[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 13:15:16.700634  9682 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.9.116.190:35419
I20260709 13:15:16.702162  9682 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 13:15:16.703205  9682 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:15:16.720862  9689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:16.720878  9691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:16.721906  9682 server_base.cc:1061] running on GCE node
W20260709 13:15:16.720882  9688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:17.918450  9682 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:15:17.918712  9682 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:15:17.919035  9682 hybrid_clock.cc:648] HybridClock initialized: now 1783602917918980 us; error 0 us; skew 500 ppm
I20260709 13:15:17.919912  9682 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:15:17.925439  9682 webserver.cc:533] Webserver started at http://127.9.116.190:35585/ using document root <none> and password file <none>
I20260709 13:15:17.926349  9682 fs_manager.cc:362] Metadata directory not provided
I20260709 13:15:17.926596  9682 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:15:17.927094  9682 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:15:17.933085  9682 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602916530764-9682-0/minicluster-data/master-0-root/instance:
uuid: "adf6cb397e6c4208ba71a60bcd491798"
format_stamp: "Formatted at 2026-07-09 13:15:17 on dist-test-slave-1t6m"
I20260709 13:15:17.947541  9682 fs_manager.cc:696] Time spent creating directory manager: real 0.014s	user 0.011s	sys 0.001s
I20260709 13:15:17.955611  9697 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:17.957968  9682 fs_manager.cc:730] Time spent opening block manager: real 0.006s	user 0.000s	sys 0.004s
I20260709 13:15:17.958511  9682 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602916530764-9682-0/minicluster-data/master-0-root
uuid: "adf6cb397e6c4208ba71a60bcd491798"
format_stamp: "Formatted at 2026-07-09 13:15:17 on dist-test-slave-1t6m"
I20260709 13:15:17.959192  9682 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602916530764-9682-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602916530764-9682-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602916530764-9682-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:15:18.031700  9682 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:15:18.033387  9682 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 13:15:18.033869  9682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:15:18.114106  9682 rpc_server.cc:307] RPC server started. Bound to: 127.9.116.190:35419
I20260709 13:15:18.114189  9748 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.116.190:35419 every 8 connection(s)
I20260709 13:15:18.123137  9749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:18.145720  9749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adf6cb397e6c4208ba71a60bcd491798: Bootstrap starting.
I20260709 13:15:18.151767  9749 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P adf6cb397e6c4208ba71a60bcd491798: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:18.153617  9749 log.cc:826] T 00000000000000000000000000000000 P adf6cb397e6c4208ba71a60bcd491798: Log is configured to *not* fsync() on all Append() calls
I20260709 13:15:18.158483  9749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adf6cb397e6c4208ba71a60bcd491798: No bootstrap required, opened a new log
I20260709 13:15:18.179265  9749 raft_consensus.cc:359] T 00000000000000000000000000000000 P adf6cb397e6c4208ba71a60bcd491798 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adf6cb397e6c4208ba71a60bcd491798" member_type: VOTER }
I20260709 13:15:18.179831  9749 raft_consensus.cc:385] T 00000000000000000000000000000000 P adf6cb397e6c4208ba71a60bcd491798 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:18.180032  9749 raft_consensus.cc:740] T 00000000000000000000000000000000 P adf6cb397e6c4208ba71a60bcd491798 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adf6cb397e6c4208ba71a60bcd491798, State: Initialized, Role: FOLLOWER
I20260709 13:15:18.180992  9749 consensus_queue.cc:260] T 00000000000000000000000000000000 P adf6cb397e6c4208ba71a60bcd491798 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adf6cb397e6c4208ba71a60bcd491798" member_type: VOTER }
I20260709 13:15:18.181468  9749 raft_consensus.cc:399] T 00000000000000000000000000000000 P adf6cb397e6c4208ba71a60bcd491798 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:15:18.181721  9749 raft_consensus.cc:493] T 00000000000000000000000000000000 P adf6cb397e6c4208ba71a60bcd491798 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:15:18.182016  9749 raft_consensus.cc:3060] T 00000000000000000000000000000000 P adf6cb397e6c4208ba71a60bcd491798 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:18.186175  9749 raft_consensus.cc:515] T 00000000000000000000000000000000 P adf6cb397e6c4208ba71a60bcd491798 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adf6cb397e6c4208ba71a60bcd491798" member_type: VOTER }
I20260709 13:15:18.187031  9749 leader_election.cc:304] T 00000000000000000000000000000000 P adf6cb397e6c4208ba71a60bcd491798 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: adf6cb397e6c4208ba71a60bcd491798; no voters: 
I20260709 13:15:18.188745  9749 leader_election.cc:290] T 00000000000000000000000000000000 P adf6cb397e6c4208ba71a60bcd491798 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:15:18.189133  9752 raft_consensus.cc:2804] T 00000000000000000000000000000000 P adf6cb397e6c4208ba71a60bcd491798 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:18.191565  9752 raft_consensus.cc:697] T 00000000000000000000000000000000 P adf6cb397e6c4208ba71a60bcd491798 [term 1 LEADER]: Becoming Leader. State: Replica: adf6cb397e6c4208ba71a60bcd491798, State: Running, Role: LEADER
I20260709 13:15:18.192521  9752 consensus_queue.cc:237] T 00000000000000000000000000000000 P adf6cb397e6c4208ba71a60bcd491798 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adf6cb397e6c4208ba71a60bcd491798" member_type: VOTER }
I20260709 13:15:18.193470  9749 sys_catalog.cc:565] T 00000000000000000000000000000000 P adf6cb397e6c4208ba71a60bcd491798 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:15:18.204660  9753 sys_catalog.cc:455] T 00000000000000000000000000000000 P adf6cb397e6c4208ba71a60bcd491798 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "adf6cb397e6c4208ba71a60bcd491798" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adf6cb397e6c4208ba71a60bcd491798" member_type: VOTER } }
I20260709 13:15:18.205403  9753 sys_catalog.cc:458] T 00000000000000000000000000000000 P adf6cb397e6c4208ba71a60bcd491798 [sys.catalog]: This master's current role is: LEADER
I20260709 13:15:18.206718  9754 sys_catalog.cc:455] T 00000000000000000000000000000000 P adf6cb397e6c4208ba71a60bcd491798 [sys.catalog]: SysCatalogTable state changed. Reason: New leader adf6cb397e6c4208ba71a60bcd491798. Latest consensus state: current_term: 1 leader_uuid: "adf6cb397e6c4208ba71a60bcd491798" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adf6cb397e6c4208ba71a60bcd491798" member_type: VOTER } }
I20260709 13:15:18.207419  9754 sys_catalog.cc:458] T 00000000000000000000000000000000 P adf6cb397e6c4208ba71a60bcd491798 [sys.catalog]: This master's current role is: LEADER
I20260709 13:15:18.208586  9761 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:15:18.220642  9761 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:15:18.223016  9682 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:15:18.238318  9761 catalog_manager.cc:1383] Generated new cluster ID: eecb22be1824449cb3e5fde9d4af0da9
I20260709 13:15:18.238672  9761 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:15:18.272120  9761 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:15:18.274312  9761 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:15:18.297453  9761 catalog_manager.cc:6070] T 00000000000000000000000000000000 P adf6cb397e6c4208ba71a60bcd491798: Generated new TSK 0
I20260709 13:15:18.298715  9761 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:15:18.357813  9682 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:15:18.367609  9771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:18.368610  9772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:18.373221  9682 server_base.cc:1061] running on GCE node
W20260709 13:15:18.373440  9774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:18.374372  9682 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:15:18.374558  9682 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:15:18.374778  9682 hybrid_clock.cc:648] HybridClock initialized: now 1783602918374744 us; error 0 us; skew 500 ppm
I20260709 13:15:18.375438  9682 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:15:18.380041  9682 webserver.cc:533] Webserver started at http://127.9.116.129:41797/ using document root <none> and password file <none>
I20260709 13:15:18.380636  9682 fs_manager.cc:362] Metadata directory not provided
I20260709 13:15:18.380846  9682 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:15:18.381152  9682 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:15:18.382285  9682 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602916530764-9682-0/minicluster-data/ts-0-root/instance:
uuid: "63a62dec28ea4a44930d9c44561f273b"
format_stamp: "Formatted at 2026-07-09 13:15:18 on dist-test-slave-1t6m"
I20260709 13:15:18.388302  9682 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.000s	sys 0.005s
I20260709 13:15:18.393064  9779 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:18.394002  9682 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.003s
I20260709 13:15:18.394377  9682 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602916530764-9682-0/minicluster-data/ts-0-root
uuid: "63a62dec28ea4a44930d9c44561f273b"
format_stamp: "Formatted at 2026-07-09 13:15:18 on dist-test-slave-1t6m"
I20260709 13:15:18.394766  9682 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602916530764-9682-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602916530764-9682-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602916530764-9682-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:15:18.431609  9682 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:15:18.433432  9682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:15:18.436259  9682 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:15:18.441802  9682 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:15:18.442119  9682 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:18.442448  9682 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:15:18.442685  9682 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:18.536916  9682 rpc_server.cc:307] RPC server started. Bound to: 127.9.116.129:41813
I20260709 13:15:18.536969  9841 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.116.129:41813 every 8 connection(s)
I20260709 13:15:18.545691  9682 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:15:18.556149  9846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:18.556815  9847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:18.565485  9682 server_base.cc:1061] running on GCE node
W20260709 13:15:18.567480  9849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:18.568807  9682 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:15:18.569171  9682 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:15:18.569438  9682 hybrid_clock.cc:648] HybridClock initialized: now 1783602918569414 us; error 0 us; skew 500 ppm
I20260709 13:15:18.570271  9682 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:15:18.573688  9682 webserver.cc:533] Webserver started at http://127.9.116.130:44713/ using document root <none> and password file <none>
I20260709 13:15:18.574222  9842 heartbeater.cc:344] Connected to a master server at 127.9.116.190:35419
I20260709 13:15:18.574369  9682 fs_manager.cc:362] Metadata directory not provided
I20260709 13:15:18.574647  9682 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:15:18.574760  9842 heartbeater.cc:461] Registering TS with master...
I20260709 13:15:18.574988  9682 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:15:18.576049  9842 heartbeater.cc:507] Master 127.9.116.190:35419 requested a full tablet report, sending...
I20260709 13:15:18.579667  9714 ts_manager.cc:194] Registered new tserver with Master: 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813)
I20260709 13:15:18.582075  9714 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58042
I20260709 13:15:18.582875  9682 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602916530764-9682-0/minicluster-data/ts-1-root/instance:
uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5"
format_stamp: "Formatted at 2026-07-09 13:15:18 on dist-test-slave-1t6m"
I20260709 13:15:18.590107  9682 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.003s	sys 0.003s
I20260709 13:15:18.595517  9854 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:18.596478  9682 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:15:18.596850  9682 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602916530764-9682-0/minicluster-data/ts-1-root
uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5"
format_stamp: "Formatted at 2026-07-09 13:15:18 on dist-test-slave-1t6m"
I20260709 13:15:18.597249  9682 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602916530764-9682-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602916530764-9682-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602916530764-9682-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:15:18.632522  9682 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:15:18.634228  9682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:15:18.636807  9682 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:15:18.640597  9682 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:15:18.640865  9682 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:18.641152  9682 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:15:18.641371  9682 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:18.731869  9682 rpc_server.cc:307] RPC server started. Bound to: 127.9.116.130:44493
I20260709 13:15:18.731933  9916 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.116.130:44493 every 8 connection(s)
I20260709 13:15:18.740612  9682 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:15:18.751454  9920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:18.752521  9917 heartbeater.cc:344] Connected to a master server at 127.9.116.190:35419
I20260709 13:15:18.752921  9917 heartbeater.cc:461] Registering TS with master...
W20260709 13:15:18.752985  9921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:18.753970  9917 heartbeater.cc:507] Master 127.9.116.190:35419 requested a full tablet report, sending...
I20260709 13:15:18.756903  9714 ts_manager.cc:194] Registered new tserver with Master: 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130:44493)
I20260709 13:15:18.759326  9714 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58058
I20260709 13:15:18.763722  9682 server_base.cc:1061] running on GCE node
W20260709 13:15:18.765440  9923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:18.766654  9682 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:15:18.766991  9682 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:15:18.767215  9682 hybrid_clock.cc:648] HybridClock initialized: now 1783602918767197 us; error 0 us; skew 500 ppm
I20260709 13:15:18.767992  9682 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:15:18.771070  9682 webserver.cc:533] Webserver started at http://127.9.116.131:45937/ using document root <none> and password file <none>
I20260709 13:15:18.771765  9682 fs_manager.cc:362] Metadata directory not provided
I20260709 13:15:18.772029  9682 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:15:18.772392  9682 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:15:18.773877  9682 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602916530764-9682-0/minicluster-data/ts-2-root/instance:
uuid: "08996682b9d042fc870e58e3db01e68e"
format_stamp: "Formatted at 2026-07-09 13:15:18 on dist-test-slave-1t6m"
I20260709 13:15:18.780452  9682 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.007s	sys 0.002s
I20260709 13:15:18.785158  9928 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:18.786065  9682 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:15:18.786441  9682 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602916530764-9682-0/minicluster-data/ts-2-root
uuid: "08996682b9d042fc870e58e3db01e68e"
format_stamp: "Formatted at 2026-07-09 13:15:18 on dist-test-slave-1t6m"
I20260709 13:15:18.786875  9682 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602916530764-9682-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602916530764-9682-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602916530764-9682-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:15:18.825591  9682 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:15:18.827440  9682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:15:18.829217  9682 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:15:18.832336  9682 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:15:18.832633  9682 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:18.832939  9682 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:15:18.833174  9682 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:18.926710  9682 rpc_server.cc:307] RPC server started. Bound to: 127.9.116.131:40693
I20260709 13:15:18.926824  9990 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.116.131:40693 every 8 connection(s)
I20260709 13:15:18.945669  9991 heartbeater.cc:344] Connected to a master server at 127.9.116.190:35419
I20260709 13:15:18.946118  9991 heartbeater.cc:461] Registering TS with master...
I20260709 13:15:18.947043  9991 heartbeater.cc:507] Master 127.9.116.190:35419 requested a full tablet report, sending...
I20260709 13:15:18.949085  9714 ts_manager.cc:194] Registered new tserver with Master: 08996682b9d042fc870e58e3db01e68e (127.9.116.131:40693)
I20260709 13:15:18.949496  9682 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016404134s
I20260709 13:15:18.950776  9714 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58062
I20260709 13:15:19.014549  9714 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58072:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:15:19.018282  9714 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:15:19.133499  9807 tablet_service.cc:1511] Processing CreateTablet for tablet f27f84491f9c4b85af0ef5524c00f5e8 (DEFAULT_TABLE table=test-workload [id=983fa9922ee2448d9acc3899d8404de2]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:15:19.134377  9806 tablet_service.cc:1511] Processing CreateTablet for tablet 40e71d19ec8244748ceb00b2ab546eb9 (DEFAULT_TABLE table=test-workload [id=983fa9922ee2448d9acc3899d8404de2]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:15:19.135002  9805 tablet_service.cc:1511] Processing CreateTablet for tablet e1e26033fa4044e89207d8960181bfad (DEFAULT_TABLE table=test-workload [id=983fa9922ee2448d9acc3899d8404de2]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:15:19.139714  9807 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f27f84491f9c4b85af0ef5524c00f5e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:19.141404  9806 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40e71d19ec8244748ceb00b2ab546eb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:19.142597  9805 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1e26033fa4044e89207d8960181bfad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:19.145390  9804 tablet_service.cc:1511] Processing CreateTablet for tablet cdcf499cc854451ab2609798dfcff264 (DEFAULT_TABLE table=test-workload [id=983fa9922ee2448d9acc3899d8404de2]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:15:19.148208  9804 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdcf499cc854451ab2609798dfcff264. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:19.164783  9882 tablet_service.cc:1511] Processing CreateTablet for tablet f27f84491f9c4b85af0ef5524c00f5e8 (DEFAULT_TABLE table=test-workload [id=983fa9922ee2448d9acc3899d8404de2]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:15:19.165839  9881 tablet_service.cc:1511] Processing CreateTablet for tablet 40e71d19ec8244748ceb00b2ab546eb9 (DEFAULT_TABLE table=test-workload [id=983fa9922ee2448d9acc3899d8404de2]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:15:19.166353  9882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f27f84491f9c4b85af0ef5524c00f5e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:19.167263  9881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40e71d19ec8244748ceb00b2ab546eb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:19.172112  9880 tablet_service.cc:1511] Processing CreateTablet for tablet e1e26033fa4044e89207d8960181bfad (DEFAULT_TABLE table=test-workload [id=983fa9922ee2448d9acc3899d8404de2]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:15:19.173624  9880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1e26033fa4044e89207d8960181bfad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:19.177171  9956 tablet_service.cc:1511] Processing CreateTablet for tablet f27f84491f9c4b85af0ef5524c00f5e8 (DEFAULT_TABLE table=test-workload [id=983fa9922ee2448d9acc3899d8404de2]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:15:19.178632  9956 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f27f84491f9c4b85af0ef5524c00f5e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:19.179977  9954 tablet_service.cc:1511] Processing CreateTablet for tablet e1e26033fa4044e89207d8960181bfad (DEFAULT_TABLE table=test-workload [id=983fa9922ee2448d9acc3899d8404de2]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:15:19.185590  9955 tablet_service.cc:1511] Processing CreateTablet for tablet 40e71d19ec8244748ceb00b2ab546eb9 (DEFAULT_TABLE table=test-workload [id=983fa9922ee2448d9acc3899d8404de2]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:15:19.194942  9954 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1e26033fa4044e89207d8960181bfad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:19.178655  9803 tablet_service.cc:1511] Processing CreateTablet for tablet 986acd03cce8455aad79beacff7aa154 (DEFAULT_TABLE table=test-workload [id=983fa9922ee2448d9acc3899d8404de2]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:15:19.196820  9879 tablet_service.cc:1511] Processing CreateTablet for tablet cdcf499cc854451ab2609798dfcff264 (DEFAULT_TABLE table=test-workload [id=983fa9922ee2448d9acc3899d8404de2]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:15:19.192351  9802 tablet_service.cc:1511] Processing CreateTablet for tablet dba81173e16b42ab92d7da17ba0518a7 (DEFAULT_TABLE table=test-workload [id=983fa9922ee2448d9acc3899d8404de2]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:15:19.198258  9878 tablet_service.cc:1511] Processing CreateTablet for tablet 986acd03cce8455aad79beacff7aa154 (DEFAULT_TABLE table=test-workload [id=983fa9922ee2448d9acc3899d8404de2]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:15:19.202955  9953 tablet_service.cc:1511] Processing CreateTablet for tablet cdcf499cc854451ab2609798dfcff264 (DEFAULT_TABLE table=test-workload [id=983fa9922ee2448d9acc3899d8404de2]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:15:19.210687  9955 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40e71d19ec8244748ceb00b2ab546eb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:19.213043  9879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdcf499cc854451ab2609798dfcff264. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:19.197454  9952 tablet_service.cc:1511] Processing CreateTablet for tablet 986acd03cce8455aad79beacff7aa154 (DEFAULT_TABLE table=test-workload [id=983fa9922ee2448d9acc3899d8404de2]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:15:19.222129  9803 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 986acd03cce8455aad79beacff7aa154. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:19.220288  9951 tablet_service.cc:1511] Processing CreateTablet for tablet dba81173e16b42ab92d7da17ba0518a7 (DEFAULT_TABLE table=test-workload [id=983fa9922ee2448d9acc3899d8404de2]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:15:19.223531  9802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dba81173e16b42ab92d7da17ba0518a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:19.215595  9877 tablet_service.cc:1511] Processing CreateTablet for tablet dba81173e16b42ab92d7da17ba0518a7 (DEFAULT_TABLE table=test-workload [id=983fa9922ee2448d9acc3899d8404de2]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:15:19.227842  9953 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdcf499cc854451ab2609798dfcff264. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:19.229401  9878 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 986acd03cce8455aad79beacff7aa154. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:19.227843  9801 tablet_service.cc:1511] Processing CreateTablet for tablet 701d79f9f0674270ab399ece95bc5c03 (DEFAULT_TABLE table=test-workload [id=983fa9922ee2448d9acc3899d8404de2]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
W20260709 13:15:19.229875  9762 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:15:19.258749  9876 tablet_service.cc:1511] Processing CreateTablet for tablet 701d79f9f0674270ab399ece95bc5c03 (DEFAULT_TABLE table=test-workload [id=983fa9922ee2448d9acc3899d8404de2]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:15:19.261650  9876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 701d79f9f0674270ab399ece95bc5c03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:19.263885  9950 tablet_service.cc:1511] Processing CreateTablet for tablet 701d79f9f0674270ab399ece95bc5c03 (DEFAULT_TABLE table=test-workload [id=983fa9922ee2448d9acc3899d8404de2]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:15:19.266681  9950 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 701d79f9f0674270ab399ece95bc5c03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:19.281272  9801 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 701d79f9f0674270ab399ece95bc5c03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:19.291846  9951 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dba81173e16b42ab92d7da17ba0518a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:19.302237  9952 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 986acd03cce8455aad79beacff7aa154. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:19.303076  9877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dba81173e16b42ab92d7da17ba0518a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:19.311924  9804 tablet_service.cc:1511] Processing CreateTablet for tablet 15d491b33a7c47dab512ed81614b9b97 (DEFAULT_TABLE table=test-workload [id=983fa9922ee2448d9acc3899d8404de2]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:15:19.316617  9804 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15d491b33a7c47dab512ed81614b9b97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:19.331441  9875 tablet_service.cc:1511] Processing CreateTablet for tablet 15d491b33a7c47dab512ed81614b9b97 (DEFAULT_TABLE table=test-workload [id=983fa9922ee2448d9acc3899d8404de2]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:15:19.333128  9875 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15d491b33a7c47dab512ed81614b9b97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:19.344056 10009 tablet_bootstrap.cc:492] T 40e71d19ec8244748ceb00b2ab546eb9 P 63a62dec28ea4a44930d9c44561f273b: Bootstrap starting.
I20260709 13:15:19.370852 10010 tablet_bootstrap.cc:492] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5: Bootstrap starting.
I20260709 13:15:19.393812 10009 tablet_bootstrap.cc:654] T 40e71d19ec8244748ceb00b2ab546eb9 P 63a62dec28ea4a44930d9c44561f273b: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:19.393443  9949 tablet_service.cc:1511] Processing CreateTablet for tablet 15d491b33a7c47dab512ed81614b9b97 (DEFAULT_TABLE table=test-workload [id=983fa9922ee2448d9acc3899d8404de2]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:15:19.397433  9949 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15d491b33a7c47dab512ed81614b9b97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:19.404476  9806 tablet_service.cc:1511] Processing CreateTablet for tablet d94510fe6d394c62a543502e8ace50eb (DEFAULT_TABLE table=test-workload [id=983fa9922ee2448d9acc3899d8404de2]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:15:19.407253  9806 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d94510fe6d394c62a543502e8ace50eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:19.413461  9881 tablet_service.cc:1511] Processing CreateTablet for tablet d94510fe6d394c62a543502e8ace50eb (DEFAULT_TABLE table=test-workload [id=983fa9922ee2448d9acc3899d8404de2]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:15:19.414880  9881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d94510fe6d394c62a543502e8ace50eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:19.458143  9952 tablet_service.cc:1511] Processing CreateTablet for tablet d94510fe6d394c62a543502e8ace50eb (DEFAULT_TABLE table=test-workload [id=983fa9922ee2448d9acc3899d8404de2]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:15:19.459419  9952 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d94510fe6d394c62a543502e8ace50eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:19.468502 10011 tablet_bootstrap.cc:492] T 40e71d19ec8244748ceb00b2ab546eb9 P 08996682b9d042fc870e58e3db01e68e: Bootstrap starting.
I20260709 13:15:19.486871 10010 tablet_bootstrap.cc:654] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:19.495904 10009 tablet_bootstrap.cc:492] T 40e71d19ec8244748ceb00b2ab546eb9 P 63a62dec28ea4a44930d9c44561f273b: No bootstrap required, opened a new log
I20260709 13:15:19.496443 10009 ts_tablet_manager.cc:1403] T 40e71d19ec8244748ceb00b2ab546eb9 P 63a62dec28ea4a44930d9c44561f273b: Time spent bootstrapping tablet: real 0.161s	user 0.011s	sys 0.007s
I20260709 13:15:19.506021 10011 tablet_bootstrap.cc:654] T 40e71d19ec8244748ceb00b2ab546eb9 P 08996682b9d042fc870e58e3db01e68e: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:19.510617 10010 tablet_bootstrap.cc:492] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5: No bootstrap required, opened a new log
I20260709 13:15:19.512905 10010 ts_tablet_manager.cc:1403] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5: Time spent bootstrapping tablet: real 0.144s	user 0.006s	sys 0.012s
I20260709 13:15:19.510912 10009 raft_consensus.cc:359] T 40e71d19ec8244748ceb00b2ab546eb9 P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.515094 10009 raft_consensus.cc:385] T 40e71d19ec8244748ceb00b2ab546eb9 P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:19.515509 10009 raft_consensus.cc:740] T 40e71d19ec8244748ceb00b2ab546eb9 P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63a62dec28ea4a44930d9c44561f273b, State: Initialized, Role: FOLLOWER
I20260709 13:15:19.516297 10009 consensus_queue.cc:260] T 40e71d19ec8244748ceb00b2ab546eb9 P 63a62dec28ea4a44930d9c44561f273b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.518088 10010 raft_consensus.cc:359] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.519186 10010 raft_consensus.cc:385] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:19.519605 10010 raft_consensus.cc:740] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68ba1cb4a67343a09adb98b1f9e9dcf5, State: Initialized, Role: FOLLOWER
I20260709 13:15:19.520618 10010 consensus_queue.cc:260] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.521811 10011 tablet_bootstrap.cc:492] T 40e71d19ec8244748ceb00b2ab546eb9 P 08996682b9d042fc870e58e3db01e68e: No bootstrap required, opened a new log
I20260709 13:15:19.522452 10011 ts_tablet_manager.cc:1403] T 40e71d19ec8244748ceb00b2ab546eb9 P 08996682b9d042fc870e58e3db01e68e: Time spent bootstrapping tablet: real 0.126s	user 0.015s	sys 0.005s
I20260709 13:15:19.526000 10011 raft_consensus.cc:359] T 40e71d19ec8244748ceb00b2ab546eb9 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.526468 10009 ts_tablet_manager.cc:1434] T 40e71d19ec8244748ceb00b2ab546eb9 P 63a62dec28ea4a44930d9c44561f273b: Time spent starting tablet: real 0.030s	user 0.009s	sys 0.005s
I20260709 13:15:19.526769 10011 raft_consensus.cc:385] T 40e71d19ec8244748ceb00b2ab546eb9 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:19.527170 10011 raft_consensus.cc:740] T 40e71d19ec8244748ceb00b2ab546eb9 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08996682b9d042fc870e58e3db01e68e, State: Initialized, Role: FOLLOWER
I20260709 13:15:19.527706  9842 heartbeater.cc:499] Master 127.9.116.190:35419 was elected leader, sending a full tablet report...
I20260709 13:15:19.527994 10011 consensus_queue.cc:260] T 40e71d19ec8244748ceb00b2ab546eb9 P 08996682b9d042fc870e58e3db01e68e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.531394  9991 heartbeater.cc:499] Master 127.9.116.190:35419 was elected leader, sending a full tablet report...
I20260709 13:15:19.532810 10011 ts_tablet_manager.cc:1434] T 40e71d19ec8244748ceb00b2ab546eb9 P 08996682b9d042fc870e58e3db01e68e: Time spent starting tablet: real 0.010s	user 0.011s	sys 0.000s
I20260709 13:15:19.533917 10011 tablet_bootstrap.cc:492] T f27f84491f9c4b85af0ef5524c00f5e8 P 08996682b9d042fc870e58e3db01e68e: Bootstrap starting.
I20260709 13:15:19.533994 10009 tablet_bootstrap.cc:492] T f27f84491f9c4b85af0ef5524c00f5e8 P 63a62dec28ea4a44930d9c44561f273b: Bootstrap starting.
I20260709 13:15:19.540488  9917 heartbeater.cc:499] Master 127.9.116.190:35419 was elected leader, sending a full tablet report...
I20260709 13:15:19.540550 10018 raft_consensus.cc:493] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.542914 10018 raft_consensus.cc:515] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.544672 10009 tablet_bootstrap.cc:654] T f27f84491f9c4b85af0ef5524c00f5e8 P 63a62dec28ea4a44930d9c44561f273b: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:19.545008 10010 ts_tablet_manager.cc:1434] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5: Time spent starting tablet: real 0.030s	user 0.008s	sys 0.016s
W20260709 13:15:19.545405  9843 tablet.cc:2406] T 40e71d19ec8244748ceb00b2ab546eb9 P 63a62dec28ea4a44930d9c44561f273b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:15:19.545969 10010 tablet_bootstrap.cc:492] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5: Bootstrap starting.
I20260709 13:15:19.554046 10010 tablet_bootstrap.cc:654] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:19.554770 10011 tablet_bootstrap.cc:654] T f27f84491f9c4b85af0ef5524c00f5e8 P 08996682b9d042fc870e58e3db01e68e: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:19.561116 10018 leader_election.cc:290] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813), 08996682b9d042fc870e58e3db01e68e (127.9.116.131:40693)
I20260709 13:15:19.568267 10011 tablet_bootstrap.cc:492] T f27f84491f9c4b85af0ef5524c00f5e8 P 08996682b9d042fc870e58e3db01e68e: No bootstrap required, opened a new log
I20260709 13:15:19.568840 10011 ts_tablet_manager.cc:1403] T f27f84491f9c4b85af0ef5524c00f5e8 P 08996682b9d042fc870e58e3db01e68e: Time spent bootstrapping tablet: real 0.035s	user 0.011s	sys 0.000s
I20260709 13:15:19.569197 10010 tablet_bootstrap.cc:492] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5: No bootstrap required, opened a new log
I20260709 13:15:19.569820 10010 ts_tablet_manager.cc:1403] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.005s
I20260709 13:15:19.571974 10011 raft_consensus.cc:359] T f27f84491f9c4b85af0ef5524c00f5e8 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.572780 10011 raft_consensus.cc:385] T f27f84491f9c4b85af0ef5524c00f5e8 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:19.573086 10011 raft_consensus.cc:740] T f27f84491f9c4b85af0ef5524c00f5e8 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08996682b9d042fc870e58e3db01e68e, State: Initialized, Role: FOLLOWER
I20260709 13:15:19.573558 10010 raft_consensus.cc:359] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.575169 10010 raft_consensus.cc:385] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:19.575583 10010 raft_consensus.cc:740] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68ba1cb4a67343a09adb98b1f9e9dcf5, State: Initialized, Role: FOLLOWER
I20260709 13:15:19.576537 10010 consensus_queue.cc:260] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.579602 10010 ts_tablet_manager.cc:1434] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.003s
I20260709 13:15:19.581015 10010 tablet_bootstrap.cc:492] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5: Bootstrap starting.
I20260709 13:15:19.574018 10011 consensus_queue.cc:260] T f27f84491f9c4b85af0ef5524c00f5e8 P 08996682b9d042fc870e58e3db01e68e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.585470 10011 ts_tablet_manager.cc:1434] T f27f84491f9c4b85af0ef5524c00f5e8 P 08996682b9d042fc870e58e3db01e68e: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:15:19.586472 10011 tablet_bootstrap.cc:492] T 986acd03cce8455aad79beacff7aa154 P 08996682b9d042fc870e58e3db01e68e: Bootstrap starting.
I20260709 13:15:19.590698 10010 tablet_bootstrap.cc:654] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:19.601735 10011 tablet_bootstrap.cc:654] T 986acd03cce8455aad79beacff7aa154 P 08996682b9d042fc870e58e3db01e68e: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:19.613929 10011 tablet_bootstrap.cc:492] T 986acd03cce8455aad79beacff7aa154 P 08996682b9d042fc870e58e3db01e68e: No bootstrap required, opened a new log
I20260709 13:15:19.614419 10011 ts_tablet_manager.cc:1403] T 986acd03cce8455aad79beacff7aa154 P 08996682b9d042fc870e58e3db01e68e: Time spent bootstrapping tablet: real 0.028s	user 0.011s	sys 0.000s
I20260709 13:15:19.617283 10011 raft_consensus.cc:359] T 986acd03cce8455aad79beacff7aa154 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.618028 10011 raft_consensus.cc:385] T 986acd03cce8455aad79beacff7aa154 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:19.623700 10011 raft_consensus.cc:740] T 986acd03cce8455aad79beacff7aa154 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08996682b9d042fc870e58e3db01e68e, State: Initialized, Role: FOLLOWER
I20260709 13:15:19.624514 10011 consensus_queue.cc:260] T 986acd03cce8455aad79beacff7aa154 P 08996682b9d042fc870e58e3db01e68e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.629462 10011 ts_tablet_manager.cc:1434] T 986acd03cce8455aad79beacff7aa154 P 08996682b9d042fc870e58e3db01e68e: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 13:15:19.630404 10011 tablet_bootstrap.cc:492] T 701d79f9f0674270ab399ece95bc5c03 P 08996682b9d042fc870e58e3db01e68e: Bootstrap starting.
I20260709 13:15:19.644234 10009 tablet_bootstrap.cc:492] T f27f84491f9c4b85af0ef5524c00f5e8 P 63a62dec28ea4a44930d9c44561f273b: No bootstrap required, opened a new log
I20260709 13:15:19.647485 10009 ts_tablet_manager.cc:1403] T f27f84491f9c4b85af0ef5524c00f5e8 P 63a62dec28ea4a44930d9c44561f273b: Time spent bootstrapping tablet: real 0.114s	user 0.038s	sys 0.070s
I20260709 13:15:19.653142 10011 tablet_bootstrap.cc:654] T 701d79f9f0674270ab399ece95bc5c03 P 08996682b9d042fc870e58e3db01e68e: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:19.653501 10010 tablet_bootstrap.cc:492] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5: No bootstrap required, opened a new log
I20260709 13:15:19.654057 10010 ts_tablet_manager.cc:1403] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5: Time spent bootstrapping tablet: real 0.073s	user 0.030s	sys 0.034s
I20260709 13:15:19.655923  9817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27f84491f9c4b85af0ef5524c00f5e8" candidate_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63a62dec28ea4a44930d9c44561f273b" is_pre_election: true
I20260709 13:15:19.656719 10009 raft_consensus.cc:359] T f27f84491f9c4b85af0ef5524c00f5e8 P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.657433 10009 raft_consensus.cc:385] T f27f84491f9c4b85af0ef5524c00f5e8 P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:19.657732 10009 raft_consensus.cc:740] T f27f84491f9c4b85af0ef5524c00f5e8 P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63a62dec28ea4a44930d9c44561f273b, State: Initialized, Role: FOLLOWER
I20260709 13:15:19.658452 10009 consensus_queue.cc:260] T f27f84491f9c4b85af0ef5524c00f5e8 P 63a62dec28ea4a44930d9c44561f273b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
W20260709 13:15:19.662281  9856 leader_election.cc:343] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:19.663959  9966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27f84491f9c4b85af0ef5524c00f5e8" candidate_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08996682b9d042fc870e58e3db01e68e" is_pre_election: true
I20260709 13:15:19.664721  9966 raft_consensus.cc:2468] T f27f84491f9c4b85af0ef5524c00f5e8 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68ba1cb4a67343a09adb98b1f9e9dcf5 in term 0.
I20260709 13:15:19.666033  9856 leader_election.cc:304] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 08996682b9d042fc870e58e3db01e68e, 68ba1cb4a67343a09adb98b1f9e9dcf5; no voters: 63a62dec28ea4a44930d9c44561f273b
I20260709 13:15:19.666213 10009 ts_tablet_manager.cc:1434] T f27f84491f9c4b85af0ef5524c00f5e8 P 63a62dec28ea4a44930d9c44561f273b: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:15:19.666942 10018 raft_consensus.cc:2804] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:19.662043 10010 raft_consensus.cc:359] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.667301 10018 raft_consensus.cc:493] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:19.667475 10010 raft_consensus.cc:385] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:19.667706 10018 raft_consensus.cc:3060] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:19.667797 10009 tablet_bootstrap.cc:492] T e1e26033fa4044e89207d8960181bfad P 63a62dec28ea4a44930d9c44561f273b: Bootstrap starting.
I20260709 13:15:19.667807 10010 raft_consensus.cc:740] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68ba1cb4a67343a09adb98b1f9e9dcf5, State: Initialized, Role: FOLLOWER
I20260709 13:15:19.668848 10010 consensus_queue.cc:260] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.683858 10009 tablet_bootstrap.cc:654] T e1e26033fa4044e89207d8960181bfad P 63a62dec28ea4a44930d9c44561f273b: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:19.684593 10010 ts_tablet_manager.cc:1434] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5: Time spent starting tablet: real 0.025s	user 0.004s	sys 0.003s
I20260709 13:15:19.684250 10018 raft_consensus.cc:515] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.685658 10010 tablet_bootstrap.cc:492] T 15d491b33a7c47dab512ed81614b9b97 P 68ba1cb4a67343a09adb98b1f9e9dcf5: Bootstrap starting.
I20260709 13:15:19.692467 10010 tablet_bootstrap.cc:654] T 15d491b33a7c47dab512ed81614b9b97 P 68ba1cb4a67343a09adb98b1f9e9dcf5: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:19.695178 10011 tablet_bootstrap.cc:492] T 701d79f9f0674270ab399ece95bc5c03 P 08996682b9d042fc870e58e3db01e68e: No bootstrap required, opened a new log
I20260709 13:15:19.695829 10011 ts_tablet_manager.cc:1403] T 701d79f9f0674270ab399ece95bc5c03 P 08996682b9d042fc870e58e3db01e68e: Time spent bootstrapping tablet: real 0.066s	user 0.010s	sys 0.003s
I20260709 13:15:19.696277 10018 leader_election.cc:290] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [CANDIDATE]: Term 1 election: Requested vote from peers 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813), 08996682b9d042fc870e58e3db01e68e (127.9.116.131:40693)
I20260709 13:15:19.696368  9966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27f84491f9c4b85af0ef5524c00f5e8" candidate_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08996682b9d042fc870e58e3db01e68e"
I20260709 13:15:19.697124  9966 raft_consensus.cc:3060] T f27f84491f9c4b85af0ef5524c00f5e8 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:19.698434  9817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27f84491f9c4b85af0ef5524c00f5e8" candidate_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63a62dec28ea4a44930d9c44561f273b"
I20260709 13:15:19.699157  9817 raft_consensus.cc:3060] T f27f84491f9c4b85af0ef5524c00f5e8 P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:19.704296  9966 raft_consensus.cc:2468] T f27f84491f9c4b85af0ef5524c00f5e8 P 08996682b9d042fc870e58e3db01e68e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68ba1cb4a67343a09adb98b1f9e9dcf5 in term 1.
I20260709 13:15:19.704454 10011 raft_consensus.cc:359] T 701d79f9f0674270ab399ece95bc5c03 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.705143 10011 raft_consensus.cc:385] T 701d79f9f0674270ab399ece95bc5c03 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:19.705451 10011 raft_consensus.cc:740] T 701d79f9f0674270ab399ece95bc5c03 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08996682b9d042fc870e58e3db01e68e, State: Initialized, Role: FOLLOWER
I20260709 13:15:19.705741  9856 leader_election.cc:304] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08996682b9d042fc870e58e3db01e68e, 68ba1cb4a67343a09adb98b1f9e9dcf5; no voters: 
I20260709 13:15:19.706771 10018 raft_consensus.cc:2804] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:19.707270 10018 raft_consensus.cc:697] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 LEADER]: Becoming Leader. State: Replica: 68ba1cb4a67343a09adb98b1f9e9dcf5, State: Running, Role: LEADER
I20260709 13:15:19.708072 10018 consensus_queue.cc:237] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.710204  9817 raft_consensus.cc:2468] T f27f84491f9c4b85af0ef5524c00f5e8 P 63a62dec28ea4a44930d9c44561f273b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68ba1cb4a67343a09adb98b1f9e9dcf5 in term 1.
I20260709 13:15:19.706194 10011 consensus_queue.cc:260] T 701d79f9f0674270ab399ece95bc5c03 P 08996682b9d042fc870e58e3db01e68e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.713531 10011 ts_tablet_manager.cc:1434] T 701d79f9f0674270ab399ece95bc5c03 P 08996682b9d042fc870e58e3db01e68e: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:15:19.719444 10011 tablet_bootstrap.cc:492] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e: Bootstrap starting.
I20260709 13:15:19.726490 10011 tablet_bootstrap.cc:654] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:19.728734  9713 catalog_manager.cc:5697] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:19.731900 10009 tablet_bootstrap.cc:492] T e1e26033fa4044e89207d8960181bfad P 63a62dec28ea4a44930d9c44561f273b: No bootstrap required, opened a new log
I20260709 13:15:19.740093 10009 ts_tablet_manager.cc:1403] T e1e26033fa4044e89207d8960181bfad P 63a62dec28ea4a44930d9c44561f273b: Time spent bootstrapping tablet: real 0.073s	user 0.027s	sys 0.029s
I20260709 13:15:19.741998 10010 tablet_bootstrap.cc:492] T 15d491b33a7c47dab512ed81614b9b97 P 68ba1cb4a67343a09adb98b1f9e9dcf5: No bootstrap required, opened a new log
I20260709 13:15:19.742523 10010 ts_tablet_manager.cc:1403] T 15d491b33a7c47dab512ed81614b9b97 P 68ba1cb4a67343a09adb98b1f9e9dcf5: Time spent bootstrapping tablet: real 0.057s	user 0.020s	sys 0.015s
I20260709 13:15:19.743880 10009 raft_consensus.cc:359] T e1e26033fa4044e89207d8960181bfad P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.746115 10009 raft_consensus.cc:385] T e1e26033fa4044e89207d8960181bfad P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:19.746943 10009 raft_consensus.cc:740] T e1e26033fa4044e89207d8960181bfad P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63a62dec28ea4a44930d9c44561f273b, State: Initialized, Role: FOLLOWER
I20260709 13:15:19.745888 10010 raft_consensus.cc:359] T 15d491b33a7c47dab512ed81614b9b97 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.748109 10010 raft_consensus.cc:385] T 15d491b33a7c47dab512ed81614b9b97 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:19.747851 10009 consensus_queue.cc:260] T e1e26033fa4044e89207d8960181bfad P 63a62dec28ea4a44930d9c44561f273b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.748421 10010 raft_consensus.cc:740] T 15d491b33a7c47dab512ed81614b9b97 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68ba1cb4a67343a09adb98b1f9e9dcf5, State: Initialized, Role: FOLLOWER
I20260709 13:15:19.749801 10011 tablet_bootstrap.cc:492] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e: No bootstrap required, opened a new log
I20260709 13:15:19.749239 10010 consensus_queue.cc:260] T 15d491b33a7c47dab512ed81614b9b97 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.750429 10011 ts_tablet_manager.cc:1403] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e: Time spent bootstrapping tablet: real 0.031s	user 0.016s	sys 0.005s
I20260709 13:15:19.751919 10010 ts_tablet_manager.cc:1434] T 15d491b33a7c47dab512ed81614b9b97 P 68ba1cb4a67343a09adb98b1f9e9dcf5: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:15:19.752907 10010 tablet_bootstrap.cc:492] T dba81173e16b42ab92d7da17ba0518a7 P 68ba1cb4a67343a09adb98b1f9e9dcf5: Bootstrap starting.
I20260709 13:15:19.754488 10011 raft_consensus.cc:359] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.755329 10011 raft_consensus.cc:385] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:19.755679 10011 raft_consensus.cc:740] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08996682b9d042fc870e58e3db01e68e, State: Initialized, Role: FOLLOWER
I20260709 13:15:19.756598 10011 consensus_queue.cc:260] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.758870 10011 ts_tablet_manager.cc:1434] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:15:19.759876 10011 tablet_bootstrap.cc:492] T cdcf499cc854451ab2609798dfcff264 P 08996682b9d042fc870e58e3db01e68e: Bootstrap starting.
I20260709 13:15:19.766346 10011 tablet_bootstrap.cc:654] T cdcf499cc854451ab2609798dfcff264 P 08996682b9d042fc870e58e3db01e68e: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:19.767621 10009 ts_tablet_manager.cc:1434] T e1e26033fa4044e89207d8960181bfad P 63a62dec28ea4a44930d9c44561f273b: Time spent starting tablet: real 0.027s	user 0.004s	sys 0.003s
I20260709 13:15:19.768659 10009 tablet_bootstrap.cc:492] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b: Bootstrap starting.
I20260709 13:15:19.768690 10010 tablet_bootstrap.cc:654] T dba81173e16b42ab92d7da17ba0518a7 P 68ba1cb4a67343a09adb98b1f9e9dcf5: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:19.780328 10009 tablet_bootstrap.cc:654] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:19.781885 10017 raft_consensus.cc:493] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.782358 10017 raft_consensus.cc:515] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.785148 10011 tablet_bootstrap.cc:492] T cdcf499cc854451ab2609798dfcff264 P 08996682b9d042fc870e58e3db01e68e: No bootstrap required, opened a new log
I20260709 13:15:19.787118 10011 ts_tablet_manager.cc:1403] T cdcf499cc854451ab2609798dfcff264 P 08996682b9d042fc870e58e3db01e68e: Time spent bootstrapping tablet: real 0.027s	user 0.009s	sys 0.013s
I20260709 13:15:19.788511 10017 leader_election.cc:290] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813), 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130:44493)
I20260709 13:15:19.798211 10011 raft_consensus.cc:359] T cdcf499cc854451ab2609798dfcff264 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.799376 10011 raft_consensus.cc:385] T cdcf499cc854451ab2609798dfcff264 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:19.799882 10011 raft_consensus.cc:740] T cdcf499cc854451ab2609798dfcff264 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08996682b9d042fc870e58e3db01e68e, State: Initialized, Role: FOLLOWER
I20260709 13:15:19.801474 10010 tablet_bootstrap.cc:492] T dba81173e16b42ab92d7da17ba0518a7 P 68ba1cb4a67343a09adb98b1f9e9dcf5: No bootstrap required, opened a new log
I20260709 13:15:19.803205 10010 ts_tablet_manager.cc:1403] T dba81173e16b42ab92d7da17ba0518a7 P 68ba1cb4a67343a09adb98b1f9e9dcf5: Time spent bootstrapping tablet: real 0.050s	user 0.014s	sys 0.005s
I20260709 13:15:19.803700 10011 consensus_queue.cc:260] T cdcf499cc854451ab2609798dfcff264 P 08996682b9d042fc870e58e3db01e68e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.806173 10011 ts_tablet_manager.cc:1434] T cdcf499cc854451ab2609798dfcff264 P 08996682b9d042fc870e58e3db01e68e: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 13:15:19.806380 10010 raft_consensus.cc:359] T dba81173e16b42ab92d7da17ba0518a7 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.810900 10010 raft_consensus.cc:385] T dba81173e16b42ab92d7da17ba0518a7 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:19.811249 10010 raft_consensus.cc:740] T dba81173e16b42ab92d7da17ba0518a7 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68ba1cb4a67343a09adb98b1f9e9dcf5, State: Initialized, Role: FOLLOWER
I20260709 13:15:19.811969 10010 consensus_queue.cc:260] T dba81173e16b42ab92d7da17ba0518a7 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.813190 10011 tablet_bootstrap.cc:492] T 15d491b33a7c47dab512ed81614b9b97 P 08996682b9d042fc870e58e3db01e68e: Bootstrap starting.
I20260709 13:15:19.814249 10010 ts_tablet_manager.cc:1434] T dba81173e16b42ab92d7da17ba0518a7 P 68ba1cb4a67343a09adb98b1f9e9dcf5: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.000s
I20260709 13:15:19.815928 10010 tablet_bootstrap.cc:492] T e1e26033fa4044e89207d8960181bfad P 68ba1cb4a67343a09adb98b1f9e9dcf5: Bootstrap starting.
I20260709 13:15:19.816205  9817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e26033fa4044e89207d8960181bfad" candidate_uuid: "08996682b9d042fc870e58e3db01e68e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63a62dec28ea4a44930d9c44561f273b" is_pre_election: true
I20260709 13:15:19.816839  9817 raft_consensus.cc:2468] T e1e26033fa4044e89207d8960181bfad P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08996682b9d042fc870e58e3db01e68e in term 0.
I20260709 13:15:19.818303  9930 leader_election.cc:304] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08996682b9d042fc870e58e3db01e68e, 63a62dec28ea4a44930d9c44561f273b; no voters: 
I20260709 13:15:19.819496 10017 raft_consensus.cc:2804] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:19.819876 10017 raft_consensus.cc:493] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:19.820211 10017 raft_consensus.cc:3060] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:19.827250 10017 raft_consensus.cc:515] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.830243  9817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e26033fa4044e89207d8960181bfad" candidate_uuid: "08996682b9d042fc870e58e3db01e68e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63a62dec28ea4a44930d9c44561f273b"
I20260709 13:15:19.830591 10010 tablet_bootstrap.cc:654] T e1e26033fa4044e89207d8960181bfad P 68ba1cb4a67343a09adb98b1f9e9dcf5: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:19.830852  9817 raft_consensus.cc:3060] T e1e26033fa4044e89207d8960181bfad P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:19.831161 10011 tablet_bootstrap.cc:654] T 15d491b33a7c47dab512ed81614b9b97 P 08996682b9d042fc870e58e3db01e68e: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:19.837916  9817 raft_consensus.cc:2468] T e1e26033fa4044e89207d8960181bfad P 63a62dec28ea4a44930d9c44561f273b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08996682b9d042fc870e58e3db01e68e in term 1.
I20260709 13:15:19.839250  9930 leader_election.cc:304] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08996682b9d042fc870e58e3db01e68e, 63a62dec28ea4a44930d9c44561f273b; no voters: 
I20260709 13:15:19.841728 10010 tablet_bootstrap.cc:492] T e1e26033fa4044e89207d8960181bfad P 68ba1cb4a67343a09adb98b1f9e9dcf5: No bootstrap required, opened a new log
I20260709 13:15:19.842221 10010 ts_tablet_manager.cc:1403] T e1e26033fa4044e89207d8960181bfad P 68ba1cb4a67343a09adb98b1f9e9dcf5: Time spent bootstrapping tablet: real 0.026s	user 0.008s	sys 0.004s
I20260709 13:15:19.844911 10018 raft_consensus.cc:493] T dba81173e16b42ab92d7da17ba0518a7 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.845137 10010 raft_consensus.cc:359] T e1e26033fa4044e89207d8960181bfad P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.845393 10018 raft_consensus.cc:515] T dba81173e16b42ab92d7da17ba0518a7 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.845885 10010 raft_consensus.cc:385] T e1e26033fa4044e89207d8960181bfad P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:19.846210 10010 raft_consensus.cc:740] T e1e26033fa4044e89207d8960181bfad P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68ba1cb4a67343a09adb98b1f9e9dcf5, State: Initialized, Role: FOLLOWER
I20260709 13:15:19.847204 10017 leader_election.cc:290] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e [CANDIDATE]: Term 1 election: Requested vote from peers 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813), 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130:44493)
I20260709 13:15:19.848627 10017 raft_consensus.cc:2804] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:19.849071 10011 tablet_bootstrap.cc:492] T 15d491b33a7c47dab512ed81614b9b97 P 08996682b9d042fc870e58e3db01e68e: No bootstrap required, opened a new log
I20260709 13:15:19.847522 10018 leader_election.cc:290] T dba81173e16b42ab92d7da17ba0518a7 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813), 08996682b9d042fc870e58e3db01e68e (127.9.116.131:40693)
I20260709 13:15:19.850610  9817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba81173e16b42ab92d7da17ba0518a7" candidate_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63a62dec28ea4a44930d9c44561f273b" is_pre_election: true
I20260709 13:15:19.852862 10011 ts_tablet_manager.cc:1403] T 15d491b33a7c47dab512ed81614b9b97 P 08996682b9d042fc870e58e3db01e68e: Time spent bootstrapping tablet: real 0.040s	user 0.013s	sys 0.000s
W20260709 13:15:19.853857  9856 leader_election.cc:343] T dba81173e16b42ab92d7da17ba0518a7 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:19.854213 10010 consensus_queue.cc:260] T e1e26033fa4044e89207d8960181bfad P 68ba1cb4a67343a09adb98b1f9e9dcf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.854693 10017 raft_consensus.cc:697] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e [term 1 LEADER]: Becoming Leader. State: Replica: 08996682b9d042fc870e58e3db01e68e, State: Running, Role: LEADER
I20260709 13:15:19.855731 10017 consensus_queue.cc:237] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.855890 10011 raft_consensus.cc:359] T 15d491b33a7c47dab512ed81614b9b97 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.857431 10011 raft_consensus.cc:385] T 15d491b33a7c47dab512ed81614b9b97 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:19.857738 10011 raft_consensus.cc:740] T 15d491b33a7c47dab512ed81614b9b97 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08996682b9d042fc870e58e3db01e68e, State: Initialized, Role: FOLLOWER
I20260709 13:15:19.858206  9966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba81173e16b42ab92d7da17ba0518a7" candidate_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08996682b9d042fc870e58e3db01e68e" is_pre_election: true
I20260709 13:15:19.858479 10011 consensus_queue.cc:260] T 15d491b33a7c47dab512ed81614b9b97 P 08996682b9d042fc870e58e3db01e68e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
W20260709 13:15:19.859603  9856 leader_election.cc:343] T dba81173e16b42ab92d7da17ba0518a7 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 08996682b9d042fc870e58e3db01e68e (127.9.116.131:40693): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:19.860035  9856 leader_election.cc:304] T dba81173e16b42ab92d7da17ba0518a7 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 68ba1cb4a67343a09adb98b1f9e9dcf5; no voters: 08996682b9d042fc870e58e3db01e68e, 63a62dec28ea4a44930d9c44561f273b
I20260709 13:15:19.861057 10011 ts_tablet_manager.cc:1434] T 15d491b33a7c47dab512ed81614b9b97 P 08996682b9d042fc870e58e3db01e68e: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.004s
I20260709 13:15:19.861111 10018 raft_consensus.cc:2749] T dba81173e16b42ab92d7da17ba0518a7 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:19.856669 10010 ts_tablet_manager.cc:1434] T e1e26033fa4044e89207d8960181bfad P 68ba1cb4a67343a09adb98b1f9e9dcf5: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.002s
I20260709 13:15:19.864260 10010 tablet_bootstrap.cc:492] T d94510fe6d394c62a543502e8ace50eb P 68ba1cb4a67343a09adb98b1f9e9dcf5: Bootstrap starting.
I20260709 13:15:19.867177 10011 tablet_bootstrap.cc:492] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e: Bootstrap starting.
I20260709 13:15:19.874897 10010 tablet_bootstrap.cc:654] T d94510fe6d394c62a543502e8ace50eb P 68ba1cb4a67343a09adb98b1f9e9dcf5: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:19.875732  9713 catalog_manager.cc:5697] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e reported cstate change: term changed from 0 to 1, leader changed from <none> to 08996682b9d042fc870e58e3db01e68e (127.9.116.131). New cstate: current_term: 1 leader_uuid: "08996682b9d042fc870e58e3db01e68e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:19.879914 10009 tablet_bootstrap.cc:492] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b: No bootstrap required, opened a new log
I20260709 13:15:19.880434 10009 ts_tablet_manager.cc:1403] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b: Time spent bootstrapping tablet: real 0.112s	user 0.015s	sys 0.002s
I20260709 13:15:19.884828 10011 tablet_bootstrap.cc:654] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:19.888037 10018 raft_consensus.cc:493] T e1e26033fa4044e89207d8960181bfad P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.888693 10018 raft_consensus.cc:515] T e1e26033fa4044e89207d8960181bfad P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.890379 10009 raft_consensus.cc:359] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.891147 10009 raft_consensus.cc:385] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:19.891427 10009 raft_consensus.cc:740] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63a62dec28ea4a44930d9c44561f273b, State: Initialized, Role: FOLLOWER
I20260709 13:15:19.892490 10018 leader_election.cc:290] T e1e26033fa4044e89207d8960181bfad P 68ba1cb4a67343a09adb98b1f9e9dcf5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813), 08996682b9d042fc870e58e3db01e68e (127.9.116.131:40693)
I20260709 13:15:19.892314 10009 consensus_queue.cc:260] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.895435  9817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e26033fa4044e89207d8960181bfad" candidate_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63a62dec28ea4a44930d9c44561f273b" is_pre_election: true
I20260709 13:15:19.897162  9817 raft_consensus.cc:2393] T e1e26033fa4044e89207d8960181bfad P 63a62dec28ea4a44930d9c44561f273b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 68ba1cb4a67343a09adb98b1f9e9dcf5 in current term 1: Already voted for candidate 08996682b9d042fc870e58e3db01e68e in this term.
I20260709 13:15:19.896061  9966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e26033fa4044e89207d8960181bfad" candidate_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08996682b9d042fc870e58e3db01e68e" is_pre_election: true
I20260709 13:15:19.898234 10009 ts_tablet_manager.cc:1434] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.003s
I20260709 13:15:19.899533 10009 tablet_bootstrap.cc:492] T d94510fe6d394c62a543502e8ace50eb P 63a62dec28ea4a44930d9c44561f273b: Bootstrap starting.
I20260709 13:15:19.899915  9856 leader_election.cc:304] T e1e26033fa4044e89207d8960181bfad P 68ba1cb4a67343a09adb98b1f9e9dcf5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 68ba1cb4a67343a09adb98b1f9e9dcf5; no voters: 08996682b9d042fc870e58e3db01e68e, 63a62dec28ea4a44930d9c44561f273b
I20260709 13:15:19.900898 10016 raft_consensus.cc:3060] T e1e26033fa4044e89207d8960181bfad P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:19.906137 10009 tablet_bootstrap.cc:654] T d94510fe6d394c62a543502e8ace50eb P 63a62dec28ea4a44930d9c44561f273b: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:19.909657 10016 raft_consensus.cc:2749] T e1e26033fa4044e89207d8960181bfad P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:19.913462 10011 tablet_bootstrap.cc:492] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e: No bootstrap required, opened a new log
I20260709 13:15:19.914090 10011 ts_tablet_manager.cc:1403] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e: Time spent bootstrapping tablet: real 0.051s	user 0.008s	sys 0.008s
I20260709 13:15:19.917163  9892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e26033fa4044e89207d8960181bfad" candidate_uuid: "08996682b9d042fc870e58e3db01e68e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" is_pre_election: true
I20260709 13:15:19.917153 10011 raft_consensus.cc:359] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.918123 10011 raft_consensus.cc:385] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:19.918522 10011 raft_consensus.cc:740] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08996682b9d042fc870e58e3db01e68e, State: Initialized, Role: FOLLOWER
I20260709 13:15:19.918642  9891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e26033fa4044e89207d8960181bfad" candidate_uuid: "08996682b9d042fc870e58e3db01e68e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5"
I20260709 13:15:19.919116 10010 tablet_bootstrap.cc:492] T d94510fe6d394c62a543502e8ace50eb P 68ba1cb4a67343a09adb98b1f9e9dcf5: No bootstrap required, opened a new log
I20260709 13:15:19.919575 10010 ts_tablet_manager.cc:1403] T d94510fe6d394c62a543502e8ace50eb P 68ba1cb4a67343a09adb98b1f9e9dcf5: Time spent bootstrapping tablet: real 0.056s	user 0.014s	sys 0.011s
I20260709 13:15:19.919490 10011 consensus_queue.cc:260] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.918157  9892 raft_consensus.cc:2468] T e1e26033fa4044e89207d8960181bfad P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08996682b9d042fc870e58e3db01e68e in term 1.
I20260709 13:15:19.922046 10011 ts_tablet_manager.cc:1434] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
I20260709 13:15:19.922629 10010 raft_consensus.cc:359] T d94510fe6d394c62a543502e8ace50eb P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.923372 10010 raft_consensus.cc:385] T d94510fe6d394c62a543502e8ace50eb P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:19.923753 10010 raft_consensus.cc:740] T d94510fe6d394c62a543502e8ace50eb P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68ba1cb4a67343a09adb98b1f9e9dcf5, State: Initialized, Role: FOLLOWER
I20260709 13:15:19.926403 10009 tablet_bootstrap.cc:492] T d94510fe6d394c62a543502e8ace50eb P 63a62dec28ea4a44930d9c44561f273b: No bootstrap required, opened a new log
I20260709 13:15:19.926935 10009 ts_tablet_manager.cc:1403] T d94510fe6d394c62a543502e8ace50eb P 63a62dec28ea4a44930d9c44561f273b: Time spent bootstrapping tablet: real 0.028s	user 0.014s	sys 0.009s
I20260709 13:15:19.924499 10010 consensus_queue.cc:260] T d94510fe6d394c62a543502e8ace50eb P 68ba1cb4a67343a09adb98b1f9e9dcf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.923221 10011 tablet_bootstrap.cc:492] T d94510fe6d394c62a543502e8ace50eb P 08996682b9d042fc870e58e3db01e68e: Bootstrap starting.
I20260709 13:15:19.929437 10010 ts_tablet_manager.cc:1434] T d94510fe6d394c62a543502e8ace50eb P 68ba1cb4a67343a09adb98b1f9e9dcf5: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:15:19.931921 10010 tablet_bootstrap.cc:492] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5: Bootstrap starting.
I20260709 13:15:19.933739 10011 tablet_bootstrap.cc:654] T d94510fe6d394c62a543502e8ace50eb P 08996682b9d042fc870e58e3db01e68e: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:19.939002 10010 tablet_bootstrap.cc:654] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:19.943612 10017 raft_consensus.cc:493] T 701d79f9f0674270ab399ece95bc5c03 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.942694 10009 raft_consensus.cc:359] T d94510fe6d394c62a543502e8ace50eb P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.944200 10009 raft_consensus.cc:385] T d94510fe6d394c62a543502e8ace50eb P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:19.944124 10017 raft_consensus.cc:515] T 701d79f9f0674270ab399ece95bc5c03 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.944705 10009 raft_consensus.cc:740] T d94510fe6d394c62a543502e8ace50eb P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63a62dec28ea4a44930d9c44561f273b, State: Initialized, Role: FOLLOWER
I20260709 13:15:19.945750 10009 consensus_queue.cc:260] T d94510fe6d394c62a543502e8ace50eb P 63a62dec28ea4a44930d9c44561f273b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.946326 10017 leader_election.cc:290] T 701d79f9f0674270ab399ece95bc5c03 P 08996682b9d042fc870e58e3db01e68e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813), 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130:44493)
I20260709 13:15:19.947610  9817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701d79f9f0674270ab399ece95bc5c03" candidate_uuid: "08996682b9d042fc870e58e3db01e68e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63a62dec28ea4a44930d9c44561f273b" is_pre_election: true
I20260709 13:15:19.947955  9892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701d79f9f0674270ab399ece95bc5c03" candidate_uuid: "08996682b9d042fc870e58e3db01e68e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" is_pre_election: true
I20260709 13:15:19.948191 10009 ts_tablet_manager.cc:1434] T d94510fe6d394c62a543502e8ace50eb P 63a62dec28ea4a44930d9c44561f273b: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
W20260709 13:15:19.949101  9930 leader_election.cc:343] T 701d79f9f0674270ab399ece95bc5c03 P 08996682b9d042fc870e58e3db01e68e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:15:19.950343  9929 leader_election.cc:343] T 701d79f9f0674270ab399ece95bc5c03 P 08996682b9d042fc870e58e3db01e68e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130:44493): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:19.950937 10009 tablet_bootstrap.cc:492] T 986acd03cce8455aad79beacff7aa154 P 63a62dec28ea4a44930d9c44561f273b: Bootstrap starting.
I20260709 13:15:19.950852  9929 leader_election.cc:304] T 701d79f9f0674270ab399ece95bc5c03 P 08996682b9d042fc870e58e3db01e68e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 08996682b9d042fc870e58e3db01e68e; no voters: 63a62dec28ea4a44930d9c44561f273b, 68ba1cb4a67343a09adb98b1f9e9dcf5
I20260709 13:15:19.952708 10017 raft_consensus.cc:2749] T 701d79f9f0674270ab399ece95bc5c03 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:19.958626 10011 tablet_bootstrap.cc:492] T d94510fe6d394c62a543502e8ace50eb P 08996682b9d042fc870e58e3db01e68e: No bootstrap required, opened a new log
I20260709 13:15:19.959213 10011 ts_tablet_manager.cc:1403] T d94510fe6d394c62a543502e8ace50eb P 08996682b9d042fc870e58e3db01e68e: Time spent bootstrapping tablet: real 0.036s	user 0.013s	sys 0.000s
I20260709 13:15:19.961225 10009 tablet_bootstrap.cc:654] T 986acd03cce8455aad79beacff7aa154 P 63a62dec28ea4a44930d9c44561f273b: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:19.962862 10011 raft_consensus.cc:359] T d94510fe6d394c62a543502e8ace50eb P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.963860 10011 raft_consensus.cc:385] T d94510fe6d394c62a543502e8ace50eb P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:19.964212 10011 raft_consensus.cc:740] T d94510fe6d394c62a543502e8ace50eb P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08996682b9d042fc870e58e3db01e68e, State: Initialized, Role: FOLLOWER
I20260709 13:15:19.965435 10017 raft_consensus.cc:493] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.965565 10011 consensus_queue.cc:260] T d94510fe6d394c62a543502e8ace50eb P 08996682b9d042fc870e58e3db01e68e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.967294 10017 raft_consensus.cc:515] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.972067  9892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba81173e16b42ab92d7da17ba0518a7" candidate_uuid: "08996682b9d042fc870e58e3db01e68e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" is_pre_election: true
I20260709 13:15:19.972900  9892 raft_consensus.cc:2468] T dba81173e16b42ab92d7da17ba0518a7 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08996682b9d042fc870e58e3db01e68e in term 0.
I20260709 13:15:19.972776  9817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba81173e16b42ab92d7da17ba0518a7" candidate_uuid: "08996682b9d042fc870e58e3db01e68e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63a62dec28ea4a44930d9c44561f273b" is_pre_election: true
I20260709 13:15:19.974470  9929 leader_election.cc:304] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08996682b9d042fc870e58e3db01e68e, 68ba1cb4a67343a09adb98b1f9e9dcf5; no voters: 
W20260709 13:15:19.975127  9930 leader_election.cc:343] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:19.975680 10042 raft_consensus.cc:2804] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:19.976105 10042 raft_consensus.cc:493] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:19.976450 10042 raft_consensus.cc:3060] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:19.978127 10015 raft_consensus.cc:493] T 40e71d19ec8244748ceb00b2ab546eb9 P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.978600 10015 raft_consensus.cc:515] T 40e71d19ec8244748ceb00b2ab546eb9 P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.981676 10015 leader_election.cc:290] T 40e71d19ec8244748ceb00b2ab546eb9 P 63a62dec28ea4a44930d9c44561f273b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130:44493), 08996682b9d042fc870e58e3db01e68e (127.9.116.131:40693)
I20260709 13:15:19.984014 10011 ts_tablet_manager.cc:1434] T d94510fe6d394c62a543502e8ace50eb P 08996682b9d042fc870e58e3db01e68e: Time spent starting tablet: real 0.024s	user 0.004s	sys 0.004s
I20260709 13:15:19.988765 10015 raft_consensus.cc:493] T d94510fe6d394c62a543502e8ace50eb P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.989264 10015 raft_consensus.cc:515] T d94510fe6d394c62a543502e8ace50eb P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.991402 10015 leader_election.cc:290] T d94510fe6d394c62a543502e8ace50eb P 63a62dec28ea4a44930d9c44561f273b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130:44493), 08996682b9d042fc870e58e3db01e68e (127.9.116.131:40693)
I20260709 13:15:19.982612 10016 raft_consensus.cc:493] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.998827 10016 raft_consensus.cc:515] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:19.987818 10017 leader_election.cc:290] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813), 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130:44493)
I20260709 13:15:20.006191 10016 leader_election.cc:290] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813), 08996682b9d042fc870e58e3db01e68e (127.9.116.131:40693)
I20260709 13:15:20.007637  9966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e71d19ec8244748ceb00b2ab546eb9" candidate_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08996682b9d042fc870e58e3db01e68e" is_pre_election: true
I20260709 13:15:20.008280  9966 raft_consensus.cc:2468] T 40e71d19ec8244748ceb00b2ab546eb9 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68ba1cb4a67343a09adb98b1f9e9dcf5 in term 0.
I20260709 13:15:20.009846  9856 leader_election.cc:304] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08996682b9d042fc870e58e3db01e68e, 68ba1cb4a67343a09adb98b1f9e9dcf5; no voters: 
I20260709 13:15:20.010664 10016 raft_consensus.cc:2804] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:20.011071 10016 raft_consensus.cc:493] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:20.011399 10016 raft_consensus.cc:3060] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:20.016528 10042 raft_consensus.cc:515] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.018030 10016 raft_consensus.cc:515] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.021694 10016 leader_election.cc:290] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [CANDIDATE]: Term 1 election: Requested vote from peers 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813), 08996682b9d042fc870e58e3db01e68e (127.9.116.131:40693)
I20260709 13:15:20.022977  9966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e71d19ec8244748ceb00b2ab546eb9" candidate_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08996682b9d042fc870e58e3db01e68e"
I20260709 13:15:20.023576  9966 raft_consensus.cc:3060] T 40e71d19ec8244748ceb00b2ab546eb9 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:20.027781 10017 raft_consensus.cc:493] T cdcf499cc854451ab2609798dfcff264 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.028313 10017 raft_consensus.cc:515] T cdcf499cc854451ab2609798dfcff264 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.031791  9966 raft_consensus.cc:2468] T 40e71d19ec8244748ceb00b2ab546eb9 P 08996682b9d042fc870e58e3db01e68e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68ba1cb4a67343a09adb98b1f9e9dcf5 in term 1.
I20260709 13:15:20.033581 10016 raft_consensus.cc:493] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.037362  9817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e71d19ec8244748ceb00b2ab546eb9" candidate_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63a62dec28ea4a44930d9c44561f273b" is_pre_election: true
I20260709 13:15:20.038077  9817 raft_consensus.cc:2468] T 40e71d19ec8244748ceb00b2ab546eb9 P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68ba1cb4a67343a09adb98b1f9e9dcf5 in term 0.
I20260709 13:15:20.038548  9856 leader_election.cc:304] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08996682b9d042fc870e58e3db01e68e, 68ba1cb4a67343a09adb98b1f9e9dcf5; no voters: 
I20260709 13:15:20.039446 10018 raft_consensus.cc:2804] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:20.039332  9816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e71d19ec8244748ceb00b2ab546eb9" candidate_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63a62dec28ea4a44930d9c44561f273b"
I20260709 13:15:20.040040  9816 raft_consensus.cc:3060] T 40e71d19ec8244748ceb00b2ab546eb9 P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:20.035655 10010 tablet_bootstrap.cc:492] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5: No bootstrap required, opened a new log
I20260709 13:15:20.043774 10010 ts_tablet_manager.cc:1403] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5: Time spent bootstrapping tablet: real 0.112s	user 0.017s	sys 0.039s
I20260709 13:15:20.045854  9816 raft_consensus.cc:2468] T 40e71d19ec8244748ceb00b2ab546eb9 P 63a62dec28ea4a44930d9c44561f273b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68ba1cb4a67343a09adb98b1f9e9dcf5 in term 1.
I20260709 13:15:20.037813 10016 raft_consensus.cc:515] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.035184 10009 tablet_bootstrap.cc:492] T 986acd03cce8455aad79beacff7aa154 P 63a62dec28ea4a44930d9c44561f273b: No bootstrap required, opened a new log
I20260709 13:15:20.047946 10009 ts_tablet_manager.cc:1403] T 986acd03cce8455aad79beacff7aa154 P 63a62dec28ea4a44930d9c44561f273b: Time spent bootstrapping tablet: real 0.097s	user 0.029s	sys 0.034s
I20260709 13:15:20.049903 10016 leader_election.cc:290] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813), 08996682b9d042fc870e58e3db01e68e (127.9.116.131:40693)
I20260709 13:15:20.050333  9965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdcf499cc854451ab2609798dfcff264" candidate_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08996682b9d042fc870e58e3db01e68e" is_pre_election: true
I20260709 13:15:20.051126  9965 raft_consensus.cc:2468] T cdcf499cc854451ab2609798dfcff264 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68ba1cb4a67343a09adb98b1f9e9dcf5 in term 0.
I20260709 13:15:20.052610  9856 leader_election.cc:304] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08996682b9d042fc870e58e3db01e68e, 68ba1cb4a67343a09adb98b1f9e9dcf5; no voters: 
I20260709 13:15:20.051685 10009 raft_consensus.cc:359] T 986acd03cce8455aad79beacff7aa154 P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.053433 10016 raft_consensus.cc:2804] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:20.053836 10016 raft_consensus.cc:493] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:20.054772 10016 raft_consensus.cc:3060] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:20.057196 10017 leader_election.cc:290] T cdcf499cc854451ab2609798dfcff264 P 08996682b9d042fc870e58e3db01e68e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813), 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130:44493)
I20260709 13:15:20.057440  9891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94510fe6d394c62a543502e8ace50eb" candidate_uuid: "63a62dec28ea4a44930d9c44561f273b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" is_pre_election: true
I20260709 13:15:20.058156 10042 leader_election.cc:290] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e [CANDIDATE]: Term 1 election: Requested vote from peers 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813), 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130:44493)
I20260709 13:15:20.053359 10009 raft_consensus.cc:385] T 986acd03cce8455aad79beacff7aa154 P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:20.058894  9892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e71d19ec8244748ceb00b2ab546eb9" candidate_uuid: "63a62dec28ea4a44930d9c44561f273b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" is_pre_election: true
I20260709 13:15:20.059621  9817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba81173e16b42ab92d7da17ba0518a7" candidate_uuid: "08996682b9d042fc870e58e3db01e68e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63a62dec28ea4a44930d9c44561f273b"
I20260709 13:15:20.060087  9815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdcf499cc854451ab2609798dfcff264" candidate_uuid: "08996682b9d042fc870e58e3db01e68e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63a62dec28ea4a44930d9c44561f273b" is_pre_election: true
I20260709 13:15:20.060763  9815 raft_consensus.cc:2468] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08996682b9d042fc870e58e3db01e68e in term 0.
W20260709 13:15:20.061095  9930 leader_election.cc:343] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:20.058094  9891 raft_consensus.cc:2468] T d94510fe6d394c62a543502e8ace50eb P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 63a62dec28ea4a44930d9c44561f273b in term 0.
I20260709 13:15:20.061949 10016 raft_consensus.cc:515] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.062220  9930 leader_election.cc:304] T cdcf499cc854451ab2609798dfcff264 P 08996682b9d042fc870e58e3db01e68e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08996682b9d042fc870e58e3db01e68e, 63a62dec28ea4a44930d9c44561f273b; no voters: 
I20260709 13:15:20.059032 10009 raft_consensus.cc:740] T 986acd03cce8455aad79beacff7aa154 P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63a62dec28ea4a44930d9c44561f273b, State: Initialized, Role: FOLLOWER
I20260709 13:15:20.063659  9780 leader_election.cc:304] T d94510fe6d394c62a543502e8ace50eb P 63a62dec28ea4a44930d9c44561f273b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63a62dec28ea4a44930d9c44561f273b, 68ba1cb4a67343a09adb98b1f9e9dcf5; no voters: 
I20260709 13:15:20.063951 10017 raft_consensus.cc:2804] T cdcf499cc854451ab2609798dfcff264 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:20.064047 10016 leader_election.cc:290] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [CANDIDATE]: Term 1 election: Requested vote from peers 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813), 08996682b9d042fc870e58e3db01e68e (127.9.116.131:40693)
I20260709 13:15:20.064456 10017 raft_consensus.cc:493] T cdcf499cc854451ab2609798dfcff264 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:20.065119 10017 raft_consensus.cc:3060] T cdcf499cc854451ab2609798dfcff264 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:20.065238 10015 raft_consensus.cc:2804] T d94510fe6d394c62a543502e8ace50eb P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:20.046445 10010 raft_consensus.cc:359] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.066030 10015 raft_consensus.cc:493] T d94510fe6d394c62a543502e8ace50eb P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:20.065970  9816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdcf499cc854451ab2609798dfcff264" candidate_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63a62dec28ea4a44930d9c44561f273b" is_pre_election: true
I20260709 13:15:20.067929 10010 raft_consensus.cc:385] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:20.065552 10042 raft_consensus.cc:493] T 15d491b33a7c47dab512ed81614b9b97 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.063329 10009 consensus_queue.cc:260] T 986acd03cce8455aad79beacff7aa154 P 63a62dec28ea4a44930d9c44561f273b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.068490 10018 raft_consensus.cc:697] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 LEADER]: Becoming Leader. State: Replica: 68ba1cb4a67343a09adb98b1f9e9dcf5, State: Running, Role: LEADER
I20260709 13:15:20.069754 10042 raft_consensus.cc:515] T 15d491b33a7c47dab512ed81614b9b97 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.085139  9966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdcf499cc854451ab2609798dfcff264" candidate_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08996682b9d042fc870e58e3db01e68e"
I20260709 13:15:20.066439  9891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdcf499cc854451ab2609798dfcff264" candidate_uuid: "08996682b9d042fc870e58e3db01e68e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" is_pre_election: true
I20260709 13:15:20.069218  9816 raft_consensus.cc:2468] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68ba1cb4a67343a09adb98b1f9e9dcf5 in term 0.
I20260709 13:15:20.068518 10015 raft_consensus.cc:3060] T d94510fe6d394c62a543502e8ace50eb P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:20.087347  9965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e71d19ec8244748ceb00b2ab546eb9" candidate_uuid: "63a62dec28ea4a44930d9c44561f273b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08996682b9d042fc870e58e3db01e68e" is_pre_election: true
I20260709 13:15:20.065876  9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba81173e16b42ab92d7da17ba0518a7" candidate_uuid: "08996682b9d042fc870e58e3db01e68e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5"
I20260709 13:15:20.068739 10010 raft_consensus.cc:740] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68ba1cb4a67343a09adb98b1f9e9dcf5, State: Initialized, Role: FOLLOWER
I20260709 13:15:20.085918 10018 consensus_queue.cc:237] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.090040  9891 raft_consensus.cc:2393] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 08996682b9d042fc870e58e3db01e68e in current term 1: Already voted for candidate 68ba1cb4a67343a09adb98b1f9e9dcf5 in this term.
I20260709 13:15:20.092576  9965 raft_consensus.cc:2393] T 40e71d19ec8244748ceb00b2ab546eb9 P 08996682b9d042fc870e58e3db01e68e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 63a62dec28ea4a44930d9c44561f273b in current term 1: Already voted for candidate 68ba1cb4a67343a09adb98b1f9e9dcf5 in this term.
I20260709 13:15:20.089955  9964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94510fe6d394c62a543502e8ace50eb" candidate_uuid: "63a62dec28ea4a44930d9c44561f273b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08996682b9d042fc870e58e3db01e68e" is_pre_election: true
I20260709 13:15:20.093466  9964 raft_consensus.cc:2468] T d94510fe6d394c62a543502e8ace50eb P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 63a62dec28ea4a44930d9c44561f273b in term 0.
I20260709 13:15:20.088683  9815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdcf499cc854451ab2609798dfcff264" candidate_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63a62dec28ea4a44930d9c44561f273b"
I20260709 13:15:20.095378  9817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d491b33a7c47dab512ed81614b9b97" candidate_uuid: "08996682b9d042fc870e58e3db01e68e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63a62dec28ea4a44930d9c44561f273b" is_pre_election: true
I20260709 13:15:20.095759  9815 raft_consensus.cc:3060] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:20.091957 10042 leader_election.cc:290] T 15d491b33a7c47dab512ed81614b9b97 P 08996682b9d042fc870e58e3db01e68e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813), 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130:44493)
W20260709 13:15:20.098690  9930 leader_election.cc:343] T 15d491b33a7c47dab512ed81614b9b97 P 08996682b9d042fc870e58e3db01e68e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:20.099265 10017 raft_consensus.cc:515] T cdcf499cc854451ab2609798dfcff264 P 08996682b9d042fc870e58e3db01e68e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.100896 10010 consensus_queue.cc:260] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.101289 10042 raft_consensus.cc:493] T 986acd03cce8455aad79beacff7aa154 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.102447  9891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d491b33a7c47dab512ed81614b9b97" candidate_uuid: "08996682b9d042fc870e58e3db01e68e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" is_pre_election: true
I20260709 13:15:20.103494  9891 raft_consensus.cc:2468] T 15d491b33a7c47dab512ed81614b9b97 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08996682b9d042fc870e58e3db01e68e in term 0.
I20260709 13:15:20.104332  9966 raft_consensus.cc:2393] T cdcf499cc854451ab2609798dfcff264 P 08996682b9d042fc870e58e3db01e68e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 68ba1cb4a67343a09adb98b1f9e9dcf5 in current term 1: Already voted for candidate 08996682b9d042fc870e58e3db01e68e in this term.
I20260709 13:15:20.105139  9929 leader_election.cc:304] T 15d491b33a7c47dab512ed81614b9b97 P 08996682b9d042fc870e58e3db01e68e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 08996682b9d042fc870e58e3db01e68e, 68ba1cb4a67343a09adb98b1f9e9dcf5; no voters: 63a62dec28ea4a44930d9c44561f273b
I20260709 13:15:20.108951 10017 leader_election.cc:290] T cdcf499cc854451ab2609798dfcff264 P 08996682b9d042fc870e58e3db01e68e [CANDIDATE]: Term 1 election: Requested vote from peers 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813), 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130:44493)
I20260709 13:15:20.097824  9890 raft_consensus.cc:3060] T dba81173e16b42ab92d7da17ba0518a7 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:20.102533 10042 raft_consensus.cc:515] T 986acd03cce8455aad79beacff7aa154 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.112632 10010 ts_tablet_manager.cc:1434] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5: Time spent starting tablet: real 0.069s	user 0.006s	sys 0.012s
I20260709 13:15:20.109781 10017 raft_consensus.cc:2804] T 15d491b33a7c47dab512ed81614b9b97 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:20.114487 10010 tablet_bootstrap.cc:492] T 986acd03cce8455aad79beacff7aa154 P 68ba1cb4a67343a09adb98b1f9e9dcf5: Bootstrap starting.
I20260709 13:15:20.116066 10042 leader_election.cc:290] T 986acd03cce8455aad79beacff7aa154 P 08996682b9d042fc870e58e3db01e68e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813), 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130:44493)
I20260709 13:15:20.117633  9816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdcf499cc854451ab2609798dfcff264" candidate_uuid: "08996682b9d042fc870e58e3db01e68e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63a62dec28ea4a44930d9c44561f273b"
I20260709 13:15:20.117991  9817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986acd03cce8455aad79beacff7aa154" candidate_uuid: "08996682b9d042fc870e58e3db01e68e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63a62dec28ea4a44930d9c44561f273b" is_pre_election: true
I20260709 13:15:20.119866  9817 raft_consensus.cc:2468] T 986acd03cce8455aad79beacff7aa154 P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08996682b9d042fc870e58e3db01e68e in term 0.
I20260709 13:15:20.111610  9815 raft_consensus.cc:2468] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68ba1cb4a67343a09adb98b1f9e9dcf5 in term 1.
I20260709 13:15:20.120934  9891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdcf499cc854451ab2609798dfcff264" candidate_uuid: "08996682b9d042fc870e58e3db01e68e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5"
I20260709 13:15:20.114446 10017 raft_consensus.cc:493] T 15d491b33a7c47dab512ed81614b9b97 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:20.121501  9889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986acd03cce8455aad79beacff7aa154" candidate_uuid: "08996682b9d042fc870e58e3db01e68e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" is_pre_election: true
I20260709 13:15:20.122839  9891 raft_consensus.cc:2393] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 08996682b9d042fc870e58e3db01e68e in current term 1: Already voted for candidate 68ba1cb4a67343a09adb98b1f9e9dcf5 in this term.
I20260709 13:15:20.122907  9856 leader_election.cc:304] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 63a62dec28ea4a44930d9c44561f273b, 68ba1cb4a67343a09adb98b1f9e9dcf5; no voters: 08996682b9d042fc870e58e3db01e68e
I20260709 13:15:20.123874  9930 leader_election.cc:304] T 986acd03cce8455aad79beacff7aa154 P 08996682b9d042fc870e58e3db01e68e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08996682b9d042fc870e58e3db01e68e, 63a62dec28ea4a44930d9c44561f273b; no voters: 
I20260709 13:15:20.124121 10009 ts_tablet_manager.cc:1434] T 986acd03cce8455aad79beacff7aa154 P 63a62dec28ea4a44930d9c44561f273b: Time spent starting tablet: real 0.076s	user 0.019s	sys 0.006s
W20260709 13:15:20.124922  9929 leader_election.cc:343] T 986acd03cce8455aad79beacff7aa154 P 08996682b9d042fc870e58e3db01e68e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130:44493): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:20.125818 10052 raft_consensus.cc:2804] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:20.126492 10052 raft_consensus.cc:697] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 LEADER]: Becoming Leader. State: Replica: 68ba1cb4a67343a09adb98b1f9e9dcf5, State: Running, Role: LEADER
I20260709 13:15:20.127023 10042 raft_consensus.cc:2804] T 986acd03cce8455aad79beacff7aa154 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:20.127449 10042 raft_consensus.cc:493] T 986acd03cce8455aad79beacff7aa154 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:20.128118 10010 tablet_bootstrap.cc:654] T 986acd03cce8455aad79beacff7aa154 P 68ba1cb4a67343a09adb98b1f9e9dcf5: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:20.123469 10017 raft_consensus.cc:3060] T 15d491b33a7c47dab512ed81614b9b97 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:20.127403 10052 consensus_queue.cc:237] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.130365  9930 leader_election.cc:304] T cdcf499cc854451ab2609798dfcff264 P 08996682b9d042fc870e58e3db01e68e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 08996682b9d042fc870e58e3db01e68e; no voters: 63a62dec28ea4a44930d9c44561f273b, 68ba1cb4a67343a09adb98b1f9e9dcf5
I20260709 13:15:20.128024 10042 raft_consensus.cc:3060] T 986acd03cce8455aad79beacff7aa154 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:20.136075 10015 raft_consensus.cc:515] T d94510fe6d394c62a543502e8ace50eb P 63a62dec28ea4a44930d9c44561f273b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.145332  9891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94510fe6d394c62a543502e8ace50eb" candidate_uuid: "63a62dec28ea4a44930d9c44561f273b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5"
I20260709 13:15:20.145462  9966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94510fe6d394c62a543502e8ace50eb" candidate_uuid: "63a62dec28ea4a44930d9c44561f273b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08996682b9d042fc870e58e3db01e68e"
I20260709 13:15:20.145975  9891 raft_consensus.cc:3060] T d94510fe6d394c62a543502e8ace50eb P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:20.146095  9966 raft_consensus.cc:3060] T d94510fe6d394c62a543502e8ace50eb P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:20.147619 10009 tablet_bootstrap.cc:492] T dba81173e16b42ab92d7da17ba0518a7 P 63a62dec28ea4a44930d9c44561f273b: Bootstrap starting.
I20260709 13:15:20.148211 10015 leader_election.cc:290] T d94510fe6d394c62a543502e8ace50eb P 63a62dec28ea4a44930d9c44561f273b [CANDIDATE]: Term 1 election: Requested vote from peers 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130:44493), 08996682b9d042fc870e58e3db01e68e (127.9.116.131:40693)
I20260709 13:15:20.151893 10017 raft_consensus.cc:515] T 15d491b33a7c47dab512ed81614b9b97 P 08996682b9d042fc870e58e3db01e68e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.152745  9966 raft_consensus.cc:2468] T d94510fe6d394c62a543502e8ace50eb P 08996682b9d042fc870e58e3db01e68e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63a62dec28ea4a44930d9c44561f273b in term 1.
I20260709 13:15:20.154069  9781 leader_election.cc:304] T d94510fe6d394c62a543502e8ace50eb P 63a62dec28ea4a44930d9c44561f273b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08996682b9d042fc870e58e3db01e68e, 63a62dec28ea4a44930d9c44561f273b; no voters: 
I20260709 13:15:20.137596 10053 raft_consensus.cc:2749] T cdcf499cc854451ab2609798dfcff264 P 08996682b9d042fc870e58e3db01e68e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:15:20.155471  9889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d491b33a7c47dab512ed81614b9b97" candidate_uuid: "08996682b9d042fc870e58e3db01e68e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5"
I20260709 13:15:20.156078  9889 raft_consensus.cc:3060] T 15d491b33a7c47dab512ed81614b9b97 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:20.156181 10015 raft_consensus.cc:2804] T d94510fe6d394c62a543502e8ace50eb P 63a62dec28ea4a44930d9c44561f273b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:20.156810 10015 raft_consensus.cc:697] T d94510fe6d394c62a543502e8ace50eb P 63a62dec28ea4a44930d9c44561f273b [term 1 LEADER]: Becoming Leader. State: Replica: 63a62dec28ea4a44930d9c44561f273b, State: Running, Role: LEADER
I20260709 13:15:20.157666 10015 consensus_queue.cc:237] T d94510fe6d394c62a543502e8ace50eb P 63a62dec28ea4a44930d9c44561f273b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.159900  9816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d491b33a7c47dab512ed81614b9b97" candidate_uuid: "08996682b9d042fc870e58e3db01e68e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63a62dec28ea4a44930d9c44561f273b"
W20260709 13:15:20.162672  9930 leader_election.cc:343] T 15d491b33a7c47dab512ed81614b9b97 P 08996682b9d042fc870e58e3db01e68e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:20.164219 10017 leader_election.cc:290] T 15d491b33a7c47dab512ed81614b9b97 P 08996682b9d042fc870e58e3db01e68e [CANDIDATE]: Term 1 election: Requested vote from peers 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813), 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130:44493)
I20260709 13:15:20.165794 10016 consensus_queue.cc:1048] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:20.167631 10009 tablet_bootstrap.cc:654] T dba81173e16b42ab92d7da17ba0518a7 P 63a62dec28ea4a44930d9c44561f273b: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:20.171577  9891 raft_consensus.cc:2468] T d94510fe6d394c62a543502e8ace50eb P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63a62dec28ea4a44930d9c44561f273b in term 1.
I20260709 13:15:20.180405 10009 tablet_bootstrap.cc:492] T dba81173e16b42ab92d7da17ba0518a7 P 63a62dec28ea4a44930d9c44561f273b: No bootstrap required, opened a new log
I20260709 13:15:20.180907 10009 ts_tablet_manager.cc:1403] T dba81173e16b42ab92d7da17ba0518a7 P 63a62dec28ea4a44930d9c44561f273b: Time spent bootstrapping tablet: real 0.033s	user 0.004s	sys 0.008s
I20260709 13:15:20.181066 10042 raft_consensus.cc:515] T 986acd03cce8455aad79beacff7aa154 P 08996682b9d042fc870e58e3db01e68e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.182508  9780 leader_election.cc:304] T 40e71d19ec8244748ceb00b2ab546eb9 P 63a62dec28ea4a44930d9c44561f273b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 63a62dec28ea4a44930d9c44561f273b; no voters: 08996682b9d042fc870e58e3db01e68e, 68ba1cb4a67343a09adb98b1f9e9dcf5
I20260709 13:15:20.183319 10015 raft_consensus.cc:2749] T 40e71d19ec8244748ceb00b2ab546eb9 P 63a62dec28ea4a44930d9c44561f273b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:20.184237 10009 raft_consensus.cc:359] T dba81173e16b42ab92d7da17ba0518a7 P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.185259 10009 raft_consensus.cc:385] T dba81173e16b42ab92d7da17ba0518a7 P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:20.185632 10009 raft_consensus.cc:740] T dba81173e16b42ab92d7da17ba0518a7 P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63a62dec28ea4a44930d9c44561f273b, State: Initialized, Role: FOLLOWER
I20260709 13:15:20.187647  9815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986acd03cce8455aad79beacff7aa154" candidate_uuid: "08996682b9d042fc870e58e3db01e68e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63a62dec28ea4a44930d9c44561f273b"
I20260709 13:15:20.188516  9815 raft_consensus.cc:3060] T 986acd03cce8455aad79beacff7aa154 P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:20.188982  9713 catalog_manager.cc:5697] T d94510fe6d394c62a543502e8ace50eb P 63a62dec28ea4a44930d9c44561f273b reported cstate change: term changed from 0 to 1, leader changed from <none> to 63a62dec28ea4a44930d9c44561f273b (127.9.116.129). New cstate: current_term: 1 leader_uuid: "63a62dec28ea4a44930d9c44561f273b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:20.194698 10009 consensus_queue.cc:260] T dba81173e16b42ab92d7da17ba0518a7 P 63a62dec28ea4a44930d9c44561f273b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.197170 10009 ts_tablet_manager.cc:1434] T dba81173e16b42ab92d7da17ba0518a7 P 63a62dec28ea4a44930d9c44561f273b: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.004s
I20260709 13:15:20.198132 10009 tablet_bootstrap.cc:492] T 15d491b33a7c47dab512ed81614b9b97 P 63a62dec28ea4a44930d9c44561f273b: Bootstrap starting.
I20260709 13:15:20.200407  9815 raft_consensus.cc:2468] T 986acd03cce8455aad79beacff7aa154 P 63a62dec28ea4a44930d9c44561f273b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08996682b9d042fc870e58e3db01e68e in term 1.
I20260709 13:15:20.202064  9930 leader_election.cc:304] T 986acd03cce8455aad79beacff7aa154 P 08996682b9d042fc870e58e3db01e68e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08996682b9d042fc870e58e3db01e68e, 63a62dec28ea4a44930d9c44561f273b; no voters: 
I20260709 13:15:20.203156 10017 raft_consensus.cc:2804] T 986acd03cce8455aad79beacff7aa154 P 08996682b9d042fc870e58e3db01e68e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:20.203631 10017 raft_consensus.cc:697] T 986acd03cce8455aad79beacff7aa154 P 08996682b9d042fc870e58e3db01e68e [term 1 LEADER]: Becoming Leader. State: Replica: 08996682b9d042fc870e58e3db01e68e, State: Running, Role: LEADER
I20260709 13:15:20.204634 10017 consensus_queue.cc:237] T 986acd03cce8455aad79beacff7aa154 P 08996682b9d042fc870e58e3db01e68e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.205801 10042 leader_election.cc:290] T 986acd03cce8455aad79beacff7aa154 P 08996682b9d042fc870e58e3db01e68e [CANDIDATE]: Term 1 election: Requested vote from peers 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813), 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130:44493)
I20260709 13:15:20.206496  9892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986acd03cce8455aad79beacff7aa154" candidate_uuid: "08996682b9d042fc870e58e3db01e68e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5"
W20260709 13:15:20.208036  9929 leader_election.cc:343] T 986acd03cce8455aad79beacff7aa154 P 08996682b9d042fc870e58e3db01e68e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130:44493): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:20.209679  9711 catalog_manager.cc:5697] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:20.213552 10016 raft_consensus.cc:493] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.214239 10016 raft_consensus.cc:515] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.216617 10016 leader_election.cc:290] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813), 08996682b9d042fc870e58e3db01e68e (127.9.116.131:40693)
I20260709 13:15:20.217937  9816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701d79f9f0674270ab399ece95bc5c03" candidate_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63a62dec28ea4a44930d9c44561f273b" is_pre_election: true
W20260709 13:15:20.222621  9856 leader_election.cc:343] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:20.223703 10061 consensus_queue.cc:1048] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:20.176250  9889 raft_consensus.cc:2468] T 15d491b33a7c47dab512ed81614b9b97 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08996682b9d042fc870e58e3db01e68e in term 1.
I20260709 13:15:20.223816  9966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701d79f9f0674270ab399ece95bc5c03" candidate_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08996682b9d042fc870e58e3db01e68e" is_pre_election: true
I20260709 13:15:20.224685  9966 raft_consensus.cc:2468] T 701d79f9f0674270ab399ece95bc5c03 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68ba1cb4a67343a09adb98b1f9e9dcf5 in term 0.
I20260709 13:15:20.225940  9856 leader_election.cc:304] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 08996682b9d042fc870e58e3db01e68e, 68ba1cb4a67343a09adb98b1f9e9dcf5; no voters: 63a62dec28ea4a44930d9c44561f273b
I20260709 13:15:20.226861 10018 raft_consensus.cc:2804] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:20.224097 10009 tablet_bootstrap.cc:654] T 15d491b33a7c47dab512ed81614b9b97 P 63a62dec28ea4a44930d9c44561f273b: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:20.227782  9929 leader_election.cc:304] T 15d491b33a7c47dab512ed81614b9b97 P 08996682b9d042fc870e58e3db01e68e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 08996682b9d042fc870e58e3db01e68e, 68ba1cb4a67343a09adb98b1f9e9dcf5; no voters: 63a62dec28ea4a44930d9c44561f273b
I20260709 13:15:20.227167  9713 catalog_manager.cc:5697] T 986acd03cce8455aad79beacff7aa154 P 08996682b9d042fc870e58e3db01e68e reported cstate change: term changed from 0 to 1, leader changed from <none> to 08996682b9d042fc870e58e3db01e68e (127.9.116.131). New cstate: current_term: 1 leader_uuid: "08996682b9d042fc870e58e3db01e68e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:20.228488 10018 raft_consensus.cc:493] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:20.229076 10018 raft_consensus.cc:3060] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:20.230131 10053 raft_consensus.cc:2804] T 15d491b33a7c47dab512ed81614b9b97 P 08996682b9d042fc870e58e3db01e68e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:20.230615 10053 raft_consensus.cc:697] T 15d491b33a7c47dab512ed81614b9b97 P 08996682b9d042fc870e58e3db01e68e [term 1 LEADER]: Becoming Leader. State: Replica: 08996682b9d042fc870e58e3db01e68e, State: Running, Role: LEADER
I20260709 13:15:20.174510  9890 raft_consensus.cc:2468] T dba81173e16b42ab92d7da17ba0518a7 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08996682b9d042fc870e58e3db01e68e in term 1.
I20260709 13:15:20.232417 10053 consensus_queue.cc:237] T 15d491b33a7c47dab512ed81614b9b97 P 08996682b9d042fc870e58e3db01e68e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.235116  9929 leader_election.cc:304] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 08996682b9d042fc870e58e3db01e68e, 68ba1cb4a67343a09adb98b1f9e9dcf5; no voters: 63a62dec28ea4a44930d9c44561f273b
I20260709 13:15:20.236320 10017 raft_consensus.cc:2804] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:20.236793 10017 raft_consensus.cc:697] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e [term 1 LEADER]: Becoming Leader. State: Replica: 08996682b9d042fc870e58e3db01e68e, State: Running, Role: LEADER
I20260709 13:15:20.237555 10017 consensus_queue.cc:237] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
W20260709 13:15:20.264899  9762 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:15:20.271068 10018 raft_consensus.cc:515] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.273373 10018 leader_election.cc:290] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [CANDIDATE]: Term 1 election: Requested vote from peers 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813), 08996682b9d042fc870e58e3db01e68e (127.9.116.131:40693)
I20260709 13:15:20.275012  9964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701d79f9f0674270ab399ece95bc5c03" candidate_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08996682b9d042fc870e58e3db01e68e"
I20260709 13:15:20.275636  9964 raft_consensus.cc:3060] T 701d79f9f0674270ab399ece95bc5c03 P 08996682b9d042fc870e58e3db01e68e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:20.276476 10010 tablet_bootstrap.cc:492] T 986acd03cce8455aad79beacff7aa154 P 68ba1cb4a67343a09adb98b1f9e9dcf5: No bootstrap required, opened a new log
I20260709 13:15:20.277004 10010 ts_tablet_manager.cc:1403] T 986acd03cce8455aad79beacff7aa154 P 68ba1cb4a67343a09adb98b1f9e9dcf5: Time spent bootstrapping tablet: real 0.163s	user 0.023s	sys 0.022s
I20260709 13:15:20.279938 10010 raft_consensus.cc:359] T 986acd03cce8455aad79beacff7aa154 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.280623 10010 raft_consensus.cc:385] T 986acd03cce8455aad79beacff7aa154 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:20.280913 10010 raft_consensus.cc:740] T 986acd03cce8455aad79beacff7aa154 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68ba1cb4a67343a09adb98b1f9e9dcf5, State: Initialized, Role: FOLLOWER
I20260709 13:15:20.281543 10010 consensus_queue.cc:260] T 986acd03cce8455aad79beacff7aa154 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.283727  9964 raft_consensus.cc:2468] T 701d79f9f0674270ab399ece95bc5c03 P 08996682b9d042fc870e58e3db01e68e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68ba1cb4a67343a09adb98b1f9e9dcf5 in term 1.
I20260709 13:15:20.284904  9815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701d79f9f0674270ab399ece95bc5c03" candidate_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63a62dec28ea4a44930d9c44561f273b"
I20260709 13:15:20.284310  9711 catalog_manager.cc:5697] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:20.285293  9856 leader_election.cc:304] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08996682b9d042fc870e58e3db01e68e, 68ba1cb4a67343a09adb98b1f9e9dcf5; no voters: 
W20260709 13:15:20.287317  9856 leader_election.cc:343] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:20.287977 10010 ts_tablet_manager.cc:1434] T 986acd03cce8455aad79beacff7aa154 P 68ba1cb4a67343a09adb98b1f9e9dcf5: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:15:20.288220 10018 raft_consensus.cc:2804] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:20.288686 10018 raft_consensus.cc:697] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 LEADER]: Becoming Leader. State: Replica: 68ba1cb4a67343a09adb98b1f9e9dcf5, State: Running, Role: LEADER
I20260709 13:15:20.289882 10018 consensus_queue.cc:237] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.304173 10009 tablet_bootstrap.cc:492] T 15d491b33a7c47dab512ed81614b9b97 P 63a62dec28ea4a44930d9c44561f273b: No bootstrap required, opened a new log
I20260709 13:15:20.304677 10009 ts_tablet_manager.cc:1403] T 15d491b33a7c47dab512ed81614b9b97 P 63a62dec28ea4a44930d9c44561f273b: Time spent bootstrapping tablet: real 0.107s	user 0.012s	sys 0.008s
I20260709 13:15:20.304672  9710 catalog_manager.cc:5697] T 15d491b33a7c47dab512ed81614b9b97 P 08996682b9d042fc870e58e3db01e68e reported cstate change: term changed from 0 to 1, leader changed from <none> to 08996682b9d042fc870e58e3db01e68e (127.9.116.131). New cstate: current_term: 1 leader_uuid: "08996682b9d042fc870e58e3db01e68e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:20.307205  9710 catalog_manager.cc:5697] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e reported cstate change: term changed from 0 to 1, leader changed from <none> to 08996682b9d042fc870e58e3db01e68e (127.9.116.131). New cstate: current_term: 1 leader_uuid: "08996682b9d042fc870e58e3db01e68e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:20.307811 10009 raft_consensus.cc:359] T 15d491b33a7c47dab512ed81614b9b97 P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.310174 10009 raft_consensus.cc:385] T 15d491b33a7c47dab512ed81614b9b97 P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:20.310616 10009 raft_consensus.cc:740] T 15d491b33a7c47dab512ed81614b9b97 P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63a62dec28ea4a44930d9c44561f273b, State: Initialized, Role: FOLLOWER
I20260709 13:15:20.311424 10009 consensus_queue.cc:260] T 15d491b33a7c47dab512ed81614b9b97 P 63a62dec28ea4a44930d9c44561f273b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.313805 10009 ts_tablet_manager.cc:1434] T 15d491b33a7c47dab512ed81614b9b97 P 63a62dec28ea4a44930d9c44561f273b: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:15:20.313702  9711 catalog_manager.cc:5697] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:20.314760 10009 tablet_bootstrap.cc:492] T 701d79f9f0674270ab399ece95bc5c03 P 63a62dec28ea4a44930d9c44561f273b: Bootstrap starting.
I20260709 13:15:20.320075 10009 tablet_bootstrap.cc:654] T 701d79f9f0674270ab399ece95bc5c03 P 63a62dec28ea4a44930d9c44561f273b: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:20.332260 10009 tablet_bootstrap.cc:492] T 701d79f9f0674270ab399ece95bc5c03 P 63a62dec28ea4a44930d9c44561f273b: No bootstrap required, opened a new log
I20260709 13:15:20.332718 10009 ts_tablet_manager.cc:1403] T 701d79f9f0674270ab399ece95bc5c03 P 63a62dec28ea4a44930d9c44561f273b: Time spent bootstrapping tablet: real 0.018s	user 0.009s	sys 0.007s
I20260709 13:15:20.335152 10009 raft_consensus.cc:359] T 701d79f9f0674270ab399ece95bc5c03 P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.335780 10009 raft_consensus.cc:385] T 701d79f9f0674270ab399ece95bc5c03 P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:20.336035 10009 raft_consensus.cc:740] T 701d79f9f0674270ab399ece95bc5c03 P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63a62dec28ea4a44930d9c44561f273b, State: Initialized, Role: FOLLOWER
I20260709 13:15:20.336634 10009 consensus_queue.cc:260] T 701d79f9f0674270ab399ece95bc5c03 P 63a62dec28ea4a44930d9c44561f273b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.338490 10009 ts_tablet_manager.cc:1434] T 701d79f9f0674270ab399ece95bc5c03 P 63a62dec28ea4a44930d9c44561f273b: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:15:20.353973  9890 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f27f84491f9c4b85af0ef5524c00f5e8"
dest_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5"
mode: GRACEFUL
new_leader_uuid: "63a62dec28ea4a44930d9c44561f273b"
 from {username='slave'} at 127.0.0.1:55700
I20260709 13:15:20.354555  9890 raft_consensus.cc:606] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 LEADER]: Received request to transfer leadership to TS 63a62dec28ea4a44930d9c44561f273b
I20260709 13:15:20.360252  9890 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "40e71d19ec8244748ceb00b2ab546eb9"
dest_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5"
mode: GRACEFUL
new_leader_uuid: "63a62dec28ea4a44930d9c44561f273b"
 from {username='slave'} at 127.0.0.1:55700
I20260709 13:15:20.360773  9890 raft_consensus.cc:606] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 LEADER]: Received request to transfer leadership to TS 63a62dec28ea4a44930d9c44561f273b
I20260709 13:15:20.365944 10017 consensus_queue.cc:1048] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e [LEADER]: Connected to new peer: Peer: permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:20.378952  9966 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e1e26033fa4044e89207d8960181bfad"
dest_uuid: "08996682b9d042fc870e58e3db01e68e"
mode: GRACEFUL
new_leader_uuid: "63a62dec28ea4a44930d9c44561f273b"
 from {username='slave'} at 127.0.0.1:53386
I20260709 13:15:20.380287  9966 raft_consensus.cc:606] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e [term 1 LEADER]: Received request to transfer leadership to TS 63a62dec28ea4a44930d9c44561f273b
I20260709 13:15:20.380833 10017 consensus_queue.cc:1048] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e [LEADER]: Connected to new peer: Peer: permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:20.391906 10015 raft_consensus.cc:493] T 15d491b33a7c47dab512ed81614b9b97 P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.392366  9890 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cdcf499cc854451ab2609798dfcff264"
dest_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5"
mode: GRACEFUL
new_leader_uuid: "63a62dec28ea4a44930d9c44561f273b"
 from {username='slave'} at 127.0.0.1:55700
I20260709 13:15:20.393065  9890 raft_consensus.cc:606] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 LEADER]: Received request to transfer leadership to TS 63a62dec28ea4a44930d9c44561f273b
I20260709 13:15:20.392407 10015 raft_consensus.cc:515] T 15d491b33a7c47dab512ed81614b9b97 P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.395952 10015 leader_election.cc:290] T 15d491b33a7c47dab512ed81614b9b97 P 63a62dec28ea4a44930d9c44561f273b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130:44493), 08996682b9d042fc870e58e3db01e68e (127.9.116.131:40693)
I20260709 13:15:20.396760  9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d491b33a7c47dab512ed81614b9b97" candidate_uuid: "63a62dec28ea4a44930d9c44561f273b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" is_pre_election: true
I20260709 13:15:20.397173  9966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d491b33a7c47dab512ed81614b9b97" candidate_uuid: "63a62dec28ea4a44930d9c44561f273b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08996682b9d042fc870e58e3db01e68e" is_pre_election: true
I20260709 13:15:20.397552  9890 raft_consensus.cc:2393] T 15d491b33a7c47dab512ed81614b9b97 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 63a62dec28ea4a44930d9c44561f273b in current term 1: Already voted for candidate 08996682b9d042fc870e58e3db01e68e in this term.
I20260709 13:15:20.398757  9780 leader_election.cc:304] T 15d491b33a7c47dab512ed81614b9b97 P 63a62dec28ea4a44930d9c44561f273b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 63a62dec28ea4a44930d9c44561f273b; no voters: 08996682b9d042fc870e58e3db01e68e, 68ba1cb4a67343a09adb98b1f9e9dcf5
I20260709 13:15:20.399595 10015 raft_consensus.cc:3060] T 15d491b33a7c47dab512ed81614b9b97 P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:20.400307 10042 raft_consensus.cc:993] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e: : Instructing follower 63a62dec28ea4a44930d9c44561f273b to start an election
I20260709 13:15:20.400730 10042 raft_consensus.cc:1081] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e [term 1 LEADER]: Signalling peer 63a62dec28ea4a44930d9c44561f273b to start an election
I20260709 13:15:20.403908  9815 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e1e26033fa4044e89207d8960181bfad"
dest_uuid: "63a62dec28ea4a44930d9c44561f273b"
 from {username='slave'} at 127.0.0.1:33738
I20260709 13:15:20.404548  9815 raft_consensus.cc:493] T e1e26033fa4044e89207d8960181bfad P 63a62dec28ea4a44930d9c44561f273b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:20.404990  9815 raft_consensus.cc:3060] T e1e26033fa4044e89207d8960181bfad P 63a62dec28ea4a44930d9c44561f273b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:20.405493  9966 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "986acd03cce8455aad79beacff7aa154"
dest_uuid: "08996682b9d042fc870e58e3db01e68e"
mode: GRACEFUL
new_leader_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5"
 from {username='slave'} at 127.0.0.1:53386
I20260709 13:15:20.406101  9966 raft_consensus.cc:606] T 986acd03cce8455aad79beacff7aa154 P 08996682b9d042fc870e58e3db01e68e [term 1 LEADER]: Received request to transfer leadership to TS 68ba1cb4a67343a09adb98b1f9e9dcf5
I20260709 13:15:20.406836 10015 raft_consensus.cc:2749] T 15d491b33a7c47dab512ed81614b9b97 P 63a62dec28ea4a44930d9c44561f273b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:20.411628  9815 raft_consensus.cc:515] T e1e26033fa4044e89207d8960181bfad P 63a62dec28ea4a44930d9c44561f273b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.412866  9966 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dba81173e16b42ab92d7da17ba0518a7"
dest_uuid: "08996682b9d042fc870e58e3db01e68e"
mode: GRACEFUL
new_leader_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5"
 from {username='slave'} at 127.0.0.1:53386
I20260709 13:15:20.413489  9966 raft_consensus.cc:606] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e [term 1 LEADER]: Received request to transfer leadership to TS 68ba1cb4a67343a09adb98b1f9e9dcf5
I20260709 13:15:20.414743  9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e26033fa4044e89207d8960181bfad" candidate_uuid: "63a62dec28ea4a44930d9c44561f273b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5"
I20260709 13:15:20.415369  9890 raft_consensus.cc:3060] T e1e26033fa4044e89207d8960181bfad P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:20.419760  9815 leader_election.cc:290] T e1e26033fa4044e89207d8960181bfad P 63a62dec28ea4a44930d9c44561f273b [CANDIDATE]: Term 2 election: Requested vote from peers 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130:44493), 08996682b9d042fc870e58e3db01e68e (127.9.116.131:40693)
I20260709 13:15:20.420307  9966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e26033fa4044e89207d8960181bfad" candidate_uuid: "63a62dec28ea4a44930d9c44561f273b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "08996682b9d042fc870e58e3db01e68e"
I20260709 13:15:20.421229  9966 raft_consensus.cc:3055] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:20.421574  9966 raft_consensus.cc:740] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 08996682b9d042fc870e58e3db01e68e, State: Running, Role: LEADER
I20260709 13:15:20.422433  9966 consensus_queue.cc:260] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.423859  9966 raft_consensus.cc:3060] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:20.423918  9964 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15d491b33a7c47dab512ed81614b9b97"
dest_uuid: "08996682b9d042fc870e58e3db01e68e"
mode: GRACEFUL
new_leader_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5"
 from {username='slave'} at 127.0.0.1:53386
I20260709 13:15:20.424533  9964 raft_consensus.cc:606] T 15d491b33a7c47dab512ed81614b9b97 P 08996682b9d042fc870e58e3db01e68e [term 1 LEADER]: Received request to transfer leadership to TS 68ba1cb4a67343a09adb98b1f9e9dcf5
I20260709 13:15:20.425598  9890 raft_consensus.cc:2468] T e1e26033fa4044e89207d8960181bfad P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63a62dec28ea4a44930d9c44561f273b in term 2.
I20260709 13:15:20.426682  9780 leader_election.cc:304] T e1e26033fa4044e89207d8960181bfad P 63a62dec28ea4a44930d9c44561f273b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63a62dec28ea4a44930d9c44561f273b, 68ba1cb4a67343a09adb98b1f9e9dcf5; no voters: 
I20260709 13:15:20.427568 10015 raft_consensus.cc:2804] T e1e26033fa4044e89207d8960181bfad P 63a62dec28ea4a44930d9c44561f273b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:20.428013 10015 raft_consensus.cc:697] T e1e26033fa4044e89207d8960181bfad P 63a62dec28ea4a44930d9c44561f273b [term 2 LEADER]: Becoming Leader. State: Replica: 63a62dec28ea4a44930d9c44561f273b, State: Running, Role: LEADER
I20260709 13:15:20.428778 10015 consensus_queue.cc:237] T e1e26033fa4044e89207d8960181bfad P 63a62dec28ea4a44930d9c44561f273b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.431514  9966 raft_consensus.cc:2468] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63a62dec28ea4a44930d9c44561f273b in term 2.
I20260709 13:15:20.442286  9815 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d94510fe6d394c62a543502e8ace50eb"
dest_uuid: "63a62dec28ea4a44930d9c44561f273b"
mode: GRACEFUL
new_leader_uuid: "08996682b9d042fc870e58e3db01e68e"
 from {username='slave'} at 127.0.0.1:33754
I20260709 13:15:20.442966  9815 raft_consensus.cc:606] T d94510fe6d394c62a543502e8ace50eb P 63a62dec28ea4a44930d9c44561f273b [term 1 LEADER]: Received request to transfer leadership to TS 08996682b9d042fc870e58e3db01e68e
I20260709 13:15:20.442131  9711 catalog_manager.cc:5697] T e1e26033fa4044e89207d8960181bfad P 63a62dec28ea4a44930d9c44561f273b reported cstate change: term changed from 1 to 2, leader changed from 08996682b9d042fc870e58e3db01e68e (127.9.116.131) to 63a62dec28ea4a44930d9c44561f273b (127.9.116.129). New cstate: current_term: 2 leader_uuid: "63a62dec28ea4a44930d9c44561f273b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:20.453850 10018 raft_consensus.cc:493] T 986acd03cce8455aad79beacff7aa154 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.454366 10018 raft_consensus.cc:515] T 986acd03cce8455aad79beacff7aa154 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.456547 10018 leader_election.cc:290] T 986acd03cce8455aad79beacff7aa154 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813), 08996682b9d042fc870e58e3db01e68e (127.9.116.131:40693)
I20260709 13:15:20.457646  9966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986acd03cce8455aad79beacff7aa154" candidate_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08996682b9d042fc870e58e3db01e68e" is_pre_election: true
I20260709 13:15:20.457619  9815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986acd03cce8455aad79beacff7aa154" candidate_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63a62dec28ea4a44930d9c44561f273b" is_pre_election: true
I20260709 13:15:20.458639  9815 raft_consensus.cc:2393] T 986acd03cce8455aad79beacff7aa154 P 63a62dec28ea4a44930d9c44561f273b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 68ba1cb4a67343a09adb98b1f9e9dcf5 in current term 1: Already voted for candidate 08996682b9d042fc870e58e3db01e68e in this term.
I20260709 13:15:20.459614  9856 leader_election.cc:304] T 986acd03cce8455aad79beacff7aa154 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 68ba1cb4a67343a09adb98b1f9e9dcf5; no voters: 08996682b9d042fc870e58e3db01e68e, 63a62dec28ea4a44930d9c44561f273b
I20260709 13:15:20.460335 10018 raft_consensus.cc:3060] T 986acd03cce8455aad79beacff7aa154 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:20.464850 10018 raft_consensus.cc:2749] T 986acd03cce8455aad79beacff7aa154 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:20.548584 10015 consensus_queue.cc:1048] T d94510fe6d394c62a543502e8ace50eb P 63a62dec28ea4a44930d9c44561f273b [LEADER]: Connected to new peer: Peer: permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:20.561697 10015 consensus_queue.cc:1048] T d94510fe6d394c62a543502e8ace50eb P 63a62dec28ea4a44930d9c44561f273b [LEADER]: Connected to new peer: Peer: permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:20.577378 10054 raft_consensus.cc:993] T d94510fe6d394c62a543502e8ace50eb P 63a62dec28ea4a44930d9c44561f273b: : Instructing follower 08996682b9d042fc870e58e3db01e68e to start an election
I20260709 13:15:20.577981 10080 raft_consensus.cc:1081] T d94510fe6d394c62a543502e8ace50eb P 63a62dec28ea4a44930d9c44561f273b [term 1 LEADER]: Signalling peer 08996682b9d042fc870e58e3db01e68e to start an election
I20260709 13:15:20.579932  9966 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d94510fe6d394c62a543502e8ace50eb"
dest_uuid: "08996682b9d042fc870e58e3db01e68e"
 from {username='slave'} at 127.0.0.1:53378
I20260709 13:15:20.580642  9966 raft_consensus.cc:493] T d94510fe6d394c62a543502e8ace50eb P 08996682b9d042fc870e58e3db01e68e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:20.581029  9966 raft_consensus.cc:3060] T d94510fe6d394c62a543502e8ace50eb P 08996682b9d042fc870e58e3db01e68e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:20.588634  9966 raft_consensus.cc:515] T d94510fe6d394c62a543502e8ace50eb P 08996682b9d042fc870e58e3db01e68e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.590857  9966 leader_election.cc:290] T d94510fe6d394c62a543502e8ace50eb P 08996682b9d042fc870e58e3db01e68e [CANDIDATE]: Term 2 election: Requested vote from peers 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813), 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130:44493)
I20260709 13:15:20.591781  9815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94510fe6d394c62a543502e8ace50eb" candidate_uuid: "08996682b9d042fc870e58e3db01e68e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "63a62dec28ea4a44930d9c44561f273b"
I20260709 13:15:20.592430  9815 raft_consensus.cc:3055] T d94510fe6d394c62a543502e8ace50eb P 63a62dec28ea4a44930d9c44561f273b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:20.592783  9815 raft_consensus.cc:740] T d94510fe6d394c62a543502e8ace50eb P 63a62dec28ea4a44930d9c44561f273b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 63a62dec28ea4a44930d9c44561f273b, State: Running, Role: LEADER
I20260709 13:15:20.593576  9815 consensus_queue.cc:260] T d94510fe6d394c62a543502e8ace50eb P 63a62dec28ea4a44930d9c44561f273b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.594944  9815 raft_consensus.cc:3060] T d94510fe6d394c62a543502e8ace50eb P 63a62dec28ea4a44930d9c44561f273b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:20.600060  9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94510fe6d394c62a543502e8ace50eb" candidate_uuid: "08996682b9d042fc870e58e3db01e68e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5"
I20260709 13:15:20.600327  9815 raft_consensus.cc:2468] T d94510fe6d394c62a543502e8ace50eb P 63a62dec28ea4a44930d9c44561f273b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08996682b9d042fc870e58e3db01e68e in term 2.
I20260709 13:15:20.600706  9890 raft_consensus.cc:3060] T d94510fe6d394c62a543502e8ace50eb P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:20.601446  9930 leader_election.cc:304] T d94510fe6d394c62a543502e8ace50eb P 08996682b9d042fc870e58e3db01e68e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08996682b9d042fc870e58e3db01e68e, 63a62dec28ea4a44930d9c44561f273b; no voters: 
I20260709 13:15:20.602312 10053 raft_consensus.cc:2804] T d94510fe6d394c62a543502e8ace50eb P 08996682b9d042fc870e58e3db01e68e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:20.602767 10053 raft_consensus.cc:697] T d94510fe6d394c62a543502e8ace50eb P 08996682b9d042fc870e58e3db01e68e [term 2 LEADER]: Becoming Leader. State: Replica: 08996682b9d042fc870e58e3db01e68e, State: Running, Role: LEADER
I20260709 13:15:20.603631 10053 consensus_queue.cc:237] T d94510fe6d394c62a543502e8ace50eb P 08996682b9d042fc870e58e3db01e68e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.606071  9890 raft_consensus.cc:2468] T d94510fe6d394c62a543502e8ace50eb P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08996682b9d042fc870e58e3db01e68e in term 2.
I20260709 13:15:20.611971  9713 catalog_manager.cc:5697] T d94510fe6d394c62a543502e8ace50eb P 08996682b9d042fc870e58e3db01e68e reported cstate change: term changed from 1 to 2, leader changed from 63a62dec28ea4a44930d9c44561f273b (127.9.116.129) to 08996682b9d042fc870e58e3db01e68e (127.9.116.131). New cstate: current_term: 2 leader_uuid: "08996682b9d042fc870e58e3db01e68e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:20.630050 10018 consensus_queue.cc:1048] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:20.636956 10080 raft_consensus.cc:493] T dba81173e16b42ab92d7da17ba0518a7 P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.637563 10080 raft_consensus.cc:515] T dba81173e16b42ab92d7da17ba0518a7 P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.640784  9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba81173e16b42ab92d7da17ba0518a7" candidate_uuid: "63a62dec28ea4a44930d9c44561f273b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" is_pre_election: true
I20260709 13:15:20.641561  9890 raft_consensus.cc:2393] T dba81173e16b42ab92d7da17ba0518a7 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 63a62dec28ea4a44930d9c44561f273b in current term 1: Already voted for candidate 08996682b9d042fc870e58e3db01e68e in this term.
I20260709 13:15:20.642688 10053 consensus_queue.cc:1048] T 15d491b33a7c47dab512ed81614b9b97 P 08996682b9d042fc870e58e3db01e68e [LEADER]: Connected to new peer: Peer: permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:20.644088 10080 leader_election.cc:290] T dba81173e16b42ab92d7da17ba0518a7 P 63a62dec28ea4a44930d9c44561f273b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130:44493), 08996682b9d042fc870e58e3db01e68e (127.9.116.131:40693)
I20260709 13:15:20.647027 10018 consensus_queue.cc:1048] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:20.648716  9966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba81173e16b42ab92d7da17ba0518a7" candidate_uuid: "63a62dec28ea4a44930d9c44561f273b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08996682b9d042fc870e58e3db01e68e" is_pre_election: true
I20260709 13:15:20.650460  9781 leader_election.cc:304] T dba81173e16b42ab92d7da17ba0518a7 P 63a62dec28ea4a44930d9c44561f273b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 63a62dec28ea4a44930d9c44561f273b; no voters: 08996682b9d042fc870e58e3db01e68e, 68ba1cb4a67343a09adb98b1f9e9dcf5
I20260709 13:15:20.652779 10080 raft_consensus.cc:3060] T dba81173e16b42ab92d7da17ba0518a7 P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:20.661329 10080 raft_consensus.cc:2749] T dba81173e16b42ab92d7da17ba0518a7 P 63a62dec28ea4a44930d9c44561f273b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:20.666196 10042 raft_consensus.cc:993] T 15d491b33a7c47dab512ed81614b9b97 P 08996682b9d042fc870e58e3db01e68e: : Instructing follower 68ba1cb4a67343a09adb98b1f9e9dcf5 to start an election
I20260709 13:15:20.666671 10042 raft_consensus.cc:1081] T 15d491b33a7c47dab512ed81614b9b97 P 08996682b9d042fc870e58e3db01e68e [term 1 LEADER]: Signalling peer 68ba1cb4a67343a09adb98b1f9e9dcf5 to start an election
I20260709 13:15:20.668896  9889 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "15d491b33a7c47dab512ed81614b9b97"
dest_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5"
 from {username='slave'} at 127.0.0.1:55684
I20260709 13:15:20.669471  9889 raft_consensus.cc:493] T 15d491b33a7c47dab512ed81614b9b97 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:20.669831  9889 raft_consensus.cc:3060] T 15d491b33a7c47dab512ed81614b9b97 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:20.676781 10017 consensus_queue.cc:1048] T 15d491b33a7c47dab512ed81614b9b97 P 08996682b9d042fc870e58e3db01e68e [LEADER]: Connected to new peer: Peer: permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:20.681843 10018 raft_consensus.cc:993] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5: : Instructing follower 63a62dec28ea4a44930d9c44561f273b to start an election
I20260709 13:15:20.681449  9889 raft_consensus.cc:515] T 15d491b33a7c47dab512ed81614b9b97 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.682299 10018 raft_consensus.cc:1081] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 LEADER]: Signalling peer 63a62dec28ea4a44930d9c44561f273b to start an election
I20260709 13:15:20.684556  9817 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "40e71d19ec8244748ceb00b2ab546eb9"
dest_uuid: "63a62dec28ea4a44930d9c44561f273b"
 from {username='slave'} at 127.0.0.1:33722
I20260709 13:15:20.685165  9817 raft_consensus.cc:493] T 40e71d19ec8244748ceb00b2ab546eb9 P 63a62dec28ea4a44930d9c44561f273b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:20.682842  9890 raft_consensus.cc:1240] T 15d491b33a7c47dab512ed81614b9b97 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 2 FOLLOWER]: Rejecting Update request from peer 08996682b9d042fc870e58e3db01e68e for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:20.685642  9817 raft_consensus.cc:3060] T 40e71d19ec8244748ceb00b2ab546eb9 P 63a62dec28ea4a44930d9c44561f273b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:20.690320 10042 consensus_queue.cc:1059] T 15d491b33a7c47dab512ed81614b9b97 P 08996682b9d042fc870e58e3db01e68e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:20.690421 10017 consensus_queue.cc:1048] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e [LEADER]: Connected to new peer: Peer: permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:20.691262 10042 raft_consensus.cc:3055] T 15d491b33a7c47dab512ed81614b9b97 P 08996682b9d042fc870e58e3db01e68e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:20.691614 10042 raft_consensus.cc:740] T 15d491b33a7c47dab512ed81614b9b97 P 08996682b9d042fc870e58e3db01e68e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 08996682b9d042fc870e58e3db01e68e, State: Running, Role: LEADER
I20260709 13:15:20.692462 10042 consensus_queue.cc:260] T 15d491b33a7c47dab512ed81614b9b97 P 08996682b9d042fc870e58e3db01e68e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.693511 10042 raft_consensus.cc:3060] T 15d491b33a7c47dab512ed81614b9b97 P 08996682b9d042fc870e58e3db01e68e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:20.694777  9966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d491b33a7c47dab512ed81614b9b97" candidate_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "08996682b9d042fc870e58e3db01e68e"
I20260709 13:15:20.693794  9814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d491b33a7c47dab512ed81614b9b97" candidate_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "63a62dec28ea4a44930d9c44561f273b"
I20260709 13:15:20.692698  9817 raft_consensus.cc:515] T 40e71d19ec8244748ceb00b2ab546eb9 P 63a62dec28ea4a44930d9c44561f273b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.685204  9889 leader_election.cc:290] T 15d491b33a7c47dab512ed81614b9b97 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [CANDIDATE]: Term 2 election: Requested vote from peers 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813), 08996682b9d042fc870e58e3db01e68e (127.9.116.131:40693)
I20260709 13:15:20.701792  9964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e71d19ec8244748ceb00b2ab546eb9" candidate_uuid: "63a62dec28ea4a44930d9c44561f273b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "08996682b9d042fc870e58e3db01e68e"
I20260709 13:15:20.702054  9817 leader_election.cc:290] T 40e71d19ec8244748ceb00b2ab546eb9 P 63a62dec28ea4a44930d9c44561f273b [CANDIDATE]: Term 2 election: Requested vote from peers 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130:44493), 08996682b9d042fc870e58e3db01e68e (127.9.116.131:40693)
I20260709 13:15:20.702394  9964 raft_consensus.cc:3060] T 40e71d19ec8244748ceb00b2ab546eb9 P 08996682b9d042fc870e58e3db01e68e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:20.703488  9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e71d19ec8244748ceb00b2ab546eb9" candidate_uuid: "63a62dec28ea4a44930d9c44561f273b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5"
I20260709 13:15:20.704124  9890 raft_consensus.cc:3055] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:20.704510  9890 raft_consensus.cc:740] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 68ba1cb4a67343a09adb98b1f9e9dcf5, State: Running, Role: LEADER
I20260709 13:15:20.705363  9890 consensus_queue.cc:260] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.706732  9890 raft_consensus.cc:3060] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:20.709235  9964 raft_consensus.cc:2468] T 40e71d19ec8244748ceb00b2ab546eb9 P 08996682b9d042fc870e58e3db01e68e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63a62dec28ea4a44930d9c44561f273b in term 2.
I20260709 13:15:20.712381  9781 leader_election.cc:304] T 40e71d19ec8244748ceb00b2ab546eb9 P 63a62dec28ea4a44930d9c44561f273b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08996682b9d042fc870e58e3db01e68e, 63a62dec28ea4a44930d9c44561f273b; no voters: 
I20260709 13:15:20.713224 10080 raft_consensus.cc:2804] T 40e71d19ec8244748ceb00b2ab546eb9 P 63a62dec28ea4a44930d9c44561f273b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:20.713693 10080 raft_consensus.cc:697] T 40e71d19ec8244748ceb00b2ab546eb9 P 63a62dec28ea4a44930d9c44561f273b [term 2 LEADER]: Becoming Leader. State: Replica: 63a62dec28ea4a44930d9c44561f273b, State: Running, Role: LEADER
I20260709 13:15:20.714514 10080 consensus_queue.cc:237] T 40e71d19ec8244748ceb00b2ab546eb9 P 63a62dec28ea4a44930d9c44561f273b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.716205  9890 raft_consensus.cc:2468] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63a62dec28ea4a44930d9c44561f273b in term 2.
I20260709 13:15:20.717465  9966 raft_consensus.cc:2468] T 15d491b33a7c47dab512ed81614b9b97 P 08996682b9d042fc870e58e3db01e68e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68ba1cb4a67343a09adb98b1f9e9dcf5 in term 2.
I20260709 13:15:20.719368  9856 leader_election.cc:304] T 15d491b33a7c47dab512ed81614b9b97 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 08996682b9d042fc870e58e3db01e68e, 68ba1cb4a67343a09adb98b1f9e9dcf5; no voters: 63a62dec28ea4a44930d9c44561f273b
I20260709 13:15:20.720498 10018 raft_consensus.cc:2804] T 15d491b33a7c47dab512ed81614b9b97 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:20.721040 10018 raft_consensus.cc:697] T 15d491b33a7c47dab512ed81614b9b97 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 2 LEADER]: Becoming Leader. State: Replica: 68ba1cb4a67343a09adb98b1f9e9dcf5, State: Running, Role: LEADER
I20260709 13:15:20.721859 10018 consensus_queue.cc:237] T 15d491b33a7c47dab512ed81614b9b97 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.726636 10042 consensus_queue.cc:1048] T 986acd03cce8455aad79beacff7aa154 P 08996682b9d042fc870e58e3db01e68e [LEADER]: Connected to new peer: Peer: permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:20.734242 10042 consensus_queue.cc:1048] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e [LEADER]: Connected to new peer: Peer: permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:20.734639  9711 catalog_manager.cc:5697] T 15d491b33a7c47dab512ed81614b9b97 P 68ba1cb4a67343a09adb98b1f9e9dcf5 reported cstate change: term changed from 1 to 2, leader changed from 08996682b9d042fc870e58e3db01e68e (127.9.116.131) to 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130). New cstate: current_term: 2 leader_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:20.744336 10065 consensus_queue.cc:1048] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:20.756196 10017 raft_consensus.cc:993] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e: : Instructing follower 68ba1cb4a67343a09adb98b1f9e9dcf5 to start an election
I20260709 13:15:20.756690 10087 raft_consensus.cc:1081] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e [term 1 LEADER]: Signalling peer 68ba1cb4a67343a09adb98b1f9e9dcf5 to start an election
I20260709 13:15:20.757380 10066 consensus_queue.cc:1048] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:20.758633  9890 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dba81173e16b42ab92d7da17ba0518a7"
dest_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5"
 from {username='slave'} at 127.0.0.1:55684
I20260709 13:15:20.759681  9890 raft_consensus.cc:493] T dba81173e16b42ab92d7da17ba0518a7 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:20.760121  9890 raft_consensus.cc:3060] T dba81173e16b42ab92d7da17ba0518a7 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:20.767251  9890 raft_consensus.cc:515] T dba81173e16b42ab92d7da17ba0518a7 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.769450  9890 leader_election.cc:290] T dba81173e16b42ab92d7da17ba0518a7 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [CANDIDATE]: Term 2 election: Requested vote from peers 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813), 08996682b9d042fc870e58e3db01e68e (127.9.116.131:40693)
I20260709 13:15:20.772610  9964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba81173e16b42ab92d7da17ba0518a7" candidate_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "08996682b9d042fc870e58e3db01e68e"
I20260709 13:15:20.773020  9813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba81173e16b42ab92d7da17ba0518a7" candidate_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "63a62dec28ea4a44930d9c44561f273b"
I20260709 13:15:20.773242  9964 raft_consensus.cc:3055] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:20.773583  9813 raft_consensus.cc:3060] T dba81173e16b42ab92d7da17ba0518a7 P 63a62dec28ea4a44930d9c44561f273b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:20.773622  9964 raft_consensus.cc:740] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 08996682b9d042fc870e58e3db01e68e, State: Running, Role: LEADER
I20260709 13:15:20.774577  9964 consensus_queue.cc:260] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.775908  9964 raft_consensus.cc:3060] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:20.782372 10080 raft_consensus.cc:493] T 701d79f9f0674270ab399ece95bc5c03 P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.782917 10080 raft_consensus.cc:515] T 701d79f9f0674270ab399ece95bc5c03 P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.785012 10080 leader_election.cc:290] T 701d79f9f0674270ab399ece95bc5c03 P 63a62dec28ea4a44930d9c44561f273b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130:44493), 08996682b9d042fc870e58e3db01e68e (127.9.116.131:40693)
I20260709 13:15:20.785849  9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701d79f9f0674270ab399ece95bc5c03" candidate_uuid: "63a62dec28ea4a44930d9c44561f273b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" is_pre_election: true
I20260709 13:15:20.791011  9964 raft_consensus.cc:2468] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68ba1cb4a67343a09adb98b1f9e9dcf5 in term 2.
I20260709 13:15:20.792512  9856 leader_election.cc:304] T dba81173e16b42ab92d7da17ba0518a7 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08996682b9d042fc870e58e3db01e68e, 68ba1cb4a67343a09adb98b1f9e9dcf5; no voters: 
I20260709 13:15:20.799528 10053 consensus_queue.cc:1048] T 986acd03cce8455aad79beacff7aa154 P 08996682b9d042fc870e58e3db01e68e [LEADER]: Connected to new peer: Peer: permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:20.799918  9966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701d79f9f0674270ab399ece95bc5c03" candidate_uuid: "63a62dec28ea4a44930d9c44561f273b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08996682b9d042fc870e58e3db01e68e" is_pre_election: true
I20260709 13:15:20.800154 10094 raft_consensus.cc:2804] T dba81173e16b42ab92d7da17ba0518a7 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:20.800616 10094 raft_consensus.cc:697] T dba81173e16b42ab92d7da17ba0518a7 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 2 LEADER]: Becoming Leader. State: Replica: 68ba1cb4a67343a09adb98b1f9e9dcf5, State: Running, Role: LEADER
I20260709 13:15:20.800772  9966 raft_consensus.cc:2393] T 701d79f9f0674270ab399ece95bc5c03 P 08996682b9d042fc870e58e3db01e68e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 63a62dec28ea4a44930d9c44561f273b in current term 1: Already voted for candidate 68ba1cb4a67343a09adb98b1f9e9dcf5 in this term.
I20260709 13:15:20.801508 10094 consensus_queue.cc:237] T dba81173e16b42ab92d7da17ba0518a7 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.802428  9781 leader_election.cc:304] T 701d79f9f0674270ab399ece95bc5c03 P 63a62dec28ea4a44930d9c44561f273b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 63a62dec28ea4a44930d9c44561f273b; no voters: 08996682b9d042fc870e58e3db01e68e, 68ba1cb4a67343a09adb98b1f9e9dcf5
I20260709 13:15:20.804471  9815 raft_consensus.cc:3060] T 701d79f9f0674270ab399ece95bc5c03 P 63a62dec28ea4a44930d9c44561f273b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:20.813547 10080 raft_consensus.cc:2749] T 701d79f9f0674270ab399ece95bc5c03 P 63a62dec28ea4a44930d9c44561f273b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:20.816751 10094 consensus_queue.cc:1048] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:20.823643  9813 raft_consensus.cc:2468] T dba81173e16b42ab92d7da17ba0518a7 P 63a62dec28ea4a44930d9c44561f273b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68ba1cb4a67343a09adb98b1f9e9dcf5 in term 2.
I20260709 13:15:20.828306 10094 raft_consensus.cc:993] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5: : Instructing follower 63a62dec28ea4a44930d9c44561f273b to start an election
I20260709 13:15:20.828838 10094 raft_consensus.cc:1081] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 LEADER]: Signalling peer 63a62dec28ea4a44930d9c44561f273b to start an election
I20260709 13:15:20.832661 10053 raft_consensus.cc:993] T 986acd03cce8455aad79beacff7aa154 P 08996682b9d042fc870e58e3db01e68e: : Instructing follower 68ba1cb4a67343a09adb98b1f9e9dcf5 to start an election
I20260709 13:15:20.837903 10053 raft_consensus.cc:1081] T 986acd03cce8455aad79beacff7aa154 P 08996682b9d042fc870e58e3db01e68e [term 1 LEADER]: Signalling peer 68ba1cb4a67343a09adb98b1f9e9dcf5 to start an election
I20260709 13:15:20.839882  9890 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "986acd03cce8455aad79beacff7aa154"
dest_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5"
 from {username='slave'} at 127.0.0.1:55684
I20260709 13:15:20.840515  9890 raft_consensus.cc:493] T 986acd03cce8455aad79beacff7aa154 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:20.840919  9890 raft_consensus.cc:3060] T 986acd03cce8455aad79beacff7aa154 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:20.841590 10018 raft_consensus.cc:993] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5: : Instructing follower 63a62dec28ea4a44930d9c44561f273b to start an election
I20260709 13:15:20.842069 10018 raft_consensus.cc:1081] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 LEADER]: Signalling peer 63a62dec28ea4a44930d9c44561f273b to start an election
I20260709 13:15:20.832592  9711 catalog_manager.cc:5697] T dba81173e16b42ab92d7da17ba0518a7 P 68ba1cb4a67343a09adb98b1f9e9dcf5 reported cstate change: term changed from 1 to 2, leader changed from 08996682b9d042fc870e58e3db01e68e (127.9.116.131) to 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130). New cstate: current_term: 2 leader_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:20.844496  9816 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f27f84491f9c4b85af0ef5524c00f5e8"
dest_uuid: "63a62dec28ea4a44930d9c44561f273b"
 from {username='slave'} at 127.0.0.1:33722
I20260709 13:15:20.845057  9816 raft_consensus.cc:493] T f27f84491f9c4b85af0ef5524c00f5e8 P 63a62dec28ea4a44930d9c44561f273b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:20.833160  9813 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cdcf499cc854451ab2609798dfcff264"
dest_uuid: "63a62dec28ea4a44930d9c44561f273b"
 from {username='slave'} at 127.0.0.1:33722
I20260709 13:15:20.845422  9816 raft_consensus.cc:3060] T f27f84491f9c4b85af0ef5524c00f5e8 P 63a62dec28ea4a44930d9c44561f273b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:20.849552  9813 raft_consensus.cc:493] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:20.850077  9813 raft_consensus.cc:3060] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:20.852130  9816 raft_consensus.cc:515] T f27f84491f9c4b85af0ef5524c00f5e8 P 63a62dec28ea4a44930d9c44561f273b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.852491  9890 raft_consensus.cc:515] T 986acd03cce8455aad79beacff7aa154 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.856141  9815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986acd03cce8455aad79beacff7aa154" candidate_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "63a62dec28ea4a44930d9c44561f273b"
I20260709 13:15:20.855856  9713 catalog_manager.cc:5697] T 40e71d19ec8244748ceb00b2ab546eb9 P 63a62dec28ea4a44930d9c44561f273b reported cstate change: term changed from 1 to 2, leader changed from 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130) to 63a62dec28ea4a44930d9c44561f273b (127.9.116.129). New cstate: current_term: 2 leader_uuid: "63a62dec28ea4a44930d9c44561f273b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:20.858048  9813 raft_consensus.cc:515] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.856720  9815 raft_consensus.cc:3060] T 986acd03cce8455aad79beacff7aa154 P 63a62dec28ea4a44930d9c44561f273b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:20.857287  9889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27f84491f9c4b85af0ef5524c00f5e8" candidate_uuid: "63a62dec28ea4a44930d9c44561f273b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5"
I20260709 13:15:20.860049  9889 raft_consensus.cc:3055] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:20.860394  9889 raft_consensus.cc:740] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 68ba1cb4a67343a09adb98b1f9e9dcf5, State: Running, Role: LEADER
I20260709 13:15:20.861192  9889 consensus_queue.cc:260] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.861970  9966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986acd03cce8455aad79beacff7aa154" candidate_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "08996682b9d042fc870e58e3db01e68e"
I20260709 13:15:20.862258  9889 raft_consensus.cc:3060] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:20.862429  9890 leader_election.cc:290] T 986acd03cce8455aad79beacff7aa154 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [CANDIDATE]: Term 2 election: Requested vote from peers 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813), 08996682b9d042fc870e58e3db01e68e (127.9.116.131:40693)
I20260709 13:15:20.862531  9966 raft_consensus.cc:3055] T 986acd03cce8455aad79beacff7aa154 P 08996682b9d042fc870e58e3db01e68e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:20.862949  9966 raft_consensus.cc:740] T 986acd03cce8455aad79beacff7aa154 P 08996682b9d042fc870e58e3db01e68e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 08996682b9d042fc870e58e3db01e68e, State: Running, Role: LEADER
I20260709 13:15:20.863737  9966 consensus_queue.cc:260] T 986acd03cce8455aad79beacff7aa154 P 08996682b9d042fc870e58e3db01e68e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.864956  9966 raft_consensus.cc:3060] T 986acd03cce8455aad79beacff7aa154 P 08996682b9d042fc870e58e3db01e68e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:20.866714  9964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27f84491f9c4b85af0ef5524c00f5e8" candidate_uuid: "63a62dec28ea4a44930d9c44561f273b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "08996682b9d042fc870e58e3db01e68e"
I20260709 13:15:20.867091 10065 consensus_queue.cc:1048] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:20.867664  9816 leader_election.cc:290] T f27f84491f9c4b85af0ef5524c00f5e8 P 63a62dec28ea4a44930d9c44561f273b [CANDIDATE]: Term 2 election: Requested vote from peers 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130:44493), 08996682b9d042fc870e58e3db01e68e (127.9.116.131:40693)
I20260709 13:15:20.867344  9964 raft_consensus.cc:3060] T f27f84491f9c4b85af0ef5524c00f5e8 P 08996682b9d042fc870e58e3db01e68e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:20.870496  9815 raft_consensus.cc:2468] T 986acd03cce8455aad79beacff7aa154 P 63a62dec28ea4a44930d9c44561f273b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68ba1cb4a67343a09adb98b1f9e9dcf5 in term 2.
I20260709 13:15:20.874595  9889 raft_consensus.cc:2468] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63a62dec28ea4a44930d9c44561f273b in term 2.
I20260709 13:15:20.874977  9964 raft_consensus.cc:2468] T f27f84491f9c4b85af0ef5524c00f5e8 P 08996682b9d042fc870e58e3db01e68e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63a62dec28ea4a44930d9c44561f273b in term 2.
I20260709 13:15:20.875756  9780 leader_election.cc:304] T f27f84491f9c4b85af0ef5524c00f5e8 P 63a62dec28ea4a44930d9c44561f273b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63a62dec28ea4a44930d9c44561f273b, 68ba1cb4a67343a09adb98b1f9e9dcf5; no voters: 
I20260709 13:15:20.877235 10080 raft_consensus.cc:2804] T f27f84491f9c4b85af0ef5524c00f5e8 P 63a62dec28ea4a44930d9c44561f273b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:20.877694 10080 raft_consensus.cc:697] T f27f84491f9c4b85af0ef5524c00f5e8 P 63a62dec28ea4a44930d9c44561f273b [term 2 LEADER]: Becoming Leader. State: Replica: 63a62dec28ea4a44930d9c44561f273b, State: Running, Role: LEADER
I20260709 13:15:20.877996  9856 leader_election.cc:304] T 986acd03cce8455aad79beacff7aa154 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63a62dec28ea4a44930d9c44561f273b, 68ba1cb4a67343a09adb98b1f9e9dcf5; no voters: 
I20260709 13:15:20.878427 10080 consensus_queue.cc:237] T f27f84491f9c4b85af0ef5524c00f5e8 P 63a62dec28ea4a44930d9c44561f273b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.879125 10065 raft_consensus.cc:2804] T 986acd03cce8455aad79beacff7aa154 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:20.879595 10065 raft_consensus.cc:697] T 986acd03cce8455aad79beacff7aa154 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 2 LEADER]: Becoming Leader. State: Replica: 68ba1cb4a67343a09adb98b1f9e9dcf5, State: Running, Role: LEADER
I20260709 13:15:20.880389 10065 consensus_queue.cc:237] T 986acd03cce8455aad79beacff7aa154 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.890928  9966 raft_consensus.cc:2468] T 986acd03cce8455aad79beacff7aa154 P 08996682b9d042fc870e58e3db01e68e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68ba1cb4a67343a09adb98b1f9e9dcf5 in term 2.
I20260709 13:15:20.895474  9711 catalog_manager.cc:5697] T 986acd03cce8455aad79beacff7aa154 P 68ba1cb4a67343a09adb98b1f9e9dcf5 reported cstate change: term changed from 1 to 2, leader changed from 08996682b9d042fc870e58e3db01e68e (127.9.116.131) to 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130). New cstate: current_term: 2 leader_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:20.890847  9889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdcf499cc854451ab2609798dfcff264" candidate_uuid: "63a62dec28ea4a44930d9c44561f273b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5"
I20260709 13:15:20.898429  9889 raft_consensus.cc:3055] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:20.898805  9889 raft_consensus.cc:740] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 68ba1cb4a67343a09adb98b1f9e9dcf5, State: Running, Role: LEADER
I20260709 13:15:20.899709  9889 consensus_queue.cc:260] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.900794  9889 raft_consensus.cc:3060] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:20.902100  9813 leader_election.cc:290] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b [CANDIDATE]: Term 2 election: Requested vote from peers 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130:44493), 08996682b9d042fc870e58e3db01e68e (127.9.116.131:40693)
I20260709 13:15:20.906075  9965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdcf499cc854451ab2609798dfcff264" candidate_uuid: "63a62dec28ea4a44930d9c44561f273b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "08996682b9d042fc870e58e3db01e68e"
I20260709 13:15:20.906721  9965 raft_consensus.cc:3060] T cdcf499cc854451ab2609798dfcff264 P 08996682b9d042fc870e58e3db01e68e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:20.907595  9889 raft_consensus.cc:2468] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63a62dec28ea4a44930d9c44561f273b in term 2.
I20260709 13:15:20.908941  9780 leader_election.cc:304] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63a62dec28ea4a44930d9c44561f273b, 68ba1cb4a67343a09adb98b1f9e9dcf5; no voters: 
I20260709 13:15:20.909830 10080 raft_consensus.cc:2804] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:20.910333 10080 raft_consensus.cc:697] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b [term 2 LEADER]: Becoming Leader. State: Replica: 63a62dec28ea4a44930d9c44561f273b, State: Running, Role: LEADER
I20260709 13:15:20.911224 10080 consensus_queue.cc:237] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:20.914353  9965 raft_consensus.cc:2468] T cdcf499cc854451ab2609798dfcff264 P 08996682b9d042fc870e58e3db01e68e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63a62dec28ea4a44930d9c44561f273b in term 2.
I20260709 13:15:20.917084  9889 raft_consensus.cc:1275] T e1e26033fa4044e89207d8960181bfad P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 2 FOLLOWER]: Refusing update from remote peer 63a62dec28ea4a44930d9c44561f273b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:20.918541 10054 consensus_queue.cc:1048] T e1e26033fa4044e89207d8960181bfad P 63a62dec28ea4a44930d9c44561f273b [LEADER]: Connected to new peer: Peer: permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:20.919437  9712 catalog_manager.cc:5697] T f27f84491f9c4b85af0ef5524c00f5e8 P 63a62dec28ea4a44930d9c44561f273b reported cstate change: term changed from 1 to 2, leader changed from 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130) to 63a62dec28ea4a44930d9c44561f273b (127.9.116.129). New cstate: current_term: 2 leader_uuid: "63a62dec28ea4a44930d9c44561f273b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:20.929955  9965 raft_consensus.cc:1275] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e [term 2 FOLLOWER]: Refusing update from remote peer 63a62dec28ea4a44930d9c44561f273b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:20.931277 10080 consensus_queue.cc:1048] T e1e26033fa4044e89207d8960181bfad P 63a62dec28ea4a44930d9c44561f273b [LEADER]: Connected to new peer: Peer: permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:20.944054  9711 catalog_manager.cc:5697] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b reported cstate change: term changed from 1 to 2, leader changed from 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130) to 63a62dec28ea4a44930d9c44561f273b (127.9.116.129). New cstate: current_term: 2 leader_uuid: "63a62dec28ea4a44930d9c44561f273b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:21.054225  9889 raft_consensus.cc:1275] T d94510fe6d394c62a543502e8ace50eb P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 2 FOLLOWER]: Refusing update from remote peer 08996682b9d042fc870e58e3db01e68e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:21.055747 10053 consensus_queue.cc:1048] T d94510fe6d394c62a543502e8ace50eb P 08996682b9d042fc870e58e3db01e68e [LEADER]: Connected to new peer: Peer: permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:21.066633  9813 raft_consensus.cc:1275] T d94510fe6d394c62a543502e8ace50eb P 63a62dec28ea4a44930d9c44561f273b [term 2 FOLLOWER]: Refusing update from remote peer 08996682b9d042fc870e58e3db01e68e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:21.068446 10017 consensus_queue.cc:1048] T d94510fe6d394c62a543502e8ace50eb P 08996682b9d042fc870e58e3db01e68e [LEADER]: Connected to new peer: Peer: permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:21.101008  9813 raft_consensus.cc:3060] T 15d491b33a7c47dab512ed81614b9b97 P 63a62dec28ea4a44930d9c44561f273b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:21.105820  9813 raft_consensus.cc:1275] T 15d491b33a7c47dab512ed81614b9b97 P 63a62dec28ea4a44930d9c44561f273b [term 2 FOLLOWER]: Refusing update from remote peer 68ba1cb4a67343a09adb98b1f9e9dcf5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:21.107205 10066 consensus_queue.cc:1048] T 15d491b33a7c47dab512ed81614b9b97 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:21.115787  9965 raft_consensus.cc:1275] T 15d491b33a7c47dab512ed81614b9b97 P 08996682b9d042fc870e58e3db01e68e [term 2 FOLLOWER]: Refusing update from remote peer 68ba1cb4a67343a09adb98b1f9e9dcf5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:21.117195 10018 consensus_queue.cc:1048] T 15d491b33a7c47dab512ed81614b9b97 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:21.188426  9965 raft_consensus.cc:1275] T 40e71d19ec8244748ceb00b2ab546eb9 P 08996682b9d042fc870e58e3db01e68e [term 2 FOLLOWER]: Refusing update from remote peer 63a62dec28ea4a44930d9c44561f273b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:21.189810 10080 consensus_queue.cc:1048] T 40e71d19ec8244748ceb00b2ab546eb9 P 63a62dec28ea4a44930d9c44561f273b [LEADER]: Connected to new peer: Peer: permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:21.197515  9889 raft_consensus.cc:1275] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 2 FOLLOWER]: Refusing update from remote peer 63a62dec28ea4a44930d9c44561f273b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:21.198905 10080 consensus_queue.cc:1048] T 40e71d19ec8244748ceb00b2ab546eb9 P 63a62dec28ea4a44930d9c44561f273b [LEADER]: Connected to new peer: Peer: permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:21.305008  9965 raft_consensus.cc:1275] T f27f84491f9c4b85af0ef5524c00f5e8 P 08996682b9d042fc870e58e3db01e68e [term 2 FOLLOWER]: Refusing update from remote peer 63a62dec28ea4a44930d9c44561f273b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:21.306484 10080 consensus_queue.cc:1048] T f27f84491f9c4b85af0ef5524c00f5e8 P 63a62dec28ea4a44930d9c44561f273b [LEADER]: Connected to new peer: Peer: permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:21.317619  9889 raft_consensus.cc:1275] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 2 FOLLOWER]: Refusing update from remote peer 63a62dec28ea4a44930d9c44561f273b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:21.319988 10079 consensus_queue.cc:1048] T f27f84491f9c4b85af0ef5524c00f5e8 P 63a62dec28ea4a44930d9c44561f273b [LEADER]: Connected to new peer: Peer: permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:21.365172  9889 raft_consensus.cc:1275] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 2 FOLLOWER]: Refusing update from remote peer 63a62dec28ea4a44930d9c44561f273b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:21.366765 10079 consensus_queue.cc:1048] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b [LEADER]: Connected to new peer: Peer: permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:21.374480  9965 raft_consensus.cc:1275] T cdcf499cc854451ab2609798dfcff264 P 08996682b9d042fc870e58e3db01e68e [term 2 FOLLOWER]: Refusing update from remote peer 63a62dec28ea4a44930d9c44561f273b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:21.376498 10054 consensus_queue.cc:1048] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b [LEADER]: Connected to new peer: Peer: permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:21.394119  9965 raft_consensus.cc:1275] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e [term 2 FOLLOWER]: Refusing update from remote peer 68ba1cb4a67343a09adb98b1f9e9dcf5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:21.395757 10065 consensus_queue.cc:1048] T dba81173e16b42ab92d7da17ba0518a7 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:21.403820  9813 raft_consensus.cc:1275] T dba81173e16b42ab92d7da17ba0518a7 P 63a62dec28ea4a44930d9c44561f273b [term 2 FOLLOWER]: Refusing update from remote peer 68ba1cb4a67343a09adb98b1f9e9dcf5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:21.405843 10018 consensus_queue.cc:1048] T dba81173e16b42ab92d7da17ba0518a7 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:21.421303  9965 raft_consensus.cc:1275] T 986acd03cce8455aad79beacff7aa154 P 08996682b9d042fc870e58e3db01e68e [term 2 FOLLOWER]: Refusing update from remote peer 68ba1cb4a67343a09adb98b1f9e9dcf5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:21.422745 10018 consensus_queue.cc:1048] T 986acd03cce8455aad79beacff7aa154 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:21.432318  9813 raft_consensus.cc:1275] T 986acd03cce8455aad79beacff7aa154 P 63a62dec28ea4a44930d9c44561f273b [term 2 FOLLOWER]: Refusing update from remote peer 68ba1cb4a67343a09adb98b1f9e9dcf5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:21.434033 10066 consensus_queue.cc:1048] T 986acd03cce8455aad79beacff7aa154 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:23.525089  9682 auto_leader_rebalancer-test.cc:316] The leader distribution is 
I20260709 13:15:23.525946  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:23.548604  9813 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "40e71d19ec8244748ceb00b2ab546eb9"
dest_uuid: "63a62dec28ea4a44930d9c44561f273b"
mode: GRACEFUL
new_leader_uuid: "08996682b9d042fc870e58e3db01e68e"
 from {username='slave'} at 127.0.0.1:33756
I20260709 13:15:23.549228  9813 raft_consensus.cc:606] T 40e71d19ec8244748ceb00b2ab546eb9 P 63a62dec28ea4a44930d9c44561f273b [term 2 LEADER]: Received request to transfer leadership to TS 08996682b9d042fc870e58e3db01e68e
I20260709 13:15:23.560487  9889 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "701d79f9f0674270ab399ece95bc5c03"
dest_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5"
mode: GRACEFUL
new_leader_uuid: "08996682b9d042fc870e58e3db01e68e"
 from {username='slave'} at 127.0.0.1:55706
I20260709 13:15:23.560992  9889 raft_consensus.cc:606] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 LEADER]: Received request to transfer leadership to TS 08996682b9d042fc870e58e3db01e68e
I20260709 13:15:23.562119  9682 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:15:23.562549  9682 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:23.607035 10065 raft_consensus.cc:993] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5: : Instructing follower 08996682b9d042fc870e58e3db01e68e to start an election
I20260709 13:15:23.607546 10107 raft_consensus.cc:1081] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 LEADER]: Signalling peer 08996682b9d042fc870e58e3db01e68e to start an election
I20260709 13:15:23.609351  9965 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "701d79f9f0674270ab399ece95bc5c03"
dest_uuid: "08996682b9d042fc870e58e3db01e68e"
 from {username='slave'} at 127.0.0.1:53368
I20260709 13:15:23.609907  9965 raft_consensus.cc:493] T 701d79f9f0674270ab399ece95bc5c03 P 08996682b9d042fc870e58e3db01e68e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:23.610272  9965 raft_consensus.cc:3060] T 701d79f9f0674270ab399ece95bc5c03 P 08996682b9d042fc870e58e3db01e68e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:23.615103  9965 raft_consensus.cc:515] T 701d79f9f0674270ab399ece95bc5c03 P 08996682b9d042fc870e58e3db01e68e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:23.617075  9965 leader_election.cc:290] T 701d79f9f0674270ab399ece95bc5c03 P 08996682b9d042fc870e58e3db01e68e [CANDIDATE]: Term 2 election: Requested vote from peers 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813), 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130:44493)
I20260709 13:15:23.618038  9813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701d79f9f0674270ab399ece95bc5c03" candidate_uuid: "08996682b9d042fc870e58e3db01e68e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "63a62dec28ea4a44930d9c44561f273b"
I20260709 13:15:23.618225  9889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701d79f9f0674270ab399ece95bc5c03" candidate_uuid: "08996682b9d042fc870e58e3db01e68e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5"
I20260709 13:15:23.618588  9813 raft_consensus.cc:3060] T 701d79f9f0674270ab399ece95bc5c03 P 63a62dec28ea4a44930d9c44561f273b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:23.618749  9889 raft_consensus.cc:3055] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:23.619200  9889 raft_consensus.cc:740] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 68ba1cb4a67343a09adb98b1f9e9dcf5, State: Running, Role: LEADER
I20260709 13:15:23.619834  9889 consensus_queue.cc:260] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:23.620725  9889 raft_consensus.cc:3060] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:23.623553  9813 raft_consensus.cc:2468] T 701d79f9f0674270ab399ece95bc5c03 P 63a62dec28ea4a44930d9c44561f273b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08996682b9d042fc870e58e3db01e68e in term 2.
I20260709 13:15:23.624752  9930 leader_election.cc:304] T 701d79f9f0674270ab399ece95bc5c03 P 08996682b9d042fc870e58e3db01e68e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08996682b9d042fc870e58e3db01e68e, 63a62dec28ea4a44930d9c44561f273b; no voters: 
I20260709 13:15:23.625358  9889 raft_consensus.cc:2468] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08996682b9d042fc870e58e3db01e68e in term 2.
I20260709 13:15:23.625524 10101 raft_consensus.cc:2804] T 701d79f9f0674270ab399ece95bc5c03 P 08996682b9d042fc870e58e3db01e68e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:23.626967 10101 raft_consensus.cc:697] T 701d79f9f0674270ab399ece95bc5c03 P 08996682b9d042fc870e58e3db01e68e [term 2 LEADER]: Becoming Leader. State: Replica: 08996682b9d042fc870e58e3db01e68e, State: Running, Role: LEADER
I20260709 13:15:23.627717 10101 consensus_queue.cc:237] T 701d79f9f0674270ab399ece95bc5c03 P 08996682b9d042fc870e58e3db01e68e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:23.636636  9712 catalog_manager.cc:5697] T 701d79f9f0674270ab399ece95bc5c03 P 08996682b9d042fc870e58e3db01e68e reported cstate change: term changed from 1 to 2, leader changed from 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130) to 08996682b9d042fc870e58e3db01e68e (127.9.116.131). New cstate: current_term: 2 leader_uuid: "08996682b9d042fc870e58e3db01e68e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } health_report { overall_health: HEALTHY } } }
W20260709 13:15:23.966274  9838 debug-util.cc:398] Leaking SignalData structure 0x7b08000e5060 after lost signal to thread 9685
W20260709 13:15:23.967265  9838 debug-util.cc:398] Leaking SignalData structure 0x7b08000e5740 after lost signal to thread 9748
W20260709 13:15:23.969085  9838 debug-util.cc:398] Leaking SignalData structure 0x7b08001b2020 after lost signal to thread 9990
I20260709 13:15:24.027645  9813 raft_consensus.cc:1275] T 701d79f9f0674270ab399ece95bc5c03 P 63a62dec28ea4a44930d9c44561f273b [term 2 FOLLOWER]: Refusing update from remote peer 08996682b9d042fc870e58e3db01e68e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:24.029083 10101 consensus_queue.cc:1048] T 701d79f9f0674270ab399ece95bc5c03 P 08996682b9d042fc870e58e3db01e68e [LEADER]: Connected to new peer: Peer: permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:24.036834  9889 raft_consensus.cc:1275] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 2 FOLLOWER]: Refusing update from remote peer 08996682b9d042fc870e58e3db01e68e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:24.039407 10108 consensus_queue.cc:1048] T 701d79f9f0674270ab399ece95bc5c03 P 08996682b9d042fc870e58e3db01e68e [LEADER]: Connected to new peer: Peer: permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:24.042297 10102 raft_consensus.cc:993] T 40e71d19ec8244748ceb00b2ab546eb9 P 63a62dec28ea4a44930d9c44561f273b: : Instructing follower 08996682b9d042fc870e58e3db01e68e to start an election
I20260709 13:15:24.042976 10102 raft_consensus.cc:1081] T 40e71d19ec8244748ceb00b2ab546eb9 P 63a62dec28ea4a44930d9c44561f273b [term 2 LEADER]: Signalling peer 08996682b9d042fc870e58e3db01e68e to start an election
I20260709 13:15:24.044999  9965 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "40e71d19ec8244748ceb00b2ab546eb9"
dest_uuid: "08996682b9d042fc870e58e3db01e68e"
 from {username='slave'} at 127.0.0.1:53378
I20260709 13:15:24.045567  9965 raft_consensus.cc:493] T 40e71d19ec8244748ceb00b2ab546eb9 P 08996682b9d042fc870e58e3db01e68e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:24.045938  9965 raft_consensus.cc:3060] T 40e71d19ec8244748ceb00b2ab546eb9 P 08996682b9d042fc870e58e3db01e68e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:24.061810  9965 raft_consensus.cc:515] T 40e71d19ec8244748ceb00b2ab546eb9 P 08996682b9d042fc870e58e3db01e68e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:24.083623  9815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e71d19ec8244748ceb00b2ab546eb9" candidate_uuid: "08996682b9d042fc870e58e3db01e68e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "63a62dec28ea4a44930d9c44561f273b"
I20260709 13:15:24.084321  9815 raft_consensus.cc:3055] T 40e71d19ec8244748ceb00b2ab546eb9 P 63a62dec28ea4a44930d9c44561f273b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:24.084679  9815 raft_consensus.cc:740] T 40e71d19ec8244748ceb00b2ab546eb9 P 63a62dec28ea4a44930d9c44561f273b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 63a62dec28ea4a44930d9c44561f273b, State: Running, Role: LEADER
I20260709 13:15:24.085064  9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e71d19ec8244748ceb00b2ab546eb9" candidate_uuid: "08996682b9d042fc870e58e3db01e68e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5"
I20260709 13:15:24.085652  9890 raft_consensus.cc:3060] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:24.085567  9815 consensus_queue.cc:260] T 40e71d19ec8244748ceb00b2ab546eb9 P 63a62dec28ea4a44930d9c44561f273b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:24.086911  9815 raft_consensus.cc:3060] T 40e71d19ec8244748ceb00b2ab546eb9 P 63a62dec28ea4a44930d9c44561f273b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:24.087291  9965 leader_election.cc:290] T 40e71d19ec8244748ceb00b2ab546eb9 P 08996682b9d042fc870e58e3db01e68e [CANDIDATE]: Term 3 election: Requested vote from peers 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813), 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130:44493)
I20260709 13:15:24.093562  9890 raft_consensus.cc:2468] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08996682b9d042fc870e58e3db01e68e in term 3.
I20260709 13:15:24.094754  9929 leader_election.cc:304] T 40e71d19ec8244748ceb00b2ab546eb9 P 08996682b9d042fc870e58e3db01e68e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08996682b9d042fc870e58e3db01e68e, 68ba1cb4a67343a09adb98b1f9e9dcf5; no voters: 
I20260709 13:15:24.095698 10108 raft_consensus.cc:2804] T 40e71d19ec8244748ceb00b2ab546eb9 P 08996682b9d042fc870e58e3db01e68e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:24.096201 10108 raft_consensus.cc:697] T 40e71d19ec8244748ceb00b2ab546eb9 P 08996682b9d042fc870e58e3db01e68e [term 3 LEADER]: Becoming Leader. State: Replica: 08996682b9d042fc870e58e3db01e68e, State: Running, Role: LEADER
I20260709 13:15:24.097023 10108 consensus_queue.cc:237] T 40e71d19ec8244748ceb00b2ab546eb9 P 08996682b9d042fc870e58e3db01e68e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:24.097999  9815 raft_consensus.cc:2468] T 40e71d19ec8244748ceb00b2ab546eb9 P 63a62dec28ea4a44930d9c44561f273b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08996682b9d042fc870e58e3db01e68e in term 3.
I20260709 13:15:24.107348  9710 catalog_manager.cc:5697] T 40e71d19ec8244748ceb00b2ab546eb9 P 08996682b9d042fc870e58e3db01e68e reported cstate change: term changed from 2 to 3, leader changed from 63a62dec28ea4a44930d9c44561f273b (127.9.116.129) to 08996682b9d042fc870e58e3db01e68e (127.9.116.131). New cstate: current_term: 3 leader_uuid: "08996682b9d042fc870e58e3db01e68e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:24.563264  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:24.573266  9815 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f27f84491f9c4b85af0ef5524c00f5e8"
dest_uuid: "63a62dec28ea4a44930d9c44561f273b"
mode: GRACEFUL
new_leader_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5"
 from {username='slave'} at 127.0.0.1:33754
I20260709 13:15:24.573957  9815 raft_consensus.cc:606] T f27f84491f9c4b85af0ef5524c00f5e8 P 63a62dec28ea4a44930d9c44561f273b [term 2 LEADER]: Received request to transfer leadership to TS 68ba1cb4a67343a09adb98b1f9e9dcf5
I20260709 13:15:24.579210  9965 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "40e71d19ec8244748ceb00b2ab546eb9"
dest_uuid: "08996682b9d042fc870e58e3db01e68e"
mode: GRACEFUL
new_leader_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5"
 from {username='slave'} at 127.0.0.1:53386
I20260709 13:15:24.579861  9965 raft_consensus.cc:606] T 40e71d19ec8244748ceb00b2ab546eb9 P 08996682b9d042fc870e58e3db01e68e [term 3 LEADER]: Received request to transfer leadership to TS 68ba1cb4a67343a09adb98b1f9e9dcf5
I20260709 13:15:24.580643  9815 raft_consensus.cc:1275] T 40e71d19ec8244748ceb00b2ab546eb9 P 63a62dec28ea4a44930d9c44561f273b [term 3 FOLLOWER]: Refusing update from remote peer 08996682b9d042fc870e58e3db01e68e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:24.582432 10108 consensus_queue.cc:1048] T 40e71d19ec8244748ceb00b2ab546eb9 P 08996682b9d042fc870e58e3db01e68e [LEADER]: Connected to new peer: Peer: permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:24.586560  9813 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e1e26033fa4044e89207d8960181bfad"
dest_uuid: "63a62dec28ea4a44930d9c44561f273b"
mode: GRACEFUL
new_leader_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5"
 from {username='slave'} at 127.0.0.1:33754
I20260709 13:15:24.587330  9813 raft_consensus.cc:606] T e1e26033fa4044e89207d8960181bfad P 63a62dec28ea4a44930d9c44561f273b [term 2 LEADER]: Received request to transfer leadership to TS 68ba1cb4a67343a09adb98b1f9e9dcf5
I20260709 13:15:24.600680  9813 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cdcf499cc854451ab2609798dfcff264"
dest_uuid: "63a62dec28ea4a44930d9c44561f273b"
mode: GRACEFUL
new_leader_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5"
 from {username='slave'} at 127.0.0.1:33754
I20260709 13:15:24.601336  9813 raft_consensus.cc:606] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b [term 2 LEADER]: Received request to transfer leadership to TS 68ba1cb4a67343a09adb98b1f9e9dcf5
I20260709 13:15:24.613950  9890 raft_consensus.cc:1275] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 3 FOLLOWER]: Refusing update from remote peer 08996682b9d042fc870e58e3db01e68e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:24.615680 10108 consensus_queue.cc:1048] T 40e71d19ec8244748ceb00b2ab546eb9 P 08996682b9d042fc870e58e3db01e68e [LEADER]: Connected to new peer: Peer: permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:24.624941  9965 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "701d79f9f0674270ab399ece95bc5c03"
dest_uuid: "08996682b9d042fc870e58e3db01e68e"
mode: GRACEFUL
new_leader_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5"
 from {username='slave'} at 127.0.0.1:53386
I20260709 13:15:24.625563  9965 raft_consensus.cc:606] T 701d79f9f0674270ab399ece95bc5c03 P 08996682b9d042fc870e58e3db01e68e [term 2 LEADER]: Received request to transfer leadership to TS 68ba1cb4a67343a09adb98b1f9e9dcf5
I20260709 13:15:24.634407 10101 raft_consensus.cc:993] T 40e71d19ec8244748ceb00b2ab546eb9 P 08996682b9d042fc870e58e3db01e68e: : Instructing follower 68ba1cb4a67343a09adb98b1f9e9dcf5 to start an election
I20260709 13:15:24.634995 10108 raft_consensus.cc:1081] T 40e71d19ec8244748ceb00b2ab546eb9 P 08996682b9d042fc870e58e3db01e68e [term 3 LEADER]: Signalling peer 68ba1cb4a67343a09adb98b1f9e9dcf5 to start an election
I20260709 13:15:24.636790  9890 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "40e71d19ec8244748ceb00b2ab546eb9"
dest_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5"
 from {username='slave'} at 127.0.0.1:55684
I20260709 13:15:24.637369  9890 raft_consensus.cc:493] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:24.637753  9890 raft_consensus.cc:3060] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:24.643446  9890 raft_consensus.cc:515] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:24.645476  9890 leader_election.cc:290] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [CANDIDATE]: Term 4 election: Requested vote from peers 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813), 08996682b9d042fc870e58e3db01e68e (127.9.116.131:40693)
I20260709 13:15:24.647207  9813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e71d19ec8244748ceb00b2ab546eb9" candidate_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "63a62dec28ea4a44930d9c44561f273b"
I20260709 13:15:24.647492  9965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e71d19ec8244748ceb00b2ab546eb9" candidate_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "08996682b9d042fc870e58e3db01e68e"
I20260709 13:15:24.647848  9813 raft_consensus.cc:3060] T 40e71d19ec8244748ceb00b2ab546eb9 P 63a62dec28ea4a44930d9c44561f273b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:24.648133  9965 raft_consensus.cc:3055] T 40e71d19ec8244748ceb00b2ab546eb9 P 08996682b9d042fc870e58e3db01e68e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:24.648514  9965 raft_consensus.cc:740] T 40e71d19ec8244748ceb00b2ab546eb9 P 08996682b9d042fc870e58e3db01e68e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 08996682b9d042fc870e58e3db01e68e, State: Running, Role: LEADER
I20260709 13:15:24.649375  9965 consensus_queue.cc:260] T 40e71d19ec8244748ceb00b2ab546eb9 P 08996682b9d042fc870e58e3db01e68e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:24.650764  9965 raft_consensus.cc:3060] T 40e71d19ec8244748ceb00b2ab546eb9 P 08996682b9d042fc870e58e3db01e68e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:24.653241  9813 raft_consensus.cc:2468] T 40e71d19ec8244748ceb00b2ab546eb9 P 63a62dec28ea4a44930d9c44561f273b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68ba1cb4a67343a09adb98b1f9e9dcf5 in term 4.
I20260709 13:15:24.654496  9856 leader_election.cc:304] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63a62dec28ea4a44930d9c44561f273b, 68ba1cb4a67343a09adb98b1f9e9dcf5; no voters: 
I20260709 13:15:24.655345 10065 raft_consensus.cc:2804] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:24.656445 10065 raft_consensus.cc:697] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 4 LEADER]: Becoming Leader. State: Replica: 68ba1cb4a67343a09adb98b1f9e9dcf5, State: Running, Role: LEADER
I20260709 13:15:24.657299 10065 consensus_queue.cc:237] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:24.658746  9965 raft_consensus.cc:2468] T 40e71d19ec8244748ceb00b2ab546eb9 P 08996682b9d042fc870e58e3db01e68e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68ba1cb4a67343a09adb98b1f9e9dcf5 in term 4.
I20260709 13:15:24.666536  9710 catalog_manager.cc:5697] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5 reported cstate change: term changed from 3 to 4, leader changed from 08996682b9d042fc870e58e3db01e68e (127.9.116.131) to 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130). New cstate: current_term: 4 leader_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:24.668684 10108 raft_consensus.cc:993] T 701d79f9f0674270ab399ece95bc5c03 P 08996682b9d042fc870e58e3db01e68e: : Instructing follower 68ba1cb4a67343a09adb98b1f9e9dcf5 to start an election
I20260709 13:15:24.669188 10108 raft_consensus.cc:1081] T 701d79f9f0674270ab399ece95bc5c03 P 08996682b9d042fc870e58e3db01e68e [term 2 LEADER]: Signalling peer 68ba1cb4a67343a09adb98b1f9e9dcf5 to start an election
I20260709 13:15:24.671474  9890 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "701d79f9f0674270ab399ece95bc5c03"
dest_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5"
 from {username='slave'} at 127.0.0.1:55684
I20260709 13:15:24.672130  9890 raft_consensus.cc:493] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:24.672560  9890 raft_consensus.cc:3060] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:24.680383  9890 raft_consensus.cc:515] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:24.683881  9815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701d79f9f0674270ab399ece95bc5c03" candidate_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "63a62dec28ea4a44930d9c44561f273b"
I20260709 13:15:24.684576  9815 raft_consensus.cc:3060] T 701d79f9f0674270ab399ece95bc5c03 P 63a62dec28ea4a44930d9c44561f273b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:24.693475 10130 raft_consensus.cc:993] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b: : Instructing follower 68ba1cb4a67343a09adb98b1f9e9dcf5 to start an election
I20260709 13:15:24.694116 10102 raft_consensus.cc:1081] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b [term 2 LEADER]: Signalling peer 68ba1cb4a67343a09adb98b1f9e9dcf5 to start an election
I20260709 13:15:24.694873  9815 raft_consensus.cc:2468] T 701d79f9f0674270ab399ece95bc5c03 P 63a62dec28ea4a44930d9c44561f273b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68ba1cb4a67343a09adb98b1f9e9dcf5 in term 3.
I20260709 13:15:24.696300  9890 leader_election.cc:290] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [CANDIDATE]: Term 3 election: Requested vote from peers 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813), 08996682b9d042fc870e58e3db01e68e (127.9.116.131:40693)
I20260709 13:15:24.697022  9889 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cdcf499cc854451ab2609798dfcff264"
dest_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5"
 from {username='slave'} at 127.0.0.1:55694
I20260709 13:15:24.697486  9856 leader_election.cc:304] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63a62dec28ea4a44930d9c44561f273b, 68ba1cb4a67343a09adb98b1f9e9dcf5; no voters: 
I20260709 13:15:24.697630  9889 raft_consensus.cc:493] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:24.698127  9889 raft_consensus.cc:3060] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:24.698495 10065 raft_consensus.cc:2804] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:24.698539  9965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701d79f9f0674270ab399ece95bc5c03" candidate_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "08996682b9d042fc870e58e3db01e68e"
I20260709 13:15:24.699126 10065 raft_consensus.cc:697] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 3 LEADER]: Becoming Leader. State: Replica: 68ba1cb4a67343a09adb98b1f9e9dcf5, State: Running, Role: LEADER
I20260709 13:15:24.699297  9965 raft_consensus.cc:3055] T 701d79f9f0674270ab399ece95bc5c03 P 08996682b9d042fc870e58e3db01e68e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:24.699704  9965 raft_consensus.cc:740] T 701d79f9f0674270ab399ece95bc5c03 P 08996682b9d042fc870e58e3db01e68e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 08996682b9d042fc870e58e3db01e68e, State: Running, Role: LEADER
I20260709 13:15:24.700037 10065 consensus_queue.cc:237] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:24.700632  9965 consensus_queue.cc:260] T 701d79f9f0674270ab399ece95bc5c03 P 08996682b9d042fc870e58e3db01e68e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:24.702068  9965 raft_consensus.cc:3060] T 701d79f9f0674270ab399ece95bc5c03 P 08996682b9d042fc870e58e3db01e68e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:24.705499  9889 raft_consensus.cc:515] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:24.707826  9889 leader_election.cc:290] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [CANDIDATE]: Term 3 election: Requested vote from peers 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813), 08996682b9d042fc870e58e3db01e68e (127.9.116.131:40693)
I20260709 13:15:24.708019  9815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdcf499cc854451ab2609798dfcff264" candidate_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "63a62dec28ea4a44930d9c44561f273b"
I20260709 13:15:24.708680  9815 raft_consensus.cc:3055] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:24.709024  9815 raft_consensus.cc:740] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 63a62dec28ea4a44930d9c44561f273b, State: Running, Role: LEADER
I20260709 13:15:24.709007  9966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdcf499cc854451ab2609798dfcff264" candidate_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "08996682b9d042fc870e58e3db01e68e"
I20260709 13:15:24.710105  9966 raft_consensus.cc:3060] T cdcf499cc854451ab2609798dfcff264 P 08996682b9d042fc870e58e3db01e68e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:24.710166  9815 consensus_queue.cc:260] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:24.711916  9815 raft_consensus.cc:3060] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:24.714202  9965 raft_consensus.cc:2468] T 701d79f9f0674270ab399ece95bc5c03 P 08996682b9d042fc870e58e3db01e68e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68ba1cb4a67343a09adb98b1f9e9dcf5 in term 3.
I20260709 13:15:24.717523  9966 raft_consensus.cc:2468] T cdcf499cc854451ab2609798dfcff264 P 08996682b9d042fc870e58e3db01e68e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68ba1cb4a67343a09adb98b1f9e9dcf5 in term 3.
I20260709 13:15:24.719012  9856 leader_election.cc:304] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08996682b9d042fc870e58e3db01e68e, 68ba1cb4a67343a09adb98b1f9e9dcf5; no voters: 
I20260709 13:15:24.719228  9815 raft_consensus.cc:2468] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68ba1cb4a67343a09adb98b1f9e9dcf5 in term 3.
I20260709 13:15:24.719954 10128 raft_consensus.cc:2804] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:24.720480 10128 raft_consensus.cc:697] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 3 LEADER]: Becoming Leader. State: Replica: 68ba1cb4a67343a09adb98b1f9e9dcf5, State: Running, Role: LEADER
I20260709 13:15:24.721351 10128 consensus_queue.cc:237] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:24.731391  9710 catalog_manager.cc:5697] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 reported cstate change: term changed from 2 to 3, leader changed from 08996682b9d042fc870e58e3db01e68e (127.9.116.131) to 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130). New cstate: current_term: 3 leader_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:24.732731  9710 catalog_manager.cc:5697] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 reported cstate change: term changed from 2 to 3, leader changed from 63a62dec28ea4a44930d9c44561f273b (127.9.116.129) to 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130). New cstate: current_term: 3 leader_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:24.762869 10102 raft_consensus.cc:993] T f27f84491f9c4b85af0ef5524c00f5e8 P 63a62dec28ea4a44930d9c44561f273b: : Instructing follower 68ba1cb4a67343a09adb98b1f9e9dcf5 to start an election
I20260709 13:15:24.763357 10130 raft_consensus.cc:1081] T f27f84491f9c4b85af0ef5524c00f5e8 P 63a62dec28ea4a44930d9c44561f273b [term 2 LEADER]: Signalling peer 68ba1cb4a67343a09adb98b1f9e9dcf5 to start an election
I20260709 13:15:24.764890  9889 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f27f84491f9c4b85af0ef5524c00f5e8"
dest_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5"
 from {username='slave'} at 127.0.0.1:55694
I20260709 13:15:24.765383  9889 raft_consensus.cc:493] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:24.765760  9889 raft_consensus.cc:3060] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:24.770419  9889 raft_consensus.cc:515] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:24.772045  9889 leader_election.cc:290] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [CANDIDATE]: Term 3 election: Requested vote from peers 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813), 08996682b9d042fc870e58e3db01e68e (127.9.116.131:40693)
I20260709 13:15:24.772994  9815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27f84491f9c4b85af0ef5524c00f5e8" candidate_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "63a62dec28ea4a44930d9c44561f273b"
I20260709 13:15:24.773619  9815 raft_consensus.cc:3055] T f27f84491f9c4b85af0ef5524c00f5e8 P 63a62dec28ea4a44930d9c44561f273b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:24.773586  9966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27f84491f9c4b85af0ef5524c00f5e8" candidate_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "08996682b9d042fc870e58e3db01e68e"
I20260709 13:15:24.774093  9815 raft_consensus.cc:740] T f27f84491f9c4b85af0ef5524c00f5e8 P 63a62dec28ea4a44930d9c44561f273b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 63a62dec28ea4a44930d9c44561f273b, State: Running, Role: LEADER
I20260709 13:15:24.774309  9966 raft_consensus.cc:3060] T f27f84491f9c4b85af0ef5524c00f5e8 P 08996682b9d042fc870e58e3db01e68e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:24.774950  9815 consensus_queue.cc:260] T f27f84491f9c4b85af0ef5524c00f5e8 P 63a62dec28ea4a44930d9c44561f273b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:24.775893  9815 raft_consensus.cc:3060] T f27f84491f9c4b85af0ef5524c00f5e8 P 63a62dec28ea4a44930d9c44561f273b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:24.780746  9815 raft_consensus.cc:2468] T f27f84491f9c4b85af0ef5524c00f5e8 P 63a62dec28ea4a44930d9c44561f273b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68ba1cb4a67343a09adb98b1f9e9dcf5 in term 3.
I20260709 13:15:24.782229  9856 leader_election.cc:304] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63a62dec28ea4a44930d9c44561f273b, 68ba1cb4a67343a09adb98b1f9e9dcf5; no voters: 
I20260709 13:15:24.782817  9966 raft_consensus.cc:2468] T f27f84491f9c4b85af0ef5524c00f5e8 P 08996682b9d042fc870e58e3db01e68e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68ba1cb4a67343a09adb98b1f9e9dcf5 in term 3.
I20260709 13:15:24.783272 10128 raft_consensus.cc:2804] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:24.783840 10128 raft_consensus.cc:697] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 3 LEADER]: Becoming Leader. State: Replica: 68ba1cb4a67343a09adb98b1f9e9dcf5, State: Running, Role: LEADER
I20260709 13:15:24.784765 10128 consensus_queue.cc:237] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:24.793437  9710 catalog_manager.cc:5697] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5 reported cstate change: term changed from 2 to 3, leader changed from 63a62dec28ea4a44930d9c44561f273b (127.9.116.129) to 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130). New cstate: current_term: 3 leader_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:25.082835 10102 raft_consensus.cc:993] T e1e26033fa4044e89207d8960181bfad P 63a62dec28ea4a44930d9c44561f273b: : Instructing follower 68ba1cb4a67343a09adb98b1f9e9dcf5 to start an election
I20260709 13:15:25.083374 10130 raft_consensus.cc:1081] T e1e26033fa4044e89207d8960181bfad P 63a62dec28ea4a44930d9c44561f273b [term 2 LEADER]: Signalling peer 68ba1cb4a67343a09adb98b1f9e9dcf5 to start an election
I20260709 13:15:25.084813  9889 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e1e26033fa4044e89207d8960181bfad"
dest_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5"
 from {username='slave'} at 127.0.0.1:55694
I20260709 13:15:25.085438  9889 raft_consensus.cc:493] T e1e26033fa4044e89207d8960181bfad P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:25.085813  9889 raft_consensus.cc:3060] T e1e26033fa4044e89207d8960181bfad P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:25.090480  9889 raft_consensus.cc:515] T e1e26033fa4044e89207d8960181bfad P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:25.092056  9889 leader_election.cc:290] T e1e26033fa4044e89207d8960181bfad P 68ba1cb4a67343a09adb98b1f9e9dcf5 [CANDIDATE]: Term 3 election: Requested vote from peers 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813), 08996682b9d042fc870e58e3db01e68e (127.9.116.131:40693)
I20260709 13:15:25.093312  9815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e26033fa4044e89207d8960181bfad" candidate_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "63a62dec28ea4a44930d9c44561f273b"
I20260709 13:15:25.093410  9966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e26033fa4044e89207d8960181bfad" candidate_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "08996682b9d042fc870e58e3db01e68e"
I20260709 13:15:25.093822  9815 raft_consensus.cc:3055] T e1e26033fa4044e89207d8960181bfad P 63a62dec28ea4a44930d9c44561f273b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:25.094035  9966 raft_consensus.cc:3060] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:25.094197  9815 raft_consensus.cc:740] T e1e26033fa4044e89207d8960181bfad P 63a62dec28ea4a44930d9c44561f273b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 63a62dec28ea4a44930d9c44561f273b, State: Running, Role: LEADER
I20260709 13:15:25.095095  9815 consensus_queue.cc:260] T e1e26033fa4044e89207d8960181bfad P 63a62dec28ea4a44930d9c44561f273b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:25.096035  9815 raft_consensus.cc:3060] T e1e26033fa4044e89207d8960181bfad P 63a62dec28ea4a44930d9c44561f273b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:25.101336  9815 raft_consensus.cc:2468] T e1e26033fa4044e89207d8960181bfad P 63a62dec28ea4a44930d9c44561f273b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68ba1cb4a67343a09adb98b1f9e9dcf5 in term 3.
I20260709 13:15:25.102490  9856 leader_election.cc:304] T e1e26033fa4044e89207d8960181bfad P 68ba1cb4a67343a09adb98b1f9e9dcf5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63a62dec28ea4a44930d9c44561f273b, 68ba1cb4a67343a09adb98b1f9e9dcf5; no voters: 
I20260709 13:15:25.102005  9966 raft_consensus.cc:2468] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68ba1cb4a67343a09adb98b1f9e9dcf5 in term 3.
I20260709 13:15:25.103492 10128 raft_consensus.cc:2804] T e1e26033fa4044e89207d8960181bfad P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:25.103952 10128 raft_consensus.cc:697] T e1e26033fa4044e89207d8960181bfad P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 3 LEADER]: Becoming Leader. State: Replica: 68ba1cb4a67343a09adb98b1f9e9dcf5, State: Running, Role: LEADER
I20260709 13:15:25.104876 10128 consensus_queue.cc:237] T e1e26033fa4044e89207d8960181bfad P 68ba1cb4a67343a09adb98b1f9e9dcf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:25.107810  9966 raft_consensus.cc:1275] T 40e71d19ec8244748ceb00b2ab546eb9 P 08996682b9d042fc870e58e3db01e68e [term 4 FOLLOWER]: Refusing update from remote peer 68ba1cb4a67343a09adb98b1f9e9dcf5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:15:25.109247 10065 consensus_queue.cc:1048] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:25.113612  9710 catalog_manager.cc:5697] T e1e26033fa4044e89207d8960181bfad P 68ba1cb4a67343a09adb98b1f9e9dcf5 reported cstate change: term changed from 2 to 3, leader changed from 63a62dec28ea4a44930d9c44561f273b (127.9.116.129) to 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130). New cstate: current_term: 3 leader_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:25.116793  9965 raft_consensus.cc:1275] T 701d79f9f0674270ab399ece95bc5c03 P 08996682b9d042fc870e58e3db01e68e [term 3 FOLLOWER]: Refusing update from remote peer 68ba1cb4a67343a09adb98b1f9e9dcf5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:25.118251  9815 raft_consensus.cc:1275] T 40e71d19ec8244748ceb00b2ab546eb9 P 63a62dec28ea4a44930d9c44561f273b [term 4 FOLLOWER]: Refusing update from remote peer 68ba1cb4a67343a09adb98b1f9e9dcf5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:15:25.120875 10128 consensus_queue.cc:1048] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:25.123152 10128 consensus_queue.cc:1048] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:25.143467  9813 raft_consensus.cc:1275] T 701d79f9f0674270ab399ece95bc5c03 P 63a62dec28ea4a44930d9c44561f273b [term 3 FOLLOWER]: Refusing update from remote peer 68ba1cb4a67343a09adb98b1f9e9dcf5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:25.145947 10065 consensus_queue.cc:1048] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:25.191010  9965 raft_consensus.cc:1275] T f27f84491f9c4b85af0ef5524c00f5e8 P 08996682b9d042fc870e58e3db01e68e [term 3 FOLLOWER]: Refusing update from remote peer 68ba1cb4a67343a09adb98b1f9e9dcf5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:25.192485 10128 consensus_queue.cc:1048] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:25.206238  9813 raft_consensus.cc:1275] T f27f84491f9c4b85af0ef5524c00f5e8 P 63a62dec28ea4a44930d9c44561f273b [term 3 FOLLOWER]: Refusing update from remote peer 68ba1cb4a67343a09adb98b1f9e9dcf5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:25.209527 10065 consensus_queue.cc:1048] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:25.270292  9965 raft_consensus.cc:1275] T cdcf499cc854451ab2609798dfcff264 P 08996682b9d042fc870e58e3db01e68e [term 3 FOLLOWER]: Refusing update from remote peer 68ba1cb4a67343a09adb98b1f9e9dcf5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:25.271829 10065 consensus_queue.cc:1048] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:25.283597  9815 raft_consensus.cc:1275] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b [term 3 FOLLOWER]: Refusing update from remote peer 68ba1cb4a67343a09adb98b1f9e9dcf5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:25.285566 10128 consensus_queue.cc:1048] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:25.510556  9965 raft_consensus.cc:1275] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e [term 3 FOLLOWER]: Refusing update from remote peer 68ba1cb4a67343a09adb98b1f9e9dcf5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:25.511935 10065 consensus_queue.cc:1048] T e1e26033fa4044e89207d8960181bfad P 68ba1cb4a67343a09adb98b1f9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:25.522894  9815 raft_consensus.cc:1275] T e1e26033fa4044e89207d8960181bfad P 63a62dec28ea4a44930d9c44561f273b [term 3 FOLLOWER]: Refusing update from remote peer 68ba1cb4a67343a09adb98b1f9e9dcf5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:25.526559 10065 consensus_queue.cc:1048] T e1e26033fa4044e89207d8960181bfad P 68ba1cb4a67343a09adb98b1f9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:27.634007  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:27.641285  9889 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15d491b33a7c47dab512ed81614b9b97"
dest_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5"
mode: GRACEFUL
new_leader_uuid: "63a62dec28ea4a44930d9c44561f273b"
 from {username='slave'} at 127.0.0.1:55706
I20260709 13:15:27.642009  9889 raft_consensus.cc:606] T 15d491b33a7c47dab512ed81614b9b97 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 2 LEADER]: Received request to transfer leadership to TS 63a62dec28ea4a44930d9c44561f273b
I20260709 13:15:27.644611  9889 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "701d79f9f0674270ab399ece95bc5c03"
dest_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5"
mode: GRACEFUL
new_leader_uuid: "63a62dec28ea4a44930d9c44561f273b"
 from {username='slave'} at 127.0.0.1:55706
I20260709 13:15:27.645128  9889 raft_consensus.cc:606] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 3 LEADER]: Received request to transfer leadership to TS 63a62dec28ea4a44930d9c44561f273b
I20260709 13:15:27.647284  9889 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cdcf499cc854451ab2609798dfcff264"
dest_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5"
mode: GRACEFUL
new_leader_uuid: "63a62dec28ea4a44930d9c44561f273b"
 from {username='slave'} at 127.0.0.1:55706
I20260709 13:15:27.647758  9889 raft_consensus.cc:606] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 3 LEADER]: Received request to transfer leadership to TS 63a62dec28ea4a44930d9c44561f273b
I20260709 13:15:27.649842  9889 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dba81173e16b42ab92d7da17ba0518a7"
dest_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5"
mode: GRACEFUL
new_leader_uuid: "08996682b9d042fc870e58e3db01e68e"
 from {username='slave'} at 127.0.0.1:55706
I20260709 13:15:27.650295  9889 raft_consensus.cc:606] T dba81173e16b42ab92d7da17ba0518a7 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 2 LEADER]: Received request to transfer leadership to TS 08996682b9d042fc870e58e3db01e68e
I20260709 13:15:27.652755  9889 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e1e26033fa4044e89207d8960181bfad"
dest_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5"
mode: GRACEFUL
new_leader_uuid: "08996682b9d042fc870e58e3db01e68e"
 from {username='slave'} at 127.0.0.1:55706
I20260709 13:15:27.653198  9889 raft_consensus.cc:606] T e1e26033fa4044e89207d8960181bfad P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 3 LEADER]: Received request to transfer leadership to TS 08996682b9d042fc870e58e3db01e68e
I20260709 13:15:27.654174  9682 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:15:27.654608  9682 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:27.979768 10137 raft_consensus.cc:993] T dba81173e16b42ab92d7da17ba0518a7 P 68ba1cb4a67343a09adb98b1f9e9dcf5: : Instructing follower 08996682b9d042fc870e58e3db01e68e to start an election
I20260709 13:15:27.980297 10140 raft_consensus.cc:1081] T dba81173e16b42ab92d7da17ba0518a7 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 2 LEADER]: Signalling peer 08996682b9d042fc870e58e3db01e68e to start an election
I20260709 13:15:27.981745  9965 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dba81173e16b42ab92d7da17ba0518a7"
dest_uuid: "08996682b9d042fc870e58e3db01e68e"
 from {username='slave'} at 127.0.0.1:53368
I20260709 13:15:27.982163  9965 raft_consensus.cc:493] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:27.982443  9965 raft_consensus.cc:3060] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:27.987771  9965 raft_consensus.cc:515] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:27.991298  9965 leader_election.cc:290] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e [CANDIDATE]: Term 3 election: Requested vote from peers 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813), 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130:44493)
I20260709 13:15:27.992784  9815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba81173e16b42ab92d7da17ba0518a7" candidate_uuid: "08996682b9d042fc870e58e3db01e68e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "63a62dec28ea4a44930d9c44561f273b"
I20260709 13:15:27.993510  9815 raft_consensus.cc:3060] T dba81173e16b42ab92d7da17ba0518a7 P 63a62dec28ea4a44930d9c44561f273b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:27.994179  9889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba81173e16b42ab92d7da17ba0518a7" candidate_uuid: "08996682b9d042fc870e58e3db01e68e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5"
I20260709 13:15:27.994638  9889 raft_consensus.cc:3055] T dba81173e16b42ab92d7da17ba0518a7 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:27.994907  9889 raft_consensus.cc:740] T dba81173e16b42ab92d7da17ba0518a7 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 68ba1cb4a67343a09adb98b1f9e9dcf5, State: Running, Role: LEADER
I20260709 13:15:27.995739  9889 consensus_queue.cc:260] T dba81173e16b42ab92d7da17ba0518a7 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:27.997448  9889 raft_consensus.cc:3060] T dba81173e16b42ab92d7da17ba0518a7 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:28.001575  9815 raft_consensus.cc:2468] T dba81173e16b42ab92d7da17ba0518a7 P 63a62dec28ea4a44930d9c44561f273b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08996682b9d042fc870e58e3db01e68e in term 3.
I20260709 13:15:28.003100  9930 leader_election.cc:304] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08996682b9d042fc870e58e3db01e68e, 63a62dec28ea4a44930d9c44561f273b; no voters: 
I20260709 13:15:28.003717  9889 raft_consensus.cc:2468] T dba81173e16b42ab92d7da17ba0518a7 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08996682b9d042fc870e58e3db01e68e in term 3.
I20260709 13:15:28.004040 10125 raft_consensus.cc:2804] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:28.005486 10125 raft_consensus.cc:697] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e [term 3 LEADER]: Becoming Leader. State: Replica: 08996682b9d042fc870e58e3db01e68e, State: Running, Role: LEADER
I20260709 13:15:28.006784 10125 consensus_queue.cc:237] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:28.009600 10137 raft_consensus.cc:993] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5: : Instructing follower 63a62dec28ea4a44930d9c44561f273b to start an election
I20260709 13:15:28.010167 10140 raft_consensus.cc:1081] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 3 LEADER]: Signalling peer 63a62dec28ea4a44930d9c44561f273b to start an election
I20260709 13:15:28.013365  9815 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cdcf499cc854451ab2609798dfcff264"
dest_uuid: "63a62dec28ea4a44930d9c44561f273b"
 from {username='slave'} at 127.0.0.1:33722
I20260709 13:15:28.013900  9815 raft_consensus.cc:493] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:28.014214  9815 raft_consensus.cc:3060] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:28.015595 10140 raft_consensus.cc:993] T e1e26033fa4044e89207d8960181bfad P 68ba1cb4a67343a09adb98b1f9e9dcf5: : Instructing follower 08996682b9d042fc870e58e3db01e68e to start an election
I20260709 13:15:28.016191 10137 raft_consensus.cc:1081] T e1e26033fa4044e89207d8960181bfad P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 3 LEADER]: Signalling peer 08996682b9d042fc870e58e3db01e68e to start an election
I20260709 13:15:28.017997  9966 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e1e26033fa4044e89207d8960181bfad"
dest_uuid: "08996682b9d042fc870e58e3db01e68e"
 from {username='slave'} at 127.0.0.1:53368
I20260709 13:15:28.018532  9966 raft_consensus.cc:493] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:28.018899  9966 raft_consensus.cc:3060] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:28.020648  9815 raft_consensus.cc:515] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:28.024228  9965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdcf499cc854451ab2609798dfcff264" candidate_uuid: "63a62dec28ea4a44930d9c44561f273b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "08996682b9d042fc870e58e3db01e68e"
I20260709 13:15:28.025892  9965 raft_consensus.cc:3060] T cdcf499cc854451ab2609798dfcff264 P 08996682b9d042fc870e58e3db01e68e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:28.027645  9712 catalog_manager.cc:5697] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e reported cstate change: term changed from 2 to 3, leader changed from 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130) to 08996682b9d042fc870e58e3db01e68e (127.9.116.131). New cstate: current_term: 3 leader_uuid: "08996682b9d042fc870e58e3db01e68e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:28.032524  9965 raft_consensus.cc:2468] T cdcf499cc854451ab2609798dfcff264 P 08996682b9d042fc870e58e3db01e68e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63a62dec28ea4a44930d9c44561f273b in term 4.
I20260709 13:15:28.034138  9781 leader_election.cc:304] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08996682b9d042fc870e58e3db01e68e, 63a62dec28ea4a44930d9c44561f273b; no voters: 
I20260709 13:15:28.035792  9966 raft_consensus.cc:515] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:28.039682  9813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e26033fa4044e89207d8960181bfad" candidate_uuid: "08996682b9d042fc870e58e3db01e68e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "63a62dec28ea4a44930d9c44561f273b"
I20260709 13:15:28.040354  9813 raft_consensus.cc:3060] T e1e26033fa4044e89207d8960181bfad P 63a62dec28ea4a44930d9c44561f273b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:28.040371  9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e26033fa4044e89207d8960181bfad" candidate_uuid: "08996682b9d042fc870e58e3db01e68e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5"
I20260709 13:15:28.040505  9889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdcf499cc854451ab2609798dfcff264" candidate_uuid: "63a62dec28ea4a44930d9c44561f273b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5"
I20260709 13:15:28.041200  9890 raft_consensus.cc:3055] T e1e26033fa4044e89207d8960181bfad P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:28.041224  9889 raft_consensus.cc:3055] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:28.041725  9890 raft_consensus.cc:740] T e1e26033fa4044e89207d8960181bfad P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 68ba1cb4a67343a09adb98b1f9e9dcf5, State: Running, Role: LEADER
I20260709 13:15:28.043031 10156 raft_consensus.cc:2804] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:28.042894  9890 consensus_queue.cc:260] T e1e26033fa4044e89207d8960181bfad P 68ba1cb4a67343a09adb98b1f9e9dcf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:28.044551  9890 raft_consensus.cc:3060] T e1e26033fa4044e89207d8960181bfad P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:28.041785  9889 raft_consensus.cc:740] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 68ba1cb4a67343a09adb98b1f9e9dcf5, State: Running, Role: LEADER
I20260709 13:15:28.038529  9966 leader_election.cc:290] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e [CANDIDATE]: Term 4 election: Requested vote from peers 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813), 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130:44493)
I20260709 13:15:28.047683  9889 consensus_queue.cc:260] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:28.049081  9889 raft_consensus.cc:3060] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:28.051577 10156 raft_consensus.cc:697] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b [term 4 LEADER]: Becoming Leader. State: Replica: 63a62dec28ea4a44930d9c44561f273b, State: Running, Role: LEADER
I20260709 13:15:28.052811 10156 consensus_queue.cc:237] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:28.023970  9815 leader_election.cc:290] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b [CANDIDATE]: Term 4 election: Requested vote from peers 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130:44493), 08996682b9d042fc870e58e3db01e68e (127.9.116.131:40693)
I20260709 13:15:28.056809  9813 raft_consensus.cc:2468] T e1e26033fa4044e89207d8960181bfad P 63a62dec28ea4a44930d9c44561f273b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08996682b9d042fc870e58e3db01e68e in term 4.
I20260709 13:15:28.058725  9930 leader_election.cc:304] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08996682b9d042fc870e58e3db01e68e, 63a62dec28ea4a44930d9c44561f273b; no voters: 
I20260709 13:15:28.059799 10125 raft_consensus.cc:2804] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:28.060295 10125 raft_consensus.cc:697] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e [term 4 LEADER]: Becoming Leader. State: Replica: 08996682b9d042fc870e58e3db01e68e, State: Running, Role: LEADER
I20260709 13:15:28.061195 10125 consensus_queue.cc:237] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:28.062773  9890 raft_consensus.cc:2468] T e1e26033fa4044e89207d8960181bfad P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08996682b9d042fc870e58e3db01e68e in term 4.
I20260709 13:15:28.068277  9889 raft_consensus.cc:2468] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63a62dec28ea4a44930d9c44561f273b in term 4.
I20260709 13:15:28.076460  9712 catalog_manager.cc:5697] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b reported cstate change: term changed from 3 to 4, leader changed from 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130) to 63a62dec28ea4a44930d9c44561f273b (127.9.116.129). New cstate: current_term: 4 leader_uuid: "63a62dec28ea4a44930d9c44561f273b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:28.080206  9710 catalog_manager.cc:5697] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e reported cstate change: term changed from 3 to 4, leader changed from 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130) to 08996682b9d042fc870e58e3db01e68e (127.9.116.131). New cstate: current_term: 4 leader_uuid: "08996682b9d042fc870e58e3db01e68e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:28.108505 10140 raft_consensus.cc:993] T 15d491b33a7c47dab512ed81614b9b97 P 68ba1cb4a67343a09adb98b1f9e9dcf5: : Instructing follower 63a62dec28ea4a44930d9c44561f273b to start an election
I20260709 13:15:28.108980 10137 raft_consensus.cc:1081] T 15d491b33a7c47dab512ed81614b9b97 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 2 LEADER]: Signalling peer 63a62dec28ea4a44930d9c44561f273b to start an election
I20260709 13:15:28.110416  9813 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "15d491b33a7c47dab512ed81614b9b97"
dest_uuid: "63a62dec28ea4a44930d9c44561f273b"
 from {username='slave'} at 127.0.0.1:33722
I20260709 13:15:28.110906  9813 raft_consensus.cc:493] T 15d491b33a7c47dab512ed81614b9b97 P 63a62dec28ea4a44930d9c44561f273b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:28.111172  9813 raft_consensus.cc:3060] T 15d491b33a7c47dab512ed81614b9b97 P 63a62dec28ea4a44930d9c44561f273b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:28.116575  9813 raft_consensus.cc:515] T 15d491b33a7c47dab512ed81614b9b97 P 63a62dec28ea4a44930d9c44561f273b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:28.118141  9813 leader_election.cc:290] T 15d491b33a7c47dab512ed81614b9b97 P 63a62dec28ea4a44930d9c44561f273b [CANDIDATE]: Term 3 election: Requested vote from peers 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130:44493), 08996682b9d042fc870e58e3db01e68e (127.9.116.131:40693)
I20260709 13:15:28.118937  9889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d491b33a7c47dab512ed81614b9b97" candidate_uuid: "63a62dec28ea4a44930d9c44561f273b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5"
I20260709 13:15:28.119190  9965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d491b33a7c47dab512ed81614b9b97" candidate_uuid: "63a62dec28ea4a44930d9c44561f273b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "08996682b9d042fc870e58e3db01e68e"
I20260709 13:15:28.119441  9889 raft_consensus.cc:3055] T 15d491b33a7c47dab512ed81614b9b97 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:28.119695  9965 raft_consensus.cc:3060] T 15d491b33a7c47dab512ed81614b9b97 P 08996682b9d042fc870e58e3db01e68e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:28.119763  9889 raft_consensus.cc:740] T 15d491b33a7c47dab512ed81614b9b97 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 68ba1cb4a67343a09adb98b1f9e9dcf5, State: Running, Role: LEADER
I20260709 13:15:28.120498  9889 consensus_queue.cc:260] T 15d491b33a7c47dab512ed81614b9b97 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:28.121336  9889 raft_consensus.cc:3060] T 15d491b33a7c47dab512ed81614b9b97 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:28.124186  9965 raft_consensus.cc:2468] T 15d491b33a7c47dab512ed81614b9b97 P 08996682b9d042fc870e58e3db01e68e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63a62dec28ea4a44930d9c44561f273b in term 3.
I20260709 13:15:28.125141  9781 leader_election.cc:304] T 15d491b33a7c47dab512ed81614b9b97 P 63a62dec28ea4a44930d9c44561f273b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08996682b9d042fc870e58e3db01e68e, 63a62dec28ea4a44930d9c44561f273b; no voters: 
I20260709 13:15:28.125526  9889 raft_consensus.cc:2468] T 15d491b33a7c47dab512ed81614b9b97 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63a62dec28ea4a44930d9c44561f273b in term 3.
I20260709 13:15:28.125862 10156 raft_consensus.cc:2804] T 15d491b33a7c47dab512ed81614b9b97 P 63a62dec28ea4a44930d9c44561f273b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:28.126300 10156 raft_consensus.cc:697] T 15d491b33a7c47dab512ed81614b9b97 P 63a62dec28ea4a44930d9c44561f273b [term 3 LEADER]: Becoming Leader. State: Replica: 63a62dec28ea4a44930d9c44561f273b, State: Running, Role: LEADER
I20260709 13:15:28.127206 10156 consensus_queue.cc:237] T 15d491b33a7c47dab512ed81614b9b97 P 63a62dec28ea4a44930d9c44561f273b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:28.134285  9710 catalog_manager.cc:5697] T 15d491b33a7c47dab512ed81614b9b97 P 63a62dec28ea4a44930d9c44561f273b reported cstate change: term changed from 2 to 3, leader changed from 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130) to 63a62dec28ea4a44930d9c44561f273b (127.9.116.129). New cstate: current_term: 3 leader_uuid: "63a62dec28ea4a44930d9c44561f273b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:28.141171 10140 raft_consensus.cc:993] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5: : Instructing follower 63a62dec28ea4a44930d9c44561f273b to start an election
I20260709 13:15:28.141623 10137 raft_consensus.cc:1081] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 3 LEADER]: Signalling peer 63a62dec28ea4a44930d9c44561f273b to start an election
I20260709 13:15:28.143179  9813 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "701d79f9f0674270ab399ece95bc5c03"
dest_uuid: "63a62dec28ea4a44930d9c44561f273b"
 from {username='slave'} at 127.0.0.1:33722
I20260709 13:15:28.143664  9813 raft_consensus.cc:493] T 701d79f9f0674270ab399ece95bc5c03 P 63a62dec28ea4a44930d9c44561f273b [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:28.143965  9813 raft_consensus.cc:3060] T 701d79f9f0674270ab399ece95bc5c03 P 63a62dec28ea4a44930d9c44561f273b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:28.148360  9813 raft_consensus.cc:515] T 701d79f9f0674270ab399ece95bc5c03 P 63a62dec28ea4a44930d9c44561f273b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:28.150028  9813 leader_election.cc:290] T 701d79f9f0674270ab399ece95bc5c03 P 63a62dec28ea4a44930d9c44561f273b [CANDIDATE]: Term 4 election: Requested vote from peers 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130:44493), 08996682b9d042fc870e58e3db01e68e (127.9.116.131:40693)
I20260709 13:15:28.150746  9889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701d79f9f0674270ab399ece95bc5c03" candidate_uuid: "63a62dec28ea4a44930d9c44561f273b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5"
I20260709 13:15:28.151134  9965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701d79f9f0674270ab399ece95bc5c03" candidate_uuid: "63a62dec28ea4a44930d9c44561f273b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "08996682b9d042fc870e58e3db01e68e"
I20260709 13:15:28.151281  9889 raft_consensus.cc:3055] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:28.151604  9889 raft_consensus.cc:740] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 68ba1cb4a67343a09adb98b1f9e9dcf5, State: Running, Role: LEADER
I20260709 13:15:28.151633  9965 raft_consensus.cc:3060] T 701d79f9f0674270ab399ece95bc5c03 P 08996682b9d042fc870e58e3db01e68e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:28.152302  9889 consensus_queue.cc:260] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:28.153239  9889 raft_consensus.cc:3060] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:28.156018  9965 raft_consensus.cc:2468] T 701d79f9f0674270ab399ece95bc5c03 P 08996682b9d042fc870e58e3db01e68e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63a62dec28ea4a44930d9c44561f273b in term 4.
I20260709 13:15:28.157001  9781 leader_election.cc:304] T 701d79f9f0674270ab399ece95bc5c03 P 63a62dec28ea4a44930d9c44561f273b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08996682b9d042fc870e58e3db01e68e, 63a62dec28ea4a44930d9c44561f273b; no voters: 
I20260709 13:15:28.157541  9889 raft_consensus.cc:2468] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63a62dec28ea4a44930d9c44561f273b in term 4.
I20260709 13:15:28.157732 10156 raft_consensus.cc:2804] T 701d79f9f0674270ab399ece95bc5c03 P 63a62dec28ea4a44930d9c44561f273b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:28.158214 10156 raft_consensus.cc:697] T 701d79f9f0674270ab399ece95bc5c03 P 63a62dec28ea4a44930d9c44561f273b [term 4 LEADER]: Becoming Leader. State: Replica: 63a62dec28ea4a44930d9c44561f273b, State: Running, Role: LEADER
I20260709 13:15:28.159070 10156 consensus_queue.cc:237] T 701d79f9f0674270ab399ece95bc5c03 P 63a62dec28ea4a44930d9c44561f273b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } }
I20260709 13:15:28.166035  9710 catalog_manager.cc:5697] T 701d79f9f0674270ab399ece95bc5c03 P 63a62dec28ea4a44930d9c44561f273b reported cstate change: term changed from 3 to 4, leader changed from 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130) to 63a62dec28ea4a44930d9c44561f273b (127.9.116.129). New cstate: current_term: 4 leader_uuid: "63a62dec28ea4a44930d9c44561f273b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:28.444154  9889 raft_consensus.cc:1275] T e1e26033fa4044e89207d8960181bfad P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 4 FOLLOWER]: Refusing update from remote peer 08996682b9d042fc870e58e3db01e68e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:15:28.445465 10125 consensus_queue.cc:1048] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e [LEADER]: Connected to new peer: Peer: permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:28.456380  9813 raft_consensus.cc:1275] T e1e26033fa4044e89207d8960181bfad P 63a62dec28ea4a44930d9c44561f273b [term 4 FOLLOWER]: Refusing update from remote peer 08996682b9d042fc870e58e3db01e68e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:15:28.458019 10125 consensus_queue.cc:1048] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e [LEADER]: Connected to new peer: Peer: permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:28.533710  9889 raft_consensus.cc:1275] T dba81173e16b42ab92d7da17ba0518a7 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 3 FOLLOWER]: Refusing update from remote peer 08996682b9d042fc870e58e3db01e68e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:28.534989 10125 consensus_queue.cc:1048] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e [LEADER]: Connected to new peer: Peer: permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:28.546456  9813 raft_consensus.cc:1275] T dba81173e16b42ab92d7da17ba0518a7 P 63a62dec28ea4a44930d9c44561f273b [term 3 FOLLOWER]: Refusing update from remote peer 08996682b9d042fc870e58e3db01e68e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:28.548034 10168 consensus_queue.cc:1048] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e [LEADER]: Connected to new peer: Peer: permanent_uuid: "63a62dec28ea4a44930d9c44561f273b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:28.555213  9889 raft_consensus.cc:1275] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 4 FOLLOWER]: Refusing update from remote peer 63a62dec28ea4a44930d9c44561f273b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:15:28.556373 10156 consensus_queue.cc:1048] T 701d79f9f0674270ab399ece95bc5c03 P 63a62dec28ea4a44930d9c44561f273b [LEADER]: Connected to new peer: Peer: permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:28.565014  9965 raft_consensus.cc:1275] T 701d79f9f0674270ab399ece95bc5c03 P 08996682b9d042fc870e58e3db01e68e [term 4 FOLLOWER]: Refusing update from remote peer 63a62dec28ea4a44930d9c44561f273b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:15:28.566529  9965 raft_consensus.cc:1275] T 15d491b33a7c47dab512ed81614b9b97 P 08996682b9d042fc870e58e3db01e68e [term 3 FOLLOWER]: Refusing update from remote peer 63a62dec28ea4a44930d9c44561f273b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:28.566741 10156 consensus_queue.cc:1048] T 701d79f9f0674270ab399ece95bc5c03 P 63a62dec28ea4a44930d9c44561f273b [LEADER]: Connected to new peer: Peer: permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:28.568324 10156 consensus_queue.cc:1048] T 15d491b33a7c47dab512ed81614b9b97 P 63a62dec28ea4a44930d9c44561f273b [LEADER]: Connected to new peer: Peer: permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:28.581543  9889 raft_consensus.cc:1275] T 15d491b33a7c47dab512ed81614b9b97 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 3 FOLLOWER]: Refusing update from remote peer 63a62dec28ea4a44930d9c44561f273b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:28.583348 10174 consensus_queue.cc:1048] T 15d491b33a7c47dab512ed81614b9b97 P 63a62dec28ea4a44930d9c44561f273b [LEADER]: Connected to new peer: Peer: permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:28.599201  9965 raft_consensus.cc:1275] T cdcf499cc854451ab2609798dfcff264 P 08996682b9d042fc870e58e3db01e68e [term 4 FOLLOWER]: Refusing update from remote peer 63a62dec28ea4a44930d9c44561f273b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:15:28.600313 10174 consensus_queue.cc:1048] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b [LEADER]: Connected to new peer: Peer: permanent_uuid: "08996682b9d042fc870e58e3db01e68e" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 40693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:28.609573  9889 raft_consensus.cc:1275] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 4 FOLLOWER]: Refusing update from remote peer 63a62dec28ea4a44930d9c44561f273b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:15:28.611199 10174 consensus_queue.cc:1048] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b [LEADER]: Connected to new peer: Peer: permanent_uuid: "68ba1cb4a67343a09adb98b1f9e9dcf5" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 44493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:28.655577  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:28.661201  9682 tablet_server.cc:179] TabletServer@127.9.116.129:0 shutting down...
W20260709 13:15:28.685834  9856 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:28.689433  9856 consensus_peers.cc:597] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5 -> Peer 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813): Couldn't send request to peer 63a62dec28ea4a44930d9c44561f273b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:28.690150  9682 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:15:28.690953  9682 tablet_replica.cc:333] T 701d79f9f0674270ab399ece95bc5c03 P 63a62dec28ea4a44930d9c44561f273b: stopping tablet replica
I20260709 13:15:28.691691  9682 raft_consensus.cc:2243] T 701d79f9f0674270ab399ece95bc5c03 P 63a62dec28ea4a44930d9c44561f273b [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:15:28.692497  9682 raft_consensus.cc:2272] T 701d79f9f0674270ab399ece95bc5c03 P 63a62dec28ea4a44930d9c44561f273b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:28.695037  9682 tablet_replica.cc:333] T 15d491b33a7c47dab512ed81614b9b97 P 63a62dec28ea4a44930d9c44561f273b: stopping tablet replica
I20260709 13:15:28.695566  9682 raft_consensus.cc:2243] T 15d491b33a7c47dab512ed81614b9b97 P 63a62dec28ea4a44930d9c44561f273b [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:15:28.696285  9682 raft_consensus.cc:2272] T 15d491b33a7c47dab512ed81614b9b97 P 63a62dec28ea4a44930d9c44561f273b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:28.698282  9682 tablet_replica.cc:333] T dba81173e16b42ab92d7da17ba0518a7 P 63a62dec28ea4a44930d9c44561f273b: stopping tablet replica
I20260709 13:15:28.698758  9682 raft_consensus.cc:2243] T dba81173e16b42ab92d7da17ba0518a7 P 63a62dec28ea4a44930d9c44561f273b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:28.699276  9682 raft_consensus.cc:2272] T dba81173e16b42ab92d7da17ba0518a7 P 63a62dec28ea4a44930d9c44561f273b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:28.701139  9682 tablet_replica.cc:333] T 986acd03cce8455aad79beacff7aa154 P 63a62dec28ea4a44930d9c44561f273b: stopping tablet replica
I20260709 13:15:28.701586  9682 raft_consensus.cc:2243] T 986acd03cce8455aad79beacff7aa154 P 63a62dec28ea4a44930d9c44561f273b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:28.701926  9682 raft_consensus.cc:2272] T 986acd03cce8455aad79beacff7aa154 P 63a62dec28ea4a44930d9c44561f273b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:28.703436  9682 tablet_replica.cc:333] T d94510fe6d394c62a543502e8ace50eb P 63a62dec28ea4a44930d9c44561f273b: stopping tablet replica
I20260709 13:15:28.703862  9682 raft_consensus.cc:2243] T d94510fe6d394c62a543502e8ace50eb P 63a62dec28ea4a44930d9c44561f273b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:28.704226  9682 raft_consensus.cc:2272] T d94510fe6d394c62a543502e8ace50eb P 63a62dec28ea4a44930d9c44561f273b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:28.705739  9682 tablet_replica.cc:333] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b: stopping tablet replica
I20260709 13:15:28.706141  9682 raft_consensus.cc:2243] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:15:28.706851  9682 raft_consensus.cc:2272] T cdcf499cc854451ab2609798dfcff264 P 63a62dec28ea4a44930d9c44561f273b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:28.708556  9682 tablet_replica.cc:333] T 40e71d19ec8244748ceb00b2ab546eb9 P 63a62dec28ea4a44930d9c44561f273b: stopping tablet replica
I20260709 13:15:28.708978  9682 raft_consensus.cc:2243] T 40e71d19ec8244748ceb00b2ab546eb9 P 63a62dec28ea4a44930d9c44561f273b [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:28.709317  9682 raft_consensus.cc:2272] T 40e71d19ec8244748ceb00b2ab546eb9 P 63a62dec28ea4a44930d9c44561f273b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:28.711145  9682 tablet_replica.cc:333] T e1e26033fa4044e89207d8960181bfad P 63a62dec28ea4a44930d9c44561f273b: stopping tablet replica
I20260709 13:15:28.711555  9682 raft_consensus.cc:2243] T e1e26033fa4044e89207d8960181bfad P 63a62dec28ea4a44930d9c44561f273b [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:28.711974  9682 raft_consensus.cc:2272] T e1e26033fa4044e89207d8960181bfad P 63a62dec28ea4a44930d9c44561f273b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:28.713829  9682 tablet_replica.cc:333] T f27f84491f9c4b85af0ef5524c00f5e8 P 63a62dec28ea4a44930d9c44561f273b: stopping tablet replica
I20260709 13:15:28.714284  9682 raft_consensus.cc:2243] T f27f84491f9c4b85af0ef5524c00f5e8 P 63a62dec28ea4a44930d9c44561f273b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:28.714684  9682 raft_consensus.cc:2272] T f27f84491f9c4b85af0ef5524c00f5e8 P 63a62dec28ea4a44930d9c44561f273b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:28.828125  9682 tablet_server.cc:196] TabletServer@127.9.116.129:0 shutdown complete.
W20260709 13:15:28.830864  9856 consensus_peers.cc:597] T 986acd03cce8455aad79beacff7aa154 P 68ba1cb4a67343a09adb98b1f9e9dcf5 -> Peer 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813): Couldn't send request to peer 63a62dec28ea4a44930d9c44561f273b. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:41813: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:28.847946  9682 tablet_server.cc:179] TabletServer@127.9.116.130:0 shutting down...
W20260709 13:15:28.876667  9930 consensus_peers.cc:597] T d94510fe6d394c62a543502e8ace50eb P 08996682b9d042fc870e58e3db01e68e -> Peer 63a62dec28ea4a44930d9c44561f273b (127.9.116.129:41813): Couldn't send request to peer 63a62dec28ea4a44930d9c44561f273b. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:41813: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:28.877161  9682 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:15:28.877820  9682 tablet_replica.cc:333] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5: stopping tablet replica
I20260709 13:15:28.878364  9682 raft_consensus.cc:2243] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:28.878897  9682 raft_consensus.cc:2272] T 701d79f9f0674270ab399ece95bc5c03 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:28.880954  9682 tablet_replica.cc:333] T 986acd03cce8455aad79beacff7aa154 P 68ba1cb4a67343a09adb98b1f9e9dcf5: stopping tablet replica
I20260709 13:15:28.881450  9682 raft_consensus.cc:2243] T 986acd03cce8455aad79beacff7aa154 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:15:28.882205  9682 raft_consensus.cc:2272] T 986acd03cce8455aad79beacff7aa154 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:28.883893  9682 tablet_replica.cc:333] T d94510fe6d394c62a543502e8ace50eb P 68ba1cb4a67343a09adb98b1f9e9dcf5: stopping tablet replica
I20260709 13:15:28.884406  9682 raft_consensus.cc:2243] T d94510fe6d394c62a543502e8ace50eb P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:28.884841  9682 raft_consensus.cc:2272] T d94510fe6d394c62a543502e8ace50eb P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:28.886467  9682 tablet_replica.cc:333] T dba81173e16b42ab92d7da17ba0518a7 P 68ba1cb4a67343a09adb98b1f9e9dcf5: stopping tablet replica
I20260709 13:15:28.887012  9682 raft_consensus.cc:2243] T dba81173e16b42ab92d7da17ba0518a7 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:28.887398  9682 raft_consensus.cc:2272] T dba81173e16b42ab92d7da17ba0518a7 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:28.889266  9682 tablet_replica.cc:333] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5: stopping tablet replica
I20260709 13:15:28.889753  9682 raft_consensus.cc:2243] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:28.890144  9682 raft_consensus.cc:2272] T cdcf499cc854451ab2609798dfcff264 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:28.892023  9682 tablet_replica.cc:333] T e1e26033fa4044e89207d8960181bfad P 68ba1cb4a67343a09adb98b1f9e9dcf5: stopping tablet replica
I20260709 13:15:28.892511  9682 raft_consensus.cc:2243] T e1e26033fa4044e89207d8960181bfad P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:28.892907  9682 raft_consensus.cc:2272] T e1e26033fa4044e89207d8960181bfad P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:28.894745  9682 tablet_replica.cc:333] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5: stopping tablet replica
I20260709 13:15:28.895290  9682 raft_consensus.cc:2243] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:15:28.895984  9682 raft_consensus.cc:2272] T 40e71d19ec8244748ceb00b2ab546eb9 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:28.897547  9682 tablet_replica.cc:333] T 15d491b33a7c47dab512ed81614b9b97 P 68ba1cb4a67343a09adb98b1f9e9dcf5: stopping tablet replica
I20260709 13:15:28.898000  9682 raft_consensus.cc:2243] T 15d491b33a7c47dab512ed81614b9b97 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:28.898379  9682 raft_consensus.cc:2272] T 15d491b33a7c47dab512ed81614b9b97 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:28.900430  9682 tablet_replica.cc:333] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5: stopping tablet replica
I20260709 13:15:28.900887  9682 raft_consensus.cc:2243] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:15:28.901624  9682 raft_consensus.cc:2272] T f27f84491f9c4b85af0ef5524c00f5e8 P 68ba1cb4a67343a09adb98b1f9e9dcf5 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:28.919291  9929 consensus_peers.cc:597] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e -> Peer 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130:44493): Couldn't send request to peer 68ba1cb4a67343a09adb98b1f9e9dcf5. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:44493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:28.929056  9682 tablet_server.cc:196] TabletServer@127.9.116.130:0 shutdown complete.
I20260709 13:15:28.946645  9682 tablet_server.cc:179] TabletServer@127.9.116.131:0 shutting down...
W20260709 13:15:28.946871  9929 consensus_peers.cc:597] T d94510fe6d394c62a543502e8ace50eb P 08996682b9d042fc870e58e3db01e68e -> Peer 68ba1cb4a67343a09adb98b1f9e9dcf5 (127.9.116.130:44493): Couldn't send request to peer 68ba1cb4a67343a09adb98b1f9e9dcf5. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:44493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:28.972384  9682 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:15:28.973067  9682 tablet_replica.cc:333] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e: stopping tablet replica
I20260709 13:15:28.973794  9682 raft_consensus.cc:2243] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:15:28.974529  9682 raft_consensus.cc:2272] T dba81173e16b42ab92d7da17ba0518a7 P 08996682b9d042fc870e58e3db01e68e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:28.976555  9682 tablet_replica.cc:333] T 15d491b33a7c47dab512ed81614b9b97 P 08996682b9d042fc870e58e3db01e68e: stopping tablet replica
I20260709 13:15:28.977021  9682 raft_consensus.cc:2243] T 15d491b33a7c47dab512ed81614b9b97 P 08996682b9d042fc870e58e3db01e68e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:28.977391  9682 raft_consensus.cc:2272] T 15d491b33a7c47dab512ed81614b9b97 P 08996682b9d042fc870e58e3db01e68e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:28.979249  9682 tablet_replica.cc:333] T cdcf499cc854451ab2609798dfcff264 P 08996682b9d042fc870e58e3db01e68e: stopping tablet replica
I20260709 13:15:28.979692  9682 raft_consensus.cc:2243] T cdcf499cc854451ab2609798dfcff264 P 08996682b9d042fc870e58e3db01e68e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:28.980077  9682 raft_consensus.cc:2272] T cdcf499cc854451ab2609798dfcff264 P 08996682b9d042fc870e58e3db01e68e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:28.981962  9682 tablet_replica.cc:333] T d94510fe6d394c62a543502e8ace50eb P 08996682b9d042fc870e58e3db01e68e: stopping tablet replica
I20260709 13:15:28.982395  9682 raft_consensus.cc:2243] T d94510fe6d394c62a543502e8ace50eb P 08996682b9d042fc870e58e3db01e68e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:15:28.983163  9682 raft_consensus.cc:2272] T d94510fe6d394c62a543502e8ace50eb P 08996682b9d042fc870e58e3db01e68e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:28.984788  9682 tablet_replica.cc:333] T 986acd03cce8455aad79beacff7aa154 P 08996682b9d042fc870e58e3db01e68e: stopping tablet replica
I20260709 13:15:28.985211  9682 raft_consensus.cc:2243] T 986acd03cce8455aad79beacff7aa154 P 08996682b9d042fc870e58e3db01e68e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:28.985581  9682 raft_consensus.cc:2272] T 986acd03cce8455aad79beacff7aa154 P 08996682b9d042fc870e58e3db01e68e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:28.987218  9682 tablet_replica.cc:333] T 40e71d19ec8244748ceb00b2ab546eb9 P 08996682b9d042fc870e58e3db01e68e: stopping tablet replica
I20260709 13:15:28.987625  9682 raft_consensus.cc:2243] T 40e71d19ec8244748ceb00b2ab546eb9 P 08996682b9d042fc870e58e3db01e68e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:28.987970  9682 raft_consensus.cc:2272] T 40e71d19ec8244748ceb00b2ab546eb9 P 08996682b9d042fc870e58e3db01e68e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:28.989573  9682 tablet_replica.cc:333] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e: stopping tablet replica
I20260709 13:15:28.989986  9682 raft_consensus.cc:2243] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:15:28.990609  9682 raft_consensus.cc:2272] T e1e26033fa4044e89207d8960181bfad P 08996682b9d042fc870e58e3db01e68e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:28.992391  9682 tablet_replica.cc:333] T 701d79f9f0674270ab399ece95bc5c03 P 08996682b9d042fc870e58e3db01e68e: stopping tablet replica
I20260709 13:15:28.992800  9682 raft_consensus.cc:2243] T 701d79f9f0674270ab399ece95bc5c03 P 08996682b9d042fc870e58e3db01e68e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:28.993150  9682 raft_consensus.cc:2272] T 701d79f9f0674270ab399ece95bc5c03 P 08996682b9d042fc870e58e3db01e68e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:28.994993  9682 tablet_replica.cc:333] T f27f84491f9c4b85af0ef5524c00f5e8 P 08996682b9d042fc870e58e3db01e68e: stopping tablet replica
I20260709 13:15:28.995463  9682 raft_consensus.cc:2243] T f27f84491f9c4b85af0ef5524c00f5e8 P 08996682b9d042fc870e58e3db01e68e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:28.995956  9682 raft_consensus.cc:2272] T f27f84491f9c4b85af0ef5524c00f5e8 P 08996682b9d042fc870e58e3db01e68e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:29.021706  9682 tablet_server.cc:196] TabletServer@127.9.116.131:0 shutdown complete.
I20260709 13:15:29.037796  9682 master.cc:562] Master@127.9.116.190:35419 shutting down...
I20260709 13:15:29.236502  9682 raft_consensus.cc:2243] T 00000000000000000000000000000000 P adf6cb397e6c4208ba71a60bcd491798 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:15:29.237051  9682 raft_consensus.cc:2272] T 00000000000000000000000000000000 P adf6cb397e6c4208ba71a60bcd491798 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:29.237352  9682 tablet_replica.cc:333] T 00000000000000000000000000000000 P adf6cb397e6c4208ba71a60bcd491798: stopping tablet replica
I20260709 13:15:29.257995  9682 master.cc:584] Master@127.9.116.190:35419 shutdown complete.
08996682b9d042fc870e58e3db01e68e  1
63a62dec28ea4a44930d9c44561f273b  4
68ba1cb4a67343a09adb98b1f9e9dcf5  4
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (12599 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 13:15:29.296865  9682 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.9.116.190:39829
I20260709 13:15:29.297981  9682 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:15:29.304200 10185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:29.305119 10186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:29.305439 10188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:29.305574  9682 server_base.cc:1061] running on GCE node
I20260709 13:15:29.306631  9682 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:15:29.306882  9682 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:15:29.307073  9682 hybrid_clock.cc:648] HybridClock initialized: now 1783602929307056 us; error 0 us; skew 500 ppm
I20260709 13:15:29.307616  9682 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:15:29.309922  9682 webserver.cc:533] Webserver started at http://127.9.116.190:45989/ using document root <none> and password file <none>
I20260709 13:15:29.310369  9682 fs_manager.cc:362] Metadata directory not provided
I20260709 13:15:29.310534  9682 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:15:29.310755  9682 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:15:29.311904  9682 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602916530764-9682-0/minicluster-data/master-0-root/instance:
uuid: "fb45f429aff74b84b7c676c2531e4c2a"
format_stamp: "Formatted at 2026-07-09 13:15:29 on dist-test-slave-1t6m"
I20260709 13:15:29.316332  9682 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:15:29.319720 10193 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:29.320494  9682 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:15:29.320814  9682 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602916530764-9682-0/minicluster-data/master-0-root
uuid: "fb45f429aff74b84b7c676c2531e4c2a"
format_stamp: "Formatted at 2026-07-09 13:15:29 on dist-test-slave-1t6m"
I20260709 13:15:29.321134  9682 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602916530764-9682-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602916530764-9682-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602916530764-9682-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:15:29.342029  9682 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:15:29.343390  9682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:15:29.380410  9682 rpc_server.cc:307] RPC server started. Bound to: 127.9.116.190:39829
I20260709 13:15:29.380511 10244 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.116.190:39829 every 8 connection(s)
I20260709 13:15:29.384227 10245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:29.393535 10245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb45f429aff74b84b7c676c2531e4c2a: Bootstrap starting.
I20260709 13:15:29.398021 10245 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb45f429aff74b84b7c676c2531e4c2a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:29.402109 10245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb45f429aff74b84b7c676c2531e4c2a: No bootstrap required, opened a new log
I20260709 13:15:29.404126 10245 raft_consensus.cc:359] T 00000000000000000000000000000000 P fb45f429aff74b84b7c676c2531e4c2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb45f429aff74b84b7c676c2531e4c2a" member_type: VOTER }
I20260709 13:15:29.404505 10245 raft_consensus.cc:385] T 00000000000000000000000000000000 P fb45f429aff74b84b7c676c2531e4c2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:29.404677 10245 raft_consensus.cc:740] T 00000000000000000000000000000000 P fb45f429aff74b84b7c676c2531e4c2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb45f429aff74b84b7c676c2531e4c2a, State: Initialized, Role: FOLLOWER
I20260709 13:15:29.405239 10245 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb45f429aff74b84b7c676c2531e4c2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb45f429aff74b84b7c676c2531e4c2a" member_type: VOTER }
I20260709 13:15:29.405665 10245 raft_consensus.cc:399] T 00000000000000000000000000000000 P fb45f429aff74b84b7c676c2531e4c2a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:15:29.405861 10245 raft_consensus.cc:493] T 00000000000000000000000000000000 P fb45f429aff74b84b7c676c2531e4c2a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:15:29.406067 10245 raft_consensus.cc:3060] T 00000000000000000000000000000000 P fb45f429aff74b84b7c676c2531e4c2a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:29.409945 10245 raft_consensus.cc:515] T 00000000000000000000000000000000 P fb45f429aff74b84b7c676c2531e4c2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb45f429aff74b84b7c676c2531e4c2a" member_type: VOTER }
I20260709 13:15:29.410423 10245 leader_election.cc:304] T 00000000000000000000000000000000 P fb45f429aff74b84b7c676c2531e4c2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb45f429aff74b84b7c676c2531e4c2a; no voters: 
I20260709 13:15:29.411502 10245 leader_election.cc:290] T 00000000000000000000000000000000 P fb45f429aff74b84b7c676c2531e4c2a [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:15:29.411923 10248 raft_consensus.cc:2804] T 00000000000000000000000000000000 P fb45f429aff74b84b7c676c2531e4c2a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:29.413244 10248 raft_consensus.cc:697] T 00000000000000000000000000000000 P fb45f429aff74b84b7c676c2531e4c2a [term 1 LEADER]: Becoming Leader. State: Replica: fb45f429aff74b84b7c676c2531e4c2a, State: Running, Role: LEADER
I20260709 13:15:29.413945 10248 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb45f429aff74b84b7c676c2531e4c2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb45f429aff74b84b7c676c2531e4c2a" member_type: VOTER }
I20260709 13:15:29.414539 10245 sys_catalog.cc:565] T 00000000000000000000000000000000 P fb45f429aff74b84b7c676c2531e4c2a [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:15:29.418740 10250 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb45f429aff74b84b7c676c2531e4c2a [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb45f429aff74b84b7c676c2531e4c2a. Latest consensus state: current_term: 1 leader_uuid: "fb45f429aff74b84b7c676c2531e4c2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb45f429aff74b84b7c676c2531e4c2a" member_type: VOTER } }
I20260709 13:15:29.418995 10249 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb45f429aff74b84b7c676c2531e4c2a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb45f429aff74b84b7c676c2531e4c2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb45f429aff74b84b7c676c2531e4c2a" member_type: VOTER } }
I20260709 13:15:29.419545 10250 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb45f429aff74b84b7c676c2531e4c2a [sys.catalog]: This master's current role is: LEADER
I20260709 13:15:29.420038 10249 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb45f429aff74b84b7c676c2531e4c2a [sys.catalog]: This master's current role is: LEADER
I20260709 13:15:29.423048 10254 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:15:29.429821 10254 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:15:29.431651  9682 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:15:29.438604 10254 catalog_manager.cc:1383] Generated new cluster ID: 961f64b88f8843eeb3ab3e13b09b0ff4
I20260709 13:15:29.438941 10254 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:15:29.453148 10254 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:15:29.454402 10254 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:15:29.473318 10254 catalog_manager.cc:6070] T 00000000000000000000000000000000 P fb45f429aff74b84b7c676c2531e4c2a: Generated new TSK 0
I20260709 13:15:29.473935 10254 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:15:29.497926  9682 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:15:29.504063 10266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:29.505285 10267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:29.508127 10269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:29.508211  9682 server_base.cc:1061] running on GCE node
I20260709 13:15:29.509274  9682 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:15:29.509516  9682 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:15:29.509691  9682 hybrid_clock.cc:648] HybridClock initialized: now 1783602929509673 us; error 0 us; skew 500 ppm
I20260709 13:15:29.510210  9682 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:15:29.512523  9682 webserver.cc:533] Webserver started at http://127.9.116.129:34993/ using document root <none> and password file <none>
I20260709 13:15:29.513000  9682 fs_manager.cc:362] Metadata directory not provided
I20260709 13:15:29.513204  9682 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:15:29.513482  9682 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:15:29.515076  9682 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602916530764-9682-0/minicluster-data/ts-0-root/instance:
uuid: "f090b64140fe4e55942c16e20a79b36b"
format_stamp: "Formatted at 2026-07-09 13:15:29 on dist-test-slave-1t6m"
I20260709 13:15:29.519512  9682 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 13:15:29.522689 10274 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:29.523631  9682 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:15:29.523872  9682 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602916530764-9682-0/minicluster-data/ts-0-root
uuid: "f090b64140fe4e55942c16e20a79b36b"
format_stamp: "Formatted at 2026-07-09 13:15:29 on dist-test-slave-1t6m"
I20260709 13:15:29.524122  9682 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602916530764-9682-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602916530764-9682-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602916530764-9682-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:15:29.554095  9682 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:15:29.555430  9682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:15:29.556980  9682 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:15:29.559442  9682 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:15:29.559623  9682 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:29.559803  9682 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:15:29.559953  9682 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:29.598902  9682 rpc_server.cc:307] RPC server started. Bound to: 127.9.116.129:42273
I20260709 13:15:29.598989 10336 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.116.129:42273 every 8 connection(s)
I20260709 13:15:29.603610  9682 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:15:29.610884 10341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:29.612154 10342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:29.616118  9682 server_base.cc:1061] running on GCE node
W20260709 13:15:29.616439 10344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:29.617587  9682 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:15:29.617832  9682 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:15:29.618026  9682 hybrid_clock.cc:648] HybridClock initialized: now 1783602929618008 us; error 0 us; skew 500 ppm
I20260709 13:15:29.618741  9682 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:15:29.620787 10337 heartbeater.cc:344] Connected to a master server at 127.9.116.190:39829
I20260709 13:15:29.621205 10337 heartbeater.cc:461] Registering TS with master...
I20260709 13:15:29.621542  9682 webserver.cc:533] Webserver started at http://127.9.116.130:44423/ using document root <none> and password file <none>
I20260709 13:15:29.622159  9682 fs_manager.cc:362] Metadata directory not provided
I20260709 13:15:29.622097 10337 heartbeater.cc:507] Master 127.9.116.190:39829 requested a full tablet report, sending...
I20260709 13:15:29.622510  9682 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:15:29.622872  9682 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:15:29.624184  9682 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602916530764-9682-0/minicluster-data/ts-1-root/instance:
uuid: "28d1d6c02b384aeaa740460051ea351b"
format_stamp: "Formatted at 2026-07-09 13:15:29 on dist-test-slave-1t6m"
I20260709 13:15:29.624521 10210 ts_manager.cc:194] Registered new tserver with Master: f090b64140fe4e55942c16e20a79b36b (127.9.116.129:42273)
I20260709 13:15:29.626981 10210 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44950
I20260709 13:15:29.629848  9682 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:15:29.632992 10349 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:29.633723  9682 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:15:29.634020  9682 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602916530764-9682-0/minicluster-data/ts-1-root
uuid: "28d1d6c02b384aeaa740460051ea351b"
format_stamp: "Formatted at 2026-07-09 13:15:29 on dist-test-slave-1t6m"
I20260709 13:15:29.634303  9682 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602916530764-9682-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602916530764-9682-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602916530764-9682-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:15:29.647742  9682 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:15:29.648959  9682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:15:29.650414  9682 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:15:29.652678  9682 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:15:29.652858  9682 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:29.653159  9682 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:15:29.653321  9682 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:29.693140  9682 rpc_server.cc:307] RPC server started. Bound to: 127.9.116.130:36353
I20260709 13:15:29.693238 10411 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.116.130:36353 every 8 connection(s)
I20260709 13:15:29.697825  9682 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:15:29.704468 10415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:29.706228 10416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:29.709504 10418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:29.710306 10412 heartbeater.cc:344] Connected to a master server at 127.9.116.190:39829
I20260709 13:15:29.710703 10412 heartbeater.cc:461] Registering TS with master...
I20260709 13:15:29.710919  9682 server_base.cc:1061] running on GCE node
I20260709 13:15:29.711638 10412 heartbeater.cc:507] Master 127.9.116.190:39829 requested a full tablet report, sending...
I20260709 13:15:29.711750  9682 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:15:29.712045  9682 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:15:29.712222  9682 hybrid_clock.cc:648] HybridClock initialized: now 1783602929712204 us; error 0 us; skew 500 ppm
I20260709 13:15:29.712956  9682 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:15:29.713878 10210 ts_manager.cc:194] Registered new tserver with Master: 28d1d6c02b384aeaa740460051ea351b (127.9.116.130:36353)
I20260709 13:15:29.715569 10210 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44952
I20260709 13:15:29.715624  9682 webserver.cc:533] Webserver started at http://127.9.116.131:43593/ using document root <none> and password file <none>
I20260709 13:15:29.716264  9682 fs_manager.cc:362] Metadata directory not provided
I20260709 13:15:29.716496  9682 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:15:29.716799  9682 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:15:29.717918  9682 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602916530764-9682-0/minicluster-data/ts-2-root/instance:
uuid: "dd07659d40f84c2aafeaabddc392aeab"
format_stamp: "Formatted at 2026-07-09 13:15:29 on dist-test-slave-1t6m"
I20260709 13:15:29.722165  9682 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 13:15:29.725594 10423 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:29.726413  9682 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:15:29.726755  9682 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602916530764-9682-0/minicluster-data/ts-2-root
uuid: "dd07659d40f84c2aafeaabddc392aeab"
format_stamp: "Formatted at 2026-07-09 13:15:29 on dist-test-slave-1t6m"
I20260709 13:15:29.727113  9682 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602916530764-9682-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602916530764-9682-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602916530764-9682-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:15:29.738485  9682 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:15:29.739758  9682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:15:29.741160  9682 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:15:29.743399  9682 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:15:29.743631  9682 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:29.743875  9682 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:15:29.744050  9682 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:29.782892  9682 rpc_server.cc:307] RPC server started. Bound to: 127.9.116.131:42413
I20260709 13:15:29.782945 10485 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.116.131:42413 every 8 connection(s)
I20260709 13:15:29.796386 10486 heartbeater.cc:344] Connected to a master server at 127.9.116.190:39829
I20260709 13:15:29.796713 10486 heartbeater.cc:461] Registering TS with master...
I20260709 13:15:29.797371 10486 heartbeater.cc:507] Master 127.9.116.190:39829 requested a full tablet report, sending...
I20260709 13:15:29.799248 10210 ts_manager.cc:194] Registered new tserver with Master: dd07659d40f84c2aafeaabddc392aeab (127.9.116.131:42413)
I20260709 13:15:29.799412  9682 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013367628s
I20260709 13:15:29.801055 10210 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44960
I20260709 13:15:29.824133 10210 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44972:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:15:29.827072 10210 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:15:29.905695 10451 tablet_service.cc:1511] Processing CreateTablet for tablet beb41e866f444744a06e30def55c3190 (DEFAULT_TABLE table=test-workload [id=0084462a390d469aad0b429e207044f1]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:15:29.907233 10451 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet beb41e866f444744a06e30def55c3190. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:29.906759 10450 tablet_service.cc:1511] Processing CreateTablet for tablet 7fa6a2759a7d43a28bc364bdd2593069 (DEFAULT_TABLE table=test-workload [id=0084462a390d469aad0b429e207044f1]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:15:29.908213 10450 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fa6a2759a7d43a28bc364bdd2593069. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:29.910082 10449 tablet_service.cc:1511] Processing CreateTablet for tablet 09786fc30e8048d9b7928fe7bf969df1 (DEFAULT_TABLE table=test-workload [id=0084462a390d469aad0b429e207044f1]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:15:29.911473 10449 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09786fc30e8048d9b7928fe7bf969df1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:29.914420 10448 tablet_service.cc:1511] Processing CreateTablet for tablet 9db04314c2c849898b82d951374e2878 (DEFAULT_TABLE table=test-workload [id=0084462a390d469aad0b429e207044f1]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:15:29.915828 10448 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9db04314c2c849898b82d951374e2878. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:29.918676 10376 tablet_service.cc:1511] Processing CreateTablet for tablet 7fa6a2759a7d43a28bc364bdd2593069 (DEFAULT_TABLE table=test-workload [id=0084462a390d469aad0b429e207044f1]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:15:29.920145 10376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fa6a2759a7d43a28bc364bdd2593069. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:29.921468 10375 tablet_service.cc:1511] Processing CreateTablet for tablet 09786fc30e8048d9b7928fe7bf969df1 (DEFAULT_TABLE table=test-workload [id=0084462a390d469aad0b429e207044f1]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:15:29.919734 10377 tablet_service.cc:1511] Processing CreateTablet for tablet beb41e866f444744a06e30def55c3190 (DEFAULT_TABLE table=test-workload [id=0084462a390d469aad0b429e207044f1]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:15:29.922888 10375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09786fc30e8048d9b7928fe7bf969df1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:29.926680 10374 tablet_service.cc:1511] Processing CreateTablet for tablet 9db04314c2c849898b82d951374e2878 (DEFAULT_TABLE table=test-workload [id=0084462a390d469aad0b429e207044f1]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:15:29.927181 10377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet beb41e866f444744a06e30def55c3190. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:29.928200 10374 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9db04314c2c849898b82d951374e2878. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:29.929896 10373 tablet_service.cc:1511] Processing CreateTablet for tablet f7a41f1857af412bb47303bd9452b65f (DEFAULT_TABLE table=test-workload [id=0084462a390d469aad0b429e207044f1]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:15:29.931296 10373 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7a41f1857af412bb47303bd9452b65f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:29.961970 10448 tablet_service.cc:1511] Processing CreateTablet for tablet f7a41f1857af412bb47303bd9452b65f (DEFAULT_TABLE table=test-workload [id=0084462a390d469aad0b429e207044f1]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:15:29.963913 10448 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7a41f1857af412bb47303bd9452b65f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:29.964342 10372 tablet_service.cc:1511] Processing CreateTablet for tablet 135e8982a8ea4fa198de62d0e60b459c (DEFAULT_TABLE table=test-workload [id=0084462a390d469aad0b429e207044f1]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:15:29.965631 10372 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 135e8982a8ea4fa198de62d0e60b459c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:29.979923 10300 tablet_service.cc:1511] Processing CreateTablet for tablet 09786fc30e8048d9b7928fe7bf969df1 (DEFAULT_TABLE table=test-workload [id=0084462a390d469aad0b429e207044f1]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:15:29.984094 10302 tablet_service.cc:1511] Processing CreateTablet for tablet beb41e866f444744a06e30def55c3190 (DEFAULT_TABLE table=test-workload [id=0084462a390d469aad0b429e207044f1]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:15:29.985512 10301 tablet_service.cc:1511] Processing CreateTablet for tablet 7fa6a2759a7d43a28bc364bdd2593069 (DEFAULT_TABLE table=test-workload [id=0084462a390d469aad0b429e207044f1]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:15:29.987906 10502 tablet_bootstrap.cc:492] T f7a41f1857af412bb47303bd9452b65f P 28d1d6c02b384aeaa740460051ea351b: Bootstrap starting.
I20260709 13:15:29.989480 10501 tablet_bootstrap.cc:492] T 9db04314c2c849898b82d951374e2878 P dd07659d40f84c2aafeaabddc392aeab: Bootstrap starting.
I20260709 13:15:29.990250 10448 tablet_service.cc:1511] Processing CreateTablet for tablet 135e8982a8ea4fa198de62d0e60b459c (DEFAULT_TABLE table=test-workload [id=0084462a390d469aad0b429e207044f1]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:15:29.991688 10448 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 135e8982a8ea4fa198de62d0e60b459c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:30.000468 10297 tablet_service.cc:1511] Processing CreateTablet for tablet 135e8982a8ea4fa198de62d0e60b459c (DEFAULT_TABLE table=test-workload [id=0084462a390d469aad0b429e207044f1]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:15:30.002723 10298 tablet_service.cc:1511] Processing CreateTablet for tablet f7a41f1857af412bb47303bd9452b65f (DEFAULT_TABLE table=test-workload [id=0084462a390d469aad0b429e207044f1]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:15:30.003746 10299 tablet_service.cc:1511] Processing CreateTablet for tablet 9db04314c2c849898b82d951374e2878 (DEFAULT_TABLE table=test-workload [id=0084462a390d469aad0b429e207044f1]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:15:30.004927 10377 tablet_service.cc:1511] Processing CreateTablet for tablet 94c17c5750834fa999137c4d6cce4cf2 (DEFAULT_TABLE table=test-workload [id=0084462a390d469aad0b429e207044f1]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:15:30.006131 10377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94c17c5750834fa999137c4d6cce4cf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:29.981341 10300 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09786fc30e8048d9b7928fe7bf969df1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:30.007459 10296 tablet_service.cc:1511] Processing CreateTablet for tablet 94c17c5750834fa999137c4d6cce4cf2 (DEFAULT_TABLE table=test-workload [id=0084462a390d469aad0b429e207044f1]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:15:30.009380 10299 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9db04314c2c849898b82d951374e2878. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:30.010325 10296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94c17c5750834fa999137c4d6cce4cf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:30.011474 10298 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7a41f1857af412bb47303bd9452b65f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:30.015133 10301 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fa6a2759a7d43a28bc364bdd2593069. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:30.019148 10297 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 135e8982a8ea4fa198de62d0e60b459c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:30.024097 10302 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet beb41e866f444744a06e30def55c3190. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:30.025949 10501 tablet_bootstrap.cc:654] T 9db04314c2c849898b82d951374e2878 P dd07659d40f84c2aafeaabddc392aeab: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:30.029028 10502 tablet_bootstrap.cc:654] T f7a41f1857af412bb47303bd9452b65f P 28d1d6c02b384aeaa740460051ea351b: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:30.050355 10501 tablet_bootstrap.cc:492] T 9db04314c2c849898b82d951374e2878 P dd07659d40f84c2aafeaabddc392aeab: No bootstrap required, opened a new log
I20260709 13:15:30.050935 10501 ts_tablet_manager.cc:1403] T 9db04314c2c849898b82d951374e2878 P dd07659d40f84c2aafeaabddc392aeab: Time spent bootstrapping tablet: real 0.062s	user 0.024s	sys 0.007s
I20260709 13:15:30.053988 10501 raft_consensus.cc:359] T 9db04314c2c849898b82d951374e2878 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.054199 10451 tablet_service.cc:1511] Processing CreateTablet for tablet 94c17c5750834fa999137c4d6cce4cf2 (DEFAULT_TABLE table=test-workload [id=0084462a390d469aad0b429e207044f1]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:15:30.054857 10501 raft_consensus.cc:385] T 9db04314c2c849898b82d951374e2878 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:30.055228 10501 raft_consensus.cc:740] T 9db04314c2c849898b82d951374e2878 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd07659d40f84c2aafeaabddc392aeab, State: Initialized, Role: FOLLOWER
I20260709 13:15:30.055635 10451 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94c17c5750834fa999137c4d6cce4cf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:30.055938 10501 consensus_queue.cc:260] T 9db04314c2c849898b82d951374e2878 P dd07659d40f84c2aafeaabddc392aeab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.070497 10299 tablet_service.cc:1511] Processing CreateTablet for tablet c5b1bc6ab2bb4893a25a192936d09209 (DEFAULT_TABLE table=test-workload [id=0084462a390d469aad0b429e207044f1]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:15:30.082319 10299 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5b1bc6ab2bb4893a25a192936d09209. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:30.093257 10506 tablet_bootstrap.cc:492] T 94c17c5750834fa999137c4d6cce4cf2 P f090b64140fe4e55942c16e20a79b36b: Bootstrap starting.
I20260709 13:15:30.097110 10448 tablet_service.cc:1511] Processing CreateTablet for tablet c5b1bc6ab2bb4893a25a192936d09209 (DEFAULT_TABLE table=test-workload [id=0084462a390d469aad0b429e207044f1]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:15:30.098369 10502 tablet_bootstrap.cc:492] T f7a41f1857af412bb47303bd9452b65f P 28d1d6c02b384aeaa740460051ea351b: No bootstrap required, opened a new log
I20260709 13:15:30.098441 10448 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5b1bc6ab2bb4893a25a192936d09209. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:30.098958 10502 ts_tablet_manager.cc:1403] T f7a41f1857af412bb47303bd9452b65f P 28d1d6c02b384aeaa740460051ea351b: Time spent bootstrapping tablet: real 0.111s	user 0.006s	sys 0.011s
I20260709 13:15:30.095184 10377 tablet_service.cc:1511] Processing CreateTablet for tablet c5b1bc6ab2bb4893a25a192936d09209 (DEFAULT_TABLE table=test-workload [id=0084462a390d469aad0b429e207044f1]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:15:30.100950 10377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5b1bc6ab2bb4893a25a192936d09209. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:30.101931 10502 raft_consensus.cc:359] T f7a41f1857af412bb47303bd9452b65f P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.103076 10486 heartbeater.cc:499] Master 127.9.116.190:39829 was elected leader, sending a full tablet report...
I20260709 13:15:30.107867 10372 tablet_service.cc:1511] Processing CreateTablet for tablet acb9159770724070b93f9bc026ffee90 (DEFAULT_TABLE table=test-workload [id=0084462a390d469aad0b429e207044f1]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:15:30.109162 10372 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acb9159770724070b93f9bc026ffee90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:30.102691 10502 raft_consensus.cc:385] T f7a41f1857af412bb47303bd9452b65f P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:30.113451 10502 raft_consensus.cc:740] T f7a41f1857af412bb47303bd9452b65f P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28d1d6c02b384aeaa740460051ea351b, State: Initialized, Role: FOLLOWER
I20260709 13:15:30.114897 10506 tablet_bootstrap.cc:654] T 94c17c5750834fa999137c4d6cce4cf2 P f090b64140fe4e55942c16e20a79b36b: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:30.115923 10300 tablet_service.cc:1511] Processing CreateTablet for tablet acb9159770724070b93f9bc026ffee90 (DEFAULT_TABLE table=test-workload [id=0084462a390d469aad0b429e207044f1]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:15:30.119102 10300 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acb9159770724070b93f9bc026ffee90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:30.118607 10502 consensus_queue.cc:260] T f7a41f1857af412bb47303bd9452b65f P 28d1d6c02b384aeaa740460051ea351b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.117998 10501 ts_tablet_manager.cc:1434] T 9db04314c2c849898b82d951374e2878 P dd07659d40f84c2aafeaabddc392aeab: Time spent starting tablet: real 0.067s	user 0.023s	sys 0.019s
I20260709 13:15:30.124305 10501 tablet_bootstrap.cc:492] T 7fa6a2759a7d43a28bc364bdd2593069 P dd07659d40f84c2aafeaabddc392aeab: Bootstrap starting.
I20260709 13:15:30.131719 10449 tablet_service.cc:1511] Processing CreateTablet for tablet acb9159770724070b93f9bc026ffee90 (DEFAULT_TABLE table=test-workload [id=0084462a390d469aad0b429e207044f1]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:15:30.134891 10449 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acb9159770724070b93f9bc026ffee90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:30.148806 10302 tablet_service.cc:1511] Processing CreateTablet for tablet b6a956a56be14d83b5f0a8db624c1609 (DEFAULT_TABLE table=test-workload [id=0084462a390d469aad0b429e207044f1]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:15:30.149243 10412 heartbeater.cc:499] Master 127.9.116.190:39829 was elected leader, sending a full tablet report...
I20260709 13:15:30.154361 10502 ts_tablet_manager.cc:1434] T f7a41f1857af412bb47303bd9452b65f P 28d1d6c02b384aeaa740460051ea351b: Time spent starting tablet: real 0.055s	user 0.016s	sys 0.009s
I20260709 13:15:30.155380 10302 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6a956a56be14d83b5f0a8db624c1609. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:30.155515 10502 tablet_bootstrap.cc:492] T 7fa6a2759a7d43a28bc364bdd2593069 P 28d1d6c02b384aeaa740460051ea351b: Bootstrap starting.
I20260709 13:15:30.159034 10373 tablet_service.cc:1511] Processing CreateTablet for tablet b6a956a56be14d83b5f0a8db624c1609 (DEFAULT_TABLE table=test-workload [id=0084462a390d469aad0b429e207044f1]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:15:30.160593 10373 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6a956a56be14d83b5f0a8db624c1609. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:30.164609 10502 tablet_bootstrap.cc:654] T 7fa6a2759a7d43a28bc364bdd2593069 P 28d1d6c02b384aeaa740460051ea351b: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:30.167430 10506 tablet_bootstrap.cc:492] T 94c17c5750834fa999137c4d6cce4cf2 P f090b64140fe4e55942c16e20a79b36b: No bootstrap required, opened a new log
I20260709 13:15:30.167945 10506 ts_tablet_manager.cc:1403] T 94c17c5750834fa999137c4d6cce4cf2 P f090b64140fe4e55942c16e20a79b36b: Time spent bootstrapping tablet: real 0.075s	user 0.019s	sys 0.018s
I20260709 13:15:30.170755 10506 raft_consensus.cc:359] T 94c17c5750834fa999137c4d6cce4cf2 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.171502 10506 raft_consensus.cc:385] T 94c17c5750834fa999137c4d6cce4cf2 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:30.171792 10506 raft_consensus.cc:740] T 94c17c5750834fa999137c4d6cce4cf2 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f090b64140fe4e55942c16e20a79b36b, State: Initialized, Role: FOLLOWER
I20260709 13:15:30.173334 10451 tablet_service.cc:1511] Processing CreateTablet for tablet b6a956a56be14d83b5f0a8db624c1609 (DEFAULT_TABLE table=test-workload [id=0084462a390d469aad0b429e207044f1]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:15:30.174576 10451 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6a956a56be14d83b5f0a8db624c1609. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:30.175423 10501 tablet_bootstrap.cc:654] T 7fa6a2759a7d43a28bc364bdd2593069 P dd07659d40f84c2aafeaabddc392aeab: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:30.185739 10506 consensus_queue.cc:260] T 94c17c5750834fa999137c4d6cce4cf2 P f090b64140fe4e55942c16e20a79b36b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.189237 10502 tablet_bootstrap.cc:492] T 7fa6a2759a7d43a28bc364bdd2593069 P 28d1d6c02b384aeaa740460051ea351b: No bootstrap required, opened a new log
I20260709 13:15:30.189616 10502 ts_tablet_manager.cc:1403] T 7fa6a2759a7d43a28bc364bdd2593069 P 28d1d6c02b384aeaa740460051ea351b: Time spent bootstrapping tablet: real 0.034s	user 0.009s	sys 0.006s
I20260709 13:15:30.192078 10502 raft_consensus.cc:359] T 7fa6a2759a7d43a28bc364bdd2593069 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.196527 10510 raft_consensus.cc:493] T 94c17c5750834fa999137c4d6cce4cf2 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:30.197204 10510 raft_consensus.cc:515] T 94c17c5750834fa999137c4d6cce4cf2 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.200609 10510 leader_election.cc:290] T 94c17c5750834fa999137c4d6cce4cf2 P f090b64140fe4e55942c16e20a79b36b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 28d1d6c02b384aeaa740460051ea351b (127.9.116.130:36353), dd07659d40f84c2aafeaabddc392aeab (127.9.116.131:42413)
I20260709 13:15:30.195291 10502 raft_consensus.cc:385] T 7fa6a2759a7d43a28bc364bdd2593069 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:30.204070 10337 heartbeater.cc:499] Master 127.9.116.190:39829 was elected leader, sending a full tablet report...
I20260709 13:15:30.205998 10506 ts_tablet_manager.cc:1434] T 94c17c5750834fa999137c4d6cce4cf2 P f090b64140fe4e55942c16e20a79b36b: Time spent starting tablet: real 0.038s	user 0.007s	sys 0.009s
I20260709 13:15:30.207173 10506 tablet_bootstrap.cc:492] T 9db04314c2c849898b82d951374e2878 P f090b64140fe4e55942c16e20a79b36b: Bootstrap starting.
I20260709 13:15:30.204746 10502 raft_consensus.cc:740] T 7fa6a2759a7d43a28bc364bdd2593069 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28d1d6c02b384aeaa740460051ea351b, State: Initialized, Role: FOLLOWER
I20260709 13:15:30.212173 10502 consensus_queue.cc:260] T 7fa6a2759a7d43a28bc364bdd2593069 P 28d1d6c02b384aeaa740460051ea351b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.213928 10501 tablet_bootstrap.cc:492] T 7fa6a2759a7d43a28bc364bdd2593069 P dd07659d40f84c2aafeaabddc392aeab: No bootstrap required, opened a new log
I20260709 13:15:30.214432 10501 ts_tablet_manager.cc:1403] T 7fa6a2759a7d43a28bc364bdd2593069 P dd07659d40f84c2aafeaabddc392aeab: Time spent bootstrapping tablet: real 0.090s	user 0.009s	sys 0.010s
I20260709 13:15:30.215866 10506 tablet_bootstrap.cc:654] T 9db04314c2c849898b82d951374e2878 P f090b64140fe4e55942c16e20a79b36b: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:30.216941 10502 ts_tablet_manager.cc:1434] T 7fa6a2759a7d43a28bc364bdd2593069 P 28d1d6c02b384aeaa740460051ea351b: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.002s
I20260709 13:15:30.217957 10502 tablet_bootstrap.cc:492] T 9db04314c2c849898b82d951374e2878 P 28d1d6c02b384aeaa740460051ea351b: Bootstrap starting.
I20260709 13:15:30.224336 10502 tablet_bootstrap.cc:654] T 9db04314c2c849898b82d951374e2878 P 28d1d6c02b384aeaa740460051ea351b: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:30.231559 10506 tablet_bootstrap.cc:492] T 9db04314c2c849898b82d951374e2878 P f090b64140fe4e55942c16e20a79b36b: No bootstrap required, opened a new log
I20260709 13:15:30.232218 10506 ts_tablet_manager.cc:1403] T 9db04314c2c849898b82d951374e2878 P f090b64140fe4e55942c16e20a79b36b: Time spent bootstrapping tablet: real 0.025s	user 0.017s	sys 0.005s
I20260709 13:15:30.235102 10506 raft_consensus.cc:359] T 9db04314c2c849898b82d951374e2878 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.235620 10502 tablet_bootstrap.cc:492] T 9db04314c2c849898b82d951374e2878 P 28d1d6c02b384aeaa740460051ea351b: No bootstrap required, opened a new log
I20260709 13:15:30.235883 10506 raft_consensus.cc:385] T 9db04314c2c849898b82d951374e2878 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:30.237512 10502 ts_tablet_manager.cc:1403] T 9db04314c2c849898b82d951374e2878 P 28d1d6c02b384aeaa740460051ea351b: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.000s
I20260709 13:15:30.238466 10501 raft_consensus.cc:359] T 7fa6a2759a7d43a28bc364bdd2593069 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.239244 10501 raft_consensus.cc:385] T 7fa6a2759a7d43a28bc364bdd2593069 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:30.239565 10501 raft_consensus.cc:740] T 7fa6a2759a7d43a28bc364bdd2593069 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd07659d40f84c2aafeaabddc392aeab, State: Initialized, Role: FOLLOWER
I20260709 13:15:30.240758 10502 raft_consensus.cc:359] T 9db04314c2c849898b82d951374e2878 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.241573 10502 raft_consensus.cc:385] T 9db04314c2c849898b82d951374e2878 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:30.241961 10502 raft_consensus.cc:740] T 9db04314c2c849898b82d951374e2878 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28d1d6c02b384aeaa740460051ea351b, State: Initialized, Role: FOLLOWER
I20260709 13:15:30.237439 10506 raft_consensus.cc:740] T 9db04314c2c849898b82d951374e2878 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f090b64140fe4e55942c16e20a79b36b, State: Initialized, Role: FOLLOWER
I20260709 13:15:30.243476 10501 consensus_queue.cc:260] T 7fa6a2759a7d43a28bc364bdd2593069 P dd07659d40f84c2aafeaabddc392aeab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.243372 10506 consensus_queue.cc:260] T 9db04314c2c849898b82d951374e2878 P f090b64140fe4e55942c16e20a79b36b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.243759 10387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c17c5750834fa999137c4d6cce4cf2" candidate_uuid: "f090b64140fe4e55942c16e20a79b36b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28d1d6c02b384aeaa740460051ea351b" is_pre_election: true
I20260709 13:15:30.245709 10461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c17c5750834fa999137c4d6cce4cf2" candidate_uuid: "f090b64140fe4e55942c16e20a79b36b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd07659d40f84c2aafeaabddc392aeab" is_pre_election: true
W20260709 13:15:30.245843 10278 leader_election.cc:343] T 94c17c5750834fa999137c4d6cce4cf2 P f090b64140fe4e55942c16e20a79b36b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28d1d6c02b384aeaa740460051ea351b (127.9.116.130:36353): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:30.246279 10502 consensus_queue.cc:260] T 9db04314c2c849898b82d951374e2878 P 28d1d6c02b384aeaa740460051ea351b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.245813 10506 ts_tablet_manager.cc:1434] T 9db04314c2c849898b82d951374e2878 P f090b64140fe4e55942c16e20a79b36b: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
W20260709 13:15:30.247239 10278 leader_election.cc:343] T 94c17c5750834fa999137c4d6cce4cf2 P f090b64140fe4e55942c16e20a79b36b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dd07659d40f84c2aafeaabddc392aeab (127.9.116.131:42413): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:30.247691 10278 leader_election.cc:304] T 94c17c5750834fa999137c4d6cce4cf2 P f090b64140fe4e55942c16e20a79b36b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f090b64140fe4e55942c16e20a79b36b; no voters: 28d1d6c02b384aeaa740460051ea351b, dd07659d40f84c2aafeaabddc392aeab
I20260709 13:15:30.248382 10501 ts_tablet_manager.cc:1434] T 7fa6a2759a7d43a28bc364bdd2593069 P dd07659d40f84c2aafeaabddc392aeab: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.000s
I20260709 13:15:30.248594 10512 raft_consensus.cc:2749] T 94c17c5750834fa999137c4d6cce4cf2 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:30.249361 10501 tablet_bootstrap.cc:492] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab: Bootstrap starting.
I20260709 13:15:30.249742 10506 tablet_bootstrap.cc:492] T f7a41f1857af412bb47303bd9452b65f P f090b64140fe4e55942c16e20a79b36b: Bootstrap starting.
I20260709 13:15:30.251214 10502 ts_tablet_manager.cc:1434] T 9db04314c2c849898b82d951374e2878 P 28d1d6c02b384aeaa740460051ea351b: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:15:30.252372 10502 tablet_bootstrap.cc:492] T beb41e866f444744a06e30def55c3190 P 28d1d6c02b384aeaa740460051ea351b: Bootstrap starting.
I20260709 13:15:30.255265 10501 tablet_bootstrap.cc:654] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:30.256752 10506 tablet_bootstrap.cc:654] T f7a41f1857af412bb47303bd9452b65f P f090b64140fe4e55942c16e20a79b36b: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:30.263437 10502 tablet_bootstrap.cc:654] T beb41e866f444744a06e30def55c3190 P 28d1d6c02b384aeaa740460051ea351b: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:30.270359 10501 tablet_bootstrap.cc:492] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab: No bootstrap required, opened a new log
I20260709 13:15:30.270928 10501 ts_tablet_manager.cc:1403] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab: Time spent bootstrapping tablet: real 0.022s	user 0.009s	sys 0.003s
I20260709 13:15:30.272425 10506 tablet_bootstrap.cc:492] T f7a41f1857af412bb47303bd9452b65f P f090b64140fe4e55942c16e20a79b36b: No bootstrap required, opened a new log
I20260709 13:15:30.272893 10506 ts_tablet_manager.cc:1403] T f7a41f1857af412bb47303bd9452b65f P f090b64140fe4e55942c16e20a79b36b: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.007s
I20260709 13:15:30.273846 10501 raft_consensus.cc:359] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.274551 10501 raft_consensus.cc:385] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:30.274906 10501 raft_consensus.cc:740] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd07659d40f84c2aafeaabddc392aeab, State: Initialized, Role: FOLLOWER
I20260709 13:15:30.275749 10506 raft_consensus.cc:359] T f7a41f1857af412bb47303bd9452b65f P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.275672 10501 consensus_queue.cc:260] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.276473 10506 raft_consensus.cc:385] T f7a41f1857af412bb47303bd9452b65f P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:30.276785 10506 raft_consensus.cc:740] T f7a41f1857af412bb47303bd9452b65f P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f090b64140fe4e55942c16e20a79b36b, State: Initialized, Role: FOLLOWER
I20260709 13:15:30.277496 10506 consensus_queue.cc:260] T f7a41f1857af412bb47303bd9452b65f P f090b64140fe4e55942c16e20a79b36b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.278030 10501 ts_tablet_manager.cc:1434] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab: Time spent starting tablet: real 0.007s	user 0.000s	sys 0.005s
I20260709 13:15:30.279060 10501 tablet_bootstrap.cc:492] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab: Bootstrap starting.
I20260709 13:15:30.280081 10506 ts_tablet_manager.cc:1434] T f7a41f1857af412bb47303bd9452b65f P f090b64140fe4e55942c16e20a79b36b: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:15:30.281035 10506 tablet_bootstrap.cc:492] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b: Bootstrap starting.
I20260709 13:15:30.284601 10502 tablet_bootstrap.cc:492] T beb41e866f444744a06e30def55c3190 P 28d1d6c02b384aeaa740460051ea351b: No bootstrap required, opened a new log
I20260709 13:15:30.285076 10502 ts_tablet_manager.cc:1403] T beb41e866f444744a06e30def55c3190 P 28d1d6c02b384aeaa740460051ea351b: Time spent bootstrapping tablet: real 0.033s	user 0.011s	sys 0.002s
I20260709 13:15:30.285411 10501 tablet_bootstrap.cc:654] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:30.287981 10502 raft_consensus.cc:359] T beb41e866f444744a06e30def55c3190 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } } peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } }
I20260709 13:15:30.288779 10502 raft_consensus.cc:385] T beb41e866f444744a06e30def55c3190 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:30.289084 10502 raft_consensus.cc:740] T beb41e866f444744a06e30def55c3190 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28d1d6c02b384aeaa740460051ea351b, State: Initialized, Role: FOLLOWER
I20260709 13:15:30.289892 10502 consensus_queue.cc:260] T beb41e866f444744a06e30def55c3190 P 28d1d6c02b384aeaa740460051ea351b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } } peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } }
I20260709 13:15:30.292901 10501 tablet_bootstrap.cc:492] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab: No bootstrap required, opened a new log
I20260709 13:15:30.293444 10501 ts_tablet_manager.cc:1403] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab: Time spent bootstrapping tablet: real 0.015s	user 0.010s	sys 0.000s
I20260709 13:15:30.296244 10501 raft_consensus.cc:359] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.296808 10501 raft_consensus.cc:385] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:30.297084 10501 raft_consensus.cc:740] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd07659d40f84c2aafeaabddc392aeab, State: Initialized, Role: FOLLOWER
I20260709 13:15:30.297175 10506 tablet_bootstrap.cc:654] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:30.297832 10501 consensus_queue.cc:260] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.299459 10502 ts_tablet_manager.cc:1434] T beb41e866f444744a06e30def55c3190 P 28d1d6c02b384aeaa740460051ea351b: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.000s
I20260709 13:15:30.299926 10501 ts_tablet_manager.cc:1434] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:15:30.301010 10501 tablet_bootstrap.cc:492] T 135e8982a8ea4fa198de62d0e60b459c P dd07659d40f84c2aafeaabddc392aeab: Bootstrap starting.
I20260709 13:15:30.307673 10501 tablet_bootstrap.cc:654] T 135e8982a8ea4fa198de62d0e60b459c P dd07659d40f84c2aafeaabddc392aeab: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:30.308288 10502 tablet_bootstrap.cc:492] T 09786fc30e8048d9b7928fe7bf969df1 P 28d1d6c02b384aeaa740460051ea351b: Bootstrap starting.
I20260709 13:15:30.308496 10506 tablet_bootstrap.cc:492] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b: No bootstrap required, opened a new log
I20260709 13:15:30.308949 10506 ts_tablet_manager.cc:1403] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b: Time spent bootstrapping tablet: real 0.028s	user 0.010s	sys 0.005s
I20260709 13:15:30.311900 10506 raft_consensus.cc:359] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.312657 10506 raft_consensus.cc:385] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:30.312963 10506 raft_consensus.cc:740] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f090b64140fe4e55942c16e20a79b36b, State: Initialized, Role: FOLLOWER
I20260709 13:15:30.313858 10506 consensus_queue.cc:260] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.315238 10502 tablet_bootstrap.cc:654] T 09786fc30e8048d9b7928fe7bf969df1 P 28d1d6c02b384aeaa740460051ea351b: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:30.316242 10506 ts_tablet_manager.cc:1434] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b: Time spent starting tablet: real 0.007s	user 0.009s	sys 0.000s
I20260709 13:15:30.317236 10506 tablet_bootstrap.cc:492] T 09786fc30e8048d9b7928fe7bf969df1 P f090b64140fe4e55942c16e20a79b36b: Bootstrap starting.
I20260709 13:15:30.321069 10501 tablet_bootstrap.cc:492] T 135e8982a8ea4fa198de62d0e60b459c P dd07659d40f84c2aafeaabddc392aeab: No bootstrap required, opened a new log
I20260709 13:15:30.321589 10501 ts_tablet_manager.cc:1403] T 135e8982a8ea4fa198de62d0e60b459c P dd07659d40f84c2aafeaabddc392aeab: Time spent bootstrapping tablet: real 0.021s	user 0.009s	sys 0.009s
I20260709 13:15:30.323491 10506 tablet_bootstrap.cc:654] T 09786fc30e8048d9b7928fe7bf969df1 P f090b64140fe4e55942c16e20a79b36b: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:30.324573 10501 raft_consensus.cc:359] T 135e8982a8ea4fa198de62d0e60b459c P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.325349 10501 raft_consensus.cc:385] T 135e8982a8ea4fa198de62d0e60b459c P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:30.325649 10501 raft_consensus.cc:740] T 135e8982a8ea4fa198de62d0e60b459c P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd07659d40f84c2aafeaabddc392aeab, State: Initialized, Role: FOLLOWER
I20260709 13:15:30.326159 10502 tablet_bootstrap.cc:492] T 09786fc30e8048d9b7928fe7bf969df1 P 28d1d6c02b384aeaa740460051ea351b: No bootstrap required, opened a new log
I20260709 13:15:30.326602 10502 ts_tablet_manager.cc:1403] T 09786fc30e8048d9b7928fe7bf969df1 P 28d1d6c02b384aeaa740460051ea351b: Time spent bootstrapping tablet: real 0.019s	user 0.011s	sys 0.001s
I20260709 13:15:30.326442 10501 consensus_queue.cc:260] T 135e8982a8ea4fa198de62d0e60b459c P dd07659d40f84c2aafeaabddc392aeab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.328722 10501 ts_tablet_manager.cc:1434] T 135e8982a8ea4fa198de62d0e60b459c P dd07659d40f84c2aafeaabddc392aeab: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:15:30.329715 10501 tablet_bootstrap.cc:492] T beb41e866f444744a06e30def55c3190 P dd07659d40f84c2aafeaabddc392aeab: Bootstrap starting.
I20260709 13:15:30.329527 10502 raft_consensus.cc:359] T 09786fc30e8048d9b7928fe7bf969df1 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.330283 10502 raft_consensus.cc:385] T 09786fc30e8048d9b7928fe7bf969df1 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:30.330581 10502 raft_consensus.cc:740] T 09786fc30e8048d9b7928fe7bf969df1 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28d1d6c02b384aeaa740460051ea351b, State: Initialized, Role: FOLLOWER
I20260709 13:15:30.331364 10502 consensus_queue.cc:260] T 09786fc30e8048d9b7928fe7bf969df1 P 28d1d6c02b384aeaa740460051ea351b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.333626 10502 ts_tablet_manager.cc:1434] T 09786fc30e8048d9b7928fe7bf969df1 P 28d1d6c02b384aeaa740460051ea351b: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.002s
I20260709 13:15:30.334600 10502 tablet_bootstrap.cc:492] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b: Bootstrap starting.
I20260709 13:15:30.336491 10501 tablet_bootstrap.cc:654] T beb41e866f444744a06e30def55c3190 P dd07659d40f84c2aafeaabddc392aeab: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:30.346168 10506 tablet_bootstrap.cc:492] T 09786fc30e8048d9b7928fe7bf969df1 P f090b64140fe4e55942c16e20a79b36b: No bootstrap required, opened a new log
I20260709 13:15:30.346171 10501 tablet_bootstrap.cc:492] T beb41e866f444744a06e30def55c3190 P dd07659d40f84c2aafeaabddc392aeab: No bootstrap required, opened a new log
I20260709 13:15:30.346814 10501 ts_tablet_manager.cc:1403] T beb41e866f444744a06e30def55c3190 P dd07659d40f84c2aafeaabddc392aeab: Time spent bootstrapping tablet: real 0.017s	user 0.009s	sys 0.007s
I20260709 13:15:30.346810 10506 ts_tablet_manager.cc:1403] T 09786fc30e8048d9b7928fe7bf969df1 P f090b64140fe4e55942c16e20a79b36b: Time spent bootstrapping tablet: real 0.030s	user 0.005s	sys 0.009s
I20260709 13:15:30.348244 10502 tablet_bootstrap.cc:654] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:30.349749 10506 raft_consensus.cc:359] T 09786fc30e8048d9b7928fe7bf969df1 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.349888 10501 raft_consensus.cc:359] T beb41e866f444744a06e30def55c3190 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } } peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } }
I20260709 13:15:30.350564 10506 raft_consensus.cc:385] T 09786fc30e8048d9b7928fe7bf969df1 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:30.350601 10501 raft_consensus.cc:385] T beb41e866f444744a06e30def55c3190 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:30.351029 10506 raft_consensus.cc:740] T 09786fc30e8048d9b7928fe7bf969df1 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f090b64140fe4e55942c16e20a79b36b, State: Initialized, Role: FOLLOWER
I20260709 13:15:30.351069 10501 raft_consensus.cc:740] T beb41e866f444744a06e30def55c3190 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd07659d40f84c2aafeaabddc392aeab, State: Initialized, Role: FOLLOWER
I20260709 13:15:30.351890 10501 consensus_queue.cc:260] T beb41e866f444744a06e30def55c3190 P dd07659d40f84c2aafeaabddc392aeab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } } peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } }
I20260709 13:15:30.351890 10506 consensus_queue.cc:260] T 09786fc30e8048d9b7928fe7bf969df1 P f090b64140fe4e55942c16e20a79b36b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.354254 10501 ts_tablet_manager.cc:1434] T beb41e866f444744a06e30def55c3190 P dd07659d40f84c2aafeaabddc392aeab: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:15:30.354547 10502 tablet_bootstrap.cc:492] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b: No bootstrap required, opened a new log
I20260709 13:15:30.355021 10502 ts_tablet_manager.cc:1403] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b: Time spent bootstrapping tablet: real 0.021s	user 0.002s	sys 0.011s
I20260709 13:15:30.355257 10501 tablet_bootstrap.cc:492] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab: Bootstrap starting.
I20260709 13:15:30.356321 10504 raft_consensus.cc:493] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:30.356786 10504 raft_consensus.cc:515] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.358299 10502 raft_consensus.cc:359] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.359063 10502 raft_consensus.cc:385] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:30.359359 10502 raft_consensus.cc:740] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28d1d6c02b384aeaa740460051ea351b, State: Initialized, Role: FOLLOWER
I20260709 13:15:30.360129 10502 consensus_queue.cc:260] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.362612 10506 ts_tablet_manager.cc:1434] T 09786fc30e8048d9b7928fe7bf969df1 P f090b64140fe4e55942c16e20a79b36b: Time spent starting tablet: real 0.015s	user 0.002s	sys 0.004s
I20260709 13:15:30.363689 10506 tablet_bootstrap.cc:492] T 7fa6a2759a7d43a28bc364bdd2593069 P f090b64140fe4e55942c16e20a79b36b: Bootstrap starting.
I20260709 13:15:30.367619 10504 leader_election.cc:290] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f090b64140fe4e55942c16e20a79b36b (127.9.116.129:42273), 28d1d6c02b384aeaa740460051ea351b (127.9.116.130:36353)
I20260709 13:15:30.369911 10506 tablet_bootstrap.cc:654] T 7fa6a2759a7d43a28bc364bdd2593069 P f090b64140fe4e55942c16e20a79b36b: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:30.369863 10502 ts_tablet_manager.cc:1434] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:15:30.376384 10501 tablet_bootstrap.cc:654] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:30.381579 10312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a41f1857af412bb47303bd9452b65f" candidate_uuid: "dd07659d40f84c2aafeaabddc392aeab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f090b64140fe4e55942c16e20a79b36b" is_pre_election: true
I20260709 13:15:30.382436 10312 raft_consensus.cc:2468] T f7a41f1857af412bb47303bd9452b65f P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd07659d40f84c2aafeaabddc392aeab in term 0.
I20260709 13:15:30.383744 10425 leader_election.cc:304] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dd07659d40f84c2aafeaabddc392aeab, f090b64140fe4e55942c16e20a79b36b; no voters: 
I20260709 13:15:30.384635 10504 raft_consensus.cc:2804] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:30.385016 10504 raft_consensus.cc:493] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.385356 10504 raft_consensus.cc:3060] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:30.385536 10506 tablet_bootstrap.cc:492] T 7fa6a2759a7d43a28bc364bdd2593069 P f090b64140fe4e55942c16e20a79b36b: No bootstrap required, opened a new log
I20260709 13:15:30.386307 10506 ts_tablet_manager.cc:1403] T 7fa6a2759a7d43a28bc364bdd2593069 P f090b64140fe4e55942c16e20a79b36b: Time spent bootstrapping tablet: real 0.023s	user 0.008s	sys 0.013s
I20260709 13:15:30.389067 10501 tablet_bootstrap.cc:492] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab: No bootstrap required, opened a new log
I20260709 13:15:30.389603 10501 ts_tablet_manager.cc:1403] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab: Time spent bootstrapping tablet: real 0.035s	user 0.016s	sys 0.000s
I20260709 13:15:30.389624 10506 raft_consensus.cc:359] T 7fa6a2759a7d43a28bc364bdd2593069 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.390106 10387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a41f1857af412bb47303bd9452b65f" candidate_uuid: "dd07659d40f84c2aafeaabddc392aeab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28d1d6c02b384aeaa740460051ea351b" is_pre_election: true
I20260709 13:15:30.390619 10506 raft_consensus.cc:385] T 7fa6a2759a7d43a28bc364bdd2593069 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:30.391055 10387 raft_consensus.cc:2468] T f7a41f1857af412bb47303bd9452b65f P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd07659d40f84c2aafeaabddc392aeab in term 0.
I20260709 13:15:30.391116 10506 raft_consensus.cc:740] T 7fa6a2759a7d43a28bc364bdd2593069 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f090b64140fe4e55942c16e20a79b36b, State: Initialized, Role: FOLLOWER
I20260709 13:15:30.392131 10506 consensus_queue.cc:260] T 7fa6a2759a7d43a28bc364bdd2593069 P f090b64140fe4e55942c16e20a79b36b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.392294 10504 raft_consensus.cc:515] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.394296 10502 tablet_bootstrap.cc:492] T 94c17c5750834fa999137c4d6cce4cf2 P 28d1d6c02b384aeaa740460051ea351b: Bootstrap starting.
I20260709 13:15:30.394587 10506 ts_tablet_manager.cc:1434] T 7fa6a2759a7d43a28bc364bdd2593069 P f090b64140fe4e55942c16e20a79b36b: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.006s
I20260709 13:15:30.392897 10501 raft_consensus.cc:359] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.395483 10501 raft_consensus.cc:385] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:30.395712 10506 tablet_bootstrap.cc:492] T beb41e866f444744a06e30def55c3190 P f090b64140fe4e55942c16e20a79b36b: Bootstrap starting.
I20260709 13:15:30.395795 10501 raft_consensus.cc:740] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd07659d40f84c2aafeaabddc392aeab, State: Initialized, Role: FOLLOWER
I20260709 13:15:30.396500 10312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a41f1857af412bb47303bd9452b65f" candidate_uuid: "dd07659d40f84c2aafeaabddc392aeab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f090b64140fe4e55942c16e20a79b36b"
I20260709 13:15:30.397094 10312 raft_consensus.cc:3060] T f7a41f1857af412bb47303bd9452b65f P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:30.396770 10501 consensus_queue.cc:260] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.401067 10502 tablet_bootstrap.cc:654] T 94c17c5750834fa999137c4d6cce4cf2 P 28d1d6c02b384aeaa740460051ea351b: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:30.404475 10506 tablet_bootstrap.cc:654] T beb41e866f444744a06e30def55c3190 P f090b64140fe4e55942c16e20a79b36b: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:30.409109 10312 raft_consensus.cc:2468] T f7a41f1857af412bb47303bd9452b65f P f090b64140fe4e55942c16e20a79b36b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd07659d40f84c2aafeaabddc392aeab in term 1.
I20260709 13:15:30.410688 10425 leader_election.cc:304] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dd07659d40f84c2aafeaabddc392aeab, f090b64140fe4e55942c16e20a79b36b; no voters: 
I20260709 13:15:30.412998 10502 tablet_bootstrap.cc:492] T 94c17c5750834fa999137c4d6cce4cf2 P 28d1d6c02b384aeaa740460051ea351b: No bootstrap required, opened a new log
I20260709 13:15:30.413620 10502 ts_tablet_manager.cc:1403] T 94c17c5750834fa999137c4d6cce4cf2 P 28d1d6c02b384aeaa740460051ea351b: Time spent bootstrapping tablet: real 0.020s	user 0.016s	sys 0.000s
I20260709 13:15:30.416740 10502 raft_consensus.cc:359] T 94c17c5750834fa999137c4d6cce4cf2 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.417449 10502 raft_consensus.cc:385] T 94c17c5750834fa999137c4d6cce4cf2 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:30.417755 10502 raft_consensus.cc:740] T 94c17c5750834fa999137c4d6cce4cf2 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28d1d6c02b384aeaa740460051ea351b, State: Initialized, Role: FOLLOWER
I20260709 13:15:30.418591 10502 consensus_queue.cc:260] T 94c17c5750834fa999137c4d6cce4cf2 P 28d1d6c02b384aeaa740460051ea351b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.424533 10504 leader_election.cc:290] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [CANDIDATE]: Term 1 election: Requested vote from peers f090b64140fe4e55942c16e20a79b36b (127.9.116.129:42273), 28d1d6c02b384aeaa740460051ea351b (127.9.116.130:36353)
I20260709 13:15:30.426025 10387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a41f1857af412bb47303bd9452b65f" candidate_uuid: "dd07659d40f84c2aafeaabddc392aeab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28d1d6c02b384aeaa740460051ea351b"
I20260709 13:15:30.426649 10387 raft_consensus.cc:3060] T f7a41f1857af412bb47303bd9452b65f P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:15:30.427348 10258 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:15:30.428511 10502 ts_tablet_manager.cc:1434] T 94c17c5750834fa999137c4d6cce4cf2 P 28d1d6c02b384aeaa740460051ea351b: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 13:15:30.429523 10504 raft_consensus.cc:2804] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:30.430018 10504 raft_consensus.cc:697] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [term 1 LEADER]: Becoming Leader. State: Replica: dd07659d40f84c2aafeaabddc392aeab, State: Running, Role: LEADER
I20260709 13:15:30.430847 10504 consensus_queue.cc:237] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.436671 10387 raft_consensus.cc:2468] T f7a41f1857af412bb47303bd9452b65f P 28d1d6c02b384aeaa740460051ea351b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd07659d40f84c2aafeaabddc392aeab in term 1.
I20260709 13:15:30.435843 10502 tablet_bootstrap.cc:492] T acb9159770724070b93f9bc026ffee90 P 28d1d6c02b384aeaa740460051ea351b: Bootstrap starting.
I20260709 13:15:30.445375 10501 ts_tablet_manager.cc:1434] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab: Time spent starting tablet: real 0.055s	user 0.028s	sys 0.008s
I20260709 13:15:30.445139 10207 catalog_manager.cc:5697] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab reported cstate change: term changed from 0 to 1, leader changed from <none> to dd07659d40f84c2aafeaabddc392aeab (127.9.116.131). New cstate: current_term: 1 leader_uuid: "dd07659d40f84c2aafeaabddc392aeab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:30.446436 10501 tablet_bootstrap.cc:492] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab: Bootstrap starting.
I20260709 13:15:30.446575 10506 tablet_bootstrap.cc:492] T beb41e866f444744a06e30def55c3190 P f090b64140fe4e55942c16e20a79b36b: No bootstrap required, opened a new log
I20260709 13:15:30.447062 10506 ts_tablet_manager.cc:1403] T beb41e866f444744a06e30def55c3190 P f090b64140fe4e55942c16e20a79b36b: Time spent bootstrapping tablet: real 0.052s	user 0.024s	sys 0.023s
I20260709 13:15:30.449945 10506 raft_consensus.cc:359] T beb41e866f444744a06e30def55c3190 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } } peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } }
I20260709 13:15:30.450678 10506 raft_consensus.cc:385] T beb41e866f444744a06e30def55c3190 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:30.451005 10506 raft_consensus.cc:740] T beb41e866f444744a06e30def55c3190 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f090b64140fe4e55942c16e20a79b36b, State: Initialized, Role: FOLLOWER
I20260709 13:15:30.451773 10506 consensus_queue.cc:260] T beb41e866f444744a06e30def55c3190 P f090b64140fe4e55942c16e20a79b36b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } } peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } }
I20260709 13:15:30.454035 10506 ts_tablet_manager.cc:1434] T beb41e866f444744a06e30def55c3190 P f090b64140fe4e55942c16e20a79b36b: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:15:30.455005 10506 tablet_bootstrap.cc:492] T c5b1bc6ab2bb4893a25a192936d09209 P f090b64140fe4e55942c16e20a79b36b: Bootstrap starting.
I20260709 13:15:30.457616 10501 tablet_bootstrap.cc:654] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:30.455044 10502 tablet_bootstrap.cc:654] T acb9159770724070b93f9bc026ffee90 P 28d1d6c02b384aeaa740460051ea351b: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:30.463508 10508 raft_consensus.cc:493] T 7fa6a2759a7d43a28bc364bdd2593069 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:30.464351 10506 tablet_bootstrap.cc:654] T c5b1bc6ab2bb4893a25a192936d09209 P f090b64140fe4e55942c16e20a79b36b: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:30.464008 10508 raft_consensus.cc:515] T 7fa6a2759a7d43a28bc364bdd2593069 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.471190 10501 tablet_bootstrap.cc:492] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab: No bootstrap required, opened a new log
I20260709 13:15:30.471825 10501 ts_tablet_manager.cc:1403] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab: Time spent bootstrapping tablet: real 0.026s	user 0.013s	sys 0.000s
I20260709 13:15:30.475143 10501 raft_consensus.cc:359] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.476051 10501 raft_consensus.cc:385] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:30.476383 10501 raft_consensus.cc:740] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd07659d40f84c2aafeaabddc392aeab, State: Initialized, Role: FOLLOWER
I20260709 13:15:30.482218 10501 consensus_queue.cc:260] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.470371 10508 leader_election.cc:290] T 7fa6a2759a7d43a28bc364bdd2593069 P 28d1d6c02b384aeaa740460051ea351b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f090b64140fe4e55942c16e20a79b36b (127.9.116.129:42273), dd07659d40f84c2aafeaabddc392aeab (127.9.116.131:42413)
I20260709 13:15:30.486091 10501 ts_tablet_manager.cc:1434] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:15:30.502413 10501 tablet_bootstrap.cc:492] T acb9159770724070b93f9bc026ffee90 P dd07659d40f84c2aafeaabddc392aeab: Bootstrap starting.
I20260709 13:15:30.503703 10506 tablet_bootstrap.cc:492] T c5b1bc6ab2bb4893a25a192936d09209 P f090b64140fe4e55942c16e20a79b36b: No bootstrap required, opened a new log
I20260709 13:15:30.504181 10506 ts_tablet_manager.cc:1403] T c5b1bc6ab2bb4893a25a192936d09209 P f090b64140fe4e55942c16e20a79b36b: Time spent bootstrapping tablet: real 0.049s	user 0.013s	sys 0.030s
I20260709 13:15:30.504465 10508 raft_consensus.cc:493] T beb41e866f444744a06e30def55c3190 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:30.504937 10508 raft_consensus.cc:515] T beb41e866f444744a06e30def55c3190 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } } peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } }
I20260709 13:15:30.506033 10312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fa6a2759a7d43a28bc364bdd2593069" candidate_uuid: "28d1d6c02b384aeaa740460051ea351b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f090b64140fe4e55942c16e20a79b36b" is_pre_election: true
I20260709 13:15:30.506662 10312 raft_consensus.cc:2468] T 7fa6a2759a7d43a28bc364bdd2593069 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28d1d6c02b384aeaa740460051ea351b in term 0.
I20260709 13:15:30.506927 10508 leader_election.cc:290] T beb41e866f444744a06e30def55c3190 P 28d1d6c02b384aeaa740460051ea351b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd07659d40f84c2aafeaabddc392aeab (127.9.116.131:42413), f090b64140fe4e55942c16e20a79b36b (127.9.116.129:42273)
I20260709 13:15:30.506687 10506 raft_consensus.cc:359] T c5b1bc6ab2bb4893a25a192936d09209 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.507531 10506 raft_consensus.cc:385] T c5b1bc6ab2bb4893a25a192936d09209 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:30.507809 10506 raft_consensus.cc:740] T c5b1bc6ab2bb4893a25a192936d09209 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f090b64140fe4e55942c16e20a79b36b, State: Initialized, Role: FOLLOWER
I20260709 13:15:30.508381 10501 tablet_bootstrap.cc:654] T acb9159770724070b93f9bc026ffee90 P dd07659d40f84c2aafeaabddc392aeab: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:30.508738 10312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "beb41e866f444744a06e30def55c3190" candidate_uuid: "28d1d6c02b384aeaa740460051ea351b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f090b64140fe4e55942c16e20a79b36b" is_pre_election: true
I20260709 13:15:30.508431 10506 consensus_queue.cc:260] T c5b1bc6ab2bb4893a25a192936d09209 P f090b64140fe4e55942c16e20a79b36b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.509389 10312 raft_consensus.cc:2468] T beb41e866f444744a06e30def55c3190 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28d1d6c02b384aeaa740460051ea351b in term 0.
I20260709 13:15:30.510558 10506 ts_tablet_manager.cc:1434] T c5b1bc6ab2bb4893a25a192936d09209 P f090b64140fe4e55942c16e20a79b36b: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:15:30.511514 10506 tablet_bootstrap.cc:492] T acb9159770724070b93f9bc026ffee90 P f090b64140fe4e55942c16e20a79b36b: Bootstrap starting.
I20260709 13:15:30.516983 10506 tablet_bootstrap.cc:654] T acb9159770724070b93f9bc026ffee90 P f090b64140fe4e55942c16e20a79b36b: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:30.517457 10351 leader_election.cc:304] T 7fa6a2759a7d43a28bc364bdd2593069 P 28d1d6c02b384aeaa740460051ea351b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28d1d6c02b384aeaa740460051ea351b, f090b64140fe4e55942c16e20a79b36b; no voters: 
I20260709 13:15:30.519214 10351 leader_election.cc:304] T beb41e866f444744a06e30def55c3190 P 28d1d6c02b384aeaa740460051ea351b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28d1d6c02b384aeaa740460051ea351b, f090b64140fe4e55942c16e20a79b36b; no voters: 
I20260709 13:15:30.523123 10508 raft_consensus.cc:2804] T 7fa6a2759a7d43a28bc364bdd2593069 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:30.523772 10502 tablet_bootstrap.cc:492] T acb9159770724070b93f9bc026ffee90 P 28d1d6c02b384aeaa740460051ea351b: No bootstrap required, opened a new log
I20260709 13:15:30.523841 10508 raft_consensus.cc:493] T 7fa6a2759a7d43a28bc364bdd2593069 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.524330 10502 ts_tablet_manager.cc:1403] T acb9159770724070b93f9bc026ffee90 P 28d1d6c02b384aeaa740460051ea351b: Time spent bootstrapping tablet: real 0.089s	user 0.018s	sys 0.017s
I20260709 13:15:30.524300 10508 raft_consensus.cc:3060] T 7fa6a2759a7d43a28bc364bdd2593069 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:30.527194 10502 raft_consensus.cc:359] T acb9159770724070b93f9bc026ffee90 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.527868 10502 raft_consensus.cc:385] T acb9159770724070b93f9bc026ffee90 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:30.528123 10502 raft_consensus.cc:740] T acb9159770724070b93f9bc026ffee90 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28d1d6c02b384aeaa740460051ea351b, State: Initialized, Role: FOLLOWER
I20260709 13:15:30.528780 10502 consensus_queue.cc:260] T acb9159770724070b93f9bc026ffee90 P 28d1d6c02b384aeaa740460051ea351b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.530663 10531 raft_consensus.cc:493] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:30.531384 10531 raft_consensus.cc:515] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.536262 10531 leader_election.cc:290] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f090b64140fe4e55942c16e20a79b36b (127.9.116.129:42273), 28d1d6c02b384aeaa740460051ea351b (127.9.116.130:36353)
I20260709 13:15:30.537364 10312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09786fc30e8048d9b7928fe7bf969df1" candidate_uuid: "dd07659d40f84c2aafeaabddc392aeab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f090b64140fe4e55942c16e20a79b36b" is_pre_election: true
I20260709 13:15:30.538218 10312 raft_consensus.cc:2468] T 09786fc30e8048d9b7928fe7bf969df1 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd07659d40f84c2aafeaabddc392aeab in term 0.
I20260709 13:15:30.539817 10425 leader_election.cc:304] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dd07659d40f84c2aafeaabddc392aeab, f090b64140fe4e55942c16e20a79b36b; no voters: 
I20260709 13:15:30.541128 10504 raft_consensus.cc:2804] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:30.541319 10531 raft_consensus.cc:493] T 7fa6a2759a7d43a28bc364bdd2593069 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:30.541554 10504 raft_consensus.cc:493] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.542065 10504 raft_consensus.cc:3060] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:30.542083 10531 raft_consensus.cc:515] T 7fa6a2759a7d43a28bc364bdd2593069 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.546618 10531 leader_election.cc:290] T 7fa6a2759a7d43a28bc364bdd2593069 P dd07659d40f84c2aafeaabddc392aeab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f090b64140fe4e55942c16e20a79b36b (127.9.116.129:42273), 28d1d6c02b384aeaa740460051ea351b (127.9.116.130:36353)
I20260709 13:15:30.555305 10531 raft_consensus.cc:493] T 9db04314c2c849898b82d951374e2878 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:30.555989 10531 raft_consensus.cc:515] T 9db04314c2c849898b82d951374e2878 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.560369 10311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db04314c2c849898b82d951374e2878" candidate_uuid: "dd07659d40f84c2aafeaabddc392aeab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f090b64140fe4e55942c16e20a79b36b" is_pre_election: true
I20260709 13:15:30.561401 10311 raft_consensus.cc:2468] T 9db04314c2c849898b82d951374e2878 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd07659d40f84c2aafeaabddc392aeab in term 0.
I20260709 13:15:30.561705 10312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fa6a2759a7d43a28bc364bdd2593069" candidate_uuid: "dd07659d40f84c2aafeaabddc392aeab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f090b64140fe4e55942c16e20a79b36b" is_pre_election: true
I20260709 13:15:30.564103 10312 raft_consensus.cc:2468] T 7fa6a2759a7d43a28bc364bdd2593069 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd07659d40f84c2aafeaabddc392aeab in term 0.
I20260709 13:15:30.561391 10531 leader_election.cc:290] T 9db04314c2c849898b82d951374e2878 P dd07659d40f84c2aafeaabddc392aeab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f090b64140fe4e55942c16e20a79b36b (127.9.116.129:42273), 28d1d6c02b384aeaa740460051ea351b (127.9.116.130:36353)
I20260709 13:15:30.563184 10425 leader_election.cc:304] T 9db04314c2c849898b82d951374e2878 P dd07659d40f84c2aafeaabddc392aeab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dd07659d40f84c2aafeaabddc392aeab, f090b64140fe4e55942c16e20a79b36b; no voters: 
I20260709 13:15:30.566253 10425 leader_election.cc:304] T 7fa6a2759a7d43a28bc364bdd2593069 P dd07659d40f84c2aafeaabddc392aeab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dd07659d40f84c2aafeaabddc392aeab, f090b64140fe4e55942c16e20a79b36b; no voters: 
I20260709 13:15:30.567358 10534 raft_consensus.cc:2804] T 9db04314c2c849898b82d951374e2878 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:30.567696 10531 raft_consensus.cc:2804] T 7fa6a2759a7d43a28bc364bdd2593069 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:30.567862 10534 raft_consensus.cc:493] T 9db04314c2c849898b82d951374e2878 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.568416 10534 raft_consensus.cc:3060] T 9db04314c2c849898b82d951374e2878 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:30.576974 10508 raft_consensus.cc:515] T 7fa6a2759a7d43a28bc364bdd2593069 P 28d1d6c02b384aeaa740460051ea351b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
W20260709 13:15:30.578919 10351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:42273, user_credentials={real_user=slave}} blocked reactor thread for 60201.5us
I20260709 13:15:30.580698 10311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fa6a2759a7d43a28bc364bdd2593069" candidate_uuid: "28d1d6c02b384aeaa740460051ea351b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f090b64140fe4e55942c16e20a79b36b"
I20260709 13:15:30.581277 10311 raft_consensus.cc:3060] T 7fa6a2759a7d43a28bc364bdd2593069 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:30.581604 10534 raft_consensus.cc:515] T 9db04314c2c849898b82d951374e2878 P dd07659d40f84c2aafeaabddc392aeab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.568284 10531 raft_consensus.cc:493] T 7fa6a2759a7d43a28bc364bdd2593069 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.588598 10548 raft_consensus.cc:493] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:30.589217 10548 raft_consensus.cc:515] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.587817 10531 raft_consensus.cc:3060] T 7fa6a2759a7d43a28bc364bdd2593069 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:30.598503 10311 raft_consensus.cc:2468] T 7fa6a2759a7d43a28bc364bdd2593069 P f090b64140fe4e55942c16e20a79b36b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28d1d6c02b384aeaa740460051ea351b in term 1.
I20260709 13:15:30.598706 10387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09786fc30e8048d9b7928fe7bf969df1" candidate_uuid: "dd07659d40f84c2aafeaabddc392aeab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28d1d6c02b384aeaa740460051ea351b" is_pre_election: true
I20260709 13:15:30.599696 10387 raft_consensus.cc:2468] T 09786fc30e8048d9b7928fe7bf969df1 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd07659d40f84c2aafeaabddc392aeab in term 0.
I20260709 13:15:30.601894 10534 leader_election.cc:290] T 9db04314c2c849898b82d951374e2878 P dd07659d40f84c2aafeaabddc392aeab [CANDIDATE]: Term 1 election: Requested vote from peers f090b64140fe4e55942c16e20a79b36b (127.9.116.129:42273), 28d1d6c02b384aeaa740460051ea351b (127.9.116.130:36353)
I20260709 13:15:30.605595 10504 raft_consensus.cc:515] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.607877 10312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c17c5750834fa999137c4d6cce4cf2" candidate_uuid: "dd07659d40f84c2aafeaabddc392aeab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f090b64140fe4e55942c16e20a79b36b" is_pre_election: true
I20260709 13:15:30.608530 10312 raft_consensus.cc:2468] T 94c17c5750834fa999137c4d6cce4cf2 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd07659d40f84c2aafeaabddc392aeab in term 0.
I20260709 13:15:30.608660 10310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09786fc30e8048d9b7928fe7bf969df1" candidate_uuid: "dd07659d40f84c2aafeaabddc392aeab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f090b64140fe4e55942c16e20a79b36b"
I20260709 13:15:30.609102 10504 leader_election.cc:290] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [CANDIDATE]: Term 1 election: Requested vote from peers f090b64140fe4e55942c16e20a79b36b (127.9.116.129:42273), 28d1d6c02b384aeaa740460051ea351b (127.9.116.130:36353)
I20260709 13:15:30.609323 10310 raft_consensus.cc:3060] T 09786fc30e8048d9b7928fe7bf969df1 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:30.609793 10425 leader_election.cc:304] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dd07659d40f84c2aafeaabddc392aeab, f090b64140fe4e55942c16e20a79b36b; no voters: 
I20260709 13:15:30.611415 10531 raft_consensus.cc:515] T 7fa6a2759a7d43a28bc364bdd2593069 P dd07659d40f84c2aafeaabddc392aeab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.613490 10531 leader_election.cc:290] T 7fa6a2759a7d43a28bc364bdd2593069 P dd07659d40f84c2aafeaabddc392aeab [CANDIDATE]: Term 1 election: Requested vote from peers f090b64140fe4e55942c16e20a79b36b (127.9.116.129:42273), 28d1d6c02b384aeaa740460051ea351b (127.9.116.130:36353)
I20260709 13:15:30.614032 10531 raft_consensus.cc:2804] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:30.614214 10312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fa6a2759a7d43a28bc364bdd2593069" candidate_uuid: "dd07659d40f84c2aafeaabddc392aeab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f090b64140fe4e55942c16e20a79b36b"
I20260709 13:15:30.615053 10312 raft_consensus.cc:2393] T 7fa6a2759a7d43a28bc364bdd2593069 P f090b64140fe4e55942c16e20a79b36b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate dd07659d40f84c2aafeaabddc392aeab in current term 1: Already voted for candidate 28d1d6c02b384aeaa740460051ea351b in this term.
I20260709 13:15:30.615392 10387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fa6a2759a7d43a28bc364bdd2593069" candidate_uuid: "dd07659d40f84c2aafeaabddc392aeab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28d1d6c02b384aeaa740460051ea351b" is_pre_election: true
I20260709 13:15:30.616245 10387 raft_consensus.cc:2393] T 7fa6a2759a7d43a28bc364bdd2593069 P 28d1d6c02b384aeaa740460051ea351b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd07659d40f84c2aafeaabddc392aeab in current term 1: Already voted for candidate 28d1d6c02b384aeaa740460051ea351b in this term.
I20260709 13:15:30.616822 10310 raft_consensus.cc:2468] T 09786fc30e8048d9b7928fe7bf969df1 P f090b64140fe4e55942c16e20a79b36b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd07659d40f84c2aafeaabddc392aeab in term 1.
I20260709 13:15:30.617646 10387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db04314c2c849898b82d951374e2878" candidate_uuid: "dd07659d40f84c2aafeaabddc392aeab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28d1d6c02b384aeaa740460051ea351b" is_pre_election: true
I20260709 13:15:30.618119 10425 leader_election.cc:304] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dd07659d40f84c2aafeaabddc392aeab, f090b64140fe4e55942c16e20a79b36b; no voters: 
I20260709 13:15:30.618278 10387 raft_consensus.cc:2468] T 9db04314c2c849898b82d951374e2878 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd07659d40f84c2aafeaabddc392aeab in term 0.
I20260709 13:15:30.619086 10534 raft_consensus.cc:2804] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:30.619270 10311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db04314c2c849898b82d951374e2878" candidate_uuid: "dd07659d40f84c2aafeaabddc392aeab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f090b64140fe4e55942c16e20a79b36b"
I20260709 13:15:30.619743 10534 raft_consensus.cc:697] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [term 1 LEADER]: Becoming Leader. State: Replica: dd07659d40f84c2aafeaabddc392aeab, State: Running, Role: LEADER
I20260709 13:15:30.620106 10311 raft_consensus.cc:3060] T 9db04314c2c849898b82d951374e2878 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:30.620924 10534 consensus_queue.cc:237] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.614367 10531 raft_consensus.cc:493] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.523825 10546 raft_consensus.cc:2804] T beb41e866f444744a06e30def55c3190 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:30.622895 10531 raft_consensus.cc:3060] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:30.623152 10546 raft_consensus.cc:493] T beb41e866f444744a06e30def55c3190 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.623536 10546 raft_consensus.cc:3060] T beb41e866f444744a06e30def55c3190 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:30.625859 10387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db04314c2c849898b82d951374e2878" candidate_uuid: "dd07659d40f84c2aafeaabddc392aeab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28d1d6c02b384aeaa740460051ea351b"
I20260709 13:15:30.627521 10386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09786fc30e8048d9b7928fe7bf969df1" candidate_uuid: "dd07659d40f84c2aafeaabddc392aeab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28d1d6c02b384aeaa740460051ea351b"
I20260709 13:15:30.633905 10386 raft_consensus.cc:3060] T 09786fc30e8048d9b7928fe7bf969df1 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:30.639081 10387 raft_consensus.cc:3060] T 9db04314c2c849898b82d951374e2878 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:30.639537 10531 raft_consensus.cc:515] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.629539 10385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fa6a2759a7d43a28bc364bdd2593069" candidate_uuid: "dd07659d40f84c2aafeaabddc392aeab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28d1d6c02b384aeaa740460051ea351b"
I20260709 13:15:30.629287 10351 leader_election.cc:304] T 7fa6a2759a7d43a28bc364bdd2593069 P 28d1d6c02b384aeaa740460051ea351b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28d1d6c02b384aeaa740460051ea351b, f090b64140fe4e55942c16e20a79b36b; no voters: 
I20260709 13:15:30.643464 10385 raft_consensus.cc:2393] T 7fa6a2759a7d43a28bc364bdd2593069 P 28d1d6c02b384aeaa740460051ea351b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate dd07659d40f84c2aafeaabddc392aeab in current term 1: Already voted for candidate 28d1d6c02b384aeaa740460051ea351b in this term.
I20260709 13:15:30.645303 10531 leader_election.cc:290] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab [CANDIDATE]: Term 1 election: Requested vote from peers f090b64140fe4e55942c16e20a79b36b (127.9.116.129:42273), 28d1d6c02b384aeaa740460051ea351b (127.9.116.130:36353)
I20260709 13:15:30.646430 10551 raft_consensus.cc:493] T 94c17c5750834fa999137c4d6cce4cf2 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:30.647421 10385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c17c5750834fa999137c4d6cce4cf2" candidate_uuid: "dd07659d40f84c2aafeaabddc392aeab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28d1d6c02b384aeaa740460051ea351b"
I20260709 13:15:30.632855 10508 leader_election.cc:290] T 7fa6a2759a7d43a28bc364bdd2593069 P 28d1d6c02b384aeaa740460051ea351b [CANDIDATE]: Term 1 election: Requested vote from peers f090b64140fe4e55942c16e20a79b36b (127.9.116.129:42273), dd07659d40f84c2aafeaabddc392aeab (127.9.116.131:42413)
I20260709 13:15:30.642501 10310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c17c5750834fa999137c4d6cce4cf2" candidate_uuid: "dd07659d40f84c2aafeaabddc392aeab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f090b64140fe4e55942c16e20a79b36b"
I20260709 13:15:30.648327 10310 raft_consensus.cc:3060] T 94c17c5750834fa999137c4d6cce4cf2 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:30.650272 10386 raft_consensus.cc:2468] T 09786fc30e8048d9b7928fe7bf969df1 P 28d1d6c02b384aeaa740460051ea351b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd07659d40f84c2aafeaabddc392aeab in term 1.
I20260709 13:15:30.651203 10512 raft_consensus.cc:493] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:30.655059 10512 raft_consensus.cc:515] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.657980 10512 leader_election.cc:290] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 28d1d6c02b384aeaa740460051ea351b (127.9.116.130:36353), dd07659d40f84c2aafeaabddc392aeab (127.9.116.131:42413)
I20260709 13:15:30.660499 10384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e8982a8ea4fa198de62d0e60b459c" candidate_uuid: "f090b64140fe4e55942c16e20a79b36b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28d1d6c02b384aeaa740460051ea351b" is_pre_election: true
I20260709 13:15:30.661895 10461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e8982a8ea4fa198de62d0e60b459c" candidate_uuid: "f090b64140fe4e55942c16e20a79b36b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd07659d40f84c2aafeaabddc392aeab" is_pre_election: true
I20260709 13:15:30.662436 10310 raft_consensus.cc:2468] T 94c17c5750834fa999137c4d6cce4cf2 P f090b64140fe4e55942c16e20a79b36b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd07659d40f84c2aafeaabddc392aeab in term 1.
I20260709 13:15:30.662706 10461 raft_consensus.cc:2468] T 135e8982a8ea4fa198de62d0e60b459c P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f090b64140fe4e55942c16e20a79b36b in term 0.
I20260709 13:15:30.663919 10425 leader_election.cc:304] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dd07659d40f84c2aafeaabddc392aeab, f090b64140fe4e55942c16e20a79b36b; no voters: 
I20260709 13:15:30.664810 10531 raft_consensus.cc:2804] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:30.646920 10427 leader_election.cc:304] T 7fa6a2759a7d43a28bc364bdd2593069 P dd07659d40f84c2aafeaabddc392aeab [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd07659d40f84c2aafeaabddc392aeab; no voters: 28d1d6c02b384aeaa740460051ea351b, f090b64140fe4e55942c16e20a79b36b
I20260709 13:15:30.665290 10531 raft_consensus.cc:697] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab [term 1 LEADER]: Becoming Leader. State: Replica: dd07659d40f84c2aafeaabddc392aeab, State: Running, Role: LEADER
I20260709 13:15:30.664531 10278 leader_election.cc:304] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dd07659d40f84c2aafeaabddc392aeab, f090b64140fe4e55942c16e20a79b36b; no voters: 
I20260709 13:15:30.666270 10387 raft_consensus.cc:2468] T 9db04314c2c849898b82d951374e2878 P 28d1d6c02b384aeaa740460051ea351b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd07659d40f84c2aafeaabddc392aeab in term 1.
I20260709 13:15:30.666191 10531 consensus_queue.cc:237] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.666533 10512 raft_consensus.cc:2804] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:30.667266 10512 raft_consensus.cc:493] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.667768 10512 raft_consensus.cc:3060] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:30.628473 10549 raft_consensus.cc:493] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:30.673812 10534 raft_consensus.cc:2749] T 7fa6a2759a7d43a28bc364bdd2593069 P dd07659d40f84c2aafeaabddc392aeab [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:15:30.673853 10502 ts_tablet_manager.cc:1434] T acb9159770724070b93f9bc026ffee90 P 28d1d6c02b384aeaa740460051ea351b: Time spent starting tablet: real 0.149s	user 0.039s	sys 0.058s
I20260709 13:15:30.676005 10502 tablet_bootstrap.cc:492] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b: Bootstrap starting.
I20260709 13:15:30.674587 10549 raft_consensus.cc:515] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.671752 10508 raft_consensus.cc:2804] T 7fa6a2759a7d43a28bc364bdd2593069 P 28d1d6c02b384aeaa740460051ea351b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:30.682518 10549 leader_election.cc:290] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f090b64140fe4e55942c16e20a79b36b (127.9.116.129:42273), dd07659d40f84c2aafeaabddc392aeab (127.9.116.131:42413)
I20260709 13:15:30.683269 10310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e8982a8ea4fa198de62d0e60b459c" candidate_uuid: "28d1d6c02b384aeaa740460051ea351b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f090b64140fe4e55942c16e20a79b36b" is_pre_election: true
I20260709 13:15:30.681407 10508 raft_consensus.cc:697] T 7fa6a2759a7d43a28bc364bdd2593069 P 28d1d6c02b384aeaa740460051ea351b [term 1 LEADER]: Becoming Leader. State: Replica: 28d1d6c02b384aeaa740460051ea351b, State: Running, Role: LEADER
I20260709 13:15:30.685159 10506 tablet_bootstrap.cc:492] T acb9159770724070b93f9bc026ffee90 P f090b64140fe4e55942c16e20a79b36b: No bootstrap required, opened a new log
I20260709 13:15:30.685698 10506 ts_tablet_manager.cc:1403] T acb9159770724070b93f9bc026ffee90 P f090b64140fe4e55942c16e20a79b36b: Time spent bootstrapping tablet: real 0.174s	user 0.043s	sys 0.056s
I20260709 13:15:30.685238 10508 consensus_queue.cc:237] T 7fa6a2759a7d43a28bc364bdd2593069 P 28d1d6c02b384aeaa740460051ea351b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.686524 10384 raft_consensus.cc:2468] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f090b64140fe4e55942c16e20a79b36b in term 0.
I20260709 13:15:30.647810 10551 raft_consensus.cc:515] T 94c17c5750834fa999137c4d6cce4cf2 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.693655 10506 raft_consensus.cc:359] T acb9159770724070b93f9bc026ffee90 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.694365 10506 raft_consensus.cc:385] T acb9159770724070b93f9bc026ffee90 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:30.694693 10506 raft_consensus.cc:740] T acb9159770724070b93f9bc026ffee90 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f090b64140fe4e55942c16e20a79b36b, State: Initialized, Role: FOLLOWER
I20260709 13:15:30.695590 10506 consensus_queue.cc:260] T acb9159770724070b93f9bc026ffee90 P f090b64140fe4e55942c16e20a79b36b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.696548 10548 leader_election.cc:290] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f090b64140fe4e55942c16e20a79b36b (127.9.116.129:42273), 28d1d6c02b384aeaa740460051ea351b (127.9.116.130:36353)
I20260709 13:15:30.697693 10385 raft_consensus.cc:3060] T 94c17c5750834fa999137c4d6cce4cf2 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:30.698874 10501 tablet_bootstrap.cc:492] T acb9159770724070b93f9bc026ffee90 P dd07659d40f84c2aafeaabddc392aeab: No bootstrap required, opened a new log
I20260709 13:15:30.699018 10551 leader_election.cc:290] T 94c17c5750834fa999137c4d6cce4cf2 P 28d1d6c02b384aeaa740460051ea351b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f090b64140fe4e55942c16e20a79b36b (127.9.116.129:42273), dd07659d40f84c2aafeaabddc392aeab (127.9.116.131:42413)
I20260709 13:15:30.699366 10501 ts_tablet_manager.cc:1403] T acb9159770724070b93f9bc026ffee90 P dd07659d40f84c2aafeaabddc392aeab: Time spent bootstrapping tablet: real 0.197s	user 0.049s	sys 0.083s
I20260709 13:15:30.702047 10501 raft_consensus.cc:359] T acb9159770724070b93f9bc026ffee90 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.702692 10501 raft_consensus.cc:385] T acb9159770724070b93f9bc026ffee90 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:30.702989 10501 raft_consensus.cc:740] T acb9159770724070b93f9bc026ffee90 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd07659d40f84c2aafeaabddc392aeab, State: Initialized, Role: FOLLOWER
I20260709 13:15:30.703572 10501 consensus_queue.cc:260] T acb9159770724070b93f9bc026ffee90 P dd07659d40f84c2aafeaabddc392aeab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.704254 10311 raft_consensus.cc:2468] T 9db04314c2c849898b82d951374e2878 P f090b64140fe4e55942c16e20a79b36b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd07659d40f84c2aafeaabddc392aeab in term 1.
I20260709 13:15:30.705737 10501 ts_tablet_manager.cc:1434] T acb9159770724070b93f9bc026ffee90 P dd07659d40f84c2aafeaabddc392aeab: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:15:30.705875 10425 leader_election.cc:304] T 9db04314c2c849898b82d951374e2878 P dd07659d40f84c2aafeaabddc392aeab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dd07659d40f84c2aafeaabddc392aeab, f090b64140fe4e55942c16e20a79b36b; no voters: 
I20260709 13:15:30.706619 10501 tablet_bootstrap.cc:492] T b6a956a56be14d83b5f0a8db624c1609 P dd07659d40f84c2aafeaabddc392aeab: Bootstrap starting.
I20260709 13:15:30.708603 10385 raft_consensus.cc:2468] T 94c17c5750834fa999137c4d6cce4cf2 P 28d1d6c02b384aeaa740460051ea351b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd07659d40f84c2aafeaabddc392aeab in term 1.
I20260709 13:15:30.709605 10548 raft_consensus.cc:2804] T 9db04314c2c849898b82d951374e2878 P dd07659d40f84c2aafeaabddc392aeab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:30.710381 10548 raft_consensus.cc:697] T 9db04314c2c849898b82d951374e2878 P dd07659d40f84c2aafeaabddc392aeab [term 1 LEADER]: Becoming Leader. State: Replica: dd07659d40f84c2aafeaabddc392aeab, State: Running, Role: LEADER
I20260709 13:15:30.710980 10506 ts_tablet_manager.cc:1434] T acb9159770724070b93f9bc026ffee90 P f090b64140fe4e55942c16e20a79b36b: Time spent starting tablet: real 0.025s	user 0.009s	sys 0.000s
I20260709 13:15:30.711918 10506 tablet_bootstrap.cc:492] T b6a956a56be14d83b5f0a8db624c1609 P f090b64140fe4e55942c16e20a79b36b: Bootstrap starting.
W20260709 13:15:30.709969 10427 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:36353, user_credentials={real_user=slave}} blocked reactor thread for 63315.6us
I20260709 13:15:30.711256 10548 consensus_queue.cc:237] T 9db04314c2c849898b82d951374e2878 P dd07659d40f84c2aafeaabddc392aeab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.711292 10207 catalog_manager.cc:5697] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab reported cstate change: term changed from 0 to 1, leader changed from <none> to dd07659d40f84c2aafeaabddc392aeab (127.9.116.131). New cstate: current_term: 1 leader_uuid: "dd07659d40f84c2aafeaabddc392aeab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:30.719424 10512 raft_consensus.cc:515] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.720613 10311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c17c5750834fa999137c4d6cce4cf2" candidate_uuid: "28d1d6c02b384aeaa740460051ea351b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f090b64140fe4e55942c16e20a79b36b" is_pre_election: true
I20260709 13:15:30.721395 10311 raft_consensus.cc:2393] T 94c17c5750834fa999137c4d6cce4cf2 P f090b64140fe4e55942c16e20a79b36b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28d1d6c02b384aeaa740460051ea351b in current term 1: Already voted for candidate dd07659d40f84c2aafeaabddc392aeab in this term.
I20260709 13:15:30.722285 10310 raft_consensus.cc:2393] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28d1d6c02b384aeaa740460051ea351b in current term 1: Already voted for candidate f090b64140fe4e55942c16e20a79b36b in this term.
I20260709 13:15:30.722879 10384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e8982a8ea4fa198de62d0e60b459c" candidate_uuid: "f090b64140fe4e55942c16e20a79b36b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28d1d6c02b384aeaa740460051ea351b"
I20260709 13:15:30.723044 10385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c17c5750834fa999137c4d6cce4cf2" candidate_uuid: "dd07659d40f84c2aafeaabddc392aeab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28d1d6c02b384aeaa740460051ea351b" is_pre_election: true
I20260709 13:15:30.723489 10384 raft_consensus.cc:3060] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:30.723676 10461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fa6a2759a7d43a28bc364bdd2593069" candidate_uuid: "28d1d6c02b384aeaa740460051ea351b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd07659d40f84c2aafeaabddc392aeab" is_pre_election: true
I20260709 13:15:30.717476 10546 raft_consensus.cc:515] T beb41e866f444744a06e30def55c3190 P 28d1d6c02b384aeaa740460051ea351b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } } peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } }
I20260709 13:15:30.726158 10546 leader_election.cc:290] T beb41e866f444744a06e30def55c3190 P 28d1d6c02b384aeaa740460051ea351b [CANDIDATE]: Term 1 election: Requested vote from peers dd07659d40f84c2aafeaabddc392aeab (127.9.116.131:42413), f090b64140fe4e55942c16e20a79b36b (127.9.116.129:42273)
I20260709 13:15:30.726819 10461 raft_consensus.cc:2393] T 7fa6a2759a7d43a28bc364bdd2593069 P dd07659d40f84c2aafeaabddc392aeab [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28d1d6c02b384aeaa740460051ea351b in current term 1: Already voted for candidate dd07659d40f84c2aafeaabddc392aeab in this term.
I20260709 13:15:30.723683 10385 raft_consensus.cc:2376] T 94c17c5750834fa999137c4d6cce4cf2 P 28d1d6c02b384aeaa740460051ea351b [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate dd07659d40f84c2aafeaabddc392aeab in term 1. Re-sending same reply.
I20260709 13:15:30.728268 10310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "beb41e866f444744a06e30def55c3190" candidate_uuid: "28d1d6c02b384aeaa740460051ea351b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f090b64140fe4e55942c16e20a79b36b"
I20260709 13:15:30.728857 10310 raft_consensus.cc:3060] T beb41e866f444744a06e30def55c3190 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:30.728739 10461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "beb41e866f444744a06e30def55c3190" candidate_uuid: "28d1d6c02b384aeaa740460051ea351b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd07659d40f84c2aafeaabddc392aeab" is_pre_election: true
I20260709 13:15:30.730479 10460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fa6a2759a7d43a28bc364bdd2593069" candidate_uuid: "28d1d6c02b384aeaa740460051ea351b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd07659d40f84c2aafeaabddc392aeab"
I20260709 13:15:30.730890 10459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e8982a8ea4fa198de62d0e60b459c" candidate_uuid: "28d1d6c02b384aeaa740460051ea351b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd07659d40f84c2aafeaabddc392aeab" is_pre_election: true
I20260709 13:15:30.730621 10461 raft_consensus.cc:2468] T beb41e866f444744a06e30def55c3190 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28d1d6c02b384aeaa740460051ea351b in term 0.
I20260709 13:15:30.731575 10459 raft_consensus.cc:2468] T 135e8982a8ea4fa198de62d0e60b459c P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28d1d6c02b384aeaa740460051ea351b in term 0.
I20260709 13:15:30.731761 10460 raft_consensus.cc:2393] T 7fa6a2759a7d43a28bc364bdd2593069 P dd07659d40f84c2aafeaabddc392aeab [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 28d1d6c02b384aeaa740460051ea351b in current term 1: Already voted for candidate dd07659d40f84c2aafeaabddc392aeab in this term.
I20260709 13:15:30.733894 10459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c17c5750834fa999137c4d6cce4cf2" candidate_uuid: "28d1d6c02b384aeaa740460051ea351b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd07659d40f84c2aafeaabddc392aeab" is_pre_election: true
I20260709 13:15:30.738003 10502 tablet_bootstrap.cc:654] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:30.738521 10459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "beb41e866f444744a06e30def55c3190" candidate_uuid: "28d1d6c02b384aeaa740460051ea351b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd07659d40f84c2aafeaabddc392aeab"
I20260709 13:15:30.739686 10459 raft_consensus.cc:3060] T beb41e866f444744a06e30def55c3190 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:30.740595 10384 raft_consensus.cc:2468] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f090b64140fe4e55942c16e20a79b36b in term 1.
I20260709 13:15:30.741886 10278 leader_election.cc:304] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28d1d6c02b384aeaa740460051ea351b, f090b64140fe4e55942c16e20a79b36b; no voters: 
I20260709 13:15:30.742735 10510 raft_consensus.cc:2804] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:30.750976 10460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e8982a8ea4fa198de62d0e60b459c" candidate_uuid: "f090b64140fe4e55942c16e20a79b36b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd07659d40f84c2aafeaabddc392aeab"
I20260709 13:15:30.751605 10460 raft_consensus.cc:3060] T 135e8982a8ea4fa198de62d0e60b459c P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:30.756650 10310 raft_consensus.cc:2468] T beb41e866f444744a06e30def55c3190 P f090b64140fe4e55942c16e20a79b36b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28d1d6c02b384aeaa740460051ea351b in term 1.
I20260709 13:15:30.757916 10351 leader_election.cc:304] T beb41e866f444744a06e30def55c3190 P 28d1d6c02b384aeaa740460051ea351b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28d1d6c02b384aeaa740460051ea351b, f090b64140fe4e55942c16e20a79b36b; no voters: 
I20260709 13:15:30.758767 10546 raft_consensus.cc:2804] T beb41e866f444744a06e30def55c3190 P 28d1d6c02b384aeaa740460051ea351b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:30.759277 10546 raft_consensus.cc:697] T beb41e866f444744a06e30def55c3190 P 28d1d6c02b384aeaa740460051ea351b [term 1 LEADER]: Becoming Leader. State: Replica: 28d1d6c02b384aeaa740460051ea351b, State: Running, Role: LEADER
I20260709 13:15:30.760277 10501 tablet_bootstrap.cc:654] T b6a956a56be14d83b5f0a8db624c1609 P dd07659d40f84c2aafeaabddc392aeab: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:30.760049 10546 consensus_queue.cc:237] T beb41e866f444744a06e30def55c3190 P 28d1d6c02b384aeaa740460051ea351b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } } peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } }
I20260709 13:15:30.765969 10460 raft_consensus.cc:2468] T 135e8982a8ea4fa198de62d0e60b459c P dd07659d40f84c2aafeaabddc392aeab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f090b64140fe4e55942c16e20a79b36b in term 1.
I20260709 13:15:30.767786 10353 leader_election.cc:304] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28d1d6c02b384aeaa740460051ea351b, dd07659d40f84c2aafeaabddc392aeab; no voters: f090b64140fe4e55942c16e20a79b36b
I20260709 13:15:30.768687 10551 raft_consensus.cc:2764] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:15:30.764219 10512 leader_election.cc:290] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [CANDIDATE]: Term 1 election: Requested vote from peers 28d1d6c02b384aeaa740460051ea351b (127.9.116.130:36353), dd07659d40f84c2aafeaabddc392aeab (127.9.116.131:42413)
I20260709 13:15:30.772154 10353 leader_election.cc:304] T 94c17c5750834fa999137c4d6cce4cf2 P 28d1d6c02b384aeaa740460051ea351b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28d1d6c02b384aeaa740460051ea351b; no voters: dd07659d40f84c2aafeaabddc392aeab, f090b64140fe4e55942c16e20a79b36b
I20260709 13:15:30.773232 10551 raft_consensus.cc:2749] T 94c17c5750834fa999137c4d6cce4cf2 P 28d1d6c02b384aeaa740460051ea351b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:30.769997 10459 raft_consensus.cc:2468] T beb41e866f444744a06e30def55c3190 P dd07659d40f84c2aafeaabddc392aeab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28d1d6c02b384aeaa740460051ea351b in term 1.
I20260709 13:15:30.776772 10548 raft_consensus.cc:493] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:30.777423 10548 raft_consensus.cc:515] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.781961 10506 tablet_bootstrap.cc:654] T b6a956a56be14d83b5f0a8db624c1609 P f090b64140fe4e55942c16e20a79b36b: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:30.791333 10510 raft_consensus.cc:697] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [term 1 LEADER]: Becoming Leader. State: Replica: f090b64140fe4e55942c16e20a79b36b, State: Running, Role: LEADER
I20260709 13:15:30.798103 10548 leader_election.cc:290] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f090b64140fe4e55942c16e20a79b36b (127.9.116.129:42273), 28d1d6c02b384aeaa740460051ea351b (127.9.116.130:36353)
I20260709 13:15:30.800029 10310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b1bc6ab2bb4893a25a192936d09209" candidate_uuid: "dd07659d40f84c2aafeaabddc392aeab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f090b64140fe4e55942c16e20a79b36b" is_pre_election: true
I20260709 13:15:30.800716 10310 raft_consensus.cc:2468] T c5b1bc6ab2bb4893a25a192936d09209 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd07659d40f84c2aafeaabddc392aeab in term 0.
I20260709 13:15:30.801748 10384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b1bc6ab2bb4893a25a192936d09209" candidate_uuid: "dd07659d40f84c2aafeaabddc392aeab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28d1d6c02b384aeaa740460051ea351b" is_pre_election: true
I20260709 13:15:30.802204 10510 consensus_queue.cc:237] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
W20260709 13:15:30.804797 10427 leader_election.cc:343] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28d1d6c02b384aeaa740460051ea351b (127.9.116.130:36353): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:30.806046 10425 leader_election.cc:304] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: dd07659d40f84c2aafeaabddc392aeab, f090b64140fe4e55942c16e20a79b36b; no voters: 28d1d6c02b384aeaa740460051ea351b
I20260709 13:15:30.815786 10548 raft_consensus.cc:2804] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:30.816303 10548 raft_consensus.cc:493] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.816745 10548 raft_consensus.cc:3060] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:30.817114 10506 tablet_bootstrap.cc:492] T b6a956a56be14d83b5f0a8db624c1609 P f090b64140fe4e55942c16e20a79b36b: No bootstrap required, opened a new log
I20260709 13:15:30.817595 10506 ts_tablet_manager.cc:1403] T b6a956a56be14d83b5f0a8db624c1609 P f090b64140fe4e55942c16e20a79b36b: Time spent bootstrapping tablet: real 0.106s	user 0.012s	sys 0.013s
I20260709 13:15:30.818522 10207 catalog_manager.cc:5697] T 7fa6a2759a7d43a28bc364bdd2593069 P 28d1d6c02b384aeaa740460051ea351b reported cstate change: term changed from 0 to 1, leader changed from <none> to 28d1d6c02b384aeaa740460051ea351b (127.9.116.130). New cstate: current_term: 1 leader_uuid: "28d1d6c02b384aeaa740460051ea351b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:30.821760 10502 tablet_bootstrap.cc:492] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b: No bootstrap required, opened a new log
I20260709 13:15:30.822302 10502 ts_tablet_manager.cc:1403] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b: Time spent bootstrapping tablet: real 0.147s	user 0.012s	sys 0.009s
I20260709 13:15:30.815943 10501 tablet_bootstrap.cc:492] T b6a956a56be14d83b5f0a8db624c1609 P dd07659d40f84c2aafeaabddc392aeab: No bootstrap required, opened a new log
I20260709 13:15:30.822379 10506 raft_consensus.cc:359] T b6a956a56be14d83b5f0a8db624c1609 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.823159 10501 ts_tablet_manager.cc:1403] T b6a956a56be14d83b5f0a8db624c1609 P dd07659d40f84c2aafeaabddc392aeab: Time spent bootstrapping tablet: real 0.117s	user 0.010s	sys 0.009s
I20260709 13:15:30.823268 10506 raft_consensus.cc:385] T b6a956a56be14d83b5f0a8db624c1609 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:30.823637 10506 raft_consensus.cc:740] T b6a956a56be14d83b5f0a8db624c1609 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f090b64140fe4e55942c16e20a79b36b, State: Initialized, Role: FOLLOWER
I20260709 13:15:30.824436 10506 consensus_queue.cc:260] T b6a956a56be14d83b5f0a8db624c1609 P f090b64140fe4e55942c16e20a79b36b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.826385 10502 raft_consensus.cc:359] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.826876 10506 ts_tablet_manager.cc:1434] T b6a956a56be14d83b5f0a8db624c1609 P f090b64140fe4e55942c16e20a79b36b: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:15:30.827241 10502 raft_consensus.cc:385] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:30.827632 10502 raft_consensus.cc:740] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28d1d6c02b384aeaa740460051ea351b, State: Initialized, Role: FOLLOWER
I20260709 13:15:30.827972 10556 raft_consensus.cc:493] T b6a956a56be14d83b5f0a8db624c1609 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:30.828428 10556 raft_consensus.cc:515] T b6a956a56be14d83b5f0a8db624c1609 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.828903 10502 consensus_queue.cc:260] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.829923 10548 raft_consensus.cc:515] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.830564 10556 leader_election.cc:290] T b6a956a56be14d83b5f0a8db624c1609 P f090b64140fe4e55942c16e20a79b36b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 28d1d6c02b384aeaa740460051ea351b (127.9.116.130:36353), dd07659d40f84c2aafeaabddc392aeab (127.9.116.131:42413)
I20260709 13:15:30.831209 10502 ts_tablet_manager.cc:1434] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 13:15:30.832096 10502 tablet_bootstrap.cc:492] T b6a956a56be14d83b5f0a8db624c1609 P 28d1d6c02b384aeaa740460051ea351b: Bootstrap starting.
I20260709 13:15:30.827307 10501 raft_consensus.cc:359] T b6a956a56be14d83b5f0a8db624c1609 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.832726 10310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b1bc6ab2bb4893a25a192936d09209" candidate_uuid: "dd07659d40f84c2aafeaabddc392aeab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f090b64140fe4e55942c16e20a79b36b"
I20260709 13:15:30.833069 10384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6a956a56be14d83b5f0a8db624c1609" candidate_uuid: "f090b64140fe4e55942c16e20a79b36b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28d1d6c02b384aeaa740460051ea351b" is_pre_election: true
I20260709 13:15:30.833421 10501 raft_consensus.cc:385] T b6a956a56be14d83b5f0a8db624c1609 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:30.833567 10310 raft_consensus.cc:3060] T c5b1bc6ab2bb4893a25a192936d09209 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:30.833757 10501 raft_consensus.cc:740] T b6a956a56be14d83b5f0a8db624c1609 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd07659d40f84c2aafeaabddc392aeab, State: Initialized, Role: FOLLOWER
I20260709 13:15:30.834223 10460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6a956a56be14d83b5f0a8db624c1609" candidate_uuid: "f090b64140fe4e55942c16e20a79b36b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd07659d40f84c2aafeaabddc392aeab" is_pre_election: true
I20260709 13:15:30.834438 10501 consensus_queue.cc:260] T b6a956a56be14d83b5f0a8db624c1609 P dd07659d40f84c2aafeaabddc392aeab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
W20260709 13:15:30.835718 10278 leader_election.cc:343] T b6a956a56be14d83b5f0a8db624c1609 P f090b64140fe4e55942c16e20a79b36b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28d1d6c02b384aeaa740460051ea351b (127.9.116.130:36353): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:30.836390 10460 raft_consensus.cc:2468] T b6a956a56be14d83b5f0a8db624c1609 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f090b64140fe4e55942c16e20a79b36b in term 0.
I20260709 13:15:30.838392 10278 leader_election.cc:304] T b6a956a56be14d83b5f0a8db624c1609 P f090b64140fe4e55942c16e20a79b36b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: dd07659d40f84c2aafeaabddc392aeab, f090b64140fe4e55942c16e20a79b36b; no voters: 28d1d6c02b384aeaa740460051ea351b
I20260709 13:15:30.839533 10510 raft_consensus.cc:2804] T b6a956a56be14d83b5f0a8db624c1609 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:30.839928 10510 raft_consensus.cc:493] T b6a956a56be14d83b5f0a8db624c1609 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.840276 10510 raft_consensus.cc:3060] T b6a956a56be14d83b5f0a8db624c1609 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:30.840858 10208 catalog_manager.cc:5697] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab reported cstate change: term changed from 0 to 1, leader changed from <none> to dd07659d40f84c2aafeaabddc392aeab (127.9.116.131). New cstate: current_term: 1 leader_uuid: "dd07659d40f84c2aafeaabddc392aeab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:30.841883 10310 raft_consensus.cc:2468] T c5b1bc6ab2bb4893a25a192936d09209 P f090b64140fe4e55942c16e20a79b36b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd07659d40f84c2aafeaabddc392aeab in term 1.
I20260709 13:15:30.843706 10210 catalog_manager.cc:5697] T beb41e866f444744a06e30def55c3190 P 28d1d6c02b384aeaa740460051ea351b reported cstate change: term changed from 0 to 1, leader changed from <none> to 28d1d6c02b384aeaa740460051ea351b (127.9.116.130). New cstate: current_term: 1 leader_uuid: "28d1d6c02b384aeaa740460051ea351b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:30.846091 10425 leader_election.cc:304] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dd07659d40f84c2aafeaabddc392aeab, f090b64140fe4e55942c16e20a79b36b; no voters: 
I20260709 13:15:30.846771 10501 ts_tablet_manager.cc:1434] T b6a956a56be14d83b5f0a8db624c1609 P dd07659d40f84c2aafeaabddc392aeab: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.001s
I20260709 13:15:30.847563 10552 raft_consensus.cc:2804] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:30.848114 10552 raft_consensus.cc:697] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [term 1 LEADER]: Becoming Leader. State: Replica: dd07659d40f84c2aafeaabddc392aeab, State: Running, Role: LEADER
I20260709 13:15:30.848768 10384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b1bc6ab2bb4893a25a192936d09209" candidate_uuid: "dd07659d40f84c2aafeaabddc392aeab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28d1d6c02b384aeaa740460051ea351b"
I20260709 13:15:30.847636 10548 leader_election.cc:290] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [CANDIDATE]: Term 1 election: Requested vote from peers f090b64140fe4e55942c16e20a79b36b (127.9.116.129:42273), 28d1d6c02b384aeaa740460051ea351b (127.9.116.130:36353)
I20260709 13:15:30.849378 10384 raft_consensus.cc:3060] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:30.848987 10552 consensus_queue.cc:237] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.849664 10209 catalog_manager.cc:5697] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b reported cstate change: term changed from 0 to 1, leader changed from <none> to f090b64140fe4e55942c16e20a79b36b (127.9.116.129). New cstate: current_term: 1 leader_uuid: "f090b64140fe4e55942c16e20a79b36b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:30.853314 10510 raft_consensus.cc:515] T b6a956a56be14d83b5f0a8db624c1609 P f090b64140fe4e55942c16e20a79b36b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.855861 10510 leader_election.cc:290] T b6a956a56be14d83b5f0a8db624c1609 P f090b64140fe4e55942c16e20a79b36b [CANDIDATE]: Term 1 election: Requested vote from peers 28d1d6c02b384aeaa740460051ea351b (127.9.116.130:36353), dd07659d40f84c2aafeaabddc392aeab (127.9.116.131:42413)
I20260709 13:15:30.854205 10502 tablet_bootstrap.cc:654] T b6a956a56be14d83b5f0a8db624c1609 P 28d1d6c02b384aeaa740460051ea351b: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:30.857175 10384 raft_consensus.cc:2468] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd07659d40f84c2aafeaabddc392aeab in term 1.
I20260709 13:15:30.863627 10502 tablet_bootstrap.cc:492] T b6a956a56be14d83b5f0a8db624c1609 P 28d1d6c02b384aeaa740460051ea351b: No bootstrap required, opened a new log
I20260709 13:15:30.864189 10502 ts_tablet_manager.cc:1403] T b6a956a56be14d83b5f0a8db624c1609 P 28d1d6c02b384aeaa740460051ea351b: Time spent bootstrapping tablet: real 0.032s	user 0.009s	sys 0.002s
I20260709 13:15:30.864133 10384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6a956a56be14d83b5f0a8db624c1609" candidate_uuid: "f090b64140fe4e55942c16e20a79b36b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28d1d6c02b384aeaa740460051ea351b"
I20260709 13:15:30.864457 10460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6a956a56be14d83b5f0a8db624c1609" candidate_uuid: "f090b64140fe4e55942c16e20a79b36b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd07659d40f84c2aafeaabddc392aeab"
I20260709 13:15:30.865057 10460 raft_consensus.cc:3060] T b6a956a56be14d83b5f0a8db624c1609 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:30.867719 10502 raft_consensus.cc:359] T b6a956a56be14d83b5f0a8db624c1609 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.868703 10502 raft_consensus.cc:385] T b6a956a56be14d83b5f0a8db624c1609 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:30.869124 10502 raft_consensus.cc:740] T b6a956a56be14d83b5f0a8db624c1609 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28d1d6c02b384aeaa740460051ea351b, State: Initialized, Role: FOLLOWER
I20260709 13:15:30.870038 10502 consensus_queue.cc:260] T b6a956a56be14d83b5f0a8db624c1609 P 28d1d6c02b384aeaa740460051ea351b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
W20260709 13:15:30.871199 10278 leader_election.cc:343] T b6a956a56be14d83b5f0a8db624c1609 P f090b64140fe4e55942c16e20a79b36b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 28d1d6c02b384aeaa740460051ea351b (127.9.116.130:36353): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:30.873020 10502 ts_tablet_manager.cc:1434] T b6a956a56be14d83b5f0a8db624c1609 P 28d1d6c02b384aeaa740460051ea351b: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:15:30.842155 10208 catalog_manager.cc:5697] T 9db04314c2c849898b82d951374e2878 P dd07659d40f84c2aafeaabddc392aeab reported cstate change: term changed from 0 to 1, leader changed from <none> to dd07659d40f84c2aafeaabddc392aeab (127.9.116.131). New cstate: current_term: 1 leader_uuid: "dd07659d40f84c2aafeaabddc392aeab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:30.882720 10460 raft_consensus.cc:2468] T b6a956a56be14d83b5f0a8db624c1609 P dd07659d40f84c2aafeaabddc392aeab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f090b64140fe4e55942c16e20a79b36b in term 1.
I20260709 13:15:30.883977 10278 leader_election.cc:304] T b6a956a56be14d83b5f0a8db624c1609 P f090b64140fe4e55942c16e20a79b36b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: dd07659d40f84c2aafeaabddc392aeab, f090b64140fe4e55942c16e20a79b36b; no voters: 28d1d6c02b384aeaa740460051ea351b
I20260709 13:15:30.884776 10510 raft_consensus.cc:2804] T b6a956a56be14d83b5f0a8db624c1609 P f090b64140fe4e55942c16e20a79b36b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:30.885282 10510 raft_consensus.cc:697] T b6a956a56be14d83b5f0a8db624c1609 P f090b64140fe4e55942c16e20a79b36b [term 1 LEADER]: Becoming Leader. State: Replica: f090b64140fe4e55942c16e20a79b36b, State: Running, Role: LEADER
I20260709 13:15:30.886054 10510 consensus_queue.cc:237] T b6a956a56be14d83b5f0a8db624c1609 P f090b64140fe4e55942c16e20a79b36b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.890414 10508 raft_consensus.cc:493] T acb9159770724070b93f9bc026ffee90 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:30.890980 10508 raft_consensus.cc:515] T acb9159770724070b93f9bc026ffee90 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.893707 10508 leader_election.cc:290] T acb9159770724070b93f9bc026ffee90 P 28d1d6c02b384aeaa740460051ea351b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f090b64140fe4e55942c16e20a79b36b (127.9.116.129:42273), dd07659d40f84c2aafeaabddc392aeab (127.9.116.131:42413)
I20260709 13:15:30.895141 10460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb9159770724070b93f9bc026ffee90" candidate_uuid: "28d1d6c02b384aeaa740460051ea351b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd07659d40f84c2aafeaabddc392aeab" is_pre_election: true
I20260709 13:15:30.895385 10310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb9159770724070b93f9bc026ffee90" candidate_uuid: "28d1d6c02b384aeaa740460051ea351b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f090b64140fe4e55942c16e20a79b36b" is_pre_election: true
I20260709 13:15:30.895994 10310 raft_consensus.cc:2468] T acb9159770724070b93f9bc026ffee90 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28d1d6c02b384aeaa740460051ea351b in term 0.
I20260709 13:15:30.896056 10460 raft_consensus.cc:2468] T acb9159770724070b93f9bc026ffee90 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28d1d6c02b384aeaa740460051ea351b in term 0.
I20260709 13:15:30.894590 10210 catalog_manager.cc:5697] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab reported cstate change: term changed from 0 to 1, leader changed from <none> to dd07659d40f84c2aafeaabddc392aeab (127.9.116.131). New cstate: current_term: 1 leader_uuid: "dd07659d40f84c2aafeaabddc392aeab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:30.896922 10351 leader_election.cc:304] T acb9159770724070b93f9bc026ffee90 P 28d1d6c02b384aeaa740460051ea351b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28d1d6c02b384aeaa740460051ea351b, f090b64140fe4e55942c16e20a79b36b; no voters: 
I20260709 13:15:30.897653 10508 raft_consensus.cc:2804] T acb9159770724070b93f9bc026ffee90 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:30.898072 10508 raft_consensus.cc:493] T acb9159770724070b93f9bc026ffee90 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.898459 10508 raft_consensus.cc:3060] T acb9159770724070b93f9bc026ffee90 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:30.905260 10508 raft_consensus.cc:515] T acb9159770724070b93f9bc026ffee90 P 28d1d6c02b384aeaa740460051ea351b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.905792 10208 catalog_manager.cc:5697] T b6a956a56be14d83b5f0a8db624c1609 P f090b64140fe4e55942c16e20a79b36b reported cstate change: term changed from 0 to 1, leader changed from <none> to f090b64140fe4e55942c16e20a79b36b (127.9.116.129). New cstate: current_term: 1 leader_uuid: "f090b64140fe4e55942c16e20a79b36b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:30.908036 10310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb9159770724070b93f9bc026ffee90" candidate_uuid: "28d1d6c02b384aeaa740460051ea351b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f090b64140fe4e55942c16e20a79b36b"
I20260709 13:15:30.908627 10310 raft_consensus.cc:3060] T acb9159770724070b93f9bc026ffee90 P f090b64140fe4e55942c16e20a79b36b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:30.909044 10508 leader_election.cc:290] T acb9159770724070b93f9bc026ffee90 P 28d1d6c02b384aeaa740460051ea351b [CANDIDATE]: Term 1 election: Requested vote from peers f090b64140fe4e55942c16e20a79b36b (127.9.116.129:42273), dd07659d40f84c2aafeaabddc392aeab (127.9.116.131:42413)
I20260709 13:15:30.909583 10460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb9159770724070b93f9bc026ffee90" candidate_uuid: "28d1d6c02b384aeaa740460051ea351b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd07659d40f84c2aafeaabddc392aeab"
I20260709 13:15:30.910154 10460 raft_consensus.cc:3060] T acb9159770724070b93f9bc026ffee90 P dd07659d40f84c2aafeaabddc392aeab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:30.913877 10310 raft_consensus.cc:2468] T acb9159770724070b93f9bc026ffee90 P f090b64140fe4e55942c16e20a79b36b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28d1d6c02b384aeaa740460051ea351b in term 1.
I20260709 13:15:30.914917 10351 leader_election.cc:304] T acb9159770724070b93f9bc026ffee90 P 28d1d6c02b384aeaa740460051ea351b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28d1d6c02b384aeaa740460051ea351b, f090b64140fe4e55942c16e20a79b36b; no voters: 
I20260709 13:15:30.915704 10508 raft_consensus.cc:2804] T acb9159770724070b93f9bc026ffee90 P 28d1d6c02b384aeaa740460051ea351b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:30.916215 10508 raft_consensus.cc:697] T acb9159770724070b93f9bc026ffee90 P 28d1d6c02b384aeaa740460051ea351b [term 1 LEADER]: Becoming Leader. State: Replica: 28d1d6c02b384aeaa740460051ea351b, State: Running, Role: LEADER
I20260709 13:15:30.916971 10508 consensus_queue.cc:237] T acb9159770724070b93f9bc026ffee90 P 28d1d6c02b384aeaa740460051ea351b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:30.917974 10460 raft_consensus.cc:2468] T acb9159770724070b93f9bc026ffee90 P dd07659d40f84c2aafeaabddc392aeab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28d1d6c02b384aeaa740460051ea351b in term 1.
I20260709 13:15:30.924607 10208 catalog_manager.cc:5697] T acb9159770724070b93f9bc026ffee90 P 28d1d6c02b384aeaa740460051ea351b reported cstate change: term changed from 0 to 1, leader changed from <none> to 28d1d6c02b384aeaa740460051ea351b (127.9.116.130). New cstate: current_term: 1 leader_uuid: "28d1d6c02b384aeaa740460051ea351b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:31.038170 10552 consensus_queue.cc:1048] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [LEADER]: Connected to new peer: Peer: permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:31.045750 10552 consensus_queue.cc:1048] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [LEADER]: Connected to new peer: Peer: permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:31.057793 10552 consensus_queue.cc:1048] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [LEADER]: Connected to new peer: Peer: permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:31.058763 10384 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "beb41e866f444744a06e30def55c3190"
dest_uuid: "28d1d6c02b384aeaa740460051ea351b"
mode: GRACEFUL
new_leader_uuid: "f090b64140fe4e55942c16e20a79b36b"
 from {username='slave'} at 127.0.0.1:57266
I20260709 13:15:31.059614 10384 raft_consensus.cc:606] T beb41e866f444744a06e30def55c3190 P 28d1d6c02b384aeaa740460051ea351b [term 1 LEADER]: Received request to transfer leadership to TS f090b64140fe4e55942c16e20a79b36b
I20260709 13:15:31.077711 10384 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7fa6a2759a7d43a28bc364bdd2593069"
dest_uuid: "28d1d6c02b384aeaa740460051ea351b"
mode: GRACEFUL
new_leader_uuid: "f090b64140fe4e55942c16e20a79b36b"
 from {username='slave'} at 127.0.0.1:57266
I20260709 13:15:31.078353 10384 raft_consensus.cc:606] T 7fa6a2759a7d43a28bc364bdd2593069 P 28d1d6c02b384aeaa740460051ea351b [term 1 LEADER]: Received request to transfer leadership to TS f090b64140fe4e55942c16e20a79b36b
I20260709 13:15:31.081019 10548 consensus_queue.cc:1048] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [LEADER]: Connected to new peer: Peer: permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:31.095633 10460 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "09786fc30e8048d9b7928fe7bf969df1"
dest_uuid: "dd07659d40f84c2aafeaabddc392aeab"
mode: GRACEFUL
new_leader_uuid: "f090b64140fe4e55942c16e20a79b36b"
 from {username='slave'} at 127.0.0.1:54420
I20260709 13:15:31.096144 10460 raft_consensus.cc:606] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [term 1 LEADER]: Received request to transfer leadership to TS f090b64140fe4e55942c16e20a79b36b
I20260709 13:15:31.101961 10460 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9db04314c2c849898b82d951374e2878"
dest_uuid: "dd07659d40f84c2aafeaabddc392aeab"
mode: GRACEFUL
new_leader_uuid: "f090b64140fe4e55942c16e20a79b36b"
 from {username='slave'} at 127.0.0.1:54420
I20260709 13:15:31.102469 10460 raft_consensus.cc:606] T 9db04314c2c849898b82d951374e2878 P dd07659d40f84c2aafeaabddc392aeab [term 1 LEADER]: Received request to transfer leadership to TS f090b64140fe4e55942c16e20a79b36b
I20260709 13:15:31.106755 10460 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f7a41f1857af412bb47303bd9452b65f"
dest_uuid: "dd07659d40f84c2aafeaabddc392aeab"
mode: GRACEFUL
new_leader_uuid: "f090b64140fe4e55942c16e20a79b36b"
 from {username='slave'} at 127.0.0.1:54420
I20260709 13:15:31.107246 10460 raft_consensus.cc:606] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [term 1 LEADER]: Received request to transfer leadership to TS f090b64140fe4e55942c16e20a79b36b
I20260709 13:15:31.121670 10310 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "135e8982a8ea4fa198de62d0e60b459c"
dest_uuid: "f090b64140fe4e55942c16e20a79b36b"
mode: GRACEFUL
new_leader_uuid: "28d1d6c02b384aeaa740460051ea351b"
 from {username='slave'} at 127.0.0.1:37710
I20260709 13:15:31.122153 10310 raft_consensus.cc:606] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [term 1 LEADER]: Received request to transfer leadership to TS 28d1d6c02b384aeaa740460051ea351b
I20260709 13:15:31.127216 10460 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "94c17c5750834fa999137c4d6cce4cf2"
dest_uuid: "dd07659d40f84c2aafeaabddc392aeab"
mode: GRACEFUL
new_leader_uuid: "28d1d6c02b384aeaa740460051ea351b"
 from {username='slave'} at 127.0.0.1:54420
I20260709 13:15:31.127703 10460 raft_consensus.cc:606] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab [term 1 LEADER]: Received request to transfer leadership to TS 28d1d6c02b384aeaa740460051ea351b
I20260709 13:15:31.132164 10460 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c5b1bc6ab2bb4893a25a192936d09209"
dest_uuid: "dd07659d40f84c2aafeaabddc392aeab"
mode: GRACEFUL
new_leader_uuid: "28d1d6c02b384aeaa740460051ea351b"
 from {username='slave'} at 127.0.0.1:54420
I20260709 13:15:31.132613 10460 raft_consensus.cc:606] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [term 1 LEADER]: Received request to transfer leadership to TS 28d1d6c02b384aeaa740460051ea351b
I20260709 13:15:31.139582 10310 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b6a956a56be14d83b5f0a8db624c1609"
dest_uuid: "f090b64140fe4e55942c16e20a79b36b"
mode: GRACEFUL
new_leader_uuid: "dd07659d40f84c2aafeaabddc392aeab"
 from {username='slave'} at 127.0.0.1:37710
I20260709 13:15:31.140030 10310 raft_consensus.cc:606] T b6a956a56be14d83b5f0a8db624c1609 P f090b64140fe4e55942c16e20a79b36b [term 1 LEADER]: Received request to transfer leadership to TS dd07659d40f84c2aafeaabddc392aeab
I20260709 13:15:31.145082 10531 consensus_queue.cc:1048] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab [LEADER]: Connected to new peer: Peer: permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:31.151211 10548 raft_consensus.cc:993] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab: : Instructing follower 28d1d6c02b384aeaa740460051ea351b to start an election
I20260709 13:15:31.151670 10552 raft_consensus.cc:1081] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab [term 1 LEADER]: Signalling peer 28d1d6c02b384aeaa740460051ea351b to start an election
I20260709 13:15:31.153841 10384 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "94c17c5750834fa999137c4d6cce4cf2"
dest_uuid: "28d1d6c02b384aeaa740460051ea351b"
 from {username='slave'} at 127.0.0.1:57264
I20260709 13:15:31.154474 10384 raft_consensus.cc:493] T 94c17c5750834fa999137c4d6cce4cf2 P 28d1d6c02b384aeaa740460051ea351b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:31.154898 10384 raft_consensus.cc:3060] T 94c17c5750834fa999137c4d6cce4cf2 P 28d1d6c02b384aeaa740460051ea351b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:31.157650 10552 consensus_queue.cc:1048] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab [LEADER]: Connected to new peer: Peer: permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:31.160307 10384 raft_consensus.cc:515] T 94c17c5750834fa999137c4d6cce4cf2 P 28d1d6c02b384aeaa740460051ea351b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:31.164100 10311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c17c5750834fa999137c4d6cce4cf2" candidate_uuid: "28d1d6c02b384aeaa740460051ea351b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f090b64140fe4e55942c16e20a79b36b"
I20260709 13:15:31.165772 10384 leader_election.cc:290] T 94c17c5750834fa999137c4d6cce4cf2 P 28d1d6c02b384aeaa740460051ea351b [CANDIDATE]: Term 2 election: Requested vote from peers f090b64140fe4e55942c16e20a79b36b (127.9.116.129:42273), dd07659d40f84c2aafeaabddc392aeab (127.9.116.131:42413)
I20260709 13:15:31.166339 10460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c17c5750834fa999137c4d6cce4cf2" candidate_uuid: "28d1d6c02b384aeaa740460051ea351b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dd07659d40f84c2aafeaabddc392aeab"
I20260709 13:15:31.166951 10460 raft_consensus.cc:3055] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:31.167251 10460 raft_consensus.cc:740] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dd07659d40f84c2aafeaabddc392aeab, State: Running, Role: LEADER
I20260709 13:15:31.168051 10460 consensus_queue.cc:260] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:31.169261 10460 raft_consensus.cc:3060] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:31.175863 10460 raft_consensus.cc:2468] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28d1d6c02b384aeaa740460051ea351b in term 2.
I20260709 13:15:31.177029 10353 leader_election.cc:304] T 94c17c5750834fa999137c4d6cce4cf2 P 28d1d6c02b384aeaa740460051ea351b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28d1d6c02b384aeaa740460051ea351b, dd07659d40f84c2aafeaabddc392aeab; no voters: f090b64140fe4e55942c16e20a79b36b
I20260709 13:15:31.177848 10508 raft_consensus.cc:2804] T 94c17c5750834fa999137c4d6cce4cf2 P 28d1d6c02b384aeaa740460051ea351b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:31.178326 10508 raft_consensus.cc:697] T 94c17c5750834fa999137c4d6cce4cf2 P 28d1d6c02b384aeaa740460051ea351b [term 2 LEADER]: Becoming Leader. State: Replica: 28d1d6c02b384aeaa740460051ea351b, State: Running, Role: LEADER
I20260709 13:15:31.179030 10508 consensus_queue.cc:237] T 94c17c5750834fa999137c4d6cce4cf2 P 28d1d6c02b384aeaa740460051ea351b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:31.186745 10510 consensus_queue.cc:1048] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [LEADER]: Connected to new peer: Peer: permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:31.186409 10210 catalog_manager.cc:5697] T 94c17c5750834fa999137c4d6cce4cf2 P 28d1d6c02b384aeaa740460051ea351b reported cstate change: term changed from 1 to 2, leader changed from dd07659d40f84c2aafeaabddc392aeab (127.9.116.131) to 28d1d6c02b384aeaa740460051ea351b (127.9.116.130). New cstate: current_term: 2 leader_uuid: "28d1d6c02b384aeaa740460051ea351b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:31.193444 10556 raft_consensus.cc:993] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b: : Instructing follower 28d1d6c02b384aeaa740460051ea351b to start an election
I20260709 13:15:31.193913 10512 raft_consensus.cc:1081] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [term 1 LEADER]: Signalling peer 28d1d6c02b384aeaa740460051ea351b to start an election
I20260709 13:15:31.198662 10552 consensus_queue.cc:1048] T 9db04314c2c849898b82d951374e2878 P dd07659d40f84c2aafeaabddc392aeab [LEADER]: Connected to new peer: Peer: permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:31.199318 10385 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "135e8982a8ea4fa198de62d0e60b459c"
dest_uuid: "28d1d6c02b384aeaa740460051ea351b"
 from {username='slave'} at 127.0.0.1:57248
I20260709 13:15:31.199929 10385 raft_consensus.cc:493] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:31.200340 10385 raft_consensus.cc:3060] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:31.203305 10510 consensus_queue.cc:1048] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:31.207820 10385 raft_consensus.cc:515] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:31.211766 10310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e8982a8ea4fa198de62d0e60b459c" candidate_uuid: "28d1d6c02b384aeaa740460051ea351b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f090b64140fe4e55942c16e20a79b36b"
I20260709 13:15:31.212559 10310 raft_consensus.cc:3055] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:31.212976 10310 raft_consensus.cc:740] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f090b64140fe4e55942c16e20a79b36b, State: Running, Role: LEADER
I20260709 13:15:31.214026 10310 consensus_queue.cc:260] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:31.215327 10310 raft_consensus.cc:3060] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:31.216495 10508 consensus_queue.cc:1048] T 7fa6a2759a7d43a28bc364bdd2593069 P 28d1d6c02b384aeaa740460051ea351b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:31.223417 10548 consensus_queue.cc:1048] T 9db04314c2c849898b82d951374e2878 P dd07659d40f84c2aafeaabddc392aeab [LEADER]: Connected to new peer: Peer: permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:31.223788 10310 raft_consensus.cc:2468] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28d1d6c02b384aeaa740460051ea351b in term 2.
I20260709 13:15:31.224980 10351 leader_election.cc:304] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28d1d6c02b384aeaa740460051ea351b, f090b64140fe4e55942c16e20a79b36b; no voters: 
I20260709 13:15:31.225849 10508 raft_consensus.cc:2804] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:31.226327 10508 raft_consensus.cc:697] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b [term 2 LEADER]: Becoming Leader. State: Replica: 28d1d6c02b384aeaa740460051ea351b, State: Running, Role: LEADER
I20260709 13:15:31.229877 10385 leader_election.cc:290] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b [CANDIDATE]: Term 2 election: Requested vote from peers f090b64140fe4e55942c16e20a79b36b (127.9.116.129:42273), dd07659d40f84c2aafeaabddc392aeab (127.9.116.131:42413)
I20260709 13:15:31.230446 10461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e8982a8ea4fa198de62d0e60b459c" candidate_uuid: "28d1d6c02b384aeaa740460051ea351b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dd07659d40f84c2aafeaabddc392aeab"
I20260709 13:15:31.232007 10551 consensus_queue.cc:1048] T beb41e866f444744a06e30def55c3190 P 28d1d6c02b384aeaa740460051ea351b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:31.227186 10508 consensus_queue.cc:237] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:31.247345 10551 consensus_queue.cc:1048] T 7fa6a2759a7d43a28bc364bdd2593069 P 28d1d6c02b384aeaa740460051ea351b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:31.249228 10546 raft_consensus.cc:993] T beb41e866f444744a06e30def55c3190 P 28d1d6c02b384aeaa740460051ea351b: : Instructing follower f090b64140fe4e55942c16e20a79b36b to start an election
I20260709 13:15:31.249900 10551 raft_consensus.cc:1081] T beb41e866f444744a06e30def55c3190 P 28d1d6c02b384aeaa740460051ea351b [term 1 LEADER]: Signalling peer f090b64140fe4e55942c16e20a79b36b to start an election
I20260709 13:15:31.264866 10312 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "beb41e866f444744a06e30def55c3190"
dest_uuid: "f090b64140fe4e55942c16e20a79b36b"
 from {username='slave'} at 127.0.0.1:37706
I20260709 13:15:31.264503 10546 consensus_queue.cc:1048] T beb41e866f444744a06e30def55c3190 P 28d1d6c02b384aeaa740460051ea351b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:31.265483 10312 raft_consensus.cc:493] T beb41e866f444744a06e30def55c3190 P f090b64140fe4e55942c16e20a79b36b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:31.265952 10312 raft_consensus.cc:3060] T beb41e866f444744a06e30def55c3190 P f090b64140fe4e55942c16e20a79b36b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:31.273579 10551 raft_consensus.cc:993] T 7fa6a2759a7d43a28bc364bdd2593069 P 28d1d6c02b384aeaa740460051ea351b: : Instructing follower f090b64140fe4e55942c16e20a79b36b to start an election
I20260709 13:15:31.274073 10551 raft_consensus.cc:1081] T 7fa6a2759a7d43a28bc364bdd2593069 P 28d1d6c02b384aeaa740460051ea351b [term 1 LEADER]: Signalling peer f090b64140fe4e55942c16e20a79b36b to start an election
I20260709 13:15:31.286216 10210 catalog_manager.cc:5697] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b reported cstate change: term changed from 1 to 2, leader changed from f090b64140fe4e55942c16e20a79b36b (127.9.116.129) to 28d1d6c02b384aeaa740460051ea351b (127.9.116.130). New cstate: current_term: 2 leader_uuid: "28d1d6c02b384aeaa740460051ea351b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:31.290550 10312 raft_consensus.cc:515] T beb41e866f444744a06e30def55c3190 P f090b64140fe4e55942c16e20a79b36b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } } peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } }
I20260709 13:15:31.292665 10312 leader_election.cc:290] T beb41e866f444744a06e30def55c3190 P f090b64140fe4e55942c16e20a79b36b [CANDIDATE]: Term 2 election: Requested vote from peers dd07659d40f84c2aafeaabddc392aeab (127.9.116.131:42413), 28d1d6c02b384aeaa740460051ea351b (127.9.116.130:36353)
I20260709 13:15:31.285207 10310 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7fa6a2759a7d43a28bc364bdd2593069"
dest_uuid: "f090b64140fe4e55942c16e20a79b36b"
 from {username='slave'} at 127.0.0.1:37706
I20260709 13:15:31.293810 10310 raft_consensus.cc:493] T 7fa6a2759a7d43a28bc364bdd2593069 P f090b64140fe4e55942c16e20a79b36b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:31.294525 10310 raft_consensus.cc:3060] T 7fa6a2759a7d43a28bc364bdd2593069 P f090b64140fe4e55942c16e20a79b36b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:31.296239 10461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "beb41e866f444744a06e30def55c3190" candidate_uuid: "f090b64140fe4e55942c16e20a79b36b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dd07659d40f84c2aafeaabddc392aeab"
I20260709 13:15:31.296828 10461 raft_consensus.cc:3060] T beb41e866f444744a06e30def55c3190 P dd07659d40f84c2aafeaabddc392aeab [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:31.303882 10385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "beb41e866f444744a06e30def55c3190" candidate_uuid: "f090b64140fe4e55942c16e20a79b36b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "28d1d6c02b384aeaa740460051ea351b"
I20260709 13:15:31.304509 10385 raft_consensus.cc:3055] T beb41e866f444744a06e30def55c3190 P 28d1d6c02b384aeaa740460051ea351b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:31.304850 10385 raft_consensus.cc:740] T beb41e866f444744a06e30def55c3190 P 28d1d6c02b384aeaa740460051ea351b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 28d1d6c02b384aeaa740460051ea351b, State: Running, Role: LEADER
I20260709 13:15:31.305630 10385 consensus_queue.cc:260] T beb41e866f444744a06e30def55c3190 P 28d1d6c02b384aeaa740460051ea351b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } } peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } }
I20260709 13:15:31.305763 10310 raft_consensus.cc:515] T 7fa6a2759a7d43a28bc364bdd2593069 P f090b64140fe4e55942c16e20a79b36b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:31.306886 10385 raft_consensus.cc:3060] T beb41e866f444744a06e30def55c3190 P 28d1d6c02b384aeaa740460051ea351b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:31.310837 10310 leader_election.cc:290] T 7fa6a2759a7d43a28bc364bdd2593069 P f090b64140fe4e55942c16e20a79b36b [CANDIDATE]: Term 2 election: Requested vote from peers 28d1d6c02b384aeaa740460051ea351b (127.9.116.130:36353), dd07659d40f84c2aafeaabddc392aeab (127.9.116.131:42413)
I20260709 13:15:31.311036 10531 raft_consensus.cc:993] T 9db04314c2c849898b82d951374e2878 P dd07659d40f84c2aafeaabddc392aeab: : Instructing follower f090b64140fe4e55942c16e20a79b36b to start an election
I20260709 13:15:31.311507 10531 raft_consensus.cc:1081] T 9db04314c2c849898b82d951374e2878 P dd07659d40f84c2aafeaabddc392aeab [term 1 LEADER]: Signalling peer f090b64140fe4e55942c16e20a79b36b to start an election
I20260709 13:15:31.312758 10461 raft_consensus.cc:2468] T beb41e866f444744a06e30def55c3190 P dd07659d40f84c2aafeaabddc392aeab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f090b64140fe4e55942c16e20a79b36b in term 2.
I20260709 13:15:31.313383 10310 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9db04314c2c849898b82d951374e2878"
dest_uuid: "f090b64140fe4e55942c16e20a79b36b"
 from {username='slave'} at 127.0.0.1:37692
I20260709 13:15:31.314182 10278 leader_election.cc:304] T beb41e866f444744a06e30def55c3190 P f090b64140fe4e55942c16e20a79b36b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dd07659d40f84c2aafeaabddc392aeab, f090b64140fe4e55942c16e20a79b36b; no voters: 
I20260709 13:15:31.314469 10310 raft_consensus.cc:493] T 9db04314c2c849898b82d951374e2878 P f090b64140fe4e55942c16e20a79b36b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:31.314930 10310 raft_consensus.cc:3060] T 9db04314c2c849898b82d951374e2878 P f090b64140fe4e55942c16e20a79b36b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:31.315083 10510 raft_consensus.cc:2804] T beb41e866f444744a06e30def55c3190 P f090b64140fe4e55942c16e20a79b36b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:31.315687 10510 raft_consensus.cc:697] T beb41e866f444744a06e30def55c3190 P f090b64140fe4e55942c16e20a79b36b [term 2 LEADER]: Becoming Leader. State: Replica: f090b64140fe4e55942c16e20a79b36b, State: Running, Role: LEADER
I20260709 13:15:31.316115 10384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fa6a2759a7d43a28bc364bdd2593069" candidate_uuid: "f090b64140fe4e55942c16e20a79b36b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "28d1d6c02b384aeaa740460051ea351b"
I20260709 13:15:31.316735 10384 raft_consensus.cc:3055] T 7fa6a2759a7d43a28bc364bdd2593069 P 28d1d6c02b384aeaa740460051ea351b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:31.316574 10510 consensus_queue.cc:237] T beb41e866f444744a06e30def55c3190 P f090b64140fe4e55942c16e20a79b36b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } } peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } }
I20260709 13:15:31.317073 10384 raft_consensus.cc:740] T 7fa6a2759a7d43a28bc364bdd2593069 P 28d1d6c02b384aeaa740460051ea351b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 28d1d6c02b384aeaa740460051ea351b, State: Running, Role: LEADER
I20260709 13:15:31.317914 10384 consensus_queue.cc:260] T 7fa6a2759a7d43a28bc364bdd2593069 P 28d1d6c02b384aeaa740460051ea351b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:31.319259 10384 raft_consensus.cc:3060] T 7fa6a2759a7d43a28bc364bdd2593069 P 28d1d6c02b384aeaa740460051ea351b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:31.320550 10548 consensus_queue.cc:1048] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [LEADER]: Connected to new peer: Peer: permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:31.325287 10386 raft_consensus.cc:3060] T b6a956a56be14d83b5f0a8db624c1609 P 28d1d6c02b384aeaa740460051ea351b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:31.325161 10310 raft_consensus.cc:515] T 9db04314c2c849898b82d951374e2878 P f090b64140fe4e55942c16e20a79b36b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:31.328181 10310 leader_election.cc:290] T 9db04314c2c849898b82d951374e2878 P f090b64140fe4e55942c16e20a79b36b [CANDIDATE]: Term 2 election: Requested vote from peers 28d1d6c02b384aeaa740460051ea351b (127.9.116.130:36353), dd07659d40f84c2aafeaabddc392aeab (127.9.116.131:42413)
I20260709 13:15:31.330232 10387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db04314c2c849898b82d951374e2878" candidate_uuid: "f090b64140fe4e55942c16e20a79b36b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "28d1d6c02b384aeaa740460051ea351b"
I20260709 13:15:31.330925 10387 raft_consensus.cc:3060] T 9db04314c2c849898b82d951374e2878 P 28d1d6c02b384aeaa740460051ea351b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:31.334241 10508 raft_consensus.cc:493] T b6a956a56be14d83b5f0a8db624c1609 P 28d1d6c02b384aeaa740460051ea351b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:31.334761 10508 raft_consensus.cc:515] T b6a956a56be14d83b5f0a8db624c1609 P 28d1d6c02b384aeaa740460051ea351b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:31.336495 10387 raft_consensus.cc:2468] T 9db04314c2c849898b82d951374e2878 P 28d1d6c02b384aeaa740460051ea351b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f090b64140fe4e55942c16e20a79b36b in term 2.
I20260709 13:15:31.335814 10461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fa6a2759a7d43a28bc364bdd2593069" candidate_uuid: "f090b64140fe4e55942c16e20a79b36b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dd07659d40f84c2aafeaabddc392aeab"
I20260709 13:15:31.337733 10278 leader_election.cc:304] T 9db04314c2c849898b82d951374e2878 P f090b64140fe4e55942c16e20a79b36b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28d1d6c02b384aeaa740460051ea351b, f090b64140fe4e55942c16e20a79b36b; no voters: 
I20260709 13:15:31.339219 10460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6a956a56be14d83b5f0a8db624c1609" candidate_uuid: "28d1d6c02b384aeaa740460051ea351b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd07659d40f84c2aafeaabddc392aeab" is_pre_election: true
I20260709 13:15:31.339552 10310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6a956a56be14d83b5f0a8db624c1609" candidate_uuid: "28d1d6c02b384aeaa740460051ea351b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f090b64140fe4e55942c16e20a79b36b" is_pre_election: true
I20260709 13:15:31.339788 10510 raft_consensus.cc:2804] T 9db04314c2c849898b82d951374e2878 P f090b64140fe4e55942c16e20a79b36b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:31.339869 10460 raft_consensus.cc:2468] T b6a956a56be14d83b5f0a8db624c1609 P dd07659d40f84c2aafeaabddc392aeab [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28d1d6c02b384aeaa740460051ea351b in term 1.
I20260709 13:15:31.341292 10510 raft_consensus.cc:697] T 9db04314c2c849898b82d951374e2878 P f090b64140fe4e55942c16e20a79b36b [term 2 LEADER]: Becoming Leader. State: Replica: f090b64140fe4e55942c16e20a79b36b, State: Running, Role: LEADER
I20260709 13:15:31.342047 10353 leader_election.cc:304] T b6a956a56be14d83b5f0a8db624c1609 P 28d1d6c02b384aeaa740460051ea351b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28d1d6c02b384aeaa740460051ea351b, dd07659d40f84c2aafeaabddc392aeab; no voters: f090b64140fe4e55942c16e20a79b36b
I20260709 13:15:31.342939 10546 raft_consensus.cc:2804] T b6a956a56be14d83b5f0a8db624c1609 P 28d1d6c02b384aeaa740460051ea351b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:31.343362 10546 raft_consensus.cc:493] T b6a956a56be14d83b5f0a8db624c1609 P 28d1d6c02b384aeaa740460051ea351b [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:31.343765 10546 raft_consensus.cc:3060] T b6a956a56be14d83b5f0a8db624c1609 P 28d1d6c02b384aeaa740460051ea351b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:31.345443 10210 catalog_manager.cc:5697] T beb41e866f444744a06e30def55c3190 P f090b64140fe4e55942c16e20a79b36b reported cstate change: term changed from 1 to 2, leader changed from 28d1d6c02b384aeaa740460051ea351b (127.9.116.130) to f090b64140fe4e55942c16e20a79b36b (127.9.116.129). New cstate: current_term: 2 leader_uuid: "f090b64140fe4e55942c16e20a79b36b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:31.346463 10552 consensus_queue.cc:1048] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [LEADER]: Connected to new peer: Peer: permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:31.348778 10584 consensus_queue.cc:1048] T acb9159770724070b93f9bc026ffee90 P 28d1d6c02b384aeaa740460051ea351b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:31.352659 10459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db04314c2c849898b82d951374e2878" candidate_uuid: "f090b64140fe4e55942c16e20a79b36b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dd07659d40f84c2aafeaabddc392aeab"
I20260709 13:15:31.353293 10459 raft_consensus.cc:3055] T 9db04314c2c849898b82d951374e2878 P dd07659d40f84c2aafeaabddc392aeab [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:31.353614 10459 raft_consensus.cc:740] T 9db04314c2c849898b82d951374e2878 P dd07659d40f84c2aafeaabddc392aeab [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dd07659d40f84c2aafeaabddc392aeab, State: Running, Role: LEADER
I20260709 13:15:31.354465 10459 consensus_queue.cc:260] T 9db04314c2c849898b82d951374e2878 P dd07659d40f84c2aafeaabddc392aeab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:31.342962 10510 consensus_queue.cc:237] T 9db04314c2c849898b82d951374e2878 P f090b64140fe4e55942c16e20a79b36b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:31.355856 10459 raft_consensus.cc:3060] T 9db04314c2c849898b82d951374e2878 P dd07659d40f84c2aafeaabddc392aeab [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:31.358892 10546 raft_consensus.cc:515] T b6a956a56be14d83b5f0a8db624c1609 P 28d1d6c02b384aeaa740460051ea351b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:31.358928 10508 leader_election.cc:290] T b6a956a56be14d83b5f0a8db624c1609 P 28d1d6c02b384aeaa740460051ea351b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f090b64140fe4e55942c16e20a79b36b (127.9.116.129:42273), dd07659d40f84c2aafeaabddc392aeab (127.9.116.131:42413)
I20260709 13:15:31.343140 10461 raft_consensus.cc:3060] T 7fa6a2759a7d43a28bc364bdd2593069 P dd07659d40f84c2aafeaabddc392aeab [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:31.365592 10384 raft_consensus.cc:2468] T 7fa6a2759a7d43a28bc364bdd2593069 P 28d1d6c02b384aeaa740460051ea351b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f090b64140fe4e55942c16e20a79b36b in term 2.
I20260709 13:15:31.361476 10510 consensus_queue.cc:1048] T b6a956a56be14d83b5f0a8db624c1609 P f090b64140fe4e55942c16e20a79b36b [LEADER]: Connected to new peer: Peer: permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:31.366475 10210 catalog_manager.cc:5697] T 9db04314c2c849898b82d951374e2878 P f090b64140fe4e55942c16e20a79b36b reported cstate change: term changed from 1 to 2, leader changed from dd07659d40f84c2aafeaabddc392aeab (127.9.116.131) to f090b64140fe4e55942c16e20a79b36b (127.9.116.129). New cstate: current_term: 2 leader_uuid: "f090b64140fe4e55942c16e20a79b36b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:31.367152 10278 leader_election.cc:304] T 7fa6a2759a7d43a28bc364bdd2593069 P f090b64140fe4e55942c16e20a79b36b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28d1d6c02b384aeaa740460051ea351b, f090b64140fe4e55942c16e20a79b36b; no voters: 
I20260709 13:15:31.368069 10512 raft_consensus.cc:2804] T 7fa6a2759a7d43a28bc364bdd2593069 P f090b64140fe4e55942c16e20a79b36b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:31.368561 10512 raft_consensus.cc:697] T 7fa6a2759a7d43a28bc364bdd2593069 P f090b64140fe4e55942c16e20a79b36b [term 2 LEADER]: Becoming Leader. State: Replica: f090b64140fe4e55942c16e20a79b36b, State: Running, Role: LEADER
I20260709 13:15:31.369454 10512 consensus_queue.cc:237] T 7fa6a2759a7d43a28bc364bdd2593069 P f090b64140fe4e55942c16e20a79b36b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:31.338680 10385 raft_consensus.cc:2468] T beb41e866f444744a06e30def55c3190 P 28d1d6c02b384aeaa740460051ea351b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f090b64140fe4e55942c16e20a79b36b in term 2.
I20260709 13:15:31.369446 10546 leader_election.cc:290] T b6a956a56be14d83b5f0a8db624c1609 P 28d1d6c02b384aeaa740460051ea351b [CANDIDATE]: Term 2 election: Requested vote from peers f090b64140fe4e55942c16e20a79b36b (127.9.116.129:42273), dd07659d40f84c2aafeaabddc392aeab (127.9.116.131:42413)
I20260709 13:15:31.376507 10458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6a956a56be14d83b5f0a8db624c1609" candidate_uuid: "28d1d6c02b384aeaa740460051ea351b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd07659d40f84c2aafeaabddc392aeab"
I20260709 13:15:31.376155 10311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6a956a56be14d83b5f0a8db624c1609" candidate_uuid: "28d1d6c02b384aeaa740460051ea351b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f090b64140fe4e55942c16e20a79b36b"
I20260709 13:15:31.381150 10384 raft_consensus.cc:1240] T b6a956a56be14d83b5f0a8db624c1609 P 28d1d6c02b384aeaa740460051ea351b [term 2 FOLLOWER]: Rejecting Update request from peer f090b64140fe4e55942c16e20a79b36b for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:15:31.388535 10458 raft_consensus.cc:3060] T b6a956a56be14d83b5f0a8db624c1609 P dd07659d40f84c2aafeaabddc392aeab [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:31.394817 10207 catalog_manager.cc:5697] T 7fa6a2759a7d43a28bc364bdd2593069 P f090b64140fe4e55942c16e20a79b36b reported cstate change: term changed from 1 to 2, leader changed from 28d1d6c02b384aeaa740460051ea351b (127.9.116.130) to f090b64140fe4e55942c16e20a79b36b (127.9.116.129). New cstate: current_term: 2 leader_uuid: "f090b64140fe4e55942c16e20a79b36b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:31.400055 10461 raft_consensus.cc:2468] T 7fa6a2759a7d43a28bc364bdd2593069 P dd07659d40f84c2aafeaabddc392aeab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f090b64140fe4e55942c16e20a79b36b in term 2.
I20260709 13:15:31.400302 10459 raft_consensus.cc:2468] T 9db04314c2c849898b82d951374e2878 P dd07659d40f84c2aafeaabddc392aeab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f090b64140fe4e55942c16e20a79b36b in term 2.
I20260709 13:15:31.404371 10551 consensus_queue.cc:1048] T acb9159770724070b93f9bc026ffee90 P 28d1d6c02b384aeaa740460051ea351b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:31.404757 10552 raft_consensus.cc:993] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab: : Instructing follower 28d1d6c02b384aeaa740460051ea351b to start an election
I20260709 13:15:31.409626 10552 raft_consensus.cc:1081] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [term 1 LEADER]: Signalling peer 28d1d6c02b384aeaa740460051ea351b to start an election
I20260709 13:15:31.406435 10512 consensus_queue.cc:1059] T b6a956a56be14d83b5f0a8db624c1609 P f090b64140fe4e55942c16e20a79b36b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:31.411676 10556 raft_consensus.cc:3055] T b6a956a56be14d83b5f0a8db624c1609 P f090b64140fe4e55942c16e20a79b36b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:31.411865 10387 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c5b1bc6ab2bb4893a25a192936d09209"
dest_uuid: "28d1d6c02b384aeaa740460051ea351b"
 from {username='slave'} at 127.0.0.1:57264
I20260709 13:15:31.412077 10556 raft_consensus.cc:740] T b6a956a56be14d83b5f0a8db624c1609 P f090b64140fe4e55942c16e20a79b36b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f090b64140fe4e55942c16e20a79b36b, State: Running, Role: LEADER
I20260709 13:15:31.412495 10387 raft_consensus.cc:493] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:31.412891 10387 raft_consensus.cc:3060] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:31.412948 10556 consensus_queue.cc:260] T b6a956a56be14d83b5f0a8db624c1609 P f090b64140fe4e55942c16e20a79b36b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:31.414228 10556 raft_consensus.cc:3060] T b6a956a56be14d83b5f0a8db624c1609 P f090b64140fe4e55942c16e20a79b36b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:31.407267 10458 raft_consensus.cc:2468] T b6a956a56be14d83b5f0a8db624c1609 P dd07659d40f84c2aafeaabddc392aeab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28d1d6c02b384aeaa740460051ea351b in term 2.
I20260709 13:15:31.415526 10460 raft_consensus.cc:1240] T b6a956a56be14d83b5f0a8db624c1609 P dd07659d40f84c2aafeaabddc392aeab [term 2 FOLLOWER]: Rejecting Update request from peer f090b64140fe4e55942c16e20a79b36b for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:31.419837 10353 leader_election.cc:304] T b6a956a56be14d83b5f0a8db624c1609 P 28d1d6c02b384aeaa740460051ea351b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28d1d6c02b384aeaa740460051ea351b, dd07659d40f84c2aafeaabddc392aeab; no voters: f090b64140fe4e55942c16e20a79b36b
I20260709 13:15:31.420923 10551 raft_consensus.cc:2804] T b6a956a56be14d83b5f0a8db624c1609 P 28d1d6c02b384aeaa740460051ea351b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:31.421460 10551 raft_consensus.cc:697] T b6a956a56be14d83b5f0a8db624c1609 P 28d1d6c02b384aeaa740460051ea351b [term 2 LEADER]: Becoming Leader. State: Replica: 28d1d6c02b384aeaa740460051ea351b, State: Running, Role: LEADER
I20260709 13:15:31.422343 10551 consensus_queue.cc:237] T b6a956a56be14d83b5f0a8db624c1609 P 28d1d6c02b384aeaa740460051ea351b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:31.429343 10387 raft_consensus.cc:515] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:31.432403 10310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b1bc6ab2bb4893a25a192936d09209" candidate_uuid: "28d1d6c02b384aeaa740460051ea351b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f090b64140fe4e55942c16e20a79b36b"
I20260709 13:15:31.433069 10310 raft_consensus.cc:3060] T c5b1bc6ab2bb4893a25a192936d09209 P f090b64140fe4e55942c16e20a79b36b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:31.432477 10208 catalog_manager.cc:5697] T b6a956a56be14d83b5f0a8db624c1609 P 28d1d6c02b384aeaa740460051ea351b reported cstate change: term changed from 1 to 2, leader changed from f090b64140fe4e55942c16e20a79b36b (127.9.116.129) to 28d1d6c02b384aeaa740460051ea351b (127.9.116.130). New cstate: current_term: 2 leader_uuid: "28d1d6c02b384aeaa740460051ea351b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:31.440320 10310 raft_consensus.cc:2468] T c5b1bc6ab2bb4893a25a192936d09209 P f090b64140fe4e55942c16e20a79b36b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28d1d6c02b384aeaa740460051ea351b in term 2.
I20260709 13:15:31.441680 10351 leader_election.cc:304] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28d1d6c02b384aeaa740460051ea351b, f090b64140fe4e55942c16e20a79b36b; no voters: 
I20260709 13:15:31.442539 10551 raft_consensus.cc:2804] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:31.443080 10551 raft_consensus.cc:697] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b [term 2 LEADER]: Becoming Leader. State: Replica: 28d1d6c02b384aeaa740460051ea351b, State: Running, Role: LEADER
I20260709 13:15:31.443990 10551 consensus_queue.cc:237] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:31.444981 10458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b1bc6ab2bb4893a25a192936d09209" candidate_uuid: "28d1d6c02b384aeaa740460051ea351b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dd07659d40f84c2aafeaabddc392aeab"
I20260709 13:15:31.445624 10458 raft_consensus.cc:3055] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:31.445988 10458 raft_consensus.cc:740] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dd07659d40f84c2aafeaabddc392aeab, State: Running, Role: LEADER
I20260709 13:15:31.446965 10458 consensus_queue.cc:260] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:31.448562 10387 leader_election.cc:290] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b [CANDIDATE]: Term 2 election: Requested vote from peers f090b64140fe4e55942c16e20a79b36b (127.9.116.129:42273), dd07659d40f84c2aafeaabddc392aeab (127.9.116.131:42413)
I20260709 13:15:31.448101 10458 raft_consensus.cc:3060] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:31.454236 10208 catalog_manager.cc:5697] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b reported cstate change: term changed from 1 to 2, leader changed from dd07659d40f84c2aafeaabddc392aeab (127.9.116.131) to 28d1d6c02b384aeaa740460051ea351b (127.9.116.130). New cstate: current_term: 2 leader_uuid: "28d1d6c02b384aeaa740460051ea351b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:31.458019 10458 raft_consensus.cc:2468] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28d1d6c02b384aeaa740460051ea351b in term 2.
I20260709 13:15:31.479635 10552 raft_consensus.cc:993] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab: : Instructing follower f090b64140fe4e55942c16e20a79b36b to start an election
I20260709 13:15:31.480069 10548 raft_consensus.cc:1081] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [term 1 LEADER]: Signalling peer f090b64140fe4e55942c16e20a79b36b to start an election
I20260709 13:15:31.481596 10311 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f7a41f1857af412bb47303bd9452b65f"
dest_uuid: "f090b64140fe4e55942c16e20a79b36b"
 from {username='slave'} at 127.0.0.1:37692
I20260709 13:15:31.482026 10311 raft_consensus.cc:493] T f7a41f1857af412bb47303bd9452b65f P f090b64140fe4e55942c16e20a79b36b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:31.482288 10311 raft_consensus.cc:3060] T f7a41f1857af412bb47303bd9452b65f P f090b64140fe4e55942c16e20a79b36b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:31.486846 10311 raft_consensus.cc:515] T f7a41f1857af412bb47303bd9452b65f P f090b64140fe4e55942c16e20a79b36b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:31.488586 10311 leader_election.cc:290] T f7a41f1857af412bb47303bd9452b65f P f090b64140fe4e55942c16e20a79b36b [CANDIDATE]: Term 2 election: Requested vote from peers 28d1d6c02b384aeaa740460051ea351b (127.9.116.130:36353), dd07659d40f84c2aafeaabddc392aeab (127.9.116.131:42413)
I20260709 13:15:31.489352 10387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a41f1857af412bb47303bd9452b65f" candidate_uuid: "f090b64140fe4e55942c16e20a79b36b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "28d1d6c02b384aeaa740460051ea351b"
I20260709 13:15:31.489691 10458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a41f1857af412bb47303bd9452b65f" candidate_uuid: "f090b64140fe4e55942c16e20a79b36b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dd07659d40f84c2aafeaabddc392aeab"
I20260709 13:15:31.489913 10387 raft_consensus.cc:3060] T f7a41f1857af412bb47303bd9452b65f P 28d1d6c02b384aeaa740460051ea351b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:31.490254 10458 raft_consensus.cc:3055] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:31.490530 10458 raft_consensus.cc:740] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dd07659d40f84c2aafeaabddc392aeab, State: Running, Role: LEADER
I20260709 13:15:31.491254 10458 consensus_queue.cc:260] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:31.492159 10458 raft_consensus.cc:3060] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:31.494710 10387 raft_consensus.cc:2468] T f7a41f1857af412bb47303bd9452b65f P 28d1d6c02b384aeaa740460051ea351b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f090b64140fe4e55942c16e20a79b36b in term 2.
I20260709 13:15:31.495844 10278 leader_election.cc:304] T f7a41f1857af412bb47303bd9452b65f P f090b64140fe4e55942c16e20a79b36b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28d1d6c02b384aeaa740460051ea351b, f090b64140fe4e55942c16e20a79b36b; no voters: 
I20260709 13:15:31.496538 10556 raft_consensus.cc:2804] T f7a41f1857af412bb47303bd9452b65f P f090b64140fe4e55942c16e20a79b36b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:31.496901 10458 raft_consensus.cc:2468] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f090b64140fe4e55942c16e20a79b36b in term 2.
I20260709 13:15:31.497051 10556 raft_consensus.cc:697] T f7a41f1857af412bb47303bd9452b65f P f090b64140fe4e55942c16e20a79b36b [term 2 LEADER]: Becoming Leader. State: Replica: f090b64140fe4e55942c16e20a79b36b, State: Running, Role: LEADER
I20260709 13:15:31.497902 10556 consensus_queue.cc:237] T f7a41f1857af412bb47303bd9452b65f P f090b64140fe4e55942c16e20a79b36b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:31.504165 10208 catalog_manager.cc:5697] T f7a41f1857af412bb47303bd9452b65f P f090b64140fe4e55942c16e20a79b36b reported cstate change: term changed from 1 to 2, leader changed from dd07659d40f84c2aafeaabddc392aeab (127.9.116.131) to f090b64140fe4e55942c16e20a79b36b (127.9.116.129). New cstate: current_term: 2 leader_uuid: "f090b64140fe4e55942c16e20a79b36b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:31.636593 10552 raft_consensus.cc:993] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab: : Instructing follower f090b64140fe4e55942c16e20a79b36b to start an election
I20260709 13:15:31.637018 10548 raft_consensus.cc:1081] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [term 1 LEADER]: Signalling peer f090b64140fe4e55942c16e20a79b36b to start an election
I20260709 13:15:31.638346 10311 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "09786fc30e8048d9b7928fe7bf969df1"
dest_uuid: "f090b64140fe4e55942c16e20a79b36b"
 from {username='slave'} at 127.0.0.1:37692
I20260709 13:15:31.638777 10311 raft_consensus.cc:493] T 09786fc30e8048d9b7928fe7bf969df1 P f090b64140fe4e55942c16e20a79b36b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:31.639127 10311 raft_consensus.cc:3060] T 09786fc30e8048d9b7928fe7bf969df1 P f090b64140fe4e55942c16e20a79b36b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:31.643502 10311 raft_consensus.cc:515] T 09786fc30e8048d9b7928fe7bf969df1 P f090b64140fe4e55942c16e20a79b36b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:31.645215 10311 leader_election.cc:290] T 09786fc30e8048d9b7928fe7bf969df1 P f090b64140fe4e55942c16e20a79b36b [CANDIDATE]: Term 2 election: Requested vote from peers 28d1d6c02b384aeaa740460051ea351b (127.9.116.130:36353), dd07659d40f84c2aafeaabddc392aeab (127.9.116.131:42413)
I20260709 13:15:31.645944 10387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09786fc30e8048d9b7928fe7bf969df1" candidate_uuid: "f090b64140fe4e55942c16e20a79b36b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "28d1d6c02b384aeaa740460051ea351b"
I20260709 13:15:31.646596 10387 raft_consensus.cc:3060] T 09786fc30e8048d9b7928fe7bf969df1 P 28d1d6c02b384aeaa740460051ea351b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:31.646479 10458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09786fc30e8048d9b7928fe7bf969df1" candidate_uuid: "f090b64140fe4e55942c16e20a79b36b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dd07659d40f84c2aafeaabddc392aeab"
I20260709 13:15:31.647253 10458 raft_consensus.cc:3055] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:31.647572 10458 raft_consensus.cc:740] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dd07659d40f84c2aafeaabddc392aeab, State: Running, Role: LEADER
I20260709 13:15:31.648393 10458 consensus_queue.cc:260] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:31.649461 10458 raft_consensus.cc:3060] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:31.653859 10387 raft_consensus.cc:2468] T 09786fc30e8048d9b7928fe7bf969df1 P 28d1d6c02b384aeaa740460051ea351b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f090b64140fe4e55942c16e20a79b36b in term 2.
I20260709 13:15:31.654651 10384 raft_consensus.cc:1240] T 09786fc30e8048d9b7928fe7bf969df1 P 28d1d6c02b384aeaa740460051ea351b [term 2 FOLLOWER]: Rejecting Update request from peer dd07659d40f84c2aafeaabddc392aeab for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:31.655139 10278 leader_election.cc:304] T 09786fc30e8048d9b7928fe7bf969df1 P f090b64140fe4e55942c16e20a79b36b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28d1d6c02b384aeaa740460051ea351b, f090b64140fe4e55942c16e20a79b36b; no voters: 
I20260709 13:15:31.655966 10556 raft_consensus.cc:2804] T 09786fc30e8048d9b7928fe7bf969df1 P f090b64140fe4e55942c16e20a79b36b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:31.656257 10458 raft_consensus.cc:2468] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f090b64140fe4e55942c16e20a79b36b in term 2.
I20260709 13:15:31.656481 10556 raft_consensus.cc:697] T 09786fc30e8048d9b7928fe7bf969df1 P f090b64140fe4e55942c16e20a79b36b [term 2 LEADER]: Becoming Leader. State: Replica: f090b64140fe4e55942c16e20a79b36b, State: Running, Role: LEADER
I20260709 13:15:31.657409 10556 consensus_queue.cc:237] T 09786fc30e8048d9b7928fe7bf969df1 P f090b64140fe4e55942c16e20a79b36b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:31.659962 10311 raft_consensus.cc:1275] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [term 2 FOLLOWER]: Refusing update from remote peer 28d1d6c02b384aeaa740460051ea351b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:31.661429 10551 consensus_queue.cc:1048] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:31.665555 10208 catalog_manager.cc:5697] T 09786fc30e8048d9b7928fe7bf969df1 P f090b64140fe4e55942c16e20a79b36b reported cstate change: term changed from 1 to 2, leader changed from dd07659d40f84c2aafeaabddc392aeab (127.9.116.131) to f090b64140fe4e55942c16e20a79b36b (127.9.116.129). New cstate: current_term: 2 leader_uuid: "f090b64140fe4e55942c16e20a79b36b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:31.670452 10458 raft_consensus.cc:3060] T 135e8982a8ea4fa198de62d0e60b459c P dd07659d40f84c2aafeaabddc392aeab [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:31.677325 10458 raft_consensus.cc:1275] T 135e8982a8ea4fa198de62d0e60b459c P dd07659d40f84c2aafeaabddc392aeab [term 2 FOLLOWER]: Refusing update from remote peer 28d1d6c02b384aeaa740460051ea351b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:31.678659 10508 consensus_queue.cc:1048] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:31.702006 10458 raft_consensus.cc:1275] T beb41e866f444744a06e30def55c3190 P dd07659d40f84c2aafeaabddc392aeab [term 2 FOLLOWER]: Refusing update from remote peer f090b64140fe4e55942c16e20a79b36b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:31.703316 10556 consensus_queue.cc:1048] T beb41e866f444744a06e30def55c3190 P f090b64140fe4e55942c16e20a79b36b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:31.708112 10311 raft_consensus.cc:3060] T 94c17c5750834fa999137c4d6cce4cf2 P f090b64140fe4e55942c16e20a79b36b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:31.711618 10384 raft_consensus.cc:1275] T beb41e866f444744a06e30def55c3190 P 28d1d6c02b384aeaa740460051ea351b [term 2 FOLLOWER]: Refusing update from remote peer f090b64140fe4e55942c16e20a79b36b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:31.715022 10512 consensus_queue.cc:1048] T beb41e866f444744a06e30def55c3190 P f090b64140fe4e55942c16e20a79b36b [LEADER]: Connected to new peer: Peer: permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:31.716516 10311 raft_consensus.cc:1275] T 94c17c5750834fa999137c4d6cce4cf2 P f090b64140fe4e55942c16e20a79b36b [term 2 FOLLOWER]: Refusing update from remote peer 28d1d6c02b384aeaa740460051ea351b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:31.720191 10584 consensus_queue.cc:1048] T 94c17c5750834fa999137c4d6cce4cf2 P 28d1d6c02b384aeaa740460051ea351b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:31.729907 10458 raft_consensus.cc:1275] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab [term 2 FOLLOWER]: Refusing update from remote peer 28d1d6c02b384aeaa740460051ea351b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:31.731207 10584 consensus_queue.cc:1048] T 94c17c5750834fa999137c4d6cce4cf2 P 28d1d6c02b384aeaa740460051ea351b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:31.767670 10384 raft_consensus.cc:1275] T 7fa6a2759a7d43a28bc364bdd2593069 P 28d1d6c02b384aeaa740460051ea351b [term 2 FOLLOWER]: Refusing update from remote peer f090b64140fe4e55942c16e20a79b36b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:31.768971 10593 consensus_queue.cc:1048] T 7fa6a2759a7d43a28bc364bdd2593069 P f090b64140fe4e55942c16e20a79b36b [LEADER]: Connected to new peer: Peer: permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:31.776805 10458 raft_consensus.cc:1275] T 7fa6a2759a7d43a28bc364bdd2593069 P dd07659d40f84c2aafeaabddc392aeab [term 2 FOLLOWER]: Refusing update from remote peer f090b64140fe4e55942c16e20a79b36b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:31.778424 10512 consensus_queue.cc:1048] T 7fa6a2759a7d43a28bc364bdd2593069 P f090b64140fe4e55942c16e20a79b36b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:31.808442 10458 raft_consensus.cc:1275] T 9db04314c2c849898b82d951374e2878 P dd07659d40f84c2aafeaabddc392aeab [term 2 FOLLOWER]: Refusing update from remote peer f090b64140fe4e55942c16e20a79b36b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:31.808722 10311 raft_consensus.cc:1275] T b6a956a56be14d83b5f0a8db624c1609 P f090b64140fe4e55942c16e20a79b36b [term 2 FOLLOWER]: Refusing update from remote peer 28d1d6c02b384aeaa740460051ea351b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260709 13:15:31.809121 10311 pending_rounds.cc:85] T b6a956a56be14d83b5f0a8db624c1609 P f090b64140fe4e55942c16e20a79b36b: Aborting all ops after (but not including) 0
I20260709 13:15:31.809446 10311 pending_rounds.cc:107] T b6a956a56be14d83b5f0a8db624c1609 P f090b64140fe4e55942c16e20a79b36b: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:15:31.809793 10311 raft_consensus.cc:2889] T b6a956a56be14d83b5f0a8db624c1609 P f090b64140fe4e55942c16e20a79b36b [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:31.809686 10512 consensus_queue.cc:1048] T 9db04314c2c849898b82d951374e2878 P f090b64140fe4e55942c16e20a79b36b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:31.811337 10584 consensus_queue.cc:1048] T b6a956a56be14d83b5f0a8db624c1609 P 28d1d6c02b384aeaa740460051ea351b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:31.820266 10384 raft_consensus.cc:1275] T 9db04314c2c849898b82d951374e2878 P 28d1d6c02b384aeaa740460051ea351b [term 2 FOLLOWER]: Refusing update from remote peer f090b64140fe4e55942c16e20a79b36b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:31.822818 10593 consensus_queue.cc:1048] T 9db04314c2c849898b82d951374e2878 P f090b64140fe4e55942c16e20a79b36b [LEADER]: Connected to new peer: Peer: permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:31.823997 10458 raft_consensus.cc:1275] T b6a956a56be14d83b5f0a8db624c1609 P dd07659d40f84c2aafeaabddc392aeab [term 2 FOLLOWER]: Refusing update from remote peer 28d1d6c02b384aeaa740460051ea351b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:15:31.825476 10508 consensus_queue.cc:1048] T b6a956a56be14d83b5f0a8db624c1609 P 28d1d6c02b384aeaa740460051ea351b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:31.938103 10311 raft_consensus.cc:1275] T c5b1bc6ab2bb4893a25a192936d09209 P f090b64140fe4e55942c16e20a79b36b [term 2 FOLLOWER]: Refusing update from remote peer 28d1d6c02b384aeaa740460051ea351b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:31.939483 10584 consensus_queue.cc:1048] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:31.946660 10460 raft_consensus.cc:1275] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [term 2 FOLLOWER]: Refusing update from remote peer 28d1d6c02b384aeaa740460051ea351b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:31.948415 10584 consensus_queue.cc:1048] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:31.996336 10460 raft_consensus.cc:1275] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [term 2 FOLLOWER]: Refusing update from remote peer f090b64140fe4e55942c16e20a79b36b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:31.997596 10556 consensus_queue.cc:1048] T f7a41f1857af412bb47303bd9452b65f P f090b64140fe4e55942c16e20a79b36b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:32.005198 10384 raft_consensus.cc:1275] T f7a41f1857af412bb47303bd9452b65f P 28d1d6c02b384aeaa740460051ea351b [term 2 FOLLOWER]: Refusing update from remote peer f090b64140fe4e55942c16e20a79b36b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:32.006676 10593 consensus_queue.cc:1048] T f7a41f1857af412bb47303bd9452b65f P f090b64140fe4e55942c16e20a79b36b [LEADER]: Connected to new peer: Peer: permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:32.119650 10460 raft_consensus.cc:1275] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [term 2 FOLLOWER]: Refusing update from remote peer f090b64140fe4e55942c16e20a79b36b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:32.120913 10593 consensus_queue.cc:1048] T 09786fc30e8048d9b7928fe7bf969df1 P f090b64140fe4e55942c16e20a79b36b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:32.130139 10384 raft_consensus.cc:1275] T 09786fc30e8048d9b7928fe7bf969df1 P 28d1d6c02b384aeaa740460051ea351b [term 2 FOLLOWER]: Refusing update from remote peer f090b64140fe4e55942c16e20a79b36b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:32.131744 10510 consensus_queue.cc:1048] T 09786fc30e8048d9b7928fe7bf969df1 P f090b64140fe4e55942c16e20a79b36b [LEADER]: Connected to new peer: Peer: permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:34.170590  9682 auto_leader_rebalancer-test.cc:348] The leader distribution is 
I20260709 13:15:34.171388  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:34.187388 10311 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "09786fc30e8048d9b7928fe7bf969df1"
dest_uuid: "f090b64140fe4e55942c16e20a79b36b"
mode: GRACEFUL
new_leader_uuid: "dd07659d40f84c2aafeaabddc392aeab"
 from {username='slave'} at 127.0.0.1:37712
I20260709 13:15:34.187944 10311 raft_consensus.cc:606] T 09786fc30e8048d9b7928fe7bf969df1 P f090b64140fe4e55942c16e20a79b36b [term 2 LEADER]: Received request to transfer leadership to TS dd07659d40f84c2aafeaabddc392aeab
I20260709 13:15:34.199368 10384 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c5b1bc6ab2bb4893a25a192936d09209"
dest_uuid: "28d1d6c02b384aeaa740460051ea351b"
mode: GRACEFUL
new_leader_uuid: "dd07659d40f84c2aafeaabddc392aeab"
 from {username='slave'} at 127.0.0.1:57278
I20260709 13:15:34.199900 10384 raft_consensus.cc:606] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b [term 2 LEADER]: Received request to transfer leadership to TS dd07659d40f84c2aafeaabddc392aeab
I20260709 13:15:34.202198 10311 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f7a41f1857af412bb47303bd9452b65f"
dest_uuid: "f090b64140fe4e55942c16e20a79b36b"
mode: GRACEFUL
new_leader_uuid: "dd07659d40f84c2aafeaabddc392aeab"
 from {username='slave'} at 127.0.0.1:37712
I20260709 13:15:34.202683 10311 raft_consensus.cc:606] T f7a41f1857af412bb47303bd9452b65f P f090b64140fe4e55942c16e20a79b36b [term 2 LEADER]: Received request to transfer leadership to TS dd07659d40f84c2aafeaabddc392aeab
I20260709 13:15:34.203830  9682 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:15:34.204330  9682 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:34.351133 10603 raft_consensus.cc:993] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b: : Instructing follower dd07659d40f84c2aafeaabddc392aeab to start an election
I20260709 13:15:34.351599 10508 raft_consensus.cc:1081] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b [term 2 LEADER]: Signalling peer dd07659d40f84c2aafeaabddc392aeab to start an election
I20260709 13:15:34.353098 10458 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c5b1bc6ab2bb4893a25a192936d09209"
dest_uuid: "dd07659d40f84c2aafeaabddc392aeab"
 from {username='slave'} at 127.0.0.1:54408
I20260709 13:15:34.353574 10458 raft_consensus.cc:493] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:34.353919 10458 raft_consensus.cc:3060] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:34.358397 10458 raft_consensus.cc:515] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:34.360028 10458 leader_election.cc:290] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [CANDIDATE]: Term 3 election: Requested vote from peers f090b64140fe4e55942c16e20a79b36b (127.9.116.129:42273), 28d1d6c02b384aeaa740460051ea351b (127.9.116.130:36353)
I20260709 13:15:34.361050 10311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b1bc6ab2bb4893a25a192936d09209" candidate_uuid: "dd07659d40f84c2aafeaabddc392aeab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f090b64140fe4e55942c16e20a79b36b"
I20260709 13:15:34.361116 10384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b1bc6ab2bb4893a25a192936d09209" candidate_uuid: "dd07659d40f84c2aafeaabddc392aeab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "28d1d6c02b384aeaa740460051ea351b"
I20260709 13:15:34.361743 10384 raft_consensus.cc:3055] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:34.361701 10311 raft_consensus.cc:3060] T c5b1bc6ab2bb4893a25a192936d09209 P f090b64140fe4e55942c16e20a79b36b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:34.362176 10384 raft_consensus.cc:740] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 28d1d6c02b384aeaa740460051ea351b, State: Running, Role: LEADER
I20260709 13:15:34.362958 10384 consensus_queue.cc:260] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:34.363865 10384 raft_consensus.cc:3060] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:34.366693 10311 raft_consensus.cc:2468] T c5b1bc6ab2bb4893a25a192936d09209 P f090b64140fe4e55942c16e20a79b36b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd07659d40f84c2aafeaabddc392aeab in term 3.
I20260709 13:15:34.367921 10425 leader_election.cc:304] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dd07659d40f84c2aafeaabddc392aeab, f090b64140fe4e55942c16e20a79b36b; no voters: 
I20260709 13:15:34.368630 10384 raft_consensus.cc:2468] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd07659d40f84c2aafeaabddc392aeab in term 3.
I20260709 13:15:34.369501 10604 raft_consensus.cc:2804] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:34.370687 10604 raft_consensus.cc:697] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [term 3 LEADER]: Becoming Leader. State: Replica: dd07659d40f84c2aafeaabddc392aeab, State: Running, Role: LEADER
I20260709 13:15:34.371364 10604 consensus_queue.cc:237] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:34.378213 10207 catalog_manager.cc:5697] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab reported cstate change: term changed from 2 to 3, leader changed from 28d1d6c02b384aeaa740460051ea351b (127.9.116.130) to dd07659d40f84c2aafeaabddc392aeab (127.9.116.131). New cstate: current_term: 3 leader_uuid: "dd07659d40f84c2aafeaabddc392aeab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:34.530898 10602 raft_consensus.cc:993] T f7a41f1857af412bb47303bd9452b65f P f090b64140fe4e55942c16e20a79b36b: : Instructing follower dd07659d40f84c2aafeaabddc392aeab to start an election
I20260709 13:15:34.531329 10556 raft_consensus.cc:1081] T f7a41f1857af412bb47303bd9452b65f P f090b64140fe4e55942c16e20a79b36b [term 2 LEADER]: Signalling peer dd07659d40f84c2aafeaabddc392aeab to start an election
I20260709 13:15:34.532672 10458 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f7a41f1857af412bb47303bd9452b65f"
dest_uuid: "dd07659d40f84c2aafeaabddc392aeab"
 from {username='slave'} at 127.0.0.1:54392
I20260709 13:15:34.533154 10458 raft_consensus.cc:493] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:34.533460 10458 raft_consensus.cc:3060] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:34.537992 10458 raft_consensus.cc:515] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:34.539635 10458 leader_election.cc:290] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [CANDIDATE]: Term 3 election: Requested vote from peers f090b64140fe4e55942c16e20a79b36b (127.9.116.129:42273), 28d1d6c02b384aeaa740460051ea351b (127.9.116.130:36353)
I20260709 13:15:34.540642 10311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a41f1857af412bb47303bd9452b65f" candidate_uuid: "dd07659d40f84c2aafeaabddc392aeab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f090b64140fe4e55942c16e20a79b36b"
I20260709 13:15:34.540774 10384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a41f1857af412bb47303bd9452b65f" candidate_uuid: "dd07659d40f84c2aafeaabddc392aeab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "28d1d6c02b384aeaa740460051ea351b"
I20260709 13:15:34.541201 10311 raft_consensus.cc:3055] T f7a41f1857af412bb47303bd9452b65f P f090b64140fe4e55942c16e20a79b36b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:34.541312 10384 raft_consensus.cc:3060] T f7a41f1857af412bb47303bd9452b65f P 28d1d6c02b384aeaa740460051ea351b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:34.541518 10311 raft_consensus.cc:740] T f7a41f1857af412bb47303bd9452b65f P f090b64140fe4e55942c16e20a79b36b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f090b64140fe4e55942c16e20a79b36b, State: Running, Role: LEADER
I20260709 13:15:34.542364 10311 consensus_queue.cc:260] T f7a41f1857af412bb47303bd9452b65f P f090b64140fe4e55942c16e20a79b36b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:34.543339 10311 raft_consensus.cc:3060] T f7a41f1857af412bb47303bd9452b65f P f090b64140fe4e55942c16e20a79b36b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:34.546343 10384 raft_consensus.cc:2468] T f7a41f1857af412bb47303bd9452b65f P 28d1d6c02b384aeaa740460051ea351b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd07659d40f84c2aafeaabddc392aeab in term 3.
I20260709 13:15:34.547458 10427 leader_election.cc:304] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28d1d6c02b384aeaa740460051ea351b, dd07659d40f84c2aafeaabddc392aeab; no voters: 
I20260709 13:15:34.548205 10311 raft_consensus.cc:2468] T f7a41f1857af412bb47303bd9452b65f P f090b64140fe4e55942c16e20a79b36b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd07659d40f84c2aafeaabddc392aeab in term 3.
I20260709 13:15:34.548256 10604 raft_consensus.cc:2804] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:34.548813 10604 raft_consensus.cc:697] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [term 3 LEADER]: Becoming Leader. State: Replica: dd07659d40f84c2aafeaabddc392aeab, State: Running, Role: LEADER
I20260709 13:15:34.549587 10604 consensus_queue.cc:237] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:34.556648 10207 catalog_manager.cc:5697] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab reported cstate change: term changed from 2 to 3, leader changed from f090b64140fe4e55942c16e20a79b36b (127.9.116.129) to dd07659d40f84c2aafeaabddc392aeab (127.9.116.131). New cstate: current_term: 3 leader_uuid: "dd07659d40f84c2aafeaabddc392aeab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:34.580379 10602 raft_consensus.cc:993] T 09786fc30e8048d9b7928fe7bf969df1 P f090b64140fe4e55942c16e20a79b36b: : Instructing follower dd07659d40f84c2aafeaabddc392aeab to start an election
I20260709 13:15:34.580807 10556 raft_consensus.cc:1081] T 09786fc30e8048d9b7928fe7bf969df1 P f090b64140fe4e55942c16e20a79b36b [term 2 LEADER]: Signalling peer dd07659d40f84c2aafeaabddc392aeab to start an election
I20260709 13:15:34.582192 10458 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "09786fc30e8048d9b7928fe7bf969df1"
dest_uuid: "dd07659d40f84c2aafeaabddc392aeab"
 from {username='slave'} at 127.0.0.1:54392
I20260709 13:15:34.582680 10458 raft_consensus.cc:493] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:34.583025 10458 raft_consensus.cc:3060] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:34.587505 10458 raft_consensus.cc:515] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:34.589152 10458 leader_election.cc:290] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [CANDIDATE]: Term 3 election: Requested vote from peers f090b64140fe4e55942c16e20a79b36b (127.9.116.129:42273), 28d1d6c02b384aeaa740460051ea351b (127.9.116.130:36353)
I20260709 13:15:34.590008 10311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09786fc30e8048d9b7928fe7bf969df1" candidate_uuid: "dd07659d40f84c2aafeaabddc392aeab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f090b64140fe4e55942c16e20a79b36b"
I20260709 13:15:34.590312 10384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09786fc30e8048d9b7928fe7bf969df1" candidate_uuid: "dd07659d40f84c2aafeaabddc392aeab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "28d1d6c02b384aeaa740460051ea351b"
I20260709 13:15:34.590561 10311 raft_consensus.cc:3055] T 09786fc30e8048d9b7928fe7bf969df1 P f090b64140fe4e55942c16e20a79b36b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:34.590906 10384 raft_consensus.cc:3060] T 09786fc30e8048d9b7928fe7bf969df1 P 28d1d6c02b384aeaa740460051ea351b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:34.590962 10311 raft_consensus.cc:740] T 09786fc30e8048d9b7928fe7bf969df1 P f090b64140fe4e55942c16e20a79b36b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f090b64140fe4e55942c16e20a79b36b, State: Running, Role: LEADER
I20260709 13:15:34.591785 10311 consensus_queue.cc:260] T 09786fc30e8048d9b7928fe7bf969df1 P f090b64140fe4e55942c16e20a79b36b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:34.592712 10311 raft_consensus.cc:3060] T 09786fc30e8048d9b7928fe7bf969df1 P f090b64140fe4e55942c16e20a79b36b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:34.597169 10384 raft_consensus.cc:2468] T 09786fc30e8048d9b7928fe7bf969df1 P 28d1d6c02b384aeaa740460051ea351b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd07659d40f84c2aafeaabddc392aeab in term 3.
I20260709 13:15:34.597412 10311 raft_consensus.cc:2468] T 09786fc30e8048d9b7928fe7bf969df1 P f090b64140fe4e55942c16e20a79b36b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd07659d40f84c2aafeaabddc392aeab in term 3.
I20260709 13:15:34.598253 10427 leader_election.cc:304] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28d1d6c02b384aeaa740460051ea351b, dd07659d40f84c2aafeaabddc392aeab; no voters: 
I20260709 13:15:34.599023 10604 raft_consensus.cc:2804] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:34.599453 10604 raft_consensus.cc:697] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [term 3 LEADER]: Becoming Leader. State: Replica: dd07659d40f84c2aafeaabddc392aeab, State: Running, Role: LEADER
I20260709 13:15:34.600174 10604 consensus_queue.cc:237] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:34.606900 10207 catalog_manager.cc:5697] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab reported cstate change: term changed from 2 to 3, leader changed from f090b64140fe4e55942c16e20a79b36b (127.9.116.129) to dd07659d40f84c2aafeaabddc392aeab (127.9.116.131). New cstate: current_term: 3 leader_uuid: "dd07659d40f84c2aafeaabddc392aeab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:34.762060 10310 raft_consensus.cc:1275] T c5b1bc6ab2bb4893a25a192936d09209 P f090b64140fe4e55942c16e20a79b36b [term 3 FOLLOWER]: Refusing update from remote peer dd07659d40f84c2aafeaabddc392aeab: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:34.763449 10604 consensus_queue.cc:1048] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [LEADER]: Connected to new peer: Peer: permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:34.773248 10384 raft_consensus.cc:1275] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b [term 3 FOLLOWER]: Refusing update from remote peer dd07659d40f84c2aafeaabddc392aeab: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:34.775468 10604 consensus_queue.cc:1048] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [LEADER]: Connected to new peer: Peer: permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:34.965380 10310 raft_consensus.cc:1275] T f7a41f1857af412bb47303bd9452b65f P f090b64140fe4e55942c16e20a79b36b [term 3 FOLLOWER]: Refusing update from remote peer dd07659d40f84c2aafeaabddc392aeab: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:34.966820 10604 consensus_queue.cc:1048] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [LEADER]: Connected to new peer: Peer: permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:34.979956 10384 raft_consensus.cc:1275] T f7a41f1857af412bb47303bd9452b65f P 28d1d6c02b384aeaa740460051ea351b [term 3 FOLLOWER]: Refusing update from remote peer dd07659d40f84c2aafeaabddc392aeab: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:34.981323 10605 consensus_queue.cc:1048] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [LEADER]: Connected to new peer: Peer: permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:35.130839 10311 raft_consensus.cc:1275] T 09786fc30e8048d9b7928fe7bf969df1 P f090b64140fe4e55942c16e20a79b36b [term 3 FOLLOWER]: Refusing update from remote peer dd07659d40f84c2aafeaabddc392aeab: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:35.132213 10605 consensus_queue.cc:1048] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [LEADER]: Connected to new peer: Peer: permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:35.142236 10384 raft_consensus.cc:1275] T 09786fc30e8048d9b7928fe7bf969df1 P 28d1d6c02b384aeaa740460051ea351b [term 3 FOLLOWER]: Refusing update from remote peer dd07659d40f84c2aafeaabddc392aeab: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:35.143632 10620 consensus_queue.cc:1048] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [LEADER]: Connected to new peer: Peer: permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:35.205128  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:35.219871 10458 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "09786fc30e8048d9b7928fe7bf969df1"
dest_uuid: "dd07659d40f84c2aafeaabddc392aeab"
mode: GRACEFUL
new_leader_uuid: "f090b64140fe4e55942c16e20a79b36b"
 from {username='slave'} at 127.0.0.1:54420
I20260709 13:15:35.220405 10458 raft_consensus.cc:606] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [term 3 LEADER]: Received request to transfer leadership to TS f090b64140fe4e55942c16e20a79b36b
I20260709 13:15:35.227692 10458 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f7a41f1857af412bb47303bd9452b65f"
dest_uuid: "dd07659d40f84c2aafeaabddc392aeab"
mode: GRACEFUL
new_leader_uuid: "f090b64140fe4e55942c16e20a79b36b"
 from {username='slave'} at 127.0.0.1:54420
I20260709 13:15:35.228214 10458 raft_consensus.cc:606] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [term 3 LEADER]: Received request to transfer leadership to TS f090b64140fe4e55942c16e20a79b36b
I20260709 13:15:35.233290 10384 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "135e8982a8ea4fa198de62d0e60b459c"
dest_uuid: "28d1d6c02b384aeaa740460051ea351b"
mode: GRACEFUL
new_leader_uuid: "f090b64140fe4e55942c16e20a79b36b"
 from {username='slave'} at 127.0.0.1:57266
I20260709 13:15:35.233745 10384 raft_consensus.cc:606] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b [term 2 LEADER]: Received request to transfer leadership to TS f090b64140fe4e55942c16e20a79b36b
I20260709 13:15:35.238405 10384 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "94c17c5750834fa999137c4d6cce4cf2"
dest_uuid: "28d1d6c02b384aeaa740460051ea351b"
mode: GRACEFUL
new_leader_uuid: "f090b64140fe4e55942c16e20a79b36b"
 from {username='slave'} at 127.0.0.1:57266
I20260709 13:15:35.239002 10384 raft_consensus.cc:606] T 94c17c5750834fa999137c4d6cce4cf2 P 28d1d6c02b384aeaa740460051ea351b [term 2 LEADER]: Received request to transfer leadership to TS f090b64140fe4e55942c16e20a79b36b
I20260709 13:15:35.243799 10458 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c5b1bc6ab2bb4893a25a192936d09209"
dest_uuid: "dd07659d40f84c2aafeaabddc392aeab"
mode: GRACEFUL
new_leader_uuid: "f090b64140fe4e55942c16e20a79b36b"
 from {username='slave'} at 127.0.0.1:54420
I20260709 13:15:35.244354 10458 raft_consensus.cc:606] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [term 3 LEADER]: Received request to transfer leadership to TS f090b64140fe4e55942c16e20a79b36b
I20260709 13:15:35.251538 10384 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b6a956a56be14d83b5f0a8db624c1609"
dest_uuid: "28d1d6c02b384aeaa740460051ea351b"
mode: GRACEFUL
new_leader_uuid: "dd07659d40f84c2aafeaabddc392aeab"
 from {username='slave'} at 127.0.0.1:57266
I20260709 13:15:35.252099 10384 raft_consensus.cc:606] T b6a956a56be14d83b5f0a8db624c1609 P 28d1d6c02b384aeaa740460051ea351b [term 2 LEADER]: Received request to transfer leadership to TS dd07659d40f84c2aafeaabddc392aeab
I20260709 13:15:35.360528 10605 raft_consensus.cc:993] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab: : Instructing follower f090b64140fe4e55942c16e20a79b36b to start an election
I20260709 13:15:35.360976 10605 raft_consensus.cc:1081] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [term 3 LEADER]: Signalling peer f090b64140fe4e55942c16e20a79b36b to start an election
I20260709 13:15:35.362422 10311 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c5b1bc6ab2bb4893a25a192936d09209"
dest_uuid: "f090b64140fe4e55942c16e20a79b36b"
 from {username='slave'} at 127.0.0.1:37692
I20260709 13:15:35.362998 10311 raft_consensus.cc:493] T c5b1bc6ab2bb4893a25a192936d09209 P f090b64140fe4e55942c16e20a79b36b [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:35.363345 10311 raft_consensus.cc:3060] T c5b1bc6ab2bb4893a25a192936d09209 P f090b64140fe4e55942c16e20a79b36b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:35.368065 10311 raft_consensus.cc:515] T c5b1bc6ab2bb4893a25a192936d09209 P f090b64140fe4e55942c16e20a79b36b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:35.369633 10311 leader_election.cc:290] T c5b1bc6ab2bb4893a25a192936d09209 P f090b64140fe4e55942c16e20a79b36b [CANDIDATE]: Term 4 election: Requested vote from peers 28d1d6c02b384aeaa740460051ea351b (127.9.116.130:36353), dd07659d40f84c2aafeaabddc392aeab (127.9.116.131:42413)
I20260709 13:15:35.371158 10458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b1bc6ab2bb4893a25a192936d09209" candidate_uuid: "f090b64140fe4e55942c16e20a79b36b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "dd07659d40f84c2aafeaabddc392aeab"
I20260709 13:15:35.371320 10384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b1bc6ab2bb4893a25a192936d09209" candidate_uuid: "f090b64140fe4e55942c16e20a79b36b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "28d1d6c02b384aeaa740460051ea351b"
I20260709 13:15:35.371790 10458 raft_consensus.cc:3055] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:35.372213 10458 raft_consensus.cc:740] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [term 3 LEADER]: Becoming Follower/Learner. State: Replica: dd07659d40f84c2aafeaabddc392aeab, State: Running, Role: LEADER
I20260709 13:15:35.372990 10458 consensus_queue.cc:260] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:35.374403 10458 raft_consensus.cc:3060] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:35.378979 10624 raft_consensus.cc:993] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b: : Instructing follower f090b64140fe4e55942c16e20a79b36b to start an election
I20260709 13:15:35.379477 10603 raft_consensus.cc:1081] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b [term 2 LEADER]: Signalling peer f090b64140fe4e55942c16e20a79b36b to start an election
I20260709 13:15:35.381167 10311 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "135e8982a8ea4fa198de62d0e60b459c"
dest_uuid: "f090b64140fe4e55942c16e20a79b36b"
 from {username='slave'} at 127.0.0.1:37706
I20260709 13:15:35.381309 10458 raft_consensus.cc:2468] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f090b64140fe4e55942c16e20a79b36b in term 4.
I20260709 13:15:35.381688 10311 raft_consensus.cc:493] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:35.382066 10311 raft_consensus.cc:3060] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:35.382712 10278 leader_election.cc:304] T c5b1bc6ab2bb4893a25a192936d09209 P f090b64140fe4e55942c16e20a79b36b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: dd07659d40f84c2aafeaabddc392aeab, f090b64140fe4e55942c16e20a79b36b; no voters: 28d1d6c02b384aeaa740460051ea351b
I20260709 13:15:35.383534 10556 raft_consensus.cc:2804] T c5b1bc6ab2bb4893a25a192936d09209 P f090b64140fe4e55942c16e20a79b36b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:35.384585 10556 raft_consensus.cc:697] T c5b1bc6ab2bb4893a25a192936d09209 P f090b64140fe4e55942c16e20a79b36b [term 4 LEADER]: Becoming Leader. State: Replica: f090b64140fe4e55942c16e20a79b36b, State: Running, Role: LEADER
I20260709 13:15:35.385531 10556 consensus_queue.cc:237] T c5b1bc6ab2bb4893a25a192936d09209 P f090b64140fe4e55942c16e20a79b36b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:35.388718 10311 raft_consensus.cc:515] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:35.390110 10311 leader_election.cc:290] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [CANDIDATE]: Term 3 election: Requested vote from peers 28d1d6c02b384aeaa740460051ea351b (127.9.116.130:36353), dd07659d40f84c2aafeaabddc392aeab (127.9.116.131:42413)
I20260709 13:15:35.391065 10384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e8982a8ea4fa198de62d0e60b459c" candidate_uuid: "f090b64140fe4e55942c16e20a79b36b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "28d1d6c02b384aeaa740460051ea351b"
I20260709 13:15:35.391409 10458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e8982a8ea4fa198de62d0e60b459c" candidate_uuid: "f090b64140fe4e55942c16e20a79b36b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "dd07659d40f84c2aafeaabddc392aeab"
I20260709 13:15:35.391628 10384 raft_consensus.cc:3055] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:35.391928 10384 raft_consensus.cc:740] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 28d1d6c02b384aeaa740460051ea351b, State: Running, Role: LEADER
I20260709 13:15:35.391957 10458 raft_consensus.cc:3060] T 135e8982a8ea4fa198de62d0e60b459c P dd07659d40f84c2aafeaabddc392aeab [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:35.392542 10207 catalog_manager.cc:5697] T c5b1bc6ab2bb4893a25a192936d09209 P f090b64140fe4e55942c16e20a79b36b reported cstate change: term changed from 3 to 4, leader changed from dd07659d40f84c2aafeaabddc392aeab (127.9.116.131) to f090b64140fe4e55942c16e20a79b36b (127.9.116.129). New cstate: current_term: 4 leader_uuid: "f090b64140fe4e55942c16e20a79b36b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:35.392735 10384 consensus_queue.cc:260] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:35.393872 10384 raft_consensus.cc:3060] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:35.398880 10458 raft_consensus.cc:2468] T 135e8982a8ea4fa198de62d0e60b459c P dd07659d40f84c2aafeaabddc392aeab [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f090b64140fe4e55942c16e20a79b36b in term 3.
I20260709 13:15:35.401309 10384 raft_consensus.cc:2468] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f090b64140fe4e55942c16e20a79b36b in term 3.
I20260709 13:15:35.402359 10278 leader_election.cc:304] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dd07659d40f84c2aafeaabddc392aeab, f090b64140fe4e55942c16e20a79b36b; no voters: 
I20260709 13:15:35.403244 10556 raft_consensus.cc:2804] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:35.403698 10556 raft_consensus.cc:697] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [term 3 LEADER]: Becoming Leader. State: Replica: f090b64140fe4e55942c16e20a79b36b, State: Running, Role: LEADER
I20260709 13:15:35.404531 10556 consensus_queue.cc:237] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:35.411773 10207 catalog_manager.cc:5697] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b reported cstate change: term changed from 2 to 3, leader changed from 28d1d6c02b384aeaa740460051ea351b (127.9.116.130) to f090b64140fe4e55942c16e20a79b36b (127.9.116.129). New cstate: current_term: 3 leader_uuid: "f090b64140fe4e55942c16e20a79b36b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:35.426846 10605 raft_consensus.cc:993] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab: : Instructing follower f090b64140fe4e55942c16e20a79b36b to start an election
I20260709 13:15:35.427305 10620 raft_consensus.cc:1081] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [term 3 LEADER]: Signalling peer f090b64140fe4e55942c16e20a79b36b to start an election
I20260709 13:15:35.428706 10311 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f7a41f1857af412bb47303bd9452b65f"
dest_uuid: "f090b64140fe4e55942c16e20a79b36b"
 from {username='slave'} at 127.0.0.1:37692
I20260709 13:15:35.429188 10311 raft_consensus.cc:493] T f7a41f1857af412bb47303bd9452b65f P f090b64140fe4e55942c16e20a79b36b [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:35.429492 10311 raft_consensus.cc:3060] T f7a41f1857af412bb47303bd9452b65f P f090b64140fe4e55942c16e20a79b36b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:35.433665 10311 raft_consensus.cc:515] T f7a41f1857af412bb47303bd9452b65f P f090b64140fe4e55942c16e20a79b36b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:35.435145 10311 leader_election.cc:290] T f7a41f1857af412bb47303bd9452b65f P f090b64140fe4e55942c16e20a79b36b [CANDIDATE]: Term 4 election: Requested vote from peers 28d1d6c02b384aeaa740460051ea351b (127.9.116.130:36353), dd07659d40f84c2aafeaabddc392aeab (127.9.116.131:42413)
I20260709 13:15:35.436474 10458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a41f1857af412bb47303bd9452b65f" candidate_uuid: "f090b64140fe4e55942c16e20a79b36b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "dd07659d40f84c2aafeaabddc392aeab"
I20260709 13:15:35.436497 10384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a41f1857af412bb47303bd9452b65f" candidate_uuid: "f090b64140fe4e55942c16e20a79b36b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "28d1d6c02b384aeaa740460051ea351b"
I20260709 13:15:35.437253 10384 raft_consensus.cc:3060] T f7a41f1857af412bb47303bd9452b65f P 28d1d6c02b384aeaa740460051ea351b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:35.437253 10458 raft_consensus.cc:3055] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:35.437724 10458 raft_consensus.cc:740] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [term 3 LEADER]: Becoming Follower/Learner. State: Replica: dd07659d40f84c2aafeaabddc392aeab, State: Running, Role: LEADER
I20260709 13:15:35.438515 10458 consensus_queue.cc:260] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:35.439733 10458 raft_consensus.cc:3060] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:35.444000 10384 raft_consensus.cc:2468] T f7a41f1857af412bb47303bd9452b65f P 28d1d6c02b384aeaa740460051ea351b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f090b64140fe4e55942c16e20a79b36b in term 4.
I20260709 13:15:35.445179 10278 leader_election.cc:304] T f7a41f1857af412bb47303bd9452b65f P f090b64140fe4e55942c16e20a79b36b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28d1d6c02b384aeaa740460051ea351b, f090b64140fe4e55942c16e20a79b36b; no voters: 
I20260709 13:15:35.445988 10556 raft_consensus.cc:2804] T f7a41f1857af412bb47303bd9452b65f P f090b64140fe4e55942c16e20a79b36b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:35.446154 10458 raft_consensus.cc:2468] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f090b64140fe4e55942c16e20a79b36b in term 4.
I20260709 13:15:35.446419 10556 raft_consensus.cc:697] T f7a41f1857af412bb47303bd9452b65f P f090b64140fe4e55942c16e20a79b36b [term 4 LEADER]: Becoming Leader. State: Replica: f090b64140fe4e55942c16e20a79b36b, State: Running, Role: LEADER
I20260709 13:15:35.447302 10556 consensus_queue.cc:237] T f7a41f1857af412bb47303bd9452b65f P f090b64140fe4e55942c16e20a79b36b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:35.452766 10207 catalog_manager.cc:5697] T f7a41f1857af412bb47303bd9452b65f P f090b64140fe4e55942c16e20a79b36b reported cstate change: term changed from 3 to 4, leader changed from dd07659d40f84c2aafeaabddc392aeab (127.9.116.131) to f090b64140fe4e55942c16e20a79b36b (127.9.116.129). New cstate: current_term: 4 leader_uuid: "f090b64140fe4e55942c16e20a79b36b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:35.459975 10624 raft_consensus.cc:993] T b6a956a56be14d83b5f0a8db624c1609 P 28d1d6c02b384aeaa740460051ea351b: : Instructing follower dd07659d40f84c2aafeaabddc392aeab to start an election
I20260709 13:15:35.460390 10603 raft_consensus.cc:1081] T b6a956a56be14d83b5f0a8db624c1609 P 28d1d6c02b384aeaa740460051ea351b [term 2 LEADER]: Signalling peer dd07659d40f84c2aafeaabddc392aeab to start an election
I20260709 13:15:35.461797 10458 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b6a956a56be14d83b5f0a8db624c1609"
dest_uuid: "dd07659d40f84c2aafeaabddc392aeab"
 from {username='slave'} at 127.0.0.1:54408
I20260709 13:15:35.462260 10458 raft_consensus.cc:493] T b6a956a56be14d83b5f0a8db624c1609 P dd07659d40f84c2aafeaabddc392aeab [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:35.462556 10458 raft_consensus.cc:3060] T b6a956a56be14d83b5f0a8db624c1609 P dd07659d40f84c2aafeaabddc392aeab [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:35.466521 10458 raft_consensus.cc:515] T b6a956a56be14d83b5f0a8db624c1609 P dd07659d40f84c2aafeaabddc392aeab [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:35.467963 10458 leader_election.cc:290] T b6a956a56be14d83b5f0a8db624c1609 P dd07659d40f84c2aafeaabddc392aeab [CANDIDATE]: Term 3 election: Requested vote from peers f090b64140fe4e55942c16e20a79b36b (127.9.116.129:42273), 28d1d6c02b384aeaa740460051ea351b (127.9.116.130:36353)
I20260709 13:15:35.468815 10311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6a956a56be14d83b5f0a8db624c1609" candidate_uuid: "dd07659d40f84c2aafeaabddc392aeab" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "f090b64140fe4e55942c16e20a79b36b"
I20260709 13:15:35.468990 10384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6a956a56be14d83b5f0a8db624c1609" candidate_uuid: "dd07659d40f84c2aafeaabddc392aeab" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "28d1d6c02b384aeaa740460051ea351b"
I20260709 13:15:35.469373 10311 raft_consensus.cc:3060] T b6a956a56be14d83b5f0a8db624c1609 P f090b64140fe4e55942c16e20a79b36b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:35.469498 10384 raft_consensus.cc:3055] T b6a956a56be14d83b5f0a8db624c1609 P 28d1d6c02b384aeaa740460051ea351b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:35.469849 10384 raft_consensus.cc:740] T b6a956a56be14d83b5f0a8db624c1609 P 28d1d6c02b384aeaa740460051ea351b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 28d1d6c02b384aeaa740460051ea351b, State: Running, Role: LEADER
I20260709 13:15:35.470461 10384 consensus_queue.cc:260] T b6a956a56be14d83b5f0a8db624c1609 P 28d1d6c02b384aeaa740460051ea351b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:35.471446 10384 raft_consensus.cc:3060] T b6a956a56be14d83b5f0a8db624c1609 P 28d1d6c02b384aeaa740460051ea351b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:35.473740 10311 raft_consensus.cc:2468] T b6a956a56be14d83b5f0a8db624c1609 P f090b64140fe4e55942c16e20a79b36b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd07659d40f84c2aafeaabddc392aeab in term 3.
I20260709 13:15:35.474757 10425 leader_election.cc:304] T b6a956a56be14d83b5f0a8db624c1609 P dd07659d40f84c2aafeaabddc392aeab [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dd07659d40f84c2aafeaabddc392aeab, f090b64140fe4e55942c16e20a79b36b; no voters: 
I20260709 13:15:35.475505 10620 raft_consensus.cc:2804] T b6a956a56be14d83b5f0a8db624c1609 P dd07659d40f84c2aafeaabddc392aeab [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:35.475658 10384 raft_consensus.cc:2468] T b6a956a56be14d83b5f0a8db624c1609 P 28d1d6c02b384aeaa740460051ea351b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd07659d40f84c2aafeaabddc392aeab in term 3.
I20260709 13:15:35.475898 10620 raft_consensus.cc:697] T b6a956a56be14d83b5f0a8db624c1609 P dd07659d40f84c2aafeaabddc392aeab [term 3 LEADER]: Becoming Leader. State: Replica: dd07659d40f84c2aafeaabddc392aeab, State: Running, Role: LEADER
I20260709 13:15:35.476634 10620 consensus_queue.cc:237] T b6a956a56be14d83b5f0a8db624c1609 P dd07659d40f84c2aafeaabddc392aeab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:35.483704 10207 catalog_manager.cc:5697] T b6a956a56be14d83b5f0a8db624c1609 P dd07659d40f84c2aafeaabddc392aeab reported cstate change: term changed from 2 to 3, leader changed from 28d1d6c02b384aeaa740460051ea351b (127.9.116.130) to dd07659d40f84c2aafeaabddc392aeab (127.9.116.131). New cstate: current_term: 3 leader_uuid: "dd07659d40f84c2aafeaabddc392aeab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:35.527333 10605 raft_consensus.cc:993] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab: : Instructing follower f090b64140fe4e55942c16e20a79b36b to start an election
I20260709 13:15:35.527750 10620 raft_consensus.cc:1081] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [term 3 LEADER]: Signalling peer f090b64140fe4e55942c16e20a79b36b to start an election
I20260709 13:15:35.529122 10311 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "09786fc30e8048d9b7928fe7bf969df1"
dest_uuid: "f090b64140fe4e55942c16e20a79b36b"
 from {username='slave'} at 127.0.0.1:37692
I20260709 13:15:35.529609 10311 raft_consensus.cc:493] T 09786fc30e8048d9b7928fe7bf969df1 P f090b64140fe4e55942c16e20a79b36b [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:35.529896 10311 raft_consensus.cc:3060] T 09786fc30e8048d9b7928fe7bf969df1 P f090b64140fe4e55942c16e20a79b36b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:35.533864 10311 raft_consensus.cc:515] T 09786fc30e8048d9b7928fe7bf969df1 P f090b64140fe4e55942c16e20a79b36b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:35.535414 10311 leader_election.cc:290] T 09786fc30e8048d9b7928fe7bf969df1 P f090b64140fe4e55942c16e20a79b36b [CANDIDATE]: Term 4 election: Requested vote from peers 28d1d6c02b384aeaa740460051ea351b (127.9.116.130:36353), dd07659d40f84c2aafeaabddc392aeab (127.9.116.131:42413)
I20260709 13:15:35.536368 10384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09786fc30e8048d9b7928fe7bf969df1" candidate_uuid: "f090b64140fe4e55942c16e20a79b36b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "28d1d6c02b384aeaa740460051ea351b"
I20260709 13:15:35.536556 10458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09786fc30e8048d9b7928fe7bf969df1" candidate_uuid: "f090b64140fe4e55942c16e20a79b36b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "dd07659d40f84c2aafeaabddc392aeab"
I20260709 13:15:35.536921 10384 raft_consensus.cc:3060] T 09786fc30e8048d9b7928fe7bf969df1 P 28d1d6c02b384aeaa740460051ea351b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:35.537019 10458 raft_consensus.cc:3055] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:35.537322 10458 raft_consensus.cc:740] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [term 3 LEADER]: Becoming Follower/Learner. State: Replica: dd07659d40f84c2aafeaabddc392aeab, State: Running, Role: LEADER
I20260709 13:15:35.538028 10458 consensus_queue.cc:260] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:35.539160 10458 raft_consensus.cc:3060] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:35.543110 10384 raft_consensus.cc:2468] T 09786fc30e8048d9b7928fe7bf969df1 P 28d1d6c02b384aeaa740460051ea351b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f090b64140fe4e55942c16e20a79b36b in term 4.
I20260709 13:15:35.544229 10278 leader_election.cc:304] T 09786fc30e8048d9b7928fe7bf969df1 P f090b64140fe4e55942c16e20a79b36b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28d1d6c02b384aeaa740460051ea351b, f090b64140fe4e55942c16e20a79b36b; no voters: 
I20260709 13:15:35.544716 10458 raft_consensus.cc:2468] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f090b64140fe4e55942c16e20a79b36b in term 4.
I20260709 13:15:35.544934 10556 raft_consensus.cc:2804] T 09786fc30e8048d9b7928fe7bf969df1 P f090b64140fe4e55942c16e20a79b36b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:35.545403 10556 raft_consensus.cc:697] T 09786fc30e8048d9b7928fe7bf969df1 P f090b64140fe4e55942c16e20a79b36b [term 4 LEADER]: Becoming Leader. State: Replica: f090b64140fe4e55942c16e20a79b36b, State: Running, Role: LEADER
I20260709 13:15:35.546185 10556 consensus_queue.cc:237] T 09786fc30e8048d9b7928fe7bf969df1 P f090b64140fe4e55942c16e20a79b36b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:35.551586 10207 catalog_manager.cc:5697] T 09786fc30e8048d9b7928fe7bf969df1 P f090b64140fe4e55942c16e20a79b36b reported cstate change: term changed from 3 to 4, leader changed from dd07659d40f84c2aafeaabddc392aeab (127.9.116.131) to f090b64140fe4e55942c16e20a79b36b (127.9.116.129). New cstate: current_term: 4 leader_uuid: "f090b64140fe4e55942c16e20a79b36b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:35.790063 10387 raft_consensus.cc:1275] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b [term 3 FOLLOWER]: Refusing update from remote peer f090b64140fe4e55942c16e20a79b36b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:35.791410 10556 consensus_queue.cc:1048] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [LEADER]: Connected to new peer: Peer: permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:35.800010 10624 raft_consensus.cc:993] T 94c17c5750834fa999137c4d6cce4cf2 P 28d1d6c02b384aeaa740460051ea351b: : Instructing follower f090b64140fe4e55942c16e20a79b36b to start an election
I20260709 13:15:35.800495 10624 raft_consensus.cc:1081] T 94c17c5750834fa999137c4d6cce4cf2 P 28d1d6c02b384aeaa740460051ea351b [term 2 LEADER]: Signalling peer f090b64140fe4e55942c16e20a79b36b to start an election
I20260709 13:15:35.803253 10458 raft_consensus.cc:1275] T 135e8982a8ea4fa198de62d0e60b459c P dd07659d40f84c2aafeaabddc392aeab [term 3 FOLLOWER]: Refusing update from remote peer f090b64140fe4e55942c16e20a79b36b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:35.803996 10311 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "94c17c5750834fa999137c4d6cce4cf2"
dest_uuid: "f090b64140fe4e55942c16e20a79b36b"
 from {username='slave'} at 127.0.0.1:37706
I20260709 13:15:35.804549 10311 raft_consensus.cc:493] T 94c17c5750834fa999137c4d6cce4cf2 P f090b64140fe4e55942c16e20a79b36b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:35.804880 10311 raft_consensus.cc:3060] T 94c17c5750834fa999137c4d6cce4cf2 P f090b64140fe4e55942c16e20a79b36b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:35.805035 10633 consensus_queue.cc:1048] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:35.812867 10311 raft_consensus.cc:515] T 94c17c5750834fa999137c4d6cce4cf2 P f090b64140fe4e55942c16e20a79b36b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:35.815354 10387 raft_consensus.cc:3060] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:35.815757 10311 leader_election.cc:290] T 94c17c5750834fa999137c4d6cce4cf2 P f090b64140fe4e55942c16e20a79b36b [CANDIDATE]: Term 3 election: Requested vote from peers 28d1d6c02b384aeaa740460051ea351b (127.9.116.130:36353), dd07659d40f84c2aafeaabddc392aeab (127.9.116.131:42413)
I20260709 13:15:35.817142 10384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c17c5750834fa999137c4d6cce4cf2" candidate_uuid: "f090b64140fe4e55942c16e20a79b36b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "28d1d6c02b384aeaa740460051ea351b"
I20260709 13:15:35.817682 10384 raft_consensus.cc:3055] T 94c17c5750834fa999137c4d6cce4cf2 P 28d1d6c02b384aeaa740460051ea351b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:35.817592 10458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c17c5750834fa999137c4d6cce4cf2" candidate_uuid: "f090b64140fe4e55942c16e20a79b36b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "dd07659d40f84c2aafeaabddc392aeab"
I20260709 13:15:35.818044 10384 raft_consensus.cc:740] T 94c17c5750834fa999137c4d6cce4cf2 P 28d1d6c02b384aeaa740460051ea351b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 28d1d6c02b384aeaa740460051ea351b, State: Running, Role: LEADER
I20260709 13:15:35.818318 10458 raft_consensus.cc:3060] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:35.818814 10384 consensus_queue.cc:260] T 94c17c5750834fa999137c4d6cce4cf2 P 28d1d6c02b384aeaa740460051ea351b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:35.820003 10384 raft_consensus.cc:3060] T 94c17c5750834fa999137c4d6cce4cf2 P 28d1d6c02b384aeaa740460051ea351b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:35.822484 10387 raft_consensus.cc:1275] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b [term 4 FOLLOWER]: Refusing update from remote peer f090b64140fe4e55942c16e20a79b36b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:15:35.822505 10458 raft_consensus.cc:2468] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f090b64140fe4e55942c16e20a79b36b in term 3.
I20260709 13:15:35.823853 10278 leader_election.cc:304] T 94c17c5750834fa999137c4d6cce4cf2 P f090b64140fe4e55942c16e20a79b36b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dd07659d40f84c2aafeaabddc392aeab, f090b64140fe4e55942c16e20a79b36b; no voters: 
I20260709 13:15:35.824818 10633 raft_consensus.cc:2804] T 94c17c5750834fa999137c4d6cce4cf2 P f090b64140fe4e55942c16e20a79b36b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:35.824925 10556 consensus_queue.cc:1048] T c5b1bc6ab2bb4893a25a192936d09209 P f090b64140fe4e55942c16e20a79b36b [LEADER]: Connected to new peer: Peer: permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:35.825398 10633 raft_consensus.cc:697] T 94c17c5750834fa999137c4d6cce4cf2 P f090b64140fe4e55942c16e20a79b36b [term 3 LEADER]: Becoming Leader. State: Replica: f090b64140fe4e55942c16e20a79b36b, State: Running, Role: LEADER
I20260709 13:15:35.826246 10633 consensus_queue.cc:237] T 94c17c5750834fa999137c4d6cce4cf2 P f090b64140fe4e55942c16e20a79b36b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:35.828127 10384 raft_consensus.cc:2468] T 94c17c5750834fa999137c4d6cce4cf2 P 28d1d6c02b384aeaa740460051ea351b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f090b64140fe4e55942c16e20a79b36b in term 3.
I20260709 13:15:35.835920 10458 raft_consensus.cc:1275] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [term 4 FOLLOWER]: Refusing update from remote peer f090b64140fe4e55942c16e20a79b36b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:15:35.835870 10207 catalog_manager.cc:5697] T 94c17c5750834fa999137c4d6cce4cf2 P f090b64140fe4e55942c16e20a79b36b reported cstate change: term changed from 2 to 3, leader changed from 28d1d6c02b384aeaa740460051ea351b (127.9.116.130) to f090b64140fe4e55942c16e20a79b36b (127.9.116.129). New cstate: current_term: 3 leader_uuid: "f090b64140fe4e55942c16e20a79b36b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:35.837241 10556 consensus_queue.cc:1048] T c5b1bc6ab2bb4893a25a192936d09209 P f090b64140fe4e55942c16e20a79b36b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:35.849134 10460 raft_consensus.cc:1275] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [term 4 FOLLOWER]: Refusing update from remote peer f090b64140fe4e55942c16e20a79b36b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:15:35.850570 10625 consensus_queue.cc:1048] T f7a41f1857af412bb47303bd9452b65f P f090b64140fe4e55942c16e20a79b36b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:35.859717 10387 raft_consensus.cc:1275] T f7a41f1857af412bb47303bd9452b65f P 28d1d6c02b384aeaa740460051ea351b [term 4 FOLLOWER]: Refusing update from remote peer f090b64140fe4e55942c16e20a79b36b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:15:35.861536 10556 consensus_queue.cc:1048] T f7a41f1857af412bb47303bd9452b65f P f090b64140fe4e55942c16e20a79b36b [LEADER]: Connected to new peer: Peer: permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:35.881618 10311 raft_consensus.cc:1275] T b6a956a56be14d83b5f0a8db624c1609 P f090b64140fe4e55942c16e20a79b36b [term 3 FOLLOWER]: Refusing update from remote peer dd07659d40f84c2aafeaabddc392aeab: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:35.882772 10620 consensus_queue.cc:1048] T b6a956a56be14d83b5f0a8db624c1609 P dd07659d40f84c2aafeaabddc392aeab [LEADER]: Connected to new peer: Peer: permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:35.891050 10387 raft_consensus.cc:1275] T b6a956a56be14d83b5f0a8db624c1609 P 28d1d6c02b384aeaa740460051ea351b [term 3 FOLLOWER]: Refusing update from remote peer dd07659d40f84c2aafeaabddc392aeab: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:35.893241 10605 consensus_queue.cc:1048] T b6a956a56be14d83b5f0a8db624c1609 P dd07659d40f84c2aafeaabddc392aeab [LEADER]: Connected to new peer: Peer: permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:36.001773 10387 raft_consensus.cc:1275] T 09786fc30e8048d9b7928fe7bf969df1 P 28d1d6c02b384aeaa740460051ea351b [term 4 FOLLOWER]: Refusing update from remote peer f090b64140fe4e55942c16e20a79b36b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:15:36.003145 10556 consensus_queue.cc:1048] T 09786fc30e8048d9b7928fe7bf969df1 P f090b64140fe4e55942c16e20a79b36b [LEADER]: Connected to new peer: Peer: permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:36.010703 10460 raft_consensus.cc:1275] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [term 4 FOLLOWER]: Refusing update from remote peer f090b64140fe4e55942c16e20a79b36b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:15:36.013016 10633 consensus_queue.cc:1048] T 09786fc30e8048d9b7928fe7bf969df1 P f090b64140fe4e55942c16e20a79b36b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:15:36.329276 10387 raft_consensus.cc:1275] T 94c17c5750834fa999137c4d6cce4cf2 P 28d1d6c02b384aeaa740460051ea351b [term 3 FOLLOWER]: Refusing update from remote peer f090b64140fe4e55942c16e20a79b36b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:36.329356 10460 raft_consensus.cc:1275] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab [term 3 FOLLOWER]: Refusing update from remote peer f090b64140fe4e55942c16e20a79b36b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:36.330762 10642 consensus_queue.cc:1048] T 94c17c5750834fa999137c4d6cce4cf2 P f090b64140fe4e55942c16e20a79b36b [LEADER]: Connected to new peer: Peer: permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:36.331455 10625 consensus_queue.cc:1048] T 94c17c5750834fa999137c4d6cce4cf2 P f090b64140fe4e55942c16e20a79b36b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:38.254498  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:38.262434 10311 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "135e8982a8ea4fa198de62d0e60b459c"
dest_uuid: "f090b64140fe4e55942c16e20a79b36b"
mode: GRACEFUL
new_leader_uuid: "28d1d6c02b384aeaa740460051ea351b"
 from {username='slave'} at 127.0.0.1:37712
I20260709 13:15:38.263163 10311 raft_consensus.cc:606] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [term 3 LEADER]: Received request to transfer leadership to TS 28d1d6c02b384aeaa740460051ea351b
I20260709 13:15:38.265818 10311 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "94c17c5750834fa999137c4d6cce4cf2"
dest_uuid: "f090b64140fe4e55942c16e20a79b36b"
mode: GRACEFUL
new_leader_uuid: "dd07659d40f84c2aafeaabddc392aeab"
 from {username='slave'} at 127.0.0.1:37712
I20260709 13:15:38.266222 10311 raft_consensus.cc:606] T 94c17c5750834fa999137c4d6cce4cf2 P f090b64140fe4e55942c16e20a79b36b [term 3 LEADER]: Received request to transfer leadership to TS dd07659d40f84c2aafeaabddc392aeab
I20260709 13:15:38.268385 10311 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "beb41e866f444744a06e30def55c3190"
dest_uuid: "f090b64140fe4e55942c16e20a79b36b"
mode: GRACEFUL
new_leader_uuid: "dd07659d40f84c2aafeaabddc392aeab"
 from {username='slave'} at 127.0.0.1:37712
I20260709 13:15:38.268777 10311 raft_consensus.cc:606] T beb41e866f444744a06e30def55c3190 P f090b64140fe4e55942c16e20a79b36b [term 2 LEADER]: Received request to transfer leadership to TS dd07659d40f84c2aafeaabddc392aeab
I20260709 13:15:38.270967 10311 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c5b1bc6ab2bb4893a25a192936d09209"
dest_uuid: "f090b64140fe4e55942c16e20a79b36b"
mode: GRACEFUL
new_leader_uuid: "28d1d6c02b384aeaa740460051ea351b"
 from {username='slave'} at 127.0.0.1:37712
I20260709 13:15:38.271370 10311 raft_consensus.cc:606] T c5b1bc6ab2bb4893a25a192936d09209 P f090b64140fe4e55942c16e20a79b36b [term 4 LEADER]: Received request to transfer leadership to TS 28d1d6c02b384aeaa740460051ea351b
I20260709 13:15:38.272383  9682 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:15:38.272868  9682 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:38.411899 10647 raft_consensus.cc:993] T beb41e866f444744a06e30def55c3190 P f090b64140fe4e55942c16e20a79b36b: : Instructing follower dd07659d40f84c2aafeaabddc392aeab to start an election
I20260709 13:15:38.412614 10556 raft_consensus.cc:1081] T beb41e866f444744a06e30def55c3190 P f090b64140fe4e55942c16e20a79b36b [term 2 LEADER]: Signalling peer dd07659d40f84c2aafeaabddc392aeab to start an election
I20260709 13:15:38.414179 10460 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "beb41e866f444744a06e30def55c3190"
dest_uuid: "dd07659d40f84c2aafeaabddc392aeab"
 from {username='slave'} at 127.0.0.1:54392
I20260709 13:15:38.414642 10460 raft_consensus.cc:493] T beb41e866f444744a06e30def55c3190 P dd07659d40f84c2aafeaabddc392aeab [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:38.415010 10460 raft_consensus.cc:3060] T beb41e866f444744a06e30def55c3190 P dd07659d40f84c2aafeaabddc392aeab [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:38.420023 10460 raft_consensus.cc:515] T beb41e866f444744a06e30def55c3190 P dd07659d40f84c2aafeaabddc392aeab [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } } peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } }
I20260709 13:15:38.421618 10460 leader_election.cc:290] T beb41e866f444744a06e30def55c3190 P dd07659d40f84c2aafeaabddc392aeab [CANDIDATE]: Term 3 election: Requested vote from peers f090b64140fe4e55942c16e20a79b36b (127.9.116.129:42273), 28d1d6c02b384aeaa740460051ea351b (127.9.116.130:36353)
I20260709 13:15:38.422729 10311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "beb41e866f444744a06e30def55c3190" candidate_uuid: "dd07659d40f84c2aafeaabddc392aeab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f090b64140fe4e55942c16e20a79b36b"
I20260709 13:15:38.422945 10384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "beb41e866f444744a06e30def55c3190" candidate_uuid: "dd07659d40f84c2aafeaabddc392aeab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "28d1d6c02b384aeaa740460051ea351b"
I20260709 13:15:38.423704 10311 raft_consensus.cc:3055] T beb41e866f444744a06e30def55c3190 P f090b64140fe4e55942c16e20a79b36b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:38.423745 10384 raft_consensus.cc:3060] T beb41e866f444744a06e30def55c3190 P 28d1d6c02b384aeaa740460051ea351b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:38.424034 10311 raft_consensus.cc:740] T beb41e866f444744a06e30def55c3190 P f090b64140fe4e55942c16e20a79b36b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f090b64140fe4e55942c16e20a79b36b, State: Running, Role: LEADER
I20260709 13:15:38.424927 10311 consensus_queue.cc:260] T beb41e866f444744a06e30def55c3190 P f090b64140fe4e55942c16e20a79b36b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } } peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } }
I20260709 13:15:38.426070 10311 raft_consensus.cc:3060] T beb41e866f444744a06e30def55c3190 P f090b64140fe4e55942c16e20a79b36b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:38.428931 10384 raft_consensus.cc:2468] T beb41e866f444744a06e30def55c3190 P 28d1d6c02b384aeaa740460051ea351b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd07659d40f84c2aafeaabddc392aeab in term 3.
I20260709 13:15:38.430110 10427 leader_election.cc:304] T beb41e866f444744a06e30def55c3190 P dd07659d40f84c2aafeaabddc392aeab [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28d1d6c02b384aeaa740460051ea351b, dd07659d40f84c2aafeaabddc392aeab; no voters: 
I20260709 13:15:38.430945 10605 raft_consensus.cc:2804] T beb41e866f444744a06e30def55c3190 P dd07659d40f84c2aafeaabddc392aeab [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:38.431859 10311 raft_consensus.cc:2468] T beb41e866f444744a06e30def55c3190 P f090b64140fe4e55942c16e20a79b36b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd07659d40f84c2aafeaabddc392aeab in term 3.
I20260709 13:15:38.431982 10605 raft_consensus.cc:697] T beb41e866f444744a06e30def55c3190 P dd07659d40f84c2aafeaabddc392aeab [term 3 LEADER]: Becoming Leader. State: Replica: dd07659d40f84c2aafeaabddc392aeab, State: Running, Role: LEADER
I20260709 13:15:38.433089 10605 consensus_queue.cc:237] T beb41e866f444744a06e30def55c3190 P dd07659d40f84c2aafeaabddc392aeab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } } peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } }
I20260709 13:15:38.441206 10207 catalog_manager.cc:5697] T beb41e866f444744a06e30def55c3190 P dd07659d40f84c2aafeaabddc392aeab reported cstate change: term changed from 2 to 3, leader changed from f090b64140fe4e55942c16e20a79b36b (127.9.116.129) to dd07659d40f84c2aafeaabddc392aeab (127.9.116.131). New cstate: current_term: 3 leader_uuid: "dd07659d40f84c2aafeaabddc392aeab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:38.447088 10648 raft_consensus.cc:993] T c5b1bc6ab2bb4893a25a192936d09209 P f090b64140fe4e55942c16e20a79b36b: : Instructing follower 28d1d6c02b384aeaa740460051ea351b to start an election
I20260709 13:15:38.447585 10556 raft_consensus.cc:1081] T c5b1bc6ab2bb4893a25a192936d09209 P f090b64140fe4e55942c16e20a79b36b [term 4 LEADER]: Signalling peer 28d1d6c02b384aeaa740460051ea351b to start an election
I20260709 13:15:38.449662 10384 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c5b1bc6ab2bb4893a25a192936d09209"
dest_uuid: "28d1d6c02b384aeaa740460051ea351b"
 from {username='slave'} at 127.0.0.1:57248
I20260709 13:15:38.450246 10384 raft_consensus.cc:493] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:38.450623 10384 raft_consensus.cc:3060] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:38.456878 10384 raft_consensus.cc:515] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:38.459242 10384 leader_election.cc:290] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b [CANDIDATE]: Term 5 election: Requested vote from peers f090b64140fe4e55942c16e20a79b36b (127.9.116.129:42273), dd07659d40f84c2aafeaabddc392aeab (127.9.116.131:42413)
I20260709 13:15:38.459619 10311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b1bc6ab2bb4893a25a192936d09209" candidate_uuid: "28d1d6c02b384aeaa740460051ea351b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "f090b64140fe4e55942c16e20a79b36b"
I20260709 13:15:38.459825 10460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b1bc6ab2bb4893a25a192936d09209" candidate_uuid: "28d1d6c02b384aeaa740460051ea351b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "dd07659d40f84c2aafeaabddc392aeab"
I20260709 13:15:38.460183 10311 raft_consensus.cc:3055] T c5b1bc6ab2bb4893a25a192936d09209 P f090b64140fe4e55942c16e20a79b36b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:38.460366 10460 raft_consensus.cc:3060] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:38.460474 10311 raft_consensus.cc:740] T c5b1bc6ab2bb4893a25a192936d09209 P f090b64140fe4e55942c16e20a79b36b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f090b64140fe4e55942c16e20a79b36b, State: Running, Role: LEADER
I20260709 13:15:38.461113 10311 consensus_queue.cc:260] T c5b1bc6ab2bb4893a25a192936d09209 P f090b64140fe4e55942c16e20a79b36b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:38.461937 10311 raft_consensus.cc:3060] T c5b1bc6ab2bb4893a25a192936d09209 P f090b64140fe4e55942c16e20a79b36b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:38.464789 10460 raft_consensus.cc:2468] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28d1d6c02b384aeaa740460051ea351b in term 5.
I20260709 13:15:38.466097 10353 leader_election.cc:304] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28d1d6c02b384aeaa740460051ea351b, dd07659d40f84c2aafeaabddc392aeab; no voters: 
I20260709 13:15:38.467157 10624 raft_consensus.cc:2804] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:38.467166 10311 raft_consensus.cc:2468] T c5b1bc6ab2bb4893a25a192936d09209 P f090b64140fe4e55942c16e20a79b36b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28d1d6c02b384aeaa740460051ea351b in term 5.
I20260709 13:15:38.468561 10624 raft_consensus.cc:697] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b [term 5 LEADER]: Becoming Leader. State: Replica: 28d1d6c02b384aeaa740460051ea351b, State: Running, Role: LEADER
I20260709 13:15:38.469360 10624 consensus_queue.cc:237] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:38.476128 10207 catalog_manager.cc:5697] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b reported cstate change: term changed from 4 to 5, leader changed from f090b64140fe4e55942c16e20a79b36b (127.9.116.129) to 28d1d6c02b384aeaa740460051ea351b (127.9.116.130). New cstate: current_term: 5 leader_uuid: "28d1d6c02b384aeaa740460051ea351b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:38.562309 10648 raft_consensus.cc:993] T 94c17c5750834fa999137c4d6cce4cf2 P f090b64140fe4e55942c16e20a79b36b: : Instructing follower dd07659d40f84c2aafeaabddc392aeab to start an election
I20260709 13:15:38.562726 10556 raft_consensus.cc:1081] T 94c17c5750834fa999137c4d6cce4cf2 P f090b64140fe4e55942c16e20a79b36b [term 3 LEADER]: Signalling peer dd07659d40f84c2aafeaabddc392aeab to start an election
I20260709 13:15:38.564005 10460 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "94c17c5750834fa999137c4d6cce4cf2"
dest_uuid: "dd07659d40f84c2aafeaabddc392aeab"
 from {username='slave'} at 127.0.0.1:54392
I20260709 13:15:38.564450 10460 raft_consensus.cc:493] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:38.564760 10460 raft_consensus.cc:3060] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:38.568831 10460 raft_consensus.cc:515] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:38.570307 10460 leader_election.cc:290] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab [CANDIDATE]: Term 4 election: Requested vote from peers f090b64140fe4e55942c16e20a79b36b (127.9.116.129:42273), 28d1d6c02b384aeaa740460051ea351b (127.9.116.130:36353)
I20260709 13:15:38.571180 10311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c17c5750834fa999137c4d6cce4cf2" candidate_uuid: "dd07659d40f84c2aafeaabddc392aeab" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "f090b64140fe4e55942c16e20a79b36b"
I20260709 13:15:38.571381 10384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c17c5750834fa999137c4d6cce4cf2" candidate_uuid: "dd07659d40f84c2aafeaabddc392aeab" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "28d1d6c02b384aeaa740460051ea351b"
I20260709 13:15:38.571755 10311 raft_consensus.cc:3055] T 94c17c5750834fa999137c4d6cce4cf2 P f090b64140fe4e55942c16e20a79b36b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:38.571841 10384 raft_consensus.cc:3060] T 94c17c5750834fa999137c4d6cce4cf2 P 28d1d6c02b384aeaa740460051ea351b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:38.572111 10311 raft_consensus.cc:740] T 94c17c5750834fa999137c4d6cce4cf2 P f090b64140fe4e55942c16e20a79b36b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f090b64140fe4e55942c16e20a79b36b, State: Running, Role: LEADER
I20260709 13:15:38.572873 10311 consensus_queue.cc:260] T 94c17c5750834fa999137c4d6cce4cf2 P f090b64140fe4e55942c16e20a79b36b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:38.574033 10311 raft_consensus.cc:3060] T 94c17c5750834fa999137c4d6cce4cf2 P f090b64140fe4e55942c16e20a79b36b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:38.578179 10384 raft_consensus.cc:2468] T 94c17c5750834fa999137c4d6cce4cf2 P 28d1d6c02b384aeaa740460051ea351b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd07659d40f84c2aafeaabddc392aeab in term 4.
I20260709 13:15:38.579310 10427 leader_election.cc:304] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28d1d6c02b384aeaa740460051ea351b, dd07659d40f84c2aafeaabddc392aeab; no voters: 
I20260709 13:15:38.579913 10311 raft_consensus.cc:2468] T 94c17c5750834fa999137c4d6cce4cf2 P f090b64140fe4e55942c16e20a79b36b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd07659d40f84c2aafeaabddc392aeab in term 4.
I20260709 13:15:38.580040 10605 raft_consensus.cc:2804] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:38.580478 10605 raft_consensus.cc:697] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab [term 4 LEADER]: Becoming Leader. State: Replica: dd07659d40f84c2aafeaabddc392aeab, State: Running, Role: LEADER
I20260709 13:15:38.581377 10605 consensus_queue.cc:237] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:38.587761 10207 catalog_manager.cc:5697] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab reported cstate change: term changed from 3 to 4, leader changed from f090b64140fe4e55942c16e20a79b36b (127.9.116.129) to dd07659d40f84c2aafeaabddc392aeab (127.9.116.131). New cstate: current_term: 4 leader_uuid: "dd07659d40f84c2aafeaabddc392aeab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:38.752909 10648 raft_consensus.cc:993] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b: : Instructing follower 28d1d6c02b384aeaa740460051ea351b to start an election
I20260709 13:15:38.753373 10556 raft_consensus.cc:1081] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [term 3 LEADER]: Signalling peer 28d1d6c02b384aeaa740460051ea351b to start an election
I20260709 13:15:38.754695 10384 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "135e8982a8ea4fa198de62d0e60b459c"
dest_uuid: "28d1d6c02b384aeaa740460051ea351b"
 from {username='slave'} at 127.0.0.1:57248
I20260709 13:15:38.755134 10384 raft_consensus.cc:493] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:38.755357 10384 raft_consensus.cc:3060] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:38.759375 10384 raft_consensus.cc:515] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:38.760872 10384 leader_election.cc:290] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b [CANDIDATE]: Term 4 election: Requested vote from peers f090b64140fe4e55942c16e20a79b36b (127.9.116.129:42273), dd07659d40f84c2aafeaabddc392aeab (127.9.116.131:42413)
I20260709 13:15:38.761585 10311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e8982a8ea4fa198de62d0e60b459c" candidate_uuid: "28d1d6c02b384aeaa740460051ea351b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "f090b64140fe4e55942c16e20a79b36b"
I20260709 13:15:38.762166 10311 raft_consensus.cc:3055] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:38.762212 10460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e8982a8ea4fa198de62d0e60b459c" candidate_uuid: "28d1d6c02b384aeaa740460051ea351b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "dd07659d40f84c2aafeaabddc392aeab"
I20260709 13:15:38.762403 10311 raft_consensus.cc:740] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f090b64140fe4e55942c16e20a79b36b, State: Running, Role: LEADER
I20260709 13:15:38.762768 10460 raft_consensus.cc:3060] T 135e8982a8ea4fa198de62d0e60b459c P dd07659d40f84c2aafeaabddc392aeab [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:38.763183 10311 consensus_queue.cc:260] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:38.764065 10311 raft_consensus.cc:3060] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:38.767284 10460 raft_consensus.cc:2468] T 135e8982a8ea4fa198de62d0e60b459c P dd07659d40f84c2aafeaabddc392aeab [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28d1d6c02b384aeaa740460051ea351b in term 4.
I20260709 13:15:38.768293 10311 raft_consensus.cc:2468] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28d1d6c02b384aeaa740460051ea351b in term 4.
I20260709 13:15:38.768298 10353 leader_election.cc:304] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28d1d6c02b384aeaa740460051ea351b, dd07659d40f84c2aafeaabddc392aeab; no voters: 
I20260709 13:15:38.769147 10624 raft_consensus.cc:2804] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:38.769613 10624 raft_consensus.cc:697] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b [term 4 LEADER]: Becoming Leader. State: Replica: 28d1d6c02b384aeaa740460051ea351b, State: Running, Role: LEADER
I20260709 13:15:38.770483 10624 consensus_queue.cc:237] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } }
I20260709 13:15:38.776885 10207 catalog_manager.cc:5697] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b reported cstate change: term changed from 3 to 4, leader changed from f090b64140fe4e55942c16e20a79b36b (127.9.116.129) to 28d1d6c02b384aeaa740460051ea351b (127.9.116.130). New cstate: current_term: 4 leader_uuid: "28d1d6c02b384aeaa740460051ea351b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:38.897501 10311 raft_consensus.cc:1275] T beb41e866f444744a06e30def55c3190 P f090b64140fe4e55942c16e20a79b36b [term 3 FOLLOWER]: Refusing update from remote peer dd07659d40f84c2aafeaabddc392aeab: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:38.898687 10605 consensus_queue.cc:1048] T beb41e866f444744a06e30def55c3190 P dd07659d40f84c2aafeaabddc392aeab [LEADER]: Connected to new peer: Peer: permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:38.908946 10384 raft_consensus.cc:1275] T beb41e866f444744a06e30def55c3190 P 28d1d6c02b384aeaa740460051ea351b [term 3 FOLLOWER]: Refusing update from remote peer dd07659d40f84c2aafeaabddc392aeab: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:38.910331 10605 consensus_queue.cc:1048] T beb41e866f444744a06e30def55c3190 P dd07659d40f84c2aafeaabddc392aeab [LEADER]: Connected to new peer: Peer: permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:38.930217 10460 raft_consensus.cc:1275] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [term 5 FOLLOWER]: Refusing update from remote peer 28d1d6c02b384aeaa740460051ea351b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:15:38.931399 10624 consensus_queue.cc:1048] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:38.940644 10311 raft_consensus.cc:1275] T c5b1bc6ab2bb4893a25a192936d09209 P f090b64140fe4e55942c16e20a79b36b [term 5 FOLLOWER]: Refusing update from remote peer 28d1d6c02b384aeaa740460051ea351b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:15:38.942070 10624 consensus_queue.cc:1048] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:39.039861 10384 raft_consensus.cc:1275] T 94c17c5750834fa999137c4d6cce4cf2 P 28d1d6c02b384aeaa740460051ea351b [term 4 FOLLOWER]: Refusing update from remote peer dd07659d40f84c2aafeaabddc392aeab: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:15:39.041265 10649 consensus_queue.cc:1048] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab [LEADER]: Connected to new peer: Peer: permanent_uuid: "28d1d6c02b384aeaa740460051ea351b" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 36353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:39.053663 10311 raft_consensus.cc:1275] T 94c17c5750834fa999137c4d6cce4cf2 P f090b64140fe4e55942c16e20a79b36b [term 4 FOLLOWER]: Refusing update from remote peer dd07659d40f84c2aafeaabddc392aeab: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:15:39.055433 10668 consensus_queue.cc:1048] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab [LEADER]: Connected to new peer: Peer: permanent_uuid: "f090b64140fe4e55942c16e20a79b36b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:39.273737  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:39.279560  9682 tablet_server.cc:179] TabletServer@127.9.116.129:0 shutting down...
W20260709 13:15:39.295555 10351 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 4 similar messages]
W20260709 13:15:39.301441 10351 consensus_peers.cc:597] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b -> Peer f090b64140fe4e55942c16e20a79b36b (127.9.116.129:42273): Couldn't send request to peer f090b64140fe4e55942c16e20a79b36b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:39.312844  9682 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:15:39.312954 10425 consensus_peers.cc:597] T beb41e866f444744a06e30def55c3190 P dd07659d40f84c2aafeaabddc392aeab -> Peer f090b64140fe4e55942c16e20a79b36b (127.9.116.129:42273): Couldn't send request to peer f090b64140fe4e55942c16e20a79b36b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:39.313748  9682 tablet_replica.cc:333] T b6a956a56be14d83b5f0a8db624c1609 P f090b64140fe4e55942c16e20a79b36b: stopping tablet replica
I20260709 13:15:39.314419  9682 raft_consensus.cc:2243] T b6a956a56be14d83b5f0a8db624c1609 P f090b64140fe4e55942c16e20a79b36b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:39.314987  9682 raft_consensus.cc:2272] T b6a956a56be14d83b5f0a8db624c1609 P f090b64140fe4e55942c16e20a79b36b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:39.316933  9682 tablet_replica.cc:333] T acb9159770724070b93f9bc026ffee90 P f090b64140fe4e55942c16e20a79b36b: stopping tablet replica
I20260709 13:15:39.317497  9682 raft_consensus.cc:2243] T acb9159770724070b93f9bc026ffee90 P f090b64140fe4e55942c16e20a79b36b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:39.317929  9682 raft_consensus.cc:2272] T acb9159770724070b93f9bc026ffee90 P f090b64140fe4e55942c16e20a79b36b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:39.319718  9682 tablet_replica.cc:333] T 7fa6a2759a7d43a28bc364bdd2593069 P f090b64140fe4e55942c16e20a79b36b: stopping tablet replica
I20260709 13:15:39.320261  9682 raft_consensus.cc:2243] T 7fa6a2759a7d43a28bc364bdd2593069 P f090b64140fe4e55942c16e20a79b36b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:15:39.321075  9682 raft_consensus.cc:2272] T 7fa6a2759a7d43a28bc364bdd2593069 P f090b64140fe4e55942c16e20a79b36b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:39.323390  9682 tablet_replica.cc:333] T 09786fc30e8048d9b7928fe7bf969df1 P f090b64140fe4e55942c16e20a79b36b: stopping tablet replica
I20260709 13:15:39.324150  9682 raft_consensus.cc:2243] T 09786fc30e8048d9b7928fe7bf969df1 P f090b64140fe4e55942c16e20a79b36b [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:15:39.324941  9682 raft_consensus.cc:2272] T 09786fc30e8048d9b7928fe7bf969df1 P f090b64140fe4e55942c16e20a79b36b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:39.326936  9682 tablet_replica.cc:333] T c5b1bc6ab2bb4893a25a192936d09209 P f090b64140fe4e55942c16e20a79b36b: stopping tablet replica
I20260709 13:15:39.327466  9682 raft_consensus.cc:2243] T c5b1bc6ab2bb4893a25a192936d09209 P f090b64140fe4e55942c16e20a79b36b [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:39.327891  9682 raft_consensus.cc:2272] T c5b1bc6ab2bb4893a25a192936d09209 P f090b64140fe4e55942c16e20a79b36b [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:39.330106  9682 tablet_replica.cc:333] T beb41e866f444744a06e30def55c3190 P f090b64140fe4e55942c16e20a79b36b: stopping tablet replica
I20260709 13:15:39.330628  9682 raft_consensus.cc:2243] T beb41e866f444744a06e30def55c3190 P f090b64140fe4e55942c16e20a79b36b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:39.331115  9682 raft_consensus.cc:2272] T beb41e866f444744a06e30def55c3190 P f090b64140fe4e55942c16e20a79b36b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:39.333016  9682 tablet_replica.cc:333] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b: stopping tablet replica
I20260709 13:15:39.333551  9682 raft_consensus.cc:2243] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:39.333945  9682 raft_consensus.cc:2272] T 135e8982a8ea4fa198de62d0e60b459c P f090b64140fe4e55942c16e20a79b36b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:39.335738  9682 tablet_replica.cc:333] T f7a41f1857af412bb47303bd9452b65f P f090b64140fe4e55942c16e20a79b36b: stopping tablet replica
I20260709 13:15:39.336269  9682 raft_consensus.cc:2243] T f7a41f1857af412bb47303bd9452b65f P f090b64140fe4e55942c16e20a79b36b [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:15:39.337021  9682 raft_consensus.cc:2272] T f7a41f1857af412bb47303bd9452b65f P f090b64140fe4e55942c16e20a79b36b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:39.338683  9682 tablet_replica.cc:333] T 9db04314c2c849898b82d951374e2878 P f090b64140fe4e55942c16e20a79b36b: stopping tablet replica
I20260709 13:15:39.339214  9682 raft_consensus.cc:2243] T 9db04314c2c849898b82d951374e2878 P f090b64140fe4e55942c16e20a79b36b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:15:39.339933  9682 raft_consensus.cc:2272] T 9db04314c2c849898b82d951374e2878 P f090b64140fe4e55942c16e20a79b36b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:39.341619  9682 tablet_replica.cc:333] T 94c17c5750834fa999137c4d6cce4cf2 P f090b64140fe4e55942c16e20a79b36b: stopping tablet replica
I20260709 13:15:39.342090  9682 raft_consensus.cc:2243] T 94c17c5750834fa999137c4d6cce4cf2 P f090b64140fe4e55942c16e20a79b36b [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:39.342533  9682 raft_consensus.cc:2272] T 94c17c5750834fa999137c4d6cce4cf2 P f090b64140fe4e55942c16e20a79b36b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:39.374531  9682 tablet_server.cc:196] TabletServer@127.9.116.129:0 shutdown complete.
W20260709 13:15:39.374761 10351 consensus_peers.cc:597] T acb9159770724070b93f9bc026ffee90 P 28d1d6c02b384aeaa740460051ea351b -> Peer f090b64140fe4e55942c16e20a79b36b (127.9.116.129:42273): Couldn't send request to peer f090b64140fe4e55942c16e20a79b36b. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:42273: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:39.385391 10460 raft_consensus.cc:1275] T 135e8982a8ea4fa198de62d0e60b459c P dd07659d40f84c2aafeaabddc392aeab [term 4 FOLLOWER]: Refusing update from remote peer 28d1d6c02b384aeaa740460051ea351b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:15:39.387086 10624 consensus_queue.cc:1048] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd07659d40f84c2aafeaabddc392aeab" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 42413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:39.403702  9682 tablet_server.cc:179] TabletServer@127.9.116.130:0 shutting down...
I20260709 13:15:39.429870  9682 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:15:39.430547  9682 tablet_replica.cc:333] T b6a956a56be14d83b5f0a8db624c1609 P 28d1d6c02b384aeaa740460051ea351b: stopping tablet replica
I20260709 13:15:39.431206  9682 raft_consensus.cc:2243] T b6a956a56be14d83b5f0a8db624c1609 P 28d1d6c02b384aeaa740460051ea351b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:39.431721  9682 raft_consensus.cc:2272] T b6a956a56be14d83b5f0a8db624c1609 P 28d1d6c02b384aeaa740460051ea351b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:39.433540  9682 tablet_replica.cc:333] T 94c17c5750834fa999137c4d6cce4cf2 P 28d1d6c02b384aeaa740460051ea351b: stopping tablet replica
I20260709 13:15:39.434119  9682 raft_consensus.cc:2243] T 94c17c5750834fa999137c4d6cce4cf2 P 28d1d6c02b384aeaa740460051ea351b [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:39.434559  9682 raft_consensus.cc:2272] T 94c17c5750834fa999137c4d6cce4cf2 P 28d1d6c02b384aeaa740460051ea351b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:39.436776  9682 tablet_replica.cc:333] T 09786fc30e8048d9b7928fe7bf969df1 P 28d1d6c02b384aeaa740460051ea351b: stopping tablet replica
I20260709 13:15:39.437290  9682 raft_consensus.cc:2243] T 09786fc30e8048d9b7928fe7bf969df1 P 28d1d6c02b384aeaa740460051ea351b [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:39.437806  9682 raft_consensus.cc:2272] T 09786fc30e8048d9b7928fe7bf969df1 P 28d1d6c02b384aeaa740460051ea351b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:39.439507  9682 tablet_replica.cc:333] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b: stopping tablet replica
I20260709 13:15:39.439981  9682 raft_consensus.cc:2243] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:15:39.440685  9682 raft_consensus.cc:2272] T c5b1bc6ab2bb4893a25a192936d09209 P 28d1d6c02b384aeaa740460051ea351b [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:39.442516  9682 tablet_replica.cc:333] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b: stopping tablet replica
I20260709 13:15:39.443059  9682 raft_consensus.cc:2243] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:15:39.443783  9682 raft_consensus.cc:2272] T 135e8982a8ea4fa198de62d0e60b459c P 28d1d6c02b384aeaa740460051ea351b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:39.445624  9682 tablet_replica.cc:333] T f7a41f1857af412bb47303bd9452b65f P 28d1d6c02b384aeaa740460051ea351b: stopping tablet replica
I20260709 13:15:39.446115  9682 raft_consensus.cc:2243] T f7a41f1857af412bb47303bd9452b65f P 28d1d6c02b384aeaa740460051ea351b [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:39.446564  9682 raft_consensus.cc:2272] T f7a41f1857af412bb47303bd9452b65f P 28d1d6c02b384aeaa740460051ea351b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:39.448237  9682 tablet_replica.cc:333] T beb41e866f444744a06e30def55c3190 P 28d1d6c02b384aeaa740460051ea351b: stopping tablet replica
I20260709 13:15:39.448700  9682 raft_consensus.cc:2243] T beb41e866f444744a06e30def55c3190 P 28d1d6c02b384aeaa740460051ea351b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:39.449096  9682 raft_consensus.cc:2272] T beb41e866f444744a06e30def55c3190 P 28d1d6c02b384aeaa740460051ea351b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:39.451262  9682 tablet_replica.cc:333] T 9db04314c2c849898b82d951374e2878 P 28d1d6c02b384aeaa740460051ea351b: stopping tablet replica
I20260709 13:15:39.451748  9682 raft_consensus.cc:2243] T 9db04314c2c849898b82d951374e2878 P 28d1d6c02b384aeaa740460051ea351b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:39.452144  9682 raft_consensus.cc:2272] T 9db04314c2c849898b82d951374e2878 P 28d1d6c02b384aeaa740460051ea351b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:39.453778  9682 tablet_replica.cc:333] T acb9159770724070b93f9bc026ffee90 P 28d1d6c02b384aeaa740460051ea351b: stopping tablet replica
I20260709 13:15:39.454238  9682 raft_consensus.cc:2243] T acb9159770724070b93f9bc026ffee90 P 28d1d6c02b384aeaa740460051ea351b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:15:39.455159  9682 raft_consensus.cc:2272] T acb9159770724070b93f9bc026ffee90 P 28d1d6c02b384aeaa740460051ea351b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:39.456696  9682 tablet_replica.cc:333] T 7fa6a2759a7d43a28bc364bdd2593069 P 28d1d6c02b384aeaa740460051ea351b: stopping tablet replica
I20260709 13:15:39.457191  9682 raft_consensus.cc:2243] T 7fa6a2759a7d43a28bc364bdd2593069 P 28d1d6c02b384aeaa740460051ea351b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:39.457648  9682 raft_consensus.cc:2272] T 7fa6a2759a7d43a28bc364bdd2593069 P 28d1d6c02b384aeaa740460051ea351b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:39.485744  9682 tablet_server.cc:196] TabletServer@127.9.116.130:0 shutdown complete.
I20260709 13:15:39.505554  9682 tablet_server.cc:179] TabletServer@127.9.116.131:0 shutting down...
I20260709 13:15:39.531457  9682 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:15:39.532302  9682 tablet_replica.cc:333] T b6a956a56be14d83b5f0a8db624c1609 P dd07659d40f84c2aafeaabddc392aeab: stopping tablet replica
I20260709 13:15:39.533097  9682 raft_consensus.cc:2243] T b6a956a56be14d83b5f0a8db624c1609 P dd07659d40f84c2aafeaabddc392aeab [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:15:39.533950  9682 raft_consensus.cc:2272] T b6a956a56be14d83b5f0a8db624c1609 P dd07659d40f84c2aafeaabddc392aeab [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:39.536051  9682 tablet_replica.cc:333] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab: stopping tablet replica
I20260709 13:15:39.536648  9682 raft_consensus.cc:2243] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:15:39.537451  9682 raft_consensus.cc:2272] T 94c17c5750834fa999137c4d6cce4cf2 P dd07659d40f84c2aafeaabddc392aeab [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:39.540570  9682 tablet_replica.cc:333] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab: stopping tablet replica
I20260709 13:15:39.541178  9682 raft_consensus.cc:2243] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:39.541817  9682 raft_consensus.cc:2272] T c5b1bc6ab2bb4893a25a192936d09209 P dd07659d40f84c2aafeaabddc392aeab [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:39.544049  9682 tablet_replica.cc:333] T beb41e866f444744a06e30def55c3190 P dd07659d40f84c2aafeaabddc392aeab: stopping tablet replica
I20260709 13:15:39.544595  9682 raft_consensus.cc:2243] T beb41e866f444744a06e30def55c3190 P dd07659d40f84c2aafeaabddc392aeab [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:15:39.545320  9682 raft_consensus.cc:2272] T beb41e866f444744a06e30def55c3190 P dd07659d40f84c2aafeaabddc392aeab [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:39.547233  9682 tablet_replica.cc:333] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab: stopping tablet replica
I20260709 13:15:39.547739  9682 raft_consensus.cc:2243] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:39.548178  9682 raft_consensus.cc:2272] T f7a41f1857af412bb47303bd9452b65f P dd07659d40f84c2aafeaabddc392aeab [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:39.549865  9682 tablet_replica.cc:333] T 135e8982a8ea4fa198de62d0e60b459c P dd07659d40f84c2aafeaabddc392aeab: stopping tablet replica
I20260709 13:15:39.550370  9682 raft_consensus.cc:2243] T 135e8982a8ea4fa198de62d0e60b459c P dd07659d40f84c2aafeaabddc392aeab [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:39.550766  9682 raft_consensus.cc:2272] T 135e8982a8ea4fa198de62d0e60b459c P dd07659d40f84c2aafeaabddc392aeab [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:39.552768  9682 tablet_replica.cc:333] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab: stopping tablet replica
I20260709 13:15:39.553274  9682 raft_consensus.cc:2243] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:39.553748  9682 raft_consensus.cc:2272] T 09786fc30e8048d9b7928fe7bf969df1 P dd07659d40f84c2aafeaabddc392aeab [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:39.555454  9682 tablet_replica.cc:333] T acb9159770724070b93f9bc026ffee90 P dd07659d40f84c2aafeaabddc392aeab: stopping tablet replica
I20260709 13:15:39.555931  9682 raft_consensus.cc:2243] T acb9159770724070b93f9bc026ffee90 P dd07659d40f84c2aafeaabddc392aeab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:39.556313  9682 raft_consensus.cc:2272] T acb9159770724070b93f9bc026ffee90 P dd07659d40f84c2aafeaabddc392aeab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:39.557901  9682 tablet_replica.cc:333] T 7fa6a2759a7d43a28bc364bdd2593069 P dd07659d40f84c2aafeaabddc392aeab: stopping tablet replica
I20260709 13:15:39.558364  9682 raft_consensus.cc:2243] T 7fa6a2759a7d43a28bc364bdd2593069 P dd07659d40f84c2aafeaabddc392aeab [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:39.558748  9682 raft_consensus.cc:2272] T 7fa6a2759a7d43a28bc364bdd2593069 P dd07659d40f84c2aafeaabddc392aeab [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:39.560550  9682 tablet_replica.cc:333] T 9db04314c2c849898b82d951374e2878 P dd07659d40f84c2aafeaabddc392aeab: stopping tablet replica
I20260709 13:15:39.561085  9682 raft_consensus.cc:2243] T 9db04314c2c849898b82d951374e2878 P dd07659d40f84c2aafeaabddc392aeab [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:39.561564  9682 raft_consensus.cc:2272] T 9db04314c2c849898b82d951374e2878 P dd07659d40f84c2aafeaabddc392aeab [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:39.589505  9682 tablet_server.cc:196] TabletServer@127.9.116.131:0 shutdown complete.
I20260709 13:15:39.607859  9682 master.cc:562] Master@127.9.116.190:39829 shutting down...
I20260709 13:15:39.627534  9682 raft_consensus.cc:2243] T 00000000000000000000000000000000 P fb45f429aff74b84b7c676c2531e4c2a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:15:39.628091  9682 raft_consensus.cc:2272] T 00000000000000000000000000000000 P fb45f429aff74b84b7c676c2531e4c2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:39.628437  9682 tablet_replica.cc:333] T 00000000000000000000000000000000 P fb45f429aff74b84b7c676c2531e4c2a: stopping tablet replica
I20260709 13:15:39.648108  9682 master.cc:584] Master@127.9.116.190:39829 shutdown complete.
28d1d6c02b384aeaa740460051ea351b  5
dd07659d40f84c2aafeaabddc392aeab  0
f090b64140fe4e55942c16e20a79b36b  5
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (10394 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 13:15:39.691442  9682 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.9.116.190:36007
I20260709 13:15:39.692749  9682 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:15:39.699137 10675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:39.699268 10674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:39.699689 10677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:39.703948  9682 server_base.cc:1061] running on GCE node
I20260709 13:15:39.705372  9682 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:15:39.705571  9682 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:15:39.705780  9682 hybrid_clock.cc:648] HybridClock initialized: now 1783602939705745 us; error 0 us; skew 500 ppm
I20260709 13:15:39.706338  9682 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:15:39.708832  9682 webserver.cc:533] Webserver started at http://127.9.116.190:34513/ using document root <none> and password file <none>
I20260709 13:15:39.709348  9682 fs_manager.cc:362] Metadata directory not provided
I20260709 13:15:39.709554  9682 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:15:39.709851  9682 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:15:39.711081  9682 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/master-0-root/instance:
uuid: "ae957c8fc66c4b6ba063d0c74cbf5484"
format_stamp: "Formatted at 2026-07-09 13:15:39 on dist-test-slave-1t6m"
I20260709 13:15:39.715535  9682 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.004s
I20260709 13:15:39.718997 10682 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:39.719754  9682 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:15:39.720068  9682 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/master-0-root
uuid: "ae957c8fc66c4b6ba063d0c74cbf5484"
format_stamp: "Formatted at 2026-07-09 13:15:39 on dist-test-slave-1t6m"
I20260709 13:15:39.720371  9682 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:15:39.753381  9682 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:15:39.754673  9682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:15:39.793545  9682 rpc_server.cc:307] RPC server started. Bound to: 127.9.116.190:36007
I20260709 13:15:39.793653 10733 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.116.190:36007 every 8 connection(s)
I20260709 13:15:39.797515 10734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.807101 10734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae957c8fc66c4b6ba063d0c74cbf5484: Bootstrap starting.
I20260709 13:15:39.811721 10734 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae957c8fc66c4b6ba063d0c74cbf5484: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:39.815752 10734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae957c8fc66c4b6ba063d0c74cbf5484: No bootstrap required, opened a new log
I20260709 13:15:39.817801 10734 raft_consensus.cc:359] T 00000000000000000000000000000000 P ae957c8fc66c4b6ba063d0c74cbf5484 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae957c8fc66c4b6ba063d0c74cbf5484" member_type: VOTER }
I20260709 13:15:39.818161 10734 raft_consensus.cc:385] T 00000000000000000000000000000000 P ae957c8fc66c4b6ba063d0c74cbf5484 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:39.818339 10734 raft_consensus.cc:740] T 00000000000000000000000000000000 P ae957c8fc66c4b6ba063d0c74cbf5484 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae957c8fc66c4b6ba063d0c74cbf5484, State: Initialized, Role: FOLLOWER
I20260709 13:15:39.818944 10734 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae957c8fc66c4b6ba063d0c74cbf5484 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae957c8fc66c4b6ba063d0c74cbf5484" member_type: VOTER }
I20260709 13:15:39.819361 10734 raft_consensus.cc:399] T 00000000000000000000000000000000 P ae957c8fc66c4b6ba063d0c74cbf5484 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:15:39.819559 10734 raft_consensus.cc:493] T 00000000000000000000000000000000 P ae957c8fc66c4b6ba063d0c74cbf5484 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:15:39.819761 10734 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ae957c8fc66c4b6ba063d0c74cbf5484 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:39.824014 10734 raft_consensus.cc:515] T 00000000000000000000000000000000 P ae957c8fc66c4b6ba063d0c74cbf5484 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae957c8fc66c4b6ba063d0c74cbf5484" member_type: VOTER }
I20260709 13:15:39.824479 10734 leader_election.cc:304] T 00000000000000000000000000000000 P ae957c8fc66c4b6ba063d0c74cbf5484 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae957c8fc66c4b6ba063d0c74cbf5484; no voters: 
I20260709 13:15:39.825510 10734 leader_election.cc:290] T 00000000000000000000000000000000 P ae957c8fc66c4b6ba063d0c74cbf5484 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:15:39.825852 10737 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ae957c8fc66c4b6ba063d0c74cbf5484 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:39.827169 10737 raft_consensus.cc:697] T 00000000000000000000000000000000 P ae957c8fc66c4b6ba063d0c74cbf5484 [term 1 LEADER]: Becoming Leader. State: Replica: ae957c8fc66c4b6ba063d0c74cbf5484, State: Running, Role: LEADER
I20260709 13:15:39.827857 10737 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae957c8fc66c4b6ba063d0c74cbf5484 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae957c8fc66c4b6ba063d0c74cbf5484" member_type: VOTER }
I20260709 13:15:39.828506 10734 sys_catalog.cc:565] T 00000000000000000000000000000000 P ae957c8fc66c4b6ba063d0c74cbf5484 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:15:39.832448 10738 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae957c8fc66c4b6ba063d0c74cbf5484 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae957c8fc66c4b6ba063d0c74cbf5484" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae957c8fc66c4b6ba063d0c74cbf5484" member_type: VOTER } }
I20260709 13:15:39.832861 10739 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae957c8fc66c4b6ba063d0c74cbf5484 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae957c8fc66c4b6ba063d0c74cbf5484. Latest consensus state: current_term: 1 leader_uuid: "ae957c8fc66c4b6ba063d0c74cbf5484" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae957c8fc66c4b6ba063d0c74cbf5484" member_type: VOTER } }
I20260709 13:15:39.833335 10738 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae957c8fc66c4b6ba063d0c74cbf5484 [sys.catalog]: This master's current role is: LEADER
I20260709 13:15:39.833694 10739 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae957c8fc66c4b6ba063d0c74cbf5484 [sys.catalog]: This master's current role is: LEADER
I20260709 13:15:39.841981 10744 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:15:39.847031 10744 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:15:39.848187  9682 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:15:39.855530 10744 catalog_manager.cc:1383] Generated new cluster ID: b3204374c6d64839b83f0d4e6ad23dec
I20260709 13:15:39.855816 10744 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:15:39.872287 10744 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:15:39.873556 10744 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:15:39.887668 10744 catalog_manager.cc:6070] T 00000000000000000000000000000000 P ae957c8fc66c4b6ba063d0c74cbf5484: Generated new TSK 0
I20260709 13:15:39.888283 10744 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:15:39.914988  9682 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:15:39.921213 10755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:39.922591 10756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:39.924474 10758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:39.924618  9682 server_base.cc:1061] running on GCE node
I20260709 13:15:39.925774  9682 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:15:39.925961  9682 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:15:39.926090  9682 hybrid_clock.cc:648] HybridClock initialized: now 1783602939926079 us; error 0 us; skew 500 ppm
I20260709 13:15:39.926537  9682 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:15:39.928783  9682 webserver.cc:533] Webserver started at http://127.9.116.129:38761/ using document root <none> and password file <none>
I20260709 13:15:39.929248  9682 fs_manager.cc:362] Metadata directory not provided
I20260709 13:15:39.929406  9682 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:15:39.929633  9682 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:15:39.930727  9682 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-0-root/instance:
uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
format_stamp: "Formatted at 2026-07-09 13:15:39 on dist-test-slave-1t6m"
I20260709 13:15:39.935496  9682 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.000s
I20260709 13:15:39.938772 10763 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:39.939625  9682 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:15:39.939939  9682 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-0-root
uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
format_stamp: "Formatted at 2026-07-09 13:15:39 on dist-test-slave-1t6m"
I20260709 13:15:39.940203  9682 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:15:39.957297  9682 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:15:39.958619  9682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:15:39.960237  9682 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:15:39.962846  9682 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:15:39.963109  9682 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:39.963374  9682 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:15:39.963568  9682 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:40.008834  9682 rpc_server.cc:307] RPC server started. Bound to: 127.9.116.129:39685
I20260709 13:15:40.008936 10825 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.116.129:39685 every 8 connection(s)
I20260709 13:15:40.013983  9682 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:15:40.021813 10830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:40.022776 10831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:40.024477 10833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:40.027494  9682 server_base.cc:1061] running on GCE node
I20260709 13:15:40.028301 10826 heartbeater.cc:344] Connected to a master server at 127.9.116.190:36007
I20260709 13:15:40.028604  9682 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:15:40.028669 10826 heartbeater.cc:461] Registering TS with master...
W20260709 13:15:40.028923  9682 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:15:40.029170  9682 hybrid_clock.cc:648] HybridClock initialized: now 1783602940029147 us; error 0 us; skew 500 ppm
I20260709 13:15:40.029731 10826 heartbeater.cc:507] Master 127.9.116.190:36007 requested a full tablet report, sending...
I20260709 13:15:40.029796  9682 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:15:40.032313 10699 ts_manager.cc:194] Registered new tserver with Master: e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685)
I20260709 13:15:40.032720  9682 webserver.cc:533] Webserver started at http://127.9.116.130:38267/ using document root <none> and password file <none>
I20260709 13:15:40.033303  9682 fs_manager.cc:362] Metadata directory not provided
I20260709 13:15:40.033546  9682 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:15:40.033838  9682 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:15:40.034422 10699 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42956
I20260709 13:15:40.035633  9682 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-1-root/instance:
uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
format_stamp: "Formatted at 2026-07-09 13:15:40 on dist-test-slave-1t6m"
I20260709 13:15:40.043257  9682 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:15:40.047447 10838 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:40.048332  9682 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.004s	sys 0.001s
I20260709 13:15:40.048604  9682 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-1-root
uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
format_stamp: "Formatted at 2026-07-09 13:15:40 on dist-test-slave-1t6m"
I20260709 13:15:40.048861  9682 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:15:40.061713  9682 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:15:40.063017  9682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:15:40.064481  9682 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:15:40.066865  9682 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:15:40.067098  9682 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:40.067356  9682 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:15:40.067540  9682 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:40.107192  9682 rpc_server.cc:307] RPC server started. Bound to: 127.9.116.130:37895
I20260709 13:15:40.107299 10900 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.116.130:37895 every 8 connection(s)
I20260709 13:15:40.112098  9682 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:15:40.119133 10904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:40.121551 10905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:40.124501  9682 server_base.cc:1061] running on GCE node
W20260709 13:15:40.125372 10907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:40.125968 10901 heartbeater.cc:344] Connected to a master server at 127.9.116.190:36007
I20260709 13:15:40.126389 10901 heartbeater.cc:461] Registering TS with master...
I20260709 13:15:40.126469  9682 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:15:40.126765  9682 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:15:40.127009  9682 hybrid_clock.cc:648] HybridClock initialized: now 1783602940126990 us; error 0 us; skew 500 ppm
I20260709 13:15:40.127389 10901 heartbeater.cc:507] Master 127.9.116.190:36007 requested a full tablet report, sending...
I20260709 13:15:40.127774  9682 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:15:40.129796 10699 ts_manager.cc:194] Registered new tserver with Master: 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:40.130625  9682 webserver.cc:533] Webserver started at http://127.9.116.131:44297/ using document root <none> and password file <none>
I20260709 13:15:40.131170  9682 fs_manager.cc:362] Metadata directory not provided
I20260709 13:15:40.131393  9682 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:15:40.131376 10699 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42968
I20260709 13:15:40.131891  9682 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:15:40.133450  9682 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-2-root/instance:
uuid: "cb0428cc55984daba1ba63744a5102a2"
format_stamp: "Formatted at 2026-07-09 13:15:40 on dist-test-slave-1t6m"
I20260709 13:15:40.138495  9682 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.006s
I20260709 13:15:40.142131 10912 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:40.142952  9682 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:15:40.143265  9682 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-2-root
uuid: "cb0428cc55984daba1ba63744a5102a2"
format_stamp: "Formatted at 2026-07-09 13:15:40 on dist-test-slave-1t6m"
I20260709 13:15:40.143573  9682 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:15:40.170104  9682 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:15:40.171406  9682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:15:40.172917  9682 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:15:40.175289  9682 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:15:40.175534  9682 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:40.175781  9682 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:15:40.175961  9682 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:40.215986  9682 rpc_server.cc:307] RPC server started. Bound to: 127.9.116.131:45553
I20260709 13:15:40.216081 10974 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.116.131:45553 every 8 connection(s)
I20260709 13:15:40.230136 10975 heartbeater.cc:344] Connected to a master server at 127.9.116.190:36007
I20260709 13:15:40.230468 10975 heartbeater.cc:461] Registering TS with master...
I20260709 13:15:40.231160 10975 heartbeater.cc:507] Master 127.9.116.190:36007 requested a full tablet report, sending...
I20260709 13:15:40.233096 10699 ts_manager.cc:194] Registered new tserver with Master: cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:40.233613  9682 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013380319s
I20260709 13:15:40.234428 10699 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42984
I20260709 13:15:40.260773 10699 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42986:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:15:40.267151 10699 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:15:40.497635 10791 tablet_service.cc:1511] Processing CreateTablet for tablet e9764b93d3a8407b8cc1ae641620c831 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:15:40.499284 10791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9764b93d3a8407b8cc1ae641620c831. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.500851 10934 tablet_service.cc:1511] Processing CreateTablet for tablet 96558ab13797493e9c168b86fe3b62cd (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:15:40.501278 10933 tablet_service.cc:1511] Processing CreateTablet for tablet ce92c99123994b928d7a775917eb22ed (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:15:40.500972 10935 tablet_service.cc:1511] Processing CreateTablet for tablet 9b4c7fc28a9b461abe203fe66da89aca (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:15:40.502594 10934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96558ab13797493e9c168b86fe3b62cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.502969 10936 tablet_service.cc:1511] Processing CreateTablet for tablet ddb097c9ce1748458d6719bc91f56e1d (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:15:40.503980 10933 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce92c99123994b928d7a775917eb22ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.505334 10937 tablet_service.cc:1511] Processing CreateTablet for tablet fc429d4d563f415d90d6f74da422a740 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:15:40.508039 10938 tablet_service.cc:1511] Processing CreateTablet for tablet 414992c8073646798991d2b43fd61f1d (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:15:40.510530 10939 tablet_service.cc:1511] Processing CreateTablet for tablet 121ef9771004451eb563dbfd77657e0e (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:15:40.512926 10940 tablet_service.cc:1511] Processing CreateTablet for tablet e9764b93d3a8407b8cc1ae641620c831 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:15:40.515262 10932 tablet_service.cc:1511] Processing CreateTablet for tablet 695ff1b5a68e4111917d909909488d39 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:15:40.517501 10931 tablet_service.cc:1511] Processing CreateTablet for tablet 86ba4060a26c42998e9f220a79f28546 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:15:40.506692 10937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc429d4d563f415d90d6f74da422a740. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.527276 10932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 695ff1b5a68e4111917d909909488d39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.529747 10931 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86ba4060a26c42998e9f220a79f28546. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.530710 10934 tablet_service.cc:1511] Processing CreateTablet for tablet 8cf3aebfc54a4652a83388ef9a48b1c2 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:15:40.532115 10934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cf3aebfc54a4652a83388ef9a48b1c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.535270 10939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 121ef9771004451eb563dbfd77657e0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.539371 10940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9764b93d3a8407b8cc1ae641620c831. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.542636 10938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 414992c8073646798991d2b43fd61f1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.545251 10932 tablet_service.cc:1511] Processing CreateTablet for tablet 417caa22425d4a25aca2f7d8e9577256 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:15:40.546550 10932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 417caa22425d4a25aca2f7d8e9577256. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.549139 10934 tablet_service.cc:1511] Processing CreateTablet for tablet f6e90b642b0d4f06b866170dc0293dc1 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:15:40.550283 10936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddb097c9ce1748458d6719bc91f56e1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.555176 10935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b4c7fc28a9b461abe203fe66da89aca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.564620 10790 tablet_service.cc:1511] Processing CreateTablet for tablet 121ef9771004451eb563dbfd77657e0e (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:15:40.566069 10790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 121ef9771004451eb563dbfd77657e0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.567688 10784 tablet_service.cc:1511] Processing CreateTablet for tablet ce92c99123994b928d7a775917eb22ed (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:15:40.567970 10789 tablet_service.cc:1511] Processing CreateTablet for tablet 414992c8073646798991d2b43fd61f1d (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:15:40.569238 10784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce92c99123994b928d7a775917eb22ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.570828 10786 tablet_service.cc:1511] Processing CreateTablet for tablet 9b4c7fc28a9b461abe203fe66da89aca (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:15:40.572026 10783 tablet_service.cc:1511] Processing CreateTablet for tablet 695ff1b5a68e4111917d909909488d39 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:15:40.572820 10785 tablet_service.cc:1511] Processing CreateTablet for tablet 96558ab13797493e9c168b86fe3b62cd (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:15:40.574083 10782 tablet_service.cc:1511] Processing CreateTablet for tablet 86ba4060a26c42998e9f220a79f28546 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:15:40.574841 10787 tablet_service.cc:1511] Processing CreateTablet for tablet ddb097c9ce1748458d6719bc91f56e1d (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:15:40.576254 10788 tablet_service.cc:1511] Processing CreateTablet for tablet fc429d4d563f415d90d6f74da422a740 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:15:40.577804 10866 tablet_service.cc:1511] Processing CreateTablet for tablet e9764b93d3a8407b8cc1ae641620c831 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:15:40.577803 10786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b4c7fc28a9b461abe203fe66da89aca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.579962 10789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 414992c8073646798991d2b43fd61f1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.579181 10866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9764b93d3a8407b8cc1ae641620c831. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.567729 10932 tablet_service.cc:1511] Processing CreateTablet for tablet 32e87e18e6c9475085e47d6a2fd71a7b (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:15:40.583171 10785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96558ab13797493e9c168b86fe3b62cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.588752 10865 tablet_service.cc:1511] Processing CreateTablet for tablet 121ef9771004451eb563dbfd77657e0e (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:15:40.595798 10865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 121ef9771004451eb563dbfd77657e0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.591212 10864 tablet_service.cc:1511] Processing CreateTablet for tablet 414992c8073646798991d2b43fd61f1d (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:15:40.599189 10783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 695ff1b5a68e4111917d909909488d39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.599622 10864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 414992c8073646798991d2b43fd61f1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.602632 10788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc429d4d563f415d90d6f74da422a740. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.607156 10782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86ba4060a26c42998e9f220a79f28546. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.611763 10787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddb097c9ce1748458d6719bc91f56e1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.612756 10863 tablet_service.cc:1511] Processing CreateTablet for tablet fc429d4d563f415d90d6f74da422a740 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:15:40.614130 10863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc429d4d563f415d90d6f74da422a740. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.579150 10934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6e90b642b0d4f06b866170dc0293dc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.625860 10862 tablet_service.cc:1511] Processing CreateTablet for tablet ddb097c9ce1748458d6719bc91f56e1d (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:15:40.628168 10862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddb097c9ce1748458d6719bc91f56e1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.631036 10861 tablet_service.cc:1511] Processing CreateTablet for tablet 9b4c7fc28a9b461abe203fe66da89aca (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:15:40.632510 10861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b4c7fc28a9b461abe203fe66da89aca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.634451 10935 tablet_service.cc:1511] Processing CreateTablet for tablet a50b3bb87a48494a8e835471e55a6f74 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:15:40.644439 10860 tablet_service.cc:1511] Processing CreateTablet for tablet 96558ab13797493e9c168b86fe3b62cd (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:15:40.645933 10860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96558ab13797493e9c168b86fe3b62cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.647610 10990 tablet_bootstrap.cc:492] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:40.650830 10933 tablet_service.cc:1511] Processing CreateTablet for tablet 76b2d93b69274dbca479bd87d7ea45cb (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:15:40.652225 10858 tablet_service.cc:1511] Processing CreateTablet for tablet 695ff1b5a68e4111917d909909488d39 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:15:40.652611 10857 tablet_service.cc:1511] Processing CreateTablet for tablet 86ba4060a26c42998e9f220a79f28546 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:15:40.655344 10859 tablet_service.cc:1511] Processing CreateTablet for tablet ce92c99123994b928d7a775917eb22ed (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:15:40.653625 10858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 695ff1b5a68e4111917d909909488d39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.658288 10937 tablet_service.cc:1511] Processing CreateTablet for tablet 171abdebfc4142cda86de477a5c89a7e (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:15:40.676817 10936 tablet_service.cc:1511] Processing CreateTablet for tablet 651c4c65e7f44b1fb4053d3757ac417f (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:15:40.678089 10791 tablet_service.cc:1511] Processing CreateTablet for tablet 417caa22425d4a25aca2f7d8e9577256 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:15:40.679270 10859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce92c99123994b928d7a775917eb22ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.683192 10857 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86ba4060a26c42998e9f220a79f28546. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.685088 10931 tablet_service.cc:1511] Processing CreateTablet for tablet e41482a95776477784d9bd9dacfed231 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:15:40.688066 10938 tablet_service.cc:1511] Processing CreateTablet for tablet b253bd741d094067902a02bc765ee0fb (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:15:40.700524 10866 tablet_service.cc:1511] Processing CreateTablet for tablet 8cf3aebfc54a4652a83388ef9a48b1c2 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:15:40.666426 10932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32e87e18e6c9475085e47d6a2fd71a7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.702698 10935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a50b3bb87a48494a8e835471e55a6f74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.703408 10931 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e41482a95776477784d9bd9dacfed231. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.707087 10937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 171abdebfc4142cda86de477a5c89a7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.701918 10866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cf3aebfc54a4652a83388ef9a48b1c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.710798 10785 tablet_service.cc:1511] Processing CreateTablet for tablet 8cf3aebfc54a4652a83388ef9a48b1c2 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:15:40.718998 10938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b253bd741d094067902a02bc765ee0fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.724797 10990 tablet_bootstrap.cc:654] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:40.722692 10783 tablet_service.cc:1511] Processing CreateTablet for tablet f6e90b642b0d4f06b866170dc0293dc1 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:15:40.732159 10788 tablet_service.cc:1511] Processing CreateTablet for tablet a50b3bb87a48494a8e835471e55a6f74 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:15:40.735510 10933 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76b2d93b69274dbca479bd87d7ea45cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.735954 10940 tablet_service.cc:1511] Processing CreateTablet for tablet f15293e178d84fe686fd66bc4439aba8 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:15:40.737300 10940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f15293e178d84fe686fd66bc4439aba8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.726991 10782 tablet_service.cc:1511] Processing CreateTablet for tablet 32e87e18e6c9475085e47d6a2fd71a7b (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:15:40.750597 10788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a50b3bb87a48494a8e835471e55a6f74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.763012 10783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6e90b642b0d4f06b866170dc0293dc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.763804 10785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cf3aebfc54a4652a83388ef9a48b1c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.765547 10782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32e87e18e6c9475085e47d6a2fd71a7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.767091 10791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 417caa22425d4a25aca2f7d8e9577256. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.771808 10859 tablet_service.cc:1511] Processing CreateTablet for tablet a50b3bb87a48494a8e835471e55a6f74 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:15:40.773248 10859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a50b3bb87a48494a8e835471e55a6f74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.754177 10936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 651c4c65e7f44b1fb4053d3757ac417f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.776288 10865 tablet_service.cc:1511] Processing CreateTablet for tablet 32e87e18e6c9475085e47d6a2fd71a7b (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:15:40.777601 10865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32e87e18e6c9475085e47d6a2fd71a7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.778098 10992 tablet_bootstrap.cc:492] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:40.779366 10934 tablet_service.cc:1511] Processing CreateTablet for tablet 741efe79e97646b396f0091dc10c745e (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:15:40.780694 10934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 741efe79e97646b396f0091dc10c745e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.780503 10858 tablet_service.cc:1511] Processing CreateTablet for tablet 76b2d93b69274dbca479bd87d7ea45cb (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:15:40.781919 10858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76b2d93b69274dbca479bd87d7ea45cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.771808 10863 tablet_service.cc:1511] Processing CreateTablet for tablet f6e90b642b0d4f06b866170dc0293dc1 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:15:40.785584 10863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6e90b642b0d4f06b866170dc0293dc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.787652 10862 tablet_service.cc:1511] Processing CreateTablet for tablet 417caa22425d4a25aca2f7d8e9577256 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:15:40.788995 10862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 417caa22425d4a25aca2f7d8e9577256. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.793291 10992 tablet_bootstrap.cc:654] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:40.795950 10864 tablet_service.cc:1511] Processing CreateTablet for tablet b253bd741d094067902a02bc765ee0fb (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:15:40.797353 10991 tablet_bootstrap.cc:492] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:40.798168 10790 tablet_service.cc:1511] Processing CreateTablet for tablet 76b2d93b69274dbca479bd87d7ea45cb (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:15:40.799552 10790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76b2d93b69274dbca479bd87d7ea45cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.804311 10789 tablet_service.cc:1511] Processing CreateTablet for tablet b253bd741d094067902a02bc765ee0fb (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:15:40.798171 10784 tablet_service.cc:1511] Processing CreateTablet for tablet 171abdebfc4142cda86de477a5c89a7e (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:15:40.805718 10789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b253bd741d094067902a02bc765ee0fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.806468 10784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 171abdebfc4142cda86de477a5c89a7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.804306 10786 tablet_service.cc:1511] Processing CreateTablet for tablet f15293e178d84fe686fd66bc4439aba8 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:15:40.815205 10787 tablet_service.cc:1511] Processing CreateTablet for tablet e41482a95776477784d9bd9dacfed231 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:15:40.824527 10864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b253bd741d094067902a02bc765ee0fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.825704 10785 tablet_service.cc:1511] Processing CreateTablet for tablet 651c4c65e7f44b1fb4053d3757ac417f (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:15:40.827416 10786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f15293e178d84fe686fd66bc4439aba8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.835109 10787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e41482a95776477784d9bd9dacfed231. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.842406 10991 tablet_bootstrap.cc:654] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:40.843986 10860 tablet_service.cc:1511] Processing CreateTablet for tablet 171abdebfc4142cda86de477a5c89a7e (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
W20260709 13:15:40.845244 10747 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:15:40.845364 10860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 171abdebfc4142cda86de477a5c89a7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.859805 10931 tablet_service.cc:1511] Processing CreateTablet for tablet 99ca4d7ec0974268ac5f35c5055da891 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:15:40.861202 10931 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99ca4d7ec0974268ac5f35c5055da891. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.863845 10785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 651c4c65e7f44b1fb4053d3757ac417f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.875042 10866 tablet_service.cc:1511] Processing CreateTablet for tablet 651c4c65e7f44b1fb4053d3757ac417f (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:15:40.883647 10932 tablet_service.cc:1511] Processing CreateTablet for tablet 5944b8838e584a3890e4e8143f3f2665 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:15:40.885666 10932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5944b8838e584a3890e4e8143f3f2665. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.886970 10857 tablet_service.cc:1511] Processing CreateTablet for tablet e41482a95776477784d9bd9dacfed231 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:15:40.882531 10782 tablet_service.cc:1511] Processing CreateTablet for tablet 99ca4d7ec0974268ac5f35c5055da891 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:15:40.892577 10782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99ca4d7ec0974268ac5f35c5055da891. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.895977 10938 tablet_service.cc:1511] Processing CreateTablet for tablet 0a574a3bcb5441568f4fd9b1c38d719a (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:15:40.897192 10938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a574a3bcb5441568f4fd9b1c38d719a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.893965 10788 tablet_service.cc:1511] Processing CreateTablet for tablet 741efe79e97646b396f0091dc10c745e (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:15:40.911252 10866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 651c4c65e7f44b1fb4053d3757ac417f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.900018 10784 tablet_service.cc:1511] Processing CreateTablet for tablet 90ad199246b34499aa32795a8082ffa8 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:15:40.901005 10788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 741efe79e97646b396f0091dc10c745e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.912663 10865 tablet_service.cc:1511] Processing CreateTablet for tablet 741efe79e97646b396f0091dc10c745e (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:15:40.931196 10992 tablet_bootstrap.cc:492] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:40.931471 10784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90ad199246b34499aa32795a8082ffa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.931740 10992 ts_tablet_manager.cc:1403] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.161s	user 0.011s	sys 0.015s
I20260709 13:15:40.937314 10791 tablet_service.cc:1511] Processing CreateTablet for tablet 5944b8838e584a3890e4e8143f3f2665 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:15:40.902580 10861 tablet_service.cc:1511] Processing CreateTablet for tablet 90ad199246b34499aa32795a8082ffa8 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:15:40.898674 10859 tablet_service.cc:1511] Processing CreateTablet for tablet f15293e178d84fe686fd66bc4439aba8 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:15:40.949627 10857 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e41482a95776477784d9bd9dacfed231. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.950047 10791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5944b8838e584a3890e4e8143f3f2665. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.951324 10939 tablet_service.cc:1511] Processing CreateTablet for tablet 90ad199246b34499aa32795a8082ffa8 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:15:40.952934 10790 tablet_service.cc:1511] Processing CreateTablet for tablet 37a1fc8ca1b34260908d185a9ebc2bb3 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:15:40.953845 10939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90ad199246b34499aa32795a8082ffa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.954445 10790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37a1fc8ca1b34260908d185a9ebc2bb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.957041 10990 tablet_bootstrap.cc:492] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:40.957567 10990 ts_tablet_manager.cc:1403] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.310s	user 0.042s	sys 0.029s
I20260709 13:15:40.948341 10783 tablet_service.cc:1511] Processing CreateTablet for tablet 0a574a3bcb5441568f4fd9b1c38d719a (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:15:40.960290 10783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a574a3bcb5441568f4fd9b1c38d719a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.934736 10992 raft_consensus.cc:359] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } }
I20260709 13:15:40.967393 10865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 741efe79e97646b396f0091dc10c745e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.967713 10992 raft_consensus.cc:385] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:40.968101 10992 raft_consensus.cc:740] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:40.969071 10992 consensus_queue.cc:260] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } }
I20260709 13:15:40.969282 10863 tablet_service.cc:1511] Processing CreateTablet for tablet 99ca4d7ec0974268ac5f35c5055da891 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:15:40.970578 10863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99ca4d7ec0974268ac5f35c5055da891. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.987730 10859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f15293e178d84fe686fd66bc4439aba8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.008056 10861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90ad199246b34499aa32795a8082ffa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.008527 10931 tablet_service.cc:1511] Processing CreateTablet for tablet 37a1fc8ca1b34260908d185a9ebc2bb3 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:15:41.009872 10931 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37a1fc8ca1b34260908d185a9ebc2bb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.026191 10992 ts_tablet_manager.cc:1434] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.094s	user 0.028s	sys 0.000s
I20260709 13:15:41.027336 10992 tablet_bootstrap.cc:492] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:41.027243 10786 tablet_service.cc:1511] Processing CreateTablet for tablet b012e79b82794524a28ea3e94dee1bb2 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:15:41.028815 10786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b012e79b82794524a28ea3e94dee1bb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.030455 10901 heartbeater.cc:499] Master 127.9.116.190:36007 was elected leader, sending a full tablet report...
I20260709 13:15:41.034189 10992 tablet_bootstrap.cc:654] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:41.035970 10937 tablet_service.cc:1511] Processing CreateTablet for tablet b012e79b82794524a28ea3e94dee1bb2 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:15:41.037277 10937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b012e79b82794524a28ea3e94dee1bb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.037912 10934 tablet_service.cc:1511] Processing CreateTablet for tablet 8949b8b2d08942939f64373bccb71936 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:15:41.038580 10826 heartbeater.cc:499] Master 127.9.116.190:36007 was elected leader, sending a full tablet report...
I20260709 13:15:41.039774 10934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8949b8b2d08942939f64373bccb71936. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.039479 10862 tablet_service.cc:1511] Processing CreateTablet for tablet 5944b8838e584a3890e4e8143f3f2665 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:15:41.043279 10862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5944b8838e584a3890e4e8143f3f2665. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.065711 10788 tablet_service.cc:1511] Processing CreateTablet for tablet 8949b8b2d08942939f64373bccb71936 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:15:41.080463 10991 tablet_bootstrap.cc:492] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:41.081362 10991 ts_tablet_manager.cc:1403] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.284s	user 0.031s	sys 0.022s
I20260709 13:15:41.091424 10990 raft_consensus.cc:359] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:41.092136 10789 tablet_service.cc:1511] Processing CreateTablet for tablet b81df256f0c644cfae9fe2d336cbe9bd (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:15:41.092880 10788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8949b8b2d08942939f64373bccb71936. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.092674 10990 raft_consensus.cc:385] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:41.093470 10990 raft_consensus.cc:740] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:41.094820 10990 consensus_queue.cc:260] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:41.161459 10859 tablet_service.cc:1511] Processing CreateTablet for tablet b012e79b82794524a28ea3e94dee1bb2 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:15:41.098371 10858 tablet_service.cc:1511] Processing CreateTablet for tablet 0a574a3bcb5441568f4fd9b1c38d719a (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:15:41.110731 10789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b81df256f0c644cfae9fe2d336cbe9bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.124291 10864 tablet_service.cc:1511] Processing CreateTablet for tablet 37a1fc8ca1b34260908d185a9ebc2bb3 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:15:41.127933 10991 raft_consensus.cc:359] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:41.163857 10991 raft_consensus.cc:385] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:41.164378 10991 raft_consensus.cc:740] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:41.165633 10991 consensus_queue.cc:260] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:41.141352 10785 tablet_service.cc:1511] Processing CreateTablet for tablet 51ae8de979e44a2d8c155bffc20a1533 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:15:41.146338 10938 tablet_service.cc:1511] Processing CreateTablet for tablet 51ae8de979e44a2d8c155bffc20a1533 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:15:41.149542 10932 tablet_service.cc:1511] Processing CreateTablet for tablet b81df256f0c644cfae9fe2d336cbe9bd (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:15:41.151254 10937 tablet_service.cc:1511] Processing CreateTablet for tablet d00373866c144f4981da73bd7ae39019 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:15:41.151656 10784 tablet_service.cc:1511] Processing CreateTablet for tablet b761e5168d4e48aeaec47b99f5c7d3d0 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:15:41.168694 10938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51ae8de979e44a2d8c155bffc20a1533. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.168851 10784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b761e5168d4e48aeaec47b99f5c7d3d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.153611 10787 tablet_service.cc:1511] Processing CreateTablet for tablet d00373866c144f4981da73bd7ae39019 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:15:41.170598 10787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d00373866c144f4981da73bd7ae39019. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.172894 10937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d00373866c144f4981da73bd7ae39019. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.175616 10932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b81df256f0c644cfae9fe2d336cbe9bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.179862 10785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51ae8de979e44a2d8c155bffc20a1533. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.185777 10992 tablet_bootstrap.cc:492] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:41.186306 10992 ts_tablet_manager.cc:1403] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.159s	user 0.049s	sys 0.078s
I20260709 13:15:41.186967 10859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b012e79b82794524a28ea3e94dee1bb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.186658 10861 tablet_service.cc:1511] Processing CreateTablet for tablet 8949b8b2d08942939f64373bccb71936 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:15:41.187602 10858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a574a3bcb5441568f4fd9b1c38d719a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.189304 10992 raft_consensus.cc:359] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:41.190026 10992 raft_consensus.cc:385] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:41.190339 10992 raft_consensus.cc:740] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:41.192724 10863 tablet_service.cc:1511] Processing CreateTablet for tablet 51ae8de979e44a2d8c155bffc20a1533 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:15:41.193684 10864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37a1fc8ca1b34260908d185a9ebc2bb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.195178 10862 tablet_service.cc:1511] Processing CreateTablet for tablet d00373866c144f4981da73bd7ae39019 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:15:41.196398 10992 consensus_queue.cc:260] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:41.198622 10992 ts_tablet_manager.cc:1434] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:15:41.199245 10863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51ae8de979e44a2d8c155bffc20a1533. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.207990 10931 tablet_service.cc:1511] Processing CreateTablet for tablet b761e5168d4e48aeaec47b99f5c7d3d0 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:15:41.209594 10861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8949b8b2d08942939f64373bccb71936. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.208292 10866 tablet_service.cc:1511] Processing CreateTablet for tablet b761e5168d4e48aeaec47b99f5c7d3d0 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:15:41.211735 10975 heartbeater.cc:499] Master 127.9.116.190:36007 was elected leader, sending a full tablet report...
I20260709 13:15:41.211864 10857 tablet_service.cc:1511] Processing CreateTablet for tablet b81df256f0c644cfae9fe2d336cbe9bd (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:15:41.212301 10990 ts_tablet_manager.cc:1434] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.254s	user 0.013s	sys 0.012s
I20260709 13:15:41.213289 10990 tablet_bootstrap.cc:492] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:41.217538 10939 tablet_service.cc:1511] Processing CreateTablet for tablet 8cab289810294719ac9686e40290a6ab (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:15:41.209282 10931 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b761e5168d4e48aeaec47b99f5c7d3d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.231562 10865 tablet_service.cc:1511] Processing CreateTablet for tablet 8cab289810294719ac9686e40290a6ab (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:15:41.236678 10939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cab289810294719ac9686e40290a6ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.237160 10862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d00373866c144f4981da73bd7ae39019. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.237586 10789 tablet_service.cc:1511] Processing CreateTablet for tablet 8cab289810294719ac9686e40290a6ab (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:15:41.238934 10789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cab289810294719ac9686e40290a6ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.257121 10857 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b81df256f0c644cfae9fe2d336cbe9bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.257802 10866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b761e5168d4e48aeaec47b99f5c7d3d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.260484 10992 tablet_bootstrap.cc:492] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:41.266317 10990 tablet_bootstrap.cc:654] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:41.284678 10865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cab289810294719ac9686e40290a6ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.293962 10934 tablet_service.cc:1511] Processing CreateTablet for tablet 225fa08b9f864105af0cd7a0e7a0d99e (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:15:41.294265 10787 tablet_service.cc:1511] Processing CreateTablet for tablet b4b13050d714473bb01ea0174d634a14 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:15:41.295419 10787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4b13050d714473bb01ea0174d634a14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.286087 10863 tablet_service.cc:1511] Processing CreateTablet for tablet 225fa08b9f864105af0cd7a0e7a0d99e (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:15:41.297253 10863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 225fa08b9f864105af0cd7a0e7a0d99e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.299401 11000 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.295419 10934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 225fa08b9f864105af0cd7a0e7a0d99e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.299942 11000 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:41.300315 10991 ts_tablet_manager.cc:1434] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.218s	user 0.011s	sys 0.006s
I20260709 13:15:41.299449 10992 tablet_bootstrap.cc:654] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:41.304096 11000 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:41.333487 10991 tablet_bootstrap.cc:492] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:41.286084 10790 tablet_service.cc:1511] Processing CreateTablet for tablet 225fa08b9f864105af0cd7a0e7a0d99e (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:15:41.350461 10991 tablet_bootstrap.cc:654] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:41.370813 10996 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.371420 10996 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } }
I20260709 13:15:41.377517 10996 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:41.421934 10991 tablet_bootstrap.cc:492] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:41.422578 10991 ts_tablet_manager.cc:1403] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.090s	user 0.006s	sys 0.010s
I20260709 13:15:41.425769 10991 raft_consensus.cc:359] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:41.428628 10991 raft_consensus.cc:385] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:41.429888 10991 raft_consensus.cc:740] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:41.430657 10991 consensus_queue.cc:260] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:41.432544 10991 ts_tablet_manager.cc:1434] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:15:41.577924 10934 tablet_service.cc:1511] Processing CreateTablet for tablet b4b13050d714473bb01ea0174d634a14 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:15:41.579423 10934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4b13050d714473bb01ea0174d634a14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.586228 10998 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.586817 10998 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:41.589398 10998 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:41.611474 10790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 225fa08b9f864105af0cd7a0e7a0d99e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.619187 10992 tablet_bootstrap.cc:492] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:41.619751 10992 ts_tablet_manager.cc:1403] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.381s	user 0.025s	sys 0.005s
I20260709 13:15:41.619982 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:15:41.621639 10766 leader_election.cc:343] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:41.622860 10992 raft_consensus.cc:359] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:41.623624 10992 raft_consensus.cc:385] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:41.623936 10992 raft_consensus.cc:740] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:41.624555 10992 consensus_queue.cc:260] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:41.628403 10865 tablet_service.cc:1511] Processing CreateTablet for tablet b4b13050d714473bb01ea0174d634a14 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:15:41.629710 10865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4b13050d714473bb01ea0174d634a14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.631201 10992 ts_tablet_manager.cc:1434] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.001s
I20260709 13:15:41.632243 10992 tablet_bootstrap.cc:492] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:41.636535 10991 tablet_bootstrap.cc:492] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:41.638732 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:15:41.643648 10916 leader_election.cc:343] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:41.644917 10990 tablet_bootstrap.cc:492] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:41.645507 10990 ts_tablet_manager.cc:1403] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.432s	user 0.012s	sys 0.019s
I20260709 13:15:41.647073 10996 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.647208 10991 tablet_bootstrap.cc:654] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:41.647567 10996 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:41.649664 10996 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:41.653337 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:15:41.655100 10841 leader_election.cc:343] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:41.655658 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:41.656348 10876 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 0.
I20260709 13:15:41.657624 10916 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:41.658615 10789 tablet_service.cc:1511] Processing CreateTablet for tablet 7595d019b2e74196bc5838962b53798d (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:15:41.659155 10998 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:41.659852 10789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7595d019b2e74196bc5838962b53798d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.659726 10998 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:41.660264 10998 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:41.667229 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:41.638938 10992 tablet_bootstrap.cc:654] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:41.641607 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:15:41.671720 10767 leader_election.cc:343] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:41.672392 10767 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:15:41.672976 10841 leader_election.cc:343] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:41.673480 11000 raft_consensus.cc:2749] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:41.679224 10934 tablet_service.cc:1511] Processing CreateTablet for tablet 7595d019b2e74196bc5838962b53798d (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:15:41.680626 10990 raft_consensus.cc:359] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:41.681536 10990 raft_consensus.cc:385] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:41.681882 10990 raft_consensus.cc:740] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:41.681725 10790 tablet_service.cc:1511] Processing CreateTablet for tablet 747220f0813a4e19a0b4b96bd98d0a2a (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:15:41.683080 10790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 747220f0813a4e19a0b4b96bd98d0a2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.682893 10990 consensus_queue.cc:260] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:41.680629 10934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7595d019b2e74196bc5838962b53798d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.691779 10865 tablet_service.cc:1511] Processing CreateTablet for tablet 7595d019b2e74196bc5838962b53798d (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:15:41.692762 10990 ts_tablet_manager.cc:1434] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.047s	user 0.004s	sys 0.008s
I20260709 13:15:41.692348 10998 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:41.693441 10865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7595d019b2e74196bc5838962b53798d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.695132 10866 tablet_service.cc:1511] Processing CreateTablet for tablet 747220f0813a4e19a0b4b96bd98d0a2a (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:15:41.696177 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:15:41.698868 10916 leader_election.cc:343] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:41.696467 10866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 747220f0813a4e19a0b4b96bd98d0a2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.704362 10931 tablet_service.cc:1511] Processing CreateTablet for tablet 747220f0813a4e19a0b4b96bd98d0a2a (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:15:41.707774 10998 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:41.711061 10785 tablet_service.cc:1511] Processing CreateTablet for tablet 12d33f7a0f8f47e5ad57e854e52ae154 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:15:41.711921 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:15:41.712471 10785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12d33f7a0f8f47e5ad57e854e52ae154. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.712563 10876 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:41.712436 10931 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 747220f0813a4e19a0b4b96bd98d0a2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.722728 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:41.724083 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:41.725744 10990 tablet_bootstrap.cc:492] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
W20260709 13:15:41.727320 10842 leader_election.cc:343] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:41.727795 10842 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:41.728638 10996 raft_consensus.cc:2749] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:15:41.728953 10842 leader_election.cc:343] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:41.729434 10842 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:41.731603 10996 raft_consensus.cc:2749] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:41.737360 10991 tablet_bootstrap.cc:492] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:41.738121 10991 ts_tablet_manager.cc:1403] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.102s	user 0.023s	sys 0.028s
I20260709 13:15:41.739619 10876 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 1.
I20260709 13:15:41.740900 10916 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:41.741037 10991 raft_consensus.cc:359] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } }
I20260709 13:15:41.742046 10991 raft_consensus.cc:385] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:41.742408 10991 raft_consensus.cc:740] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:41.748504 10934 tablet_service.cc:1511] Processing CreateTablet for tablet 12d33f7a0f8f47e5ad57e854e52ae154 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:15:41.754081 10865 tablet_service.cc:1511] Processing CreateTablet for tablet 12d33f7a0f8f47e5ad57e854e52ae154 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:15:41.755446 10865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12d33f7a0f8f47e5ad57e854e52ae154. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.754173 10785 tablet_service.cc:1511] Processing CreateTablet for tablet 1d9e744bb20c4325969f4fbc01c672e8 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:15:41.757999 10785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d9e744bb20c4325969f4fbc01c672e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.754448 10991 consensus_queue.cc:260] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } }
I20260709 13:15:41.763517 10991 ts_tablet_manager.cc:1434] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.025s	user 0.001s	sys 0.005s
I20260709 13:15:41.750842 10990 tablet_bootstrap.cc:654] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:41.751741 10939 tablet_service.cc:1511] Processing CreateTablet for tablet 1d9e744bb20c4325969f4fbc01c672e8 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:15:41.742424 11015 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:41.764802 10991 tablet_bootstrap.cc:492] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:41.764828 11015 raft_consensus.cc:697] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:15:41.765976 11015 consensus_queue.cc:237] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:41.754513 10934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12d33f7a0f8f47e5ad57e854e52ae154. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.767587 10866 tablet_service.cc:1511] Processing CreateTablet for tablet 1d9e744bb20c4325969f4fbc01c672e8 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:15:41.768867 10866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d9e744bb20c4325969f4fbc01c672e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.771104 10939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d9e744bb20c4325969f4fbc01c672e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.771697 10991 tablet_bootstrap.cc:654] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:41.750401 10992 tablet_bootstrap.cc:492] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:41.775221 10992 ts_tablet_manager.cc:1403] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.143s	user 0.010s	sys 0.011s
I20260709 13:15:41.783411 10991 tablet_bootstrap.cc:492] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:41.783999 10991 ts_tablet_manager.cc:1403] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.019s	user 0.015s	sys 0.003s
I20260709 13:15:41.784979 10990 tablet_bootstrap.cc:492] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:41.785535 10990 ts_tablet_manager.cc:1403] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.060s	user 0.007s	sys 0.003s
I20260709 13:15:41.787384 10991 raft_consensus.cc:359] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:41.788276 10991 raft_consensus.cc:385] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:41.788597 10992 raft_consensus.cc:359] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:41.788601 10991 raft_consensus.cc:740] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:41.790436 10991 consensus_queue.cc:260] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:41.789569 10990 raft_consensus.cc:359] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:41.791392 10990 raft_consensus.cc:385] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:41.791729 10990 raft_consensus.cc:740] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:41.789362 10992 raft_consensus.cc:385] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:41.792529 10992 raft_consensus.cc:740] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:41.793509 10992 consensus_queue.cc:260] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:41.794610 10991 ts_tablet_manager.cc:1434] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:15:41.795718 10991 tablet_bootstrap.cc:492] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:41.796627 10785 tablet_service.cc:1511] Processing CreateTablet for tablet 4dd4c0ff1def4960931dda3b52a4a884 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:15:41.798089 10785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4dd4c0ff1def4960931dda3b52a4a884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.797636 10990 consensus_queue.cc:260] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:41.799965 10990 ts_tablet_manager.cc:1434] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.001s
I20260709 13:15:41.800920 10990 tablet_bootstrap.cc:492] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:41.803094 10991 tablet_bootstrap.cc:654] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:41.808947 10990 tablet_bootstrap.cc:654] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:41.814956 10991 tablet_bootstrap.cc:492] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:41.815559 10991 ts_tablet_manager.cc:1403] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.005s
I20260709 13:15:41.816721 10992 ts_tablet_manager.cc:1434] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.041s	user 0.012s	sys 0.008s
I20260709 13:15:41.818621 10991 raft_consensus.cc:359] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:41.819671 10991 raft_consensus.cc:385] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:41.820021 10991 raft_consensus.cc:740] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:41.820924 10991 consensus_queue.cc:260] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:41.823119 10992 tablet_bootstrap.cc:492] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:41.826186 10991 ts_tablet_manager.cc:1434] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:15:41.827355 10991 tablet_bootstrap.cc:492] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:41.829855 10699 catalog_manager.cc:5697] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 1 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:41.832392 10992 tablet_bootstrap.cc:654] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:41.833873 10990 tablet_bootstrap.cc:492] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:41.834393 10990 ts_tablet_manager.cc:1403] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.034s	user 0.008s	sys 0.022s
I20260709 13:15:41.835511 10991 tablet_bootstrap.cc:654] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:41.838266 10990 raft_consensus.cc:359] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:41.840653 10990 raft_consensus.cc:385] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:41.841207 10990 raft_consensus.cc:740] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:41.843366 10990 consensus_queue.cc:260] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:41.843626 10866 tablet_service.cc:1511] Processing CreateTablet for tablet 4dd4c0ff1def4960931dda3b52a4a884 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:15:41.844941 10866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4dd4c0ff1def4960931dda3b52a4a884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.845752 10990 ts_tablet_manager.cc:1434] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.003s
I20260709 13:15:41.846709 10990 tablet_bootstrap.cc:492] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:41.847795 10939 tablet_service.cc:1511] Processing CreateTablet for tablet 4dd4c0ff1def4960931dda3b52a4a884 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:15:41.849050 10939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4dd4c0ff1def4960931dda3b52a4a884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.862469 10790 tablet_service.cc:1511] Processing CreateTablet for tablet bc6987a00cef4dff9545eee242db0982 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:15:41.863886 10790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc6987a00cef4dff9545eee242db0982. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.864698 10865 tablet_service.cc:1511] Processing CreateTablet for tablet bc6987a00cef4dff9545eee242db0982 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:15:41.865964 10865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc6987a00cef4dff9545eee242db0982. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.866096 11000 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.866545 11000 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:41.868734 11000 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:41.870396 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:41.871073 10876 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 0.
I20260709 13:15:41.872522 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:41.873157 10950 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 0.
I20260709 13:15:41.873558 10992 tablet_bootstrap.cc:492] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:41.874037 10992 ts_tablet_manager.cc:1403] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.051s	user 0.016s	sys 0.004s
I20260709 13:15:41.874509 10766 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:15:41.877157 11000 raft_consensus.cc:2804] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:41.877544 11000 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:41.877883 11000 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:15:41.880985 10747 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:15:41.882177 10931 tablet_service.cc:1511] Processing CreateTablet for tablet bc6987a00cef4dff9545eee242db0982 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:15:41.882936 10990 tablet_bootstrap.cc:654] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:41.883598 10931 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc6987a00cef4dff9545eee242db0982. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.890982 11000 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:41.893956 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:15:41.894550 10876 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:41.897004 10992 raft_consensus.cc:359] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:41.897737 10992 raft_consensus.cc:385] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:41.898072 10992 raft_consensus.cc:740] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:41.897881 10789 tablet_service.cc:1511] Processing CreateTablet for tablet 9354ad0d744d43bbbe2ce992ecbc4e65 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:15:41.900154 11000 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:41.900734 10789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9354ad0d744d43bbbe2ce992ecbc4e65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.901248 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:15:41.901834 10950 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:41.901146 10992 consensus_queue.cc:260] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:41.905261 10934 tablet_service.cc:1511] Processing CreateTablet for tablet 9354ad0d744d43bbbe2ce992ecbc4e65 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:15:41.906411 10934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9354ad0d744d43bbbe2ce992ecbc4e65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.906502 10787 tablet_service.cc:1511] Processing CreateTablet for tablet 8fd9828f212b47c19244738ee2a27a95 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:15:41.907697 10787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fd9828f212b47c19244738ee2a27a95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.907647 10876 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 1.
I20260709 13:15:41.909003 10767 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:15:41.910689 10950 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 1.
I20260709 13:15:41.911285 11000 raft_consensus.cc:2804] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:41.918186 10857 tablet_service.cc:1511] Processing CreateTablet for tablet 8fd9828f212b47c19244738ee2a27a95 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:15:41.919566 10857 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fd9828f212b47c19244738ee2a27a95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.919688 10865 tablet_service.cc:1511] Processing CreateTablet for tablet 9354ad0d744d43bbbe2ce992ecbc4e65 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:15:41.920826 10865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9354ad0d744d43bbbe2ce992ecbc4e65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.932905 10937 tablet_service.cc:1511] Processing CreateTablet for tablet 8fd9828f212b47c19244738ee2a27a95 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:15:41.933529 10990 tablet_bootstrap.cc:492] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:41.934065 10990 ts_tablet_manager.cc:1403] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.088s	user 0.011s	sys 0.003s
I20260709 13:15:41.934237 10937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fd9828f212b47c19244738ee2a27a95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.937000 10990 raft_consensus.cc:359] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:41.937765 10990 raft_consensus.cc:385] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:41.938078 10990 raft_consensus.cc:740] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:41.938779 10990 consensus_queue.cc:260] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:41.946815 10990 ts_tablet_manager.cc:1434] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:15:41.942567 10788 tablet_service.cc:1511] Processing CreateTablet for tablet 185480268a684958b8347a4f53af57c5 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:15:41.947814 10990 tablet_bootstrap.cc:492] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:41.948066 10788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 185480268a684958b8347a4f53af57c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.954115 10990 tablet_bootstrap.cc:654] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:41.946676 10861 tablet_service.cc:1511] Processing CreateTablet for tablet 185480268a684958b8347a4f53af57c5 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:15:41.962953 10932 tablet_service.cc:1511] Processing CreateTablet for tablet 185480268a684958b8347a4f53af57c5 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:15:41.963433 10991 tablet_bootstrap.cc:492] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:41.964167 10932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 185480268a684958b8347a4f53af57c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.964120 10991 ts_tablet_manager.cc:1403] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.137s	user 0.019s	sys 0.014s
I20260709 13:15:41.964039 10861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 185480268a684958b8347a4f53af57c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.971655 11000 raft_consensus.cc:697] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:15:41.975675 11000 consensus_queue.cc:237] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:41.979045 10787 tablet_service.cc:1511] Processing CreateTablet for tablet 35190606a0c04bc4bac6c8ca7df14ed3 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:15:41.980562 10787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35190606a0c04bc4bac6c8ca7df14ed3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.977401 10865 tablet_service.cc:1511] Processing CreateTablet for tablet 35190606a0c04bc4bac6c8ca7df14ed3 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:15:41.984335 10992 ts_tablet_manager.cc:1434] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.110s	user 0.010s	sys 0.000s
I20260709 13:15:41.985785 10992 tablet_bootstrap.cc:492] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:41.988947 10865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35190606a0c04bc4bac6c8ca7df14ed3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:42.001519 11026 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.002054 11026 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } }
I20260709 13:15:42.003762 10991 raft_consensus.cc:359] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.004489 10991 raft_consensus.cc:385] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:42.004766 10991 raft_consensus.cc:740] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:42.005208 10789 tablet_service.cc:1511] Processing CreateTablet for tablet d32f5e230e464e61b23a41962708d120 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:15:42.005475 10991 consensus_queue.cc:260] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:41.995647 11015 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.006507 10789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d32f5e230e464e61b23a41962708d120. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:42.006677 11015 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.008741 11015 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:42.009771 10785 tablet_service.cc:1511] Processing CreateTablet for tablet 2b82f18992434ed5ba58b805107060d4 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:15:42.010345 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:42.011211 10801 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 0.
I20260709 13:15:42.011511 10785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b82f18992434ed5ba58b805107060d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:42.009815 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:15:42.013929 10916 leader_election.cc:343] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:42.015574 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:15:42.017463 10766 leader_election.cc:343] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:42.020061 10939 tablet_service.cc:1511] Processing CreateTablet for tablet 35190606a0c04bc4bac6c8ca7df14ed3 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:15:42.021482 10939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35190606a0c04bc4bac6c8ca7df14ed3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:42.025046 10916 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:15:42.025977 11015 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.026192 10998 raft_consensus.cc:2804] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:42.026610 10998 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:42.026481 11015 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.027127 10998 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:42.028739 11015 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:42.036022 10991 ts_tablet_manager.cc:1434] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.071s	user 0.009s	sys 0.001s
I20260709 13:15:42.035878 10931 tablet_service.cc:1511] Processing CreateTablet for tablet d32f5e230e464e61b23a41962708d120 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:15:42.041281 10998 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.044700 10934 tablet_service.cc:1511] Processing CreateTablet for tablet 2b82f18992434ed5ba58b805107060d4 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:15:42.045061 10998 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:42.045419 10931 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d32f5e230e464e61b23a41962708d120. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:42.046054 10992 tablet_bootstrap.cc:654] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:42.046192 10934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b82f18992434ed5ba58b805107060d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:42.049702 10863 tablet_service.cc:1511] Processing CreateTablet for tablet d32f5e230e464e61b23a41962708d120 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:15:42.050419 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:42.051055 10863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d32f5e230e464e61b23a41962708d120. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:42.051806 10866 tablet_service.cc:1511] Processing CreateTablet for tablet 2b82f18992434ed5ba58b805107060d4 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:15:42.053172 10866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b82f18992434ed5ba58b805107060d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:42.053246 11026 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:42.053692 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:15:42.059270 10916 leader_election.cc:343] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:15:42.060073 10916 leader_election.cc:343] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:42.059428 10991 tablet_bootstrap.cc:492] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:42.061630 10790 tablet_service.cc:1511] Processing CreateTablet for tablet 73af6c7e9b8e4ffaa65ba65a6199eb61 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:15:42.062948 10790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73af6c7e9b8e4ffaa65ba65a6199eb61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:42.068195 10991 tablet_bootstrap.cc:654] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:42.072103 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:42.100996 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:15:42.101879 10800 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:15:42.108763 10916 leader_election.cc:343] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:42.109249 10916 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:42.110944 11028 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.111429 11028 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.114876 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:15:42.116367 10766 leader_election.cc:343] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:42.116936 11028 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:42.117662 10998 raft_consensus.cc:2749] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:42.118911 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:42.119571 10876 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 0.
I20260709 13:15:42.119472 10934 tablet_service.cc:1511] Processing CreateTablet for tablet 73af6c7e9b8e4ffaa65ba65a6199eb61 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:15:42.120649 10934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73af6c7e9b8e4ffaa65ba65a6199eb61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:42.122810 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:42.125224 10866 tablet_service.cc:1511] Processing CreateTablet for tablet 73af6c7e9b8e4ffaa65ba65a6199eb61 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:15:42.126399 10866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73af6c7e9b8e4ffaa65ba65a6199eb61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:42.130674 10789 tablet_service.cc:1511] Processing CreateTablet for tablet ad1b662471ff4b9ebda3845b4f224a34 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:15:42.132043 10789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad1b662471ff4b9ebda3845b4f224a34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:42.132421 10865 tablet_service.cc:1511] Processing CreateTablet for tablet ad1b662471ff4b9ebda3845b4f224a34 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:15:42.133544 10865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad1b662471ff4b9ebda3845b4f224a34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:42.135003 10767 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:15:42.135985 11031 raft_consensus.cc:2804] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:15:42.136196 10767 leader_election.cc:343] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:42.136423 11031 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:42.136700 10767 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:15:42.136816 11031 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:42.139040 11028 raft_consensus.cc:2749] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:42.140481 10990 tablet_bootstrap.cc:492] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:42.140980 10990 ts_tablet_manager.cc:1403] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.193s	user 0.005s	sys 0.034s
I20260709 13:15:42.148119 10931 tablet_service.cc:1511] Processing CreateTablet for tablet ad1b662471ff4b9ebda3845b4f224a34 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:15:42.152778 10788 tablet_service.cc:1511] Processing CreateTablet for tablet 90ea9e6b562e4600be1cb566739a6a30 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:15:42.153745 10996 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.155198 10931 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad1b662471ff4b9ebda3845b4f224a34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:42.159849 10996 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.168766 10996 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:42.170610 11031 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } }
I20260709 13:15:42.171172 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:42.171783 10801 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 0.
I20260709 13:15:42.172765 11031 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:42.173113 10842 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:15:42.173640 10788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90ea9e6b562e4600be1cb566739a6a30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:42.173458 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:15:42.170373 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:15:42.174957 10766 leader_election.cc:343] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:15:42.175426 10841 leader_election.cc:343] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:42.175658 10696 catalog_manager.cc:5697] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 0 to 1, leader changed from <none> to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 1 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:42.178308 10996 raft_consensus.cc:2804] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:42.178809 10996 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:42.179230 10996 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:42.180966 10800 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 1.
I20260709 13:15:42.182251 10916 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:15:42.183386 10998 raft_consensus.cc:2804] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:42.183869 10998 raft_consensus.cc:697] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:15:42.184691 10998 consensus_queue.cc:237] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.185591 10934 tablet_service.cc:1511] Processing CreateTablet for tablet 418b3711416f4a3e8fa80068035b5dd8 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:15:42.186769 10934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 418b3711416f4a3e8fa80068035b5dd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:42.187493 10992 tablet_bootstrap.cc:492] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:42.187992 10992 ts_tablet_manager.cc:1403] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.203s	user 0.013s	sys 0.023s
I20260709 13:15:42.189561 10785 tablet_service.cc:1511] Processing CreateTablet for tablet 418b3711416f4a3e8fa80068035b5dd8 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:15:42.190724 10785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 418b3711416f4a3e8fa80068035b5dd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:42.190584 10866 tablet_service.cc:1511] Processing CreateTablet for tablet 90ea9e6b562e4600be1cb566739a6a30 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:15:42.191475 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:15:42.192030 10866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90ea9e6b562e4600be1cb566739a6a30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:42.192095 10876 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:42.189558 10784 tablet_service.cc:1511] Processing CreateTablet for tablet e54874d9d0dc4cb6a82081a61f03a309 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:15:42.193477 10784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e54874d9d0dc4cb6a82081a61f03a309. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:42.193403 10937 tablet_service.cc:1511] Processing CreateTablet for tablet 90ea9e6b562e4600be1cb566739a6a30 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:15:42.194669 10937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90ea9e6b562e4600be1cb566739a6a30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:42.192057 10939 tablet_service.cc:1511] Processing CreateTablet for tablet e54874d9d0dc4cb6a82081a61f03a309 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:15:42.196731 10939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e54874d9d0dc4cb6a82081a61f03a309. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:42.200388 10863 tablet_service.cc:1511] Processing CreateTablet for tablet 418b3711416f4a3e8fa80068035b5dd8 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:15:42.201656 10863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 418b3711416f4a3e8fa80068035b5dd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:42.209704 10876 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 1.
I20260709 13:15:42.217463 11015 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.222711 11015 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.224884 11015 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:42.225217 10996 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.226815 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:42.222714 10767 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:15:42.229363 11031 raft_consensus.cc:2804] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:42.229820 11031 raft_consensus.cc:697] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:15:42.230611 11031 consensus_queue.cc:237] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } }
W20260709 13:15:42.234495 10916 leader_election.cc:343] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:42.222853 10861 tablet_service.cc:1511] Processing CreateTablet for tablet e54874d9d0dc4cb6a82081a61f03a309 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:15:42.236603 10861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e54874d9d0dc4cb6a82081a61f03a309. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:42.236496 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:15:42.237130 10800 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:42.240088 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:42.241037 10991 tablet_bootstrap.cc:492] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
W20260709 13:15:42.241391 10916 leader_election.cc:343] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:42.240003 10790 tablet_service.cc:1511] Processing CreateTablet for tablet 2778f463396643809ca1f6bef2755afc (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:15:42.241520 10991 ts_tablet_manager.cc:1403] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.182s	user 0.010s	sys 0.014s
I20260709 13:15:42.241855 10916 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:42.217669 10990 raft_consensus.cc:359] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.243224 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:15:42.259482 10696 catalog_manager.cc:5697] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 0 to 1, leader changed from <none> to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 1 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:42.272380 10992 raft_consensus.cc:359] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.273164 10992 raft_consensus.cc:385] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:42.273473 10992 raft_consensus.cc:740] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
W20260709 13:15:42.273520 10841 leader_election.cc:343] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:42.274257 10990 raft_consensus.cc:385] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:42.244063 11015 raft_consensus.cc:2749] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:42.244830 10790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2778f463396643809ca1f6bef2755afc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:42.274374 10992 consensus_queue.cc:260] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.243327 10697 catalog_manager.cc:5697] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 1 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:42.262643 10998 consensus_queue.cc:1048] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:42.285213 10939 tablet_service.cc:1511] Processing CreateTablet for tablet 2778f463396643809ca1f6bef2755afc (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:15:42.243779 10996 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:42.258986 10800 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 1.
I20260709 13:15:42.274626 10990 raft_consensus.cc:740] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:42.288506 10990 consensus_queue.cc:260] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.289739 10939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2778f463396643809ca1f6bef2755afc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:42.291203 10842 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:15:42.292340 10996 raft_consensus.cc:2804] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:42.292824 10996 raft_consensus.cc:697] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:15:42.286839 10931 tablet_service.cc:1511] Processing CreateTablet for tablet dde624b07ed446e5b7347791196396e2 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:15:42.293648 10996 consensus_queue.cc:237] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.294479 10931 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dde624b07ed446e5b7347791196396e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:42.295868 10861 tablet_service.cc:1511] Processing CreateTablet for tablet dde624b07ed446e5b7347791196396e2 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:15:42.297133 10861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dde624b07ed446e5b7347791196396e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:42.297542 10990 ts_tablet_manager.cc:1434] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.156s	user 0.009s	sys 0.000s
I20260709 13:15:42.298583 10990 tablet_bootstrap.cc:492] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:42.307883 10788 tablet_service.cc:1511] Processing CreateTablet for tablet dde624b07ed446e5b7347791196396e2 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:15:42.309252 10788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dde624b07ed446e5b7347791196396e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:42.246881 10991 raft_consensus.cc:359] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.315069 10991 raft_consensus.cc:385] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:42.315501 10991 raft_consensus.cc:740] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:42.316640 10934 tablet_service.cc:1511] Processing CreateTablet for tablet 9094285c302f4de4aab986dc171c11ff (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:15:42.323546 10934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9094285c302f4de4aab986dc171c11ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:42.323397 10991 consensus_queue.cc:260] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.318850 10784 tablet_service.cc:1511] Processing CreateTablet for tablet 9094285c302f4de4aab986dc171c11ff (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:15:42.321027 10865 tablet_service.cc:1511] Processing CreateTablet for tablet 9094285c302f4de4aab986dc171c11ff (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:15:42.327728 10865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9094285c302f4de4aab986dc171c11ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:42.327720 10784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9094285c302f4de4aab986dc171c11ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:42.331434 10863 tablet_service.cc:1511] Processing CreateTablet for tablet 2778f463396643809ca1f6bef2755afc (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:15:42.332769 10863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2778f463396643809ca1f6bef2755afc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:42.334120 10991 ts_tablet_manager.cc:1434] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.090s	user 0.007s	sys 0.010s
I20260709 13:15:42.322077 10990 tablet_bootstrap.cc:654] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:42.341656 10785 tablet_service.cc:1511] Processing CreateTablet for tablet 416a6d12184f4fbda69570b95be47851 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:15:42.342901 10785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 416a6d12184f4fbda69570b95be47851. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:42.342988 10991 tablet_bootstrap.cc:492] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:42.355829 10992 ts_tablet_manager.cc:1434] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.168s	user 0.011s	sys 0.014s
I20260709 13:15:42.356928 10992 tablet_bootstrap.cc:492] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:42.356230 10697 catalog_manager.cc:5697] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 0 to 1, leader changed from <none> to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:42.358026 10800 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:42.363583 10990 tablet_bootstrap.cc:492] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:42.364081 10990 ts_tablet_manager.cc:1403] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.066s	user 0.009s	sys 0.003s
I20260709 13:15:42.367012 10990 raft_consensus.cc:359] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.367736 10990 raft_consensus.cc:385] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:42.368023 10990 raft_consensus.cc:740] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:42.368775 10992 tablet_bootstrap.cc:654] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:42.371521 10991 tablet_bootstrap.cc:654] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:42.378448 10998 consensus_queue.cc:1048] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:42.370343 10990 consensus_queue.cc:260] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.379669 11015 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.380074 10992 tablet_bootstrap.cc:492] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:42.380138 11015 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.380643 10992 ts_tablet_manager.cc:1403] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.003s
I20260709 13:15:42.383960 10992 raft_consensus.cc:359] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.384702 10992 raft_consensus.cc:385] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:42.385010 10992 raft_consensus.cc:740] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:42.385187 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:42.385826 10799 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 0.
I20260709 13:15:42.385913 10992 consensus_queue.cc:260] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.387182 10916 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:15:42.387986 10998 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:42.388373 11015 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:42.388620 10992 ts_tablet_manager.cc:1434] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.002s
I20260709 13:15:42.388443 10998 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:42.389081 10998 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:42.390187 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:15:42.391705 10916 leader_election.cc:343] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:42.393958 10992 tablet_bootstrap.cc:492] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:42.395341 10863 tablet_service.cc:1511] Processing CreateTablet for tablet 416a6d12184f4fbda69570b95be47851 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:15:42.396862 10863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 416a6d12184f4fbda69570b95be47851. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:42.397014 10998 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.400280 10992 tablet_bootstrap.cc:654] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:42.401690 10990 ts_tablet_manager.cc:1434] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.037s	user 0.010s	sys 0.000s
I20260709 13:15:42.402732 10990 tablet_bootstrap.cc:492] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:42.411778 10939 tablet_service.cc:1511] Processing CreateTablet for tablet 416a6d12184f4fbda69570b95be47851 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:15:42.412549 10998 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:42.413089 10788 tablet_service.cc:1511] Processing CreateTablet for tablet 5c8a233bc6c44bd2b5d2d44a726b147c (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:15:42.414492 10788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c8a233bc6c44bd2b5d2d44a726b147c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:42.414547 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:15:42.415855 10916 leader_election.cc:343] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:42.421164 10939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 416a6d12184f4fbda69570b95be47851. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:42.413491 10990 tablet_bootstrap.cc:654] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:42.438414 10865 tablet_service.cc:1511] Processing CreateTablet for tablet 5c8a233bc6c44bd2b5d2d44a726b147c (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:15:42.439659 10865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c8a233bc6c44bd2b5d2d44a726b147c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:42.439800 10937 tablet_service.cc:1511] Processing CreateTablet for tablet 5c8a233bc6c44bd2b5d2d44a726b147c (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:15:42.441463 10937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c8a233bc6c44bd2b5d2d44a726b147c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:42.445354 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:15:42.446439 10800 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:15:42.456491 10897 debug-util.cc:398] Leaking SignalData structure 0x7b08001601c0 after lost signal to thread 9685
W20260709 13:15:42.457595 10897 debug-util.cc:398] Leaking SignalData structure 0x7b08000bd1a0 after lost signal to thread 10733
W20260709 13:15:42.467567 10897 debug-util.cc:398] Leaking SignalData structure 0x7b0800087b60 after lost signal to thread 10974
I20260709 13:15:42.469031 11031 consensus_queue.cc:1048] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:42.481284 10991 tablet_bootstrap.cc:492] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:42.481285 10990 tablet_bootstrap.cc:492] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:42.482007 10991 ts_tablet_manager.cc:1403] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.139s	user 0.007s	sys 0.011s
I20260709 13:15:42.482007 10990 ts_tablet_manager.cc:1403] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.079s	user 0.007s	sys 0.007s
I20260709 13:15:42.484825 10990 raft_consensus.cc:359] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.485510 10990 raft_consensus.cc:385] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:42.485776 10990 raft_consensus.cc:740] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:42.488123 10991 raft_consensus.cc:359] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.488894 10991 raft_consensus.cc:385] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:42.489228 10991 raft_consensus.cc:740] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:42.490032 10800 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 1.
I20260709 13:15:42.490804 10990 consensus_queue.cc:260] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.490084 10991 consensus_queue.cc:260] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.494159 10991 ts_tablet_manager.cc:1434] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.012s	user 0.002s	sys 0.005s
I20260709 13:15:42.495344 10916 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:15:42.500192 10937 tablet_service.cc:1511] Processing CreateTablet for tablet 19c20efd6c04443596bb712e3c93c650 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:15:42.501578 10937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19c20efd6c04443596bb712e3c93c650. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:42.507876 10863 tablet_service.cc:1511] Processing CreateTablet for tablet 19c20efd6c04443596bb712e3c93c650 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:15:42.509186 10863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19c20efd6c04443596bb712e3c93c650. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:42.510438 10785 tablet_service.cc:1511] Processing CreateTablet for tablet 19c20efd6c04443596bb712e3c93c650 (DEFAULT_TABLE table=test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:15:42.511698 10785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19c20efd6c04443596bb712e3c93c650. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:42.512218 10990 ts_tablet_manager.cc:1434] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.030s	user 0.003s	sys 0.003s
I20260709 13:15:42.512989 10990 tablet_bootstrap.cc:492] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:42.514220 10991 tablet_bootstrap.cc:492] T 414992c8073646798991d2b43fd61f1d P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:42.510350 10998 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:42.515213 10998 raft_consensus.cc:697] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:15:42.516129 10998 consensus_queue.cc:237] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.517818 10992 tablet_bootstrap.cc:492] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:42.518299 10992 ts_tablet_manager.cc:1403] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.125s	user 0.039s	sys 0.038s
I20260709 13:15:42.519364 11015 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.522430 10992 raft_consensus.cc:359] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.522601 11015 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.523197 10992 raft_consensus.cc:385] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:42.523485 10992 raft_consensus.cc:740] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:42.524132 10992 consensus_queue.cc:260] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.528507 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:42.524683 11015 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:42.529023 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:42.530916 10992 ts_tablet_manager.cc:1434] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:15:42.529327 10800 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 0.
I20260709 13:15:42.532248 10992 tablet_bootstrap.cc:492] T 86ba4060a26c42998e9f220a79f28546 P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:42.532333 10916 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:15:42.533300 10916 leader_election.cc:343] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:42.534432 11015 raft_consensus.cc:2804] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:42.538211 11015 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:42.539008 11015 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:42.545105 10990 tablet_bootstrap.cc:654] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:42.551992 11031 consensus_queue.cc:1048] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:15:42.559259 11040 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.559804 11040 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.553738 11015 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.562611 11040 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:42.565038 11015 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:42.568701 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:15:42.569505 10991 tablet_bootstrap.cc:654] T 414992c8073646798991d2b43fd61f1d P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:42.569381 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:42.570194 10800 raft_consensus.cc:2393] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 1: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:15:42.571151 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:15:42.571775 10801 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:15:42.578035 10916 leader_election.cc:343] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:42.583700 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:42.584779 10801 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 1.
I20260709 13:15:42.585196 10841 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:42.586395 11040 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:42.587476 10992 tablet_bootstrap.cc:654] T 86ba4060a26c42998e9f220a79f28546 P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:42.591284 10990 tablet_bootstrap.cc:492] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:42.591940 10990 ts_tablet_manager.cc:1403] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.079s	user 0.010s	sys 0.002s
I20260709 13:15:42.589849 10916 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:15:42.593271 11015 raft_consensus.cc:2804] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:42.594035 11015 raft_consensus.cc:697] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:15:42.592116 10996 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.594863 10996 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.595489 10990 raft_consensus.cc:359] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.596387 10990 raft_consensus.cc:385] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:42.596753 11040 raft_consensus.cc:2749] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:42.596751 10990 raft_consensus.cc:740] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:42.598749 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:42.598716 10990 consensus_queue.cc:260] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.599797 10801 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 0.
I20260709 13:15:42.601138 10990 ts_tablet_manager.cc:1434] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:15:42.602054 10990 tablet_bootstrap.cc:492] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:42.601732 10842 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:15:42.602221 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:15:42.603714 10841 leader_election.cc:343] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:42.603027 11015 consensus_queue.cc:237] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.604847 10998 consensus_queue.cc:1048] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:42.596944 10996 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:42.606832 10996 raft_consensus.cc:2804] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:42.607334 10996 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:42.607779 10996 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:42.616458 10990 tablet_bootstrap.cc:654] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:42.640506 10697 catalog_manager.cc:5697] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 1 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:42.655222 10992 tablet_bootstrap.cc:492] T 86ba4060a26c42998e9f220a79f28546 P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:42.655763 10992 ts_tablet_manager.cc:1403] T 86ba4060a26c42998e9f220a79f28546 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.124s	user 0.018s	sys 0.017s
I20260709 13:15:42.658488 10990 tablet_bootstrap.cc:492] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:42.658991 10990 ts_tablet_manager.cc:1403] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.057s	user 0.020s	sys 0.000s
I20260709 13:15:42.660200 10992 raft_consensus.cc:359] T 86ba4060a26c42998e9f220a79f28546 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.661100 10992 raft_consensus.cc:385] T 86ba4060a26c42998e9f220a79f28546 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:42.656133 10996 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.663802 10991 tablet_bootstrap.cc:492] T 414992c8073646798991d2b43fd61f1d P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:42.664261 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:15:42.664871 10801 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:42.672523 10801 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 1.
I20260709 13:15:42.673918 10842 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:15:42.674917 11040 raft_consensus.cc:2804] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:42.675619 11040 raft_consensus.cc:697] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:15:42.676533 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:15:42.676743 11040 consensus_queue.cc:237] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:15:42.678247 10841 leader_election.cc:343] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:42.661433 10992 raft_consensus.cc:740] T 86ba4060a26c42998e9f220a79f28546 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:42.679548 10699 catalog_manager.cc:5697] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 1 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:42.680356 10996 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:42.679715 10992 consensus_queue.cc:260] T 86ba4060a26c42998e9f220a79f28546 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.683372 10992 ts_tablet_manager.cc:1434] T 86ba4060a26c42998e9f220a79f28546 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.002s
I20260709 13:15:42.684649 10992 tablet_bootstrap.cc:492] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:42.664281 10991 ts_tablet_manager.cc:1403] T 414992c8073646798991d2b43fd61f1d P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.150s	user 0.016s	sys 0.025s
I20260709 13:15:42.689419 10991 raft_consensus.cc:359] T 414992c8073646798991d2b43fd61f1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.690189 10991 raft_consensus.cc:385] T 414992c8073646798991d2b43fd61f1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:42.690529 10991 raft_consensus.cc:740] T 414992c8073646798991d2b43fd61f1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:42.661654 10990 raft_consensus.cc:359] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.691931 10990 raft_consensus.cc:385] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:42.692224 10990 raft_consensus.cc:740] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:42.691289 10991 consensus_queue.cc:260] T 414992c8073646798991d2b43fd61f1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.693346 10990 consensus_queue.cc:260] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.697233 10992 tablet_bootstrap.cc:654] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:42.698740 10991 ts_tablet_manager.cc:1434] T 414992c8073646798991d2b43fd61f1d P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.012s	user 0.002s	sys 0.003s
I20260709 13:15:42.702229 10991 tablet_bootstrap.cc:492] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:42.704941 10990 ts_tablet_manager.cc:1434] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.046s	user 0.006s	sys 0.000s
I20260709 13:15:42.706050 10990 tablet_bootstrap.cc:492] T e9764b93d3a8407b8cc1ae641620c831 P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:42.705137 11038 consensus_queue.cc:1048] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:15:42.718191 10990 tablet_bootstrap.cc:654] T e9764b93d3a8407b8cc1ae641620c831 P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:42.734216 10991 tablet_bootstrap.cc:654] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:42.741257 10998 consensus_queue.cc:1048] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:42.748189 10950 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:42.753957 10990 tablet_bootstrap.cc:492] T e9764b93d3a8407b8cc1ae641620c831 P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:42.754460 10990 ts_tablet_manager.cc:1403] T e9764b93d3a8407b8cc1ae641620c831 P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.049s	user 0.012s	sys 0.008s
I20260709 13:15:42.754545 10992 tablet_bootstrap.cc:492] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:42.754997 10992 ts_tablet_manager.cc:1403] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.071s	user 0.015s	sys 0.007s
I20260709 13:15:42.757071 10990 raft_consensus.cc:359] T e9764b93d3a8407b8cc1ae641620c831 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } }
I20260709 13:15:42.757289 10992 raft_consensus.cc:359] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.757802 10990 raft_consensus.cc:385] T e9764b93d3a8407b8cc1ae641620c831 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:42.757838 10992 raft_consensus.cc:385] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:42.758175 10990 raft_consensus.cc:740] T e9764b93d3a8407b8cc1ae641620c831 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:42.758301 10992 raft_consensus.cc:740] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:42.762315 10992 consensus_queue.cc:260] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.763679 11031 consensus_queue.cc:1048] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:42.764873 10992 ts_tablet_manager.cc:1434] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:15:42.765805 10992 tablet_bootstrap.cc:492] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:42.770021 10990 consensus_queue.cc:260] T e9764b93d3a8407b8cc1ae641620c831 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } }
I20260709 13:15:42.764885 10699 catalog_manager.cc:5697] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 0 to 1, leader changed from <none> to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:42.776062 10990 ts_tablet_manager.cc:1434] T e9764b93d3a8407b8cc1ae641620c831 P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
I20260709 13:15:42.770219 11040 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.776532 11040 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.779382 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:42.780057 10800 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 0.
I20260709 13:15:42.781353 10842 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:15:42.783298 10990 tablet_bootstrap.cc:492] T f6e90b642b0d4f06b866170dc0293dc1 P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:42.783967 10996 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:42.784550 10996 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:42.785007 10996 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:42.789664 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:42.790591 10949 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 0.
I20260709 13:15:42.791622 11040 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:42.793888 10949 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:42.796507 10996 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.807468 10996 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:42.811756 11031 consensus_queue.cc:1048] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:42.812793 10990 tablet_bootstrap.cc:654] T f6e90b642b0d4f06b866170dc0293dc1 P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:42.813979 10998 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.818466 10998 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.820608 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:15:42.821237 10800 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:42.821440 10998 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:42.821753 10991 tablet_bootstrap.cc:492] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:42.822247 10991 ts_tablet_manager.cc:1403] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.120s	user 0.022s	sys 0.021s
I20260709 13:15:42.823881 11066 consensus_queue.cc:1048] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:42.826203 10992 tablet_bootstrap.cc:654] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:42.826560 10991 raft_consensus.cc:359] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.827270 10991 raft_consensus.cc:385] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:42.827607 10991 raft_consensus.cc:740] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:42.828465 10991 consensus_queue.cc:260] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.829855 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:15:42.830160 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:42.830474 10950 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:42.831029 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:42.833256 10991 ts_tablet_manager.cc:1434] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 13:15:42.831223 10800 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 1.
I20260709 13:15:42.833639 10916 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:42.834220 10991 tablet_bootstrap.cc:492] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:42.835062 11015 raft_consensus.cc:2749] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:42.834954 10842 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:15:42.836202 10996 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:42.836838 10996 raft_consensus.cc:697] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:15:42.838413 10996 consensus_queue.cc:237] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.853686 11066 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.854550 11066 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.862574 11031 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.863592 10950 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 1.
I20260709 13:15:42.863303 11031 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.869377 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:42.869733 11066 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:42.870107 10875 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 0.
I20260709 13:15:42.870932 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:42.871600 10950 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 0.
I20260709 13:15:42.872270 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:42.872720 10841 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:15:42.873062 10991 tablet_bootstrap.cc:654] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:42.874104 11066 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:42.874501 11066 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:42.874902 11066 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:42.879323 11031 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:42.879194 10767 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:15:42.879215 10842 leader_election.cc:343] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:42.883778 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:42.884421 10950 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 0.
I20260709 13:15:42.885150 11031 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:42.885637 11031 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:42.886057 11031 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:15:42.892261 10747 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:15:42.893484 10990 tablet_bootstrap.cc:492] T f6e90b642b0d4f06b866170dc0293dc1 P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:42.895074 10990 ts_tablet_manager.cc:1403] T f6e90b642b0d4f06b866170dc0293dc1 P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.112s	user 0.007s	sys 0.022s
I20260709 13:15:42.898622 11031 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.899699 10990 raft_consensus.cc:359] T f6e90b642b0d4f06b866170dc0293dc1 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.903728 10990 raft_consensus.cc:385] T f6e90b642b0d4f06b866170dc0293dc1 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:42.903594 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:15:42.904129 10990 raft_consensus.cc:740] T f6e90b642b0d4f06b866170dc0293dc1 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:42.904983 10990 consensus_queue.cc:260] T f6e90b642b0d4f06b866170dc0293dc1 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.906881 10875 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:42.907094 10697 catalog_manager.cc:5697] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 0 to 1, leader changed from <none> to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:42.909118 11066 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.909689 10992 tablet_bootstrap.cc:492] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:42.911127 10990 ts_tablet_manager.cc:1434] T f6e90b642b0d4f06b866170dc0293dc1 P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:15:42.912076 10990 tablet_bootstrap.cc:492] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:42.912400 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:15:42.912556 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:15:42.913424 10992 ts_tablet_manager.cc:1403] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.148s	user 0.007s	sys 0.006s
I20260709 13:15:42.913303 11066 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:42.915028 10948 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:15:42.915550 10842 leader_election.cc:343] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:42.919323 11031 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:42.920825 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:15:42.921458 10950 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:42.921741 10875 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 1.
I20260709 13:15:42.923822 10767 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:15:42.925669 10948 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 1.
I20260709 13:15:42.927124 10841 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:42.928459 10990 tablet_bootstrap.cc:654] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:42.929180 11066 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:42.929927 11066 raft_consensus.cc:697] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:15:42.930878 11066 consensus_queue.cc:237] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.932231 10950 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 1.
I20260709 13:15:42.933162 11031 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:42.934372 11031 raft_consensus.cc:697] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:15:42.935509 11031 consensus_queue.cc:237] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.938596 10991 tablet_bootstrap.cc:492] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:42.939122 10991 ts_tablet_manager.cc:1403] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.105s	user 0.011s	sys 0.006s
I20260709 13:15:42.940438 10992 raft_consensus.cc:359] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.941361 10992 raft_consensus.cc:385] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:42.941722 10992 raft_consensus.cc:740] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:42.945990 10990 tablet_bootstrap.cc:492] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:42.946496 10990 ts_tablet_manager.cc:1403] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.035s	user 0.010s	sys 0.013s
I20260709 13:15:42.949455 10990 raft_consensus.cc:359] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.950165 10990 raft_consensus.cc:385] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:42.950428 10990 raft_consensus.cc:740] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:42.951121 10990 consensus_queue.cc:260] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.951210 10699 catalog_manager.cc:5697] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 0 to 1, leader changed from <none> to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:42.952761 10992 consensus_queue.cc:260] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.953928 10990 ts_tablet_manager.cc:1434] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:15:42.955314 10990 tablet_bootstrap.cc:492] T 121ef9771004451eb563dbfd77657e0e P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:42.955581 10992 ts_tablet_manager.cc:1434] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.042s	user 0.005s	sys 0.002s
I20260709 13:15:42.961107 10990 tablet_bootstrap.cc:654] T 121ef9771004451eb563dbfd77657e0e P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:42.964289 10991 raft_consensus.cc:359] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.965395 10991 raft_consensus.cc:385] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:42.965837 10991 raft_consensus.cc:740] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:42.966905 10991 consensus_queue.cc:260] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.971499 10992 tablet_bootstrap.cc:492] T 9b4c7fc28a9b461abe203fe66da89aca P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:42.976622 10991 ts_tablet_manager.cc:1434] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.001s
I20260709 13:15:42.983955 10991 tablet_bootstrap.cc:492] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
W20260709 13:15:42.988621 10916 consensus_peers.cc:597] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:42.989516 10697 catalog_manager.cc:5697] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 0 to 1, leader changed from <none> to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 1 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:42.990491 10991 tablet_bootstrap.cc:654] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:42.992697 10990 tablet_bootstrap.cc:492] T 121ef9771004451eb563dbfd77657e0e P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:42.993170 10990 ts_tablet_manager.cc:1403] T 121ef9771004451eb563dbfd77657e0e P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.038s	user 0.014s	sys 0.015s
I20260709 13:15:42.995819 10990 raft_consensus.cc:359] T 121ef9771004451eb563dbfd77657e0e P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.996467 10990 raft_consensus.cc:385] T 121ef9771004451eb563dbfd77657e0e P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:42.996737 10990 raft_consensus.cc:740] T 121ef9771004451eb563dbfd77657e0e P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:42.997372 10990 consensus_queue.cc:260] T 121ef9771004451eb563dbfd77657e0e P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:42.999464 10990 ts_tablet_manager.cc:1434] T 121ef9771004451eb563dbfd77657e0e P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:15:43.000767 10990 tablet_bootstrap.cc:492] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:43.009388 10991 tablet_bootstrap.cc:492] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:43.009930 10991 ts_tablet_manager.cc:1403] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.026s	user 0.012s	sys 0.012s
I20260709 13:15:43.010730 10990 tablet_bootstrap.cc:654] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:43.012877 10991 raft_consensus.cc:359] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.013648 10991 raft_consensus.cc:385] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:43.013938 10991 raft_consensus.cc:740] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:43.014674 10991 consensus_queue.cc:260] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.017251 10991 ts_tablet_manager.cc:1434] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:15:43.018297 10991 tablet_bootstrap.cc:492] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:42.980240 10992 tablet_bootstrap.cc:654] T 9b4c7fc28a9b461abe203fe66da89aca P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:43.027946 10992 tablet_bootstrap.cc:492] T 9b4c7fc28a9b461abe203fe66da89aca P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:43.028373 10992 ts_tablet_manager.cc:1403] T 9b4c7fc28a9b461abe203fe66da89aca P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.057s	user 0.013s	sys 0.000s
I20260709 13:15:43.031344 10992 raft_consensus.cc:359] T 9b4c7fc28a9b461abe203fe66da89aca P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.032013 10992 raft_consensus.cc:385] T 9b4c7fc28a9b461abe203fe66da89aca P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:43.032292 10992 raft_consensus.cc:740] T 9b4c7fc28a9b461abe203fe66da89aca P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:43.032876 10990 tablet_bootstrap.cc:492] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:43.033504 10990 ts_tablet_manager.cc:1403] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.033s	user 0.016s	sys 0.004s
I20260709 13:15:43.032907 10992 consensus_queue.cc:260] T 9b4c7fc28a9b461abe203fe66da89aca P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.036284 10990 raft_consensus.cc:359] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.037046 10990 raft_consensus.cc:385] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:43.037338 10990 raft_consensus.cc:740] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:43.038401 10992 ts_tablet_manager.cc:1434] T 9b4c7fc28a9b461abe203fe66da89aca P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.001s
I20260709 13:15:43.038039 10990 consensus_queue.cc:260] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.040418 10990 ts_tablet_manager.cc:1434] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:15:43.041299 10992 tablet_bootstrap.cc:492] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:43.041716 10991 tablet_bootstrap.cc:654] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:43.047092 10875 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:43.047607 10992 tablet_bootstrap.cc:654] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:43.065757 10991 tablet_bootstrap.cc:492] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:43.065979 10990 tablet_bootstrap.cc:492] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:43.066294 10991 ts_tablet_manager.cc:1403] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.048s	user 0.012s	sys 0.017s
I20260709 13:15:43.067447 11015 consensus_queue.cc:1048] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:43.069691 10991 raft_consensus.cc:359] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.070521 10991 raft_consensus.cc:385] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:43.070839 10991 raft_consensus.cc:740] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:43.072594 10990 tablet_bootstrap.cc:654] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:43.074208 11066 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.074746 11066 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.076916 11066 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:43.077633 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:43.078522 10800 raft_consensus.cc:2393] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 1: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:15:43.075976 10991 consensus_queue.cc:260] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.079957 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:43.081468 10841 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:43.083650 10996 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:43.087915 11040 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.089890 10991 ts_tablet_manager.cc:1434] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.023s	user 0.004s	sys 0.004s
I20260709 13:15:43.090108 11040 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.091379 10991 tablet_bootstrap.cc:492] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:43.094281 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:43.095024 10800 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 0.
I20260709 13:15:43.098042 10842 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:15:43.100318 11066 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:43.100858 11066 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:43.101322 11066 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:43.102577 10991 tablet_bootstrap.cc:654] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:43.103124 11015 consensus_queue.cc:1048] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:43.109527 10990 tablet_bootstrap.cc:492] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:43.110167 10990 ts_tablet_manager.cc:1403] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.044s	user 0.017s	sys 0.020s
I20260709 13:15:43.112946 10990 raft_consensus.cc:359] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.113806 10990 raft_consensus.cc:385] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:43.114133 10990 raft_consensus.cc:740] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:43.114874 10990 consensus_queue.cc:260] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.115361 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:43.116000 10950 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 0.
I20260709 13:15:43.117203 10990 ts_tablet_manager.cc:1434] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 13:15:43.117471 11040 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:43.121515 10990 tablet_bootstrap.cc:492] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:43.129599 11066 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.132704 10950 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:43.133244 10990 tablet_bootstrap.cc:654] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:43.140447 10992 tablet_bootstrap.cc:492] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:43.141150 10992 ts_tablet_manager.cc:1403] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.100s	user 0.010s	sys 0.015s
I20260709 13:15:43.156533 10992 raft_consensus.cc:359] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.157512 10992 raft_consensus.cc:385] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:43.157850 10992 raft_consensus.cc:740] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:43.160210 11066 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:43.162046 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:15:43.163034 10948 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:43.163763 10992 consensus_queue.cc:260] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.167191 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:15:43.168052 10800 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:43.175395 11052 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.176177 11052 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.178311 10996 raft_consensus.cc:2749] T 9b4c7fc28a9b461abe203fe66da89aca P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:43.181388 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:15:43.185738 10916 leader_election.cc:343] T f6e90b642b0d4f06b866170dc0293dc1 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:43.183382 10800 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 1.
I20260709 13:15:43.188071 11052 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:43.188846 10948 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 1.
I20260709 13:15:43.189356 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:15:43.191718 10916 leader_election.cc:343] T f6e90b642b0d4f06b866170dc0293dc1 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:43.192226 10916 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:43.192566 10841 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:15:43.193521 10996 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:43.195088 11066 consensus_queue.cc:1048] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:15:43.196694 11015 raft_consensus.cc:2749] T f6e90b642b0d4f06b866170dc0293dc1 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:43.216605 10992 ts_tablet_manager.cc:1434] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.075s	user 0.009s	sys 0.001s
I20260709 13:15:43.217715 10992 tablet_bootstrap.cc:492] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:43.222889 10991 tablet_bootstrap.cc:492] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:43.224009 10991 ts_tablet_manager.cc:1403] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.133s	user 0.047s	sys 0.024s
I20260709 13:15:43.225458 10990 tablet_bootstrap.cc:492] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:43.225607 11038 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:15:43.225976 10990 ts_tablet_manager.cc:1403] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.105s	user 0.008s	sys 0.012s
I20260709 13:15:43.226269 11038 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.232210 10990 raft_consensus.cc:359] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.233172 10990 raft_consensus.cc:385] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:43.233542 10990 raft_consensus.cc:740] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:43.234431 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:43.234336 10990 consensus_queue.cc:260] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.234429 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:43.237018 10990 ts_tablet_manager.cc:1434] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.001s
I20260709 13:15:43.232721 11038 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:43.237924 10842 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:43.238278 10990 tablet_bootstrap.cc:492] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:43.238957 11038 raft_consensus.cc:2749] T 86ba4060a26c42998e9f220a79f28546 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:43.240690 11015 consensus_queue.cc:1048] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:43.252871 11089 consensus_queue.cc:1048] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:43.257072 10996 raft_consensus.cc:697] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:15:43.259913 10996 consensus_queue.cc:237] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.260201 10991 raft_consensus.cc:359] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.261025 10991 raft_consensus.cc:385] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:43.261369 10991 raft_consensus.cc:740] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:43.262127 10991 consensus_queue.cc:260] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.272939 10991 ts_tablet_manager.cc:1434] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.046s	user 0.006s	sys 0.002s
I20260709 13:15:43.273725 10990 tablet_bootstrap.cc:654] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:43.274005 10991 tablet_bootstrap.cc:492] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:43.260996 10992 tablet_bootstrap.cc:654] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:43.287333 10991 tablet_bootstrap.cc:654] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:43.298941 11038 consensus_queue.cc:1048] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:43.315907 10990 tablet_bootstrap.cc:492] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:43.316434 10990 ts_tablet_manager.cc:1403] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.078s	user 0.007s	sys 0.018s
I20260709 13:15:43.318529 10990 raft_consensus.cc:359] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.319247 10990 raft_consensus.cc:385] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:43.319554 10990 raft_consensus.cc:740] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:43.320338 10990 consensus_queue.cc:260] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.322656 10990 ts_tablet_manager.cc:1434] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:15:43.328280 10990 tablet_bootstrap.cc:492] T f15293e178d84fe686fd66bc4439aba8 P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:43.335346 10992 tablet_bootstrap.cc:492] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:43.335752 10992 ts_tablet_manager.cc:1403] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.118s	user 0.025s	sys 0.019s
I20260709 13:15:43.336694 10991 tablet_bootstrap.cc:492] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:43.337239 10991 ts_tablet_manager.cc:1403] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.063s	user 0.013s	sys 0.000s
I20260709 13:15:43.352844 10992 raft_consensus.cc:359] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.353440 10990 tablet_bootstrap.cc:654] T f15293e178d84fe686fd66bc4439aba8 P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:43.353621 10992 raft_consensus.cc:385] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:43.353945 10992 raft_consensus.cc:740] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:43.354609 10992 consensus_queue.cc:260] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.356941 10992 ts_tablet_manager.cc:1434] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 13:15:43.357946 10992 tablet_bootstrap.cc:492] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:43.357671 10699 catalog_manager.cc:5697] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 0 to 1, leader changed from <none> to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:43.354316 10991 raft_consensus.cc:359] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.359349 10991 raft_consensus.cc:385] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:43.359635 10991 raft_consensus.cc:740] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:43.360666 10991 consensus_queue.cc:260] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.363942 10991 ts_tablet_manager.cc:1434] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260709 13:15:43.365298 10991 tablet_bootstrap.cc:492] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:43.369026 10992 tablet_bootstrap.cc:654] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:43.379729 11040 consensus_queue.cc:1048] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:43.399642 11015 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.400190 11015 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.401163 10801 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:43.402290 11015 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:43.403643 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:43.404142 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:15:43.405367 10916 leader_election.cc:343] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:15:43.406481 10916 leader_election.cc:343] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:43.406936 10916 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:43.408720 11015 raft_consensus.cc:2749] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:43.415561 10990 tablet_bootstrap.cc:492] T f15293e178d84fe686fd66bc4439aba8 P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:43.415932 10991 tablet_bootstrap.cc:654] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:43.416076 10990 ts_tablet_manager.cc:1403] T f15293e178d84fe686fd66bc4439aba8 P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.088s	user 0.014s	sys 0.009s
I20260709 13:15:43.416837 11040 consensus_queue.cc:1048] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:15:43.419867 11031 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.420347 11031 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.422699 10990 raft_consensus.cc:359] T f15293e178d84fe686fd66bc4439aba8 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.422957 11028 consensus_queue.cc:1048] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:43.423434 10990 raft_consensus.cc:385] T f15293e178d84fe686fd66bc4439aba8 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:43.423520 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:43.423751 10990 raft_consensus.cc:740] T f15293e178d84fe686fd66bc4439aba8 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:43.424130 10948 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 1.
I20260709 13:15:43.424409 10990 consensus_queue.cc:260] T f15293e178d84fe686fd66bc4439aba8 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.425628 10766 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:15:43.427789 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:43.430012 10991 tablet_bootstrap.cc:492] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:43.430325 11031 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:43.432166 10990 ts_tablet_manager.cc:1434] T f15293e178d84fe686fd66bc4439aba8 P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.001s
I20260709 13:15:43.433200 10990 tablet_bootstrap.cc:492] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:43.430456 10991 ts_tablet_manager.cc:1403] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.065s	user 0.009s	sys 0.002s
I20260709 13:15:43.440204 10991 raft_consensus.cc:359] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.440889 10991 raft_consensus.cc:385] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:43.441155 11031 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.441645 11031 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.443197 10992 tablet_bootstrap.cc:492] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:43.443637 10992 ts_tablet_manager.cc:1403] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.086s	user 0.010s	sys 0.007s
I20260709 13:15:43.443766 11031 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:43.445783 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:43.445783 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:15:43.447614 10766 leader_election.cc:343] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:15:43.448957 10767 leader_election.cc:343] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:43.449998 10992 raft_consensus.cc:359] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.450713 10992 raft_consensus.cc:385] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:43.451021 10992 raft_consensus.cc:740] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:43.451143 10767 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:15:43.452039 11101 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:43.456347 10990 tablet_bootstrap.cc:654] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:43.458568 11028 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:43.441164 10991 raft_consensus.cc:740] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:43.459617 10991 consensus_queue.cc:260] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.460678 11028 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:15:43.461164 11028 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:43.461719 10991 ts_tablet_manager.cc:1434] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.026s	user 0.004s	sys 0.000s
I20260709 13:15:43.462677 10991 tablet_bootstrap.cc:492] T 76b2d93b69274dbca479bd87d7ea45cb P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:43.467084 10992 consensus_queue.cc:260] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.469190 10992 ts_tablet_manager.cc:1434] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.025s	user 0.001s	sys 0.004s
I20260709 13:15:43.469890 11028 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.471712 10991 tablet_bootstrap.cc:654] T 76b2d93b69274dbca479bd87d7ea45cb P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:43.472611 11028 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:43.473714 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:15:43.477504 10990 tablet_bootstrap.cc:492] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:43.477571 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:15:43.478153 10990 ts_tablet_manager.cc:1403] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.045s	user 0.010s	sys 0.001s
I20260709 13:15:43.481108 10990 raft_consensus.cc:359] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.481622 10801 raft_consensus.cc:1240] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:43.481876 10990 raft_consensus.cc:385] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:43.482224 10990 raft_consensus.cc:740] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:43.478511 10948 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:43.483129 10990 consensus_queue.cc:260] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.483429 11040 consensus_queue.cc:1059] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:43.484419 10996 raft_consensus.cc:3055] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:43.484784 10996 raft_consensus.cc:740] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:15:43.485296 10990 ts_tablet_manager.cc:1434] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.006s
I20260709 13:15:43.485265 11028 consensus_queue.cc:1048] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:43.485605 10996 consensus_queue.cc:260] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.491379 10996 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:43.486431 10990 tablet_bootstrap.cc:492] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:43.498134 10948 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 2.
I20260709 13:15:43.499116 10950 raft_consensus.cc:1240] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:43.499673 10766 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:15:43.501425 11028 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:43.501904 11028 raft_consensus.cc:697] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:15:43.500015 10992 tablet_bootstrap.cc:492] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:43.502698 11028 consensus_queue.cc:237] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.507084 10990 tablet_bootstrap.cc:654] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:43.508775 10992 tablet_bootstrap.cc:654] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:43.529188 10996 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.529942 10996 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.534308 10996 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:43.534737 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:43.536238 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:43.536939 10799 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 0.
I20260709 13:15:43.536867 10949 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 0.
I20260709 13:15:43.538810 10697 catalog_manager.cc:5697] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 1 to 2, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 2 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:43.540781 10841 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:15:43.542820 10996 raft_consensus.cc:2804] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:43.543339 10996 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:43.543808 10996 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:43.545047 11015 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.545760 11015 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.548542 11015 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:43.550661 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:43.551580 10998 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.552222 10998 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.556058 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:15:43.556691 10916 leader_election.cc:343] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:15:43.558661 10916 leader_election.cc:343] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:43.559306 10916 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:43.559726 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:43.560370 10998 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:43.561154 11052 raft_consensus.cc:2749] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:43.562147 10991 tablet_bootstrap.cc:492] T 76b2d93b69274dbca479bd87d7ea45cb P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:43.562644 10991 ts_tablet_manager.cc:1403] T 76b2d93b69274dbca479bd87d7ea45cb P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.100s	user 0.006s	sys 0.012s
I20260709 13:15:43.564018 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:43.565364 10991 raft_consensus.cc:359] T 76b2d93b69274dbca479bd87d7ea45cb P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.566128 10991 raft_consensus.cc:385] T 76b2d93b69274dbca479bd87d7ea45cb P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:43.566403 10991 raft_consensus.cc:740] T 76b2d93b69274dbca479bd87d7ea45cb P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:43.569201 10991 consensus_queue.cc:260] T 76b2d93b69274dbca479bd87d7ea45cb P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.571211 10991 ts_tablet_manager.cc:1434] T 76b2d93b69274dbca479bd87d7ea45cb P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
W20260709 13:15:43.571384 10916 leader_election.cc:343] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:15:43.572333 10916 leader_election.cc:343] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:43.572896 10916 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:43.575419 10998 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.575414 10991 tablet_bootstrap.cc:492] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:43.576692 11052 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.577340 11052 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.583370 11052 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:43.584138 11015 raft_consensus.cc:2749] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:43.576081 10998 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.587687 10992 tablet_bootstrap.cc:492] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:43.588244 10992 ts_tablet_manager.cc:1403] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.088s	user 0.021s	sys 0.021s
I20260709 13:15:43.587541 10998 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:43.590485 11028 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.591926 11028 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.593925 11028 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:43.594935 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:43.595346 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:43.595611 10875 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 0.
I20260709 13:15:43.595937 10949 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 0.
I20260709 13:15:43.596843 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:43.597452 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:43.598002 10766 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:15:43.598979 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:43.599722 11031 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:43.600126 11031 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:43.600483 11031 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:43.601425 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:43.601624 10996 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.602052 10801 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 0.
I20260709 13:15:43.598474 11028 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.605598 10990 tablet_bootstrap.cc:492] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:43.606204 10990 ts_tablet_manager.cc:1403] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.120s	user 0.029s	sys 0.038s
I20260709 13:15:43.605860 11028 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:15:43.608186 10916 leader_election.cc:343] T f15293e178d84fe686fd66bc4439aba8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:43.610440 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:15:43.610929 10916 leader_election.cc:343] T f15293e178d84fe686fd66bc4439aba8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:43.611119 10875 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 0.
I20260709 13:15:43.611371 10916 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:43.612136 10998 raft_consensus.cc:2749] T f15293e178d84fe686fd66bc4439aba8 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:43.612591 10916 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:15:43.613406 10998 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:43.613795 10998 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:43.614156 10998 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:43.615728 11028 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:43.616411 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:43.617095 10949 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 0.
I20260709 13:15:43.617758 11031 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.618434 10766 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:15:43.619138 10990 raft_consensus.cc:359] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.619874 10990 raft_consensus.cc:385] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:43.620206 10990 raft_consensus.cc:740] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:43.620954 10990 consensus_queue.cc:260] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.619477 11028 raft_consensus.cc:2804] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:43.621953 11028 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:43.622076 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:15:43.622349 11028 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:43.622701 10875 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:43.627566 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:15:43.628225 10801 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:43.635286 10990 ts_tablet_manager.cc:1434] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.029s	user 0.003s	sys 0.003s
I20260709 13:15:43.639501 10990 tablet_bootstrap.cc:492] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:43.639614 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:15:43.640417 10949 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:43.641789 10998 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.643822 10801 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 1.
I20260709 13:15:43.630573 10875 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 1.
I20260709 13:15:43.645754 10842 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:15:43.646749 11031 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:43.647017 11040 raft_consensus.cc:2804] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:43.647550 11040 raft_consensus.cc:697] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:15:43.648391 11040 consensus_queue.cc:237] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.648438 10949 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 1.
I20260709 13:15:43.653122 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:15:43.653748 10799 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:43.655902 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:15:43.657150 10916 leader_election.cc:343] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:43.659835 10991 tablet_bootstrap.cc:654] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:43.650998 10767 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:15:43.663671 11031 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:43.664227 11031 raft_consensus.cc:697] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:15:43.665092 11031 consensus_queue.cc:237] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.649269 11028 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.672124 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:15:43.672798 10873 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:43.654189 10998 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:15:43.672670 10916 leader_election.cc:343] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:43.673904 10992 raft_consensus.cc:359] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.676301 10992 raft_consensus.cc:385] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:43.676614 10992 raft_consensus.cc:740] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:43.677294 10992 consensus_queue.cc:260] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.680490 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:15:43.681291 10949 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:43.663941 10799 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 1.
I20260709 13:15:43.686916 11028 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:43.688119 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:15:43.688659 10948 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:43.685968 10996 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:43.693254 10916 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:15:43.697045 10998 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:43.697690 10998 raft_consensus.cc:697] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:15:43.698760 10998 consensus_queue.cc:237] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.708011 10949 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 1.
I20260709 13:15:43.696424 10697 catalog_manager.cc:5697] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 0 to 1, leader changed from <none> to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 1 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:43.711508 10696 catalog_manager.cc:5697] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 0 to 1, leader changed from <none> to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:43.715863 10992 ts_tablet_manager.cc:1434] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.127s	user 0.004s	sys 0.004s
I20260709 13:15:43.716872 10992 tablet_bootstrap.cc:492] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:43.725184 10996 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.735230 10996 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.737748 10996 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:43.738315 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:43.739481 10801 raft_consensus.cc:2393] T 76b2d93b69274dbca479bd87d7ea45cb P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 1: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:15:43.739368 10948 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 1.
I20260709 13:15:43.739881 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:43.745049 10842 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:43.747015 11040 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:43.743681 10699 catalog_manager.cc:5697] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 1 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:43.726114 10873 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 1.
I20260709 13:15:43.752797 10766 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:15:43.753705 11031 raft_consensus.cc:2804] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:43.754189 11031 raft_consensus.cc:697] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:15:43.755007 11031 consensus_queue.cc:237] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.758535 10996 consensus_queue.cc:1048] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:43.764703 10990 tablet_bootstrap.cc:654] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:43.768386 10991 tablet_bootstrap.cc:492] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:43.768864 10991 ts_tablet_manager.cc:1403] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.194s	user 0.014s	sys 0.012s
I20260709 13:15:43.771708 10991 raft_consensus.cc:359] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.768158 11040 raft_consensus.cc:2749] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:43.772439 10991 raft_consensus.cc:385] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:43.772743 10991 raft_consensus.cc:740] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:43.773403 10991 consensus_queue.cc:260] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.775959 10991 ts_tablet_manager.cc:1434] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:15:43.776861 10991 tablet_bootstrap.cc:492] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:43.780167 10992 tablet_bootstrap.cc:654] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:43.792939 10990 tablet_bootstrap.cc:492] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:43.793510 10990 ts_tablet_manager.cc:1403] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.154s	user 0.011s	sys 0.005s
I20260709 13:15:43.793937 10991 tablet_bootstrap.cc:654] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:43.797523 10990 raft_consensus.cc:359] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.800187 10990 raft_consensus.cc:385] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:43.800642 10990 raft_consensus.cc:740] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:43.799858 10992 tablet_bootstrap.cc:492] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:43.801494 10992 ts_tablet_manager.cc:1403] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.085s	user 0.012s	sys 0.002s
I20260709 13:15:43.804723 10992 raft_consensus.cc:359] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.805473 10992 raft_consensus.cc:385] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:43.806906 10990 consensus_queue.cc:260] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.817062 10992 raft_consensus.cc:740] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:43.818011 10992 consensus_queue.cc:260] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.822340 10992 ts_tablet_manager.cc:1434] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.000s
I20260709 13:15:43.813946 10699 catalog_manager.cc:5697] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 0 to 1, leader changed from <none> to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 1 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:43.818501 10990 ts_tablet_manager.cc:1434] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.025s	user 0.003s	sys 0.003s
I20260709 13:15:43.828169 10990 tablet_bootstrap.cc:492] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:43.831611 11066 consensus_queue.cc:1048] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:43.839241 10992 tablet_bootstrap.cc:492] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:43.844723 10998 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.845256 10998 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.845788 10990 tablet_bootstrap.cc:654] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:43.846179 10992 tablet_bootstrap.cc:654] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:43.847357 10998 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:43.847802 11031 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.848285 11031 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.849303 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:43.850404 11031 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:43.851047 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:43.852169 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:43.852865 10948 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 0.
W20260709 13:15:43.853884 10916 leader_election.cc:343] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:43.854234 10766 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:15:43.854636 10916 leader_election.cc:343] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:43.855104 10916 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:43.855387 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:15:43.857389 10767 leader_election.cc:343] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:43.858352 11031 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:43.858767 11031 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:43.859159 11031 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:43.864480 10991 tablet_bootstrap.cc:492] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:43.864962 10991 ts_tablet_manager.cc:1403] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.088s	user 0.011s	sys 0.006s
I20260709 13:15:43.858561 10998 raft_consensus.cc:2749] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:43.867790 10991 raft_consensus.cc:359] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.868475 10991 raft_consensus.cc:385] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:43.868733 10991 raft_consensus.cc:740] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:43.884654 11031 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.888309 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:15:43.889062 10949 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:43.889617 11052 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.869390 10991 consensus_queue.cc:260] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.890123 11052 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.888372 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:15:43.892032 10991 ts_tablet_manager.cc:1434] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.001s
I20260709 13:15:43.892176 11052 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:15:43.892589 10767 leader_election.cc:343] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:43.892997 10991 tablet_bootstrap.cc:492] T b253bd741d094067902a02bc765ee0fb P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:43.893574 11031 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:43.897313 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:43.899540 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:15:43.899951 10916 leader_election.cc:343] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:15:43.901979 10916 leader_election.cc:343] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:43.902410 10916 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:43.903210 11052 raft_consensus.cc:2749] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:15:43.910395 10747 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:15:43.925782 10992 tablet_bootstrap.cc:492] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:43.926380 10992 ts_tablet_manager.cc:1403] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.087s	user 0.013s	sys 0.000s
I20260709 13:15:43.930356 10990 tablet_bootstrap.cc:492] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:43.930269 10949 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 1.
I20260709 13:15:43.930961 10990 ts_tablet_manager.cc:1403] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.103s	user 0.019s	sys 0.017s
I20260709 13:15:43.931938 10766 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:15:43.933305 11031 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:43.933810 11031 raft_consensus.cc:697] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:15:43.933969 10991 tablet_bootstrap.cc:654] T b253bd741d094067902a02bc765ee0fb P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:43.934685 11031 consensus_queue.cc:237] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.935698 10992 raft_consensus.cc:359] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.936558 10992 raft_consensus.cc:385] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:43.934988 10990 raft_consensus.cc:359] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.937836 10990 raft_consensus.cc:385] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:43.938283 10990 raft_consensus.cc:740] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:43.937398 10992 raft_consensus.cc:740] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:43.941165 10990 consensus_queue.cc:260] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.945566 11038 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.946082 11038 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.940055 10992 consensus_queue.cc:260] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.948213 11038 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:43.951445 10992 ts_tablet_manager.cc:1434] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 13:15:43.953095 10990 ts_tablet_manager.cc:1434] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.022s	user 0.004s	sys 0.002s
I20260709 13:15:43.951472 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:43.956284 10990 tablet_bootstrap.cc:492] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
W20260709 13:15:43.956539 10842 leader_election.cc:343] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:43.951789 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:43.961690 10949 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 0.
I20260709 13:15:43.965969 10841 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:43.956823 10992 tablet_bootstrap.cc:492] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:43.957472 10991 tablet_bootstrap.cc:492] T b253bd741d094067902a02bc765ee0fb P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:43.967154 11038 raft_consensus.cc:2804] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:43.967667 11038 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:43.967844 10991 ts_tablet_manager.cc:1403] T b253bd741d094067902a02bc765ee0fb P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.075s	user 0.009s	sys 0.020s
I20260709 13:15:43.968056 11038 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:43.970959 10991 raft_consensus.cc:359] T b253bd741d094067902a02bc765ee0fb P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.971755 10991 raft_consensus.cc:385] T b253bd741d094067902a02bc765ee0fb P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:43.972083 10991 raft_consensus.cc:740] T b253bd741d094067902a02bc765ee0fb P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:43.972958 10991 consensus_queue.cc:260] T b253bd741d094067902a02bc765ee0fb P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.975561 10991 ts_tablet_manager.cc:1434] T b253bd741d094067902a02bc765ee0fb P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:15:43.976390 11038 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:43.984576 10991 tablet_bootstrap.cc:492] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:43.985654 10990 tablet_bootstrap.cc:654] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:43.991720 10991 tablet_bootstrap.cc:654] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:43.992198 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:15:43.992828 10799 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:43.994059 10699 catalog_manager.cc:5697] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 0 to 1, leader changed from <none> to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 1 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:43.997020 10992 tablet_bootstrap.cc:654] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.000000 10799 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 1.
I20260709 13:15:44.001487 10842 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:15:44.002305 11066 raft_consensus.cc:2804] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:44.002771 11066 raft_consensus.cc:697] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:15:44.003605 11066 consensus_queue.cc:237] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.007885 10991 tablet_bootstrap.cc:492] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:44.008421 10991 ts_tablet_manager.cc:1403] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.024s	user 0.010s	sys 0.012s
I20260709 13:15:44.011897 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:15:44.012594 10949 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:44.014626 10991 raft_consensus.cc:359] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.015419 10991 raft_consensus.cc:385] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.015707 10991 raft_consensus.cc:740] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.016307 10948 raft_consensus.cc:1275] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:44.016535 10991 consensus_queue.cc:260] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.018083 11031 consensus_queue.cc:1048] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:44.019343 10991 ts_tablet_manager.cc:1434] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.011s	user 0.001s	sys 0.007s
I20260709 13:15:44.020426 10991 tablet_bootstrap.cc:492] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:44.027354 10991 tablet_bootstrap.cc:654] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.036239 10949 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 1.
I20260709 13:15:44.044821 11038 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:44.045359 11101 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.048364 11101 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.049535 10875 raft_consensus.cc:1275] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:44.050612 11101 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:44.053925 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:44.054667 11101 consensus_queue.cc:1048] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:44.055763 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:44.056231 10991 tablet_bootstrap.cc:492] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:44.056867 10991 ts_tablet_manager.cc:1403] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.037s	user 0.006s	sys 0.012s
W20260709 13:15:44.056988 10766 leader_election.cc:343] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:15:44.057534 10767 leader_election.cc:343] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:44.057986 10767 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:15:44.058951 11028 raft_consensus.cc:2749] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:44.060659 10991 raft_consensus.cc:359] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.061621 10991 raft_consensus.cc:385] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.061966 10991 raft_consensus.cc:740] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.062927 10992 tablet_bootstrap.cc:492] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:44.063519 10992 ts_tablet_manager.cc:1403] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.107s	user 0.006s	sys 0.007s
I20260709 13:15:44.065968 10697 catalog_manager.cc:5697] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 0 to 1, leader changed from <none> to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:44.071286 10991 consensus_queue.cc:260] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.074759 10992 raft_consensus.cc:359] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.075770 10992 raft_consensus.cc:385] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.076117 10992 raft_consensus.cc:740] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.077750 10992 consensus_queue.cc:260] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.081960 10992 ts_tablet_manager.cc:1434] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.001s
I20260709 13:15:44.083132 10992 tablet_bootstrap.cc:492] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:44.083470 10990 tablet_bootstrap.cc:492] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:44.083922 10990 ts_tablet_manager.cc:1403] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.128s	user 0.022s	sys 0.005s
I20260709 13:15:44.094290 10992 tablet_bootstrap.cc:654] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.096356 10991 ts_tablet_manager.cc:1434] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.039s	user 0.011s	sys 0.006s
I20260709 13:15:44.097286 10991 tablet_bootstrap.cc:492] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:44.103138 10991 tablet_bootstrap.cc:654] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.093887 10990 raft_consensus.cc:359] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.105345 10990 raft_consensus.cc:385] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.105679 10990 raft_consensus.cc:740] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.106307 10990 consensus_queue.cc:260] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.111996 10990 ts_tablet_manager.cc:1434] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.028s	user 0.004s	sys 0.003s
I20260709 13:15:44.113179 10990 tablet_bootstrap.cc:492] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:44.114643 10992 tablet_bootstrap.cc:492] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:44.115137 10992 ts_tablet_manager.cc:1403] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.032s	user 0.007s	sys 0.004s
I20260709 13:15:44.114929 11015 consensus_queue.cc:1048] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:44.118881 10992 raft_consensus.cc:359] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.119484 10990 tablet_bootstrap.cc:654] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.119642 10992 raft_consensus.cc:385] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.119930 10992 raft_consensus.cc:740] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.120597 10992 consensus_queue.cc:260] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.122637 10992 ts_tablet_manager.cc:1434] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.000s
I20260709 13:15:44.123569 10992 tablet_bootstrap.cc:492] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:44.125089 11038 consensus_queue.cc:1048] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:44.137485 11052 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.138494 11052 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.160131 11052 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:44.162042 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:15:44.169322 10916 leader_election.cc:343] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:44.172477 11066 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.173005 11066 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.175889 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:44.175927 11031 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.172152 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:15:44.177850 10842 leader_election.cc:343] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:44.177981 10799 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 0.
I20260709 13:15:44.178714 10991 tablet_bootstrap.cc:492] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:44.179196 10991 ts_tablet_manager.cc:1403] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.082s	user 0.032s	sys 0.021s
I20260709 13:15:44.176990 11031 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.181109 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:44.179695 10916 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:15:44.182644 11031 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:44.182852 11015 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:44.183362 11015 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:44.183723 11015 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:44.183887 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:44.184504 10949 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 0.
W20260709 13:15:44.182901 10841 leader_election.cc:343] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:44.186424 10992 tablet_bootstrap.cc:654] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.186430 10766 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:15:44.192597 11031 raft_consensus.cc:2804] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:44.192971 11031 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:44.193287 11031 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:44.181617 11066 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:44.194904 10841 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:44.196086 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:44.197163 11066 raft_consensus.cc:2749] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:44.197779 11066 consensus_queue.cc:1048] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:15:44.197952 10767 leader_election.cc:343] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:44.199785 11015 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.202412 11015 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:44.213544 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:15:44.215246 10998 consensus_queue.cc:1048] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:15:44.217788 10916 leader_election.cc:343] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:44.218153 10991 raft_consensus.cc:359] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.218850 10991 raft_consensus.cc:385] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.219127 10991 raft_consensus.cc:740] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.219786 10991 consensus_queue.cc:260] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.219146 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:15:44.222506 10801 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:44.229331 10991 ts_tablet_manager.cc:1434] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.050s	user 0.006s	sys 0.000s
I20260709 13:15:44.230732 10991 tablet_bootstrap.cc:492] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:44.239183 11031 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.248973 10990 tablet_bootstrap.cc:492] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:44.249658 10990 ts_tablet_manager.cc:1403] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.137s	user 0.016s	sys 0.007s
I20260709 13:15:44.251184 11066 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.252673 11066 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.257437 11066 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:44.258522 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:44.259421 10948 raft_consensus.cc:2393] T f15293e178d84fe686fd66bc4439aba8 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 1: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:15:44.262169 10801 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 1.
I20260709 13:15:44.263015 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:44.265117 10842 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:44.266714 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:15:44.267315 10990 raft_consensus.cc:359] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.267270 11038 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:44.268755 10990 raft_consensus.cc:385] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.269070 10990 raft_consensus.cc:740] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.271003 11031 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:15:44.272185 10767 leader_election.cc:343] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:44.272790 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:15:44.273392 10948 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:44.275859 10990 consensus_queue.cc:260] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.274443 10916 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:15:44.279830 11028 consensus_queue.cc:1048] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:44.280894 10948 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 1.
I20260709 13:15:44.282148 11038 raft_consensus.cc:2749] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:44.282928 11015 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:44.284958 10766 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:15:44.285380 11031 consensus_queue.cc:1048] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:44.288043 10990 ts_tablet_manager.cc:1434] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.038s	user 0.008s	sys 0.000s
I20260709 13:15:44.288589 10992 tablet_bootstrap.cc:492] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:44.288895 10990 tablet_bootstrap.cc:492] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:44.288986 10992 ts_tablet_manager.cc:1403] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.166s	user 0.024s	sys 0.014s
I20260709 13:15:44.287793 11015 raft_consensus.cc:697] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:15:44.290140 11028 raft_consensus.cc:2804] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:44.295310 11028 raft_consensus.cc:697] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:15:44.296649 11028 consensus_queue.cc:237] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.295351 11015 consensus_queue.cc:237] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.300014 10992 raft_consensus.cc:359] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.300853 10992 raft_consensus.cc:385] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.301204 10992 raft_consensus.cc:740] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.324448 10990 tablet_bootstrap.cc:654] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.301822 10992 consensus_queue.cc:260] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.301707 11131 consensus_queue.cc:1048] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.030s
I20260709 13:15:44.312804 10991 tablet_bootstrap.cc:654] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.337723 10697 catalog_manager.cc:5697] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 1 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:44.341776 10992 ts_tablet_manager.cc:1434] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.052s	user 0.006s	sys 0.004s
I20260709 13:15:44.341899 11101 consensus_queue.cc:1048] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:44.344341 10992 tablet_bootstrap.cc:492] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:44.376439 11066 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.384904 10990 tablet_bootstrap.cc:492] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:44.385411 10990 ts_tablet_manager.cc:1403] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.097s	user 0.020s	sys 0.015s
I20260709 13:15:44.387943 10696 catalog_manager.cc:5697] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 0 to 1, leader changed from <none> to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 1 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:44.381963 11066 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.401846 11066 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:44.403934 11052 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.404453 11052 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.410508 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:44.410616 10990 raft_consensus.cc:359] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.411232 10800 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 0.
I20260709 13:15:44.411363 10990 raft_consensus.cc:385] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.411805 11052 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:44.411963 10990 raft_consensus.cc:740] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.412791 10990 consensus_queue.cc:260] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.403985 11066 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.414860 11066 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.415594 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:44.416955 11066 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:44.414857 10990 ts_tablet_manager.cc:1434] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.001s
I20260709 13:15:44.417712 10842 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:15:44.418506 10990 tablet_bootstrap.cc:492] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:44.417073 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:44.419399 10996 raft_consensus.cc:2804] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:44.419819 10996 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:44.419592 10949 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 0.
I20260709 13:15:44.408594 10992 tablet_bootstrap.cc:654] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.420271 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:15:44.422834 10916 leader_election.cc:343] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:44.423185 10799 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 0.
I20260709 13:15:44.423277 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:44.420179 10996 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:44.425583 11066 consensus_queue.cc:1048] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:44.427003 10842 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:15:44.430244 11040 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:44.430797 11040 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:44.431223 11040 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:44.433873 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:15:44.436549 10841 leader_election.cc:343] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:44.444216 11040 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:15:44.437810 10916 leader_election.cc:343] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:44.445353 10916 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:44.448467 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:15:44.436985 11015 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.449324 11015 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:15:44.449834 10841 leader_election.cc:343] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:44.450049 11101 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.451258 11015 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:44.451745 11015 raft_consensus.cc:2749] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:44.453471 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:44.454080 10875 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 0.
I20260709 13:15:44.450486 11101 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.456064 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:15:44.456643 10800 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:44.456709 11101 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:44.459292 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:44.459553 10991 tablet_bootstrap.cc:492] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:44.459970 10991 ts_tablet_manager.cc:1403] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.230s	user 0.024s	sys 0.019s
I20260709 13:15:44.461663 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:44.462296 10948 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 0.
I20260709 13:15:44.463533 10766 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:15:44.465152 11101 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:44.465741 11101 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:44.466264 11101 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:15:44.470384 10916 leader_election.cc:343] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:44.471223 10916 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:44.471952 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:44.472196 10991 raft_consensus.cc:359] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.472904 10991 raft_consensus.cc:385] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.473176 10991 raft_consensus.cc:740] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.473882 10991 consensus_queue.cc:260] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.472003 11052 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:44.458595 11040 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:44.475553 11052 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:44.475792 10991 ts_tablet_manager.cc:1434] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.003s
I20260709 13:15:44.476065 11052 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:44.477041 10991 tablet_bootstrap.cc:492] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
W20260709 13:15:44.489831 10767 leader_election.cc:343] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:44.490669 10800 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 1.
I20260709 13:15:44.499253 10842 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:15:44.500042 11040 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:44.500499 11040 raft_consensus.cc:697] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:15:44.500530 11028 consensus_queue.cc:1048] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:44.501416 11040 consensus_queue.cc:237] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.503934 10990 tablet_bootstrap.cc:654] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.511356 10996 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.517719 10996 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:44.517510 11052 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.525734 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:15:44.526392 10949 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:44.528498 11052 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:44.534193 10992 tablet_bootstrap.cc:492] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:44.534737 10992 ts_tablet_manager.cc:1403] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.192s	user 0.008s	sys 0.004s
I20260709 13:15:44.535758 11031 consensus_queue.cc:1048] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:44.538362 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:15:44.539165 10800 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:44.541492 10996 consensus_queue.cc:1048] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:44.541046 10697 catalog_manager.cc:5697] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 0 to 1, leader changed from <none> to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:44.542834 10992 raft_consensus.cc:359] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.543540 10992 raft_consensus.cc:385] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.543810 10992 raft_consensus.cc:740] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.544672 10992 consensus_queue.cc:260] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.547961 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:15:44.548597 10875 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:44.554877 10949 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 1.
I20260709 13:15:44.555958 10875 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 1.
I20260709 13:15:44.556105 10841 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:15:44.557037 10996 raft_consensus.cc:2804] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:44.557265 10916 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:15:44.571669 10996 raft_consensus.cc:697] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:15:44.572613 11052 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:44.572763 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:15:44.573128 11052 raft_consensus.cc:697] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:15:44.573202 10996 consensus_queue.cc:237] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:15:44.574308 10916 leader_election.cc:343] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:44.573959 11052 consensus_queue.cc:237] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.575497 10991 tablet_bootstrap.cc:654] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.577709 10992 ts_tablet_manager.cc:1434] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.043s	user 0.005s	sys 0.001s
I20260709 13:15:44.578702 10992 tablet_bootstrap.cc:492] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:44.599280 11101 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.605325 10990 tablet_bootstrap.cc:492] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:44.606032 10990 ts_tablet_manager.cc:1403] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.188s	user 0.023s	sys 0.005s
I20260709 13:15:44.615360 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:15:44.616008 10949 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:44.617635 10990 raft_consensus.cc:359] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.618320 10990 raft_consensus.cc:385] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.618569 10990 raft_consensus.cc:740] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.620314 10990 consensus_queue.cc:260] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.614228 11101 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:44.625169 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:15:44.625856 10875 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:44.628381 10949 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 1.
I20260709 13:15:44.629557 10766 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:15:44.630396 11031 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:44.630977 11031 raft_consensus.cc:697] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:15:44.631843 11031 consensus_queue.cc:237] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.633788 10875 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 1.
I20260709 13:15:44.641664 10990 ts_tablet_manager.cc:1434] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.001s
I20260709 13:15:44.642455 10990 tablet_bootstrap.cc:492] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:44.642943 10800 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 1.
I20260709 13:15:44.652117 10697 catalog_manager.cc:5697] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 1 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:44.657485 10992 tablet_bootstrap.cc:654] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.656366 10696 catalog_manager.cc:5697] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 0 to 1, leader changed from <none> to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:44.702263 10991 tablet_bootstrap.cc:492] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:44.702761 10991 ts_tablet_manager.cc:1403] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.226s	user 0.019s	sys 0.008s
I20260709 13:15:44.705034 11052 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.705560 11052 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.707705 11052 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:44.709609 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:44.709685 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:44.710379 10801 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 0.
W20260709 13:15:44.711320 10916 leader_election.cc:343] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:44.707078 10699 catalog_manager.cc:5697] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 0 to 1, leader changed from <none> to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 1 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:44.714512 10992 tablet_bootstrap.cc:492] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:44.715013 10992 ts_tablet_manager.cc:1403] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.136s	user 0.009s	sys 0.004s
I20260709 13:15:44.715770 10916 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:15:44.721339 11015 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:44.721748 11015 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:44.722115 11015 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:44.717852 10992 raft_consensus.cc:359] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.726069 10992 raft_consensus.cc:385] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.726444 10992 raft_consensus.cc:740] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.727190 10992 consensus_queue.cc:260] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.729466 10992 ts_tablet_manager.cc:1434] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.001s
I20260709 13:15:44.730396 10992 tablet_bootstrap.cc:492] T 741efe79e97646b396f0091dc10c745e P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:44.718617 10990 tablet_bootstrap.cc:654] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.735487 11015 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.722095 10991 raft_consensus.cc:359] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.738312 10991 raft_consensus.cc:385] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.738583 10991 raft_consensus.cc:740] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.739266 11015 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:44.739324 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:15:44.740000 10800 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:44.739285 10991 consensus_queue.cc:260] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.743225 10991 ts_tablet_manager.cc:1434] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.040s	user 0.005s	sys 0.002s
I20260709 13:15:44.744160 10991 tablet_bootstrap.cc:492] T 5944b8838e584a3890e4e8143f3f2665 P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
W20260709 13:15:44.744261 10916 consensus_peers.cc:597] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:44.745342 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:15:44.747845 10916 leader_election.cc:343] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:44.764880 10800 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 1.
I20260709 13:15:44.766927 10916 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:15:44.768571 10998 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:44.769066 10998 raft_consensus.cc:697] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:15:44.769949 10991 tablet_bootstrap.cc:654] T 5944b8838e584a3890e4e8143f3f2665 P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.769891 10998 consensus_queue.cc:237] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.777935 10990 tablet_bootstrap.cc:492] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:44.778573 10990 ts_tablet_manager.cc:1403] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.136s	user 0.014s	sys 0.005s
I20260709 13:15:44.784463 10990 raft_consensus.cc:359] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.785233 10990 raft_consensus.cc:385] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.785535 10990 raft_consensus.cc:740] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.786616 10992 tablet_bootstrap.cc:654] T 741efe79e97646b396f0091dc10c745e P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.786850 10990 consensus_queue.cc:260] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.789155 10990 ts_tablet_manager.cc:1434] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:15:44.789392 11052 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.790113 10990 tablet_bootstrap.cc:492] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:44.792954 11052 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.795207 11052 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:44.796466 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:15:44.797992 10916 leader_election.cc:343] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:44.801153 10699 catalog_manager.cc:5697] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 1 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:44.802254 10990 tablet_bootstrap.cc:654] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.803124 10991 tablet_bootstrap.cc:492] T 5944b8838e584a3890e4e8143f3f2665 P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:44.803704 10991 ts_tablet_manager.cc:1403] T 5944b8838e584a3890e4e8143f3f2665 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.060s	user 0.010s	sys 0.003s
I20260709 13:15:44.804082 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:15:44.805631 10916 leader_election.cc:343] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:44.806241 10916 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:44.807075 10991 raft_consensus.cc:359] T 5944b8838e584a3890e4e8143f3f2665 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.807636 11052 raft_consensus.cc:2749] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:44.807928 10991 raft_consensus.cc:385] T 5944b8838e584a3890e4e8143f3f2665 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.808373 10991 raft_consensus.cc:740] T 5944b8838e584a3890e4e8143f3f2665 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.809480 10992 tablet_bootstrap.cc:492] T 741efe79e97646b396f0091dc10c745e P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:44.809937 10992 ts_tablet_manager.cc:1403] T 741efe79e97646b396f0091dc10c745e P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.080s	user 0.020s	sys 0.008s
I20260709 13:15:44.809078 10991 consensus_queue.cc:260] T 5944b8838e584a3890e4e8143f3f2665 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.812546 10992 raft_consensus.cc:359] T 741efe79e97646b396f0091dc10c745e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.813189 10992 raft_consensus.cc:385] T 741efe79e97646b396f0091dc10c745e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.813455 10992 raft_consensus.cc:740] T 741efe79e97646b396f0091dc10c745e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.814208 10991 ts_tablet_manager.cc:1434] T 5944b8838e584a3890e4e8143f3f2665 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.001s
I20260709 13:15:44.814103 10992 consensus_queue.cc:260] T 741efe79e97646b396f0091dc10c745e P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.816112 10992 ts_tablet_manager.cc:1434] T 741efe79e97646b396f0091dc10c745e P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:15:44.817556 10991 tablet_bootstrap.cc:492] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:44.822536 10992 tablet_bootstrap.cc:492] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:44.825944 10990 tablet_bootstrap.cc:492] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:44.826467 10991 tablet_bootstrap.cc:654] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.826514 10990 ts_tablet_manager.cc:1403] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.037s	user 0.011s	sys 0.005s
I20260709 13:15:44.828329 10992 tablet_bootstrap.cc:654] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.829627 10990 raft_consensus.cc:359] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.830439 10990 raft_consensus.cc:385] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.830770 10990 raft_consensus.cc:740] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.831516 10990 consensus_queue.cc:260] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.833777 10990 ts_tablet_manager.cc:1434] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:15:44.834733 10990 tablet_bootstrap.cc:492] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:44.841452 10991 tablet_bootstrap.cc:492] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:44.842001 10991 ts_tablet_manager.cc:1403] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.025s	user 0.004s	sys 0.012s
I20260709 13:15:44.844980 10990 tablet_bootstrap.cc:654] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.844715 10991 raft_consensus.cc:359] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.845499 10992 tablet_bootstrap.cc:492] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:44.845674 10991 raft_consensus.cc:385] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.845978 10992 ts_tablet_manager.cc:1403] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.024s	user 0.007s	sys 0.003s
I20260709 13:15:44.845956 10991 raft_consensus.cc:740] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.846957 10991 consensus_queue.cc:260] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.848783 10992 raft_consensus.cc:359] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.849431 10992 raft_consensus.cc:385] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.849690 10992 raft_consensus.cc:740] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.850337 10992 consensus_queue.cc:260] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.852681 10991 ts_tablet_manager.cc:1434] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:15:44.853719 10991 tablet_bootstrap.cc:492] T b81df256f0c644cfae9fe2d336cbe9bd P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:44.859109 10992 ts_tablet_manager.cc:1434] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.004s
I20260709 13:15:44.860128 10992 tablet_bootstrap.cc:492] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:44.868019 10992 tablet_bootstrap.cc:654] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.872646 11031 consensus_queue.cc:1048] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:44.882553 10873 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:44.884980 10990 tablet_bootstrap.cc:492] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:44.885504 10990 ts_tablet_manager.cc:1403] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.051s	user 0.007s	sys 0.009s
I20260709 13:15:44.885540 11052 consensus_queue.cc:1048] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:44.886098 10992 tablet_bootstrap.cc:492] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:44.888434 10990 raft_consensus.cc:359] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.889240 10990 raft_consensus.cc:385] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.889519 10990 raft_consensus.cc:740] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.890159 10990 consensus_queue.cc:260] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.886583 10992 ts_tablet_manager.cc:1403] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.027s	user 0.010s	sys 0.012s
I20260709 13:15:44.892447 10990 ts_tablet_manager.cc:1434] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:15:44.893508 10990 tablet_bootstrap.cc:492] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:44.893713 10992 raft_consensus.cc:359] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.894356 10992 raft_consensus.cc:385] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.894623 10992 raft_consensus.cc:740] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.895272 10992 consensus_queue.cc:260] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.897322 10992 ts_tablet_manager.cc:1434] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:15:44.901034 10991 tablet_bootstrap.cc:654] T b81df256f0c644cfae9fe2d336cbe9bd P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.902837 11031 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.903340 11031 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.905441 11031 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:44.907528 10992 tablet_bootstrap.cc:492] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:44.907317 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:44.908046 10876 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 0.
I20260709 13:15:44.909332 10767 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:15:44.910136 11031 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:44.910476 11031 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:44.910831 11031 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:44.912791 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:44.913457 10950 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 0.
I20260709 13:15:44.921164 10996 consensus_queue.cc:1048] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:44.922180 10992 tablet_bootstrap.cc:654] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.958205 10990 tablet_bootstrap.cc:654] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.965390 11031 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:44.969139 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:15:44.969776 10950 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:44.975153 11031 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:44.984311 11031 consensus_queue.cc:1048] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:44.989135 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:15:44.991865 10876 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:44.992743 10996 consensus_queue.cc:1048] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:44.988834 11031 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.986721 10950 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 1.
I20260709 13:15:45.001588 10766 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:15:44.998965 11031 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:45.010637 11015 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.016549 10799 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.016721 11015 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:45.015663 11131 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.005652 10992 tablet_bootstrap.cc:492] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:45.019315 10992 ts_tablet_manager.cc:1403] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.112s	user 0.025s	sys 0.022s
I20260709 13:15:45.006083 11101 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:45.032248 10992 raft_consensus.cc:359] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:45.032910 10992 raft_consensus.cc:385] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:45.033144 10991 tablet_bootstrap.cc:492] T b81df256f0c644cfae9fe2d336cbe9bd P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:45.033572 10991 ts_tablet_manager.cc:1403] T b81df256f0c644cfae9fe2d336cbe9bd P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.180s	user 0.016s	sys 0.039s
I20260709 13:15:45.020063 11131 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:45.033171 10992 raft_consensus.cc:740] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:45.031751 11031 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:45.026535 11101 raft_consensus.cc:697] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:15:45.040510 10992 consensus_queue.cc:260] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:45.048951 10992 ts_tablet_manager.cc:1434] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.029s	user 0.004s	sys 0.005s
I20260709 13:15:45.055960 10876 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 1.
I20260709 13:15:45.057866 10991 raft_consensus.cc:359] T b81df256f0c644cfae9fe2d336cbe9bd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:45.058636 10991 raft_consensus.cc:385] T b81df256f0c644cfae9fe2d336cbe9bd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:45.059522 10991 raft_consensus.cc:740] T b81df256f0c644cfae9fe2d336cbe9bd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:45.060648 10992 tablet_bootstrap.cc:492] T 0a574a3bcb5441568f4fd9b1c38d719a P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:45.062261 10998 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.062117 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:45.062913 10876 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 0.
I20260709 13:15:45.062770 10998 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:45.060240 10991 consensus_queue.cc:260] T b81df256f0c644cfae9fe2d336cbe9bd P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:45.064924 10991 ts_tablet_manager.cc:1434] T b81df256f0c644cfae9fe2d336cbe9bd P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.002s
I20260709 13:15:45.065194 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:45.065819 10991 tablet_bootstrap.cc:492] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:45.065747 10998 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:45.042544 10950 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.034937 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:45.067296 10916 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:15:45.072003 10998 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:45.067746 11101 consensus_queue.cc:237] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:45.072427 10998 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:45.072837 10998 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.068503 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:15:45.076046 10916 leader_election.cc:343] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:45.059988 11015 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:45.076172 11131 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:45.089440 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:15:45.096421 10916 leader_election.cc:343] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:45.090667 10876 raft_consensus.cc:2393] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 1: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:15:45.099517 11052 consensus_queue.cc:1048] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:45.113533 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:45.125244 10992 tablet_bootstrap.cc:654] T 0a574a3bcb5441568f4fd9b1c38d719a P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
W20260709 13:15:45.125900 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 56151.3us
I20260709 13:15:45.120133 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:45.135058 10991 tablet_bootstrap.cc:654] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:45.139650 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:45.140348 10947 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 0.
I20260709 13:15:45.121748 10767 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:15:45.151023 10747 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:15:45.152762 10767 leader_election.cc:343] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:15:45.160365 10916 leader_election.cc:343] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:45.161164 10916 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:45.163633 10766 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:15:45.168612 10873 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.169283 10990 tablet_bootstrap.cc:492] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:45.169739 10990 ts_tablet_manager.cc:1403] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.276s	user 0.010s	sys 0.004s
I20260709 13:15:45.172942 10998 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:45.175062 11028 raft_consensus.cc:2749] T 5944b8838e584a3890e4e8143f3f2665 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:45.175129 10998 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:45.167754 11101 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:45.191059 11101 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:45.191514 11101 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.192539 10996 consensus_queue.cc:1048] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:45.198333 10990 raft_consensus.cc:359] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:45.199200 10990 raft_consensus.cc:385] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:45.199504 10990 raft_consensus.cc:740] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:45.200122 10990 consensus_queue.cc:260] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:15:45.184098 11180 log.cc:927] Time spent T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:15:45.209548 11052 consensus_queue.cc:1048] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:15:45.203146 10998 raft_consensus.cc:2749] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:45.213583 10990 ts_tablet_manager.cc:1434] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.044s	user 0.004s	sys 0.004s
I20260709 13:15:45.213696 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:15:45.214298 10876 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.214677 10990 tablet_bootstrap.cc:492] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:45.221280 10991 tablet_bootstrap.cc:492] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:45.221782 10991 ts_tablet_manager.cc:1403] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.156s	user 0.026s	sys 0.048s
I20260709 13:15:45.219183 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:15:45.240192 10916 leader_election.cc:343] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:45.238221 11028 consensus_queue.cc:1048] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:15:45.255409 11028 consensus_queue.cc:1048] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:45.286377 10699 catalog_manager.cc:5697] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 0 to 1, leader changed from <none> to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 1 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:45.355639 10992 tablet_bootstrap.cc:492] T 0a574a3bcb5441568f4fd9b1c38d719a P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:45.310731 10991 raft_consensus.cc:359] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:45.358242 10992 ts_tablet_manager.cc:1403] T 0a574a3bcb5441568f4fd9b1c38d719a P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.298s	user 0.011s	sys 0.045s
I20260709 13:15:45.358569 10991 raft_consensus.cc:385] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:45.359126 10991 raft_consensus.cc:740] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:45.362084 10991 consensus_queue.cc:260] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:45.368615 10990 tablet_bootstrap.cc:654] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:45.371810 10992 raft_consensus.cc:359] T 0a574a3bcb5441568f4fd9b1c38d719a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:45.372705 10992 raft_consensus.cc:385] T 0a574a3bcb5441568f4fd9b1c38d719a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:45.373014 10992 raft_consensus.cc:740] T 0a574a3bcb5441568f4fd9b1c38d719a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:45.373615 10992 consensus_queue.cc:260] T 0a574a3bcb5441568f4fd9b1c38d719a P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:45.316452 10876 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 1.
I20260709 13:15:45.319330 11101 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:45.382481 10991 ts_tablet_manager.cc:1434] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.160s	user 0.009s	sys 0.002s
I20260709 13:15:45.384068 10991 tablet_bootstrap.cc:492] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:45.387642 10992 ts_tablet_manager.cc:1434] T 0a574a3bcb5441568f4fd9b1c38d719a P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.000s
I20260709 13:15:45.388757 10992 tablet_bootstrap.cc:492] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:45.423360 11188 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.431550 11141 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.432077 11141 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:45.429723 11188 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:45.435416 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:15:45.435990 10948 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.424873 11101 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:15:45.485116 11189 log.cc:927] Time spent T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:15:45.497632 11031 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.498349 11031 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:45.505240 10948 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 1.
I20260709 13:15:45.535697 10991 tablet_bootstrap.cc:654] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:45.554981 11031 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:45.595567 11031 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:45.595815 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 153564us
I20260709 13:15:45.596100 11031 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:45.597422 11040 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.597998 11031 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:45.597878 11040 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:45.600221 11040 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:45.600582 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:45.601266 10799 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 0.
I20260709 13:15:45.600589 11153 consensus_queue.cc:1048] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:15:45.604143 10990 tablet_bootstrap.cc:492] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:45.604604 10990 ts_tablet_manager.cc:1403] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.390s	user 0.030s	sys 0.027s
W20260709 13:15:45.605340 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 48695us
I20260709 13:15:45.608157 10992 tablet_bootstrap.cc:654] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:45.555584 11188 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:45.648722 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:45.651517 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:45.654237 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:15:45.655078 10875 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.604012 11052 consensus_queue.cc:1048] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.037s
W20260709 13:15:45.671036 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 64431.4us
I20260709 13:15:45.671921 10916 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:15:45.663898 11192 log.cc:927] Time spent T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:15:45.661837 11141 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:45.682576 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:45.693239 10990 raft_consensus.cc:359] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:45.694293 10990 raft_consensus.cc:385] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:45.694664 10990 raft_consensus.cc:740] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:45.696316 10766 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:15:45.695569 10990 consensus_queue.cc:260] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:45.684762 11141 raft_consensus.cc:2804] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:45.714282 11141 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:45.712258 10948 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 0.
W20260709 13:15:45.740823 10767 leader_election.cc:343] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:45.747670 10916 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:45.748625 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:45.749632 10990 ts_tablet_manager.cc:1434] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.145s	user 0.008s	sys 0.004s
I20260709 13:15:45.752218 10990 tablet_bootstrap.cc:492] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:45.735781 11141 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.759438 10992 tablet_bootstrap.cc:492] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:45.761605 10992 ts_tablet_manager.cc:1403] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.373s	user 0.014s	sys 0.021s
I20260709 13:15:45.737267 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:15:45.764344 10916 leader_election.cc:343] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:45.750563 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:15:45.766333 10916 consensus_peers.cc:597] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:45.768440 11028 consensus_queue.cc:1048] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:45.771409 10991 tablet_bootstrap.cc:492] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:45.772058 10991 ts_tablet_manager.cc:1403] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.388s	user 0.036s	sys 0.025s
I20260709 13:15:45.760720 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:45.754673 10875 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 1.
I20260709 13:15:45.763259 11188 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:45.773990 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:45.791381 11178 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:45.791975 11178 raft_consensus.cc:697] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:15:45.792841 11178 consensus_queue.cc:237] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:45.812366 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:15:45.823589 10766 leader_election.cc:343] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:45.832703 11141 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:45.834498 11141 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:45.826844 11188 raft_consensus.cc:697] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:15:45.838562 10842 leader_election.cc:343] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:45.838420 10949 raft_consensus.cc:2393] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 1: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
W20260709 13:15:45.841727 11211 log.cc:927] Time spent T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:15:45.842458 10992 raft_consensus.cc:359] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:45.843488 10992 raft_consensus.cc:385] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:45.843850 10992 raft_consensus.cc:740] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:45.841885 10990 tablet_bootstrap.cc:654] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
W20260709 13:15:45.825404 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 56337.8us
I20260709 13:15:45.846971 10991 raft_consensus.cc:359] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:45.837456 10766 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:15:45.847673 10991 raft_consensus.cc:385] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:45.847976 10991 raft_consensus.cc:740] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:45.848196 11031 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:45.848654 10991 consensus_queue.cc:260] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:45.850746 10991 ts_tablet_manager.cc:1434] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.078s	user 0.004s	sys 0.000s
I20260709 13:15:45.851830 10991 tablet_bootstrap.cc:492] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:45.852991 11031 consensus_queue.cc:1048] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:45.858690 11188 consensus_queue.cc:237] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:45.870159 10992 consensus_queue.cc:260] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:45.873044 10841 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:45.878435 11038 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:45.879526 11038 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:45.881843 11038 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:15:45.884135 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 72784.3us
I20260709 13:15:45.892019 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:15:45.886536 10916 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:45.899284 10992 ts_tablet_manager.cc:1434] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.137s	user 0.009s	sys 0.000s
W20260709 13:15:45.901875 10916 leader_election.cc:343] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:45.903242 10992 tablet_bootstrap.cc:492] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
W20260709 13:15:45.880470 10767 leader_election.cc:343] T b81df256f0c644cfae9fe2d336cbe9bd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:45.907450 10767 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:15:45.906505 10998 raft_consensus.cc:2749] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:45.910630 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:15:45.925186 10798 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.991693 10990 tablet_bootstrap.cc:492] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:46.000255 10990 ts_tablet_manager.cc:1403] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.250s	user 0.015s	sys 0.012s
I20260709 13:15:45.997383 11038 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.002696 11038 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:46.002924 10990 raft_consensus.cc:359] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.003645 10990 raft_consensus.cc:385] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.003942 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:15:46.007092 10947 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:46.003958 10990 raft_consensus.cc:740] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:46.012239 10990 consensus_queue.cc:260] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.014832 10990 ts_tablet_manager.cc:1434] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.014s	user 0.002s	sys 0.005s
I20260709 13:15:46.015784 10990 tablet_bootstrap.cc:492] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:46.018339 10991 tablet_bootstrap.cc:654] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
W20260709 13:15:45.989209 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 109822us
I20260709 13:15:46.022167 10992 tablet_bootstrap.cc:654] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:46.023715 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:15:46.026058 10842 leader_election.cc:343] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:46.023341 10947 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 1.
I20260709 13:15:46.028302 10841 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:46.029119 10996 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:46.029546 10996 raft_consensus.cc:697] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:15:46.030259 10996 consensus_queue.cc:237] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.035588 10798 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 1.
I20260709 13:15:46.041642 11201 raft_consensus.cc:2749] T b81df256f0c644cfae9fe2d336cbe9bd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:46.048056 10916 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:15:46.049145 11188 raft_consensus.cc:2804] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:46.049631 11188 raft_consensus.cc:697] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:15:46.050571 11188 consensus_queue.cc:237] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.074216 10699 catalog_manager.cc:5697] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 0 to 1, leader changed from <none> to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:46.079073 10990 tablet_bootstrap.cc:654] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:46.096057 10991 tablet_bootstrap.cc:492] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:46.096565 10991 ts_tablet_manager.cc:1403] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.245s	user 0.038s	sys 0.022s
I20260709 13:15:46.097638 10992 tablet_bootstrap.cc:492] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:46.098093 10992 ts_tablet_manager.cc:1403] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.197s	user 0.037s	sys 0.065s
I20260709 13:15:46.101102 10992 raft_consensus.cc:359] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.102052 10992 raft_consensus.cc:385] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.102380 10992 raft_consensus.cc:740] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:46.099171 10991 raft_consensus.cc:359] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.103003 10991 raft_consensus.cc:385] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.103273 10991 raft_consensus.cc:740] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:46.103931 10991 consensus_queue.cc:260] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.104118 10992 consensus_queue.cc:260] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.107657 10992 ts_tablet_manager.cc:1434] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:15:46.108145 10991 ts_tablet_manager.cc:1434] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:15:46.108564 10992 tablet_bootstrap.cc:492] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:46.109018 10991 tablet_bootstrap.cc:492] T 37a1fc8ca1b34260908d185a9ebc2bb3 P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:46.115839 10992 tablet_bootstrap.cc:654] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:46.117344 10991 tablet_bootstrap.cc:654] T 37a1fc8ca1b34260908d185a9ebc2bb3 P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:46.120801 10990 tablet_bootstrap.cc:492] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:46.121349 10990 ts_tablet_manager.cc:1403] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.106s	user 0.017s	sys 0.011s
I20260709 13:15:46.121806 10992 tablet_bootstrap.cc:492] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:46.122849 10992 ts_tablet_manager.cc:1403] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.014s	user 0.012s	sys 0.001s
I20260709 13:15:46.125240 10990 raft_consensus.cc:359] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.125463 10992 raft_consensus.cc:359] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.126010 10990 raft_consensus.cc:385] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.126171 10992 raft_consensus.cc:385] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.126538 10992 raft_consensus.cc:740] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:46.126577 10990 raft_consensus.cc:740] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:46.127909 10990 consensus_queue.cc:260] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.130077 10992 consensus_queue.cc:260] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.130831 10990 ts_tablet_manager.cc:1434] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:15:46.131896 10990 tablet_bootstrap.cc:492] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:46.134524 10992 ts_tablet_manager.cc:1434] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:15:46.129848 10697 catalog_manager.cc:5697] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 1 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:46.137867 10992 tablet_bootstrap.cc:492] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:46.138257 10990 tablet_bootstrap.cc:654] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:46.144692 10992 tablet_bootstrap.cc:654] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:46.173568 10698 catalog_manager.cc:5697] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 0 to 1, leader changed from <none> to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 1 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:46.176076 11213 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:46.176810 11213 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.193457 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:15:46.195569 10767 leader_election.cc:343] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:46.202622 11213 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:46.205217 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:46.206194 10942 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 0.
I20260709 13:15:46.207826 10766 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:15:46.209543 11213 raft_consensus.cc:2804] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:46.210000 11213 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:46.210507 11213 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:46.214433 10992 tablet_bootstrap.cc:492] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:46.214949 10992 ts_tablet_manager.cc:1403] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.080s	user 0.038s	sys 0.035s
I20260709 13:15:46.215998 10990 tablet_bootstrap.cc:492] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:46.216558 10990 ts_tablet_manager.cc:1403] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.085s	user 0.049s	sys 0.017s
I20260709 13:15:46.217624 10992 raft_consensus.cc:359] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.218415 10992 raft_consensus.cc:385] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.218691 10992 raft_consensus.cc:740] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:46.219422 10992 consensus_queue.cc:260] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.219597 10990 raft_consensus.cc:359] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.224086 10990 raft_consensus.cc:385] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.224459 10990 raft_consensus.cc:740] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:46.225878 11188 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:46.222805 11213 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.226171 10990 consensus_queue.cc:260] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.226397 11188 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.221519 10992 ts_tablet_manager.cc:1434] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:15:46.228027 10992 tablet_bootstrap.cc:492] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:46.228634 11188 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:46.229614 10991 tablet_bootstrap.cc:492] T 37a1fc8ca1b34260908d185a9ebc2bb3 P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:46.230068 10991 ts_tablet_manager.cc:1403] T 37a1fc8ca1b34260908d185a9ebc2bb3 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.121s	user 0.010s	sys 0.004s
I20260709 13:15:46.232276 10990 ts_tablet_manager.cc:1434] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:15:46.232820 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:46.232757 10991 raft_consensus.cc:359] T 37a1fc8ca1b34260908d185a9ebc2bb3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.233237 10990 tablet_bootstrap.cc:492] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:46.233510 10991 raft_consensus.cc:385] T 37a1fc8ca1b34260908d185a9ebc2bb3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.233798 10992 tablet_bootstrap.cc:654] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:46.233852 10991 raft_consensus.cc:740] T 37a1fc8ca1b34260908d185a9ebc2bb3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:46.237196 10991 consensus_queue.cc:260] T 37a1fc8ca1b34260908d185a9ebc2bb3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.240273 10990 tablet_bootstrap.cc:654] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:46.240361 10991 ts_tablet_manager.cc:1434] T 37a1fc8ca1b34260908d185a9ebc2bb3 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.002s
W20260709 13:15:46.240902 10916 leader_election.cc:343] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:46.244928 10992 tablet_bootstrap.cc:492] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:46.245417 10992 ts_tablet_manager.cc:1403] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.018s	user 0.005s	sys 0.009s
I20260709 13:15:46.247838 10992 raft_consensus.cc:359] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.248502 10992 raft_consensus.cc:385] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.248800 10992 raft_consensus.cc:740] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:46.249416 10992 consensus_queue.cc:260] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.251917 10992 ts_tablet_manager.cc:1434] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260709 13:15:46.232071 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:46.252818 10992 tablet_bootstrap.cc:492] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:46.252991 11200 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:46.253489 11200 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.256592 11200 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:46.257896 10990 tablet_bootstrap.cc:492] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:46.258015 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:46.259871 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:46.258713 10992 tablet_bootstrap.cc:654] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:46.260566 10990 ts_tablet_manager.cc:1403] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.028s	user 0.014s	sys 0.004s
I20260709 13:15:46.261569 10841 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:46.262382 11200 raft_consensus.cc:2749] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:46.263739 10990 raft_consensus.cc:359] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.259189 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:15:46.264401 10990 raft_consensus.cc:385] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.264681 10990 raft_consensus.cc:740] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:46.264768 10872 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:46.265460 10991 tablet_bootstrap.cc:492] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:46.265357 10990 consensus_queue.cc:260] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.272167 10990 ts_tablet_manager.cc:1434] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
W20260709 13:15:46.273423 10916 leader_election.cc:343] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:46.273836 10916 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:46.274418 11188 raft_consensus.cc:2749] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:15:46.281340 10747 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:15:46.282538 10991 tablet_bootstrap.cc:654] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:46.282516 10998 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:46.283392 10998 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.285418 10872 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 1.
I20260709 13:15:46.286295 10998 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:46.287128 10996 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:46.287761 10996 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.289100 10990 tablet_bootstrap.cc:492] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:46.286909 10767 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:15:46.290980 11031 raft_consensus.cc:2804] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:46.291543 11031 raft_consensus.cc:697] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:15:46.292484 11031 consensus_queue.cc:237] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.294456 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:46.296211 10996 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:46.299775 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:46.300755 10990 tablet_bootstrap.cc:654] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
W20260709 13:15:46.302230 10916 leader_election.cc:343] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:46.310380 10697 catalog_manager.cc:5697] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 1 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:46.312801 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:46.313520 10798 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 0.
I20260709 13:15:46.314284 11015 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:46.307226 10942 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 0.
I20260709 13:15:46.314805 11015 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.317157 11031 consensus_queue.cc:1048] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:46.318324 10872 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:46.318917 10998 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:46.325076 10699 catalog_manager.cc:5697] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 0 to 1, leader changed from <none> to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 1 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:46.327724 10842 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:15:46.335798 10996 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:46.336154 10996 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:46.336232 10991 tablet_bootstrap.cc:492] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:46.336508 10996 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:46.336854 10991 ts_tablet_manager.cc:1403] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.072s	user 0.018s	sys 0.007s
I20260709 13:15:46.336953 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:46.337697 10798 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 0.
I20260709 13:15:46.341327 11031 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:46.341924 11031 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.346771 10916 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:15:46.348268 11141 raft_consensus.cc:2804] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:46.348758 11141 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:46.349130 11141 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:46.349424 10996 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.354935 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:15:46.355615 10942 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:46.334765 11213 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:46.339123 10998 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.361498 10998 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:46.362076 10942 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 1.
I20260709 13:15:46.364363 10991 raft_consensus.cc:359] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.367754 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:46.368775 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:46.370399 11205 consensus_queue.cc:1048] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:46.371326 11015 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:46.373109 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:46.374185 11141 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.375937 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:46.378468 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:46.379136 11031 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:46.383880 10991 raft_consensus.cc:385] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.383901 10996 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:46.385236 10873 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 0.
I20260709 13:15:46.391137 10991 raft_consensus.cc:740] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:46.387353 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:15:46.392671 10947 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:46.395290 10991 consensus_queue.cc:260] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.398358 10991 ts_tablet_manager.cc:1434] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.061s	user 0.005s	sys 0.003s
I20260709 13:15:46.400157 10948 raft_consensus.cc:2393] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 1: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:15:46.406251 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:46.407274 10800 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 0.
I20260709 13:15:46.409572 10767 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:15:46.413069 11201 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:46.414095 11031 consensus_queue.cc:1048] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:46.420410 10991 tablet_bootstrap.cc:492] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
W20260709 13:15:46.417389 10916 leader_election.cc:343] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:46.423275 10916 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:15:46.431348 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:15:46.432238 10800 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:46.429113 11141 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:46.425722 11015 raft_consensus.cc:2804] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:46.435956 11015 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:46.436432 11015 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:46.438668 10947 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 1.
W20260709 13:15:46.442005 10916 consensus_peers.cc:597] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:46.442585 10916 leader_election.cc:343] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:46.443347 10916 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:15:46.444902 10998 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:46.445291 10998 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:46.445199 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:15:46.445713 10998 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:46.449455 10800 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 1.
I20260709 13:15:46.443704 10841 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:15:46.451177 10996 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:46.452543 10996 raft_consensus.cc:697] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:15:46.453400 10996 consensus_queue.cc:237] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:15:46.455767 10916 leader_election.cc:343] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:46.457296 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:15:46.457571 10992 tablet_bootstrap.cc:492] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:46.459214 10992 ts_tablet_manager.cc:1403] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.207s	user 0.035s	sys 0.037s
I20260709 13:15:46.461556 10800 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:46.469226 11015 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.471060 10992 raft_consensus.cc:359] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.471773 10992 raft_consensus.cc:385] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.472060 10992 raft_consensus.cc:740] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:46.472723 10992 consensus_queue.cc:260] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.474761 10992 ts_tablet_manager.cc:1434] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:15:46.475221 10990 tablet_bootstrap.cc:492] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:46.475677 10990 ts_tablet_manager.cc:1403] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.187s	user 0.032s	sys 0.045s
I20260709 13:15:46.479074 10998 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.480845 11201 raft_consensus.cc:2749] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:46.483529 11015 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:46.486690 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:15:46.487190 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:15:46.487385 11200 consensus_queue.cc:1048] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:46.487416 10875 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:46.482322 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:15:46.494987 10800 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 1.
I20260709 13:15:46.504119 10991 tablet_bootstrap.cc:654] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:46.505913 11040 consensus_queue.cc:1048] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:46.507303 10875 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 1.
I20260709 13:15:46.481266 10998 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:46.482301 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:15:46.482366 10990 raft_consensus.cc:359] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.510133 10797 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:46.510434 10990 raft_consensus.cc:385] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.510710 10990 raft_consensus.cc:740] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:46.511449 10990 consensus_queue.cc:260] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.513566 10990 ts_tablet_manager.cc:1434] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.038s	user 0.000s	sys 0.004s
W20260709 13:15:46.513672 10916 leader_election.cc:343] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:46.514494 10990 tablet_bootstrap.cc:492] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:46.514367 10916 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:15:46.515218 10916 leader_election.cc:343] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:46.515601 11198 raft_consensus.cc:2804] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:46.516016 10916 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:15:46.516146 11198 raft_consensus.cc:697] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:15:46.516719 11141 raft_consensus.cc:2804] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:46.517407 11141 raft_consensus.cc:697] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:15:46.517083 11198 consensus_queue.cc:237] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.522001 10797 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 1.
I20260709 13:15:46.518433 11141 consensus_queue.cc:237] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.475695 10992 tablet_bootstrap.cc:492] T b4b13050d714473bb01ea0174d634a14 P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:46.530444 10916 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:15:46.531942 11015 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:46.532433 11015 raft_consensus.cc:697] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:15:46.535223 11015 consensus_queue.cc:237] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.541320 10991 tablet_bootstrap.cc:492] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:46.541863 10991 ts_tablet_manager.cc:1403] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.122s	user 0.008s	sys 0.003s
I20260709 13:15:46.544644 10991 raft_consensus.cc:359] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.545288 10991 raft_consensus.cc:385] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.545543 10991 raft_consensus.cc:740] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:46.546517 10699 catalog_manager.cc:5697] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 0 to 1, leader changed from <none> to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
W20260709 13:15:46.549059 10916 consensus_peers.cc:597] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:46.558874 11040 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:46.559427 11040 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.560827 10992 tablet_bootstrap.cc:654] T b4b13050d714473bb01ea0174d634a14 P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:46.562718 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:46.563520 10991 consensus_queue.cc:260] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.565306 11040 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:46.567101 10698 catalog_manager.cc:5697] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 1 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:46.568674 10991 ts_tablet_manager.cc:1434] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.002s
I20260709 13:15:46.569756 10991 tablet_bootstrap.cc:492] T 51ae8de979e44a2d8c155bffc20a1533 P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:46.565521 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:46.571547 10798 raft_consensus.cc:2393] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 1: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:15:46.572682 10842 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:46.573657 11040 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:46.554685 10990 tablet_bootstrap.cc:654] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:46.584589 11040 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:46.569793 10698 catalog_manager.cc:5697] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 1 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:46.595027 10991 tablet_bootstrap.cc:654] T 51ae8de979e44a2d8c155bffc20a1533 P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:46.604357 11015 consensus_queue.cc:1048] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:46.609099 10992 tablet_bootstrap.cc:492] T b4b13050d714473bb01ea0174d634a14 P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:46.613037 10992 ts_tablet_manager.cc:1403] T b4b13050d714473bb01ea0174d634a14 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.138s	user 0.014s	sys 0.008s
I20260709 13:15:46.612349 10990 tablet_bootstrap.cc:492] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:46.615983 10990 ts_tablet_manager.cc:1403] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.102s	user 0.010s	sys 0.004s
I20260709 13:15:46.623570 10990 raft_consensus.cc:359] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.625924 10990 raft_consensus.cc:385] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.627604 11031 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:46.628041 11031 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.629738 11031 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:46.632629 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:46.633339 10947 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 0.
I20260709 13:15:46.633169 10699 catalog_manager.cc:5697] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 1 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:46.627169 10990 raft_consensus.cc:740] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:46.634869 10766 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:15:46.635397 10990 consensus_queue.cc:260] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.637452 10990 ts_tablet_manager.cc:1434] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.021s	user 0.004s	sys 0.000s
I20260709 13:15:46.638368 10990 tablet_bootstrap.cc:492] T 1d9e744bb20c4325969f4fbc01c672e8 P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:46.642163 11201 raft_consensus.cc:2804] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:46.642602 11201 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:46.644147 10990 tablet_bootstrap.cc:654] T 1d9e744bb20c4325969f4fbc01c672e8 P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:46.644421 11201 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:46.644706 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:46.645750 10875 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 0.
I20260709 13:15:46.626498 10992 raft_consensus.cc:359] T b4b13050d714473bb01ea0174d634a14 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.653692 10992 raft_consensus.cc:385] T b4b13050d714473bb01ea0174d634a14 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.654102 10992 raft_consensus.cc:740] T b4b13050d714473bb01ea0174d634a14 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:46.653947 11201 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.654942 10992 consensus_queue.cc:260] T b4b13050d714473bb01ea0174d634a14 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.656550 11201 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:46.657222 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:15:46.657979 10947 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:46.659631 10990 tablet_bootstrap.cc:492] T 1d9e744bb20c4325969f4fbc01c672e8 P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:46.660137 10990 ts_tablet_manager.cc:1403] T 1d9e744bb20c4325969f4fbc01c672e8 P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.000s
I20260709 13:15:46.662814 10990 raft_consensus.cc:359] T 1d9e744bb20c4325969f4fbc01c672e8 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.663478 10990 raft_consensus.cc:385] T 1d9e744bb20c4325969f4fbc01c672e8 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.663740 10990 raft_consensus.cc:740] T 1d9e744bb20c4325969f4fbc01c672e8 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:46.663736 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:15:46.664507 10875 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:46.664621 10990 consensus_queue.cc:260] T 1d9e744bb20c4325969f4fbc01c672e8 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.666894 10990 ts_tablet_manager.cc:1434] T 1d9e744bb20c4325969f4fbc01c672e8 P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:15:46.667827 10990 tablet_bootstrap.cc:492] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:46.672861 10875 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 1.
I20260709 13:15:46.673909 10990 tablet_bootstrap.cc:654] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:46.673949 11015 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:46.675444 10947 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 1.
I20260709 13:15:46.679630 10991 tablet_bootstrap.cc:492] T 51ae8de979e44a2d8c155bffc20a1533 P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:46.690428 10991 ts_tablet_manager.cc:1403] T 51ae8de979e44a2d8c155bffc20a1533 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.121s	user 0.017s	sys 0.005s
I20260709 13:15:46.690160 11015 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.691051 10766 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:15:46.691996 11201 raft_consensus.cc:2804] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:46.692443 11201 raft_consensus.cc:697] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:15:46.693197 10991 raft_consensus.cc:359] T 51ae8de979e44a2d8c155bffc20a1533 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.693462 11201 consensus_queue.cc:237] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.693945 10991 raft_consensus.cc:385] T 51ae8de979e44a2d8c155bffc20a1533 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.694222 10991 raft_consensus.cc:740] T 51ae8de979e44a2d8c155bffc20a1533 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:46.694263 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:46.694893 10991 consensus_queue.cc:260] T 51ae8de979e44a2d8c155bffc20a1533 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.692380 11015 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:46.697896 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:15:46.699939 10916 leader_election.cc:343] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:15:46.700788 10916 leader_election.cc:343] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:46.701279 10916 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:46.702188 11015 raft_consensus.cc:2749] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:46.707825 10991 ts_tablet_manager.cc:1434] T 51ae8de979e44a2d8c155bffc20a1533 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:15:46.708803 10991 tablet_bootstrap.cc:492] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:46.714367 10992 ts_tablet_manager.cc:1434] T b4b13050d714473bb01ea0174d634a14 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.101s	user 0.006s	sys 0.000s
I20260709 13:15:46.715399 10992 tablet_bootstrap.cc:492] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:46.718286 10697 catalog_manager.cc:5697] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 0 to 1, leader changed from <none> to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 1 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:46.721047 10992 tablet_bootstrap.cc:654] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:46.736382 10991 tablet_bootstrap.cc:654] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:46.738152 10990 tablet_bootstrap.cc:492] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:46.738842 10990 ts_tablet_manager.cc:1403] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.071s	user 0.024s	sys 0.018s
I20260709 13:15:46.742776 10990 raft_consensus.cc:359] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.743803 10990 raft_consensus.cc:385] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.744174 10990 raft_consensus.cc:740] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:46.748050 10990 consensus_queue.cc:260] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.752501 10990 ts_tablet_manager.cc:1434] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.003s
I20260709 13:15:46.753389 10990 tablet_bootstrap.cc:492] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:46.760241 10992 tablet_bootstrap.cc:492] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:46.760728 10990 tablet_bootstrap.cc:654] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:46.760800 10992 ts_tablet_manager.cc:1403] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.046s	user 0.015s	sys 0.015s
I20260709 13:15:46.763787 10992 raft_consensus.cc:359] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.764534 10992 raft_consensus.cc:385] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.764804 10992 raft_consensus.cc:740] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:46.765468 10992 consensus_queue.cc:260] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.767985 10992 ts_tablet_manager.cc:1434] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:15:46.773833 10991 tablet_bootstrap.cc:492] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:46.774130 10990 tablet_bootstrap.cc:492] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:46.774160 10992 tablet_bootstrap.cc:492] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:46.774331 10991 ts_tablet_manager.cc:1403] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.066s	user 0.014s	sys 0.011s
I20260709 13:15:46.774737 10990 ts_tablet_manager.cc:1403] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.022s	user 0.011s	sys 0.008s
I20260709 13:15:46.777004 10991 raft_consensus.cc:359] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.777427 10990 raft_consensus.cc:359] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.777808 10991 raft_consensus.cc:385] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.778101 10990 raft_consensus.cc:385] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.778147 10991 raft_consensus.cc:740] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:46.778497 10990 raft_consensus.cc:740] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:46.779215 10990 consensus_queue.cc:260] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.781006 10992 tablet_bootstrap.cc:654] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:46.781234 10990 ts_tablet_manager.cc:1434] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:15:46.781430 10991 consensus_queue.cc:260] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.782469 10990 tablet_bootstrap.cc:492] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:46.783435 10991 ts_tablet_manager.cc:1434] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 13:15:46.784298 10991 tablet_bootstrap.cc:492] T 7595d019b2e74196bc5838962b53798d P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:46.790402 10990 tablet_bootstrap.cc:654] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:46.799275 10800 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:46.802898 10991 tablet_bootstrap.cc:654] T 7595d019b2e74196bc5838962b53798d P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:46.810616 11201 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:46.811219 11201 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.814466 11031 consensus_queue.cc:1048] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:46.819816 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:46.828474 11201 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:46.828998 10991 tablet_bootstrap.cc:492] T 7595d019b2e74196bc5838962b53798d P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:46.828909 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:46.829643 10991 ts_tablet_manager.cc:1403] T 7595d019b2e74196bc5838962b53798d P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.046s	user 0.007s	sys 0.005s
I20260709 13:15:46.830252 10871 raft_consensus.cc:2393] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 1: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:15:46.833271 10991 raft_consensus.cc:359] T 7595d019b2e74196bc5838962b53798d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.833993 10991 raft_consensus.cc:385] T 7595d019b2e74196bc5838962b53798d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.834260 10991 raft_consensus.cc:740] T 7595d019b2e74196bc5838962b53798d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:46.834911 10991 consensus_queue.cc:260] T 7595d019b2e74196bc5838962b53798d P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.842069 10767 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:15:46.842724 10991 ts_tablet_manager.cc:1434] T 7595d019b2e74196bc5838962b53798d P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:15:46.843968 10991 tablet_bootstrap.cc:492] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:46.848677 10990 tablet_bootstrap.cc:492] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:46.849188 10990 ts_tablet_manager.cc:1403] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.067s	user 0.024s	sys 0.020s
I20260709 13:15:46.851859 10990 raft_consensus.cc:359] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.852027 11201 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:46.858716 10991 tablet_bootstrap.cc:654] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:46.859058 11201 raft_consensus.cc:2749] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:46.859900 11015 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:46.860351 11015 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.864595 11201 consensus_queue.cc:1048] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:46.866453 10992 tablet_bootstrap.cc:492] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:46.867424 10990 raft_consensus.cc:385] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.867851 10990 raft_consensus.cc:740] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:46.870328 10990 consensus_queue.cc:260] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.872330 10990 ts_tablet_manager.cc:1434] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 13:15:46.873209 10990 tablet_bootstrap.cc:492] T 4dd4c0ff1def4960931dda3b52a4a884 P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:46.874171 11015 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:46.870750 10992 ts_tablet_manager.cc:1403] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.097s	user 0.011s	sys 0.001s
I20260709 13:15:46.877270 10992 raft_consensus.cc:359] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.877529 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:46.877988 10992 raft_consensus.cc:385] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.878312 10992 raft_consensus.cc:740] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
W20260709 13:15:46.879161 10916 leader_election.cc:343] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:46.879760 10990 tablet_bootstrap.cc:654] T 4dd4c0ff1def4960931dda3b52a4a884 P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:46.883375 10992 consensus_queue.cc:260] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.883553 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:46.885126 10992 ts_tablet_manager.cc:1434] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.000s
W20260709 13:15:46.885649 10916 leader_election.cc:343] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:46.885973 10992 tablet_bootstrap.cc:492] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:46.886085 10916 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:46.887696 11015 raft_consensus.cc:2749] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:46.904610 10990 tablet_bootstrap.cc:492] T 4dd4c0ff1def4960931dda3b52a4a884 P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:46.905224 10990 ts_tablet_manager.cc:1403] T 4dd4c0ff1def4960931dda3b52a4a884 P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.032s	user 0.009s	sys 0.003s
I20260709 13:15:46.909580 10990 raft_consensus.cc:359] T 4dd4c0ff1def4960931dda3b52a4a884 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.910506 10990 raft_consensus.cc:385] T 4dd4c0ff1def4960931dda3b52a4a884 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.910892 10990 raft_consensus.cc:740] T 4dd4c0ff1def4960931dda3b52a4a884 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:46.911942 11015 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:46.912478 11015 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.911908 10990 consensus_queue.cc:260] T 4dd4c0ff1def4960931dda3b52a4a884 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.914348 11015 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:46.916139 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:46.917423 10990 ts_tablet_manager.cc:1434] T 4dd4c0ff1def4960931dda3b52a4a884 P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.001s
I20260709 13:15:46.917263 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:46.918341 10990 tablet_bootstrap.cc:492] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:46.918568 10992 tablet_bootstrap.cc:654] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:46.918509 11040 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:46.919899 11040 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:46.922159 11040 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:46.924356 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:46.924754 10990 tablet_bootstrap.cc:654] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
W20260709 13:15:46.924724 10916 leader_election.cc:343] T 1d9e744bb20c4325969f4fbc01c672e8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:46.932709 10992 tablet_bootstrap.cc:492] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:46.933082 10992 ts_tablet_manager.cc:1403] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.047s	user 0.010s	sys 0.000s
I20260709 13:15:46.932986 10991 tablet_bootstrap.cc:492] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:46.933754 10991 ts_tablet_manager.cc:1403] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.090s	user 0.019s	sys 0.018s
I20260709 13:15:46.976471 10875 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:47.015237 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:47.020527 10798 raft_consensus.cc:2393] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 1: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:15:47.043246 10990 tablet_bootstrap.cc:492] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:47.043121 11052 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:47.044284 10990 ts_tablet_manager.cc:1403] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.126s	user 0.011s	sys 0.011s
I20260709 13:15:47.044037 11052 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.053573 11198 consensus_queue.cc:1048] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:15:47.055388 11052 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:47.056406 10992 raft_consensus.cc:359] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.057402 10992 raft_consensus.cc:385] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:47.057812 10992 raft_consensus.cc:740] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:47.060869 10875 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:47.058499 10992 consensus_queue.cc:260] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.063948 11198 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:47.016263 11271 consensus_queue.cc:1048] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.036s
I20260709 13:15:47.072340 10842 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:47.077315 10996 consensus_queue.cc:1048] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:47.081422 10992 ts_tablet_manager.cc:1434] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.148s	user 0.008s	sys 0.001s
I20260709 13:15:47.083314 11015 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:47.083959 10991 raft_consensus.cc:359] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.084077 11015 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.084832 10991 raft_consensus.cc:385] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:47.085172 10991 raft_consensus.cc:740] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:47.085865 10991 consensus_queue.cc:260] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.086915 11015 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:47.087272 11198 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.090190 11038 raft_consensus.cc:2749] T b4b13050d714473bb01ea0174d634a14 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:47.102481 10991 ts_tablet_manager.cc:1434] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.168s	user 0.005s	sys 0.005s
I20260709 13:15:47.104964 10990 raft_consensus.cc:359] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.105849 10990 raft_consensus.cc:385] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:47.106139 10990 raft_consensus.cc:740] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:47.109345 11198 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:47.116997 10990 consensus_queue.cc:260] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.124058 10992 tablet_bootstrap.cc:492] T 12d33f7a0f8f47e5ad57e854e52ae154 P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
W20260709 13:15:47.128074 10916 leader_election.cc:343] T 1d9e744bb20c4325969f4fbc01c672e8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:47.129040 10916 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:47.146514 10990 ts_tablet_manager.cc:1434] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.102s	user 0.005s	sys 0.005s
I20260709 13:15:47.165004 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:47.165841 10871 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 0.
I20260709 13:15:47.166905 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:47.173199 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:47.174881 10991 tablet_bootstrap.cc:492] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:47.177742 11274 raft_consensus.cc:2749] T 1d9e744bb20c4325969f4fbc01c672e8 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:47.179333 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:47.185503 11200 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:15:47.186111 11200 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.197104 11213 consensus_queue.cc:1048] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:15:47.201460 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:47.203608 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:47.203670 11200 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:47.218395 11141 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:47.220070 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:47.219111 11141 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.222471 11141 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:47.223170 10996 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:47.223703 10996 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.222523 10992 tablet_bootstrap.cc:654] T 12d33f7a0f8f47e5ad57e854e52ae154 P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:47.227208 10842 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:47.229743 10990 tablet_bootstrap.cc:492] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:47.230358 11275 consensus_queue.cc:1048] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:47.231442 10996 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:47.230337 10916 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:15:47.232293 10996 raft_consensus.cc:2749] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:15:47.232923 10916 leader_election.cc:343] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:47.229162 11213 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:47.234747 10916 leader_election.cc:343] T 4dd4c0ff1def4960931dda3b52a4a884 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:47.234440 11213 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.237758 11141 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:47.238468 11141 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:47.239048 11141 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:47.239809 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:47.237797 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:47.228888 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:47.243791 10948 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 0.
I20260709 13:15:47.245345 10841 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:15:47.248039 10996 raft_consensus.cc:2804] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:47.248462 10996 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:47.248790 10996 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:15:47.253957 10916 leader_election.cc:343] T 4dd4c0ff1def4960931dda3b52a4a884 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:47.255115 10916 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:47.250053 11274 consensus_queue.cc:1048] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:15:47.257486 10916 leader_election.cc:343] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:47.259918 10998 consensus_queue.cc:1048] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:15:47.262178 11015 consensus_queue.cc:1048] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:47.262984 11188 raft_consensus.cc:2749] T 4dd4c0ff1def4960931dda3b52a4a884 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:47.265125 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:15:47.269420 10916 leader_election.cc:343] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:47.271799 11275 consensus_queue.cc:1048] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 13:15:47.277758 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:47.283586 10796 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 0.
I20260709 13:15:47.289724 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:47.291167 10949 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 0.
I20260709 13:15:47.275272 11213 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:15:47.302385 10916 leader_election.cc:343] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:47.302862 10916 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:47.303632 10991 tablet_bootstrap.cc:654] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:47.304064 11275 raft_consensus.cc:2749] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:47.304767 10996 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.306255 10875 raft_consensus.cc:2393] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 1: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:15:47.310246 10992 tablet_bootstrap.cc:492] T 12d33f7a0f8f47e5ad57e854e52ae154 P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:47.310832 10992 ts_tablet_manager.cc:1403] T 12d33f7a0f8f47e5ad57e854e52ae154 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.204s	user 0.016s	sys 0.003s
I20260709 13:15:47.321614 10992 raft_consensus.cc:359] T 12d33f7a0f8f47e5ad57e854e52ae154 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.322346 10992 raft_consensus.cc:385] T 12d33f7a0f8f47e5ad57e854e52ae154 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:47.322445 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:15:47.322703 10992 raft_consensus.cc:740] T 12d33f7a0f8f47e5ad57e854e52ae154 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:47.323145 10942 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:47.323561 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:47.323395 10992 consensus_queue.cc:260] T 12d33f7a0f8f47e5ad57e854e52ae154 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.325351 10992 ts_tablet_manager.cc:1434] T 12d33f7a0f8f47e5ad57e854e52ae154 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:15:47.326464 10992 tablet_bootstrap.cc:492] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:47.329793 10996 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:47.333638 10942 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 1.
I20260709 13:15:47.342186 10841 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:15:47.344340 10996 raft_consensus.cc:2804] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:47.343743 10766 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:15:47.347080 11213 raft_consensus.cc:2804] T 7595d019b2e74196bc5838962b53798d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:47.347484 11213 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:47.344761 10996 raft_consensus.cc:697] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:15:47.354105 10996 consensus_queue.cc:237] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.322038 11141 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.357272 11213 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:47.386236 11101 consensus_queue.cc:1048] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:47.402803 10992 tablet_bootstrap.cc:654] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:47.410108 10991 tablet_bootstrap.cc:492] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:47.410621 10991 ts_tablet_manager.cc:1403] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.275s	user 0.019s	sys 0.013s
I20260709 13:15:47.417507 10990 tablet_bootstrap.cc:654] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:47.427527 11141 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:47.430244 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:15:47.433301 10876 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:47.439395 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:15:47.443527 10916 leader_election.cc:343] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:47.443940 10916 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:47.452952 11198 raft_consensus.cc:2749] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:47.409785 11213 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.491869 10991 raft_consensus.cc:359] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.497694 10991 raft_consensus.cc:385] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:47.498047 10991 raft_consensus.cc:740] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:47.463400 11205 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:47.499480 11205 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.472231 11101 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:47.502381 11101 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.508471 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:15:47.510267 10916 leader_election.cc:343] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:47.511075 11205 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:47.513820 10876 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 1.
I20260709 13:15:47.519361 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:15:47.536442 10991 consensus_queue.cc:260] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:15:47.537649 10916 leader_election.cc:343] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:47.541247 10916 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:47.540616 11213 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:47.553188 10991 ts_tablet_manager.cc:1434] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.142s	user 0.009s	sys 0.001s
I20260709 13:15:47.548521 11141 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:15:47.471124 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 44984us
I20260709 13:15:47.563074 10991 tablet_bootstrap.cc:492] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:47.564117 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:15:47.564927 10800 raft_consensus.cc:2393] T 7595d019b2e74196bc5838962b53798d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 1: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:15:47.560273 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:47.560868 11141 raft_consensus.cc:697] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:15:47.571930 10990 tablet_bootstrap.cc:492] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:47.569092 10876 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 0.
I20260709 13:15:47.572422 10990 ts_tablet_manager.cc:1403] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.343s	user 0.019s	sys 0.008s
I20260709 13:15:47.573078 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:15:47.573868 10949 raft_consensus.cc:2393] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 1: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:15:47.577880 11141 consensus_queue.cc:237] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.578648 10992 tablet_bootstrap.cc:492] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:47.579171 10992 ts_tablet_manager.cc:1403] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.253s	user 0.022s	sys 0.019s
I20260709 13:15:47.582521 10990 raft_consensus.cc:359] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.583834 10990 raft_consensus.cc:385] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:47.584177 10990 raft_consensus.cc:740] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:47.584887 10990 consensus_queue.cc:260] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.581152 10699 catalog_manager.cc:5697] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 0 to 1, leader changed from <none> to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:47.588116 10990 ts_tablet_manager.cc:1434] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.001s
I20260709 13:15:47.590003 10990 tablet_bootstrap.cc:492] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:47.590282 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:47.590512 10766 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:15:47.596709 11273 raft_consensus.cc:2749] T 7595d019b2e74196bc5838962b53798d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:15:47.593508 10991 tablet_bootstrap.cc:654] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:47.600728 10992 raft_consensus.cc:359] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.601466 10992 raft_consensus.cc:385] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:47.601788 10992 raft_consensus.cc:740] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:47.607046 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:47.602494 10992 consensus_queue.cc:260] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.594218 11101 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:15:47.594746 10747 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:15:47.608799 10947 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 0.
I20260709 13:15:47.613291 10766 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:15:47.614233 11273 raft_consensus.cc:2804] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:47.614650 11273 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:47.615028 11273 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:47.617120 10992 ts_tablet_manager.cc:1434] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.038s	user 0.006s	sys 0.000s
I20260709 13:15:47.618294 10992 tablet_bootstrap.cc:492] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:47.622941 11273 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.624559 10990 tablet_bootstrap.cc:654] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
W20260709 13:15:47.628506 10916 leader_election.cc:343] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:47.631029 10916 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:47.631150 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:15:47.635502 10876 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:47.628715 10992 tablet_bootstrap.cc:654] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:47.638257 11015 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:47.639312 11015 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:47.639894 11015 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.642282 11015 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:47.643677 10876 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 1.
I20260709 13:15:47.643762 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:15:47.644418 10947 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:47.646982 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:47.648440 10991 tablet_bootstrap.cc:492] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:47.648936 10991 ts_tablet_manager.cc:1403] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.089s	user 0.007s	sys 0.010s
I20260709 13:15:47.649012 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:15:47.656906 10916 leader_election.cc:343] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:15:47.657500 10916 leader_election.cc:343] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:47.657795 10916 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:47.649493 11273 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:47.662902 11205 raft_consensus.cc:2749] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:47.668088 10767 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:15:47.672137 11273 raft_consensus.cc:2804] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:47.673177 11273 raft_consensus.cc:697] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:15:47.675612 10947 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 1.
I20260709 13:15:47.684029 11038 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:47.684801 11038 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.687768 11038 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:47.674683 11273 consensus_queue.cc:237] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.689348 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:47.690045 10948 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 0.
I20260709 13:15:47.690220 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:47.691524 10841 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:15:47.694361 11038 raft_consensus.cc:2804] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:47.697962 11038 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:47.701308 11038 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:15:47.708660 10842 leader_election.cc:343] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:47.713356 10992 tablet_bootstrap.cc:492] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:47.713858 10992 ts_tablet_manager.cc:1403] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.096s	user 0.025s	sys 0.025s
I20260709 13:15:47.716573 10992 raft_consensus.cc:359] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.717228 10992 raft_consensus.cc:385] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:47.717501 10992 raft_consensus.cc:740] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:47.720088 10991 raft_consensus.cc:359] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.720810 10991 raft_consensus.cc:385] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:47.721084 10991 raft_consensus.cc:740] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:47.721735 10991 consensus_queue.cc:260] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.722123 11038 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.724416 10990 tablet_bootstrap.cc:492] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:47.724885 10990 ts_tablet_manager.cc:1403] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.135s	user 0.024s	sys 0.034s
I20260709 13:15:47.725441 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:15:47.726187 10801 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:47.726100 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:15:47.724915 10992 consensus_queue.cc:260] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.727047 10948 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:47.727514 11038 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:47.727669 10990 raft_consensus.cc:359] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.728318 10990 raft_consensus.cc:385] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:47.728593 10990 raft_consensus.cc:740] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:47.729276 10990 consensus_queue.cc:260] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.729856 10992 ts_tablet_manager.cc:1434] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:15:47.731433 10990 ts_tablet_manager.cc:1434] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:15:47.732488 10990 tablet_bootstrap.cc:492] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:47.732610 10992 tablet_bootstrap.cc:492] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:47.743026 10991 ts_tablet_manager.cc:1434] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.094s	user 0.006s	sys 0.006s
I20260709 13:15:47.744059 10991 tablet_bootstrap.cc:492] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:47.755303 10992 tablet_bootstrap.cc:654] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:47.756156 10801 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 1.
I20260709 13:15:47.757478 10842 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:15:47.759013 10948 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 1.
I20260709 13:15:47.759403 10996 raft_consensus.cc:2804] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:47.759912 10996 raft_consensus.cc:697] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:15:47.760646 10996 consensus_queue.cc:237] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.760645 10990 tablet_bootstrap.cc:654] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:47.785764 10991 tablet_bootstrap.cc:654] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:47.790192 10992 tablet_bootstrap.cc:492] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:47.790720 10992 ts_tablet_manager.cc:1403] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.059s	user 0.015s	sys 0.003s
I20260709 13:15:47.796160 10992 raft_consensus.cc:359] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.796962 10992 raft_consensus.cc:385] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:47.797300 10992 raft_consensus.cc:740] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:47.798036 10992 consensus_queue.cc:260] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.799676 10990 tablet_bootstrap.cc:492] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:47.800202 10990 ts_tablet_manager.cc:1403] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.068s	user 0.006s	sys 0.025s
I20260709 13:15:47.801268 10992 ts_tablet_manager.cc:1434] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:15:47.802366 10992 tablet_bootstrap.cc:492] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:47.805908 11200 consensus_queue.cc:1048] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:47.809151 10992 tablet_bootstrap.cc:654] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:47.820102 11038 consensus_queue.cc:1048] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:47.823150 10992 tablet_bootstrap.cc:492] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:47.823679 10992 ts_tablet_manager.cc:1403] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.022s	user 0.009s	sys 0.007s
I20260709 13:15:47.827162 10992 raft_consensus.cc:359] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.827862 10992 raft_consensus.cc:385] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:47.828109 10992 raft_consensus.cc:740] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:47.828701 10992 consensus_queue.cc:260] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.833539 10992 ts_tablet_manager.cc:1434] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:15:47.834481 10992 tablet_bootstrap.cc:492] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:47.842936 10992 tablet_bootstrap.cc:654] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:47.862775 10699 catalog_manager.cc:5697] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 0 to 1, leader changed from <none> to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:47.862499 11038 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:47.870573 10990 raft_consensus.cc:359] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.871371 10990 raft_consensus.cc:385] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:47.871666 10990 raft_consensus.cc:740] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:47.872721 10990 consensus_queue.cc:260] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.888170 10990 ts_tablet_manager.cc:1434] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.088s	user 0.008s	sys 0.000s
I20260709 13:15:47.889040 10990 tablet_bootstrap.cc:492] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:47.871799 11038 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.892508 11038 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:47.873536 10992 tablet_bootstrap.cc:492] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:47.885788 10697 catalog_manager.cc:5697] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 0 to 1, leader changed from <none> to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 1 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:47.893885 10992 ts_tablet_manager.cc:1403] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.060s	user 0.013s	sys 0.017s
I20260709 13:15:47.895614 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:47.896189 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:47.896821 10992 raft_consensus.cc:359] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.897591 10992 raft_consensus.cc:385] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:47.897898 10992 raft_consensus.cc:740] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:47.898696 10992 consensus_queue.cc:260] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:15:47.903034 10841 leader_election.cc:343] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:47.904405 10992 ts_tablet_manager.cc:1434] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:15:47.905424 10992 tablet_bootstrap.cc:492] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:47.907248 11273 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:47.907770 11273 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.908634 10991 tablet_bootstrap.cc:492] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:47.909089 10991 ts_tablet_manager.cc:1403] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.165s	user 0.025s	sys 0.026s
I20260709 13:15:47.909971 11273 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:15:47.910406 10842 leader_election.cc:343] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:47.910884 10842 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:47.911722 10991 raft_consensus.cc:359] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.912431 10991 raft_consensus.cc:385] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:47.912696 10991 raft_consensus.cc:740] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:47.912936 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:47.913353 10991 consensus_queue.cc:260] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.913749 10871 raft_consensus.cc:2393] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 1: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:15:47.913673 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:47.914770 10990 tablet_bootstrap.cc:654] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:47.911906 11038 raft_consensus.cc:2749] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:47.915402 10766 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:15:47.916862 11276 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:47.918664 10991 ts_tablet_manager.cc:1434] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:15:47.920306 10991 tablet_bootstrap.cc:492] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:47.928625 10990 tablet_bootstrap.cc:492] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:47.929163 10990 ts_tablet_manager.cc:1403] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.040s	user 0.008s	sys 0.004s
I20260709 13:15:47.929910 11276 raft_consensus.cc:2749] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:47.932723 10699 catalog_manager.cc:5697] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 1 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:47.936136 10990 raft_consensus.cc:359] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.936866 10990 raft_consensus.cc:385] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:47.937157 10990 raft_consensus.cc:740] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:47.939505 10992 tablet_bootstrap.cc:654] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:47.941542 10991 tablet_bootstrap.cc:654] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:47.943293 10990 consensus_queue.cc:260] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.948500 10990 ts_tablet_manager.cc:1434] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
I20260709 13:15:47.949585 10990 tablet_bootstrap.cc:492] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:47.969347 10990 tablet_bootstrap.cc:654] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:47.973676 10991 tablet_bootstrap.cc:492] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:47.976969 10991 ts_tablet_manager.cc:1403] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.057s	user 0.013s	sys 0.000s
I20260709 13:15:47.980355 10991 raft_consensus.cc:359] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.981161 10991 raft_consensus.cc:385] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:47.981518 10991 raft_consensus.cc:740] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:47.982316 10991 consensus_queue.cc:260] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:47.984007 10991 ts_tablet_manager.cc:1434] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:15:47.985013 10991 tablet_bootstrap.cc:492] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:48.001885 10990 tablet_bootstrap.cc:492] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:48.002540 10990 ts_tablet_manager.cc:1403] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.053s	user 0.007s	sys 0.004s
I20260709 13:15:48.005716 10990 raft_consensus.cc:359] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.008406 10990 raft_consensus.cc:385] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:48.009071 10990 raft_consensus.cc:740] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:48.009740 10990 consensus_queue.cc:260] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.011823 10991 tablet_bootstrap.cc:654] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:48.017874 10990 ts_tablet_manager.cc:1434] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.002s
I20260709 13:15:48.019168 10990 tablet_bootstrap.cc:492] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:48.025411 10990 tablet_bootstrap.cc:654] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:48.032382 11038 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:48.033123 11038 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.036744 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:48.037377 10990 tablet_bootstrap.cc:492] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:48.037674 10991 tablet_bootstrap.cc:492] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:48.037530 10801 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 0.
I20260709 13:15:48.037879 10990 ts_tablet_manager.cc:1403] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.019s	user 0.011s	sys 0.005s
I20260709 13:15:48.038288 10991 ts_tablet_manager.cc:1403] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.053s	user 0.011s	sys 0.000s
I20260709 13:15:48.038914 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:48.039427 11038 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:48.039662 10949 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 0.
I20260709 13:15:48.041260 10841 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:15:48.041525 10991 raft_consensus.cc:359] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.041411 10990 raft_consensus.cc:359] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.042265 10991 raft_consensus.cc:385] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:48.042330 10990 raft_consensus.cc:385] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:48.042619 10991 raft_consensus.cc:740] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:48.042725 10990 raft_consensus.cc:740] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:48.043535 11038 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:48.043376 10991 consensus_queue.cc:260] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.043576 10990 consensus_queue.cc:260] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.045780 10990 ts_tablet_manager.cc:1434] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:15:48.045782 10991 ts_tablet_manager.cc:1434] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:15:48.046919 10990 tablet_bootstrap.cc:492] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:48.044235 11038 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:48.047698 11038 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:48.049434 11200 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:48.053593 10990 tablet_bootstrap.cc:654] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:48.053411 11200 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.056030 11200 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:48.057781 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:48.058610 10949 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 0.
I20260709 13:15:48.060156 10841 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:15:48.061064 11200 raft_consensus.cc:2804] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:48.061542 11200 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:48.061949 11200 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:48.056192 10991 tablet_bootstrap.cc:492] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:48.068887 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:15:48.070575 10842 leader_election.cc:343] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:48.071342 11200 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.081508 11200 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:48.082229 11205 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:48.086723 11038 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.087246 11205 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.088824 11038 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:48.089214 11205 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:48.093711 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:15:48.094559 10801 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:48.095373 10990 tablet_bootstrap.cc:492] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:48.097491 10990 ts_tablet_manager.cc:1403] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.051s	user 0.029s	sys 0.015s
I20260709 13:15:48.100291 10990 raft_consensus.cc:359] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.101049 10990 raft_consensus.cc:385] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:48.101312 10990 raft_consensus.cc:740] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:48.102124 10990 consensus_queue.cc:260] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.103752 11276 consensus_queue.cc:1048] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:48.104260 10991 tablet_bootstrap.cc:654] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:48.086383 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:15:48.104920 10949 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:48.105698 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:15:48.106431 10948 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:48.113013 10949 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 1.
I20260709 13:15:48.104316 10990 ts_tablet_manager.cc:1434] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:15:48.115232 10990 tablet_bootstrap.cc:492] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:48.116345 10841 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:15:48.117415 11038 raft_consensus.cc:2804] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:48.118088 11038 raft_consensus.cc:697] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:15:48.119290 11038 consensus_queue.cc:237] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.123828 10990 tablet_bootstrap.cc:654] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:48.129266 10948 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 1.
I20260709 13:15:48.084607 10992 tablet_bootstrap.cc:492] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:48.130656 10992 ts_tablet_manager.cc:1403] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.225s	user 0.016s	sys 0.001s
I20260709 13:15:48.144526 10841 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:15:48.149376 11038 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:48.150265 10992 raft_consensus.cc:359] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.151027 10992 raft_consensus.cc:385] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:48.151317 10992 raft_consensus.cc:740] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:48.144726 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:15:48.152725 10992 consensus_queue.cc:260] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.151911 11038 raft_consensus.cc:697] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:15:48.160575 10842 leader_election.cc:343] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:48.163599 11200 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:48.164125 11200 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.165477 10990 tablet_bootstrap.cc:492] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:48.166107 10990 ts_tablet_manager.cc:1403] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.051s	user 0.016s	sys 0.014s
I20260709 13:15:48.166978 11200 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:48.167282 11213 consensus_queue.cc:1048] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:48.168414 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:48.169276 10949 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 0.
I20260709 13:15:48.170753 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:48.171042 10841 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:15:48.175189 11200 raft_consensus.cc:2804] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:48.176595 11200 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:48.177182 11200 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:15:48.177553 10842 leader_election.cc:343] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:48.178651 10992 ts_tablet_manager.cc:1434] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.041s	user 0.006s	sys 0.002s
I20260709 13:15:48.179790 10992 tablet_bootstrap.cc:492] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:48.180104 10990 raft_consensus.cc:359] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.189743 10990 raft_consensus.cc:385] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:48.191290 10992 tablet_bootstrap.cc:654] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:48.191536 10990 raft_consensus.cc:740] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:48.192595 10990 consensus_queue.cc:260] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.199489 10990 ts_tablet_manager.cc:1434] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.033s	user 0.009s	sys 0.000s
I20260709 13:15:48.202461 10990 tablet_bootstrap.cc:492] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:48.217669 10801 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 1.
I20260709 13:15:48.226962 11038 consensus_queue.cc:237] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.248809 10996 consensus_queue.cc:1048] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:48.250325 10992 tablet_bootstrap.cc:492] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:48.250865 10992 ts_tablet_manager.cc:1403] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.071s	user 0.018s	sys 0.002s
I20260709 13:15:48.234135 11200 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.262460 10991 tablet_bootstrap.cc:492] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:48.263150 10991 ts_tablet_manager.cc:1403] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.207s	user 0.023s	sys 0.058s
I20260709 13:15:48.268471 11200 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:48.269743 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:15:48.270387 10949 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:48.280089 10990 tablet_bootstrap.cc:654] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
W20260709 13:15:48.293720 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 216992us
I20260709 13:15:48.294075 10949 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 1.
I20260709 13:15:48.295610 10841 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:15:48.300501 11015 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:48.301231 11015 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.302242 11038 raft_consensus.cc:2804] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:48.303861 11015 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:48.307085 11038 raft_consensus.cc:697] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:15:48.308182 11038 consensus_queue.cc:237] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.312798 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:48.317740 10992 raft_consensus.cc:359] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.325219 10992 raft_consensus.cc:385] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:48.325609 10992 raft_consensus.cc:740] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:48.315526 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:48.320986 11198 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:48.326538 10992 consensus_queue.cc:260] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.324853 10990 tablet_bootstrap.cc:492] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:48.327823 10990 ts_tablet_manager.cc:1403] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.126s	user 0.014s	sys 0.000s
I20260709 13:15:48.327220 11198 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.330070 11198 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:48.332008 10992 ts_tablet_manager.cc:1434] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.081s	user 0.006s	sys 0.000s
I20260709 13:15:48.330596 10990 raft_consensus.cc:359] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.333056 10990 raft_consensus.cc:385] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:48.333369 10990 raft_consensus.cc:740] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:48.290848 10991 raft_consensus.cc:359] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.336853 10991 raft_consensus.cc:385] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:48.337175 10991 raft_consensus.cc:740] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:48.337845 10991 consensus_queue.cc:260] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.340207 10991 ts_tablet_manager.cc:1434] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.077s	user 0.006s	sys 0.000s
I20260709 13:15:48.341627 11198 consensus_queue.cc:1048] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:15:48.346400 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 97969us
W20260709 13:15:48.347517 10916 leader_election.cc:343] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:48.348356 10990 consensus_queue.cc:260] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.334303 10992 tablet_bootstrap.cc:492] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:48.350068 11198 consensus_queue.cc:1048] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:15:48.351611 10916 leader_election.cc:343] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:48.353282 11031 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:48.353797 10916 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:48.353914 11031 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.357663 11031 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:48.357875 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:48.358515 11328 raft_consensus.cc:2749] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:48.358585 10949 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 0.
I20260709 13:15:48.359153 10996 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:48.359624 10996 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.360318 10766 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:15:48.361593 10996 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:48.362208 10990 ts_tablet_manager.cc:1434] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.001s
I20260709 13:15:48.362323 11031 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:48.362772 11031 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:48.363145 11031 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:48.363310 10990 tablet_bootstrap.cc:492] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:48.364118 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:15:48.367676 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:48.371024 10948 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 0.
I20260709 13:15:48.373265 10991 tablet_bootstrap.cc:492] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:48.375278 10841 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:15:48.367908 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:48.368223 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:48.380139 11271 consensus_queue.cc:1048] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:48.381484 11271 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:48.381807 11271 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:48.382097 11271 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:48.386358 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:48.387295 10870 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 0.
I20260709 13:15:48.388070 11031 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.389904 11031 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:48.395349 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:15:48.396137 10948 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:48.396384 10990 tablet_bootstrap.cc:654] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:48.397953 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:48.379366 10992 tablet_bootstrap.cc:654] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:48.358263 10699 catalog_manager.cc:5697] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 0 to 1, leader changed from <none> to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:48.407125 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:15:48.409504 10916 leader_election.cc:343] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:48.413686 10699 catalog_manager.cc:5697] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 0 to 1, leader changed from <none> to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:48.418053 10948 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 1.
I20260709 13:15:48.428205 11205 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:48.428486 10916 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:15:48.428726 11205 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.429979 11141 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:48.430370 11141 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:48.430720 11141 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:48.430753 11205 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:48.431900 10766 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:15:48.433266 11213 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:48.433718 11213 raft_consensus.cc:697] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:15:48.434551 11213 consensus_queue.cc:237] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.437836 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:15:48.427578 10842 leader_election.cc:343] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:48.438702 10800 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 0.
I20260709 13:15:48.441804 11141 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.452280 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:15:48.491401 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 61856.6us
I20260709 13:15:48.505050 11031 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:48.505676 11031 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.565610 11031 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:48.584420 10990 tablet_bootstrap.cc:492] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:48.584985 10990 ts_tablet_manager.cc:1403] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.222s	user 0.028s	sys 0.013s
I20260709 13:15:48.588167 10990 raft_consensus.cc:359] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.588986 10990 raft_consensus.cc:385] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:48.589843 10990 raft_consensus.cc:740] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:48.590575 10990 consensus_queue.cc:260] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:15:48.595975 10916 leader_election.cc:343] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:48.596647 10916 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:48.599037 10990 ts_tablet_manager.cc:1434] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:15:48.599169 11015 raft_consensus.cc:2749] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:48.601711 10992 tablet_bootstrap.cc:492] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:48.602293 10992 ts_tablet_manager.cc:1403] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.268s	user 0.020s	sys 0.004s
I20260709 13:15:48.577142 11328 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:48.608013 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:48.617534 11015 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:48.618214 11015 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.623049 10990 tablet_bootstrap.cc:492] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:48.627848 11015 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:48.606655 11328 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.642889 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:48.643909 10948 raft_consensus.cc:2393] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 1: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:15:48.632251 11141 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:48.647564 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:48.638978 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:48.661917 11271 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.663266 10766 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:15:48.670559 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:48.674943 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:15:48.679200 10798 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:48.679760 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:48.681449 10991 tablet_bootstrap.cc:654] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:48.677050 11276 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:48.703032 10870 raft_consensus.cc:2393] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 1: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:15:48.705160 11329 log.cc:927] Time spent T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 13:15:48.723951 10990 tablet_bootstrap.cc:654] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:48.710517 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:15:48.733490 10916 leader_election.cc:343] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:15:48.741909 10916 leader_election.cc:343] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:15:48.750514 10916 leader_election.cc:343] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:48.751103 10916 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:48.736392 10871 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:15:48.758469 10916 leader_election.cc:343] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:48.759162 10916 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:48.752136 10992 raft_consensus.cc:359] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.760059 10992 raft_consensus.cc:385] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:48.760495 10992 raft_consensus.cc:740] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:48.762106 11141 raft_consensus.cc:2749] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:48.769289 11271 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:48.771809 10798 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 1.
I20260709 13:15:48.781389 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:48.765275 11328 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:48.784192 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:15:48.785224 11271 consensus_queue.cc:1048] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:48.787772 10996 consensus_queue.cc:1048] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:48.788079 11271 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:48.788600 11271 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.804795 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:15:48.810182 11338 raft_consensus.cc:2749] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:48.805753 10942 raft_consensus.cc:2393] T 4dd4c0ff1def4960931dda3b52a4a884 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 1: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:15:48.824604 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:48.832381 10992 consensus_queue.cc:260] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.808158 11271 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:15:48.841455 10747 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:15:48.849942 11337 log.cc:927] Time spent T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:15:48.850674 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:48.842585 11141 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:48.852125 11276 raft_consensus.cc:2749] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:48.852031 11141 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.854189 11141 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:48.855872 10871 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 1.
I20260709 13:15:48.862862 10992 ts_tablet_manager.cc:1434] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.259s	user 0.006s	sys 0.005s
I20260709 13:15:48.863920 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:48.864614 10944 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 0.
I20260709 13:15:48.870445 10992 tablet_bootstrap.cc:492] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:48.872305 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:48.872555 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:48.875814 10990 tablet_bootstrap.cc:492] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:48.876272 10990 ts_tablet_manager.cc:1403] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.261s	user 0.029s	sys 0.020s
W20260709 13:15:48.874564 10916 leader_election.cc:343] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:48.881372 10990 raft_consensus.cc:359] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.882202 10990 raft_consensus.cc:385] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:48.882534 10990 raft_consensus.cc:740] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:48.883298 10990 consensus_queue.cc:260] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.883472 10916 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:15:48.887424 11141 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:48.888178 11141 raft_consensus.cc:697] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:15:48.888804 10991 tablet_bootstrap.cc:492] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:48.888906 10990 ts_tablet_manager.cc:1434] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
W20260709 13:15:48.889855 10842 leader_election.cc:343] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:48.889312 11141 consensus_queue.cc:237] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:15:48.891072 10842 consensus_peers.cc:597] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:48.889320 10991 ts_tablet_manager.cc:1403] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.516s	user 0.018s	sys 0.003s
I20260709 13:15:48.891719 10990 tablet_bootstrap.cc:492] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:48.893373 11200 consensus_queue.cc:1048] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:48.921981 11200 consensus_queue.cc:1048] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:48.931964 11276 consensus_queue.cc:1048] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:48.931349 10991 raft_consensus.cc:359] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:48.947558 10991 raft_consensus.cc:385] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:48.947886 10991 raft_consensus.cc:740] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:48.948513 10991 consensus_queue.cc:260] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:15:48.885350 10916 leader_election.cc:343] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:15:48.950353 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 65054.2us
I20260709 13:15:48.957194 10991 ts_tablet_manager.cc:1434] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.065s	user 0.005s	sys 0.001s
W20260709 13:15:48.972025 10916 leader_election.cc:343] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:48.980397 10916 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:48.981115 11015 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:15:48.982162 10916 leader_election.cc:343] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:48.982564 10916 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:48.983392 11015 raft_consensus.cc:2749] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:48.986593 11213 consensus_queue.cc:1048] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 13:15:49.004444 10697 catalog_manager.cc:5697] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 0 to 1, leader changed from <none> to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 1 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:49.006824 10991 tablet_bootstrap.cc:492] T 185480268a684958b8347a4f53af57c5 P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:49.010468 11328 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:49.011122 11328 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.011637 10841 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:49.013552 11328 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:49.021600 10841 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:49.023583 11325 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:49.023980 11325 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:49.024389 11325 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:49.023978 10992 tablet_bootstrap.cc:654] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:49.036454 11325 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.039742 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:15:49.040372 10798 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:49.042631 10699 catalog_manager.cc:5697] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 0 to 1, leader changed from <none> to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:49.044376 11038 consensus_queue.cc:1048] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:49.059990 10990 tablet_bootstrap.cc:654] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:49.063985 11271 raft_consensus.cc:2749] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:15:49.065246 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:49.067677 11198 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:49.068663 11198 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.069628 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:15:49.067646 10916 leader_election.cc:343] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:49.073964 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:15:49.077445 10916 leader_election.cc:343] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:49.077920 10916 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:49.082072 10798 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 1.
W20260709 13:15:49.083913 10916 leader_election.cc:343] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:49.084901 10842 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:15:49.082901 11198 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:49.091517 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:49.087862 11271 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:49.093824 11271 raft_consensus.cc:697] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:15:49.102823 11333 raft_consensus.cc:2749] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:49.105062 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:15:49.105643 10948 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:49.095498 11271 consensus_queue.cc:237] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.117274 10948 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 1.
W20260709 13:15:49.099961 10916 leader_election.cc:343] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:49.119271 10916 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:49.120062 11328 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:49.121186 10698 catalog_manager.cc:5697] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 1 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:49.096408 11325 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:49.144752 10990 tablet_bootstrap.cc:492] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:49.145470 10990 ts_tablet_manager.cc:1403] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.254s	user 0.004s	sys 0.012s
I20260709 13:15:49.150758 10991 tablet_bootstrap.cc:654] T 185480268a684958b8347a4f53af57c5 P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:49.152343 10990 raft_consensus.cc:359] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.164819 10990 raft_consensus.cc:385] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:49.165138 10990 raft_consensus.cc:740] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:49.165915 10990 consensus_queue.cc:260] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.158213 10699 catalog_manager.cc:5697] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 0 to 1, leader changed from <none> to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:49.164443 10992 tablet_bootstrap.cc:492] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:49.176759 10992 ts_tablet_manager.cc:1403] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.307s	user 0.015s	sys 0.022s
I20260709 13:15:49.189009 10990 ts_tablet_manager.cc:1434] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.043s	user 0.001s	sys 0.005s
I20260709 13:15:49.190235 10990 tablet_bootstrap.cc:492] T dde624b07ed446e5b7347791196396e2 P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:49.199555 10992 raft_consensus.cc:359] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.200416 10992 raft_consensus.cc:385] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:49.200714 10992 raft_consensus.cc:740] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:49.201454 10992 consensus_queue.cc:260] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.206480 10991 tablet_bootstrap.cc:492] T 185480268a684958b8347a4f53af57c5 P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:49.207212 10991 ts_tablet_manager.cc:1403] T 185480268a684958b8347a4f53af57c5 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.206s	user 0.009s	sys 0.004s
I20260709 13:15:49.209990 10991 raft_consensus.cc:359] T 185480268a684958b8347a4f53af57c5 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.210722 10991 raft_consensus.cc:385] T 185480268a684958b8347a4f53af57c5 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:49.211027 10991 raft_consensus.cc:740] T 185480268a684958b8347a4f53af57c5 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:49.214922 10992 ts_tablet_manager.cc:1434] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.038s	user 0.006s	sys 0.002s
I20260709 13:15:49.216063 10992 tablet_bootstrap.cc:492] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:49.225190 10990 tablet_bootstrap.cc:654] T dde624b07ed446e5b7347791196396e2 P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:49.227068 10991 consensus_queue.cc:260] T 185480268a684958b8347a4f53af57c5 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.229831 10991 ts_tablet_manager.cc:1434] T 185480268a684958b8347a4f53af57c5 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.000s
I20260709 13:15:49.235843 10992 tablet_bootstrap.cc:654] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:49.251140 10991 tablet_bootstrap.cc:492] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:49.258589 10991 tablet_bootstrap.cc:654] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:49.259049 10990 tablet_bootstrap.cc:492] T dde624b07ed446e5b7347791196396e2 P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:49.267374 10990 ts_tablet_manager.cc:1403] T dde624b07ed446e5b7347791196396e2 P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.077s	user 0.010s	sys 0.007s
I20260709 13:15:49.270138 10990 raft_consensus.cc:359] T dde624b07ed446e5b7347791196396e2 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.261348 11325 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:49.271082 11325 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.274338 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:49.275101 10947 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 0.
I20260709 13:15:49.276317 10841 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:15:49.277253 11200 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:49.277616 11200 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:49.277915 11200 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:49.279042 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:15:49.280620 10842 leader_election.cc:343] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:49.282981 11325 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:49.290588 11200 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.260392 10992 tablet_bootstrap.cc:492] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:49.294246 10992 ts_tablet_manager.cc:1403] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.078s	user 0.011s	sys 0.003s
I20260709 13:15:49.294118 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:15:49.296257 10842 leader_election.cc:343] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:49.291653 10990 raft_consensus.cc:385] T dde624b07ed446e5b7347791196396e2 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:49.297183 11198 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:49.296790 10990 raft_consensus.cc:740] T dde624b07ed446e5b7347791196396e2 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:49.297814 11198 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.298511 10990 consensus_queue.cc:260] T dde624b07ed446e5b7347791196396e2 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.299836 11198 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:49.302452 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:49.300698 10990 ts_tablet_manager.cc:1434] T dde624b07ed446e5b7347791196396e2 P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.033s	user 0.005s	sys 0.000s
I20260709 13:15:49.305096 10990 tablet_bootstrap.cc:492] T 5c8a233bc6c44bd2b5d2d44a726b147c P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
W20260709 13:15:49.306545 10916 leader_election.cc:343] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:49.307427 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:49.308530 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:15:49.309075 10916 leader_election.cc:343] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:49.309175 10947 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:49.309494 10916 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:49.310597 11328 raft_consensus.cc:2749] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:49.298898 10992 raft_consensus.cc:359] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.314162 10992 raft_consensus.cc:385] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:49.314473 10992 raft_consensus.cc:740] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:49.316042 11200 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:49.320611 10990 tablet_bootstrap.cc:654] T 5c8a233bc6c44bd2b5d2d44a726b147c P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:49.316242 10992 consensus_queue.cc:260] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.325413 10991 tablet_bootstrap.cc:492] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:49.325897 10991 ts_tablet_manager.cc:1403] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.075s	user 0.009s	sys 0.005s
I20260709 13:15:49.327737 10992 ts_tablet_manager.cc:1434] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.000s
I20260709 13:15:49.328694 10992 tablet_bootstrap.cc:492] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:49.332099 10947 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 1.
I20260709 13:15:49.333477 10841 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:49.334540 11200 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:49.335246 11200 raft_consensus.cc:697] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:15:49.336133 11200 consensus_queue.cc:237] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.346287 10992 tablet_bootstrap.cc:654] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:49.349285 10990 tablet_bootstrap.cc:492] T 5c8a233bc6c44bd2b5d2d44a726b147c P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:49.349774 10990 ts_tablet_manager.cc:1403] T 5c8a233bc6c44bd2b5d2d44a726b147c P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.045s	user 0.008s	sys 0.002s
I20260709 13:15:49.352587 10990 raft_consensus.cc:359] T 5c8a233bc6c44bd2b5d2d44a726b147c P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.353246 10990 raft_consensus.cc:385] T 5c8a233bc6c44bd2b5d2d44a726b147c P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:49.353520 10990 raft_consensus.cc:740] T 5c8a233bc6c44bd2b5d2d44a726b147c P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:49.362864 10991 raft_consensus.cc:359] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.363610 10991 raft_consensus.cc:385] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:49.363878 10991 raft_consensus.cc:740] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:49.363965 10992 tablet_bootstrap.cc:492] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:49.364429 10992 ts_tablet_manager.cc:1403] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.036s	user 0.016s	sys 0.000s
I20260709 13:15:49.364573 10991 consensus_queue.cc:260] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.367216 10992 raft_consensus.cc:359] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.367851 10992 raft_consensus.cc:385] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:49.368124 10992 raft_consensus.cc:740] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:49.368846 10992 consensus_queue.cc:260] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.373692 10698 catalog_manager.cc:5697] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 0 to 1, leader changed from <none> to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:49.375190 10991 ts_tablet_manager.cc:1434] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.049s	user 0.005s	sys 0.002s
I20260709 13:15:49.376266 10991 tablet_bootstrap.cc:492] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:49.354235 10990 consensus_queue.cc:260] T 5c8a233bc6c44bd2b5d2d44a726b147c P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.382745 10990 ts_tablet_manager.cc:1434] T 5c8a233bc6c44bd2b5d2d44a726b147c P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.000s
I20260709 13:15:49.385583 10992 ts_tablet_manager.cc:1434] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.002s
I20260709 13:15:49.386689 10992 tablet_bootstrap.cc:492] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:49.387526 10991 tablet_bootstrap.cc:654] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:49.390990 10990 tablet_bootstrap.cc:492] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2: Bootstrap starting.
I20260709 13:15:49.392529 10992 tablet_bootstrap.cc:654] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:49.403532 10992 tablet_bootstrap.cc:492] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:49.404024 10992 ts_tablet_manager.cc:1403] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.018s	user 0.003s	sys 0.007s
I20260709 13:15:49.406805 10992 raft_consensus.cc:359] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.407569 10992 raft_consensus.cc:385] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:49.407876 10992 raft_consensus.cc:740] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:49.408632 10992 consensus_queue.cc:260] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.410867 10992 ts_tablet_manager.cc:1434] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:15:49.412416 10990 tablet_bootstrap.cc:654] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:49.417418 10991 tablet_bootstrap.cc:492] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:49.417910 10991 ts_tablet_manager.cc:1403] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.042s	user 0.007s	sys 0.002s
I20260709 13:15:49.420244 10992 tablet_bootstrap.cc:492] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:49.428351 10991 raft_consensus.cc:359] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.429343 10991 raft_consensus.cc:385] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:49.429553 10990 tablet_bootstrap.cc:492] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2: No bootstrap required, opened a new log
I20260709 13:15:49.430444 10990 ts_tablet_manager.cc:1403] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2: Time spent bootstrapping tablet: real 0.040s	user 0.010s	sys 0.002s
I20260709 13:15:49.433267 10990 raft_consensus.cc:359] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.434029 10990 raft_consensus.cc:385] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:49.434302 10990 raft_consensus.cc:740] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Initialized, Role: FOLLOWER
I20260709 13:15:49.435282 10990 consensus_queue.cc:260] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.436568 10992 tablet_bootstrap.cc:654] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:49.438184 10990 ts_tablet_manager.cc:1434] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:15:49.438614 10991 raft_consensus.cc:740] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:49.439821 10991 consensus_queue.cc:260] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.451320 10991 ts_tablet_manager.cc:1434] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.033s	user 0.008s	sys 0.000s
I20260709 13:15:49.454293 10991 tablet_bootstrap.cc:492] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:49.455118 11328 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:49.463310 10991 tablet_bootstrap.cc:654] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:49.463140 11328 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.465420 11328 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:49.466118 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:49.466813 10799 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 0.
I20260709 13:15:49.468961 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:49.468976 10916 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:15:49.458629 10992 tablet_bootstrap.cc:492] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:49.469792 10873 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 0.
I20260709 13:15:49.470065 10992 ts_tablet_manager.cc:1403] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.050s	user 0.013s	sys 0.006s
I20260709 13:15:49.470191 11328 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:49.471621 11328 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:49.471962 11328 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:49.473282 10992 raft_consensus.cc:359] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.474021 10992 raft_consensus.cc:385] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:49.474283 10992 raft_consensus.cc:740] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:49.475056 10992 consensus_queue.cc:260] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.477331 10992 ts_tablet_manager.cc:1434] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.004s
I20260709 13:15:49.478857 10992 tablet_bootstrap.cc:492] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:49.480326 11328 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.485689 10992 tablet_bootstrap.cc:654] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:49.486312 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:15:49.487015 10799 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:49.492578 11325 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:49.493335 11325 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.496949 11325 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:49.501806 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:49.503348 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:49.510159 10947 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 0.
I20260709 13:15:49.510334 10798 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 0.
I20260709 13:15:49.519907 10842 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:15:49.521685 11200 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:49.522114 11200 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:49.522475 11200 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:49.523610 11198 consensus_queue.cc:1048] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:49.538003 11356 consensus_queue.cc:1048] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:49.544957 11038 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:49.545450 11038 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.547418 11038 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:15:49.549063 10827 tablet.cc:2406] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:15:49.556012 11198 consensus_queue.cc:1048] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:49.559000 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:49.559679 10947 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 0.
I20260709 13:15:49.560848 10841 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:15:49.561606 11325 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:49.561975 11325 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:49.562397 11325 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:49.572758 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:15:49.579064 10842 leader_election.cc:343] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:49.583519 10799 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 1.
I20260709 13:15:49.586120 11325 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.590642 10916 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:15:49.591645 11333 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:49.592129 11333 raft_consensus.cc:697] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:15:49.593016 11333 consensus_queue.cc:237] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.594542 11328 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:49.598587 11213 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:49.599161 11213 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.599349 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:15:49.600229 10873 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:49.602993 11213 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:49.604373 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:49.611950 11328 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:49.620040 11328 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.623672 10991 tablet_bootstrap.cc:492] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:49.624882 10991 ts_tablet_manager.cc:1403] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.171s	user 0.010s	sys 0.004s
I20260709 13:15:49.628087 10991 raft_consensus.cc:359] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.634886 10992 tablet_bootstrap.cc:492] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:49.631107 11328 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:49.651400 10992 ts_tablet_manager.cc:1403] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.173s	user 0.058s	sys 0.040s
I20260709 13:15:49.637349 11198 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:49.652966 11198 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.638281 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:49.639456 11271 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:49.647225 11325 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:49.648387 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:49.651237 10991 raft_consensus.cc:385] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:49.658877 10991 raft_consensus.cc:740] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:49.657063 11200 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:15:49.657444 10916 leader_election.cc:343] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:49.657799 10873 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 1.
I20260709 13:15:49.657872 11271 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.661703 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:15:49.668696 11271 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:49.669497 11198 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:49.670861 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:49.671617 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:15:49.673555 10916 leader_election.cc:343] T dde624b07ed446e5b7347791196396e2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:49.685606 11325 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:49.686187 11325 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.685428 10698 catalog_manager.cc:5697] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 1 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:49.689038 11325 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:49.691251 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:15:49.692138 10947 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:15:49.696476 10916 leader_election.cc:343] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:49.696883 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:49.697440 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:49.698036 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:49.698378 10992 raft_consensus.cc:359] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:15:49.698693 10842 leader_election.cc:343] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:49.696942 10916 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:49.699169 10992 raft_consensus.cc:385] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:49.699502 10992 raft_consensus.cc:740] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:49.700127 11328 raft_consensus.cc:2749] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:49.700212 10992 consensus_queue.cc:260] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:15:49.701162 10842 leader_election.cc:343] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:49.702621 10992 ts_tablet_manager.cc:1434] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.051s	user 0.011s	sys 0.024s
I20260709 13:15:49.703886 11271 consensus_queue.cc:1048] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:49.708212 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:15:49.706997 10916 leader_election.cc:343] T dde624b07ed446e5b7347791196396e2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:49.709367 10800 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:49.709348 10916 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:15:49.708546 10842 leader_election.cc:343] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:49.708973 10991 consensus_queue.cc:260] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.712954 11380 raft_consensus.cc:2749] T dde624b07ed446e5b7347791196396e2 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:49.713449 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:49.714088 10944 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 0.
I20260709 13:15:49.707504 10767 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:15:49.698985 11200 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:49.717980 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:49.718302 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:15:49.718605 10948 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 0.
I20260709 13:15:49.718976 10942 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:49.717978 10800 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 1.
I20260709 13:15:49.723294 10841 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:49.723276 10842 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:15:49.725346 11271 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:49.729246 11356 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:49.729745 11356 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:49.730173 11356 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:49.731468 10947 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 1.
I20260709 13:15:49.724840 10841 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:49.735948 11038 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:49.736347 11038 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:49.736717 11038 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:49.735019 11271 raft_consensus.cc:697] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:15:49.746232 11383 raft_consensus.cc:2749] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:49.739571 11271 consensus_queue.cc:237] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.749563 10992 tablet_bootstrap.cc:492] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
W20260709 13:15:49.777181 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 52405.6us
I20260709 13:15:49.778168 10841 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:15:49.779191 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 71756.5us
I20260709 13:15:49.781836 11038 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.783520 11325 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:49.785658 11325 raft_consensus.cc:697] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:15:49.786825 11325 consensus_queue.cc:237] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.800824 10991 ts_tablet_manager.cc:1434] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.176s	user 0.015s	sys 0.008s
I20260709 13:15:49.811887 11038 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:49.830905 10992 tablet_bootstrap.cc:654] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
W20260709 13:15:49.832702 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 48256.2us
I20260709 13:15:49.841085 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:15:49.841723 10949 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:49.830525 11380 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:49.843024 11380 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.845050 11380 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:49.845549 11356 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:49.847795 10991 tablet_bootstrap.cc:492] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:49.852054 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:49.856011 10949 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 1.
I20260709 13:15:49.872677 10801 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:15:49.893985 10916 leader_election.cc:343] T 5c8a233bc6c44bd2b5d2d44a726b147c P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:49.936195 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:15:49.950176 10747 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:15:49.972990 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 51759.1us
I20260709 13:15:49.972548 11271 consensus_queue.cc:1048] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:15:49.969523 10942 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 1.
I20260709 13:15:49.913753 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:15:49.966686 11356 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:15:50.013593 10842 leader_election.cc:343] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:50.019922 10991 tablet_bootstrap.cc:654] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:50.020794 10841 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:15:50.023116 10698 catalog_manager.cc:5697] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 0 to 1, leader changed from <none> to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:50.025941 10992 tablet_bootstrap.cc:492] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:50.026564 10992 ts_tablet_manager.cc:1403] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.298s	user 0.010s	sys 0.013s
I20260709 13:15:50.029136 10992 raft_consensus.cc:359] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:50.029803 10992 raft_consensus.cc:385] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:50.030078 10992 raft_consensus.cc:740] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:50.031023 10992 consensus_queue.cc:260] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:50.030158 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:15:50.035362 10944 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:50.034945 10992 ts_tablet_manager.cc:1434] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:15:50.042327 11389 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:50.042949 11389 raft_consensus.cc:697] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:15:50.043766 11389 consensus_queue.cc:237] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:50.067122 10992 tablet_bootstrap.cc:492] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
W20260709 13:15:50.091571 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 52186.3us
W20260709 13:15:50.101665 10842 leader_election.cc:343] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:50.109900 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:15:50.148248 10916 leader_election.cc:343] T 5c8a233bc6c44bd2b5d2d44a726b147c P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:50.149191 10916 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:50.147231 10796 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:50.152251 11380 raft_consensus.cc:2749] T 5c8a233bc6c44bd2b5d2d44a726b147c P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:50.153184 10991 tablet_bootstrap.cc:492] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:50.153685 10991 ts_tablet_manager.cc:1403] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.306s	user 0.021s	sys 0.023s
I20260709 13:15:50.171612 10991 raft_consensus.cc:359] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:50.172540 10991 raft_consensus.cc:385] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:50.172906 10991 raft_consensus.cc:740] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:50.172993 11388 consensus_queue.cc:1048] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:50.173803 10991 consensus_queue.cc:260] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:50.176625 10991 ts_tablet_manager.cc:1434] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.023s	user 0.009s	sys 0.000s
I20260709 13:15:50.178179 10991 tablet_bootstrap.cc:492] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:50.181439 11380 consensus_queue.cc:1048] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 13:15:50.166625 10944 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 1.
I20260709 13:15:50.204478 10841 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:50.211098 11388 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:50.211896 11388 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:50.214200 11388 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:50.217557 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:50.217955 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:50.219458 10945 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 0.
I20260709 13:15:50.220963 10841 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:15:50.221791 11200 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:50.222198 11200 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:50.222566 11200 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:50.228940 11200 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:50.206575 11387 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:50.232224 11387 raft_consensus.cc:697] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:15:50.218406 10796 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 0.
I20260709 13:15:50.277825 10991 tablet_bootstrap.cc:654] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:50.267019 11387 consensus_queue.cc:237] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:50.278528 10697 catalog_manager.cc:5697] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 0 to 1, leader changed from <none> to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:50.302346 11356 consensus_queue.cc:1048] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:50.307540 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:15:50.313158 10992 tablet_bootstrap.cc:654] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
W20260709 13:15:50.314939 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 59392.4us
I20260709 13:15:50.315207 11200 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:50.280943 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:15:50.331593 10799 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:15:50.295107 11396 log.cc:927] Time spent T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
I20260709 13:15:50.329306 10697 catalog_manager.cc:5697] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 0 to 1, leader changed from <none> to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:50.344884 10942 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:50.381258 11394 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:50.394374 10799 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 1.
I20260709 13:15:50.381870 11394 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:15:50.407819 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 55313.3us
I20260709 13:15:50.403811 11356 consensus_queue.cc:1048] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 13:15:50.410755 10942 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 1.
I20260709 13:15:50.412765 10842 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:15:50.432395 10842 consensus_peers.cc:597] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:50.428814 11389 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:15:50.455029 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 85211.6us
I20260709 13:15:50.456097 10991 tablet_bootstrap.cc:492] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:50.467159 10991 ts_tablet_manager.cc:1403] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.289s	user 0.013s	sys 0.022s
I20260709 13:15:50.472570 10992 tablet_bootstrap.cc:492] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:50.473598 10992 ts_tablet_manager.cc:1403] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.412s	user 0.022s	sys 0.020s
I20260709 13:15:50.476967 11411 consensus_queue.cc:1048] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 13:15:50.481673 10991 raft_consensus.cc:359] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:50.482441 10991 raft_consensus.cc:385] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:50.482707 10991 raft_consensus.cc:740] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:50.500530 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:50.492738 11389 raft_consensus.cc:697] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:15:50.510169 10992 raft_consensus.cc:359] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:50.498225 11271 consensus_queue.cc:1048] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260709 13:15:50.512197 10992 raft_consensus.cc:385] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:50.514829 10992 raft_consensus.cc:740] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:50.515473 10992 consensus_queue.cc:260] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:50.502600 11394 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:50.532933 10992 ts_tablet_manager.cc:1434] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.059s	user 0.004s	sys 0.003s
I20260709 13:15:50.533108 10991 consensus_queue.cc:260] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:50.535789 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:50.536651 10948 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 0.
W20260709 13:15:50.549355 10842 leader_election.cc:343] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:50.550887 11389 consensus_queue.cc:237] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:50.559919 10991 ts_tablet_manager.cc:1434] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.092s	user 0.006s	sys 0.003s
I20260709 13:15:50.578418 10991 tablet_bootstrap.cc:492] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:50.581738 10841 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:50.584088 11388 raft_consensus.cc:2804] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:50.584512 11388 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:50.584848 11388 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:50.585249 10992 tablet_bootstrap.cc:492] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:50.665133 10992 tablet_bootstrap.cc:654] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:50.680630 10991 tablet_bootstrap.cc:654] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:50.653944 11388 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:15:50.685027 10842 consensus_peers.cc:597] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:50.695530 11408 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:50.696063 11408 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:50.700644 11408 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:50.705895 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:50.706735 10944 raft_consensus.cc:2393] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 1: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:15:50.708751 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:50.717324 10767 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:15:50.724251 11213 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:50.725018 11394 consensus_queue.cc:1048] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:50.743680 11388 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:50.745075 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:15:50.747195 10945 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:50.749750 11388 consensus_queue.cc:1048] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:50.757682 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:15:50.759735 11213 raft_consensus.cc:2749] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:15:50.761701 10842 leader_election.cc:343] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:50.764935 10945 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 1.
I20260709 13:15:50.768767 10841 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:50.770082 11038 raft_consensus.cc:2804] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:50.770926 11038 raft_consensus.cc:697] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:15:50.771921 11038 consensus_queue.cc:237] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:50.785017 10991 tablet_bootstrap.cc:492] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:50.785542 10992 tablet_bootstrap.cc:492] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:50.795574 10992 ts_tablet_manager.cc:1403] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.220s	user 0.013s	sys 0.010s
I20260709 13:15:50.800475 10992 raft_consensus.cc:359] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:50.801209 10992 raft_consensus.cc:385] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:50.801493 10992 raft_consensus.cc:740] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:50.802161 10992 consensus_queue.cc:260] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:50.804360 10992 ts_tablet_manager.cc:1434] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:15:50.805567 10992 tablet_bootstrap.cc:492] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:50.793123 10699 catalog_manager.cc:5697] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 0 to 1, leader changed from <none> to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:50.811018 10991 ts_tablet_manager.cc:1403] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.233s	user 0.014s	sys 0.004s
I20260709 13:15:50.812987 11394 consensus_queue.cc:1048] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:50.813603 10991 raft_consensus.cc:359] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:50.817775 10699 catalog_manager.cc:5697] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 0 to 1, leader changed from <none> to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:50.814275 10991 raft_consensus.cc:385] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:50.820632 10991 raft_consensus.cc:740] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:50.821373 10991 consensus_queue.cc:260] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:50.823522 10992 tablet_bootstrap.cc:654] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:50.830470 10991 ts_tablet_manager.cc:1434] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.019s	user 0.004s	sys 0.002s
I20260709 13:15:50.831707 10991 tablet_bootstrap.cc:492] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:50.841682 11325 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:50.842442 11325 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:50.859197 10991 tablet_bootstrap.cc:654] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:50.858541 10992 tablet_bootstrap.cc:492] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:50.861994 10992 ts_tablet_manager.cc:1403] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.057s	user 0.011s	sys 0.002s
I20260709 13:15:50.862897 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:50.863101 11325 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:15:50.864372 10842 leader_election.cc:343] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:50.865392 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:50.864552 10992 raft_consensus.cc:359] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:50.870736 10949 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 0.
I20260709 13:15:50.871884 10841 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:50.872927 11038 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:50.873281 11038 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:50.873585 11038 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:50.875427 10991 tablet_bootstrap.cc:492] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:50.876077 10991 ts_tablet_manager.cc:1403] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.045s	user 0.011s	sys 0.000s
I20260709 13:15:50.870718 10992 raft_consensus.cc:385] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:50.880394 11038 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:50.884016 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:15:50.885622 10842 leader_election.cc:343] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:50.885910 10991 raft_consensus.cc:359] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:50.886834 10991 raft_consensus.cc:385] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:50.887622 10991 raft_consensus.cc:740] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:50.887759 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:15:50.888342 10949 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:50.888664 10991 consensus_queue.cc:260] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:50.881148 10992 raft_consensus.cc:740] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:50.890533 10992 consensus_queue.cc:260] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:50.891541 10991 ts_tablet_manager.cc:1434] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:15:50.892834 10991 tablet_bootstrap.cc:492] T ad1b662471ff4b9ebda3845b4f224a34 P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:50.897421 10949 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 1.
I20260709 13:15:50.897513 11405 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:50.898223 11405 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:50.898594 10841 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:15:50.899379 11394 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:50.899724 11038 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 1 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:50.899832 11394 raft_consensus.cc:697] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:15:50.900671 11394 consensus_queue.cc:237] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:50.901109 11405 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:50.903525 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:50.904228 10992 ts_tablet_manager.cc:1434] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.042s	user 0.006s	sys 0.000s
I20260709 13:15:50.904281 10800 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 0.
I20260709 13:15:50.905303 10992 tablet_bootstrap.cc:492] T 416a6d12184f4fbda69570b95be47851 P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:50.905527 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:50.906280 10869 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 0.
I20260709 13:15:50.909596 10916 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:15:50.912274 11328 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:50.912703 11328 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:50.913035 11328 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:50.914381 10697 catalog_manager.cc:5697] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 0 to 1, leader changed from <none> to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:50.922613 10991 tablet_bootstrap.cc:654] T ad1b662471ff4b9ebda3845b4f224a34 P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:50.934324 11328 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:50.924127 10992 tablet_bootstrap.cc:654] T 416a6d12184f4fbda69570b95be47851 P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:50.946367 11328 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:50.947793 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:15:50.948802 10869 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:50.964208 11394 consensus_queue.cc:1048] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:50.965682 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:15:50.986845 10793 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:51.001451 10991 tablet_bootstrap.cc:492] T ad1b662471ff4b9ebda3845b4f224a34 P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:51.001989 10991 ts_tablet_manager.cc:1403] T ad1b662471ff4b9ebda3845b4f224a34 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.109s	user 0.012s	sys 0.006s
I20260709 13:15:51.004559 10992 tablet_bootstrap.cc:492] T 416a6d12184f4fbda69570b95be47851 P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:51.005095 10992 ts_tablet_manager.cc:1403] T 416a6d12184f4fbda69570b95be47851 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.100s	user 0.010s	sys 0.000s
I20260709 13:15:51.004815 10991 raft_consensus.cc:359] T ad1b662471ff4b9ebda3845b4f224a34 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:51.005548 10991 raft_consensus.cc:385] T ad1b662471ff4b9ebda3845b4f224a34 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:51.005888 10991 raft_consensus.cc:740] T ad1b662471ff4b9ebda3845b4f224a34 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:51.006562 10991 consensus_queue.cc:260] T ad1b662471ff4b9ebda3845b4f224a34 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:51.002745 10697 catalog_manager.cc:5697] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 0 to 1, leader changed from <none> to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:51.008879 10991 ts_tablet_manager.cc:1434] T ad1b662471ff4b9ebda3845b4f224a34 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:15:51.010650 10991 tablet_bootstrap.cc:492] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:51.011283 10869 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 1.
I20260709 13:15:51.016135 10992 raft_consensus.cc:359] T 416a6d12184f4fbda69570b95be47851 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:51.016906 10992 raft_consensus.cc:385] T 416a6d12184f4fbda69570b95be47851 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:51.017179 10992 raft_consensus.cc:740] T 416a6d12184f4fbda69570b95be47851 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:51.017838 10991 tablet_bootstrap.cc:654] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:51.019634 10916 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:15:51.024350 10793 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 1.
I20260709 13:15:51.028398 10992 consensus_queue.cc:260] T 416a6d12184f4fbda69570b95be47851 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:51.030992 11328 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:51.031741 11328 raft_consensus.cc:697] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:15:51.032804 11328 consensus_queue.cc:237] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:51.034380 10992 ts_tablet_manager.cc:1434] T 416a6d12184f4fbda69570b95be47851 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.029s	user 0.009s	sys 0.000s
I20260709 13:15:51.084298 11394 consensus_queue.cc:1048] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:51.089334 10699 catalog_manager.cc:5697] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 1 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:51.093406 11328 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:51.094169 11328 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:51.097939 10992 tablet_bootstrap.cc:492] T 19c20efd6c04443596bb712e3c93c650 P 468a8fd46e824f62b9c2c0d3532a6d26: Bootstrap starting.
I20260709 13:15:51.105152 11328 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:51.105343 11393 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:51.105846 11393 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:51.107077 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:51.107862 10870 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 0.
I20260709 13:15:51.109087 10916 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:15:51.109768 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:51.111078 11393 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:51.111600 11333 raft_consensus.cc:2804] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:51.116169 11333 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:51.116641 11333 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:51.120236 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:15:51.121649 10992 tablet_bootstrap.cc:654] T 19c20efd6c04443596bb712e3c93c650 P 468a8fd46e824f62b9c2c0d3532a6d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:51.123587 10991 tablet_bootstrap.cc:492] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:51.124140 10991 ts_tablet_manager.cc:1403] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.114s	user 0.016s	sys 0.037s
I20260709 13:15:51.128181 10991 raft_consensus.cc:359] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:51.129060 10991 raft_consensus.cc:385] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:51.129458 10991 raft_consensus.cc:740] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:51.130347 10991 consensus_queue.cc:260] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:51.131297 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:51.132854 10767 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:15:51.133872 11391 raft_consensus.cc:2749] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:51.135463 10991 ts_tablet_manager.cc:1434] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.003s
I20260709 13:15:51.138837 10991 tablet_bootstrap.cc:492] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:51.146025 10991 tablet_bootstrap.cc:654] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
W20260709 13:15:51.148339 10747 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:15:51.152719 10992 tablet_bootstrap.cc:492] T 19c20efd6c04443596bb712e3c93c650 P 468a8fd46e824f62b9c2c0d3532a6d26: No bootstrap required, opened a new log
I20260709 13:15:51.153236 10992 ts_tablet_manager.cc:1403] T 19c20efd6c04443596bb712e3c93c650 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent bootstrapping tablet: real 0.056s	user 0.005s	sys 0.012s
I20260709 13:15:51.155890 10992 raft_consensus.cc:359] T 19c20efd6c04443596bb712e3c93c650 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:51.157044 10992 raft_consensus.cc:385] T 19c20efd6c04443596bb712e3c93c650 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:51.157349 10992 raft_consensus.cc:740] T 19c20efd6c04443596bb712e3c93c650 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Initialized, Role: FOLLOWER
I20260709 13:15:51.157943 10992 consensus_queue.cc:260] T 19c20efd6c04443596bb712e3c93c650 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:51.159447 10991 tablet_bootstrap.cc:492] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:51.159969 10991 ts_tablet_manager.cc:1403] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.021s	user 0.007s	sys 0.011s
I20260709 13:15:51.163197 10991 raft_consensus.cc:359] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:51.163959 10991 raft_consensus.cc:385] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:51.164268 10991 raft_consensus.cc:740] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:51.165591 10991 consensus_queue.cc:260] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:51.166373 11333 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:51.168584 11333 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:51.169694 10992 ts_tablet_manager.cc:1434] T 19c20efd6c04443596bb712e3c93c650 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.001s
I20260709 13:15:51.169638 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:15:51.170378 10870 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:51.171265 10991 ts_tablet_manager.cc:1434] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.001s
I20260709 13:15:51.172405 10991 tablet_bootstrap.cc:492] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
W20260709 13:15:51.176008 10916 leader_election.cc:343] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:51.178047 10870 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 1.
I20260709 13:15:51.179522 10991 tablet_bootstrap.cc:654] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:51.179446 10916 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:15:51.185711 10991 tablet_bootstrap.cc:492] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:51.186506 10991 ts_tablet_manager.cc:1403] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.014s	user 0.011s	sys 0.001s
I20260709 13:15:51.187058 10797 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:51.189353 10991 raft_consensus.cc:359] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:51.190142 10991 raft_consensus.cc:385] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:51.190460 10991 raft_consensus.cc:740] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:51.191433 10991 consensus_queue.cc:260] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:51.193993 10991 ts_tablet_manager.cc:1434] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:15:51.194579 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:15:51.195235 10792 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:51.195410 10991 tablet_bootstrap.cc:492] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:51.195662 11328 raft_consensus.cc:2804] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:51.196198 11328 raft_consensus.cc:697] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:15:51.197118 11328 consensus_queue.cc:237] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:15:51.197695 10842 consensus_peers.cc:597] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:51.203012 10991 tablet_bootstrap.cc:654] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:51.204859 11393 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:51.205461 11393 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:51.216766 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:51.217183 11393 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:51.217463 10945 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 0.
I20260709 13:15:51.218355 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:51.218828 10766 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:15:51.219007 10870 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 0.
I20260709 13:15:51.219709 11393 raft_consensus.cc:2804] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:51.220077 11393 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:51.220391 11393 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:51.221451 10792 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 1.
I20260709 13:15:51.231554 11393 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:51.236682 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:15:51.237334 10870 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:51.250427 11393 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:51.245258 10870 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 1.
I20260709 13:15:51.251749 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:15:51.252274 10767 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:15:51.253337 11393 raft_consensus.cc:2804] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:51.257167 11393 raft_consensus.cc:697] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:15:51.252417 10945 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:51.258215 11393 consensus_queue.cc:237] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:51.250955 11391 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:51.263065 11391 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:51.266898 11391 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:51.267030 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:51.267853 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:51.270056 10949 raft_consensus.cc:2393] T dde624b07ed446e5b7347791196396e2 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 1: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:15:51.271193 10766 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:15:51.272096 11393 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:51.278478 10945 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 1.
I20260709 13:15:51.282716 10697 catalog_manager.cc:5697] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 1 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:51.317137 11393 raft_consensus.cc:2749] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:51.351449 11387 consensus_queue.cc:1048] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:51.353762 10991 tablet_bootstrap.cc:492] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:51.354379 10991 ts_tablet_manager.cc:1403] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.159s	user 0.028s	sys 0.041s
I20260709 13:15:51.359283 10991 raft_consensus.cc:359] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:51.361555 10991 raft_consensus.cc:385] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:51.361938 10991 raft_consensus.cc:740] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:51.362722 10991 consensus_queue.cc:260] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:51.365459 10991 ts_tablet_manager.cc:1434] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:15:51.366720 10991 tablet_bootstrap.cc:492] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87: Bootstrap starting.
I20260709 13:15:51.366608 10698 catalog_manager.cc:5697] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 0 to 1, leader changed from <none> to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 1 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:51.373399 10991 tablet_bootstrap.cc:654] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:51.394901 11449 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:51.395581 11449 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:51.396991 11387 consensus_queue.cc:1048] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:51.399284 11449 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:51.400791 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:51.403288 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:51.407270 10767 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:15:51.408416 11449 raft_consensus.cc:2749] T ad1b662471ff4b9ebda3845b4f224a34 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:51.412421 11387 consensus_queue.cc:1048] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:51.451891 10991 tablet_bootstrap.cc:492] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87: No bootstrap required, opened a new log
I20260709 13:15:51.452535 10991 ts_tablet_manager.cc:1403] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent bootstrapping tablet: real 0.086s	user 0.021s	sys 0.037s
I20260709 13:15:51.455585 10991 raft_consensus.cc:359] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:51.456554 10991 raft_consensus.cc:385] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:51.456909 10991 raft_consensus.cc:740] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Initialized, Role: FOLLOWER
I20260709 13:15:51.457744 10991 consensus_queue.cc:260] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:51.459275 11449 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:51.459755 11449 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:51.460796 10991 ts_tablet_manager.cc:1434] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:15:51.463752 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:15:51.464455 10944 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 0.
I20260709 13:15:51.465489 10766 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:15:51.466256 11393 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:51.466616 11393 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:51.467005 11393 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:51.469218 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:15:51.469904 10870 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 0.
I20260709 13:15:51.475425 11393 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:51.477811 11449 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:51.477561 11328 consensus_queue.cc:1048] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:51.480053 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:15:51.480727 10945 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:51.477561 11393 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:51.502574 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:15:51.507990 10870 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:51.521234 10870 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 1.
I20260709 13:15:51.490448 10945 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 1.
I20260709 13:15:51.523963 10767 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:15:51.525054 10797 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:51.527120 11449 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:51.527611 11449 raft_consensus.cc:697] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:15:51.528436 11449 consensus_queue.cc:237] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:15:51.551214 11328 consensus_queue.cc:1048] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:51.594208 10697 catalog_manager.cc:5697] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 0 to 1, leader changed from <none> to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 1 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:51.649487 10797 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:51.666061 11328 consensus_queue.cc:1048] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:51.684267 11411 consensus_queue.cc:1048] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:51.697005 11213 consensus_queue.cc:1048] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:51.743098 11391 consensus_queue.cc:1048] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:51.954129 11391 consensus_queue.cc:1048] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:51.968951 11391 consensus_queue.cc:1048] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:52.784492  9682 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:15:52.801750 11466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:52.809194 11467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:52.810566  9682 server_base.cc:1061] running on GCE node
W20260709 13:15:52.811553 11469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:52.812894  9682 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:15:52.813184  9682 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:15:52.813408  9682 hybrid_clock.cc:648] HybridClock initialized: now 1783602952813388 us; error 0 us; skew 500 ppm
I20260709 13:15:52.814163  9682 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:15:52.830889  9682 webserver.cc:533] Webserver started at http://127.9.116.132:42835/ using document root <none> and password file <none>
I20260709 13:15:52.831511  9682 fs_manager.cc:362] Metadata directory not provided
I20260709 13:15:52.831820  9682 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:15:52.832242  9682 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:15:52.833957  9682 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-3-root/instance:
uuid: "12682eb70bc84c76879e04a8628ee586"
format_stamp: "Formatted at 2026-07-09 13:15:52 on dist-test-slave-1t6m"
I20260709 13:15:52.846596  9682 fs_manager.cc:696] Time spent creating directory manager: real 0.012s	user 0.009s	sys 0.002s
I20260709 13:15:52.851632 11474 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:52.852658  9682 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:15:52.853000  9682 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-3-root
uuid: "12682eb70bc84c76879e04a8628ee586"
format_stamp: "Formatted at 2026-07-09 13:15:52 on dist-test-slave-1t6m"
I20260709 13:15:52.853368  9682 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:15:52.897770  9682 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:15:52.899255  9682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:15:52.900856  9682 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:15:52.906141  9682 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:15:52.906379  9682 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:52.906693  9682 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:15:52.906947  9682 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:15:53.032557  9682 rpc_server.cc:307] RPC server started. Bound to: 127.9.116.132:40555
I20260709 13:15:53.032696 11536 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.116.132:40555 every 8 connection(s)
I20260709 13:15:53.044785  9682 auto_leader_rebalancer-test.cc:384] add a tserver: 12682eb70bc84c76879e04a8628ee586
I20260709 13:15:53.060320 11537 heartbeater.cc:344] Connected to a master server at 127.9.116.190:36007
I20260709 13:15:53.060837 11537 heartbeater.cc:461] Registering TS with master...
I20260709 13:15:53.061606 11537 heartbeater.cc:507] Master 127.9.116.190:36007 requested a full tablet report, sending...
I20260709 13:15:53.063794 10699 ts_manager.cc:194] Registered new tserver with Master: 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:15:53.065733 10699 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42506
I20260709 13:15:53.332856 10945 consensus_queue.cc:237] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:15:53.343920 10870 raft_consensus.cc:1275] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:53.344826 10799 raft_consensus.cc:1275] T 51ae8de979e44a2d8c155bffc20a1533 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:53.347700 11405 consensus_queue.cc:1048] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:53.350724 11405 consensus_queue.cc:1048] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:53.363611 11465 raft_consensus.cc:2955] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:15:53.367779 10799 raft_consensus.cc:2955] T 51ae8de979e44a2d8c155bffc20a1533 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
W20260709 13:15:53.373528 10915 consensus_peers.cc:597] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 -> Peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Couldn't send request to peer 12682eb70bc84c76879e04a8628ee586. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 51ae8de979e44a2d8c155bffc20a1533. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:53.371788 10870 raft_consensus.cc:2955] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:15:53.415287 10699 catalog_manager.cc:5697] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New cstate: current_term: 1 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:15:53.443684 10870 consensus_queue.cc:237] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:15:53.456235 10949 raft_consensus.cc:1275] T dde624b07ed446e5b7347791196396e2 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:53.458097 11387 consensus_queue.cc:1048] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:53.462754 10797 raft_consensus.cc:1275] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:53.465448 11038 consensus_queue.cc:1048] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:53.503815 11557 raft_consensus.cc:2955] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
W20260709 13:15:53.510083 10841 consensus_peers.cc:597] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Couldn't send request to peer 12682eb70bc84c76879e04a8628ee586. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dde624b07ed446e5b7347791196396e2. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:53.513685 10949 raft_consensus.cc:2955] T dde624b07ed446e5b7347791196396e2 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:15:53.519649 10797 raft_consensus.cc:2955] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:15:53.527082 10698 catalog_manager.cc:5697] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:53.560606 10793 consensus_queue.cc:237] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:15:53.568848 10944 raft_consensus.cc:1275] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:53.570677 10870 raft_consensus.cc:1275] T 12d33f7a0f8f47e5ad57e854e52ae154 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:53.572849 11541 consensus_queue.cc:1048] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:53.573841 11391 consensus_queue.cc:1048] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:15:53.586352 10766 consensus_peers.cc:597] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 -> Peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Couldn't send request to peer 12682eb70bc84c76879e04a8628ee586. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 12d33f7a0f8f47e5ad57e854e52ae154. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:53.589051 11213 raft_consensus.cc:2955] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:15:53.594714 10870 raft_consensus.cc:2955] T 12d33f7a0f8f47e5ad57e854e52ae154 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:15:53.604589 10949 raft_consensus.cc:2955] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:15:53.609351 10698 catalog_manager.cc:5697] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New cstate: current_term: 1 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:53.614012 10944 consensus_queue.cc:237] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
W20260709 13:15:53.655385 10915 consensus_peers.cc:597] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 -> Peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Couldn't send request to peer 12682eb70bc84c76879e04a8628ee586. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9b4c7fc28a9b461abe203fe66da89aca. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:53.663748 10793 raft_consensus.cc:1275] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:53.665721 11411 consensus_queue.cc:1048] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:53.667450 10870 raft_consensus.cc:1275] T 9b4c7fc28a9b461abe203fe66da89aca P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:53.668819 11405 consensus_queue.cc:1048] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:53.680751 11465 raft_consensus.cc:2955] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:15:53.684120 10793 raft_consensus.cc:2955] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:15:53.698726 10869 raft_consensus.cc:2955] T 9b4c7fc28a9b461abe203fe66da89aca P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:15:53.719934 10698 catalog_manager.cc:5697] T 9b4c7fc28a9b461abe203fe66da89aca P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New cstate: current_term: 1 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:15:53.893900 11569 ts_tablet_manager.cc:933] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586: Initiating tablet copy from peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:53.909418 11569 tablet_copy_client.cc:323] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Beginning tablet copy session from remote peer at address 127.9.116.131:45553
I20260709 13:15:53.949328 10960 tablet_copy_service.cc:140] P cb0428cc55984daba1ba63744a5102a2: Received BeginTabletCopySession request for tablet 51ae8de979e44a2d8c155bffc20a1533 from peer 12682eb70bc84c76879e04a8628ee586 ({username='slave'} at 127.0.0.1:36714)
I20260709 13:15:53.950003 10960 tablet_copy_service.cc:161] P cb0428cc55984daba1ba63744a5102a2: Beginning new tablet copy session on tablet 51ae8de979e44a2d8c155bffc20a1533 from peer 12682eb70bc84c76879e04a8628ee586 at {username='slave'} at 127.0.0.1:36714: session id = 12682eb70bc84c76879e04a8628ee586-51ae8de979e44a2d8c155bffc20a1533
I20260709 13:15:53.960043 10960 tablet_copy_source_session.cc:215] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:53.963092 11573 ts_tablet_manager.cc:933] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586: Initiating tablet copy from peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:53.964175 11569 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51ae8de979e44a2d8c155bffc20a1533. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:54.015082 11569 tablet_copy_client.cc:806] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:54.015820 11569 tablet_copy_client.cc:670] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:54.020675 11569 tablet_copy_client.cc:538] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:54.031200 11573 tablet_copy_client.cc:323] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Beginning tablet copy session from remote peer at address 127.9.116.130:37895
I20260709 13:15:54.045842 11569 tablet_bootstrap.cc:492] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586: Bootstrap starting.
I20260709 13:15:54.062642 11576 ts_tablet_manager.cc:933] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586: Initiating tablet copy from peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685)
I20260709 13:15:54.068962 11537 heartbeater.cc:499] Master 127.9.116.190:36007 was elected leader, sending a full tablet report...
I20260709 13:15:54.083878 11576 tablet_copy_client.cc:323] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Beginning tablet copy session from remote peer at address 127.9.116.129:39685
I20260709 13:15:54.084954 10886 tablet_copy_service.cc:140] P 468a8fd46e824f62b9c2c0d3532a6d26: Received BeginTabletCopySession request for tablet dde624b07ed446e5b7347791196396e2 from peer 12682eb70bc84c76879e04a8628ee586 ({username='slave'} at 127.0.0.1:49290)
I20260709 13:15:54.085489 10886 tablet_copy_service.cc:161] P 468a8fd46e824f62b9c2c0d3532a6d26: Beginning new tablet copy session on tablet dde624b07ed446e5b7347791196396e2 from peer 12682eb70bc84c76879e04a8628ee586 at {username='slave'} at 127.0.0.1:49290: session id = 12682eb70bc84c76879e04a8628ee586-dde624b07ed446e5b7347791196396e2
I20260709 13:15:54.096587 10886 tablet_copy_source_session.cc:215] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:54.101037 11573 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dde624b07ed446e5b7347791196396e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:54.119866 10811 tablet_copy_service.cc:140] P e01ecd008edd4aa2b6b2b47efbc2ea87: Received BeginTabletCopySession request for tablet 12d33f7a0f8f47e5ad57e854e52ae154 from peer 12682eb70bc84c76879e04a8628ee586 ({username='slave'} at 127.0.0.1:39442)
I20260709 13:15:54.120450 10811 tablet_copy_service.cc:161] P e01ecd008edd4aa2b6b2b47efbc2ea87: Beginning new tablet copy session on tablet 12d33f7a0f8f47e5ad57e854e52ae154 from peer 12682eb70bc84c76879e04a8628ee586 at {username='slave'} at 127.0.0.1:39442: session id = 12682eb70bc84c76879e04a8628ee586-12d33f7a0f8f47e5ad57e854e52ae154
I20260709 13:15:54.127287 10811 tablet_copy_source_session.cc:215] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:54.130834 11576 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12d33f7a0f8f47e5ad57e854e52ae154. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:54.133889 11573 tablet_copy_client.cc:806] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:54.135262 11573 tablet_copy_client.cc:670] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:54.140059 11573 tablet_copy_client.cc:538] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:54.161680 11573 tablet_bootstrap.cc:492] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586: Bootstrap starting.
I20260709 13:15:54.172932 11580 ts_tablet_manager.cc:933] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586: Initiating tablet copy from peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:54.189540 11576 tablet_copy_client.cc:806] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:54.190268 11576 tablet_copy_client.cc:670] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:54.196659 11576 tablet_copy_client.cc:538] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:54.202543 11569 tablet_bootstrap.cc:492] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:54.204124 11569 tablet_bootstrap.cc:492] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586: Bootstrap complete.
I20260709 13:15:54.204960 11569 ts_tablet_manager.cc:1403] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586: Time spent bootstrapping tablet: real 0.159s	user 0.028s	sys 0.032s
I20260709 13:15:54.205799 11580 tablet_copy_client.cc:323] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586: tablet copy: Beginning tablet copy session from remote peer at address 127.9.116.131:45553
I20260709 13:15:54.206508 11576 tablet_bootstrap.cc:492] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586: Bootstrap starting.
I20260709 13:15:54.207906 10960 tablet_copy_service.cc:140] P cb0428cc55984daba1ba63744a5102a2: Received BeginTabletCopySession request for tablet 9b4c7fc28a9b461abe203fe66da89aca from peer 12682eb70bc84c76879e04a8628ee586 ({username='slave'} at 127.0.0.1:36714)
I20260709 13:15:54.208518 10960 tablet_copy_service.cc:161] P cb0428cc55984daba1ba63744a5102a2: Beginning new tablet copy session on tablet 9b4c7fc28a9b461abe203fe66da89aca from peer 12682eb70bc84c76879e04a8628ee586 at {username='slave'} at 127.0.0.1:36714: session id = 12682eb70bc84c76879e04a8628ee586-9b4c7fc28a9b461abe203fe66da89aca
I20260709 13:15:54.215364 11569 raft_consensus.cc:359] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:15:54.216107 11569 raft_consensus.cc:740] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Initialized, Role: LEARNER
I20260709 13:15:54.216595 11573 tablet_bootstrap.cc:492] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:54.216730 11569 consensus_queue.cc:260] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:15:54.217585 11573 tablet_bootstrap.cc:492] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586: Bootstrap complete.
I20260709 13:15:54.217711 10960 tablet_copy_source_session.cc:215] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:54.229207 11580 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b4c7fc28a9b461abe203fe66da89aca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:54.247125 11573 ts_tablet_manager.cc:1403] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586: Time spent bootstrapping tablet: real 0.086s	user 0.021s	sys 0.004s
I20260709 13:15:54.250396 11573 raft_consensus.cc:359] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:15:54.255935 11573 raft_consensus.cc:740] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Initialized, Role: LEARNER
I20260709 13:15:54.259208 11569 ts_tablet_manager.cc:1434] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586: Time spent starting tablet: real 0.053s	user 0.010s	sys 0.000s
I20260709 13:15:54.260392 11573 consensus_queue.cc:260] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:15:54.263965 10960 tablet_copy_service.cc:342] P cb0428cc55984daba1ba63744a5102a2: Request end of tablet copy session 12682eb70bc84c76879e04a8628ee586-51ae8de979e44a2d8c155bffc20a1533 received from {username='slave'} at 127.0.0.1:36714
I20260709 13:15:54.264515 10960 tablet_copy_service.cc:434] P cb0428cc55984daba1ba63744a5102a2: ending tablet copy session 12682eb70bc84c76879e04a8628ee586-51ae8de979e44a2d8c155bffc20a1533 on tablet 51ae8de979e44a2d8c155bffc20a1533 with peer 12682eb70bc84c76879e04a8628ee586
I20260709 13:15:54.270532 11580 tablet_copy_client.cc:806] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:54.271448 11580 tablet_copy_client.cc:670] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:54.286643 11580 tablet_copy_client.cc:538] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:54.298250 11580 tablet_bootstrap.cc:492] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586: Bootstrap starting.
I20260709 13:15:54.308319 11573 ts_tablet_manager.cc:1434] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586: Time spent starting tablet: real 0.061s	user 0.011s	sys 0.005s
I20260709 13:15:54.315910 10886 tablet_copy_service.cc:342] P 468a8fd46e824f62b9c2c0d3532a6d26: Request end of tablet copy session 12682eb70bc84c76879e04a8628ee586-dde624b07ed446e5b7347791196396e2 received from {username='slave'} at 127.0.0.1:49290
I20260709 13:15:54.316372 10886 tablet_copy_service.cc:434] P 468a8fd46e824f62b9c2c0d3532a6d26: ending tablet copy session 12682eb70bc84c76879e04a8628ee586-dde624b07ed446e5b7347791196396e2 on tablet dde624b07ed446e5b7347791196396e2 with peer 12682eb70bc84c76879e04a8628ee586
I20260709 13:15:54.321631 11576 tablet_bootstrap.cc:492] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:54.322561 11576 tablet_bootstrap.cc:492] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586: Bootstrap complete.
I20260709 13:15:54.323225 11576 ts_tablet_manager.cc:1403] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586: Time spent bootstrapping tablet: real 0.117s	user 0.014s	sys 0.024s
I20260709 13:15:54.326833 11576 raft_consensus.cc:359] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:15:54.329545 11576 raft_consensus.cc:740] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Initialized, Role: LEARNER
I20260709 13:15:54.330245 11576 consensus_queue.cc:260] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:15:54.333207 11576 ts_tablet_manager.cc:1434] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:15:54.335139 10811 tablet_copy_service.cc:342] P e01ecd008edd4aa2b6b2b47efbc2ea87: Request end of tablet copy session 12682eb70bc84c76879e04a8628ee586-12d33f7a0f8f47e5ad57e854e52ae154 received from {username='slave'} at 127.0.0.1:39442
I20260709 13:15:54.335623 10811 tablet_copy_service.cc:434] P e01ecd008edd4aa2b6b2b47efbc2ea87: ending tablet copy session 12682eb70bc84c76879e04a8628ee586-12d33f7a0f8f47e5ad57e854e52ae154 on tablet 12d33f7a0f8f47e5ad57e854e52ae154 with peer 12682eb70bc84c76879e04a8628ee586
I20260709 13:15:54.388103 11512 raft_consensus.cc:1217] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:15:54.389703 11580 tablet_bootstrap.cc:492] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:54.390895 11580 tablet_bootstrap.cc:492] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586: Bootstrap complete.
I20260709 13:15:54.391556 11580 ts_tablet_manager.cc:1403] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586: Time spent bootstrapping tablet: real 0.094s	user 0.024s	sys 0.022s
I20260709 13:15:54.394533 11580 raft_consensus.cc:359] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:15:54.399739 11580 raft_consensus.cc:740] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Initialized, Role: LEARNER
I20260709 13:15:54.400851 11580 consensus_queue.cc:260] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:15:54.404414 11580 ts_tablet_manager.cc:1434] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.001s
I20260709 13:15:54.407403 10960 tablet_copy_service.cc:342] P cb0428cc55984daba1ba63744a5102a2: Request end of tablet copy session 12682eb70bc84c76879e04a8628ee586-9b4c7fc28a9b461abe203fe66da89aca received from {username='slave'} at 127.0.0.1:36714
I20260709 13:15:54.407881 10960 tablet_copy_service.cc:434] P cb0428cc55984daba1ba63744a5102a2: ending tablet copy session 12682eb70bc84c76879e04a8628ee586-9b4c7fc28a9b461abe203fe66da89aca on tablet 9b4c7fc28a9b461abe203fe66da89aca with peer 12682eb70bc84c76879e04a8628ee586
I20260709 13:15:54.536742 11512 raft_consensus.cc:1217] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:15:54.550748 11512 raft_consensus.cc:1217] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:15:54.657960 11512 raft_consensus.cc:1217] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:15:54.751730 11411 raft_consensus.cc:1064] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2: attempting to promote NON_VOTER 12682eb70bc84c76879e04a8628ee586 to VOTER
I20260709 13:15:54.754722 11411 consensus_queue.cc:237] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:15:54.767750 11512 raft_consensus.cc:1275] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:54.769627 10797 raft_consensus.cc:1275] T 51ae8de979e44a2d8c155bffc20a1533 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:54.771193 10870 raft_consensus.cc:1275] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:54.772104 11587 consensus_queue.cc:1048] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:54.773587 11465 consensus_queue.cc:1048] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:54.774356 11587 consensus_queue.cc:1048] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:54.852624 11411 raft_consensus.cc:2955] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:54.871852 10793 raft_consensus.cc:2955] T 51ae8de979e44a2d8c155bffc20a1533 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:54.896643 10699 catalog_manager.cc:5697] T 51ae8de979e44a2d8c155bffc20a1533 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:54.919642 11512 raft_consensus.cc:2955] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
W20260709 13:15:54.933233 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 106160us
I20260709 13:15:54.959726 11391 raft_consensus.cc:1064] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87: attempting to promote NON_VOTER 12682eb70bc84c76879e04a8628ee586 to VOTER
I20260709 13:15:54.961984 11391 consensus_queue.cc:237] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:15:54.967347 10944 consensus_queue.cc:237] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:15:54.972935 10870 raft_consensus.cc:2955] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:54.983896 11511 raft_consensus.cc:1275] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:54.992924 11411 consensus_queue.cc:1048] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:55.001929 11465 raft_consensus.cc:2955] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:55.016831 10868 raft_consensus.cc:1275] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:55.026257 11411 consensus_queue.cc:1048] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:55.030489 10870 raft_consensus.cc:1275] T 12d33f7a0f8f47e5ad57e854e52ae154 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:55.031070 10944 raft_consensus.cc:1275] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:55.029994 11512 raft_consensus.cc:2955] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:55.032122 11391 consensus_queue.cc:1048] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:55.032975 11594 consensus_queue.cc:1048] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:55.036221 11511 raft_consensus.cc:1275] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:55.033162 10869 raft_consensus.cc:2955] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:55.037642 11595 consensus_queue.cc:1048] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:55.042385 10685 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 51ae8de979e44a2d8c155bffc20a1533 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:55.075383 11391 raft_consensus.cc:2955] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:55.077246 10869 raft_consensus.cc:2955] T 12d33f7a0f8f47e5ad57e854e52ae154 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:55.079614 11511 raft_consensus.cc:2955] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:55.083695 10699 catalog_manager.cc:5697] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: config changed from index 3 to 4, VOTER e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) evicted. New cstate: current_term: 1 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:55.099690 10785 tablet_service.cc:1558] Processing DeleteTablet for tablet 51ae8de979e44a2d8c155bffc20a1533 with delete_type TABLET_DATA_TOMBSTONED (TS e01ecd008edd4aa2b6b2b47efbc2ea87 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46752
I20260709 13:15:55.103861 10699 catalog_manager.cc:5697] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:55.104285 10950 raft_consensus.cc:2955] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:55.136073 11542 raft_consensus.cc:1064] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26: attempting to promote NON_VOTER 12682eb70bc84c76879e04a8628ee586 to VOTER
I20260709 13:15:55.137593 10799 consensus_queue.cc:237] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:15:55.138616 11542 consensus_queue.cc:237] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:15:55.134585 11604 tablet_replica.cc:333] T 51ae8de979e44a2d8c155bffc20a1533 P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:15:55.142987 11511 raft_consensus.cc:1275] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:55.143828 10950 raft_consensus.cc:1275] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:55.144913 11391 consensus_queue.cc:1048] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:55.146569 11594 consensus_queue.cc:1048] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:55.157163 11594 raft_consensus.cc:2955] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:55.162981 10948 raft_consensus.cc:1275] T dde624b07ed446e5b7347791196396e2 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:55.166896 11387 consensus_queue.cc:1048] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:55.167809 11511 raft_consensus.cc:2955] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:55.168267 10948 raft_consensus.cc:2955] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:55.194083 10686 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 12d33f7a0f8f47e5ad57e854e52ae154 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:55.196486 11411 raft_consensus.cc:1064] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2: attempting to promote NON_VOTER 12682eb70bc84c76879e04a8628ee586 to VOTER
I20260709 13:15:55.199177 11411 consensus_queue.cc:237] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:15:55.201179 10696 catalog_manager.cc:5697] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: config changed from index 3 to 4, VOTER 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) evicted. New cstate: current_term: 1 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:55.211372 10793 raft_consensus.cc:1275] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:55.213037 11512 raft_consensus.cc:1275] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:55.214646 11542 consensus_queue.cc:1048] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:55.224107 11606 consensus_queue.cc:1048] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 13:15:55.233265 11511 raft_consensus.cc:1275] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:55.234973 11405 consensus_queue.cc:1048] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:55.235399 10793 raft_consensus.cc:1275] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:55.236778 11405 consensus_queue.cc:1048] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:55.238557 10869 raft_consensus.cc:1275] T 9b4c7fc28a9b461abe203fe66da89aca P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:55.238947 11604 raft_consensus.cc:2243] T 51ae8de979e44a2d8c155bffc20a1533 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:55.239749 11604 raft_consensus.cc:2272] T 51ae8de979e44a2d8c155bffc20a1533 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:55.244052 11604 ts_tablet_manager.cc:1916] T 51ae8de979e44a2d8c155bffc20a1533 P e01ecd008edd4aa2b6b2b47efbc2ea87: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:55.249087 10863 tablet_service.cc:1558] Processing DeleteTablet for tablet 12d33f7a0f8f47e5ad57e854e52ae154 with delete_type TABLET_DATA_TOMBSTONED (TS 468a8fd46e824f62b9c2c0d3532a6d26 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54946
I20260709 13:15:55.250376 11587 consensus_queue.cc:1048] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:55.288329 11615 tablet_replica.cc:333] T 12d33f7a0f8f47e5ad57e854e52ae154 P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
W20260709 13:15:55.289013 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 67599.2us
I20260709 13:15:55.296183 11465 raft_consensus.cc:2955] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:55.290169 11615 raft_consensus.cc:2243] T 12d33f7a0f8f47e5ad57e854e52ae154 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:55.311786 11615 raft_consensus.cc:2272] T 12d33f7a0f8f47e5ad57e854e52ae154 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:55.318943 10699 catalog_manager.cc:5697] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 reported cstate change: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:15:55.330343 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 44962.1us
I20260709 13:15:55.330909 11615 ts_tablet_manager.cc:1916] T 12d33f7a0f8f47e5ad57e854e52ae154 P 468a8fd46e824f62b9c2c0d3532a6d26: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:55.339478 11612 raft_consensus.cc:2955] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:55.343410 11512 raft_consensus.cc:2955] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:55.355367 11510 raft_consensus.cc:2955] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:55.356551 10799 raft_consensus.cc:2955] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:55.358242 10868 raft_consensus.cc:2955] T 9b4c7fc28a9b461abe203fe66da89aca P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:55.361265 11604 ts_tablet_manager.cc:1929] T 51ae8de979e44a2d8c155bffc20a1533 P e01ecd008edd4aa2b6b2b47efbc2ea87: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:15:55.385329 11604 log.cc:1199] T 51ae8de979e44a2d8c155bffc20a1533 P e01ecd008edd4aa2b6b2b47efbc2ea87: Deleting WAL directory at /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-0-root/wals/51ae8de979e44a2d8c155bffc20a1533
I20260709 13:15:55.352463 10793 raft_consensus.cc:2955] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:55.387777 10686 catalog_manager.cc:5028] TS e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): tablet 51ae8de979e44a2d8c155bffc20a1533 (table test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]) successfully deleted
I20260709 13:15:55.371124 10948 consensus_queue.cc:237] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:15:55.429230 11510 raft_consensus.cc:1275] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:55.431526 11587 consensus_queue.cc:1048] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:55.437692 10944 raft_consensus.cc:2955] T dde624b07ed446e5b7347791196396e2 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:55.450467 10794 raft_consensus.cc:1275] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:55.460156 10696 catalog_manager.cc:5697] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:55.462001 11465 raft_consensus.cc:2955] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:55.465452 11587 consensus_queue.cc:1048] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:55.472757 10685 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9b4c7fc28a9b461abe203fe66da89aca with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:55.469807 10794 raft_consensus.cc:2955] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:55.461066 11512 raft_consensus.cc:2955] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:55.495874 10870 consensus_queue.cc:237] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:15:55.512948 11510 raft_consensus.cc:1275] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:55.512514 10698 catalog_manager.cc:5697] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 reported cstate change: config changed from index 3 to 4, VOTER 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) evicted. New cstate: current_term: 1 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:55.514389 10794 raft_consensus.cc:1275] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:55.520102 11612 consensus_queue.cc:1048] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:55.529174 11612 consensus_queue.cc:1048] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:55.531773 10863 tablet_service.cc:1558] Processing DeleteTablet for tablet 9b4c7fc28a9b461abe203fe66da89aca with delete_type TABLET_DATA_TOMBSTONED (TS 468a8fd46e824f62b9c2c0d3532a6d26 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54946
I20260709 13:15:55.533689 11615 ts_tablet_manager.cc:1929] T 12d33f7a0f8f47e5ad57e854e52ae154 P 468a8fd46e824f62b9c2c0d3532a6d26: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:15:55.534436 11611 raft_consensus.cc:2955] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:55.538101 10794 raft_consensus.cc:2955] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:55.551165 11510 raft_consensus.cc:2955] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:55.544956 11615 log.cc:1199] T 12d33f7a0f8f47e5ad57e854e52ae154 P 468a8fd46e824f62b9c2c0d3532a6d26: Deleting WAL directory at /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-1-root/wals/12d33f7a0f8f47e5ad57e854e52ae154
I20260709 13:15:55.555749 11615 tablet_replica.cc:333] T 9b4c7fc28a9b461abe203fe66da89aca P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:15:55.555704 10686 catalog_manager.cc:5028] TS 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): tablet 12d33f7a0f8f47e5ad57e854e52ae154 (table test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]) successfully deleted
I20260709 13:15:55.556895 11615 raft_consensus.cc:2243] T 9b4c7fc28a9b461abe203fe66da89aca P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:55.557690 11615 raft_consensus.cc:2272] T 9b4c7fc28a9b461abe203fe66da89aca P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:55.571316 10686 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet dde624b07ed446e5b7347791196396e2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:55.574221 10699 catalog_manager.cc:5697] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: config changed from index 3 to 4, VOTER cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) evicted. New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:55.575867 11615 ts_tablet_manager.cc:1916] T 9b4c7fc28a9b461abe203fe66da89aca P 468a8fd46e824f62b9c2c0d3532a6d26: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:55.591918 10937 tablet_service.cc:1558] Processing DeleteTablet for tablet dde624b07ed446e5b7347791196396e2 with delete_type TABLET_DATA_TOMBSTONED (TS cb0428cc55984daba1ba63744a5102a2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41006
I20260709 13:15:55.613036 11623 tablet_replica.cc:333] T dde624b07ed446e5b7347791196396e2 P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:15:55.617795 11615 ts_tablet_manager.cc:1929] T 9b4c7fc28a9b461abe203fe66da89aca P 468a8fd46e824f62b9c2c0d3532a6d26: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:15:55.618480 11615 log.cc:1199] T 9b4c7fc28a9b461abe203fe66da89aca P 468a8fd46e824f62b9c2c0d3532a6d26: Deleting WAL directory at /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-1-root/wals/9b4c7fc28a9b461abe203fe66da89aca
I20260709 13:15:55.620483 10686 catalog_manager.cc:5028] TS 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): tablet 9b4c7fc28a9b461abe203fe66da89aca (table test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]) successfully deleted
I20260709 13:15:55.621325 11623 raft_consensus.cc:2243] T dde624b07ed446e5b7347791196396e2 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:55.622264 11623 raft_consensus.cc:2272] T dde624b07ed446e5b7347791196396e2 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:55.627255 11623 ts_tablet_manager.cc:1916] T dde624b07ed446e5b7347791196396e2 P cb0428cc55984daba1ba63744a5102a2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:55.642584 11623 ts_tablet_manager.cc:1929] T dde624b07ed446e5b7347791196396e2 P cb0428cc55984daba1ba63744a5102a2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:15:55.643011 11623 log.cc:1199] T dde624b07ed446e5b7347791196396e2 P cb0428cc55984daba1ba63744a5102a2: Deleting WAL directory at /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-2-root/wals/dde624b07ed446e5b7347791196396e2
I20260709 13:15:55.644585 10685 catalog_manager.cc:5028] TS cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): tablet dde624b07ed446e5b7347791196396e2 (table test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]) successfully deleted
I20260709 13:15:56.688017 10870 consensus_queue.cc:237] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:15:56.697489 10799 raft_consensus.cc:1275] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:15:56.697887 10841 consensus_peers.cc:597] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Couldn't send request to peer 12682eb70bc84c76879e04a8628ee586. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 73af6c7e9b8e4ffaa65ba65a6199eb61. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:56.698474 10944 raft_consensus.cc:1275] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:56.699450 11612 consensus_queue.cc:1048] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:56.700366 11611 consensus_queue.cc:1048] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:56.709180 11611 raft_consensus.cc:2955] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:15:56.724434 10794 raft_consensus.cc:2955] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:15:56.730219 10950 raft_consensus.cc:2955] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:15:56.748032 10696 catalog_manager.cc:5697] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:56.754693 10944 consensus_queue.cc:237] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:15:56.762967 10870 raft_consensus.cc:1275] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:15:56.764840 10915 consensus_peers.cc:597] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 -> Peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Couldn't send request to peer 12682eb70bc84c76879e04a8628ee586. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b81df256f0c644cfae9fe2d336cbe9bd. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:56.766613 11587 consensus_queue.cc:1048] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:56.767202 10794 raft_consensus.cc:1275] T b81df256f0c644cfae9fe2d336cbe9bd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:56.768599 11411 consensus_queue.cc:1048] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:56.780246 11587 raft_consensus.cc:2955] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:15:56.782594 10870 raft_consensus.cc:2955] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:15:56.785909 10794 raft_consensus.cc:2955] T b81df256f0c644cfae9fe2d336cbe9bd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:15:56.796897 10944 consensus_queue.cc:237] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:15:56.817204 10699 catalog_manager.cc:5697] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 reported cstate change: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New cstate: current_term: 1 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:15:56.832507 10915 consensus_peers.cc:597] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 -> Peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Couldn't send request to peer 12682eb70bc84c76879e04a8628ee586. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b4b13050d714473bb01ea0174d634a14. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:56.833314 10870 raft_consensus.cc:1275] T b4b13050d714473bb01ea0174d634a14 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:56.834830 10799 raft_consensus.cc:1275] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:56.834901 11587 consensus_queue.cc:1048] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:56.838928 11587 consensus_queue.cc:1048] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:56.847534 11587 raft_consensus.cc:2955] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:15:56.866463 10799 consensus_queue.cc:237] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
W20260709 13:15:56.895395 10766 consensus_peers.cc:597] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 -> Peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Couldn't send request to peer 12682eb70bc84c76879e04a8628ee586. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e9764b93d3a8407b8cc1ae641620c831. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:56.925874 10868 raft_consensus.cc:1275] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:56.927238 10950 raft_consensus.cc:1275] T e9764b93d3a8407b8cc1ae641620c831 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:56.927512 11594 consensus_queue.cc:1048] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:56.932508 11594 consensus_queue.cc:1048] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 13:15:56.971536 11595 raft_consensus.cc:2955] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:15:56.983719 10868 raft_consensus.cc:2955] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:15:56.990149 10944 raft_consensus.cc:2955] T e9764b93d3a8407b8cc1ae641620c831 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
W20260709 13:15:56.991954 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 144694us
I20260709 13:15:57.003582 10698 catalog_manager.cc:5697] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New cstate: current_term: 1 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:57.011823 10868 raft_consensus.cc:2955] T b4b13050d714473bb01ea0174d634a14 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:15:57.005376 10696 catalog_manager.cc:5697] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New cstate: current_term: 1 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:57.012951 10797 raft_consensus.cc:2955] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:15:57.157150 11653 ts_tablet_manager.cc:933] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586: Initiating tablet copy from peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:15:57.182235 11653 tablet_copy_client.cc:323] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Beginning tablet copy session from remote peer at address 127.9.116.130:37895
I20260709 13:15:57.186400 10886 tablet_copy_service.cc:140] P 468a8fd46e824f62b9c2c0d3532a6d26: Received BeginTabletCopySession request for tablet 73af6c7e9b8e4ffaa65ba65a6199eb61 from peer 12682eb70bc84c76879e04a8628ee586 ({username='slave'} at 127.0.0.1:49290)
I20260709 13:15:57.187284 10886 tablet_copy_service.cc:161] P 468a8fd46e824f62b9c2c0d3532a6d26: Beginning new tablet copy session on tablet 73af6c7e9b8e4ffaa65ba65a6199eb61 from peer 12682eb70bc84c76879e04a8628ee586 at {username='slave'} at 127.0.0.1:49290: session id = 12682eb70bc84c76879e04a8628ee586-73af6c7e9b8e4ffaa65ba65a6199eb61
I20260709 13:15:57.205678 10886 tablet_copy_source_session.cc:215] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:57.210186 11653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73af6c7e9b8e4ffaa65ba65a6199eb61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.272894 11655 ts_tablet_manager.cc:933] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586: Initiating tablet copy from peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:57.300275 11653 tablet_copy_client.cc:806] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:57.303226 11653 tablet_copy_client.cc:670] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:57.307341 11657 ts_tablet_manager.cc:933] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586: Initiating tablet copy from peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:15:57.313964 11658 ts_tablet_manager.cc:933] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586: Initiating tablet copy from peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685)
I20260709 13:15:57.323189 11658 tablet_copy_client.cc:323] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Beginning tablet copy session from remote peer at address 127.9.116.129:39685
I20260709 13:15:57.326097 10811 tablet_copy_service.cc:140] P e01ecd008edd4aa2b6b2b47efbc2ea87: Received BeginTabletCopySession request for tablet e9764b93d3a8407b8cc1ae641620c831 from peer 12682eb70bc84c76879e04a8628ee586 ({username='slave'} at 127.0.0.1:39442)
I20260709 13:15:57.327024 10811 tablet_copy_service.cc:161] P e01ecd008edd4aa2b6b2b47efbc2ea87: Beginning new tablet copy session on tablet e9764b93d3a8407b8cc1ae641620c831 from peer 12682eb70bc84c76879e04a8628ee586 at {username='slave'} at 127.0.0.1:39442: session id = 12682eb70bc84c76879e04a8628ee586-e9764b93d3a8407b8cc1ae641620c831
I20260709 13:15:57.337407 11653 tablet_copy_client.cc:538] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:57.340637 10811 tablet_copy_source_session.cc:215] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:57.345177 11658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9764b93d3a8407b8cc1ae641620c831. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.353456 11655 tablet_copy_client.cc:323] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586: tablet copy: Beginning tablet copy session from remote peer at address 127.9.116.131:45553
I20260709 13:15:57.356156 11657 tablet_copy_client.cc:323] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Beginning tablet copy session from remote peer at address 127.9.116.131:45553
I20260709 13:15:57.356392 10960 tablet_copy_service.cc:140] P cb0428cc55984daba1ba63744a5102a2: Received BeginTabletCopySession request for tablet b81df256f0c644cfae9fe2d336cbe9bd from peer 12682eb70bc84c76879e04a8628ee586 ({username='slave'} at 127.0.0.1:36714)
I20260709 13:15:57.357108 10960 tablet_copy_service.cc:161] P cb0428cc55984daba1ba63744a5102a2: Beginning new tablet copy session on tablet b81df256f0c644cfae9fe2d336cbe9bd from peer 12682eb70bc84c76879e04a8628ee586 at {username='slave'} at 127.0.0.1:36714: session id = 12682eb70bc84c76879e04a8628ee586-b81df256f0c644cfae9fe2d336cbe9bd
I20260709 13:15:57.363747 10959 tablet_copy_service.cc:140] P cb0428cc55984daba1ba63744a5102a2: Received BeginTabletCopySession request for tablet b4b13050d714473bb01ea0174d634a14 from peer 12682eb70bc84c76879e04a8628ee586 ({username='slave'} at 127.0.0.1:36714)
I20260709 13:15:57.364377 10959 tablet_copy_service.cc:161] P cb0428cc55984daba1ba63744a5102a2: Beginning new tablet copy session on tablet b4b13050d714473bb01ea0174d634a14 from peer 12682eb70bc84c76879e04a8628ee586 at {username='slave'} at 127.0.0.1:36714: session id = 12682eb70bc84c76879e04a8628ee586-b4b13050d714473bb01ea0174d634a14
I20260709 13:15:57.383292 11653 tablet_bootstrap.cc:492] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586: Bootstrap starting.
I20260709 13:15:57.384117 10960 tablet_copy_source_session.cc:215] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:57.387570 10959 tablet_copy_source_session.cc:215] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:57.393622 11657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4b13050d714473bb01ea0174d634a14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.401962 11655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b81df256f0c644cfae9fe2d336cbe9bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.418823 11657 tablet_copy_client.cc:806] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:57.419752 11657 tablet_copy_client.cc:670] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:57.424005 11658 tablet_copy_client.cc:806] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:57.424468 11657 tablet_copy_client.cc:538] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:57.427119 11658 tablet_copy_client.cc:670] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:57.436156 11658 tablet_copy_client.cc:538] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:57.449810 11653 tablet_bootstrap.cc:492] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:57.452775 11653 tablet_bootstrap.cc:492] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586: Bootstrap complete.
I20260709 13:15:57.453768 11653 ts_tablet_manager.cc:1403] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586: Time spent bootstrapping tablet: real 0.073s	user 0.018s	sys 0.014s
I20260709 13:15:57.457880 11653 raft_consensus.cc:359] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:15:57.459239 11658 tablet_bootstrap.cc:492] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586: Bootstrap starting.
I20260709 13:15:57.459367 11653 raft_consensus.cc:740] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Initialized, Role: LEARNER
I20260709 13:15:57.461246 11653 consensus_queue.cc:260] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:15:57.459924 11655 tablet_copy_client.cc:806] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:57.465185 11657 tablet_bootstrap.cc:492] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586: Bootstrap starting.
I20260709 13:15:57.471922 11655 tablet_copy_client.cc:670] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:57.474207 11653 ts_tablet_manager.cc:1434] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.004s
I20260709 13:15:57.481503 10886 tablet_copy_service.cc:342] P 468a8fd46e824f62b9c2c0d3532a6d26: Request end of tablet copy session 12682eb70bc84c76879e04a8628ee586-73af6c7e9b8e4ffaa65ba65a6199eb61 received from {username='slave'} at 127.0.0.1:49290
I20260709 13:15:57.481863 11655 tablet_copy_client.cc:538] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:57.482030 10886 tablet_copy_service.cc:434] P 468a8fd46e824f62b9c2c0d3532a6d26: ending tablet copy session 12682eb70bc84c76879e04a8628ee586-73af6c7e9b8e4ffaa65ba65a6199eb61 on tablet 73af6c7e9b8e4ffaa65ba65a6199eb61 with peer 12682eb70bc84c76879e04a8628ee586
I20260709 13:15:57.516185 11655 tablet_bootstrap.cc:492] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586: Bootstrap starting.
I20260709 13:15:57.582501 11512 raft_consensus.cc:1217] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:15:57.641763 11657 tablet_bootstrap.cc:492] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:57.642885 11657 tablet_bootstrap.cc:492] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586: Bootstrap complete.
I20260709 13:15:57.643699 11657 ts_tablet_manager.cc:1403] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586: Time spent bootstrapping tablet: real 0.179s	user 0.022s	sys 0.004s
I20260709 13:15:57.647665 11657 raft_consensus.cc:359] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:15:57.648756 11657 raft_consensus.cc:740] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Initialized, Role: LEARNER
I20260709 13:15:57.649523 11657 consensus_queue.cc:260] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:15:57.653234 11657 ts_tablet_manager.cc:1434] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:15:57.674093 10959 tablet_copy_service.cc:342] P cb0428cc55984daba1ba63744a5102a2: Request end of tablet copy session 12682eb70bc84c76879e04a8628ee586-b4b13050d714473bb01ea0174d634a14 received from {username='slave'} at 127.0.0.1:36714
I20260709 13:15:57.674592 10959 tablet_copy_service.cc:434] P cb0428cc55984daba1ba63744a5102a2: ending tablet copy session 12682eb70bc84c76879e04a8628ee586-b4b13050d714473bb01ea0174d634a14 on tablet b4b13050d714473bb01ea0174d634a14 with peer 12682eb70bc84c76879e04a8628ee586
I20260709 13:15:57.696813 11658 tablet_bootstrap.cc:492] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:57.711519 11655 tablet_bootstrap.cc:492] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:57.716023 11658 tablet_bootstrap.cc:492] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586: Bootstrap complete.
I20260709 13:15:57.717159 11655 tablet_bootstrap.cc:492] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586: Bootstrap complete.
I20260709 13:15:57.739331 11658 ts_tablet_manager.cc:1403] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586: Time spent bootstrapping tablet: real 0.287s	user 0.041s	sys 0.021s
I20260709 13:15:57.743142 11658 raft_consensus.cc:359] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:15:57.745501 11658 raft_consensus.cc:740] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Initialized, Role: LEARNER
I20260709 13:15:57.746352 11658 consensus_queue.cc:260] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:15:57.749542 11658 ts_tablet_manager.cc:1434] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:15:57.752625 10811 tablet_copy_service.cc:342] P e01ecd008edd4aa2b6b2b47efbc2ea87: Request end of tablet copy session 12682eb70bc84c76879e04a8628ee586-e9764b93d3a8407b8cc1ae641620c831 received from {username='slave'} at 127.0.0.1:39442
I20260709 13:15:57.753109 10811 tablet_copy_service.cc:434] P e01ecd008edd4aa2b6b2b47efbc2ea87: ending tablet copy session 12682eb70bc84c76879e04a8628ee586-e9764b93d3a8407b8cc1ae641620c831 on tablet e9764b93d3a8407b8cc1ae641620c831 with peer 12682eb70bc84c76879e04a8628ee586
I20260709 13:15:57.753243 11655 ts_tablet_manager.cc:1403] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586: Time spent bootstrapping tablet: real 0.237s	user 0.020s	sys 0.053s
I20260709 13:15:57.756866 11655 raft_consensus.cc:359] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:15:57.757829 11655 raft_consensus.cc:740] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Initialized, Role: LEARNER
I20260709 13:15:57.758555 11655 consensus_queue.cc:260] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:15:57.762806 11655 ts_tablet_manager.cc:1434] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.002s
I20260709 13:15:57.764554 10959 tablet_copy_service.cc:342] P cb0428cc55984daba1ba63744a5102a2: Request end of tablet copy session 12682eb70bc84c76879e04a8628ee586-b81df256f0c644cfae9fe2d336cbe9bd received from {username='slave'} at 127.0.0.1:36714
I20260709 13:15:57.765036 10959 tablet_copy_service.cc:434] P cb0428cc55984daba1ba63744a5102a2: ending tablet copy session 12682eb70bc84c76879e04a8628ee586-b81df256f0c644cfae9fe2d336cbe9bd on tablet b81df256f0c644cfae9fe2d336cbe9bd with peer 12682eb70bc84c76879e04a8628ee586
I20260709 13:15:57.782141 11512 raft_consensus.cc:1217] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:15:58.121965 11512 raft_consensus.cc:1217] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:15:58.156859 11626 raft_consensus.cc:1064] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26: attempting to promote NON_VOTER 12682eb70bc84c76879e04a8628ee586 to VOTER
I20260709 13:15:58.158948 11626 consensus_queue.cc:237] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:15:58.167595 10950 raft_consensus.cc:1275] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:58.168051 10794 raft_consensus.cc:1275] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:58.170228 11512 raft_consensus.cc:1275] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:58.170316 11626 consensus_queue.cc:1048] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:58.175602 11624 consensus_queue.cc:1048] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:58.177918 11611 consensus_queue.cc:1048] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:58.207507 11618 raft_consensus.cc:2955] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:58.210230 11512 raft_consensus.cc:2955] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:58.228045 10948 raft_consensus.cc:2955] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:58.241595 10696 catalog_manager.cc:5697] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:58.253363 10800 raft_consensus.cc:2955] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:58.262375 11512 raft_consensus.cc:1217] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 13:15:58.331493 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 66898.6us
I20260709 13:15:58.345528 11643 raft_consensus.cc:1064] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2: attempting to promote NON_VOTER 12682eb70bc84c76879e04a8628ee586 to VOTER
I20260709 13:15:58.348263 11643 consensus_queue.cc:237] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:15:58.363852 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 96857.3us
I20260709 13:15:58.379626 10872 raft_consensus.cc:1275] T b4b13050d714473bb01ea0174d634a14 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:58.382663 11512 raft_consensus.cc:1275] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:58.384248 11680 consensus_queue.cc:1048] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:58.386229 10871 consensus_queue.cc:237] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:15:58.394559 10793 raft_consensus.cc:1275] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:58.396298 11673 consensus_queue.cc:1048] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:58.406900 11679 consensus_queue.cc:1048] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:58.414747 11512 raft_consensus.cc:1275] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:58.416471 11611 consensus_queue.cc:1048] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:58.425603 10794 raft_consensus.cc:1275] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:58.427737 11611 raft_consensus.cc:2955] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:58.439023 11624 consensus_queue.cc:1048] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:15:58.445186 11510 raft_consensus.cc:2955] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:58.450371 10794 raft_consensus.cc:2955] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:58.470093 11672 raft_consensus.cc:2955] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:58.472198 10871 raft_consensus.cc:2955] T b4b13050d714473bb01ea0174d634a14 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:58.474334 10793 raft_consensus.cc:2955] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:58.475049 10697 catalog_manager.cc:5697] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: config changed from index 3 to 4, VOTER cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) evicted. New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:58.492556 11512 raft_consensus.cc:2955] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:58.494658 10937 tablet_service.cc:1558] Processing DeleteTablet for tablet 73af6c7e9b8e4ffaa65ba65a6199eb61 with delete_type TABLET_DATA_TOMBSTONED (TS cb0428cc55984daba1ba63744a5102a2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41006
I20260709 13:15:58.513471 10686 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 73af6c7e9b8e4ffaa65ba65a6199eb61 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:58.522301 10696 catalog_manager.cc:5697] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:58.535758 11684 tablet_replica.cc:333] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:15:58.542651 10944 consensus_queue.cc:237] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:15:58.549132 11684 raft_consensus.cc:2243] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:58.550009 11684 raft_consensus.cc:2272] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:58.550410 11512 raft_consensus.cc:1275] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:58.551823 11652 raft_consensus.cc:1064] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87: attempting to promote NON_VOTER 12682eb70bc84c76879e04a8628ee586 to VOTER
I20260709 13:15:58.554479 11679 consensus_queue.cc:1048] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:58.557296 10793 raft_consensus.cc:1275] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:58.561295 11679 consensus_queue.cc:1048] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:58.558553 11652 consensus_queue.cc:237] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:15:58.565898 11680 raft_consensus.cc:2955] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:58.570746 11684 ts_tablet_manager.cc:1916] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P cb0428cc55984daba1ba63744a5102a2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:58.577848 11510 raft_consensus.cc:2955] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:58.587002 10793 raft_consensus.cc:2955] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:58.598920 10685 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b4b13050d714473bb01ea0174d634a14 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:58.607676 10871 raft_consensus.cc:1275] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:58.608961 10697 catalog_manager.cc:5697] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: config changed from index 3 to 4, VOTER 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) evicted. New cstate: current_term: 1 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:58.612991 10948 raft_consensus.cc:1275] T e9764b93d3a8407b8cc1ae641620c831 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:58.610600 11652 consensus_queue.cc:1048] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 13:15:58.614836 11512 raft_consensus.cc:1275] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:58.619585 11595 consensus_queue.cc:1048] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:58.621990 11684 ts_tablet_manager.cc:1929] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P cb0428cc55984daba1ba63744a5102a2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:15:58.622541 11684 log.cc:1199] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P cb0428cc55984daba1ba63744a5102a2: Deleting WAL directory at /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-2-root/wals/73af6c7e9b8e4ffaa65ba65a6199eb61
I20260709 13:15:58.623180 11652 consensus_queue.cc:1048] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:58.626020 10863 tablet_service.cc:1558] Processing DeleteTablet for tablet b4b13050d714473bb01ea0174d634a14 with delete_type TABLET_DATA_TOMBSTONED (TS 468a8fd46e824f62b9c2c0d3532a6d26 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54946
I20260709 13:15:58.626834 10685 catalog_manager.cc:5028] TS cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): tablet 73af6c7e9b8e4ffaa65ba65a6199eb61 (table test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]) successfully deleted
I20260709 13:15:58.639753 11687 tablet_replica.cc:333] T b4b13050d714473bb01ea0174d634a14 P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:15:58.660478 11652 raft_consensus.cc:2955] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:58.662288 10950 raft_consensus.cc:2955] T e9764b93d3a8407b8cc1ae641620c831 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:58.674983 10872 raft_consensus.cc:2955] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:58.701987 11687 raft_consensus.cc:2243] T b4b13050d714473bb01ea0174d634a14 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:58.703008 11687 raft_consensus.cc:2272] T b4b13050d714473bb01ea0174d634a14 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:58.714257 11687 ts_tablet_manager.cc:1916] T b4b13050d714473bb01ea0174d634a14 P 468a8fd46e824f62b9c2c0d3532a6d26: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:58.725168 11512 raft_consensus.cc:2955] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:58.736318 10697 catalog_manager.cc:5697] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:58.749348 11687 ts_tablet_manager.cc:1929] T b4b13050d714473bb01ea0174d634a14 P 468a8fd46e824f62b9c2c0d3532a6d26: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:15:58.753688 11687 log.cc:1199] T b4b13050d714473bb01ea0174d634a14 P 468a8fd46e824f62b9c2c0d3532a6d26: Deleting WAL directory at /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-1-root/wals/b4b13050d714473bb01ea0174d634a14
I20260709 13:15:58.755661 10686 catalog_manager.cc:5028] TS 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): tablet b4b13050d714473bb01ea0174d634a14 (table test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]) successfully deleted
I20260709 13:15:58.776111 10794 consensus_queue.cc:237] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:15:58.782446 11512 raft_consensus.cc:1275] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:58.782958 10871 raft_consensus.cc:1275] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:58.784272 11652 consensus_queue.cc:1048] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:58.785959 11652 consensus_queue.cc:1048] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:58.792667 11627 raft_consensus.cc:2955] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:58.800822 10871 raft_consensus.cc:2955] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:58.801187 11512 raft_consensus.cc:2955] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:58.804589 10686 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e9764b93d3a8407b8cc1ae641620c831 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:58.813731 10698 catalog_manager.cc:5697] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: config changed from index 3 to 4, VOTER cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) evicted. New cstate: current_term: 1 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:58.833835 11679 raft_consensus.cc:1064] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2: attempting to promote NON_VOTER 12682eb70bc84c76879e04a8628ee586 to VOTER
I20260709 13:15:58.836635 11679 consensus_queue.cc:237] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:15:58.839823 10937 tablet_service.cc:1558] Processing DeleteTablet for tablet e9764b93d3a8407b8cc1ae641620c831 with delete_type TABLET_DATA_TOMBSTONED (TS cb0428cc55984daba1ba63744a5102a2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41006
I20260709 13:15:58.840885 11684 tablet_replica.cc:333] T e9764b93d3a8407b8cc1ae641620c831 P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:15:58.842089 11684 raft_consensus.cc:2243] T e9764b93d3a8407b8cc1ae641620c831 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:58.843750 11684 raft_consensus.cc:2272] T e9764b93d3a8407b8cc1ae641620c831 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:58.847958 11684 ts_tablet_manager.cc:1916] T e9764b93d3a8407b8cc1ae641620c831 P cb0428cc55984daba1ba63744a5102a2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:58.853164 10793 raft_consensus.cc:1275] T b81df256f0c644cfae9fe2d336cbe9bd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:58.853775 10871 raft_consensus.cc:1275] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:58.855643 11674 consensus_queue.cc:1048] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:58.859262 11512 raft_consensus.cc:1275] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:58.862185 11679 consensus_queue.cc:1048] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:58.864732 11674 consensus_queue.cc:1048] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:58.893074 11684 ts_tablet_manager.cc:1929] T e9764b93d3a8407b8cc1ae641620c831 P cb0428cc55984daba1ba63744a5102a2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:15:58.893555 11684 log.cc:1199] T e9764b93d3a8407b8cc1ae641620c831 P cb0428cc55984daba1ba63744a5102a2: Deleting WAL directory at /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-2-root/wals/e9764b93d3a8407b8cc1ae641620c831
I20260709 13:15:58.892612 11411 raft_consensus.cc:2955] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:58.892915 10872 raft_consensus.cc:2955] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:58.895375 10685 catalog_manager.cc:5028] TS cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): tablet e9764b93d3a8407b8cc1ae641620c831 (table test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]) successfully deleted
I20260709 13:15:58.905673 11512 raft_consensus.cc:2955] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:58.914752 10793 raft_consensus.cc:2955] T b81df256f0c644cfae9fe2d336cbe9bd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:58.917490 10696 catalog_manager.cc:5697] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:58.943745 10948 consensus_queue.cc:237] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:15:58.952251 11512 raft_consensus.cc:1275] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:58.954074 11679 consensus_queue.cc:1048] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:58.956970 10871 raft_consensus.cc:1275] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:58.958436 11679 consensus_queue.cc:1048] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:58.964347 11642 raft_consensus.cc:2955] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:58.966452 11510 raft_consensus.cc:2955] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:58.971598 10872 raft_consensus.cc:2955] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:58.983439 10685 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b81df256f0c644cfae9fe2d336cbe9bd with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:58.983590 10698 catalog_manager.cc:5697] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 reported cstate change: config changed from index 3 to 4, VOTER e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) evicted. New cstate: current_term: 1 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:15:58.996269 10785 tablet_service.cc:1558] Processing DeleteTablet for tablet b81df256f0c644cfae9fe2d336cbe9bd with delete_type TABLET_DATA_TOMBSTONED (TS e01ecd008edd4aa2b6b2b47efbc2ea87 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46752
I20260709 13:15:58.998384 11696 tablet_replica.cc:333] T b81df256f0c644cfae9fe2d336cbe9bd P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:15:59.006773 11696 raft_consensus.cc:2243] T b81df256f0c644cfae9fe2d336cbe9bd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:59.007458 11696 raft_consensus.cc:2272] T b81df256f0c644cfae9fe2d336cbe9bd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:59.010406 11696 ts_tablet_manager.cc:1916] T b81df256f0c644cfae9fe2d336cbe9bd P e01ecd008edd4aa2b6b2b47efbc2ea87: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:59.024784 11696 ts_tablet_manager.cc:1929] T b81df256f0c644cfae9fe2d336cbe9bd P e01ecd008edd4aa2b6b2b47efbc2ea87: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:15:59.025166 11696 log.cc:1199] T b81df256f0c644cfae9fe2d336cbe9bd P e01ecd008edd4aa2b6b2b47efbc2ea87: Deleting WAL directory at /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-0-root/wals/b81df256f0c644cfae9fe2d336cbe9bd
I20260709 13:15:59.026643 10686 catalog_manager.cc:5028] TS e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): tablet b81df256f0c644cfae9fe2d336cbe9bd (table test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]) successfully deleted
I20260709 13:16:00.072129 10871 consensus_queue.cc:237] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:00.077559 10800 raft_consensus.cc:1275] T 185480268a684958b8347a4f53af57c5 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:16:00.078225 10841 consensus_peers.cc:597] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Couldn't send request to peer 12682eb70bc84c76879e04a8628ee586. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 185480268a684958b8347a4f53af57c5. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:00.078460 10948 raft_consensus.cc:1275] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:00.079058 11626 consensus_queue.cc:1048] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:00.080034 11697 consensus_queue.cc:1048] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:00.088490 11626 raft_consensus.cc:2955] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:00.090372 10794 raft_consensus.cc:2955] T 185480268a684958b8347a4f53af57c5 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:00.096216 10950 raft_consensus.cc:2955] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:00.107522 10698 catalog_manager.cc:5697] T 185480268a684958b8347a4f53af57c5 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:00.112241 10794 consensus_queue.cc:237] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:00.128593 10871 raft_consensus.cc:1275] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:16:00.128790 10766 consensus_peers.cc:597] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 -> Peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Couldn't send request to peer 12682eb70bc84c76879e04a8628ee586. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f6e90b642b0d4f06b866170dc0293dc1. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:00.130385 11595 consensus_queue.cc:1048] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:00.130677 10948 raft_consensus.cc:1275] T f6e90b642b0d4f06b866170dc0293dc1 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:00.133301 11595 consensus_queue.cc:1048] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:00.140964 11652 raft_consensus.cc:2955] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:00.144580 10872 raft_consensus.cc:2955] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:00.168967 10872 consensus_queue.cc:237] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:00.174594 10950 raft_consensus.cc:2955] T f6e90b642b0d4f06b866170dc0293dc1 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:00.177183 10948 raft_consensus.cc:1275] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:00.179010 11669 consensus_queue.cc:1048] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:00.179994 10794 raft_consensus.cc:1275] T 7595d019b2e74196bc5838962b53798d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:00.181798 11669 consensus_queue.cc:1048] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:16:00.188395 10841 consensus_peers.cc:597] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Couldn't send request to peer 12682eb70bc84c76879e04a8628ee586. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7595d019b2e74196bc5838962b53798d. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:00.207194 11626 raft_consensus.cc:2955] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:00.208374 10800 raft_consensus.cc:2955] T 7595d019b2e74196bc5838962b53798d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:00.203191 10698 catalog_manager.cc:5697] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New cstate: current_term: 1 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:00.218556 10950 raft_consensus.cc:2955] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:00.233290 10794 consensus_queue.cc:237] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:00.249058 10696 catalog_manager.cc:5697] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:00.255427 10950 raft_consensus.cc:1275] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:16:00.257076 10766 consensus_peers.cc:597] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 -> Peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Couldn't send request to peer 12682eb70bc84c76879e04a8628ee586. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 741efe79e97646b396f0091dc10c745e. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:00.257650 11652 consensus_queue.cc:1048] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:00.258271 10872 raft_consensus.cc:1275] T 741efe79e97646b396f0091dc10c745e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:00.262719 11652 consensus_queue.cc:1048] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:00.266968 11595 raft_consensus.cc:2955] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:00.269516 10950 raft_consensus.cc:2955] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:00.286274 10697 catalog_manager.cc:5697] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 reported cstate change: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New cstate: current_term: 1 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:00.288136 10872 raft_consensus.cc:2955] T 741efe79e97646b396f0091dc10c745e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:00.476444 11717 ts_tablet_manager.cc:933] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586: Initiating tablet copy from peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:00.480746 11717 tablet_copy_client.cc:323] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Beginning tablet copy session from remote peer at address 127.9.116.130:37895
I20260709 13:16:00.482388 10886 tablet_copy_service.cc:140] P 468a8fd46e824f62b9c2c0d3532a6d26: Received BeginTabletCopySession request for tablet 185480268a684958b8347a4f53af57c5 from peer 12682eb70bc84c76879e04a8628ee586 ({username='slave'} at 127.0.0.1:49290)
I20260709 13:16:00.482986 10886 tablet_copy_service.cc:161] P 468a8fd46e824f62b9c2c0d3532a6d26: Beginning new tablet copy session on tablet 185480268a684958b8347a4f53af57c5 from peer 12682eb70bc84c76879e04a8628ee586 at {username='slave'} at 127.0.0.1:49290: session id = 12682eb70bc84c76879e04a8628ee586-185480268a684958b8347a4f53af57c5
I20260709 13:16:00.490720 10886 tablet_copy_source_session.cc:215] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:00.494082 11717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 185480268a684958b8347a4f53af57c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:00.506935 11717 tablet_copy_client.cc:806] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:00.507579 11717 tablet_copy_client.cc:670] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:00.511251 11717 tablet_copy_client.cc:538] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:00.518455 11717 tablet_bootstrap.cc:492] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586: Bootstrap starting.
I20260709 13:16:00.541505 11717 tablet_bootstrap.cc:492] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:00.542651 11717 tablet_bootstrap.cc:492] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586: Bootstrap complete.
I20260709 13:16:00.543503 11717 ts_tablet_manager.cc:1403] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586: Time spent bootstrapping tablet: real 0.025s	user 0.021s	sys 0.005s
I20260709 13:16:00.547066 11717 raft_consensus.cc:359] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:00.548214 11717 raft_consensus.cc:740] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Initialized, Role: LEARNER
I20260709 13:16:00.548972 11717 consensus_queue.cc:260] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:00.556612 11717 ts_tablet_manager.cc:1434] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.005s
I20260709 13:16:00.558744 10886 tablet_copy_service.cc:342] P 468a8fd46e824f62b9c2c0d3532a6d26: Request end of tablet copy session 12682eb70bc84c76879e04a8628ee586-185480268a684958b8347a4f53af57c5 received from {username='slave'} at 127.0.0.1:49290
I20260709 13:16:00.559321 10886 tablet_copy_service.cc:434] P 468a8fd46e824f62b9c2c0d3532a6d26: ending tablet copy session 12682eb70bc84c76879e04a8628ee586-185480268a684958b8347a4f53af57c5 on tablet 185480268a684958b8347a4f53af57c5 with peer 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:00.615995 11717 ts_tablet_manager.cc:933] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586: Initiating tablet copy from peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685)
I20260709 13:16:00.618255 11717 tablet_copy_client.cc:323] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Beginning tablet copy session from remote peer at address 127.9.116.129:39685
I20260709 13:16:00.619927 10811 tablet_copy_service.cc:140] P e01ecd008edd4aa2b6b2b47efbc2ea87: Received BeginTabletCopySession request for tablet f6e90b642b0d4f06b866170dc0293dc1 from peer 12682eb70bc84c76879e04a8628ee586 ({username='slave'} at 127.0.0.1:39442)
I20260709 13:16:00.620497 10811 tablet_copy_service.cc:161] P e01ecd008edd4aa2b6b2b47efbc2ea87: Beginning new tablet copy session on tablet f6e90b642b0d4f06b866170dc0293dc1 from peer 12682eb70bc84c76879e04a8628ee586 at {username='slave'} at 127.0.0.1:39442: session id = 12682eb70bc84c76879e04a8628ee586-f6e90b642b0d4f06b866170dc0293dc1
I20260709 13:16:00.626912 10811 tablet_copy_source_session.cc:215] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:00.630052 11717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6e90b642b0d4f06b866170dc0293dc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:00.646886 11717 tablet_copy_client.cc:806] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:00.647578 11717 tablet_copy_client.cc:670] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:00.651352 11717 tablet_copy_client.cc:538] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:00.660852 11717 tablet_bootstrap.cc:492] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586: Bootstrap starting.
I20260709 13:16:00.699365 11717 tablet_bootstrap.cc:492] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:00.700423 11717 tablet_bootstrap.cc:492] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586: Bootstrap complete.
I20260709 13:16:00.701643 11717 ts_tablet_manager.cc:1403] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586: Time spent bootstrapping tablet: real 0.041s	user 0.024s	sys 0.016s
I20260709 13:16:00.704818 11717 raft_consensus.cc:359] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:00.705813 11717 raft_consensus.cc:740] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Initialized, Role: LEARNER
I20260709 13:16:00.706503 11717 consensus_queue.cc:260] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:00.708942 11717 ts_tablet_manager.cc:1434] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:16:00.713251 10811 tablet_copy_service.cc:342] P e01ecd008edd4aa2b6b2b47efbc2ea87: Request end of tablet copy session 12682eb70bc84c76879e04a8628ee586-f6e90b642b0d4f06b866170dc0293dc1 received from {username='slave'} at 127.0.0.1:39442
I20260709 13:16:00.713704 10811 tablet_copy_service.cc:434] P e01ecd008edd4aa2b6b2b47efbc2ea87: ending tablet copy session 12682eb70bc84c76879e04a8628ee586-f6e90b642b0d4f06b866170dc0293dc1 on tablet f6e90b642b0d4f06b866170dc0293dc1 with peer 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:00.735143 11717 ts_tablet_manager.cc:933] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586: Initiating tablet copy from peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:00.738664 11717 tablet_copy_client.cc:323] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586: tablet copy: Beginning tablet copy session from remote peer at address 127.9.116.130:37895
I20260709 13:16:00.742548 10886 tablet_copy_service.cc:140] P 468a8fd46e824f62b9c2c0d3532a6d26: Received BeginTabletCopySession request for tablet 7595d019b2e74196bc5838962b53798d from peer 12682eb70bc84c76879e04a8628ee586 ({username='slave'} at 127.0.0.1:49290)
I20260709 13:16:00.743140 10886 tablet_copy_service.cc:161] P 468a8fd46e824f62b9c2c0d3532a6d26: Beginning new tablet copy session on tablet 7595d019b2e74196bc5838962b53798d from peer 12682eb70bc84c76879e04a8628ee586 at {username='slave'} at 127.0.0.1:49290: session id = 12682eb70bc84c76879e04a8628ee586-7595d019b2e74196bc5838962b53798d
I20260709 13:16:00.751669 10886 tablet_copy_source_session.cc:215] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:00.754856 11717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7595d019b2e74196bc5838962b53798d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:00.771842 11717 tablet_copy_client.cc:806] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:00.772413 11717 tablet_copy_client.cc:670] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:00.775763 11717 tablet_copy_client.cc:538] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:00.783532 11717 tablet_bootstrap.cc:492] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586: Bootstrap starting.
I20260709 13:16:00.799022 11717 tablet_bootstrap.cc:492] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:00.800004 11717 tablet_bootstrap.cc:492] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586: Bootstrap complete.
I20260709 13:16:00.800719 11717 ts_tablet_manager.cc:1403] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586: Time spent bootstrapping tablet: real 0.017s	user 0.011s	sys 0.006s
I20260709 13:16:00.803558 11717 raft_consensus.cc:359] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:00.804725 11717 raft_consensus.cc:740] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Initialized, Role: LEARNER
I20260709 13:16:00.805361 11717 consensus_queue.cc:260] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:00.810410 11717 ts_tablet_manager.cc:1434] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:16:00.812655 10886 tablet_copy_service.cc:342] P 468a8fd46e824f62b9c2c0d3532a6d26: Request end of tablet copy session 12682eb70bc84c76879e04a8628ee586-7595d019b2e74196bc5838962b53798d received from {username='slave'} at 127.0.0.1:49290
I20260709 13:16:00.813138 10886 tablet_copy_service.cc:434] P 468a8fd46e824f62b9c2c0d3532a6d26: ending tablet copy session 12682eb70bc84c76879e04a8628ee586-7595d019b2e74196bc5838962b53798d on tablet 7595d019b2e74196bc5838962b53798d with peer 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:00.813967 11726 ts_tablet_manager.cc:933] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586: Initiating tablet copy from peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685)
I20260709 13:16:00.822119 11726 tablet_copy_client.cc:323] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586: tablet copy: Beginning tablet copy session from remote peer at address 127.9.116.129:39685
I20260709 13:16:00.824337 10811 tablet_copy_service.cc:140] P e01ecd008edd4aa2b6b2b47efbc2ea87: Received BeginTabletCopySession request for tablet 741efe79e97646b396f0091dc10c745e from peer 12682eb70bc84c76879e04a8628ee586 ({username='slave'} at 127.0.0.1:39442)
I20260709 13:16:00.824967 10811 tablet_copy_service.cc:161] P e01ecd008edd4aa2b6b2b47efbc2ea87: Beginning new tablet copy session on tablet 741efe79e97646b396f0091dc10c745e from peer 12682eb70bc84c76879e04a8628ee586 at {username='slave'} at 127.0.0.1:39442: session id = 12682eb70bc84c76879e04a8628ee586-741efe79e97646b396f0091dc10c745e
I20260709 13:16:00.832401 10811 tablet_copy_source_session.cc:215] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:00.840092 11726 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 741efe79e97646b396f0091dc10c745e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:00.864468 11726 tablet_copy_client.cc:806] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:00.865036 11726 tablet_copy_client.cc:670] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:00.868800 11726 tablet_copy_client.cc:538] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:00.876735 11726 tablet_bootstrap.cc:492] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586: Bootstrap starting.
I20260709 13:16:00.893883 11726 tablet_bootstrap.cc:492] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:00.894621 11726 tablet_bootstrap.cc:492] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586: Bootstrap complete.
I20260709 13:16:00.895203 11726 ts_tablet_manager.cc:1403] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586: Time spent bootstrapping tablet: real 0.019s	user 0.011s	sys 0.008s
I20260709 13:16:00.897471 11726 raft_consensus.cc:359] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:00.898083 11726 raft_consensus.cc:740] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Initialized, Role: LEARNER
I20260709 13:16:00.898619 11726 consensus_queue.cc:260] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:00.900492 11726 ts_tablet_manager.cc:1434] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:16:00.902531 10811 tablet_copy_service.cc:342] P e01ecd008edd4aa2b6b2b47efbc2ea87: Request end of tablet copy session 12682eb70bc84c76879e04a8628ee586-741efe79e97646b396f0091dc10c745e received from {username='slave'} at 127.0.0.1:39442
I20260709 13:16:00.902973 10811 tablet_copy_service.cc:434] P e01ecd008edd4aa2b6b2b47efbc2ea87: ending tablet copy session 12682eb70bc84c76879e04a8628ee586-741efe79e97646b396f0091dc10c745e on tablet 741efe79e97646b396f0091dc10c745e with peer 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:00.926507 11510 raft_consensus.cc:1217] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:01.196677 11510 raft_consensus.cc:1217] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:01.281629 11510 raft_consensus.cc:1217] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:01.385186 11510 raft_consensus.cc:1217] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:01.448781 11730 raft_consensus.cc:1064] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26: attempting to promote NON_VOTER 12682eb70bc84c76879e04a8628ee586 to VOTER
I20260709 13:16:01.450933 11730 consensus_queue.cc:237] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:01.459154 10950 raft_consensus.cc:1275] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:01.460302 10793 raft_consensus.cc:1275] T 185480268a684958b8347a4f53af57c5 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:01.461601 11510 raft_consensus.cc:1275] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:01.461414 11730 consensus_queue.cc:1048] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:01.462906 11731 consensus_queue.cc:1048] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:01.463801 11697 consensus_queue.cc:1048] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:01.474953 11697 raft_consensus.cc:2955] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:01.476838 11510 raft_consensus.cc:2955] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:01.490581 10950 raft_consensus.cc:2955] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:01.497359 10794 raft_consensus.cc:2955] T 185480268a684958b8347a4f53af57c5 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:01.502961 10698 catalog_manager.cc:5697] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:01.562522 10871 consensus_queue.cc:237] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:01.569375 11512 raft_consensus.cc:1275] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:01.570712 10950 raft_consensus.cc:1275] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:01.570868 11697 consensus_queue.cc:1048] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:16:01.572881 11731 consensus_queue.cc:1048] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:01.580266 11697 raft_consensus.cc:2955] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:01.585059 11512 raft_consensus.cc:2955] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:01.585248 10950 raft_consensus.cc:2955] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:01.599524 10699 catalog_manager.cc:5697] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: config changed from index 3 to 4, VOTER e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) evicted. New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:01.604437 10686 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 185480268a684958b8347a4f53af57c5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:01.614001 10785 tablet_service.cc:1558] Processing DeleteTablet for tablet 185480268a684958b8347a4f53af57c5 with delete_type TABLET_DATA_TOMBSTONED (TS e01ecd008edd4aa2b6b2b47efbc2ea87 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46752
I20260709 13:16:01.616765 11740 tablet_replica.cc:333] T 185480268a684958b8347a4f53af57c5 P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:16:01.618352 11740 raft_consensus.cc:2243] T 185480268a684958b8347a4f53af57c5 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:01.619124 11740 raft_consensus.cc:2272] T 185480268a684958b8347a4f53af57c5 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:01.623679 11740 ts_tablet_manager.cc:1916] T 185480268a684958b8347a4f53af57c5 P e01ecd008edd4aa2b6b2b47efbc2ea87: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:01.640729 11740 ts_tablet_manager.cc:1929] T 185480268a684958b8347a4f53af57c5 P e01ecd008edd4aa2b6b2b47efbc2ea87: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:01.641240 11740 log.cc:1199] T 185480268a684958b8347a4f53af57c5 P e01ecd008edd4aa2b6b2b47efbc2ea87: Deleting WAL directory at /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-0-root/wals/185480268a684958b8347a4f53af57c5
I20260709 13:16:01.642992 10686 catalog_manager.cc:5028] TS e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): tablet 185480268a684958b8347a4f53af57c5 (table test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]) successfully deleted
I20260709 13:16:01.661158 11652 raft_consensus.cc:1064] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87: attempting to promote NON_VOTER 12682eb70bc84c76879e04a8628ee586 to VOTER
I20260709 13:16:01.663645 11652 consensus_queue.cc:237] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:01.670917 10949 raft_consensus.cc:1275] T f6e90b642b0d4f06b866170dc0293dc1 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:01.671460 11512 raft_consensus.cc:1275] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:01.672886 10871 raft_consensus.cc:1275] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:01.673017 11595 consensus_queue.cc:1048] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:01.674909 11724 consensus_queue.cc:1048] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:01.676098 11595 consensus_queue.cc:1048] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:01.688850 11731 raft_consensus.cc:1064] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26: attempting to promote NON_VOTER 12682eb70bc84c76879e04a8628ee586 to VOTER
I20260709 13:16:01.694557 11731 consensus_queue.cc:237] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:01.730363 11724 raft_consensus.cc:2955] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:01.735482 11510 raft_consensus.cc:2955] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:01.742374 10872 raft_consensus.cc:2955] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:01.748689 10948 raft_consensus.cc:1275] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:01.747128 10949 raft_consensus.cc:2955] T f6e90b642b0d4f06b866170dc0293dc1 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:01.750111 11626 consensus_queue.cc:1048] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:01.756800 10794 raft_consensus.cc:1275] T 7595d019b2e74196bc5838962b53798d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:01.759482 11730 consensus_queue.cc:1048] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:01.770713 11510 raft_consensus.cc:1275] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:01.772970 11731 consensus_queue.cc:1048] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:01.751302 10696 catalog_manager.cc:5697] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:01.800138 11595 raft_consensus.cc:1064] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87: attempting to promote NON_VOTER 12682eb70bc84c76879e04a8628ee586 to VOTER
I20260709 13:16:01.802454 11595 consensus_queue.cc:237] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:01.824757 10793 consensus_queue.cc:237] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:01.834419 10872 raft_consensus.cc:1275] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:01.836301 11652 consensus_queue.cc:1048] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:01.842743 11510 raft_consensus.cc:1275] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:01.844756 11697 raft_consensus.cc:2955] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:01.852344 11652 raft_consensus.cc:2955] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:01.853495 11724 consensus_queue.cc:1048] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:01.862869 10869 raft_consensus.cc:1275] T 741efe79e97646b396f0091dc10c745e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:01.864302 10950 raft_consensus.cc:1275] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:01.864445 11595 consensus_queue.cc:1048] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:01.865685 10793 raft_consensus.cc:2955] T 7595d019b2e74196bc5838962b53798d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:01.866967 11512 raft_consensus.cc:1275] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:01.868183 11510 raft_consensus.cc:2955] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:01.868804 11724 consensus_queue.cc:1048] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:01.876441 10686 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f6e90b642b0d4f06b866170dc0293dc1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:01.888247 11746 consensus_queue.cc:1048] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:01.892490 10949 raft_consensus.cc:2955] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:01.915149 11512 raft_consensus.cc:2955] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:01.925057 11724 raft_consensus.cc:2955] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:01.934168 10948 raft_consensus.cc:2955] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:01.942063 10699 catalog_manager.cc:5697] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:16:01.955436 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 85626.4us
I20260709 13:16:01.965126 10869 raft_consensus.cc:2955] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:01.966735 11512 raft_consensus.cc:2955] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:01.991959 10696 catalog_manager.cc:5697] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: config changed from index 3 to 4, VOTER cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) evicted. New cstate: current_term: 1 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:02.020955 10872 raft_consensus.cc:2955] T 741efe79e97646b396f0091dc10c745e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:01.965173 10699 catalog_manager.cc:5697] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 reported cstate change: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:02.028008 10875 consensus_queue.cc:237] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:02.030987 10937 tablet_service.cc:1558] Processing DeleteTablet for tablet f6e90b642b0d4f06b866170dc0293dc1 with delete_type TABLET_DATA_TOMBSTONED (TS cb0428cc55984daba1ba63744a5102a2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41006
I20260709 13:16:02.046250 11512 raft_consensus.cc:1275] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:02.047853 11730 consensus_queue.cc:1048] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:02.060972 11697 raft_consensus.cc:2955] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:02.063537 11756 tablet_replica.cc:333] T f6e90b642b0d4f06b866170dc0293dc1 P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:16:02.064610 11756 raft_consensus.cc:2243] T f6e90b642b0d4f06b866170dc0293dc1 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:02.065397 11756 raft_consensus.cc:2272] T f6e90b642b0d4f06b866170dc0293dc1 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:02.072393 10949 raft_consensus.cc:1275] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:02.089847 11512 raft_consensus.cc:2955] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:02.091720 10797 consensus_queue.cc:237] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:02.094252 10686 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7595d019b2e74196bc5838962b53798d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:02.097002 11731 consensus_queue.cc:1048] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:02.106256 11510 raft_consensus.cc:1275] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:02.105368 10949 raft_consensus.cc:2955] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:02.107885 11595 consensus_queue.cc:1048] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:02.109531 11756 ts_tablet_manager.cc:1916] T f6e90b642b0d4f06b866170dc0293dc1 P cb0428cc55984daba1ba63744a5102a2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:02.117583 11595 raft_consensus.cc:2955] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:02.119096 11510 raft_consensus.cc:2955] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:02.125765 10950 raft_consensus.cc:1275] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:02.140427 10686 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 741efe79e97646b396f0091dc10c745e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:02.142748 11595 consensus_queue.cc:1048] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:02.151355 10697 catalog_manager.cc:5697] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: config changed from index 3 to 4, VOTER 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) evicted. New cstate: current_term: 1 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:02.158162 10950 raft_consensus.cc:2955] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:02.164613 10863 tablet_service.cc:1558] Processing DeleteTablet for tablet 741efe79e97646b396f0091dc10c745e with delete_type TABLET_DATA_TOMBSTONED (TS 468a8fd46e824f62b9c2c0d3532a6d26 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54946
I20260709 13:16:02.165867 10699 catalog_manager.cc:5697] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 reported cstate change: config changed from index 3 to 4, VOTER e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) evicted. New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:02.170025 11758 tablet_replica.cc:333] T 741efe79e97646b396f0091dc10c745e P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:16:02.171178 11758 raft_consensus.cc:2243] T 741efe79e97646b396f0091dc10c745e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:02.171981 11758 raft_consensus.cc:2272] T 741efe79e97646b396f0091dc10c745e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:02.175488 11758 ts_tablet_manager.cc:1916] T 741efe79e97646b396f0091dc10c745e P 468a8fd46e824f62b9c2c0d3532a6d26: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:02.187563 10785 tablet_service.cc:1558] Processing DeleteTablet for tablet 7595d019b2e74196bc5838962b53798d with delete_type TABLET_DATA_TOMBSTONED (TS e01ecd008edd4aa2b6b2b47efbc2ea87 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46752
I20260709 13:16:02.195048 11756 ts_tablet_manager.cc:1929] T f6e90b642b0d4f06b866170dc0293dc1 P cb0428cc55984daba1ba63744a5102a2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:02.195566 11756 log.cc:1199] T f6e90b642b0d4f06b866170dc0293dc1 P cb0428cc55984daba1ba63744a5102a2: Deleting WAL directory at /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-2-root/wals/f6e90b642b0d4f06b866170dc0293dc1
I20260709 13:16:02.197705 10685 catalog_manager.cc:5028] TS cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): tablet f6e90b642b0d4f06b866170dc0293dc1 (table test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]) successfully deleted
I20260709 13:16:02.204802 11758 ts_tablet_manager.cc:1929] T 741efe79e97646b396f0091dc10c745e P 468a8fd46e824f62b9c2c0d3532a6d26: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:02.205084 11759 tablet_replica.cc:333] T 7595d019b2e74196bc5838962b53798d P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:16:02.205421 11758 log.cc:1199] T 741efe79e97646b396f0091dc10c745e P 468a8fd46e824f62b9c2c0d3532a6d26: Deleting WAL directory at /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-1-root/wals/741efe79e97646b396f0091dc10c745e
I20260709 13:16:02.207832 10686 catalog_manager.cc:5028] TS 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): tablet 741efe79e97646b396f0091dc10c745e (table test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]) successfully deleted
I20260709 13:16:02.208590 11759 raft_consensus.cc:2243] T 7595d019b2e74196bc5838962b53798d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:02.209888 11759 raft_consensus.cc:2272] T 7595d019b2e74196bc5838962b53798d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:02.213501 11759 ts_tablet_manager.cc:1916] T 7595d019b2e74196bc5838962b53798d P e01ecd008edd4aa2b6b2b47efbc2ea87: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:02.225930 11759 ts_tablet_manager.cc:1929] T 7595d019b2e74196bc5838962b53798d P e01ecd008edd4aa2b6b2b47efbc2ea87: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:02.226375 11759 log.cc:1199] T 7595d019b2e74196bc5838962b53798d P e01ecd008edd4aa2b6b2b47efbc2ea87: Deleting WAL directory at /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-0-root/wals/7595d019b2e74196bc5838962b53798d
I20260709 13:16:02.228364 10686 catalog_manager.cc:5028] TS e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): tablet 7595d019b2e74196bc5838962b53798d (table test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]) successfully deleted
I20260709 13:16:03.261009 10869 consensus_queue.cc:237] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:03.267731 10797 raft_consensus.cc:1275] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:16:03.268064 10841 consensus_peers.cc:597] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Couldn't send request to peer 12682eb70bc84c76879e04a8628ee586. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 121ef9771004451eb563dbfd77657e0e. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:03.267735 10950 raft_consensus.cc:1275] T 121ef9771004451eb563dbfd77657e0e P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:03.269498 11731 consensus_queue.cc:1048] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:03.270469 11697 consensus_queue.cc:1048] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:03.277485 11697 raft_consensus.cc:2955] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:03.280088 10797 raft_consensus.cc:2955] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:03.286836 10948 raft_consensus.cc:2955] T 121ef9771004451eb563dbfd77657e0e P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:03.291229 10869 consensus_queue.cc:237] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:03.295436 10699 catalog_manager.cc:5697] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:03.298771 10950 raft_consensus.cc:1275] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:16:03.300206 10841 consensus_peers.cc:597] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Couldn't send request to peer 12682eb70bc84c76879e04a8628ee586. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ad1b662471ff4b9ebda3845b4f224a34. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:03.301035 10793 raft_consensus.cc:1275] T ad1b662471ff4b9ebda3845b4f224a34 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:03.301903 11697 consensus_queue.cc:1048] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:03.303341 11731 consensus_queue.cc:1048] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:03.314033 11761 raft_consensus.cc:2955] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:03.317821 10948 raft_consensus.cc:2955] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:03.354485 10948 consensus_queue.cc:237] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:03.368345 10800 raft_consensus.cc:2955] T ad1b662471ff4b9ebda3845b4f224a34 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
W20260709 13:16:03.371556 10915 consensus_peers.cc:597] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 -> Peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Couldn't send request to peer 12682eb70bc84c76879e04a8628ee586. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 416a6d12184f4fbda69570b95be47851. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:03.375039 10869 raft_consensus.cc:1275] T 416a6d12184f4fbda69570b95be47851 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:03.375648 10797 raft_consensus.cc:1275] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:03.376525 11679 consensus_queue.cc:1048] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:16:03.379175 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 50959us
I20260709 13:16:03.379995 11679 consensus_queue.cc:1048] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:03.415244 10697 catalog_manager.cc:5697] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:03.416836 11679 raft_consensus.cc:2955] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:03.419312 10869 raft_consensus.cc:2955] T 416a6d12184f4fbda69570b95be47851 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:03.420925 10794 raft_consensus.cc:2955] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:03.430909 10950 consensus_queue.cc:237] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
W20260709 13:16:03.439932 10915 consensus_peers.cc:597] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 -> Peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Couldn't send request to peer 12682eb70bc84c76879e04a8628ee586. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5944b8838e584a3890e4e8143f3f2665. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:03.440896 10800 raft_consensus.cc:1275] T 5944b8838e584a3890e4e8143f3f2665 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:03.440948 10869 raft_consensus.cc:1275] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:03.442427 11776 consensus_queue.cc:1048] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:03.441977 10697 catalog_manager.cc:5697] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New cstate: current_term: 1 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:03.443571 11411 consensus_queue.cc:1048] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:03.453853 11411 raft_consensus.cc:2955] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:03.461150 10871 raft_consensus.cc:2955] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:03.476852 10794 raft_consensus.cc:2955] T 5944b8838e584a3890e4e8143f3f2665 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:03.493186 10699 catalog_manager.cc:5697] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New cstate: current_term: 1 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:03.678009 11780 ts_tablet_manager.cc:933] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586: Initiating tablet copy from peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:03.680549 11780 tablet_copy_client.cc:323] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586: tablet copy: Beginning tablet copy session from remote peer at address 127.9.116.130:37895
I20260709 13:16:03.682024 10886 tablet_copy_service.cc:140] P 468a8fd46e824f62b9c2c0d3532a6d26: Received BeginTabletCopySession request for tablet 121ef9771004451eb563dbfd77657e0e from peer 12682eb70bc84c76879e04a8628ee586 ({username='slave'} at 127.0.0.1:49290)
I20260709 13:16:03.682474 10886 tablet_copy_service.cc:161] P 468a8fd46e824f62b9c2c0d3532a6d26: Beginning new tablet copy session on tablet 121ef9771004451eb563dbfd77657e0e from peer 12682eb70bc84c76879e04a8628ee586 at {username='slave'} at 127.0.0.1:49290: session id = 12682eb70bc84c76879e04a8628ee586-121ef9771004451eb563dbfd77657e0e
I20260709 13:16:03.691407 10886 tablet_copy_source_session.cc:215] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:03.694692 11780 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 121ef9771004451eb563dbfd77657e0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:03.705905 11780 tablet_copy_client.cc:806] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:03.706528 11780 tablet_copy_client.cc:670] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:03.710256 11780 tablet_copy_client.cc:538] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:03.717741 11780 tablet_bootstrap.cc:492] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586: Bootstrap starting.
I20260709 13:16:03.737962 11780 tablet_bootstrap.cc:492] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:03.738837 11780 tablet_bootstrap.cc:492] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586: Bootstrap complete.
I20260709 13:16:03.739502 11780 ts_tablet_manager.cc:1403] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586: Time spent bootstrapping tablet: real 0.022s	user 0.018s	sys 0.004s
I20260709 13:16:03.742668 11780 raft_consensus.cc:359] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:03.743570 11780 raft_consensus.cc:740] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Initialized, Role: LEARNER
I20260709 13:16:03.744278 11780 consensus_queue.cc:260] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:03.747311 11780 ts_tablet_manager.cc:1434] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 13:16:03.758697 10886 tablet_copy_service.cc:342] P 468a8fd46e824f62b9c2c0d3532a6d26: Request end of tablet copy session 12682eb70bc84c76879e04a8628ee586-121ef9771004451eb563dbfd77657e0e received from {username='slave'} at 127.0.0.1:49290
I20260709 13:16:03.759181 10886 tablet_copy_service.cc:434] P 468a8fd46e824f62b9c2c0d3532a6d26: ending tablet copy session 12682eb70bc84c76879e04a8628ee586-121ef9771004451eb563dbfd77657e0e on tablet 121ef9771004451eb563dbfd77657e0e with peer 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:03.904651 11780 ts_tablet_manager.cc:933] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586: Initiating tablet copy from peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:03.928841 11780 tablet_copy_client.cc:323] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Beginning tablet copy session from remote peer at address 127.9.116.130:37895
I20260709 13:16:03.930945 10886 tablet_copy_service.cc:140] P 468a8fd46e824f62b9c2c0d3532a6d26: Received BeginTabletCopySession request for tablet ad1b662471ff4b9ebda3845b4f224a34 from peer 12682eb70bc84c76879e04a8628ee586 ({username='slave'} at 127.0.0.1:49290)
I20260709 13:16:03.931550 10886 tablet_copy_service.cc:161] P 468a8fd46e824f62b9c2c0d3532a6d26: Beginning new tablet copy session on tablet ad1b662471ff4b9ebda3845b4f224a34 from peer 12682eb70bc84c76879e04a8628ee586 at {username='slave'} at 127.0.0.1:49290: session id = 12682eb70bc84c76879e04a8628ee586-ad1b662471ff4b9ebda3845b4f224a34
I20260709 13:16:03.940069 10886 tablet_copy_source_session.cc:215] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:03.943341 11780 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad1b662471ff4b9ebda3845b4f224a34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:03.969772 11780 tablet_copy_client.cc:806] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:03.970448 11780 tablet_copy_client.cc:670] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:03.974251 11780 tablet_copy_client.cc:538] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:03.986327 11780 tablet_bootstrap.cc:492] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586: Bootstrap starting.
I20260709 13:16:04.033625 11787 ts_tablet_manager.cc:933] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586: Initiating tablet copy from peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:04.035697 11785 ts_tablet_manager.cc:933] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586: Initiating tablet copy from peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:04.048714 11787 tablet_copy_client.cc:323] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Beginning tablet copy session from remote peer at address 127.9.116.131:45553
I20260709 13:16:04.051105 10959 tablet_copy_service.cc:140] P cb0428cc55984daba1ba63744a5102a2: Received BeginTabletCopySession request for tablet 416a6d12184f4fbda69570b95be47851 from peer 12682eb70bc84c76879e04a8628ee586 ({username='slave'} at 127.0.0.1:36714)
I20260709 13:16:04.051664 10959 tablet_copy_service.cc:161] P cb0428cc55984daba1ba63744a5102a2: Beginning new tablet copy session on tablet 416a6d12184f4fbda69570b95be47851 from peer 12682eb70bc84c76879e04a8628ee586 at {username='slave'} at 127.0.0.1:36714: session id = 12682eb70bc84c76879e04a8628ee586-416a6d12184f4fbda69570b95be47851
I20260709 13:16:04.068235 11510 raft_consensus.cc:1217] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:04.075718 10959 tablet_copy_source_session.cc:215] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:04.086180 11785 tablet_copy_client.cc:323] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Beginning tablet copy session from remote peer at address 127.9.116.131:45553
I20260709 13:16:04.093583 11780 tablet_bootstrap.cc:492] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:04.094662 11780 tablet_bootstrap.cc:492] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586: Bootstrap complete.
I20260709 13:16:04.095459 11780 ts_tablet_manager.cc:1403] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586: Time spent bootstrapping tablet: real 0.109s	user 0.032s	sys 0.019s
I20260709 13:16:04.098125 11787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 416a6d12184f4fbda69570b95be47851. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:04.098403 11780 raft_consensus.cc:359] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:04.099154 10959 tablet_copy_service.cc:140] P cb0428cc55984daba1ba63744a5102a2: Received BeginTabletCopySession request for tablet 5944b8838e584a3890e4e8143f3f2665 from peer 12682eb70bc84c76879e04a8628ee586 ({username='slave'} at 127.0.0.1:36714)
I20260709 13:16:04.099398 11780 raft_consensus.cc:740] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Initialized, Role: LEARNER
I20260709 13:16:04.099802 10959 tablet_copy_service.cc:161] P cb0428cc55984daba1ba63744a5102a2: Beginning new tablet copy session on tablet 5944b8838e584a3890e4e8143f3f2665 from peer 12682eb70bc84c76879e04a8628ee586 at {username='slave'} at 127.0.0.1:36714: session id = 12682eb70bc84c76879e04a8628ee586-5944b8838e584a3890e4e8143f3f2665
I20260709 13:16:04.100201 11780 consensus_queue.cc:260] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:04.107838 10959 tablet_copy_source_session.cc:215] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:04.110085 11780 ts_tablet_manager.cc:1434] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 13:16:04.111325 11785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5944b8838e584a3890e4e8143f3f2665. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:04.119040 11787 tablet_copy_client.cc:806] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:04.119374 10886 tablet_copy_service.cc:342] P 468a8fd46e824f62b9c2c0d3532a6d26: Request end of tablet copy session 12682eb70bc84c76879e04a8628ee586-ad1b662471ff4b9ebda3845b4f224a34 received from {username='slave'} at 127.0.0.1:49290
I20260709 13:16:04.119850 10886 tablet_copy_service.cc:434] P 468a8fd46e824f62b9c2c0d3532a6d26: ending tablet copy session 12682eb70bc84c76879e04a8628ee586-ad1b662471ff4b9ebda3845b4f224a34 on tablet ad1b662471ff4b9ebda3845b4f224a34 with peer 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:04.124655 11787 tablet_copy_client.cc:670] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:04.129093 11787 tablet_copy_client.cc:538] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:04.135757 11785 tablet_copy_client.cc:806] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:04.136853 11785 tablet_copy_client.cc:670] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:04.141193 11785 tablet_copy_client.cc:538] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:04.142937 11787 tablet_bootstrap.cc:492] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586: Bootstrap starting.
I20260709 13:16:04.174058 11785 tablet_bootstrap.cc:492] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586: Bootstrap starting.
I20260709 13:16:04.174538 11787 tablet_bootstrap.cc:492] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:04.175590 11787 tablet_bootstrap.cc:492] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586: Bootstrap complete.
I20260709 13:16:04.176340 11787 ts_tablet_manager.cc:1403] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586: Time spent bootstrapping tablet: real 0.034s	user 0.020s	sys 0.010s
I20260709 13:16:04.179548 11787 raft_consensus.cc:359] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:04.180544 11787 raft_consensus.cc:740] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Initialized, Role: LEARNER
I20260709 13:16:04.181530 11787 consensus_queue.cc:260] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:04.186618 11787 ts_tablet_manager.cc:1434] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.001s
I20260709 13:16:04.188745 10959 tablet_copy_service.cc:342] P cb0428cc55984daba1ba63744a5102a2: Request end of tablet copy session 12682eb70bc84c76879e04a8628ee586-416a6d12184f4fbda69570b95be47851 received from {username='slave'} at 127.0.0.1:36714
I20260709 13:16:04.189168 10959 tablet_copy_service.cc:434] P cb0428cc55984daba1ba63744a5102a2: ending tablet copy session 12682eb70bc84c76879e04a8628ee586-416a6d12184f4fbda69570b95be47851 on tablet 416a6d12184f4fbda69570b95be47851 with peer 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:04.207229 11785 tablet_bootstrap.cc:492] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:04.209122 11785 tablet_bootstrap.cc:492] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586: Bootstrap complete.
I20260709 13:16:04.210824 11785 ts_tablet_manager.cc:1403] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586: Time spent bootstrapping tablet: real 0.037s	user 0.018s	sys 0.004s
I20260709 13:16:04.213555 11785 raft_consensus.cc:359] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:04.214172 11785 raft_consensus.cc:740] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Initialized, Role: LEARNER
I20260709 13:16:04.214699 11785 consensus_queue.cc:260] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:04.216789 11785 ts_tablet_manager.cc:1434] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586: Time spent starting tablet: real 0.006s	user 0.000s	sys 0.003s
I20260709 13:16:04.218744 10959 tablet_copy_service.cc:342] P cb0428cc55984daba1ba63744a5102a2: Request end of tablet copy session 12682eb70bc84c76879e04a8628ee586-5944b8838e584a3890e4e8143f3f2665 received from {username='slave'} at 127.0.0.1:36714
I20260709 13:16:04.219192 10959 tablet_copy_service.cc:434] P cb0428cc55984daba1ba63744a5102a2: ending tablet copy session 12682eb70bc84c76879e04a8628ee586-5944b8838e584a3890e4e8143f3f2665 on tablet 5944b8838e584a3890e4e8143f3f2665 with peer 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:04.359283 11510 raft_consensus.cc:1217] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:04.444154 11510 raft_consensus.cc:1217] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:04.510064 11510 raft_consensus.cc:1217] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:04.572995 11697 raft_consensus.cc:1064] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26: attempting to promote NON_VOTER 12682eb70bc84c76879e04a8628ee586 to VOTER
I20260709 13:16:04.575345 11697 consensus_queue.cc:237] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:04.584983 10948 raft_consensus.cc:1275] T 121ef9771004451eb563dbfd77657e0e P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:04.586265 11791 consensus_queue.cc:1048] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:04.589293 11510 raft_consensus.cc:1275] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:04.589293 10800 raft_consensus.cc:1275] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:04.591025 11791 consensus_queue.cc:1048] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:04.592113 11766 consensus_queue.cc:1048] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:04.609352 11766 raft_consensus.cc:2955] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:04.610958 10800 raft_consensus.cc:2955] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:04.630013 11510 raft_consensus.cc:2955] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:04.631032 10699 catalog_manager.cc:5697] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:04.643811 10948 raft_consensus.cc:2955] T 121ef9771004451eb563dbfd77657e0e P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:04.661823 10869 consensus_queue.cc:237] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:04.679899 11510 raft_consensus.cc:1275] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:04.681811 11791 consensus_queue.cc:1048] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:04.683038 10800 raft_consensus.cc:1275] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:04.686400 11797 consensus_queue.cc:1048] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:04.691262 11797 raft_consensus.cc:2955] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:04.693652 11510 raft_consensus.cc:2955] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:04.695664 10800 raft_consensus.cc:2955] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:04.709905 10686 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 121ef9771004451eb563dbfd77657e0e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:04.719856 10697 catalog_manager.cc:5697] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: config changed from index 3 to 4, VOTER cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) evicted. New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:04.735044 10937 tablet_service.cc:1558] Processing DeleteTablet for tablet 121ef9771004451eb563dbfd77657e0e with delete_type TABLET_DATA_TOMBSTONED (TS cb0428cc55984daba1ba63744a5102a2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41006
I20260709 13:16:04.760018 11806 tablet_replica.cc:333] T 121ef9771004451eb563dbfd77657e0e P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:16:04.761360 11806 raft_consensus.cc:2243] T 121ef9771004451eb563dbfd77657e0e P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:04.762151 11806 raft_consensus.cc:2272] T 121ef9771004451eb563dbfd77657e0e P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:04.765365 11806 ts_tablet_manager.cc:1916] T 121ef9771004451eb563dbfd77657e0e P cb0428cc55984daba1ba63744a5102a2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:04.780426 11806 ts_tablet_manager.cc:1929] T 121ef9771004451eb563dbfd77657e0e P cb0428cc55984daba1ba63744a5102a2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:04.780788 11806 log.cc:1199] T 121ef9771004451eb563dbfd77657e0e P cb0428cc55984daba1ba63744a5102a2: Deleting WAL directory at /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-2-root/wals/121ef9771004451eb563dbfd77657e0e
I20260709 13:16:04.782126 10685 catalog_manager.cc:5028] TS cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): tablet 121ef9771004451eb563dbfd77657e0e (table test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]) successfully deleted
I20260709 13:16:04.810232 11766 raft_consensus.cc:1064] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26: attempting to promote NON_VOTER 12682eb70bc84c76879e04a8628ee586 to VOTER
I20260709 13:16:04.812741 11766 consensus_queue.cc:237] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:04.819628 10950 raft_consensus.cc:1275] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:04.820003 10800 raft_consensus.cc:1275] T ad1b662471ff4b9ebda3845b4f224a34 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:04.820369 11510 raft_consensus.cc:1275] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:04.821476 11797 consensus_queue.cc:1048] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:04.824312 11791 consensus_queue.cc:1048] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:04.825451 11697 consensus_queue.cc:1048] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:04.846328 11791 raft_consensus.cc:2955] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:04.848155 10794 raft_consensus.cc:2955] T ad1b662471ff4b9ebda3845b4f224a34 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:04.854064 11510 raft_consensus.cc:2955] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:04.860023 10950 raft_consensus.cc:2955] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:04.864514 10697 catalog_manager.cc:5697] T ad1b662471ff4b9ebda3845b4f224a34 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:04.894182 10872 consensus_queue.cc:237] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:04.902953 10948 raft_consensus.cc:1275] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:04.904929 11766 consensus_queue.cc:1048] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:04.905448 11510 raft_consensus.cc:1275] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:04.906852 11766 consensus_queue.cc:1048] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:04.913252 11797 raft_consensus.cc:2955] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:04.916210 10948 raft_consensus.cc:2955] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:04.931527 11510 raft_consensus.cc:2955] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:04.935683 10686 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ad1b662471ff4b9ebda3845b4f224a34 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:04.948814 10696 catalog_manager.cc:5697] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: config changed from index 3 to 4, VOTER e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) evicted. New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:04.965358 10785 tablet_service.cc:1558] Processing DeleteTablet for tablet ad1b662471ff4b9ebda3845b4f224a34 with delete_type TABLET_DATA_TOMBSTONED (TS e01ecd008edd4aa2b6b2b47efbc2ea87 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46752
I20260709 13:16:04.967280 11811 tablet_replica.cc:333] T ad1b662471ff4b9ebda3845b4f224a34 P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:16:04.968803 11811 raft_consensus.cc:2243] T ad1b662471ff4b9ebda3845b4f224a34 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:04.969554 11811 raft_consensus.cc:2272] T ad1b662471ff4b9ebda3845b4f224a34 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:04.973383 11811 ts_tablet_manager.cc:1916] T ad1b662471ff4b9ebda3845b4f224a34 P e01ecd008edd4aa2b6b2b47efbc2ea87: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:04.990367 11795 raft_consensus.cc:1064] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2: attempting to promote NON_VOTER 12682eb70bc84c76879e04a8628ee586 to VOTER
I20260709 13:16:04.992409 11795 consensus_queue.cc:237] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:04.994122 11811 ts_tablet_manager.cc:1929] T ad1b662471ff4b9ebda3845b4f224a34 P e01ecd008edd4aa2b6b2b47efbc2ea87: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:04.994648 11811 log.cc:1199] T ad1b662471ff4b9ebda3845b4f224a34 P e01ecd008edd4aa2b6b2b47efbc2ea87: Deleting WAL directory at /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-0-root/wals/ad1b662471ff4b9ebda3845b4f224a34
I20260709 13:16:04.996385 10686 catalog_manager.cc:5028] TS e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): tablet ad1b662471ff4b9ebda3845b4f224a34 (table test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]) successfully deleted
I20260709 13:16:04.999737 10872 raft_consensus.cc:1275] T 416a6d12184f4fbda69570b95be47851 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:04.999748 11510 raft_consensus.cc:1275] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:05.001318 11411 consensus_queue.cc:1048] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:05.002687 11776 consensus_queue.cc:1048] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:05.005393 10800 raft_consensus.cc:1275] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:05.007630 11411 consensus_queue.cc:1048] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:05.029778 11776 raft_consensus.cc:1064] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2: attempting to promote NON_VOTER 12682eb70bc84c76879e04a8628ee586 to VOTER
I20260709 13:16:05.032701 11776 consensus_queue.cc:237] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:05.041390 11794 raft_consensus.cc:2955] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:05.065240 11510 raft_consensus.cc:2955] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:05.065255 11512 raft_consensus.cc:1275] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:05.071355 11818 consensus_queue.cc:1048] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:16:05.093212 10697 catalog_manager.cc:5697] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
W20260709 13:16:05.096838 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 49912.8us
I20260709 13:16:05.101799 10872 raft_consensus.cc:1275] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:05.105384 10800 raft_consensus.cc:1275] T 5944b8838e584a3890e4e8143f3f2665 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:05.109311 10800 raft_consensus.cc:2955] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:05.110908 11818 consensus_queue.cc:1048] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:05.115341 11817 consensus_queue.cc:1048] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:05.119865 10869 raft_consensus.cc:2955] T 416a6d12184f4fbda69570b95be47851 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:05.158275 10948 consensus_queue.cc:237] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:05.167126 11512 raft_consensus.cc:1275] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:05.159024 11818 raft_consensus.cc:2955] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:05.168627 11794 consensus_queue.cc:1048] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:05.176236 11820 raft_consensus.cc:2955] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:05.179829 11512 raft_consensus.cc:2955] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:05.182453 10869 raft_consensus.cc:2955] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:05.187355 11510 raft_consensus.cc:2955] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:05.192169 10800 raft_consensus.cc:1275] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:05.202111 11411 consensus_queue.cc:1048] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:05.205030 10800 raft_consensus.cc:2955] T 5944b8838e584a3890e4e8143f3f2665 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:05.206118 10699 catalog_manager.cc:5697] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:05.211792 10685 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 416a6d12184f4fbda69570b95be47851 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:05.213382 10797 raft_consensus.cc:2955] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:05.220992 10696 catalog_manager.cc:5697] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: config changed from index 3 to 4, VOTER 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) evicted. New cstate: current_term: 1 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:05.233718 10863 tablet_service.cc:1558] Processing DeleteTablet for tablet 416a6d12184f4fbda69570b95be47851 with delete_type TABLET_DATA_TOMBSTONED (TS 468a8fd46e824f62b9c2c0d3532a6d26 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54946
I20260709 13:16:05.241379 10948 consensus_queue.cc:237] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:05.250228 10869 raft_consensus.cc:1275] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:05.253336 11512 raft_consensus.cc:1275] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:05.253954 11411 consensus_queue.cc:1048] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:05.254823 11817 consensus_queue.cc:1048] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:05.262509 11817 raft_consensus.cc:2955] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:05.267570 11824 tablet_replica.cc:333] T 416a6d12184f4fbda69570b95be47851 P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:16:05.268778 11824 raft_consensus.cc:2243] T 416a6d12184f4fbda69570b95be47851 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:05.269618 11824 raft_consensus.cc:2272] T 416a6d12184f4fbda69570b95be47851 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:05.271574 11511 raft_consensus.cc:2955] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:05.280254 10869 raft_consensus.cc:2955] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:05.285179 10685 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 5944b8838e584a3890e4e8143f3f2665 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:05.289301 10699 catalog_manager.cc:5697] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: config changed from index 3 to 4, VOTER e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) evicted. New cstate: current_term: 1 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:05.299638 11824 ts_tablet_manager.cc:1916] T 416a6d12184f4fbda69570b95be47851 P 468a8fd46e824f62b9c2c0d3532a6d26: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:05.304275 10785 tablet_service.cc:1558] Processing DeleteTablet for tablet 5944b8838e584a3890e4e8143f3f2665 with delete_type TABLET_DATA_TOMBSTONED (TS e01ecd008edd4aa2b6b2b47efbc2ea87 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46752
I20260709 13:16:05.305290 11811 tablet_replica.cc:333] T 5944b8838e584a3890e4e8143f3f2665 P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:16:05.306360 11811 raft_consensus.cc:2243] T 5944b8838e584a3890e4e8143f3f2665 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:05.307026 11811 raft_consensus.cc:2272] T 5944b8838e584a3890e4e8143f3f2665 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:05.310506 11811 ts_tablet_manager.cc:1916] T 5944b8838e584a3890e4e8143f3f2665 P e01ecd008edd4aa2b6b2b47efbc2ea87: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:05.319834 11824 ts_tablet_manager.cc:1929] T 416a6d12184f4fbda69570b95be47851 P 468a8fd46e824f62b9c2c0d3532a6d26: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:05.320271 11824 log.cc:1199] T 416a6d12184f4fbda69570b95be47851 P 468a8fd46e824f62b9c2c0d3532a6d26: Deleting WAL directory at /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-1-root/wals/416a6d12184f4fbda69570b95be47851
I20260709 13:16:05.321606 10686 catalog_manager.cc:5028] TS 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): tablet 416a6d12184f4fbda69570b95be47851 (table test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]) successfully deleted
I20260709 13:16:05.325978 11811 ts_tablet_manager.cc:1929] T 5944b8838e584a3890e4e8143f3f2665 P e01ecd008edd4aa2b6b2b47efbc2ea87: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:05.326530 11811 log.cc:1199] T 5944b8838e584a3890e4e8143f3f2665 P e01ecd008edd4aa2b6b2b47efbc2ea87: Deleting WAL directory at /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-0-root/wals/5944b8838e584a3890e4e8143f3f2665
I20260709 13:16:05.327771 10686 catalog_manager.cc:5028] TS e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): tablet 5944b8838e584a3890e4e8143f3f2665 (table test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]) successfully deleted
I20260709 13:16:06.377030 10948 consensus_queue.cc:237] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
W20260709 13:16:06.384016 10915 consensus_peers.cc:597] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 -> Peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Couldn't send request to peer 12682eb70bc84c76879e04a8628ee586. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 86ba4060a26c42998e9f220a79f28546. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:06.383730 10871 raft_consensus.cc:1275] T 86ba4060a26c42998e9f220a79f28546 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:06.384419 10797 raft_consensus.cc:1275] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:06.385759 11794 consensus_queue.cc:1048] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:06.386421 11817 consensus_queue.cc:1048] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:06.398494 11828 raft_consensus.cc:2955] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:06.400600 10871 raft_consensus.cc:2955] T 86ba4060a26c42998e9f220a79f28546 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:06.402166 10797 raft_consensus.cc:2955] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:06.417723 10698 catalog_manager.cc:5697] T 86ba4060a26c42998e9f220a79f28546 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New cstate: current_term: 1 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:06.425489 10800 consensus_queue.cc:237] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:06.435448 10948 raft_consensus.cc:1275] T f15293e178d84fe686fd66bc4439aba8 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:16:06.436123 10766 consensus_peers.cc:597] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 -> Peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Couldn't send request to peer 12682eb70bc84c76879e04a8628ee586. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f15293e178d84fe686fd66bc4439aba8. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:06.437258 11805 consensus_queue.cc:1048] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:06.442581 10871 raft_consensus.cc:1275] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:06.445644 11751 consensus_queue.cc:1048] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:06.453936 11751 raft_consensus.cc:2955] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:06.462901 10950 raft_consensus.cc:2955] T f15293e178d84fe686fd66bc4439aba8 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:06.479534 10800 consensus_queue.cc:237] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:06.479825 10869 raft_consensus.cc:2955] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:06.487849 10697 catalog_manager.cc:5697] T f15293e178d84fe686fd66bc4439aba8 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New cstate: current_term: 1 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:06.496641 10950 raft_consensus.cc:1275] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 13:16:06.497087 10766 consensus_peers.cc:597] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 -> Peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Couldn't send request to peer 12682eb70bc84c76879e04a8628ee586. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8cf3aebfc54a4652a83388ef9a48b1c2. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:06.498145 10871 raft_consensus.cc:1275] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:16:06.498910 11834 consensus_queue.cc:1048] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:06.501619 11834 consensus_queue.cc:1048] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:06.511379 11805 raft_consensus.cc:2955] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:06.512724 10950 raft_consensus.cc:2955] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:06.518211 10871 raft_consensus.cc:2955] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:06.534525 10950 consensus_queue.cc:237] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:06.535108 10699 catalog_manager.cc:5697] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: config changed from index -1 to 3, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New cstate: current_term: 2 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
W20260709 13:16:06.554724 10915 consensus_peers.cc:597] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 -> Peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Couldn't send request to peer 12682eb70bc84c76879e04a8628ee586. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76b2d93b69274dbca479bd87d7ea45cb. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:06.564396 10871 raft_consensus.cc:1275] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:06.565928 11817 consensus_queue.cc:1048] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:06.566246 10800 raft_consensus.cc:1275] T 76b2d93b69274dbca479bd87d7ea45cb P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:06.568241 11794 consensus_queue.cc:1048] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:06.583039 11794 raft_consensus.cc:2955] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:06.586972 10869 raft_consensus.cc:2955] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:06.591655 10800 raft_consensus.cc:2955] T 76b2d93b69274dbca479bd87d7ea45cb P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:06.618410 10697 catalog_manager.cc:5697] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 reported cstate change: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New cstate: current_term: 1 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:06.820503 11846 ts_tablet_manager.cc:933] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586: Initiating tablet copy from peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685)
I20260709 13:16:06.822507 11846 tablet_copy_client.cc:323] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Beginning tablet copy session from remote peer at address 127.9.116.129:39685
I20260709 13:16:06.824252 10811 tablet_copy_service.cc:140] P e01ecd008edd4aa2b6b2b47efbc2ea87: Received BeginTabletCopySession request for tablet f15293e178d84fe686fd66bc4439aba8 from peer 12682eb70bc84c76879e04a8628ee586 ({username='slave'} at 127.0.0.1:39442)
I20260709 13:16:06.824827 10811 tablet_copy_service.cc:161] P e01ecd008edd4aa2b6b2b47efbc2ea87: Beginning new tablet copy session on tablet f15293e178d84fe686fd66bc4439aba8 from peer 12682eb70bc84c76879e04a8628ee586 at {username='slave'} at 127.0.0.1:39442: session id = 12682eb70bc84c76879e04a8628ee586-f15293e178d84fe686fd66bc4439aba8
I20260709 13:16:06.838397 10811 tablet_copy_source_session.cc:215] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:06.846565 11846 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f15293e178d84fe686fd66bc4439aba8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:06.861981 11848 ts_tablet_manager.cc:933] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586: Initiating tablet copy from peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:06.866744 11846 tablet_copy_client.cc:806] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:06.867426 11846 tablet_copy_client.cc:670] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:06.869685 11848 tablet_copy_client.cc:323] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Beginning tablet copy session from remote peer at address 127.9.116.131:45553
I20260709 13:16:06.872421 11846 tablet_copy_client.cc:538] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:06.872366 10959 tablet_copy_service.cc:140] P cb0428cc55984daba1ba63744a5102a2: Received BeginTabletCopySession request for tablet 86ba4060a26c42998e9f220a79f28546 from peer 12682eb70bc84c76879e04a8628ee586 ({username='slave'} at 127.0.0.1:36714)
I20260709 13:16:06.872985 10959 tablet_copy_service.cc:161] P cb0428cc55984daba1ba63744a5102a2: Beginning new tablet copy session on tablet 86ba4060a26c42998e9f220a79f28546 from peer 12682eb70bc84c76879e04a8628ee586 at {username='slave'} at 127.0.0.1:36714: session id = 12682eb70bc84c76879e04a8628ee586-86ba4060a26c42998e9f220a79f28546
I20260709 13:16:06.881958 10959 tablet_copy_source_session.cc:215] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:06.888934 11846 tablet_bootstrap.cc:492] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586: Bootstrap starting.
I20260709 13:16:06.892814 11848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86ba4060a26c42998e9f220a79f28546. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:06.922271 11848 tablet_copy_client.cc:806] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:06.922892 11848 tablet_copy_client.cc:670] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:06.927340 11848 tablet_copy_client.cc:538] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:06.940882 11848 tablet_bootstrap.cc:492] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586: Bootstrap starting.
I20260709 13:16:06.975931 11848 tablet_bootstrap.cc:492] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:06.976893 11848 tablet_bootstrap.cc:492] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586: Bootstrap complete.
I20260709 13:16:06.977562 11848 ts_tablet_manager.cc:1403] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586: Time spent bootstrapping tablet: real 0.037s	user 0.018s	sys 0.009s
I20260709 13:16:06.982154 11848 raft_consensus.cc:359] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:06.983067 11848 raft_consensus.cc:740] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Initialized, Role: LEARNER
I20260709 13:16:06.983858 11848 consensus_queue.cc:260] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:06.998260 11848 ts_tablet_manager.cc:1434] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586: Time spent starting tablet: real 0.020s	user 0.012s	sys 0.003s
I20260709 13:16:07.000315 10959 tablet_copy_service.cc:342] P cb0428cc55984daba1ba63744a5102a2: Request end of tablet copy session 12682eb70bc84c76879e04a8628ee586-86ba4060a26c42998e9f220a79f28546 received from {username='slave'} at 127.0.0.1:36714
I20260709 13:16:07.000767 10959 tablet_copy_service.cc:434] P cb0428cc55984daba1ba63744a5102a2: ending tablet copy session 12682eb70bc84c76879e04a8628ee586-86ba4060a26c42998e9f220a79f28546 on tablet 86ba4060a26c42998e9f220a79f28546 with peer 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:07.003527 11848 ts_tablet_manager.cc:933] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586: Initiating tablet copy from peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685)
I20260709 13:16:07.023631 11848 tablet_copy_client.cc:323] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Beginning tablet copy session from remote peer at address 127.9.116.129:39685
I20260709 13:16:07.025390 10811 tablet_copy_service.cc:140] P e01ecd008edd4aa2b6b2b47efbc2ea87: Received BeginTabletCopySession request for tablet 8cf3aebfc54a4652a83388ef9a48b1c2 from peer 12682eb70bc84c76879e04a8628ee586 ({username='slave'} at 127.0.0.1:39442)
I20260709 13:16:07.025978 10811 tablet_copy_service.cc:161] P e01ecd008edd4aa2b6b2b47efbc2ea87: Beginning new tablet copy session on tablet 8cf3aebfc54a4652a83388ef9a48b1c2 from peer 12682eb70bc84c76879e04a8628ee586 at {username='slave'} at 127.0.0.1:39442: session id = 12682eb70bc84c76879e04a8628ee586-8cf3aebfc54a4652a83388ef9a48b1c2
I20260709 13:16:07.031585 11846 tablet_bootstrap.cc:492] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:07.032523 11846 tablet_bootstrap.cc:492] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586: Bootstrap complete.
I20260709 13:16:07.033205 11846 ts_tablet_manager.cc:1403] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586: Time spent bootstrapping tablet: real 0.144s	user 0.059s	sys 0.046s
I20260709 13:16:07.034255 10811 tablet_copy_source_session.cc:215] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:07.036228 11846 raft_consensus.cc:359] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:07.037117 11846 raft_consensus.cc:740] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Initialized, Role: LEARNER
I20260709 13:16:07.037833 11846 consensus_queue.cc:260] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:07.041225 11846 ts_tablet_manager.cc:1434] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:16:07.045799 11851 ts_tablet_manager.cc:933] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586: Initiating tablet copy from peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:07.047900 11851 tablet_copy_client.cc:323] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586: tablet copy: Beginning tablet copy session from remote peer at address 127.9.116.131:45553
I20260709 13:16:07.049175 10811 tablet_copy_service.cc:342] P e01ecd008edd4aa2b6b2b47efbc2ea87: Request end of tablet copy session 12682eb70bc84c76879e04a8628ee586-f15293e178d84fe686fd66bc4439aba8 received from {username='slave'} at 127.0.0.1:39442
I20260709 13:16:07.049469 11848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cf3aebfc54a4652a83388ef9a48b1c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:07.049654 10811 tablet_copy_service.cc:434] P e01ecd008edd4aa2b6b2b47efbc2ea87: ending tablet copy session 12682eb70bc84c76879e04a8628ee586-f15293e178d84fe686fd66bc4439aba8 on tablet f15293e178d84fe686fd66bc4439aba8 with peer 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:07.049790 10959 tablet_copy_service.cc:140] P cb0428cc55984daba1ba63744a5102a2: Received BeginTabletCopySession request for tablet 76b2d93b69274dbca479bd87d7ea45cb from peer 12682eb70bc84c76879e04a8628ee586 ({username='slave'} at 127.0.0.1:36714)
I20260709 13:16:07.050348 10959 tablet_copy_service.cc:161] P cb0428cc55984daba1ba63744a5102a2: Beginning new tablet copy session on tablet 76b2d93b69274dbca479bd87d7ea45cb from peer 12682eb70bc84c76879e04a8628ee586 at {username='slave'} at 127.0.0.1:36714: session id = 12682eb70bc84c76879e04a8628ee586-76b2d93b69274dbca479bd87d7ea45cb
I20260709 13:16:07.063858 10959 tablet_copy_source_session.cc:215] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:07.069023 11848 tablet_copy_client.cc:806] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:07.069708 11848 tablet_copy_client.cc:670] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:07.069855 11851 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76b2d93b69274dbca479bd87d7ea45cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:07.073447 11848 tablet_copy_client.cc:538] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:07.082626 11848 tablet_bootstrap.cc:492] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586: Bootstrap starting.
I20260709 13:16:07.093245 11851 tablet_copy_client.cc:806] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:07.093927 11851 tablet_copy_client.cc:670] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:07.101114 11851 tablet_copy_client.cc:538] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:07.107517 11848 tablet_bootstrap.cc:492] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:07.108367 11848 tablet_bootstrap.cc:492] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586: Bootstrap complete.
I20260709 13:16:07.108975 11848 ts_tablet_manager.cc:1403] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586: Time spent bootstrapping tablet: real 0.027s	user 0.023s	sys 0.004s
I20260709 13:16:07.110569 11851 tablet_bootstrap.cc:492] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586: Bootstrap starting.
I20260709 13:16:07.111577 11848 raft_consensus.cc:359] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:07.112489 11848 raft_consensus.cc:740] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Initialized, Role: LEARNER
I20260709 13:16:07.113178 11848 consensus_queue.cc:260] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:07.119431 11848 ts_tablet_manager.cc:1434] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:16:07.122295 10811 tablet_copy_service.cc:342] P e01ecd008edd4aa2b6b2b47efbc2ea87: Request end of tablet copy session 12682eb70bc84c76879e04a8628ee586-8cf3aebfc54a4652a83388ef9a48b1c2 received from {username='slave'} at 127.0.0.1:39442
I20260709 13:16:07.122740 10811 tablet_copy_service.cc:434] P e01ecd008edd4aa2b6b2b47efbc2ea87: ending tablet copy session 12682eb70bc84c76879e04a8628ee586-8cf3aebfc54a4652a83388ef9a48b1c2 on tablet 8cf3aebfc54a4652a83388ef9a48b1c2 with peer 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:07.149659 11851 tablet_bootstrap.cc:492] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:07.150619 11851 tablet_bootstrap.cc:492] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586: Bootstrap complete.
I20260709 13:16:07.151337 11851 ts_tablet_manager.cc:1403] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586: Time spent bootstrapping tablet: real 0.041s	user 0.036s	sys 0.001s
I20260709 13:16:07.154248 11851 raft_consensus.cc:359] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:07.155150 11851 raft_consensus.cc:740] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Initialized, Role: LEARNER
I20260709 13:16:07.155792 11851 consensus_queue.cc:260] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:07.158725 11851 ts_tablet_manager.cc:1434] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:16:07.160404 10959 tablet_copy_service.cc:342] P cb0428cc55984daba1ba63744a5102a2: Request end of tablet copy session 12682eb70bc84c76879e04a8628ee586-76b2d93b69274dbca479bd87d7ea45cb received from {username='slave'} at 127.0.0.1:36714
I20260709 13:16:07.160821 10959 tablet_copy_service.cc:434] P cb0428cc55984daba1ba63744a5102a2: ending tablet copy session 12682eb70bc84c76879e04a8628ee586-76b2d93b69274dbca479bd87d7ea45cb on tablet 76b2d93b69274dbca479bd87d7ea45cb with peer 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:07.306311 11511 raft_consensus.cc:1217] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:07.325595 11511 raft_consensus.cc:1217] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:16:07.342305 11510 raft_consensus.cc:1217] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:07.421317 11511 raft_consensus.cc:1217] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:07.722110 11828 raft_consensus.cc:1064] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2: attempting to promote NON_VOTER 12682eb70bc84c76879e04a8628ee586 to VOTER
I20260709 13:16:07.725631 11828 consensus_queue.cc:237] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:07.733450 11511 raft_consensus.cc:1275] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:07.735035 11828 consensus_queue.cc:1048] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:07.735939 10794 raft_consensus.cc:1275] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:07.737294 10872 raft_consensus.cc:1275] T 86ba4060a26c42998e9f220a79f28546 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:07.737567 11817 consensus_queue.cc:1048] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:07.740024 11817 consensus_queue.cc:1048] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:07.751863 11794 raft_consensus.cc:2955] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:07.753695 11511 raft_consensus.cc:2955] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:07.756806 10871 raft_consensus.cc:2955] T 86ba4060a26c42998e9f220a79f28546 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:07.800170 10696 catalog_manager.cc:5697] T 86ba4060a26c42998e9f220a79f28546 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:07.818733 10800 raft_consensus.cc:2955] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:07.826546 11860 raft_consensus.cc:1064] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87: attempting to promote NON_VOTER 12682eb70bc84c76879e04a8628ee586 to VOTER
I20260709 13:16:07.829319 11860 consensus_queue.cc:237] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:07.846375 11511 raft_consensus.cc:1275] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 2 LEARNER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:16:07.848713 10875 raft_consensus.cc:1275] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:16:07.849064 10950 raft_consensus.cc:1275] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:16:07.850625 11834 consensus_queue.cc:1048] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:07.852988 11751 consensus_queue.cc:1048] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:07.865077 10950 consensus_queue.cc:237] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:07.875572 11834 consensus_queue.cc:1048] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 13:16:07.882961 11751 raft_consensus.cc:1064] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87: attempting to promote NON_VOTER 12682eb70bc84c76879e04a8628ee586 to VOTER
I20260709 13:16:07.889705 11751 consensus_queue.cc:237] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:07.913818 11512 raft_consensus.cc:1275] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:07.923745 11794 consensus_queue.cc:1048] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:07.929759 10875 raft_consensus.cc:1275] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:07.933368 11878 consensus_queue.cc:1048] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:07.934445 10800 raft_consensus.cc:1275] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:07.937695 11817 raft_consensus.cc:2955] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:07.941032 10950 raft_consensus.cc:1275] T f15293e178d84fe686fd66bc4439aba8 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:07.940537 11507 raft_consensus.cc:2955] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:07.942205 11874 raft_consensus.cc:2955] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:07.947576 11511 raft_consensus.cc:1275] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:07.948468 11869 consensus_queue.cc:1048] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:07.954723 11511 raft_consensus.cc:2955] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:07.973655 10698 catalog_manager.cc:5697] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: config changed from index 3 to 4, VOTER 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) evicted. New cstate: current_term: 1 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:07.981017 10794 raft_consensus.cc:2955] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:07.985085 10685 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 86ba4060a26c42998e9f220a79f28546 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:07.987936 10863 tablet_service.cc:1558] Processing DeleteTablet for tablet 86ba4060a26c42998e9f220a79f28546 with delete_type TABLET_DATA_TOMBSTONED (TS 468a8fd46e824f62b9c2c0d3532a6d26 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54946
I20260709 13:16:07.988862 11751 consensus_queue.cc:1048] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:07.989972 11878 consensus_queue.cc:1048] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:07.999960 10872 raft_consensus.cc:2955] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:08.017817 11794 raft_consensus.cc:1064] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2: attempting to promote NON_VOTER 12682eb70bc84c76879e04a8628ee586 to VOTER
I20260709 13:16:08.014571 10698 catalog_manager.cc:5697] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: config changed from index 3 to 4, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:08.051548 11794 consensus_queue.cc:237] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:08.071997 10950 raft_consensus.cc:2955] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:08.074518 11507 raft_consensus.cc:2955] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:08.072705 11877 raft_consensus.cc:2955] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:08.107673 10872 raft_consensus.cc:1275] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:08.107776 11511 raft_consensus.cc:1275] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:08.110066 11869 consensus_queue.cc:1048] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:08.115062 11884 consensus_queue.cc:1048] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:08.115969 10950 raft_consensus.cc:2955] T f15293e178d84fe686fd66bc4439aba8 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:08.119380 10797 raft_consensus.cc:1275] T 76b2d93b69274dbca479bd87d7ea45cb P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:08.122258 11882 tablet_replica.cc:333] T 86ba4060a26c42998e9f220a79f28546 P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:16:08.129226 10871 raft_consensus.cc:2955] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:08.130642 11869 consensus_queue.cc:1048] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:08.145390 10696 catalog_manager.cc:5697] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:08.206084 11885 raft_consensus.cc:2955] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:08.208384 10794 raft_consensus.cc:2955] T 76b2d93b69274dbca479bd87d7ea45cb P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:08.214602 11882 raft_consensus.cc:2243] T 86ba4060a26c42998e9f220a79f28546 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:08.215628 11882 raft_consensus.cc:2272] T 86ba4060a26c42998e9f220a79f28546 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.223439 11882 ts_tablet_manager.cc:1916] T 86ba4060a26c42998e9f220a79f28546 P 468a8fd46e824f62b9c2c0d3532a6d26: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:08.226260 10797 consensus_queue.cc:237] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:08.233305 11511 raft_consensus.cc:1275] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:08.234925 11874 consensus_queue.cc:1048] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:08.240073 10948 raft_consensus.cc:1275] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:08.245849 11877 raft_consensus.cc:2955] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:08.255754 11511 raft_consensus.cc:2955] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:08.260082 10871 raft_consensus.cc:2955] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:08.266397 11878 consensus_queue.cc:1048] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:08.274960 10697 catalog_manager.cc:5697] T 76b2d93b69274dbca479bd87d7ea45cb P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:08.285130 11509 raft_consensus.cc:2955] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:08.297003 10948 consensus_queue.cc:237] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:08.304562 10686 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8cf3aebfc54a4652a83388ef9a48b1c2 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:08.318455 10948 raft_consensus.cc:2955] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:08.320444 10696 catalog_manager.cc:5697] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: config changed from index 4 to 5, VOTER 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) evicted. New cstate: current_term: 2 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:08.323356 10871 raft_consensus.cc:1275] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:08.326824 11882 ts_tablet_manager.cc:1929] T 86ba4060a26c42998e9f220a79f28546 P 468a8fd46e824f62b9c2c0d3532a6d26: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:08.327400 11882 log.cc:1199] T 86ba4060a26c42998e9f220a79f28546 P 468a8fd46e824f62b9c2c0d3532a6d26: Deleting WAL directory at /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-1-root/wals/86ba4060a26c42998e9f220a79f28546
I20260709 13:16:08.325536 11884 consensus_queue.cc:1048] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 13:16:08.325137 10797 consensus_queue.cc:237] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:08.334722 11509 raft_consensus.cc:1275] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:08.335080 11885 raft_consensus.cc:2955] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:08.338639 11794 consensus_queue.cc:1048] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:08.343147 11507 raft_consensus.cc:2955] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:08.349089 10685 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 76b2d93b69274dbca479bd87d7ea45cb with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:08.352996 10872 raft_consensus.cc:2955] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:08.335151 10686 catalog_manager.cc:5028] TS 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): tablet 86ba4060a26c42998e9f220a79f28546 (table test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]) successfully deleted
I20260709 13:16:08.360746 10871 raft_consensus.cc:1275] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:08.362332 11511 raft_consensus.cc:1275] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:08.363123 11873 consensus_queue.cc:1048] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:08.367061 10863 tablet_service.cc:1558] Processing DeleteTablet for tablet 8cf3aebfc54a4652a83388ef9a48b1c2 with delete_type TABLET_DATA_TOMBSTONED (TS 468a8fd46e824f62b9c2c0d3532a6d26 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:54946
I20260709 13:16:08.372006 11873 consensus_queue.cc:1048] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:08.373374 10698 catalog_manager.cc:5697] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 reported cstate change: config changed from index 3 to 4, VOTER e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) evicted. New cstate: current_term: 1 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:08.373970 11882 tablet_replica.cc:333] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:16:08.376006 11874 raft_consensus.cc:2955] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:08.377758 11882 raft_consensus.cc:2243] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:08.379794 11882 raft_consensus.cc:2272] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.385279 11882 ts_tablet_manager.cc:1916] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 468a8fd46e824f62b9c2c0d3532a6d26: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:08.387698 10871 raft_consensus.cc:2955] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:08.391763 10686 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f15293e178d84fe686fd66bc4439aba8 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:08.392956 11511 raft_consensus.cc:2955] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:08.401042 10699 catalog_manager.cc:5697] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: config changed from index 3 to 4, VOTER cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) evicted. New cstate: current_term: 1 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:08.411270 10785 tablet_service.cc:1558] Processing DeleteTablet for tablet 76b2d93b69274dbca479bd87d7ea45cb with delete_type TABLET_DATA_TOMBSTONED (TS e01ecd008edd4aa2b6b2b47efbc2ea87 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46752
I20260709 13:16:08.426769 10937 tablet_service.cc:1558] Processing DeleteTablet for tablet f15293e178d84fe686fd66bc4439aba8 with delete_type TABLET_DATA_TOMBSTONED (TS cb0428cc55984daba1ba63744a5102a2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41006
I20260709 13:16:08.433748 11892 tablet_replica.cc:333] T f15293e178d84fe686fd66bc4439aba8 P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:16:08.435673 11892 raft_consensus.cc:2243] T f15293e178d84fe686fd66bc4439aba8 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:08.436877 11892 raft_consensus.cc:2272] T f15293e178d84fe686fd66bc4439aba8 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.441126 11892 ts_tablet_manager.cc:1916] T f15293e178d84fe686fd66bc4439aba8 P cb0428cc55984daba1ba63744a5102a2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:08.453199 11891 tablet_replica.cc:333] T 76b2d93b69274dbca479bd87d7ea45cb P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:16:08.454454 11891 raft_consensus.cc:2243] T 76b2d93b69274dbca479bd87d7ea45cb P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:08.455389 11891 raft_consensus.cc:2272] T 76b2d93b69274dbca479bd87d7ea45cb P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.462941 11891 ts_tablet_manager.cc:1916] T 76b2d93b69274dbca479bd87d7ea45cb P e01ecd008edd4aa2b6b2b47efbc2ea87: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:08.467296 11892 ts_tablet_manager.cc:1929] T f15293e178d84fe686fd66bc4439aba8 P cb0428cc55984daba1ba63744a5102a2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:08.468004 11892 log.cc:1199] T f15293e178d84fe686fd66bc4439aba8 P cb0428cc55984daba1ba63744a5102a2: Deleting WAL directory at /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-2-root/wals/f15293e178d84fe686fd66bc4439aba8
I20260709 13:16:08.469133 11882 ts_tablet_manager.cc:1929] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 468a8fd46e824f62b9c2c0d3532a6d26: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:16:08.469626 11882 log.cc:1199] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 468a8fd46e824f62b9c2c0d3532a6d26: Deleting WAL directory at /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-1-root/wals/8cf3aebfc54a4652a83388ef9a48b1c2
I20260709 13:16:08.476851 10686 catalog_manager.cc:5028] TS 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): tablet 8cf3aebfc54a4652a83388ef9a48b1c2 (table test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]) successfully deleted
I20260709 13:16:08.478381 10685 catalog_manager.cc:5028] TS cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): tablet f15293e178d84fe686fd66bc4439aba8 (table test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]) successfully deleted
I20260709 13:16:08.478978 11891 ts_tablet_manager.cc:1929] T 76b2d93b69274dbca479bd87d7ea45cb P e01ecd008edd4aa2b6b2b47efbc2ea87: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:08.479903 11891 log.cc:1199] T 76b2d93b69274dbca479bd87d7ea45cb P e01ecd008edd4aa2b6b2b47efbc2ea87: Deleting WAL directory at /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-0-root/wals/76b2d93b69274dbca479bd87d7ea45cb
I20260709 13:16:08.483554 10686 catalog_manager.cc:5028] TS e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): tablet 76b2d93b69274dbca479bd87d7ea45cb (table test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]) successfully deleted
I20260709 13:16:09.508077 10797 consensus_queue.cc:237] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:09.514102 10948 raft_consensus.cc:1275] T 5c8a233bc6c44bd2b5d2d44a726b147c P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:09.515305 10871 raft_consensus.cc:1275] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:09.515524 11874 consensus_queue.cc:1048] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:16:09.516645 10766 consensus_peers.cc:597] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 -> Peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Couldn't send request to peer 12682eb70bc84c76879e04a8628ee586. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5c8a233bc6c44bd2b5d2d44a726b147c. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:09.517275 11873 consensus_queue.cc:1048] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:09.523772 11874 raft_consensus.cc:2955] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:09.527091 10948 raft_consensus.cc:2955] T 5c8a233bc6c44bd2b5d2d44a726b147c P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:09.527974 10871 raft_consensus.cc:2955] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:09.539853 10797 consensus_queue.cc:237] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:09.555271 10699 catalog_manager.cc:5697] T 5c8a233bc6c44bd2b5d2d44a726b147c P cb0428cc55984daba1ba63744a5102a2 reported cstate change: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New cstate: current_term: 1 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:09.566931 10872 raft_consensus.cc:1275] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:16:09.567193 10766 consensus_peers.cc:597] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 -> Peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Couldn't send request to peer 12682eb70bc84c76879e04a8628ee586. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4dd4c0ff1def4960931dda3b52a4a884. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:09.567257 10948 raft_consensus.cc:1275] T 4dd4c0ff1def4960931dda3b52a4a884 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:09.569823 11893 consensus_queue.cc:1048] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:09.571233 11874 consensus_queue.cc:1048] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:09.588699 11873 raft_consensus.cc:2955] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:09.590333 10871 raft_consensus.cc:2955] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:09.590148 10948 raft_consensus.cc:2955] T 4dd4c0ff1def4960931dda3b52a4a884 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:09.606076 10797 consensus_queue.cc:237] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:09.613334 10698 catalog_manager.cc:5697] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New cstate: current_term: 1 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:09.617636 10948 raft_consensus.cc:1275] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:16:09.618980 10766 consensus_peers.cc:597] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 -> Peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Couldn't send request to peer 12682eb70bc84c76879e04a8628ee586. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0a574a3bcb5441568f4fd9b1c38d719a. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:09.619268 10871 raft_consensus.cc:1275] T 0a574a3bcb5441568f4fd9b1c38d719a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:09.619913 11873 consensus_queue.cc:1048] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:09.620914 11897 consensus_queue.cc:1048] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:09.630263 11897 raft_consensus.cc:2955] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:09.632475 10948 raft_consensus.cc:2955] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:09.632138 10871 raft_consensus.cc:2955] T 0a574a3bcb5441568f4fd9b1c38d719a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:09.649223 10696 catalog_manager.cc:5697] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New cstate: current_term: 1 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:09.654079 10872 consensus_queue.cc:237] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:09.669341 10948 raft_consensus.cc:1275] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:09.670812 11861 consensus_queue.cc:1048] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:09.677943 10800 raft_consensus.cc:1275] T 37a1fc8ca1b34260908d185a9ebc2bb3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:09.679909 11899 consensus_queue.cc:1048] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:16:09.683784 10841 consensus_peers.cc:597] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Couldn't send request to peer 12682eb70bc84c76879e04a8628ee586. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 37a1fc8ca1b34260908d185a9ebc2bb3. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:09.692188 11911 raft_consensus.cc:2955] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:09.693897 10948 raft_consensus.cc:2955] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:09.696280 10800 raft_consensus.cc:2955] T 37a1fc8ca1b34260908d185a9ebc2bb3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:09.716413 10699 catalog_manager.cc:5697] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:10.033494 11914 ts_tablet_manager.cc:933] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586: Initiating tablet copy from peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685)
I20260709 13:16:10.036765 11914 tablet_copy_client.cc:323] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586: tablet copy: Beginning tablet copy session from remote peer at address 127.9.116.129:39685
I20260709 13:16:10.038594 10811 tablet_copy_service.cc:140] P e01ecd008edd4aa2b6b2b47efbc2ea87: Received BeginTabletCopySession request for tablet 0a574a3bcb5441568f4fd9b1c38d719a from peer 12682eb70bc84c76879e04a8628ee586 ({username='slave'} at 127.0.0.1:39442)
I20260709 13:16:10.039191 10811 tablet_copy_service.cc:161] P e01ecd008edd4aa2b6b2b47efbc2ea87: Beginning new tablet copy session on tablet 0a574a3bcb5441568f4fd9b1c38d719a from peer 12682eb70bc84c76879e04a8628ee586 at {username='slave'} at 127.0.0.1:39442: session id = 12682eb70bc84c76879e04a8628ee586-0a574a3bcb5441568f4fd9b1c38d719a
I20260709 13:16:10.046077 10811 tablet_copy_source_session.cc:215] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:10.052800 11914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a574a3bcb5441568f4fd9b1c38d719a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.070423 11914 tablet_copy_client.cc:806] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:10.071054 11914 tablet_copy_client.cc:670] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:10.074959 11914 tablet_copy_client.cc:538] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:10.083500 11914 tablet_bootstrap.cc:492] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586: Bootstrap starting.
I20260709 13:16:10.089658 11916 ts_tablet_manager.cc:933] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586: Initiating tablet copy from peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685)
I20260709 13:16:10.099452 11916 tablet_copy_client.cc:323] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586: tablet copy: Beginning tablet copy session from remote peer at address 127.9.116.129:39685
I20260709 13:16:10.101250 10811 tablet_copy_service.cc:140] P e01ecd008edd4aa2b6b2b47efbc2ea87: Received BeginTabletCopySession request for tablet 5c8a233bc6c44bd2b5d2d44a726b147c from peer 12682eb70bc84c76879e04a8628ee586 ({username='slave'} at 127.0.0.1:39442)
I20260709 13:16:10.101809 10811 tablet_copy_service.cc:161] P e01ecd008edd4aa2b6b2b47efbc2ea87: Beginning new tablet copy session on tablet 5c8a233bc6c44bd2b5d2d44a726b147c from peer 12682eb70bc84c76879e04a8628ee586 at {username='slave'} at 127.0.0.1:39442: session id = 12682eb70bc84c76879e04a8628ee586-5c8a233bc6c44bd2b5d2d44a726b147c
I20260709 13:16:10.107474 11914 tablet_bootstrap.cc:492] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:10.108479 11914 tablet_bootstrap.cc:492] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586: Bootstrap complete.
I20260709 13:16:10.109179 11914 ts_tablet_manager.cc:1403] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586: Time spent bootstrapping tablet: real 0.026s	user 0.018s	sys 0.004s
I20260709 13:16:10.110030 10811 tablet_copy_source_session.cc:215] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:10.112715 11914 raft_consensus.cc:359] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:10.113511 11916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c8a233bc6c44bd2b5d2d44a726b147c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.113667 11914 raft_consensus.cc:740] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Initialized, Role: LEARNER
I20260709 13:16:10.114405 11914 consensus_queue.cc:260] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:10.124475 11914 ts_tablet_manager.cc:1434] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586: Time spent starting tablet: real 0.015s	user 0.015s	sys 0.000s
I20260709 13:16:10.126569 10811 tablet_copy_service.cc:342] P e01ecd008edd4aa2b6b2b47efbc2ea87: Request end of tablet copy session 12682eb70bc84c76879e04a8628ee586-0a574a3bcb5441568f4fd9b1c38d719a received from {username='slave'} at 127.0.0.1:39442
I20260709 13:16:10.127112 10811 tablet_copy_service.cc:434] P e01ecd008edd4aa2b6b2b47efbc2ea87: ending tablet copy session 12682eb70bc84c76879e04a8628ee586-0a574a3bcb5441568f4fd9b1c38d719a on tablet 0a574a3bcb5441568f4fd9b1c38d719a with peer 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:10.133360 11916 tablet_copy_client.cc:806] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:10.133949 11916 tablet_copy_client.cc:670] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:10.143972 11916 tablet_copy_client.cc:538] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:10.156266 11920 ts_tablet_manager.cc:933] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586: Initiating tablet copy from peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685)
I20260709 13:16:10.158946 11916 tablet_bootstrap.cc:492] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586: Bootstrap starting.
I20260709 13:16:10.176149 11920 tablet_copy_client.cc:323] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Beginning tablet copy session from remote peer at address 127.9.116.129:39685
I20260709 13:16:10.178076 10811 tablet_copy_service.cc:140] P e01ecd008edd4aa2b6b2b47efbc2ea87: Received BeginTabletCopySession request for tablet 4dd4c0ff1def4960931dda3b52a4a884 from peer 12682eb70bc84c76879e04a8628ee586 ({username='slave'} at 127.0.0.1:39442)
I20260709 13:16:10.178728 10811 tablet_copy_service.cc:161] P e01ecd008edd4aa2b6b2b47efbc2ea87: Beginning new tablet copy session on tablet 4dd4c0ff1def4960931dda3b52a4a884 from peer 12682eb70bc84c76879e04a8628ee586 at {username='slave'} at 127.0.0.1:39442: session id = 12682eb70bc84c76879e04a8628ee586-4dd4c0ff1def4960931dda3b52a4a884
I20260709 13:16:10.195747 11916 tablet_bootstrap.cc:492] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:10.196691 11916 tablet_bootstrap.cc:492] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586: Bootstrap complete.
I20260709 13:16:10.197364 11916 ts_tablet_manager.cc:1403] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586: Time spent bootstrapping tablet: real 0.039s	user 0.021s	sys 0.001s
I20260709 13:16:10.200343 11916 raft_consensus.cc:359] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:10.201143 11916 raft_consensus.cc:740] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Initialized, Role: LEARNER
I20260709 13:16:10.201793 11916 consensus_queue.cc:260] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:10.204227 11916 ts_tablet_manager.cc:1434] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:16:10.205976 10810 tablet_copy_service.cc:342] P e01ecd008edd4aa2b6b2b47efbc2ea87: Request end of tablet copy session 12682eb70bc84c76879e04a8628ee586-5c8a233bc6c44bd2b5d2d44a726b147c received from {username='slave'} at 127.0.0.1:39442
I20260709 13:16:10.206418 10810 tablet_copy_service.cc:434] P e01ecd008edd4aa2b6b2b47efbc2ea87: ending tablet copy session 12682eb70bc84c76879e04a8628ee586-5c8a233bc6c44bd2b5d2d44a726b147c on tablet 5c8a233bc6c44bd2b5d2d44a726b147c with peer 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:10.212517 10811 tablet_copy_source_session.cc:215] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:10.216641 11920 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4dd4c0ff1def4960931dda3b52a4a884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.235270 11920 tablet_copy_client.cc:806] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:10.235882 11920 tablet_copy_client.cc:670] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:10.239696 11920 tablet_copy_client.cc:538] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:10.254493 11920 tablet_bootstrap.cc:492] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586: Bootstrap starting.
I20260709 13:16:10.279440 11920 tablet_bootstrap.cc:492] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:10.281635 11920 tablet_bootstrap.cc:492] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586: Bootstrap complete.
I20260709 13:16:10.282325 11920 ts_tablet_manager.cc:1403] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586: Time spent bootstrapping tablet: real 0.028s	user 0.014s	sys 0.012s
I20260709 13:16:10.283764 11916 ts_tablet_manager.cc:933] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586: Initiating tablet copy from peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:10.285800 11916 tablet_copy_client.cc:323] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Beginning tablet copy session from remote peer at address 127.9.116.130:37895
I20260709 13:16:10.285518 11920 raft_consensus.cc:359] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:10.286485 11920 raft_consensus.cc:740] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Initialized, Role: LEARNER
I20260709 13:16:10.287196 11920 consensus_queue.cc:260] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:10.288060 10886 tablet_copy_service.cc:140] P 468a8fd46e824f62b9c2c0d3532a6d26: Received BeginTabletCopySession request for tablet 37a1fc8ca1b34260908d185a9ebc2bb3 from peer 12682eb70bc84c76879e04a8628ee586 ({username='slave'} at 127.0.0.1:49290)
I20260709 13:16:10.288647 10886 tablet_copy_service.cc:161] P 468a8fd46e824f62b9c2c0d3532a6d26: Beginning new tablet copy session on tablet 37a1fc8ca1b34260908d185a9ebc2bb3 from peer 12682eb70bc84c76879e04a8628ee586 at {username='slave'} at 127.0.0.1:49290: session id = 12682eb70bc84c76879e04a8628ee586-37a1fc8ca1b34260908d185a9ebc2bb3
I20260709 13:16:10.296042 11920 ts_tablet_manager.cc:1434] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.001s
I20260709 13:16:10.297947 10811 tablet_copy_service.cc:342] P e01ecd008edd4aa2b6b2b47efbc2ea87: Request end of tablet copy session 12682eb70bc84c76879e04a8628ee586-4dd4c0ff1def4960931dda3b52a4a884 received from {username='slave'} at 127.0.0.1:39442
I20260709 13:16:10.298404 10811 tablet_copy_service.cc:434] P e01ecd008edd4aa2b6b2b47efbc2ea87: ending tablet copy session 12682eb70bc84c76879e04a8628ee586-4dd4c0ff1def4960931dda3b52a4a884 on tablet 4dd4c0ff1def4960931dda3b52a4a884 with peer 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:10.301800 10886 tablet_copy_source_session.cc:215] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:10.306392 11916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37a1fc8ca1b34260908d185a9ebc2bb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.322117 11916 tablet_copy_client.cc:806] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:10.322765 11916 tablet_copy_client.cc:670] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:10.326864 11916 tablet_copy_client.cc:538] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:10.336464 11916 tablet_bootstrap.cc:492] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586: Bootstrap starting.
I20260709 13:16:10.357023 11916 tablet_bootstrap.cc:492] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:10.357956 11916 tablet_bootstrap.cc:492] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586: Bootstrap complete.
I20260709 13:16:10.358647 11916 ts_tablet_manager.cc:1403] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.005s
I20260709 13:16:10.361680 11916 raft_consensus.cc:359] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:10.362551 11916 raft_consensus.cc:740] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Initialized, Role: LEARNER
I20260709 13:16:10.363198 11916 consensus_queue.cc:260] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:10.365638 11916 ts_tablet_manager.cc:1434] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:16:10.367386 10886 tablet_copy_service.cc:342] P 468a8fd46e824f62b9c2c0d3532a6d26: Request end of tablet copy session 12682eb70bc84c76879e04a8628ee586-37a1fc8ca1b34260908d185a9ebc2bb3 received from {username='slave'} at 127.0.0.1:49290
I20260709 13:16:10.367834 10886 tablet_copy_service.cc:434] P 468a8fd46e824f62b9c2c0d3532a6d26: ending tablet copy session 12682eb70bc84c76879e04a8628ee586-37a1fc8ca1b34260908d185a9ebc2bb3 on tablet 37a1fc8ca1b34260908d185a9ebc2bb3 with peer 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:10.413488 11511 raft_consensus.cc:1217] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:10.654155 11511 raft_consensus.cc:1217] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:10.685609 11511 raft_consensus.cc:1217] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:10.707945 11511 raft_consensus.cc:1217] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:10.979338 11873 raft_consensus.cc:1064] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87: attempting to promote NON_VOTER 12682eb70bc84c76879e04a8628ee586 to VOTER
I20260709 13:16:10.981819 11873 consensus_queue.cc:237] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:10.989034 10950 raft_consensus.cc:1275] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:10.991160 11893 consensus_queue.cc:1048] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:10.991632 11511 raft_consensus.cc:1275] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:10.993171 11897 consensus_queue.cc:1048] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:10.994135 10875 raft_consensus.cc:1275] T 0a574a3bcb5441568f4fd9b1c38d719a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:11.000382 11927 consensus_queue.cc:1048] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:11.013832 11893 raft_consensus.cc:2955] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:11.019649 10875 raft_consensus.cc:2955] T 0a574a3bcb5441568f4fd9b1c38d719a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:11.020259 10950 raft_consensus.cc:2955] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:11.037312 11511 raft_consensus.cc:2955] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:11.041667 10698 catalog_manager.cc:5697] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 reported cstate change: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:11.093515 10794 consensus_queue.cc:237] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:11.101228 11509 raft_consensus.cc:1275] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:11.103020 10950 raft_consensus.cc:1275] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:11.102821 11897 consensus_queue.cc:1048] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:11.104645 11893 consensus_queue.cc:1048] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:11.112428 11873 raft_consensus.cc:2955] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:11.114080 11507 raft_consensus.cc:2955] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:11.121170 10950 raft_consensus.cc:2955] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:11.123889 10686 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0a574a3bcb5441568f4fd9b1c38d719a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:11.129889 11897 raft_consensus.cc:1064] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87: attempting to promote NON_VOTER 12682eb70bc84c76879e04a8628ee586 to VOTER
I20260709 13:16:11.132580 11897 consensus_queue.cc:237] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:11.140630 10697 catalog_manager.cc:5697] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: config changed from index 3 to 4, VOTER 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) evicted. New cstate: current_term: 1 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:11.149443 11511 raft_consensus.cc:1275] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:11.154241 10948 raft_consensus.cc:1275] T 4dd4c0ff1def4960931dda3b52a4a884 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:11.160337 10863 tablet_service.cc:1558] Processing DeleteTablet for tablet 0a574a3bcb5441568f4fd9b1c38d719a with delete_type TABLET_DATA_TOMBSTONED (TS 468a8fd46e824f62b9c2c0d3532a6d26 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54946
I20260709 13:16:11.164311 11937 consensus_queue.cc:1048] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:11.172315 11893 consensus_queue.cc:1048] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:11.178079 11942 raft_consensus.cc:1064] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87: attempting to promote NON_VOTER 12682eb70bc84c76879e04a8628ee586 to VOTER
I20260709 13:16:11.187624 11941 tablet_replica.cc:333] T 0a574a3bcb5441568f4fd9b1c38d719a P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:16:11.190493 11941 raft_consensus.cc:2243] T 0a574a3bcb5441568f4fd9b1c38d719a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:11.191972 11942 consensus_queue.cc:237] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:11.191972 11941 raft_consensus.cc:2272] T 0a574a3bcb5441568f4fd9b1c38d719a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:11.206902 11941 ts_tablet_manager.cc:1916] T 0a574a3bcb5441568f4fd9b1c38d719a P 468a8fd46e824f62b9c2c0d3532a6d26: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 13:16:11.212653 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 46271.3us
I20260709 13:16:11.214685 10875 raft_consensus.cc:1275] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:11.225116 11507 raft_consensus.cc:1275] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:11.226707 11937 consensus_queue.cc:1048] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:11.230255 10949 raft_consensus.cc:1275] T 5c8a233bc6c44bd2b5d2d44a726b147c P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:11.230710 10875 raft_consensus.cc:1275] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:11.230469 11942 consensus_queue.cc:1048] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:11.233168 11942 consensus_queue.cc:1048] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:11.256143 11897 consensus_queue.cc:1048] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:11.271118 11937 raft_consensus.cc:2955] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:11.277043 10872 raft_consensus.cc:2955] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:11.278242 11826 raft_consensus.cc:1064] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26: attempting to promote NON_VOTER 12682eb70bc84c76879e04a8628ee586 to VOTER
I20260709 13:16:11.280929 11826 consensus_queue.cc:237] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:11.294940 10697 catalog_manager.cc:5697] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:11.297226 11507 raft_consensus.cc:2955] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:11.335071 10949 raft_consensus.cc:1275] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:11.335542 11511 raft_consensus.cc:1275] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:11.337788 11899 consensus_queue.cc:1048] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:11.341259 10797 raft_consensus.cc:1275] T 37a1fc8ca1b34260908d185a9ebc2bb3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:11.343631 11899 consensus_queue.cc:1048] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:11.346349 11911 consensus_queue.cc:1048] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:11.384248 10797 raft_consensus.cc:2955] T 37a1fc8ca1b34260908d185a9ebc2bb3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
W20260709 13:16:11.387974 11949 log.cc:927] Time spent T 5c8a233bc6c44bd2b5d2d44a726b147c P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:16:11.386727 10949 raft_consensus.cc:2955] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:11.392149 11937 raft_consensus.cc:2955] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:11.401412 10950 raft_consensus.cc:2955] T 5c8a233bc6c44bd2b5d2d44a726b147c P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:11.383175 11948 raft_consensus.cc:2955] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:11.418984 10948 raft_consensus.cc:2955] T 4dd4c0ff1def4960931dda3b52a4a884 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:11.424994 11511 raft_consensus.cc:2955] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:11.437397 10872 raft_consensus.cc:2955] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:11.441309 11941 ts_tablet_manager.cc:1929] T 0a574a3bcb5441568f4fd9b1c38d719a P 468a8fd46e824f62b9c2c0d3532a6d26: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:11.442001 11941 log.cc:1199] T 0a574a3bcb5441568f4fd9b1c38d719a P 468a8fd46e824f62b9c2c0d3532a6d26: Deleting WAL directory at /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-1-root/wals/0a574a3bcb5441568f4fd9b1c38d719a
I20260709 13:16:11.444394 11507 raft_consensus.cc:2955] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:11.451947 10686 catalog_manager.cc:5028] TS 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): tablet 0a574a3bcb5441568f4fd9b1c38d719a (table test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]) successfully deleted
I20260709 13:16:11.468026 10696 catalog_manager.cc:5697] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:11.472587 10797 consensus_queue.cc:237] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:11.473914 10696 catalog_manager.cc:5697] T 5c8a233bc6c44bd2b5d2d44a726b147c P cb0428cc55984daba1ba63744a5102a2 reported cstate change: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:11.485580 11511 raft_consensus.cc:1275] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:11.485765 10872 raft_consensus.cc:1275] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:11.487697 11939 consensus_queue.cc:1048] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:11.494354 11939 consensus_queue.cc:1048] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:11.497673 11939 raft_consensus.cc:2955] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:11.499457 10875 raft_consensus.cc:2955] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:11.509961 11511 raft_consensus.cc:2955] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:11.515013 10686 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4dd4c0ff1def4960931dda3b52a4a884 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:11.518891 10875 consensus_queue.cc:237] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:11.523729 10800 consensus_queue.cc:237] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:11.526188 11507 raft_consensus.cc:1275] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:11.527755 10949 raft_consensus.cc:1275] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:11.530133 11826 consensus_queue.cc:1048] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:11.531852 11826 consensus_queue.cc:1048] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:11.543689 10698 catalog_manager.cc:5697] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: config changed from index 3 to 4, VOTER cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) evicted. New cstate: current_term: 1 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:11.551623 11955 raft_consensus.cc:2955] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:11.557767 10937 tablet_service.cc:1558] Processing DeleteTablet for tablet 4dd4c0ff1def4960931dda3b52a4a884 with delete_type TABLET_DATA_TOMBSTONED (TS cb0428cc55984daba1ba63744a5102a2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41006
I20260709 13:16:11.560355 10872 raft_consensus.cc:1275] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:11.560792 11508 raft_consensus.cc:2955] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:11.562125 11937 consensus_queue.cc:1048] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:11.556955 11507 raft_consensus.cc:1275] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:11.566440 11937 consensus_queue.cc:1048] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:11.570664 11897 raft_consensus.cc:2955] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:11.573195 10949 raft_consensus.cc:2955] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:11.579056 11507 raft_consensus.cc:2955] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:11.587294 10872 raft_consensus.cc:2955] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:11.593360 10686 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 5c8a233bc6c44bd2b5d2d44a726b147c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:11.625465 11956 tablet_replica.cc:333] T 4dd4c0ff1def4960931dda3b52a4a884 P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:16:11.628944 11956 raft_consensus.cc:2243] T 4dd4c0ff1def4960931dda3b52a4a884 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:11.630167 11956 raft_consensus.cc:2272] T 4dd4c0ff1def4960931dda3b52a4a884 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:11.639006 10686 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 37a1fc8ca1b34260908d185a9ebc2bb3 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:11.646063 11956 ts_tablet_manager.cc:1916] T 4dd4c0ff1def4960931dda3b52a4a884 P cb0428cc55984daba1ba63744a5102a2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:11.663375 11956 ts_tablet_manager.cc:1929] T 4dd4c0ff1def4960931dda3b52a4a884 P cb0428cc55984daba1ba63744a5102a2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:11.663889 11956 log.cc:1199] T 4dd4c0ff1def4960931dda3b52a4a884 P cb0428cc55984daba1ba63744a5102a2: Deleting WAL directory at /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-2-root/wals/4dd4c0ff1def4960931dda3b52a4a884
I20260709 13:16:11.658888 10698 catalog_manager.cc:5697] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: config changed from index 3 to 4, VOTER cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) evicted. New cstate: current_term: 1 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:11.665465 10685 catalog_manager.cc:5028] TS cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): tablet 4dd4c0ff1def4960931dda3b52a4a884 (table test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]) successfully deleted
I20260709 13:16:11.673262 10696 catalog_manager.cc:5697] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: config changed from index 3 to 4, VOTER e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) evicted. New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:11.674399 10937 tablet_service.cc:1558] Processing DeleteTablet for tablet 5c8a233bc6c44bd2b5d2d44a726b147c with delete_type TABLET_DATA_TOMBSTONED (TS cb0428cc55984daba1ba63744a5102a2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41006
I20260709 13:16:11.677680 11956 tablet_replica.cc:333] T 5c8a233bc6c44bd2b5d2d44a726b147c P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:16:11.679311 11956 raft_consensus.cc:2243] T 5c8a233bc6c44bd2b5d2d44a726b147c P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:11.680172 11956 raft_consensus.cc:2272] T 5c8a233bc6c44bd2b5d2d44a726b147c P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:11.684110 11956 ts_tablet_manager.cc:1916] T 5c8a233bc6c44bd2b5d2d44a726b147c P cb0428cc55984daba1ba63744a5102a2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:11.693068 10785 tablet_service.cc:1558] Processing DeleteTablet for tablet 37a1fc8ca1b34260908d185a9ebc2bb3 with delete_type TABLET_DATA_TOMBSTONED (TS e01ecd008edd4aa2b6b2b47efbc2ea87 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46752
I20260709 13:16:11.695729 11958 tablet_replica.cc:333] T 37a1fc8ca1b34260908d185a9ebc2bb3 P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:16:11.696595 11958 raft_consensus.cc:2243] T 37a1fc8ca1b34260908d185a9ebc2bb3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:11.697273 11958 raft_consensus.cc:2272] T 37a1fc8ca1b34260908d185a9ebc2bb3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:11.701692 11958 ts_tablet_manager.cc:1916] T 37a1fc8ca1b34260908d185a9ebc2bb3 P e01ecd008edd4aa2b6b2b47efbc2ea87: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:11.707561 11956 ts_tablet_manager.cc:1929] T 5c8a233bc6c44bd2b5d2d44a726b147c P cb0428cc55984daba1ba63744a5102a2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:11.708141 11956 log.cc:1199] T 5c8a233bc6c44bd2b5d2d44a726b147c P cb0428cc55984daba1ba63744a5102a2: Deleting WAL directory at /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-2-root/wals/5c8a233bc6c44bd2b5d2d44a726b147c
I20260709 13:16:11.709944 10685 catalog_manager.cc:5028] TS cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): tablet 5c8a233bc6c44bd2b5d2d44a726b147c (table test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]) successfully deleted
I20260709 13:16:11.720253 11958 ts_tablet_manager.cc:1929] T 37a1fc8ca1b34260908d185a9ebc2bb3 P e01ecd008edd4aa2b6b2b47efbc2ea87: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:11.720678 11958 log.cc:1199] T 37a1fc8ca1b34260908d185a9ebc2bb3 P e01ecd008edd4aa2b6b2b47efbc2ea87: Deleting WAL directory at /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-0-root/wals/37a1fc8ca1b34260908d185a9ebc2bb3
I20260709 13:16:11.722008 10686 catalog_manager.cc:5028] TS e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): tablet 37a1fc8ca1b34260908d185a9ebc2bb3 (table test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]) successfully deleted
I20260709 13:16:12.758977 10872 consensus_queue.cc:237] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:12.767055 10800 raft_consensus.cc:1275] T 414992c8073646798991d2b43fd61f1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:12.767048 10950 raft_consensus.cc:1275] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:12.769347 11961 consensus_queue.cc:1048] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:12.770288 11911 consensus_queue.cc:1048] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:16:12.777441 10841 consensus_peers.cc:597] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Couldn't send request to peer 12682eb70bc84c76879e04a8628ee586. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 414992c8073646798991d2b43fd61f1d. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:12.786199 11899 raft_consensus.cc:2955] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:12.787588 10800 raft_consensus.cc:2955] T 414992c8073646798991d2b43fd61f1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:12.789032 10950 raft_consensus.cc:2955] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:12.805431 10697 catalog_manager.cc:5697] T 414992c8073646798991d2b43fd61f1d P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:12.808663 10872 consensus_queue.cc:237] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:12.827245 10950 raft_consensus.cc:1275] T 1d9e744bb20c4325969f4fbc01c672e8 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:12.827765 10800 raft_consensus.cc:1275] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:16:12.828001 10841 consensus_peers.cc:597] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Couldn't send request to peer 12682eb70bc84c76879e04a8628ee586. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1d9e744bb20c4325969f4fbc01c672e8. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:12.829242 11965 consensus_queue.cc:1048] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:12.830248 11911 consensus_queue.cc:1048] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:12.848968 11965 raft_consensus.cc:2955] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:12.851339 10950 raft_consensus.cc:2955] T 1d9e744bb20c4325969f4fbc01c672e8 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:12.854995 10800 raft_consensus.cc:2955] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:12.863781 10797 consensus_queue.cc:237] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:12.873463 10699 catalog_manager.cc:5697] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:12.880815 10944 raft_consensus.cc:1275] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:12.883033 10872 raft_consensus.cc:1275] T 19c20efd6c04443596bb712e3c93c650 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:12.882740 11960 consensus_queue.cc:1048] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:12.884536 11937 consensus_queue.cc:1048] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:16:12.887166 10766 consensus_peers.cc:597] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 -> Peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Couldn't send request to peer 12682eb70bc84c76879e04a8628ee586. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 19c20efd6c04443596bb712e3c93c650. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:12.901960 11960 raft_consensus.cc:2955] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:12.903738 10949 raft_consensus.cc:2955] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:12.916512 10875 consensus_queue.cc:237] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:12.920075 10699 catalog_manager.cc:5697] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New cstate: current_term: 1 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:12.941001 10949 raft_consensus.cc:1275] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:16:12.941680 10841 consensus_peers.cc:597] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Couldn't send request to peer 12682eb70bc84c76879e04a8628ee586. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b253bd741d094067902a02bc765ee0fb. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:12.942683 11911 consensus_queue.cc:1048] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:12.947201 10800 raft_consensus.cc:1275] T b253bd741d094067902a02bc765ee0fb P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:12.949528 11911 consensus_queue.cc:1048] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:12.945957 10872 raft_consensus.cc:2955] T 19c20efd6c04443596bb712e3c93c650 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:12.962004 10949 raft_consensus.cc:2955] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:12.960011 11911 raft_consensus.cc:2955] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:12.983237 10699 catalog_manager.cc:5697] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 reported cstate change: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:12.989284 10797 raft_consensus.cc:2955] T b253bd741d094067902a02bc765ee0fb P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:13.046137  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:13.111461 10944 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e"
dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
mode: GRACEFUL
new_leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
 from {username='slave'} at 127.0.0.1:46052
I20260709 13:16:13.112195 10944 raft_consensus.cc:606] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Received request to transfer leadership to TS 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:13.128925 10872 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:16:13.129554 10872 raft_consensus.cc:606] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Received request to transfer leadership to TS e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:13.132287 10872 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7595d019b2e74196bc5838962b53798d"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "12682eb70bc84c76879e04a8628ee586"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:16:13.132848 10872 raft_consensus.cc:606] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Received request to transfer leadership to TS 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:13.137774 10872 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:16:13.138326 10872 raft_consensus.cc:606] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Received request to transfer leadership to TS cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:13.140957 10872 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:16:13.141480 10872 raft_consensus.cc:606] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Received request to transfer leadership to TS e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:13.144506 10872 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:16:13.145021 10872 raft_consensus.cc:606] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Received request to transfer leadership to TS cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:13.160938 10797 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d"
dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
mode: GRACEFUL
new_leader_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:16:13.161536 10797 raft_consensus.cc:606] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Received request to transfer leadership to TS cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:13.162746  9682 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 7
I20260709 13:16:13.164021  9682 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:13.167151 11983 ts_tablet_manager.cc:933] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586: Initiating tablet copy from peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:13.169028 11983 tablet_copy_client.cc:323] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586: tablet copy: Beginning tablet copy session from remote peer at address 127.9.116.130:37895
I20260709 13:16:13.170554 10886 tablet_copy_service.cc:140] P 468a8fd46e824f62b9c2c0d3532a6d26: Received BeginTabletCopySession request for tablet 414992c8073646798991d2b43fd61f1d from peer 12682eb70bc84c76879e04a8628ee586 ({username='slave'} at 127.0.0.1:49290)
I20260709 13:16:13.171046 10886 tablet_copy_service.cc:161] P 468a8fd46e824f62b9c2c0d3532a6d26: Beginning new tablet copy session on tablet 414992c8073646798991d2b43fd61f1d from peer 12682eb70bc84c76879e04a8628ee586 at {username='slave'} at 127.0.0.1:49290: session id = 12682eb70bc84c76879e04a8628ee586-414992c8073646798991d2b43fd61f1d
I20260709 13:16:13.176460 11961 raft_consensus.cc:993] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26: : Instructing follower cb0428cc55984daba1ba63744a5102a2 to start an election
I20260709 13:16:13.179108 11961 raft_consensus.cc:1081] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Signalling peer cb0428cc55984daba1ba63744a5102a2 to start an election
I20260709 13:16:13.177541 10886 tablet_copy_source_session.cc:215] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:13.187566 10944 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74"
dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:41020
I20260709 13:16:13.188198 10944 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:13.188594 10944 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:13.189802 11983 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 414992c8073646798991d2b43fd61f1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:13.196174 10944 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:13.198762 10944 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:13.199860 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:13.200584 10797 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:13.199923 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:13.203493 10875 raft_consensus.cc:3055] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:13.203794 10875 raft_consensus.cc:740] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:13.204545 10875 consensus_queue.cc:260] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:13.205763 10875 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:13.207199 10797 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 2.
I20260709 13:16:13.208658 10916 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:13.209479 11934 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:13.209955 11934 raft_consensus.cc:697] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 2 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:13.210841 11934 consensus_queue.cc:237] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:13.214272 10875 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 2.
I20260709 13:16:13.228248 11983 tablet_copy_client.cc:806] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:13.227856 10697 catalog_manager.cc:5697] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 1 to 2, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 2 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:13.229039 11983 tablet_copy_client.cc:670] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:13.232818 11983 tablet_copy_client.cc:538] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:13.241849 11983 tablet_bootstrap.cc:492] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586: Bootstrap starting.
I20260709 13:16:13.263906 11987 ts_tablet_manager.cc:933] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586: Initiating tablet copy from peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:13.271375 11987 tablet_copy_client.cc:323] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Beginning tablet copy session from remote peer at address 127.9.116.130:37895
I20260709 13:16:13.273926 10886 tablet_copy_service.cc:140] P 468a8fd46e824f62b9c2c0d3532a6d26: Received BeginTabletCopySession request for tablet 1d9e744bb20c4325969f4fbc01c672e8 from peer 12682eb70bc84c76879e04a8628ee586 ({username='slave'} at 127.0.0.1:49290)
I20260709 13:16:13.274477 10886 tablet_copy_service.cc:161] P 468a8fd46e824f62b9c2c0d3532a6d26: Beginning new tablet copy session on tablet 1d9e744bb20c4325969f4fbc01c672e8 from peer 12682eb70bc84c76879e04a8628ee586 at {username='slave'} at 127.0.0.1:49290: session id = 12682eb70bc84c76879e04a8628ee586-1d9e744bb20c4325969f4fbc01c672e8
I20260709 13:16:13.275231 11983 tablet_bootstrap.cc:492] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:13.276158 11983 tablet_bootstrap.cc:492] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586: Bootstrap complete.
I20260709 13:16:13.276839 11983 ts_tablet_manager.cc:1403] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586: Time spent bootstrapping tablet: real 0.035s	user 0.026s	sys 0.001s
I20260709 13:16:13.283481 11983 raft_consensus.cc:359] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:13.284353 11983 raft_consensus.cc:740] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Initialized, Role: LEARNER
I20260709 13:16:13.284997 11983 consensus_queue.cc:260] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:13.288823 11983 ts_tablet_manager.cc:1434] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 13:16:13.298509 10885 tablet_copy_service.cc:342] P 468a8fd46e824f62b9c2c0d3532a6d26: Request end of tablet copy session 12682eb70bc84c76879e04a8628ee586-414992c8073646798991d2b43fd61f1d received from {username='slave'} at 127.0.0.1:49290
I20260709 13:16:13.299010 10885 tablet_copy_service.cc:434] P 468a8fd46e824f62b9c2c0d3532a6d26: ending tablet copy session 12682eb70bc84c76879e04a8628ee586-414992c8073646798991d2b43fd61f1d on tablet 414992c8073646798991d2b43fd61f1d with peer 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:13.286131 10886 tablet_copy_source_session.cc:215] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:13.306452 11965 raft_consensus.cc:993] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26: : Instructing follower 12682eb70bc84c76879e04a8628ee586 to start an election
I20260709 13:16:13.306962 11965 raft_consensus.cc:1081] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Signalling peer 12682eb70bc84c76879e04a8628ee586 to start an election
I20260709 13:16:13.310565 11987 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d9e744bb20c4325969f4fbc01c672e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:13.310456 11508 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7595d019b2e74196bc5838962b53798d"
dest_uuid: "12682eb70bc84c76879e04a8628ee586"
 from {username='slave'} at 127.0.0.1:56410
I20260709 13:16:13.313478 11508 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:13.313908 11508 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:13.322937 11508 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:13.325629 11508 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:13.327229 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:13.327834 10944 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:13.327926 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:13.328576 10875 raft_consensus.cc:3055] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:13.328873 10875 raft_consensus.cc:740] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:13.329602 10875 consensus_queue.cc:260] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:13.332981 11987 tablet_copy_client.cc:806] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:13.333586 11987 tablet_copy_client.cc:670] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:13.331629 10875 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:13.337832 10944 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2.
I20260709 13:16:13.339017 11477 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:13.339989 11989 raft_consensus.cc:2804] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:13.341181 11989 raft_consensus.cc:697] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 2 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:13.342073 11989 consensus_queue.cc:237] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:13.342059 11987 tablet_copy_client.cc:538] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:13.347667 10875 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2.
I20260709 13:16:13.359645 10697 catalog_manager.cc:5697] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 1 to 2, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 2 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:13.361438 11987 tablet_bootstrap.cc:492] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586: Bootstrap starting.
I20260709 13:16:13.372491 11960 raft_consensus.cc:993] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87: : Instructing follower cb0428cc55984daba1ba63744a5102a2 to start an election
I20260709 13:16:13.373121 11937 raft_consensus.cc:1081] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Signalling peer cb0428cc55984daba1ba63744a5102a2 to start an election
I20260709 13:16:13.375027 10949 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d"
dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:41014
I20260709 13:16:13.375643 10949 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:13.376021 10949 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:13.388262 10949 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:13.393420 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:13.394019 10797 raft_consensus.cc:3055] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:13.394332 10797 raft_consensus.cc:740] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:13.395049 10797 consensus_queue.cc:260] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:13.396090 10797 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:13.396658 11987 tablet_bootstrap.cc:492] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:13.397727 11987 tablet_bootstrap.cc:492] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586: Bootstrap complete.
I20260709 13:16:13.397825 11983 ts_tablet_manager.cc:933] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586: Initiating tablet copy from peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685)
I20260709 13:16:13.398669 11987 ts_tablet_manager.cc:1403] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586: Time spent bootstrapping tablet: real 0.037s	user 0.012s	sys 0.013s
I20260709 13:16:13.401914 11987 raft_consensus.cc:359] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:13.402745 11987 raft_consensus.cc:740] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Initialized, Role: LEARNER
I20260709 13:16:13.403455 11987 consensus_queue.cc:260] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:13.403965 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:13.404517 10875 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:13.408824 11987 ts_tablet_manager.cc:1434] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586: Time spent starting tablet: real 0.010s	user 0.001s	sys 0.005s
I20260709 13:16:13.411216 11934 raft_consensus.cc:993] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2: : Instructing follower 468a8fd46e824f62b9c2c0d3532a6d26 to start an election
I20260709 13:16:13.411724 11934 raft_consensus.cc:1081] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Signalling peer 468a8fd46e824f62b9c2c0d3532a6d26 to start an election
I20260709 13:16:13.413183 10886 tablet_copy_service.cc:342] P 468a8fd46e824f62b9c2c0d3532a6d26: Request end of tablet copy session 12682eb70bc84c76879e04a8628ee586-1d9e744bb20c4325969f4fbc01c672e8 received from {username='slave'} at 127.0.0.1:49290
I20260709 13:16:13.413636 10886 tablet_copy_service.cc:434] P 468a8fd46e824f62b9c2c0d3532a6d26: ending tablet copy session 12682eb70bc84c76879e04a8628ee586-1d9e744bb20c4325969f4fbc01c672e8 on tablet 1d9e744bb20c4325969f4fbc01c672e8 with peer 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:13.411257 10949 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:13.414641 10871 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
 from {username='slave'} at 127.0.0.1:54950
I20260709 13:16:13.415234 10871 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:13.415597 10871 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:13.416239 10797 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 2.
I20260709 13:16:13.417388 10916 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:13.418222 11884 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:13.418648 11884 raft_consensus.cc:697] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 2 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:13.419456 11884 consensus_queue.cc:237] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:13.422457 10871 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:13.424747 10875 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 2.
I20260709 13:16:13.425423 11961 raft_consensus.cc:993] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26: : Instructing follower cb0428cc55984daba1ba63744a5102a2 to start an election
I20260709 13:16:13.425956 11965 raft_consensus.cc:1081] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Signalling peer cb0428cc55984daba1ba63744a5102a2 to start an election
I20260709 13:16:13.429569 10871 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:13.430922 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:13.431543 10800 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:13.435254 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:13.436067 10950 raft_consensus.cc:3055] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:13.436432 10950 raft_consensus.cc:740] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:13.437386 10950 consensus_queue.cc:260] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:13.442344 10950 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:13.444842 10949 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8"
dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:41020
I20260709 13:16:13.451577 10949 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:13.451998 10949 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:13.479087 10800 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:13.483596 10842 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:13.484401 11911 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:13.484885 11911 raft_consensus.cc:697] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:13.484530 10697 catalog_manager.cc:5697] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 1 to 2, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 2 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:13.485680 11911 consensus_queue.cc:237] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:13.493283 10950 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:13.498661 11983 tablet_copy_client.cc:323] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Beginning tablet copy session from remote peer at address 127.9.116.129:39685
I20260709 13:16:13.500361 10811 tablet_copy_service.cc:140] P e01ecd008edd4aa2b6b2b47efbc2ea87: Received BeginTabletCopySession request for tablet 19c20efd6c04443596bb712e3c93c650 from peer 12682eb70bc84c76879e04a8628ee586 ({username='slave'} at 127.0.0.1:39442)
I20260709 13:16:13.500957 10811 tablet_copy_service.cc:161] P e01ecd008edd4aa2b6b2b47efbc2ea87: Beginning new tablet copy session on tablet 19c20efd6c04443596bb712e3c93c650 from peer 12682eb70bc84c76879e04a8628ee586 at {username='slave'} at 127.0.0.1:39442: session id = 12682eb70bc84c76879e04a8628ee586-19c20efd6c04443596bb712e3c93c650
I20260709 13:16:13.500710 10949 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:13.502912 10949 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:13.504765 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:13.505340 10871 raft_consensus.cc:3055] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:13.505628 10871 raft_consensus.cc:740] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:13.506338 10871 consensus_queue.cc:260] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:13.506939 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:13.507581 10797 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:13.507673 10871 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:13.511807 10811 tablet_copy_source_session.cc:215] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:13.514660 10797 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 2.
I20260709 13:16:13.517307 11983 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19c20efd6c04443596bb712e3c93c650. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:13.519763 10871 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 2.
I20260709 13:16:13.515803 10916 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:13.525359 11884 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:13.525830 11884 raft_consensus.cc:697] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 2 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:13.526607 11884 consensus_queue.cc:237] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:13.532788 10697 catalog_manager.cc:5697] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 1 to 2, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 2 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:13.551247 11911 raft_consensus.cc:993] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26: : Instructing follower e01ecd008edd4aa2b6b2b47efbc2ea87 to start an election
I20260709 13:16:13.551713 11911 raft_consensus.cc:1081] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Signalling peer e01ecd008edd4aa2b6b2b47efbc2ea87 to start an election
I20260709 13:16:13.551614 10699 catalog_manager.cc:5697] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 1 to 2, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 2 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:13.553491 11987 ts_tablet_manager.cc:933] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586: Initiating tablet copy from peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:13.558509 11987 tablet_copy_client.cc:323] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586: tablet copy: Beginning tablet copy session from remote peer at address 127.9.116.130:37895
I20260709 13:16:13.553434 11983 tablet_copy_client.cc:806] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:13.563257 11983 tablet_copy_client.cc:670] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:13.566243 11965 raft_consensus.cc:993] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26: : Instructing follower e01ecd008edd4aa2b6b2b47efbc2ea87 to start an election
I20260709 13:16:13.560981 10886 tablet_copy_service.cc:140] P 468a8fd46e824f62b9c2c0d3532a6d26: Received BeginTabletCopySession request for tablet b253bd741d094067902a02bc765ee0fb from peer 12682eb70bc84c76879e04a8628ee586 ({username='slave'} at 127.0.0.1:49290)
I20260709 13:16:13.565600 10800 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30"
dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
 from {username='slave'} at 127.0.0.1:46758
I20260709 13:16:13.567224 10886 tablet_copy_service.cc:161] P 468a8fd46e824f62b9c2c0d3532a6d26: Beginning new tablet copy session on tablet b253bd741d094067902a02bc765ee0fb from peer 12682eb70bc84c76879e04a8628ee586 at {username='slave'} at 127.0.0.1:49290: session id = 12682eb70bc84c76879e04a8628ee586-b253bd741d094067902a02bc765ee0fb
I20260709 13:16:13.567917 10800 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:13.568303 10800 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:13.568674 11983 tablet_copy_client.cc:538] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:13.566733 11965 raft_consensus.cc:1081] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Signalling peer e01ecd008edd4aa2b6b2b47efbc2ea87 to start an election
I20260709 13:16:13.574889 10800 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:13.577271 10800 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:13.578251 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:13.578893 10949 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:13.580524 10886 tablet_copy_source_session.cc:215] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:13.582664 11983 tablet_bootstrap.cc:492] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586: Bootstrap starting.
I20260709 13:16:13.584117 11987 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b253bd741d094067902a02bc765ee0fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:13.586925 10949 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 2.
I20260709 13:16:13.588234 10800 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8"
dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
 from {username='slave'} at 127.0.0.1:46758
I20260709 13:16:13.588805 10800 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:13.588749 10766 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:13.589282 10800 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:13.589926 11937 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:13.590446 11937 raft_consensus.cc:697] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:13.591409 11937 consensus_queue.cc:237] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:13.596622 10800 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:13.598976 10800 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:13.600098 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:13.600678 10950 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:13.607107 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:13.607714 10875 raft_consensus.cc:3055] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:13.608011 10875 raft_consensus.cc:740] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:13.608734 10950 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 2.
I20260709 13:16:13.608702 10875 consensus_queue.cc:260] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:13.609927 10875 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:13.610265 10766 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:13.613260 11897 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:13.613843 11897 raft_consensus.cc:697] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:13.614662 11897 consensus_queue.cc:237] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:13.616860 10875 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 2.
I20260709 13:16:13.620070 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:13.620648 10872 raft_consensus.cc:3055] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:13.620920 10872 raft_consensus.cc:740] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:13.621588 10872 consensus_queue.cc:260] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:13.623905 10872 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:13.635195 11983 tablet_bootstrap.cc:492] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:13.635509 10869 raft_consensus.cc:1275] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:13.636166 11983 tablet_bootstrap.cc:492] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586: Bootstrap complete.
I20260709 13:16:13.636902 11983 ts_tablet_manager.cc:1403] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586: Time spent bootstrapping tablet: real 0.054s	user 0.022s	sys 0.013s
I20260709 13:16:13.636525 10697 catalog_manager.cc:5697] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 1 to 2, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 2 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:13.640507 11884 consensus_queue.cc:1048] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:13.641006 11983 raft_consensus.cc:359] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:13.641875 11983 raft_consensus.cc:740] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Initialized, Role: LEARNER
I20260709 13:16:13.642519 11983 consensus_queue.cc:260] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:13.645113 11983 ts_tablet_manager.cc:1434] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:16:13.647069 10811 tablet_copy_service.cc:342] P e01ecd008edd4aa2b6b2b47efbc2ea87: Request end of tablet copy session 12682eb70bc84c76879e04a8628ee586-19c20efd6c04443596bb712e3c93c650 received from {username='slave'} at 127.0.0.1:39442
I20260709 13:16:13.647589 10811 tablet_copy_service.cc:434] P e01ecd008edd4aa2b6b2b47efbc2ea87: ending tablet copy session 12682eb70bc84c76879e04a8628ee586-19c20efd6c04443596bb712e3c93c650 on tablet 19c20efd6c04443596bb712e3c93c650 with peer 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:13.655776 10872 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 2.
I20260709 13:16:13.676802 11508 raft_consensus.cc:1217] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:13.679440 11987 tablet_copy_client.cc:806] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:13.685856 11987 tablet_copy_client.cc:670] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:13.690850 11987 tablet_copy_client.cc:538] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:13.726197 10697 catalog_manager.cc:5697] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 1 to 2, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 2 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:13.739132 11987 tablet_bootstrap.cc:492] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586: Bootstrap starting.
I20260709 13:16:13.740262 10797 raft_consensus.cc:1275] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:13.742992 11934 consensus_queue.cc:1048] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:13.765386 11507 raft_consensus.cc:1217] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:13.781672 10869 raft_consensus.cc:1275] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:13.784909 11989 consensus_queue.cc:1048] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:13.792618 11987 tablet_bootstrap.cc:492] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:13.795666 11987 tablet_bootstrap.cc:492] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586: Bootstrap complete.
I20260709 13:16:13.797667 11987 ts_tablet_manager.cc:1403] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586: Time spent bootstrapping tablet: real 0.059s	user 0.023s	sys 0.022s
I20260709 13:16:13.802691 11987 raft_consensus.cc:359] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:13.803539 11987 raft_consensus.cc:740] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Initialized, Role: LEARNER
I20260709 13:16:13.804173 11987 consensus_queue.cc:260] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:13.806605 11987 ts_tablet_manager.cc:1434] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.000s
I20260709 13:16:13.818660 10800 raft_consensus.cc:1275] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:13.824561 11994 consensus_queue.cc:1048] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:13.830123 10886 tablet_copy_service.cc:342] P 468a8fd46e824f62b9c2c0d3532a6d26: Request end of tablet copy session 12682eb70bc84c76879e04a8628ee586-b253bd741d094067902a02bc765ee0fb received from {username='slave'} at 127.0.0.1:49290
I20260709 13:16:13.830569 10886 tablet_copy_service.cc:434] P 468a8fd46e824f62b9c2c0d3532a6d26: ending tablet copy session 12682eb70bc84c76879e04a8628ee586-b253bd741d094067902a02bc765ee0fb on tablet b253bd741d094067902a02bc765ee0fb with peer 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:13.832341 10950 raft_consensus.cc:1275] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:13.834574 11989 consensus_queue.cc:1048] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:13.984014 10797 raft_consensus.cc:1275] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:13.992231 11911 consensus_queue.cc:1048] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:13.995522 10948 raft_consensus.cc:1275] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:14.016499 11507 raft_consensus.cc:1217] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:14.041319 11911 consensus_queue.cc:1048] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:14.044925 10945 raft_consensus.cc:1275] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:14.053743 11508 raft_consensus.cc:1217] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:14.063050 12016 consensus_queue.cc:1048] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:16:14.108424 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 75278.6us
I20260709 13:16:14.135106 10869 raft_consensus.cc:1275] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:14.142095 10870 raft_consensus.cc:1275] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:14.143989 10793 raft_consensus.cc:1275] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:14.145334 10868 raft_consensus.cc:1275] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:14.151345 10871 raft_consensus.cc:1275] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:14.155750 12016 consensus_queue.cc:1048] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:14.162115 12016 consensus_queue.cc:1048] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:14.165144  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:14.165998 10945 raft_consensus.cc:1275] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:14.167102 12018 raft_consensus.cc:1064] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26: attempting to promote NON_VOTER 12682eb70bc84c76879e04a8628ee586 to VOTER
I20260709 13:16:14.167501 11937 consensus_queue.cc:1048] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:14.169692 12018 consensus_queue.cc:237] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:14.245538 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 61696.3us
I20260709 13:16:14.247417 10797 raft_consensus.cc:1275] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:14.251865 11899 consensus_queue.cc:1048] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:16:14.254256 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 94208.9us
I20260709 13:16:14.255780 12017 consensus_queue.cc:1048] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:14.250342 10945 raft_consensus.cc:1275] T 1d9e744bb20c4325969f4fbc01c672e8 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:14.260989 11934 consensus_queue.cc:1048] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:14.262270 11934 consensus_queue.cc:1048] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:14.269891 11507 raft_consensus.cc:1275] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:14.285017 11899 consensus_queue.cc:1048] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:14.289258 11899 raft_consensus.cc:1064] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26: attempting to promote NON_VOTER 12682eb70bc84c76879e04a8628ee586 to VOTER
I20260709 13:16:14.289911 12027 consensus_queue.cc:1048] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:14.289263  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:14.292671 11899 consensus_queue.cc:237] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:14.319687 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 61035.7us
I20260709 13:16:14.364262 12028 raft_consensus.cc:2955] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:14.395957 11507 raft_consensus.cc:2955] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:14.403932 10799 raft_consensus.cc:1275] T 414992c8073646798991d2b43fd61f1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:14.405421 11899 consensus_queue.cc:1048] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:14.405792 11508 raft_consensus.cc:1275] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:14.414011 12018 consensus_queue.cc:1048] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:14.418898 10793 raft_consensus.cc:2955] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:14.433653 10949 raft_consensus.cc:1275] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:14.451429 10945 raft_consensus.cc:2955] T 1d9e744bb20c4325969f4fbc01c672e8 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:14.452425 11899 consensus_queue.cc:1048] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:14.457913 10698 catalog_manager.cc:5697] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:14.506096 10869 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:16:14.506989 10869 raft_consensus.cc:606] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 LEADER]: Received request to transfer leadership to TS e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:14.516820 12014 raft_consensus.cc:1064] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87: attempting to promote NON_VOTER 12682eb70bc84c76879e04a8628ee586 to VOTER
I20260709 13:16:14.520205 12014 consensus_queue.cc:237] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:14.525578 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 48020.9us
I20260709 13:16:14.530548 10869 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "185480268a684958b8347a4f53af57c5"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "12682eb70bc84c76879e04a8628ee586"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:16:14.535871 10869 raft_consensus.cc:606] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Received request to transfer leadership to TS 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:14.539434 10869 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "12682eb70bc84c76879e04a8628ee586"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:16:14.540127 10869 raft_consensus.cc:606] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Received request to transfer leadership to TS 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:14.543228 11961 raft_consensus.cc:2955] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:14.547379 10797 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256"
dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
mode: GRACEFUL
new_leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:16:14.548043 10797 raft_consensus.cc:606] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Received request to transfer leadership to TS 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:14.551674 10945 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533"
dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
mode: GRACEFUL
new_leader_uuid: "12682eb70bc84c76879e04a8628ee586"
 from {username='slave'} at 127.0.0.1:46052
I20260709 13:16:14.552464 10945 raft_consensus.cc:606] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Received request to transfer leadership to TS 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:14.555807 10872 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "12682eb70bc84c76879e04a8628ee586"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:16:14.556422 10872 raft_consensus.cc:606] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Received request to transfer leadership to TS 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:14.562472 11507 raft_consensus.cc:2955] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:14.566257 10868 consensus_queue.cc:237] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:14.573849 11899 raft_consensus.cc:1064] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26: attempting to promote NON_VOTER 12682eb70bc84c76879e04a8628ee586 to VOTER
I20260709 13:16:14.577083 10868 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:16:14.577729 10868 raft_consensus.cc:606] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Received request to transfer leadership to TS e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:14.579000  9682 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 7
I20260709 13:16:14.580714  9682 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:14.584586 11899 consensus_queue.cc:237] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:14.592474 11508 raft_consensus.cc:1275] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:14.595824 10945 raft_consensus.cc:2955] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:14.601958 10797 raft_consensus.cc:1275] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:14.610553 12028 consensus_queue.cc:1048] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:14.617338 10945 raft_consensus.cc:1275] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:14.620110 11508 raft_consensus.cc:1275] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:14.626732 10699 catalog_manager.cc:5697] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 reported cstate change: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:14.629093 11965 raft_consensus.cc:993] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26: : Instructing follower e01ecd008edd4aa2b6b2b47efbc2ea87 to start an election
I20260709 13:16:14.629637 11965 raft_consensus.cc:1081] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Signalling peer e01ecd008edd4aa2b6b2b47efbc2ea87 to start an election
I20260709 13:16:14.629792 12026 raft_consensus.cc:2955] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:14.641752 10686 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1d9e744bb20c4325969f4fbc01c672e8 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:14.644948 10793 raft_consensus.cc:1275] T b253bd741d094067902a02bc765ee0fb P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:14.646082 11899 consensus_queue.cc:1048] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:14.647277 10793 raft_consensus.cc:2955] T 414992c8073646798991d2b43fd61f1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:14.649276 11899 consensus_queue.cc:1048] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:14.651433 10799 raft_consensus.cc:2955] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:14.654670 12026 raft_consensus.cc:993] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26: : Instructing follower e01ecd008edd4aa2b6b2b47efbc2ea87 to start an election
I20260709 13:16:14.659623 12026 raft_consensus.cc:1081] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 LEADER]: Signalling peer e01ecd008edd4aa2b6b2b47efbc2ea87 to start an election
I20260709 13:16:14.662520 10699 catalog_manager.cc:5697] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: config changed from index 3 to 4, VOTER cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) evicted. New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:14.664965 11508 raft_consensus.cc:1275] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:14.667120 10872 raft_consensus.cc:1275] T 19c20efd6c04443596bb712e3c93c650 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:14.668180 12016 consensus_queue.cc:1048] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:14.641542 10797 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740"
dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
 from {username='slave'} at 127.0.0.1:46758
I20260709 13:16:14.670274 10800 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e"
dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
 from {username='slave'} at 127.0.0.1:46758
I20260709 13:16:14.670765 10797 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:14.670854 11961 consensus_queue.cc:1048] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:14.670881 10800 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:14.671756 10800 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:14.671187 10797 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:14.694993 12015 consensus_queue.cc:1048] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:14.703059 10800 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:14.708177 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:14.708775 10949 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:14.708856 10797 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:14.711453 10797 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:14.713551 10937 tablet_service.cc:1558] Processing DeleteTablet for tablet 1d9e744bb20c4325969f4fbc01c672e8 with delete_type TABLET_DATA_TOMBSTONED (TS cb0428cc55984daba1ba63744a5102a2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41006
I20260709 13:16:14.714582 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:14.715401 10944 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:14.720891 10949 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
I20260709 13:16:14.722301 10766 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:14.726606 10944 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 2.
I20260709 13:16:14.728559 12015 raft_consensus.cc:993] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87: : Instructing follower 468a8fd46e824f62b9c2c0d3532a6d26 to start an election
I20260709 13:16:14.726044 10868 consensus_queue.cc:237] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:14.736763 12012 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:14.737437 12012 raft_consensus.cc:697] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:14.738945 10766 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:14.741747 12012 consensus_queue.cc:237] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:14.743430 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:14.744168 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:14.744336 10868 raft_consensus.cc:3055] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:14.744762 10869 raft_consensus.cc:3055] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:14.744802 10868 raft_consensus.cc:740] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:14.665696 10945 raft_consensus.cc:1275] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:14.745225 10869 raft_consensus.cc:740] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:14.746055 10869 consensus_queue.cc:260] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:14.745930 10868 consensus_queue.cc:260] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:14.747175 11961 consensus_queue.cc:1048] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:14.746723 10800 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:14.747695 10869 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:14.747375 10868 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:14.754715 11507 raft_consensus.cc:2955] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:14.767150 10869 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 2.
I20260709 13:16:14.768337 12012 raft_consensus.cc:1081] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Signalling peer 468a8fd46e824f62b9c2c0d3532a6d26 to start an election
I20260709 13:16:14.769052 12016 raft_consensus.cc:2804] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:14.771414 10871 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
 from {username='slave'} at 127.0.0.1:54948
I20260709 13:16:14.772169 10871 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:14.772600 10871 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:14.781822 10868 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
I20260709 13:16:14.781931 12016 raft_consensus.cc:697] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:14.784477 12016 consensus_queue.cc:237] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:14.788669 10945 raft_consensus.cc:1240] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:14.790624 10944 raft_consensus.cc:1275] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:14.797811 10871 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:14.813172 11508 raft_consensus.cc:1275] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:14.818849 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:14.821892 10797 raft_consensus.cc:3055] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:14.822227 10797 raft_consensus.cc:740] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:14.823457 10797 consensus_queue.cc:260] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:14.826066 11961 consensus_queue.cc:1048] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:14.817966 10871 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:14.828626 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 89849.3us
I20260709 13:16:14.832412 10797 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:14.826053 12054 tablet_replica.cc:333] T 1d9e744bb20c4325969f4fbc01c672e8 P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:16:14.839406 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:14.843086 10949 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:14.849326 11899 raft_consensus.cc:993] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26: : Instructing follower 12682eb70bc84c76879e04a8628ee586 to start an election
I20260709 13:16:14.849171 12054 raft_consensus.cc:2243] T 1d9e744bb20c4325969f4fbc01c672e8 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:14.852375 10797 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:14.856539 10842 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:14.852591 10949 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:14.835824 12018 raft_consensus.cc:993] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26: : Instructing follower 12682eb70bc84c76879e04a8628ee586 to start an election
I20260709 13:16:14.849852 12028 raft_consensus.cc:1081] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Signalling peer 12682eb70bc84c76879e04a8628ee586 to start an election
I20260709 13:16:14.860352 11911 raft_consensus.cc:2804] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:14.860821 11911 raft_consensus.cc:697] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:14.862498 12054 raft_consensus.cc:2272] T 1d9e744bb20c4325969f4fbc01c672e8 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:14.864540 12018 raft_consensus.cc:1081] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Signalling peer 12682eb70bc84c76879e04a8628ee586 to start an election
I20260709 13:16:14.837109 12027 consensus_queue.cc:1048] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 13:16:14.873684 11511 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3"
dest_uuid: "12682eb70bc84c76879e04a8628ee586"
 from {username='slave'} at 127.0.0.1:56410
I20260709 13:16:14.874369 11511 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:14.874832 11511 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:14.852013 11937 consensus_queue.cc:1048] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.016s
I20260709 13:16:14.884409 11511 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:14.887408 12024 raft_consensus.cc:993] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2: : Instructing follower 12682eb70bc84c76879e04a8628ee586 to start an election
I20260709 13:16:14.887676 11511 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:14.887933 12024 raft_consensus.cc:1081] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Signalling peer 12682eb70bc84c76879e04a8628ee586 to start an election
I20260709 13:16:14.889665 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:14.890249 10944 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:14.891562 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:14.888864 11911 consensus_queue.cc:237] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:14.902765 10872 raft_consensus.cc:3055] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:14.903337 10872 raft_consensus.cc:740] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:14.904312 10872 consensus_queue.cc:260] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:14.905844 10872 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:14.913664 11510 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533"
dest_uuid: "12682eb70bc84c76879e04a8628ee586"
 from {username='slave'} at 127.0.0.1:56404
I20260709 13:16:14.914289 11510 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:14.914650 11510 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:14.920713 12054 ts_tablet_manager.cc:1916] T 1d9e744bb20c4325969f4fbc01c672e8 P cb0428cc55984daba1ba63744a5102a2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:14.923511 11510 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:14.925994 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 49076.8us
I20260709 13:16:14.925595 11965 raft_consensus.cc:2955] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:14.928526 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:14.929080 10871 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:14.929687 12012 raft_consensus.cc:2955] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:14.932577 10869 raft_consensus.cc:2955] T 19c20efd6c04443596bb712e3c93c650 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:14.952400 10871 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2.
I20260709 13:16:14.953466 10870 raft_consensus.cc:1240] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:14.957091 12032 consensus_queue.cc:1059] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:14.959708 12032 raft_consensus.cc:3055] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:14.960117 12032 raft_consensus.cc:740] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:16:14.961158 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 87699.2us
I20260709 13:16:14.961280 12032 consensus_queue.cc:260] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:14.963357 10948 raft_consensus.cc:2955] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:14.967972 11510 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:14.969835 11478 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:16:14.972663 11507 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "185480268a684958b8347a4f53af57c5"
dest_uuid: "12682eb70bc84c76879e04a8628ee586"
 from {username='slave'} at 127.0.0.1:56410
I20260709 13:16:14.972690 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:14.973421 11507 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:14.973806 11507 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:14.974259 11989 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:14.974763 11989 raft_consensus.cc:697] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 2 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:14.975986 11989 consensus_queue.cc:237] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:14.980037 11512 raft_consensus.cc:2955] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:14.986032 12032 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:14.988054 11507 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:14.992079 11507 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:14.993744 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:14.994459 10949 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:14.918484 10872 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2.
I20260709 13:16:14.978488 10944 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2.
I20260709 13:16:15.002939 10696 catalog_manager.cc:5697] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 reported cstate change: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:15.007211 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:15.008002 10871 raft_consensus.cc:3055] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:15.008373 10871 raft_consensus.cc:740] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:15.010058 11512 raft_consensus.cc:2955] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:15.013713 10871 consensus_queue.cc:260] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:15.014988 10871 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:15.035378 11961 raft_consensus.cc:2955] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:15.038487 10871 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2.
I20260709 13:16:15.039815 11478 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:16:15.040709 11989 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:15.041361 11989 raft_consensus.cc:697] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 2 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:15.043928 10949 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2.
I20260709 13:16:15.044900 10948 raft_consensus.cc:1240] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:15.045687 11477 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:15.049211 12046 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:15.057946 10797 raft_consensus.cc:2955] T b253bd741d094067902a02bc765ee0fb P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:15.080572 10945 raft_consensus.cc:2955] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:15.101279 10949 raft_consensus.cc:2955] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:15.042485 11989 consensus_queue.cc:237] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:15.109161 12054 ts_tablet_manager.cc:1929] T 1d9e744bb20c4325969f4fbc01c672e8 P cb0428cc55984daba1ba63744a5102a2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:15.122152 12054 log.cc:1199] T 1d9e744bb20c4325969f4fbc01c672e8 P cb0428cc55984daba1ba63744a5102a2: Deleting WAL directory at /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-2-root/wals/1d9e744bb20c4325969f4fbc01c672e8
I20260709 13:16:15.124188 10685 catalog_manager.cc:5028] TS cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): tablet 1d9e744bb20c4325969f4fbc01c672e8 (table test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]) successfully deleted
I20260709 13:16:15.187965 10699 catalog_manager.cc:5697] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 1 to 2, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 2 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:15.096160 12046 raft_consensus.cc:697] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 2 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:15.121325 10950 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2.
W20260709 13:16:15.230767 12060 log.cc:927] Time spent T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:16:15.264020 12046 consensus_queue.cc:237] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:15.324815 10686 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 414992c8073646798991d2b43fd61f1d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:15.380658 12063 raft_consensus.cc:993] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26: : Instructing follower 12682eb70bc84c76879e04a8628ee586 to start an election
W20260709 13:16:15.401526 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 192002us
I20260709 13:16:15.397143 12063 raft_consensus.cc:1081] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Signalling peer 12682eb70bc84c76879e04a8628ee586 to start an election
W20260709 13:16:15.423295 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 62388.2us
I20260709 13:16:15.426669 10949 raft_consensus.cc:1275] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:15.430099 10797 raft_consensus.cc:1275] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:15.432281 10871 raft_consensus.cc:1275] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:15.436254 11512 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61"
dest_uuid: "12682eb70bc84c76879e04a8628ee586"
 from {username='slave'} at 127.0.0.1:56410
I20260709 13:16:15.437038 11512 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:15.437532 11512 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:15.437404 12012 consensus_queue.cc:1048] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:15.453833 11512 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:15.469511 11512 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:15.470059 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:15.470703 10871 raft_consensus.cc:3055] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:15.471024 10871 raft_consensus.cc:740] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:15.474673 10871 consensus_queue.cc:260] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:15.476128 10871 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:15.493804 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:15.499640 10871 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2.
I20260709 13:16:15.503126 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:15.508589 10797 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:15.522938 10696 catalog_manager.cc:5697] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 1 to 2, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 2 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:15.478551 12018 consensus_queue.cc:1048] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
W20260709 13:16:15.591020 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 174021us
I20260709 13:16:15.601564 10797 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2.
I20260709 13:16:15.632283 10699 catalog_manager.cc:5697] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 2 to 3, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 3 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:15.632268 11478 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:15.636430 11989 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:15.637063 11989 raft_consensus.cc:697] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 2 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:15.640426 10698 catalog_manager.cc:5697] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 1 to 2, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 2 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:15.638110 11989 consensus_queue.cc:237] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:15.634469 10699 catalog_manager.cc:5697] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 1 to 2, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 2 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:15.642620 10698 catalog_manager.cc:5697] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: config changed from index 2 to 3, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:15.653779 10698 catalog_manager.cc:5697] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 1 to 2, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 2 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:15.660597  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:16:15.669059 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 75783.1us
W20260709 13:16:15.669291 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 141515us
I20260709 13:16:15.693758 11507 raft_consensus.cc:2955] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:15.708699 10697 catalog_manager.cc:5697] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 1 to 2, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 2 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:15.694000 10696 catalog_manager.cc:5697] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 reported cstate change: config changed from index 3 to 4, VOTER e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) evicted. New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
W20260709 13:16:15.742924 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 246000us
I20260709 13:16:15.744191 10948 raft_consensus.cc:1275] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:15.747640 10949 raft_consensus.cc:1275] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:15.763684 10872 raft_consensus.cc:1275] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:15.768034 10872 raft_consensus.cc:1275] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:15.775826 10872 raft_consensus.cc:1275] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:15.776626 12046 consensus_queue.cc:1048] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:15.778741 12067 consensus_queue.cc:1048] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 13:16:15.818990 10945 raft_consensus.cc:1275] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:15.824328 12066 consensus_queue.cc:1048] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 13:16:15.826288 10870 raft_consensus.cc:1275] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:15.828593 12065 consensus_queue.cc:1048] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:16:15.833583 12067 consensus_queue.cc:1048] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:16:15.849423 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 205586us
I20260709 13:16:15.851960 10785 tablet_service.cc:1558] Processing DeleteTablet for tablet 414992c8073646798991d2b43fd61f1d with delete_type TABLET_DATA_TOMBSTONED (TS e01ecd008edd4aa2b6b2b47efbc2ea87 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46752
I20260709 13:16:15.859463 10950 raft_consensus.cc:1275] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:15.863920 10799 consensus_queue.cc:237] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:15.872506 12015 consensus_queue.cc:1048] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:16:15.903802 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 91915.9us
I20260709 13:16:15.905155 12105 tablet_replica.cc:333] T 414992c8073646798991d2b43fd61f1d P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:16:15.907332 12066 consensus_queue.cc:1048] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:15.909976 10943 raft_consensus.cc:1275] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:16:15.913067 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 63089.7us
I20260709 13:16:15.956212 12089 consensus_queue.cc:1048] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:16:15.961535 12105 raft_consensus.cc:2243] T 414992c8073646798991d2b43fd61f1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:15.953509 12015 consensus_queue.cc:1048] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 13:16:15.967984 12066 consensus_queue.cc:1048] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:15.970969 12105 raft_consensus.cc:2272] T 414992c8073646798991d2b43fd61f1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:15.972726 10869 consensus_queue.cc:237] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:15.974102 12105 ts_tablet_manager.cc:1916] T 414992c8073646798991d2b43fd61f1d P e01ecd008edd4aa2b6b2b47efbc2ea87: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:15.987831  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:16:16.008240 12103 log.cc:927] Time spent T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:16:16.041481 11507 raft_consensus.cc:1275] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:16.055436 10946 raft_consensus.cc:1275] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:16.079743 12012 consensus_queue.cc:1048] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 13:16:16.114807 12085 consensus_queue.cc:1048] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 13:16:16.189905 11507 raft_consensus.cc:1275] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20260709 13:16:16.215638 12102 log.cc:927] Time spent T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 13:16:16.262106 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 186636us
I20260709 13:16:16.264782 10868 raft_consensus.cc:1275] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:16.222113 12090 consensus_queue.cc:1048] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
W20260709 13:16:16.278604 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 202180us
W20260709 13:16:16.294299 12116 log.cc:927] Time spent T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
W20260709 13:16:16.347254 12109 log.cc:927] Time spent T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 13:16:16.359262 10950 raft_consensus.cc:1275] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20260709 13:16:16.360522 12111 log.cc:927] Time spent T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
I20260709 13:16:16.369035 12069 consensus_queue.cc:1048] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 13:16:16.417711 12067 consensus_queue.cc:1048] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 13:16:16.529430 10797 raft_consensus.cc:1275] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:16.538992 11989 consensus_queue.cc:1048] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:16:16.570534 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 132663us
I20260709 13:16:16.542578 12105 ts_tablet_manager.cc:1929] T 414992c8073646798991d2b43fd61f1d P e01ecd008edd4aa2b6b2b47efbc2ea87: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:16.583132 12105 log.cc:1199] T 414992c8073646798991d2b43fd61f1d P e01ecd008edd4aa2b6b2b47efbc2ea87: Deleting WAL directory at /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-0-root/wals/414992c8073646798991d2b43fd61f1d
I20260709 13:16:16.584215 10946 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665"
dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
mode: GRACEFUL
new_leader_uuid: "12682eb70bc84c76879e04a8628ee586"
 from {username='slave'} at 127.0.0.1:46052
I20260709 13:16:16.615227 10946 raft_consensus.cc:606] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Received request to transfer leadership to TS 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:16.639854 12065 raft_consensus.cc:2955] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
W20260709 13:16:16.648398 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 73774us
I20260709 13:16:16.648696 10944 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546"
dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
mode: GRACEFUL
new_leader_uuid: "12682eb70bc84c76879e04a8628ee586"
 from {username='slave'} at 127.0.0.1:46052
I20260709 13:16:16.650156 10944 raft_consensus.cc:606] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Received request to transfer leadership to TS 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:16.656878 10686 catalog_manager.cc:5028] TS e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): tablet 414992c8073646798991d2b43fd61f1d (table test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]) successfully deleted
I20260709 13:16:16.668947 10946 raft_consensus.cc:2955] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:16.664933 12070 raft_consensus.cc:2955] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:16.678678 10945 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95"
dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
mode: GRACEFUL
new_leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
 from {username='slave'} at 127.0.0.1:46052
I20260709 13:16:16.682750 10945 raft_consensus.cc:606] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Received request to transfer leadership to TS 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:16.694950 10945 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0"
dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
mode: GRACEFUL
new_leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
 from {username='slave'} at 127.0.0.1:46052
I20260709 13:16:16.695567 10945 raft_consensus.cc:606] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Received request to transfer leadership to TS 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:16:16.695322 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 114996us
I20260709 13:16:16.703452 10869 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dde624b07ed446e5b7347791196396e2"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "12682eb70bc84c76879e04a8628ee586"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:16:16.704099 10869 raft_consensus.cc:606] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Received request to transfer leadership to TS 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:16.707350 10946 raft_consensus.cc:2955] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:16.719638 10799 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831"
dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
mode: GRACEFUL
new_leader_uuid: "12682eb70bc84c76879e04a8628ee586"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:16:16.720269 10799 raft_consensus.cc:606] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Received request to transfer leadership to TS 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:16.721681  9682 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:16:16.725204  9682 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:16.727986 11511 raft_consensus.cc:2955] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:16.728945 12077 raft_consensus.cc:993] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2: : Instructing follower 12682eb70bc84c76879e04a8628ee586 to start an election
I20260709 13:16:16.732122 12077 raft_consensus.cc:1081] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Signalling peer 12682eb70bc84c76879e04a8628ee586 to start an election
I20260709 13:16:16.739078 11507 raft_consensus.cc:2955] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:16.756935 10686 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 19c20efd6c04443596bb712e3c93c650 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:16.762413 11512 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546"
dest_uuid: "12682eb70bc84c76879e04a8628ee586"
 from {username='slave'} at 127.0.0.1:56404
I20260709 13:16:16.763082 11512 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:16.763432 11512 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:16.775457 10696 catalog_manager.cc:5697] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: config changed from index 3 to 4, VOTER 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) evicted. New cstate: current_term: 1 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:16.776140 11512 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:16.779336 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:16.780193 10946 raft_consensus.cc:3055] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:16.780635 10946 raft_consensus.cc:740] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:16.781714 10946 consensus_queue.cc:260] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:16.783838 10946 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:16.793850 10946 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2.
I20260709 13:16:16.797497 11477 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:16.798817 12101 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:16.799541 12101 raft_consensus.cc:697] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 2 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:16.800478 12101 consensus_queue.cc:237] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:16.802541 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:16.803174 10799 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:16.809685 10799 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2.
I20260709 13:16:16.824250 11512 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:16.827876 10686 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b253bd741d094067902a02bc765ee0fb with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:16.828655 12077 raft_consensus.cc:993] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2: : Instructing follower 468a8fd46e824f62b9c2c0d3532a6d26 to start an election
I20260709 13:16:16.829312 12126 raft_consensus.cc:1081] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Signalling peer 468a8fd46e824f62b9c2c0d3532a6d26 to start an election
I20260709 13:16:16.839244 10699 catalog_manager.cc:5697] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: config changed from index 3 to 4, VOTER e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) evicted. New cstate: current_term: 1 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:16.843010 12084 raft_consensus.cc:993] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2: : Instructing follower 468a8fd46e824f62b9c2c0d3532a6d26 to start an election
I20260709 13:16:16.843524 12126 raft_consensus.cc:1081] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Signalling peer 468a8fd46e824f62b9c2c0d3532a6d26 to start an election
I20260709 13:16:16.850353 10863 tablet_service.cc:1558] Processing DeleteTablet for tablet 19c20efd6c04443596bb712e3c93c650 with delete_type TABLET_DATA_TOMBSTONED (TS 468a8fd46e824f62b9c2c0d3532a6d26 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54946
I20260709 13:16:16.857784 10869 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
 from {username='slave'} at 127.0.0.1:54950
I20260709 13:16:16.858489 10869 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:16.858902 10869 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:16.863915 10872 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
 from {username='slave'} at 127.0.0.1:54950
I20260709 13:16:16.864554 10872 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:16.864892 10872 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:16.872960 10872 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:16.875622 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:16.876252 10799 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:16.876113 10697 catalog_manager.cc:5697] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 1 to 2, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 2 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:16.876950 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:16.875674 12148 tablet_replica.cc:333] T 19c20efd6c04443596bb712e3c93c650 P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:16:16.877571 10946 raft_consensus.cc:3055] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:16.877928 10946 raft_consensus.cc:740] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:16.879335 10946 consensus_queue.cc:260] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:16.882006 10872 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:16.885797 10785 tablet_service.cc:1558] Processing DeleteTablet for tablet b253bd741d094067902a02bc765ee0fb with delete_type TABLET_DATA_TOMBSTONED (TS e01ecd008edd4aa2b6b2b47efbc2ea87 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46752
I20260709 13:16:16.886739 12105 tablet_replica.cc:333] T b253bd741d094067902a02bc765ee0fb P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:16:16.880470 10946 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:16.887699 12105 raft_consensus.cc:2243] T b253bd741d094067902a02bc765ee0fb P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:16.888427 12105 raft_consensus.cc:2272] T b253bd741d094067902a02bc765ee0fb P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:16.892993 10946 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:16.885012 12057 raft_consensus.cc:993] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87: : Instructing follower 12682eb70bc84c76879e04a8628ee586 to start an election
I20260709 13:16:16.894233 12057 raft_consensus.cc:1081] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Signalling peer 12682eb70bc84c76879e04a8628ee586 to start an election
I20260709 13:16:16.894621 10841 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:16.895498 11911 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:16.895953 11911 raft_consensus.cc:697] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:16.896026 12105 ts_tablet_manager.cc:1916] T b253bd741d094067902a02bc765ee0fb P e01ecd008edd4aa2b6b2b47efbc2ea87: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:16.896333 11512 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831"
dest_uuid: "12682eb70bc84c76879e04a8628ee586"
 from {username='slave'} at 127.0.0.1:56416
I20260709 13:16:16.896924 11512 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:16.896781 11911 consensus_queue.cc:237] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:16.897311 11512 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:16.897778 10799 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:16.900084 10869 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:16.902356 10869 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:16.904249 11512 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:16.905318 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:16.905926 10799 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:16.907215 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:16.907841 10792 raft_consensus.cc:3055] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:16.907899 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:16.908236 10792 raft_consensus.cc:740] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:16.908478 10945 raft_consensus.cc:3055] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:16.908777 10945 raft_consensus.cc:740] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:16.909096 10792 consensus_queue.cc:260] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:16.909463 10945 consensus_queue.cc:260] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:16.910261 10792 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:16.910452 10945 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:16.912357 10799 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:16.913952 10842 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:16.915038 12140 raft_consensus.cc:2804] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:16.915711 12140 raft_consensus.cc:697] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:16.916671 12140 consensus_queue.cc:237] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:16.933450 11512 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:16.936267 10792 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2.
I20260709 13:16:16.937451 11478 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:16.939321 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:16.939940 10872 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:16.940816 12067 raft_consensus.cc:2804] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:16.941398 12067 raft_consensus.cc:697] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 2 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:16.942317 12067 consensus_queue.cc:237] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:16.882221 12148 raft_consensus.cc:2243] T 19c20efd6c04443596bb712e3c93c650 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:16.944222 12148 raft_consensus.cc:2272] T 19c20efd6c04443596bb712e3c93c650 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:16.949839 12069 raft_consensus.cc:993] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26: : Instructing follower 12682eb70bc84c76879e04a8628ee586 to start an election
I20260709 13:16:16.950510 12069 raft_consensus.cc:1081] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Signalling peer 12682eb70bc84c76879e04a8628ee586 to start an election
I20260709 13:16:16.951489 10872 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2.
I20260709 13:16:16.954138 11512 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dde624b07ed446e5b7347791196396e2"
dest_uuid: "12682eb70bc84c76879e04a8628ee586"
 from {username='slave'} at 127.0.0.1:56410
I20260709 13:16:16.954895 11512 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:16.955825 10945 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:16.956106 11512 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:16.965821 12148 ts_tablet_manager.cc:1916] T 19c20efd6c04443596bb712e3c93c650 P 468a8fd46e824f62b9c2c0d3532a6d26: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:16.993268 11512 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:16.995354 12077 raft_consensus.cc:993] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2: : Instructing follower 12682eb70bc84c76879e04a8628ee586 to start an election
I20260709 13:16:16.995929 12077 raft_consensus.cc:1081] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Signalling peer 12682eb70bc84c76879e04a8628ee586 to start an election
I20260709 13:16:16.996275 11512 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:16.996937 12105 ts_tablet_manager.cc:1929] T b253bd741d094067902a02bc765ee0fb P e01ecd008edd4aa2b6b2b47efbc2ea87: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:16.997390 12105 log.cc:1199] T b253bd741d094067902a02bc765ee0fb P e01ecd008edd4aa2b6b2b47efbc2ea87: Deleting WAL directory at /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-0-root/wals/b253bd741d094067902a02bc765ee0fb
I20260709 13:16:16.997707 11507 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665"
dest_uuid: "12682eb70bc84c76879e04a8628ee586"
 from {username='slave'} at 127.0.0.1:56404
I20260709 13:16:16.998282 11507 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:16.998636 11507 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:17.006728 11507 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:17.010291 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:17.010968 10945 raft_consensus.cc:3055] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:17.011220 12148 ts_tablet_manager.cc:1929] T 19c20efd6c04443596bb712e3c93c650 P 468a8fd46e824f62b9c2c0d3532a6d26: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:17.011305 10945 raft_consensus.cc:740] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:17.011740 12148 log.cc:1199] T 19c20efd6c04443596bb712e3c93c650 P 468a8fd46e824f62b9c2c0d3532a6d26: Deleting WAL directory at /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-1-root/wals/19c20efd6c04443596bb712e3c93c650
I20260709 13:16:17.012440 10945 consensus_queue.cc:260] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:17.013607 11507 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:17.013645 10945 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:17.023509 10945 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2.
I20260709 13:16:17.024928 11477 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:17.025717 12067 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:17.026079 12067 raft_consensus.cc:697] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 2 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:17.026672 12067 consensus_queue.cc:237] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:17.088433 10686 catalog_manager.cc:5028] TS e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): tablet b253bd741d094067902a02bc765ee0fb (table test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]) successfully deleted
I20260709 13:16:17.637892 10799 raft_consensus.cc:1275] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:17.679925 10799 raft_consensus.cc:1275] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:17.812315 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:17.813210 10872 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:17.817298 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:17.818060 10869 raft_consensus.cc:3055] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:17.818439 10869 raft_consensus.cc:740] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:17.836436 10872 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2.
I20260709 13:16:17.846402 12139 consensus_queue.cc:1048] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 13:16:17.666473 12018 consensus_queue.cc:1048] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.173s
I20260709 13:16:18.037127 12156 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:18.037932 12156 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:18.076313 12067 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:18.077152 12067 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:18.081130 12067 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:18.081919 12067 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:18.082641 12067 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:18.085882 12067 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:18.041003 12156 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:16:18.187892 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 415338us
W20260709 13:16:18.216993 10897 debug-util.cc:398] Leaking SignalData structure 0x7b0800154ee0 after lost signal to thread 9685
W20260709 13:16:18.219295 10897 debug-util.cc:398] Leaking SignalData structure 0x7b080009cc40 after lost signal to thread 10733
I20260709 13:16:18.218685 10869 consensus_queue.cc:260] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:18.221673 10869 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:18.228768 12089 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:16:18.544080 10839 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.763s	user 0.005s	sys 0.017s
I20260709 13:16:18.556608 12078 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:18.557610 12078 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:18.633042 12154 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:17.768049 10686 catalog_manager.cc:5028] TS 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): tablet 19c20efd6c04443596bb712e3c93c650 (table test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]) successfully deleted
W20260709 13:16:18.644095 10686 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 875838us
I20260709 13:16:18.172827  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:18.229925 12089 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:18.669126 12015 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:18.670431 12158 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:18.671453 12158 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:18.673547 10943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 1089) took 1498 ms. Trace:
I20260709 13:16:18.670149 12015 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:18.674939 12156 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:18.675580 12158 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:18.675976 12129 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:18.676316 12158 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:18.676823 12129 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:18.677518 12015 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:18.677152 12158 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:18.679021 12015 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:18.675993 12156 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:18.680029 12015 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:18.681761 12156 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:18.682396 12156 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:18.660256 10947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 1088) took 1475 ms. Trace:
W20260709 13:16:18.657088 11475 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.004s	sys 0.001s
I20260709 13:16:18.682739 12015 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:18.661444 12160 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:16:18.694738 11475 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.005s	sys 0.001s
I20260709 13:16:18.634516 12154 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:18.700709 12160 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:18.703799 12078 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:18.686653 10947 rpcz_store.cc:276] 0709 13:16:17.174877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:18.648864 (+1473987us) service_pool.cc:224] Handling call
0709 13:16:18.649577 (+   713us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:18.650161 (+   584us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:18.650199 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:18.650279 (+    80us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:16:18.650343 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:18.650418 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:18.650586 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:16:18.712457 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 156344us
I20260709 13:16:18.714761 12158 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:18.716804 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:18.720468 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:16:18.721817 10841 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.835s	user 0.068s	sys 0.108s
W20260709 13:16:18.722276 10841 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.903s	user 0.068s	sys 0.108s
I20260709 13:16:18.739620 12156 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:18.739317 12160 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:18.741863 12160 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:18.745455 12160 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:18.746218 12089 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:18.748436 12160 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:18.744530 12156 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:18.748729 12165 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:18.702469 12015 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
W20260709 13:16:18.734669 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 944699us
I20260709 13:16:18.674238 10943 rpcz_store.cc:276] 0709 13:16:17.175169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:18.642380 (+1467211us) service_pool.cc:224] Handling call
0709 13:16:18.672286 (+ 29906us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:18.672758 (+   472us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:18.672804 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:18.672910 (+   106us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:18.672965 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:18.673089 (+   124us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:18.673278 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:18.753913 10696 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42968 (request call id 241) took 1856 ms. Trace:
I20260709 13:16:18.703007 12068 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:18.711644 12078 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:18.755796 12015 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:18.762202 12165 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:18.757532 10876 raft_consensus.cc:1275] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:18.757841 10696 rpcz_store.cc:276] 0709 13:16:16.895355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:16.895494 (+   139us) service_pool.cc:224] Handling call
0709 13:16:18.751284 (+1855790us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":106517,"reported_tablets":1}
I20260709 13:16:18.768900 12118 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:18.770350 12129 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:18.757513 10941 raft_consensus.cc:1275] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:18.774428 12068 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:18.779556 12068 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:18.779882 12158 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:18.780642 12158 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:18.781535 12015 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:18.782948 12166 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:18.769889 12118 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:18.783923 12166 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:18.782990 12071 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:18.789619 12161 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:18.789489 12164 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:18.790385 12071 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:18.791505 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:18.796568 12176 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:18.796773 12015 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:18.797868 12015 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:18.773026 12078 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:18.836082 12068 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:18.836946 12068 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:18.839368 12068 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:18.839988 12068 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:18.840265 12078 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:18.932241 12167 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:18.933171 12168 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:18.797479 12176 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:18.936303 12174 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:18.937083 12164 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:18.940040 10948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 1430) took 1768 ms. Trace:
I20260709 13:16:18.940302 12171 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:18.940975 12172 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:18.941776 12068 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:18.838263 12156 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:18.942929 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:18.943290 12071 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:18.946810 10946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 1427) took 1831 ms. Trace:
W20260709 13:16:18.948055 11478 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.116.129:39685 timed out after 1.581s (SENT) [suppressed 2 similar messages]
I20260709 13:16:18.951740 12166 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:18.814375 12158 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:18.993409 10948 rpcz_store.cc:276] 0709 13:16:17.169095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:18.804502 (+1635407us) service_pool.cc:224] Handling call
0709 13:16:18.810702 (+  6200us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:18.937542 (+126840us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:18.937571 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:18.937637 (+    66us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:18.937687 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:18.937757 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:18.937893 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:18.994024 12159 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:19.000073 10697 catalog_manager.cc:5697] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 1 to 2, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 2 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:18.790680 12161 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:19.002332 12015 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:19.004371 12160 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:19.161158 12175 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:19.162638 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:19.060190 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:19.063624 12178 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:16:19.185298 10869 consensus_meta.cc:287] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.554s	user 0.006s	sys 0.007s
I20260709 13:16:19.200100 10869 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2.
W20260709 13:16:19.202075 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 417209us
I20260709 13:16:19.202544 12179 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:19.134985 12156 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:19.143225 12166 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:19.187973 10697 catalog_manager.cc:5697] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 1 to 2, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 2 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:19.202071 12180 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:19.179237 12015 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:19.040978 10946 rpcz_store.cc:276] 0709 13:16:17.115630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:18.934597 (+1818967us) service_pool.cc:224] Handling call
0709 13:16:18.945679 (+ 11082us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:18.946081 (+   402us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:18.946128 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:18.946209 (+    81us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:18.946307 (+    98us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:18.946417 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:18.946585 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:19.224799 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:16:19.221586 10869 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49290 (request call id 60) took 1438 ms (client timeout 1581 ms). Trace:
I20260709 13:16:19.185524 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:16:19.233712 10869 rpcz_store.cc:269] 0709 13:16:17.773601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:17.817036 (+ 43435us) service_pool.cc:224] Handling call
0709 13:16:19.205113 (+1388077us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:19.189194 12160 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:19.239079 12187 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:19.239939 12161 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:19.220139 12170 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:19.219214 12186 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:19.227128 12165 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:19.097257 12167 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:19.135010 12174 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:19.255903 12181 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:19.219789 12175 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:19.286832 12015 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:19.256171 12154 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:19.064085 12168 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:19.187942 12172 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:19.266631 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 361857us
I20260709 13:16:19.186141 12171 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:19.295429 11899 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:19.203164 12159 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:19.292841 12184 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:19.312965 12068 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:19.316033 12078 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:19.316293 10867 raft_consensus.cc:1275] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:19.294185 12165 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.292418 12181 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:19.291026 12166 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:19.306443 12177 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.227327 12178 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:19.313303 12118 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:19.307819 12154 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.335098 10947 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 2.
I20260709 13:16:19.344182 12123 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.252671 12180 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:19.323894 12016 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:19.353528 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:19.343093 12176 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:19.344971 12123 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:19.349926 12118 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:19.293916 12179 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:19.345568 12078 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:19.355278 12015 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:19.354772 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 67) took 1175 ms. Trace:
I20260709 13:16:19.356948 11899 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:19.349439 12165 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:19.320303 12170 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:19.365052 12176 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:19.365968 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:19.307451 12187 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:19.373706 12160 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:19.370762 12127 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:19.363397 12161 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:19.390745 12168 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:19.379104 12175 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:19.365687 12016 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:19.381038 12118 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:19.378600 12167 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:19.370776 12154 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:19.403949 11899 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:19.349313 12186 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:19.399400 12156 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:19.403977 12169 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.400292 12160 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:19.358513 12177 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:19.424966 10897 debug-util.cc:398] Leaking SignalData structure 0x7b0800161a40 after lost signal to thread 10825
I20260709 13:16:19.385861 12176 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:19.428932 10897 debug-util.cc:398] Leaking SignalData structure 0x7b0800161920 after lost signal to thread 10826
I20260709 13:16:19.429665 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:19.430683 12185 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:19.374686 10867 rpcz_store.cc:276] 0709 13:16:18.175074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:18.217144 (+ 42070us) service_pool.cc:224] Handling call
0709 13:16:19.219315 (+1002171us) spinlock_profiling.cc:243] Waited 320 ms on lock 0x7b5800177098. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd90a22 00007f9c86ca9c09 00007f9c85e03ee0 00007f9c85ecd0ab 00007f9c85ecd131 00007f9c85e8efc0 00007f9c85e79929 00007f9c85e793d2 00007f9c7f3f0bd4
0709 13:16:19.226491 (+  7176us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:19.332545 (+106054us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:19.347861 (+ 15316us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":735951744}
I20260709 13:16:19.445268 12181 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:19.427944 12172 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:19.435667 11512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56416 (request call id 299) took 2246 ms. Trace:
I20260709 13:16:19.437016 12171 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:19.437242 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:19.444207 12123 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:19.411129 12068 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:19.405462 12168 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:19.430987 12016 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:19.361155 12184 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:19.447443 11512 rpcz_store.cc:276] 0709 13:16:17.188884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:19.434029 (+2245145us) service_pool.cc:224] Handling call
0709 13:16:19.434564 (+   535us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:19.435007 (+   443us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:19.435048 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:19.435123 (+    75us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:19.435170 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:19.435249 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:19.435425 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:19.449532 12196 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:19.456226 12191 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:19.456843 12191 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:19.456772 12196 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:19.446063 12156 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:19.426621 12161 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:19.445143 12185 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:19.463981 12018 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:19.466732 12135 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.466858 12139 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:19.468928 12140 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:19.466255 10941 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 1.
I20260709 13:16:19.449569 12138 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:19.466740 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:19.460512 12181 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:19.450528 12172 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:19.478441 10800 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:19.452877 12160 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:19.482149 11899 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:19.511798 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:19.507513 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:19.544116 12185 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:19.512138 12018 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:19.478436 12171 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:19.510911 12177 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:19.510900 12176 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:19.479703 10943 raft_consensus.cc:1275] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:19.546583 12185 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:19.512581 11509 raft_consensus.cc:1240] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:19.514135 12179 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:19.513481 12165 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:19.547914 12018 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:19.510537 12189 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:19.548739 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:19.482920 12168 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:19.510141 12068 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:19.549917 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:19.550879 12168 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:19.551129 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:19.551471 12068 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:19.550971 12171 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:19.547318 12185 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:19.555670 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:19.452966 12169 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:19.505574 12199 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:19.555516 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:19.564143 11899 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:19.553133 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:19.483635 12156 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:19.565689 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:19.568832 10793 raft_consensus.cc:1275] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:19.570835 10794 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:19.562659 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:19.449641 12127 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:19.576751 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:19.577972 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:19.578189 12182 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:19.578507 12184 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:19.579079 12170 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:19.570086 12192 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.573832 12160 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:19.581197 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:19.569090 12078 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:19.559489 12172 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:19.520206 12135 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:19.511322 12138 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:19.585939 12191 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:19.583313 11511 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 1.
I20260709 13:16:19.581053 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:19.555627 12140 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:19.554100 12139 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:19.565446 12177 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:19.609335 12177 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:19.564496 12181 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:19.548867 12189 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:19.574924 12154 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:19.587976 12161 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:19.590457 12118 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:19.596061 12196 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:19.594085 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:19.591637 12184 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:19.592079 12192 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:19.655476 10766 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:19.627113 10873 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 1.
I20260709 13:16:19.673957 12177 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:19.619905 11899 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:19.674234 12127 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:19.594755 11509 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 1.
I20260709 13:16:19.674280 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49290 (request call id 69) took 1498 ms. Trace:
I20260709 13:16:19.592196 12197 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:19.676589 10949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 1435) took 1015 ms. Trace:
I20260709 13:16:19.651243 11510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56416 (request call id 304) took 2112 ms. Trace:
I20260709 13:16:19.681710 10949 rpcz_store.cc:276] 0709 13:16:18.667562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:18.672128 (+  4566us) service_pool.cc:224] Handling call
0709 13:16:19.653932 (+981804us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b5800112598. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd90a22 00007f9c86ca9c09 00007f9c85e03ee0 00007f9c85ecd0ab 00007f9c85ecd131 00007f9c85e8efc0 00007f9c85e79929 00007f9c85e793d2 00007f9c7f3f0bd4
0709 13:16:19.654107 (+   175us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:19.676114 (+ 22007us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:19.676149 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:19.676203 (+    54us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:19.676236 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:19.676292 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:19.676405 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26621824}
I20260709 13:16:19.681778 11510 rpcz_store.cc:276] 0709 13:16:17.476320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:19.582274 (+2105954us) service_pool.cc:224] Handling call
0709 13:16:19.587873 (+  5599us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:19.588079 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:19.588112 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:19.588169 (+    57us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:19.588198 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:19.588252 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:19.588359 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:19.651347 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:19.677865 11899 consensus_queue.cc:1048] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 13:16:19.652623 12172 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:19.650974 12185 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:19.653616 12187 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:19.696553 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 214302us
I20260709 13:16:19.680922 12127 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:19.619942 12138 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:19.589017 12199 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:19.699765 10946 raft_consensus.cc:1275] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:19.653379 10942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 1431) took 2483 ms. Trace:
I20260709 13:16:19.626745 10943 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 1.
I20260709 13:16:19.697584 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:19.704900 12172 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:19.700920 10941 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 2.
I20260709 13:16:19.702872 12127 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:19.713946 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:19.680202 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:19.679000 10873 rpcz_store.cc:276] 0709 13:16:18.176104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:18.217144 (+ 41040us) service_pool.cc:224] Handling call
0709 13:16:19.580686 (+1363542us) spinlock_profiling.cc:243] Waited 320 ms on lock 0x7b5800177098. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd90a22 00007f9c86ca9c09 00007f9c85e03ee0 00007f9c85ecd0ab 00007f9c85ecd131 00007f9c85e7b0a0 00007f9c85e7a984 00007f9c7f3f56d4 00007f9c7f3f5662
0709 13:16:19.674209 (+ 93523us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":735634176}
I20260709 13:16:19.702653 12140 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:19.714074 12171 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:19.738718 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:19.739104 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:19.739267 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:19.739761 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:19.739903 12156 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:19.740198 12190 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:19.716552 12118 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:19.741705 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:19.727219 12127 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:19.738744 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:19.744884 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:19.752386 12187 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.726104 12015 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:19.745045 12169 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:19.702689 12184 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:19.747908 12135 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:19.743966 10870 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 2.
I20260709 13:16:19.751524 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:19.752362 12160 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:19.637264 12182 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:19.749383 10872 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 2.
I20260709 13:16:19.699638 12197 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:19.755419 12015 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:19.755234 12187 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:19.761505 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:19.760259 10794 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2.
I20260709 13:16:19.737624 12172 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:19.705260 10942 rpcz_store.cc:276] 0709 13:16:17.169411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:19.652254 (+2482843us) service_pool.cc:224] Handling call
0709 13:16:19.652654 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:19.652903 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:19.652935 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:19.653004 (+    69us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:19.653035 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:19.653103 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:19.653227 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:19.753854 12196 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
W20260709 13:16:19.905112 10897 debug-util.cc:398] Leaking SignalData structure 0x7b0800087360 after lost signal to thread 10900
I20260709 13:16:19.904721 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:19.897406 11477 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:19.973129 12182 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:19.973222 12187 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:19.928264 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:19.928165 10944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 1434) took 1677 ms. Trace:
I20260709 13:16:19.899362 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:19.817595 10945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 1426) took 2702 ms. Trace:
W20260709 13:16:19.975658 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 93549.5us
I20260709 13:16:19.978921 10949 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 1.
I20260709 13:16:19.977437 10945 rpcz_store.cc:276] 0709 13:16:17.115263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:19.806340 (+2691077us) service_pool.cc:224] Handling call
0709 13:16:19.806731 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:19.817087 (+ 10356us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:19.817117 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:19.817156 (+    39us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:19.817192 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:19.817248 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:19.817401 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:19.960690 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:19.899838 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:19.806192 12195 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:19.826730 12179 consensus_queue.cc:1048] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.169s
W20260709 13:16:19.945510 12203 log.cc:927] Time spent T 00000000000000000000000000000000 P ae957c8fc66c4b6ba063d0c74cbf5484: Append to log took a long time: real 0.115s	user 0.002s	sys 0.000s
I20260709 13:16:19.767678 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:19.789680 12199 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:19.976006 10944 rpcz_store.cc:276] 0709 13:16:18.558697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:18.559216 (+   519us) service_pool.cc:224] Handling call
0709 13:16:19.900751 (+1341535us) spinlock_profiling.cc:243] Waited 1.95 ms on lock 0x7b5800112598. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd90eee 0000000000375939 00007f9c85e041e0 00007f9c85ecd0c4 00007f9c85ecd131 00007f9c85e8efc0 00007f9c85e79929 00007f9c85e793d2 00007f9c7f3f0bd4
0709 13:16:19.900933 (+   182us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:19.904539 (+  3606us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:19.904569 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:19.904612 (+    43us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:19.904658 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:19.904723 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:19.904852 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4490624}
I20260709 13:16:19.766099 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:19.796479 12139 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:19.830554 12184 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:19.797456 12181 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:19.827744 12192 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:20.000367 10950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 1428) took 2849 ms. Trace:
I20260709 13:16:19.975418 12196 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:19.976743 12171 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:19.820446 12189 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:20.046388 12171 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:20.046519 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:20.046929 12171 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:20.048178 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:20.049664 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:19.836048 12186 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:20.005569 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:20.006898 12127 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:19.791616 12118 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:20.054215 12127 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:20.054641 12118 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:19.789965 12190 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:20.011165 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:20.007854 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:20.012944 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:20.012342 12172 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:20.013417 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:19.993295 12160 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:20.013067 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:20.019157 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:20.019433 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:20.027510 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:20.027201 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:20.019671 10841 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:20.028102 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:20.020953 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:20.023638 10875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 64) took 2206 ms. Trace:
I20260709 13:16:20.014493 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:20.024406 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:20.030740 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:20.023900 10698 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42956 (request call id 255) took 3093 ms. Trace:
I20260709 13:16:20.024299 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:20.025367 12197 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:20.013257 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:20.025226 10766 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:20.014358 12015 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:20.018652 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:20.028841 12180 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:20.027586 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:20.034514 12195 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:20.036188 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:20.033528 10950 rpcz_store.cc:276] 0709 13:16:17.150897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:19.999415 (+2848518us) service_pool.cc:224] Handling call
0709 13:16:19.999759 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:19.999932 (+   173us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:19.999955 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:20.000005 (+    50us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:20.000037 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:20.000097 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:20.000214 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:20.026849 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:20.041180 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:20.036814 10842 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:20.123190 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:20.243369 10873 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 2.
I20260709 13:16:20.244437 10876 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 1.
W20260709 13:16:20.245922 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 220776us
W20260709 13:16:20.242592 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 205857us
I20260709 13:16:20.242568 10944 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 1.
W20260709 13:16:20.217334 10897 debug-util.cc:398] Leaking SignalData structure 0x7b0800268680 after lost signal to thread 10974
I20260709 13:16:20.243316 10872 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 1.
I20260709 13:16:20.209353 12189 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:20.243825 10868 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 2.
I20260709 13:16:20.249006 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:20.242077 10875 rpcz_store.cc:276] 0709 13:16:17.816871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:18.217797 (+400926us) service_pool.cc:224] Handling call
0709 13:16:20.022735 (+1804938us) spinlock_profiling.cc:243] Waited 14.6 ms on lock 0x7b5800177098. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd90a22 00007f9c86ca9c09 00007f9c85e03ee0 00007f9c85ecd0ab 00007f9c85ecd131 00007f9c85e8efc0 00007f9c85e79929 00007f9c85e793d2 00007f9c7f3f0bd4
0709 13:16:20.022968 (+   233us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:20.023183 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:20.023210 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:20.023269 (+    59us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:20.023301 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:20.023358 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:20.023466 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":33587456}
I20260709 13:16:20.250667 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:20.250859 10697 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42506 (request call id 150) took 3057 ms. Trace:
W20260709 13:16:20.251073 10897 debug-util.cc:398] Leaking SignalData structure 0x7b08002bd2e0 after lost signal to thread 11536
I20260709 13:16:20.249212 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:20.232554 11507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56416 (request call id 298) took 3044 ms. Trace:
W20260709 13:16:20.252816 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 233222us
I20260709 13:16:20.250427 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:20.245952 10794 raft_consensus.cc:2376] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2. Re-sending same reply.
I20260709 13:16:20.255151 10876 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 2.
I20260709 13:16:20.242692 10698 rpcz_store.cc:276] 0709 13:16:16.930636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:16.945166 (+ 14530us) service_pool.cc:224] Handling call
0709 13:16:20.023673 (+3078507us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1470845,"reported_tablets":4}
I20260709 13:16:20.235846 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:20.332082 12090 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:20.332244 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:20.332448 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:20.254354 11507 rpcz_store.cc:276] 0709 13:16:17.188536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:20.231582 (+3043046us) service_pool.cc:224] Handling call
0709 13:16:20.231932 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:20.232132 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:20.232155 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:20.232210 (+    55us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:20.232236 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:20.232285 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:20.232394 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:20.257830 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:20.252480 11899 raft_consensus.cc:2749] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:20.330088 10799 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 1.
I20260709 13:16:20.330399 10766 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:20.332877 10871 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 1.
I20260709 13:16:20.216001 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:20.254552 12118 raft_consensus.cc:2804] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:20.215657 12077 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:20.109663 12198 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:20.344506 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:20.344810 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:16:20.337647 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 83050.6us
I20260709 13:16:20.055699 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:20.344467 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:20.256141 12174 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:20.347431 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:20.347365 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:20.202823 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:20.355983 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:20.357335 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:20.349010 10799 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 2.
I20260709 13:16:20.349640 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:20.054981 12170 consensus_queue.cc:1048] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.285s
I20260709 13:16:20.351519 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:20.099587 12179 consensus_queue.cc:1048] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.253s
I20260709 13:16:20.111997 12171 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:20.352052 12160 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:20.336026 10792 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 1.
I20260709 13:16:20.345763 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:20.355320 12070 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:20.358547 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:20.359542 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:20.365890 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:20.372516 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:20.340576 10942 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:20.249070 12189 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:20.254029 10941 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:20.348294 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:20.344071 12118 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:20.373044 10871 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 2.
I20260709 13:16:20.374750 12070 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:20.375100 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:20.373200 10801 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 1.
I20260709 13:16:20.376535 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:20.373154 10875 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 2.
I20260709 13:16:20.375975 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:20.372362 12064 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.359647 12077 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:20.386548 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:20.391040 12196 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:20.384400 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:20.373416 10874 raft_consensus.cc:2376] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2. Re-sending same reply.
I20260709 13:16:20.392259 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:20.390426 11478 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:20.363790 12187 consensus_queue.cc:1048] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 13:16:20.378928 10792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 1461) took 1021 ms. Trace:
I20260709 13:16:20.381323 12118 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:20.435372 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:20.409258 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:20.382666 12070 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:20.408810 10793 raft_consensus.cc:2410] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:16:20.465966 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:20.385589 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:20.466638 12077 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:20.475031 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:20.475664 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:20.482839 10871 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 1.
I20260709 13:16:20.489141 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:20.493767 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:20.497156 10767 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:16:20.497962 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 116526us
W20260709 13:16:20.499454 12205 log.cc:927] Time spent T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
I20260709 13:16:20.500284 12124 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:20.500727 12124 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:20.501087 12124 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:20.379582 12198 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:20.466408 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:20.488149 10916 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:20.527959 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:20.498713 10767 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:20.440747 12171 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:20.526371 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 135971us
I20260709 13:16:20.530002 11478 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:20.530915 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:20.532291 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:20.532311 10767 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:20.534035 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:20.534616 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:20.533115 10799 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:20.535279 10949 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:20.536523 10767 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:20.251258 10697 rpcz_store.cc:276] 0709 13:16:17.193443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:17.193607 (+   164us) service_pool.cc:224] Handling call
0709 13:16:20.250764 (+3057157us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:16:19.515037 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:16:19.595376 (+ 80339us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:16:19.595442 (+    66us) write_op.cc:435] Acquired schema lock
0709 13:16:19.595482 (+    40us) tablet.cc:662] Decoding operations
0709 13:16:19.596117 (+   635us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:16:19.596206 (+    89us) write_op.cc:641] Partition lock acquired for write op
0709 13:16:19.596243 (+    37us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:16:19.596751 (+   508us) tablet.cc:701] Row locks acquired
0709 13:16:19.596797 (+    46us) write_op.cc:260] PREPARE: finished
0709 13:16:19.596971 (+   174us) write_op.cc:270] Start()
0709 13:16:19.609081 (+ 12110us) write_op.cc:276] Timestamp: P: 1783602979596904 usec, L: 0
0709 13:16:19.609134 (+    53us) op_driver.cc:348] REPLICATION: starting
0709 13:16:19.609942 (+   808us) log.cc:844] Serialized 915 byte log entry
0709 13:16:20.052601 (+442659us) op_driver.cc:464] REPLICATION: finished
0709 13:16:20.216217 (+163616us) write_op.cc:301] APPLY: starting
0709 13:16:20.216343 (+   126us) tablet.cc:1367] starting BulkCheckPresence
0709 13:16:20.216532 (+   189us) tablet.cc:1370] finished BulkCheckPresence
0709 13:16:20.216700 (+   168us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:16:20.230062 (+ 13362us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:16:20.230516 (+   454us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:16:20.230606 (+    90us) write_op.cc:312] APPLY: finished
0709 13:16:20.231020 (+   414us) log.cc:844] Serialized 29 byte log entry
0709 13:16:20.231273 (+   253us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:20.231386 (+   113us) write_op.cc:454] Released schema lock
0709 13:16:20.231462 (+    76us) write_op.cc:341] FINISH: Updating metrics
0709 13:16:20.231942 (+   480us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:20.231979 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1442875,"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":163273,"apply.run_cpu_time_us":3313,"apply.run_wall_time_us":15740,"num_ops":2,"prepare.queue_time_us":84815,"prepare.run_cpu_time_us":16948,"prepare.run_wall_time_us":199826,"raft.queue_time_us":44564,"raft.run_cpu_time_us":8935,"raft.run_wall_time_us":140953,"replication_time_us":443308,"thread_start_us":181454,"threads_started":3,"wal-append.queue_time_us":218523}]]}
I20260709 13:16:20.540747 10767 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:16:20.541378 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 206484us
I20260709 13:16:20.537711 10871 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 2.
I20260709 13:16:20.542073 10916 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:20.542614 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:20.542749 12179 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:20.543267 12179 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:20.543565 10916 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:20.543833 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:20.544445 10793 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 1.
I20260709 13:16:20.544631 12126 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:20.545012 12126 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:20.545320 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:20.546132 12180 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:20.543673 12179 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:20.546515 12180 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:20.547156 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:20.545354 12126 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:20.547344 12180 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:20.548480 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:20.537582 12127 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:20.546486 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:20.550290 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:20.529345 12170 raft_consensus.cc:2749] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:20.529453 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:20.555555 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:20.556070 10799 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 2.
I20260709 13:16:20.537799 12128 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:20.441800 10792 rpcz_store.cc:276] 0709 13:16:19.355541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:19.357743 (+  2202us) service_pool.cc:224] Handling call
0709 13:16:20.249052 (+891309us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b5800002798. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd90a22 00007f9c86ca9c09 00007f9c85e03ee0 00007f9c85ecd0ab 00007f9c85ecd131 00007f9c85e7b0a0 00007f9c85e7a984 00007f9c7f3f56d4 00007f9c7f3f5662
0709 13:16:20.376699 (+127647us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24532736}
I20260709 13:16:20.554442 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:20.564433 10801 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:20.389415 12064 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:20.561231 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:20.551702 12077 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:20.551460 12127 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:20.574617 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:20.573859 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:20.391754 10796 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2.
I20260709 13:16:20.549219 12071 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:20.577806 12015 raft_consensus.cc:2749] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:20.579525 12198 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:20.580511 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:20.585417 10942 raft_consensus.cc:1275] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:20.615358 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:20.616878 12178 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:20.576246 12077 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:20.623875 12190 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:20.624295 12077 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:20.581705 12127 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:20.589197 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:20.560425 12128 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:20.649693 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1126) took 1052 ms. Trace:
I20260709 13:16:20.649984 12128 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:20.650271 10867 rpcz_store.cc:276] 0709 13:16:19.597405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:19.602936 (+  5531us) service_pool.cc:224] Handling call
0709 13:16:19.649965 (+ 47029us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:19.676465 (+ 26500us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:19.676502 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:19.676574 (+    72us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:19.676620 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:20.649369 (+972749us) spinlock_profiling.cc:243] Waited 21.9 ms on lock 0x7b540006c7d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:20.649429 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:20.649617 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":50476800}
I20260709 13:16:20.652470 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:20.658746 12164 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:20.592188 12195 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:20.593545 10947 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41014 (request call id 1106) took 1651 ms (client timeout 2038 ms). Trace:
I20260709 13:16:20.583315 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:16:20.661976 10947 rpcz_store.cc:269] 0709 13:16:18.941811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:18.942615 (+   804us) service_pool.cc:224] Handling call
0709 13:16:20.593324 (+1650709us) spinlock_profiling.cc:243] Waited 323 ms on lock 0x7b5400399c54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:20.593460 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":742669184}
I20260709 13:16:20.627753 11478 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:16:20.639928 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 96404us
W20260709 13:16:20.645306 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 107194us
I20260709 13:16:20.690604 10916 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:20.691759 10793 raft_consensus.cc:1240] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:20.586333 12071 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:20.692803 12071 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:20.693388 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:20.694023 10869 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2.
I20260709 13:16:20.704486 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:20.705184 12187 raft_consensus.cc:2749] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:20.705674 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:20.692485 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:20.714337 10869 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 2.
I20260709 13:16:20.643806 12189 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:20.717948 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:20.719815 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:20.720499 10873 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 1.
I20260709 13:16:20.720826 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:20.722261 10869 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 1.
I20260709 13:16:20.721388 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:16:20.723320 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 95670.3us
I20260709 13:16:20.723555 10797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 1460) took 1367 ms. Trace:
I20260709 13:16:20.724715 10797 rpcz_store.cc:276] 0709 13:16:19.354709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:19.354889 (+   180us) service_pool.cc:224] Handling call
0709 13:16:19.365709 (+ 10820us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:19.602938 (+237229us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:19.602961 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:19.609041 (+  6080us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:19.609086 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:20.721986 (+1112900us) spinlock_profiling.cc:243] Waited 229 ms on lock 0x7b540026b954. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:20.722034 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:20.722167 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":526955904}
I20260709 13:16:20.726979 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:20.727200 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:20.727974 10869 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 2.
I20260709 13:16:20.728389 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:20.729476 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:20.730010 10873 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 1.
I20260709 13:16:20.728777 10916 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:20.732769 12166 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:20.727862 11478 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:16:20.735251 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:20.726217 12189 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:20.740365 12189 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:20.740939 12189 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:20.741020 12090 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:20.741554 12090 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:20.736307 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:20.743300 10869 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 2.
I20260709 13:16:20.746618 11478 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:16:20.751363 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 210689us
I20260709 13:16:20.751220 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:20.752195 12171 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:20.752770 12171 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:20.753779 11478 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:20.753249 10767 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:20.755399 10767 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:20.756779 10767 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:20.753232 12171 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:20.759176 12129 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:20.759497 12064 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:20.759622 12129 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:20.760134 12129 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:20.758704 10767 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:20.763260 11478 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:16:20.763402 10767 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:20.767248 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:20.765197 11478 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
W20260709 13:16:20.771101 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 43839.1us
I20260709 13:16:20.771790 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:20.749274 12187 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:20.780628 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:20.761358 12166 raft_consensus.cc:2749] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:20.766166 12195 raft_consensus.cc:2749] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:20.787348 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:20.819857 10943 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 1.
I20260709 13:16:20.765972 12170 raft_consensus.cc:2749] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:20.766338 12186 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:20.750133 12207 raft_consensus.cc:2804] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:20.818544 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:20.815819 12164 raft_consensus.cc:2749] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:20.844033 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:20.788151 12178 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:20.851143 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:20.852800 10873 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 2.
I20260709 13:16:20.841895 12186 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:20.842926 12207 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:20.850056 12170 raft_consensus.cc:2804] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:20.874553 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:16:20.874706 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 109591us
I20260709 13:16:20.876228 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:20.878378 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:20.881686 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:20.892233 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:20.898272 10799 raft_consensus.cc:1275] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:20.878542 12186 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:20.912504 10766 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:20.911470 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:20.913522 10950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41020 (request call id 1453) took 1030 ms. Trace:
I20260709 13:16:20.913828 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:20.913815 10950 rpcz_store.cc:276] 0709 13:16:19.883584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:20.889726 (+1006142us) service_pool.cc:224] Handling call
0709 13:16:20.913464 (+ 23738us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:20.917577 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:20.917874 12208 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:20.918498 12208 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:20.918962 12208 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:20.919116 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:20.885844 12170 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:20.885156 12207 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:20.921236 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:20.928146 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:20.856223 12178 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:20.931808 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:20.935233 10798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 1462) took 1557 ms. Trace:
I20260709 13:16:20.935043 12215 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:20.935779 10798 rpcz_store.cc:276] 0709 13:16:19.356432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:19.356660 (+   228us) service_pool.cc:224] Handling call
0709 13:16:19.383342 (+ 26682us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:19.444851 (+ 61509us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:19.444888 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:19.444946 (+    58us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:19.444995 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:20.889848 (+1444853us) spinlock_profiling.cc:243] Waited 41.9 ms on lock 0x7b54000d7354. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:20.891594 (+  1746us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:20.911987 (+ 20393us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":96420224}
I20260709 13:16:20.936163 12215 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:20.941958 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:16:20.946096 10915 connection.cc:256] RPC call timeout handler was delayed by -1.1451873410001099s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.168s for a response
I20260709 13:16:20.951713 11478 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:20.893987 12213 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:20.927591 12170 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:20.962704 11510 raft_consensus.cc:1240] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:20.966219 10916 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:20.967535 10916 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:20.948895 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:20.940864 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:20.992035 10767 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:21.008423 10767 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:21.009738 10767 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:21.016659 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:20.966698 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:21.021981 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:21.019761 12190 raft_consensus.cc:2804] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:21.023337 12172 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:21.023496 12190 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:21.023782 12172 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:21.023895 12190 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:21.024035 12214 raft_consensus.cc:2749] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:21.024159 12172 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:21.028420  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:21.002270 12215 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:21.011459 12078 raft_consensus.cc:2749] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:21.011710 12195 raft_consensus.cc:2749] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:20.969765 12180 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:21.049510 12170 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:21.051813 12170 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:21.032974 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:20.966178 12070 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:20.964059 12213 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:21.034677 12118 consensus_meta.cc:287] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.544s	user 0.011s	sys 0.013s
I20260709 13:16:21.016829 12159 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:21.012481 10795 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 1.
I20260709 13:16:21.039481 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:21.081924 10947 raft_consensus.cc:2393] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 3: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:21.066219 12215 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:21.081375 12219 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:21.093592 12219 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:21.093863 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:21.094450 12070 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:21.112409 10697 catalog_manager.cc:5697] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 1 to 2, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 2 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:21.091144 12215 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:21.114681 10697 catalog_manager.cc:5697] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 1 to 2, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 2 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:21.119539 10842 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:21.120539 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:21.120817 12224 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:21.121212 12159 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:21.080914 12118 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:21.135041 10842 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:16:21.148500 12179 consensus_meta.cc:287] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.602s	user 0.019s	sys 0.007s
I20260709 13:16:21.148381 12129 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:21.148985 12179 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:21.150918 12128 consensus_meta.cc:287] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.500s	user 0.022s	sys 0.004s
I20260709 13:16:21.151576 12129 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:21.151331 12128 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:21.152765 10947 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2778f463396643809ca1f6bef2755afc"
dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
mode: GRACEFUL
new_leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
 from {username='slave'} at 127.0.0.1:46052
I20260709 13:16:21.153544 10947 raft_consensus.cc:606] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Received request to transfer leadership to TS 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:21.154013 12128 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:21.156407 12179 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:21.087534 12187 raft_consensus.cc:697] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 2 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:21.091691 12127 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:21.161227 10842 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:21.166358 10842 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:21.117995 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:21.172708 10842 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:21.180531 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:21.159978 12178 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:21.135645 12215 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:16:21.220824 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 211116us
I20260709 13:16:21.179858 12064 raft_consensus.cc:2804] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:21.170022 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:16:21.219527 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 46919.5us
I20260709 13:16:21.138898 12168 raft_consensus.cc:2749] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:16:21.222209 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 254699us
I20260709 13:16:21.227316 10767 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:21.236066 10842 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:21.236780 10767 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:16:21.224242 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 272653us
I20260709 13:16:21.263095 12065 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:21.230254 12064 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:21.263782 10767 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:21.263139 11478 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:16:21.266397 12223 log.cc:927] Time spent T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 13:16:21.226735 10792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1135) took 1192 ms. Trace:
W20260709 13:16:21.268790 12124 consensus_meta.cc:287] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.767s	user 0.035s	sys 0.021s
W20260709 13:16:21.233330 12216 log.cc:927] Time spent T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.225s	user 0.001s	sys 0.000s
I20260709 13:16:21.223219 12184 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:21.160331 12224 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:21.300223 11478 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:21.180022 12158 raft_consensus.cc:2749] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:21.287122 12180 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:21.293885 12015 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:21.285742 12218 raft_consensus.cc:2749] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:16:21.278699 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 158576us
I20260709 13:16:21.295943 12072 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:21.179095 12159 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:21.293819 10792 rpcz_store.cc:276] 0709 13:16:20.032805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:21.053055 (+1020250us) service_pool.cc:224] Handling call
0709 13:16:21.083597 (+ 30542us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:21.093353 (+  9756us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:21.093393 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:21.093445 (+    52us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:21.093501 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:21.093570 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:21.219473 (+125903us) spinlock_profiling.cc:243] Waited 28.1 ms on lock 0x7b5400294d50. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:21.225594 (+  6121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":64646912}
I20260709 13:16:21.307332 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:16:21.303659 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 138570us
I20260709 13:16:21.288144 10916 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:21.283162 12065 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:21.332090 12072 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:21.343245 12065 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:21.343508 12072 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:21.291301 12064 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:21.275152 10795 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 1.
W20260709 13:16:21.321182 12220 log.cc:927] Time spent T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 13:16:21.298209 12184 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:21.328291 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:21.325013 12172 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:21.304827 12179 raft_consensus.cc:2749] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:21.331432 10915 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:21.319238 12219 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:21.297283 12124 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:21.368470 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 45800.5us
I20260709 13:16:21.319095 12015 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:21.316224 12229 raft_consensus.cc:2749] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:16:21.354586 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 66526us
I20260709 13:16:21.325867 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:21.320001 12214 consensus_queue.cc:1048] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 13:16:21.364336 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:16:21.381629 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 53126us
I20260709 13:16:21.370416 11477 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:21.415493 10950 raft_consensus.cc:2393] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 3: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
W20260709 13:16:21.414809 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 98836.2us
W20260709 13:16:21.417119 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 46793.5us
I20260709 13:16:21.367662 12184 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:21.421536 11477 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:21.366029 12227 raft_consensus.cc:2804] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:16:21.393985 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 64352.8us
I20260709 13:16:21.366111 12070 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:16:21.427250 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 95899.6us
I20260709 13:16:21.356145 12164 consensus_queue.cc:1059] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:16:21.432822 10801 raft_consensus.cc:1240] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:21.326025 12187 consensus_queue.cc:237] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:21.381183 12170 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:21.399478 12118 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:21.415182 12015 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:21.370507 12213 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:21.445647 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:21.418431 12219 raft_consensus.cc:2749] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:21.424566 12179 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:21.426427 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:21.426359 12227 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:21.456214 10801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1136) took 1422 ms. Trace:
I20260709 13:16:21.460948 12127 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:21.452950 12179 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:21.424458 10941 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:21.473394 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:21.475605 12070 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:21.479437 12214 raft_consensus.cc:2749] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:21.480289 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:21.481952 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:21.481019 10943 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 1.
I20260709 13:16:21.425235 12208 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:21.484633 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:21.484134 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:21.466064 10801 rpcz_store.cc:276] 0709 13:16:20.034269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:20.646051 (+611782us) service_pool.cc:224] Handling call
0709 13:16:20.646481 (+   430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:21.435028 (+788547us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:21.455968 (+ 20940us) spinlock_profiling.cc:243] Waited 786 ms on lock 0x7b540039f8d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:21.456154 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1807999744}
I20260709 13:16:21.487170 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:16:21.471627 12077 consensus_meta.cc:287] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.847s	user 0.016s	sys 0.022s
I20260709 13:16:21.480938 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:21.472569 12227 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:16:21.485729 12126 consensus_meta.cc:287] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.934s	user 0.019s	sys 0.009s
I20260709 13:16:21.477370 12179 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:21.490700 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:21.481458 12070 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:21.517223 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 281223us
I20260709 13:16:21.510211 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:21.519726 10842 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:16:21.485044 12189 consensus_meta.cc:287] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.743s	user 0.009s	sys 0.013s
I20260709 13:16:21.527760 10799 raft_consensus.cc:1240] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:16:21.527875 10944 raft_consensus.cc:1240] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:21.530102 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:21.444553 12170 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:21.523587 10943 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 2.
I20260709 13:16:21.544653 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:21.546995 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:16:21.558458 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 126137us
I20260709 13:16:21.580790 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:16:21.549929 12071 consensus_meta.cc:287] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.856s	user 0.024s	sys 0.003s
I20260709 13:16:21.541705 12168 raft_consensus.cc:2749] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:16:21.597417 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 161326us
I20260709 13:16:21.523602 12189 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:21.599998 10841 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:21.510720 12077 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:21.523029 12126 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:21.603127 10841 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:21.583050 12224 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:21.577487 12158 consensus_queue.cc:1059] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 13:16:21.600576 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:21.608584 12189 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:21.609798 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:21.610891 10841 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:21.591099 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:21.613513 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:21.615751 10766 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:21.616803 12127 raft_consensus.cc:2749] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:16:21.616921 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 72866.6us
I20260709 13:16:21.617201 10766 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:21.618714 10766 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:21.623227 10766 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:21.626682 10766 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:21.606575 12172 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:21.591423 12071 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:21.642000 12195 consensus_queue.cc:1059] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:21.643857 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:21.614462 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:16:21.660334 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 49559.7us
I20260709 13:16:21.628955 12128 raft_consensus.cc:2749] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:21.624087 12127 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:21.629258 12078 raft_consensus.cc:2749] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:21.664618 10841 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:21.665537 10841 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:21.618620 12232 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:21.672390 10841 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:21.642870 10946 raft_consensus.cc:1240] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:21.668862 10800 raft_consensus.cc:1240] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:21.680359 12248 raft_consensus.cc:3055] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:21.681052 12248 raft_consensus.cc:740] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:21.682235 12248 consensus_queue.cc:260] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:21.684115 12248 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:21.691011 12227 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:21.667896 12190 consensus_meta.cc:287] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.638s	user 0.008s	sys 0.005s
I20260709 13:16:21.695324 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:21.629069 12236 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:21.693995 12215 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:21.697602 12243 raft_consensus.cc:2749] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:16:21.697082 12171 consensus_meta.cc:287] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.938s	user 0.009s	sys 0.014s
I20260709 13:16:21.673817 12127 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:16:21.694882 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 145319us
I20260709 13:16:21.627630 12244 raft_consensus.cc:2749] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:21.704859 10916 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:16:21.709487 12186 consensus_meta.cc:287] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.784s	user 0.003s	sys 0.007s
I20260709 13:16:21.704346 12190 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:21.711551 12170 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:21.712049 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:21.692432 12124 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:21.698661 12077 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:21.713007 12071 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:21.713140 12207 consensus_meta.cc:287] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.756s	user 0.008s	sys 0.003s
I20260709 13:16:21.674839 12232 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:21.714591 12227 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:21.715507 12168 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:16:21.695904 12234 log.cc:927] Time spent T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.250s	user 0.000s	sys 0.001s
I20260709 13:16:21.647853 12158 raft_consensus.cc:2749] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:16:21.717685 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 291517us
W20260709 13:16:21.752729 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 331257us
I20260709 13:16:21.700630 12228 raft_consensus.cc:3055] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:21.722533 10916 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:21.729771 10947 raft_consensus.cc:2393] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 3: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:21.733781 12070 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:21.731050 12208 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:21.737231 12190 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:21.729395 12186 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:21.738322 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:21.738353 10794 raft_consensus.cc:2393] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 3: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
W20260709 13:16:21.739261 12090 consensus_meta.cc:287] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.997s	user 0.002s	sys 0.023s
I20260709 13:16:21.736971 12015 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:21.722065 12127 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:21.729027 12224 raft_consensus.cc:2749] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:21.758165 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:21.734382 12065 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:21.772490 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:16:21.759344 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 115087us
I20260709 13:16:21.773921 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:21.747838 12232 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:21.780150 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 104545us
I20260709 13:16:21.721985 12171 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:21.800884 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:21.786175 10916 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:21.747865 12251 raft_consensus.cc:2749] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:21.803647 12077 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:21.804047 12065 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:21.797148 12126 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:21.787196 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:21.787714 11510 raft_consensus.cc:1240] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:21.747700 12207 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:21.784042 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:16:21.815119 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 59066.7us
I20260709 13:16:21.815420 11507 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:21.749387 12168 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:21.791810 10948 raft_consensus.cc:1240] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:21.755594 12228 raft_consensus.cc:740] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:21.819839 12171 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:21.796794 10766 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:21.820024 12228 consensus_queue.cc:260] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:21.821280 12228 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:21.822038 10950 raft_consensus.cc:1240] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:21.803094 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:21.803442 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:21.806459 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:21.788486 11512 raft_consensus.cc:1240] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:21.807701 10916 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:21.804666 12243 consensus_queue.cc:1048] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:21.809298 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:21.796217 12072 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:21.809835 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:21.809739 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:21.799099 11509 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:21.811261 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:21.811952 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:21.811265 12070 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:21.759534 12170 consensus_queue.cc:1048] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:16:21.785774 12242 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:21.843021 10842 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:21.827824 12195 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:21.823310 10841 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:21.832887 11504 raft_consensus.cc:1240] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:21.836483 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:21.822461 12168 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:21.808646 12179 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:21.841006 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:21.824831 12215 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:21.821424 12184 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:21.843698 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:21.856416 11507 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 2.
I20260709 13:16:21.857230 11511 raft_consensus.cc:1240] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:21.829901 10916 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:21.833765 12221 raft_consensus.cc:2804] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:21.790326 12090 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:21.846817 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:21.848644 10801 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c"
dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
mode: GRACEFUL
new_leader_uuid: "12682eb70bc84c76879e04a8628ee586"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:16:21.848976 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:21.863829 10801 raft_consensus.cc:606] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Received request to transfer leadership to TS 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:21.850127 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:21.840543 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:21.853443 12179 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:21.850069 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:21.860378 10916 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:21.861143 11478 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:21.846082 12195 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:21.867029 10876 raft_consensus.cc:1240] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:21.844235 12242 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:21.868178 10841 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:21.865036 12251 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:21.868820 11478 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:21.869860 12072 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:21.865146 12213 raft_consensus.cc:2749] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:21.862723 12221 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:21.864775 12249 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:21.841806 12187 consensus_queue.cc:1059] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:16:21.897533 10801 raft_consensus.cc:1240] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:21.892362 10944 raft_consensus.cc:1240] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:16:21.874459 11478 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:21.886168 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:21.876003 12184 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:21.878326 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:21.924252 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:21.924752 10792 raft_consensus.cc:1240] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:21.876158 12090 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:21.873600 12242 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:21.871459 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:21.928143 10792 consensus_queue.cc:237] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:21.928835 10793 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "741efe79e97646b396f0091dc10c745e"
dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
mode: GRACEFUL
new_leader_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:16:21.915488 12127 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:21.929399 10793 raft_consensus.cc:606] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Received request to transfer leadership to TS cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:21.881690 12244 raft_consensus.cc:2749] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:21.931195 12127 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:21.876874 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:21.872407 10916 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:21.869648 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:21.866993 12256 raft_consensus.cc:2749] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:16:21.879577 12064 consensus_meta.cc:287] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.514s	user 0.010s	sys 0.000s
W20260709 13:16:21.932651 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 60286.7us
I20260709 13:16:21.905603 12179 raft_consensus.cc:2804] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:21.877812 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:21.916610 12249 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:21.920370 12170 raft_consensus.cc:2749] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:21.953758 12249 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:21.921772 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:21.927132 12187 raft_consensus.cc:3055] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:21.952963 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:21.895684 10950 raft_consensus.cc:1240] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:21.917583 12221 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:21.932634 10915 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:21.933039 12213 raft_consensus.cc:2749] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:21.918164 10801 raft_consensus.cc:1240] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:21.954574 12187 raft_consensus.cc:740] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:21.936378 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:21.934317 11509 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 2.
I20260709 13:16:21.872991 12215 raft_consensus.cc:2749] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:21.937430 12171 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:21.873070 12195 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:21.939157 12064 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:21.876514 12186 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:21.944419 12232 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:21.946982 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:21.938278 12168 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:21.947575 10800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1137) took 1913 ms. Trace:
I20260709 13:16:21.967608 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:21.968647 10766 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:21.940311 10916 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:21.951522 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:21.951757 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:21.952361 10941 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:21.878485 12015 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:21.947660 12179 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:21.955566 10944 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d32f5e230e464e61b23a41962708d120"
dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
mode: GRACEFUL
new_leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
 from {username='slave'} at 127.0.0.1:46052
I20260709 13:16:21.960024 11477 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:16:21.964869 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 96764us
I20260709 13:16:21.963429 12213 raft_consensus.cc:2749] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:21.972086 10800 rpcz_store.cc:276] 0709 13:16:20.034562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:20.654374 (+619812us) service_pool.cc:224] Handling call
0709 13:16:20.654747 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:21.679366 (+1024619us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:21.947338 (+267972us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400386154. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:21.947515 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2331706752}
I20260709 13:16:21.980620 10944 raft_consensus.cc:606] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Received request to transfer leadership to TS 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:21.960238 12242 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:21.976475 12259 consensus_queue.cc:1059] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:21.979832 10841 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:21.967537 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:21.974437 12172 raft_consensus.cc:2804] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:21.990458 12172 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:21.990958 12172 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:21.992463 12242 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:21.976930 12127 raft_consensus.cc:2749] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:21.998474 10916 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:22.000559 10841 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:22.001420 10943 raft_consensus.cc:1240] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:16:22.001608 12264 raft_consensus.cc:2749] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:21.967653 12171 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:21.980996 12179 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:22.006840 10841 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:21.973292 12126 consensus_queue.cc:1059] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:16:21.980752 12256 raft_consensus.cc:2749] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:22.003391 12184 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:22.006034 12213 consensus_queue.cc:1059] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 13:16:22.026270 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:22.026419 12184 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:22.027009 12184 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:22.027097 12126 raft_consensus.cc:3055] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:22.003746 12064 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:22.027565 12126 raft_consensus.cc:740] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:22.028514 12126 consensus_queue.cc:260] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:22.028831 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:22.004838 12217 raft_consensus.cc:993] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2: : Instructing follower 468a8fd46e824f62b9c2c0d3532a6d26 to start an election
I20260709 13:16:21.982391 12229 consensus_queue.cc:1059] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:16:21.982690 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:22.030701 12126 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:22.002677 10916 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:22.008836 12264 raft_consensus.cc:2749] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:22.009548 12236 consensus_queue.cc:1059] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:22.009471 12127 consensus_queue.cc:1059] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:22.011909 12171 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:22.005292 12187 consensus_queue.cc:260] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:22.008011 10766 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:22.040853 12236 raft_consensus.cc:3055] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:22.041036 12128 raft_consensus.cc:2804] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:22.041309 12236 raft_consensus.cc:740] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:22.040860 12127 raft_consensus.cc:3055] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:22.041886 12127 raft_consensus.cc:740] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:22.042752 12127 consensus_queue.cc:260] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:22.044008 10766 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:22.015435 12232 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:22.019569 10871 raft_consensus.cc:1240] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:22.046118 12208 raft_consensus.cc:2749] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:22.006285 12242 raft_consensus.cc:2749] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:22.047992 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:22.019848 10794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 1490) took 1011 ms. Trace:
I20260709 13:16:22.049543 10794 rpcz_store.cc:276] 0709 13:16:21.008886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:21.009022 (+   136us) service_pool.cc:224] Handling call
0709 13:16:22.019665 (+1010643us) spinlock_profiling.cc:243] Waited 715 ms on lock 0x7b54001aa4d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:22.019794 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1645626240}
I20260709 13:16:22.020012 10798 raft_consensus.cc:1240] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:16:22.004179 12259 raft_consensus.cc:3055] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:22.022305 10799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 53) took 1128 ms. Trace:
I20260709 13:16:22.008104 10841 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:16:22.592128 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 584058us
I20260709 13:16:22.592859 10841 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:22.593958 10841 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:22.036502 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:22.596169 10801 raft_consensus.cc:2393] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 3: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:22.028980 10795 raft_consensus.cc:1240] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:22.020574 10876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1134) took 1964 ms. Trace:
I20260709 13:16:22.035531 10916 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:22.039076 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:22.602873 12224 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:16:22.603034 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 567556us
I20260709 13:16:22.043051 12128 raft_consensus.cc:697] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:22.605193 12064 raft_consensus.cc:2749] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:22.605904 12064 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:22.606578 12064 raft_consensus.cc:2749] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:22.056111 12160 consensus_queue.cc:1059] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:16:22.039983 12172 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:22.611528 12172 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:22.034080 12215 raft_consensus.cc:2749] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:22.055058 12127 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:22.617738 10875 raft_consensus.cc:1240] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:22.004343 12168 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:21.988209 12170 consensus_queue.cc:1059] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:16:22.622239 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:16:22.044806 12190 consensus_queue.cc:1175] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" responder_term: 1 status { last_received { term: 1 index: 4 } last_committed_idx: 4 last_received_current_leader { term: 1 index: 4 } }
I20260709 13:16:22.031602 12213 raft_consensus.cc:3055] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:22.029807 12217 raft_consensus.cc:1081] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Signalling peer 468a8fd46e824f62b9c2c0d3532a6d26 to start an election
I20260709 13:16:22.064385 12171 consensus_queue.cc:1059] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 13:16:22.170033 12249 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:22.631655 10867 raft_consensus.cc:2393] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 2: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:22.631793 10942 raft_consensus.cc:1240] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:22.632440 12249 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:22.245049 12259 raft_consensus.cc:740] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:22.358106 12159 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:22.377406 12267 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:22.410538 11504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56416 (request call id 321) took 1484 ms. Trace:
I20260709 13:16:22.633414 12267 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:22.633550 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:22.633747 11504 rpcz_store.cc:276] 0709 13:16:20.926097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:20.928185 (+  2088us) service_pool.cc:224] Handling call
0709 13:16:20.938692 (+ 10507us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:21.848380 (+909688us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:22.410251 (+561871us) spinlock_profiling.cc:243] Waited 884 ms on lock 0x7b54001f90d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:22.410452 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2032709888}
W20260709 13:16:22.431645 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:22.635855 12267 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:22.636458 12171 raft_consensus.cc:3055] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:16:22.636633 10947 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41020 (request call id 1473) took 1477 ms (client timeout 1523 ms). Trace:
I20260709 13:16:22.438724 10870 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dde624b07ed446e5b7347791196396e2"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "12682eb70bc84c76879e04a8628ee586"
 from {username='slave'} at 127.0.0.1:53748
W20260709 13:16:22.637458 10947 rpcz_store.cc:269] 0709 13:16:21.159772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:21.170161 (+ 10389us) service_pool.cc:224] Handling call
0709 13:16:22.631469 (+1461308us) spinlock_profiling.cc:243] Waited 505 ms on lock 0x7b54001e2b54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:22.636571 (+  5102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1162506496}
W20260709 13:16:22.639307 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:22.639061 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:22.639914 10943 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 1.
I20260709 13:16:22.642544 11477 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:22.643409 12267 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:22.643772 12267 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:22.077226 12236 consensus_queue.cc:260] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:22.458262 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:22.644855 10942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 1129) took 1208 ms. Trace:
I20260709 13:16:22.645412 10942 rpcz_store.cc:276] 0709 13:16:21.436302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:21.444162 (+  7860us) service_pool.cc:224] Handling call
0709 13:16:22.631601 (+1187439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:22.632293 (+   692us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:22.644607 (+ 12314us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54001e2b50. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:22.644784 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2684368128}
I20260709 13:16:22.647861 12217 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:22.481813 12070 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:22.648334 12217 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:22.648545 12070 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:22.650455 12217 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:22.650862 10798 raft_consensus.cc:1275] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:22.515107 11478 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:16:22.652117 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 137128us
I20260709 13:16:22.517474 10946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 56) took 2953 ms. Trace:
I20260709 13:16:22.650485 12070 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:22.524233 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:22.655218 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:22.655786 10797 raft_consensus.cc:2393] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 2: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:22.655827 10944 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:22.657186 10797 raft_consensus.cc:1240] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:22.654672 10946 rpcz_store.cc:276] 0709 13:16:19.564275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:20.526498 (+962223us) service_pool.cc:224] Handling call
0709 13:16:20.559929 (+ 33431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:21.693542 (+1133613us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:22.517162 (+823620us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b540018ecd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:22.517392 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2442249088}
I20260709 13:16:22.525647 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:16:22.527074 10792 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.004s	sys 0.006s
W20260709 13:16:22.658368 10792 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.698s	user 0.005s	sys 0.006s
I20260709 13:16:22.658447 10794 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 1.
I20260709 13:16:22.528578 12255 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:22.660326 12255 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:22.662498 12255 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:22.560137 10941 consensus_meta.cc:287] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.583s	user 0.013s	sys 0.000s
I20260709 13:16:22.665406 10941 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2.
I20260709 13:16:22.563282 10696 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42984 (request call id 242) took 1027 ms. Trace:
I20260709 13:16:22.565008 12261 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:22.666411 10696 rpcz_store.cc:276] 0709 13:16:21.535388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:21.535692 (+   304us) service_pool.cc:224] Handling call
0709 13:16:22.563209 (+1027517us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1}
I20260709 13:16:22.666671 12261 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:22.668289 10871 raft_consensus.cc:1275] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:22.669210 12261 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:22.565832 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:22.508602 10766 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:22.566916 10698 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42956 (request call id 256) took 1133 ms. Trace:
W20260709 13:16:22.670706 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 162177us
I20260709 13:16:22.670778 10870 raft_consensus.cc:606] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Received request to transfer leadership to TS 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:22.671280 12255 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:22.127638 12184 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:22.670732 10698 rpcz_store.cc:276] 0709 13:16:21.433282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:21.441396 (+  8114us) service_pool.cc:224] Handling call
0709 13:16:22.566848 (+1125452us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44236,"reported_tablets":2}
I20260709 13:16:22.673373 12184 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:22.673601 10946 raft_consensus.cc:1240] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:22.581758 10799 rpcz_store.cc:276] 0709 13:16:20.893461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:20.933186 (+ 39725us) service_pool.cc:224] Handling call
0709 13:16:21.047612 (+114426us) spinlock_profiling.cc:243] Waited 7.41 ms on lock 0x7b5800002798. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd90a22 00007f9c86ca9c09 00007f9c85e03ee0 00007f9c85ecd0ab 00007f9c85ecd131 00007f9c85e8efc0 00007f9c85e79929 00007f9c85e793d2 00007f9c7f3f0bd4
0709 13:16:21.054449 (+  6837us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:21.528296 (+473847us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:22.022030 (+493734us) spinlock_profiling.cc:243] Waited 465 ms on lock 0x7b54000e8b54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:22.022218 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1085545600}
I20260709 13:16:22.584566 10767 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:22.674036 11477 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:16:22.674810 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 90314.1us
I20260709 13:16:22.675117 12239 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:22.675697 12239 raft_consensus.cc:697] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 2 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:22.675735 10946 raft_consensus.cc:1240] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:22.676808 12239 consensus_queue.cc:237] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:22.586611 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:22.678081 10800 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:22.588100 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:22.588702 12187 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:22.679890 12164 raft_consensus.cc:2749] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:22.600418 10795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 54) took 1175 ms. Trace:
I20260709 13:16:22.571707 12015 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:22.603964 10916 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:22.680606 10795 rpcz_store.cc:276] 0709 13:16:21.424967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:21.426887 (+  1920us) service_pool.cc:224] Handling call
0709 13:16:22.028633 (+601746us) spinlock_profiling.cc:243] Waited 6.76 ms on lock 0x7b5800002798. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd90eee 0000000000375939 00007f9c85e041e0 00007f9c85ecd0c4 00007f9c85ecd131 00007f9c85e8efc0 00007f9c85e79929 00007f9c85e793d2 00007f9c7f3f0bd4
0709 13:16:22.028805 (+   172us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:22.600220 (+571415us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:22.600372 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15557376}
I20260709 13:16:22.601864 10876 rpcz_store.cc:276] 0709 13:16:20.056205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:20.938122 (+881917us) service_pool.cc:224] Handling call
0709 13:16:20.938358 (+   236us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:21.867428 (+929070us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:22.020376 (+152948us) spinlock_profiling.cc:243] Waited 928 ms on lock 0x7b5400340654. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:22.020524 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2135363456}
W20260709 13:16:22.681807 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 77926.8us
I20260709 13:16:22.610414 10841 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:22.682745 10916 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:16:22.683640 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 73302.9us
I20260709 13:16:22.609089 12214 raft_consensus.cc:3055] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:22.684209 12215 raft_consensus.cc:2749] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:22.684444 12214 raft_consensus.cc:740] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:22.685016 12215 raft_consensus.cc:2749] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:22.622677 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:22.021677 12179 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:22.689445 12179 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:22.690272 11507 raft_consensus.cc:1240] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:22.684489 10841 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:22.692346 12168 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:22.692464 10841 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:22.694185 12243 raft_consensus.cc:2749] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:22.685590 12015 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:22.695828 10841 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:22.697454 12015 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:22.578399 12207 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:22.625778 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:22.622355 12170 raft_consensus.cc:3055] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:22.684250 10916 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:22.627127 12213 raft_consensus.cc:740] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:22.700961 12172 raft_consensus.cc:2749] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:22.700766 12213 consensus_queue.cc:260] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:22.701789 12172 consensus_queue.cc:1059] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:22.702137 12213 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:22.628602 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:22.703195 10868 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:22.606098 12128 consensus_queue.cc:237] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:22.704552 10841 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:22.636880 12171 raft_consensus.cc:740] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:22.706926 12171 consensus_queue.cc:260] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:22.707866 12171 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:22.709229 12261 raft_consensus.cc:2804] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:22.703635 12172 raft_consensus.cc:3055] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:22.709678 12261 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:22.709959 12172 raft_consensus.cc:740] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:22.710305 12261 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:22.633579 12259 consensus_queue.cc:260] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:22.711186 12172 consensus_queue.cc:260] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:22.660565 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:22.532250 12256 raft_consensus.cc:3055] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:22.712420 12256 raft_consensus.cc:740] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:22.713229 12256 consensus_queue.cc:260] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:22.663420 12160 raft_consensus.cc:2749] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:22.664454 12235 consensus_queue.cc:1059] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:16:22.714279 12256 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:22.536444 12208 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:22.671304 10870 raft_consensus.cc:614] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:22.715016 12208 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:22.717056 10841 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:22.626940 10842 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:22.718345  9682 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 13:16:22.585106 12195 consensus_meta.cc:287] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.611s	user 0.014s	sys 0.001s
I20260709 13:16:22.719192 12243 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:22.719239 12195 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:22.720412 10796 raft_consensus.cc:1240] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:22.620764 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:22.721483 12195 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:22.721796 10875 raft_consensus.cc:2393] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 3: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:22.723726 12214 consensus_queue.cc:260] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:22.725425 12214 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:22.683679 12184 raft_consensus.cc:2749] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:22.623132 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:22.699559 12170 raft_consensus.cc:740] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:22.702185 12255 raft_consensus.cc:697] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:22.729146 12259 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:22.729265 12255 consensus_queue.cc:237] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:22.567696 12078 raft_consensus.cc:3055] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:22.717247 12208 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:22.677371 10767 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:22.720481  9682 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:16:22.731724 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 54429.7us
I20260709 13:16:22.699666 10874 raft_consensus.cc:2393] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 3: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:22.650743 10943 raft_consensus.cc:1240] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:16:22.732574 10767 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:22.733858 12195 raft_consensus.cc:2749] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:22.734474 12195 raft_consensus.cc:2749] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:22.734963 10944 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 2.
I20260709 13:16:22.727825 12170 consensus_queue.cc:260] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:22.737418 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:22.738008 12170 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:22.700790 10916 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:22.739619 12184 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:22.741677 11507 raft_consensus.cc:1240] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:22.743974 12236 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:22.755725 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:22.754381 12243 raft_consensus.cc:2749] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:16:22.759145 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 58435.6us
I20260709 13:16:22.760363 10916 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:22.761420 12215 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:22.673169 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:22.445055 12221 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:22.765383 12221 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:22.768067 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:22.768674 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:22.768638 12235 consensus_queue.cc:1048] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:22.769613 10950 raft_consensus.cc:2393] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 3: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:22.770551 10800 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:22.644236 12267 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:22.750841 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:22.750209 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:22.774573 12215 consensus_queue.cc:1059] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:22.751513 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:22.730973 12078 raft_consensus.cc:740] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:22.775815 12217 raft_consensus.cc:3055] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:22.776327 12078 consensus_queue.cc:260] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:22.776865 12221 consensus_queue.cc:1059] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:22.777479 12078 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:22.776242 12217 raft_consensus.cc:740] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:22.777923 12221 raft_consensus.cc:3055] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:22.778472 12221 raft_consensus.cc:740] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:22.773394 10916 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:22.747543 10943 raft_consensus.cc:1240] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:22.754352 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:22.812254 10794 raft_consensus.cc:1240] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:22.778717 12172 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:22.775213 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:22.772743 11478 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:16:22.817610 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 44920us
I20260709 13:16:22.818338 11478 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:22.782624 10868 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 2.
I20260709 13:16:22.785442 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:16:22.739120 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 112379us
I20260709 13:16:22.779135 12217 consensus_queue.cc:260] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:22.793057 12215 raft_consensus.cc:2749] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:22.793315 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:22.793550 12195 consensus_queue.cc:1059] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:22.830330 10793 raft_consensus.cc:1240] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:22.794759 10944 raft_consensus.cc:1240] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:22.794633 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:22.786988 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:22.795251 10945 raft_consensus.cc:1240] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:22.795661 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:22.843109 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:22.843963 12256 consensus_queue.cc:1048] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:22.796293 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:22.796749 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:22.797676 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:22.789237 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:22.798413 12168 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:22.799763 10874 raft_consensus.cc:1240] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:22.801105 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:22.802103 11504 raft_consensus.cc:2410] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:16:22.803750 12070 raft_consensus.cc:993] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87: : Instructing follower cb0428cc55984daba1ba63744a5102a2 to start an election
I20260709 13:16:22.862366 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:22.864179 10872 raft_consensus.cc:1240] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:22.865547 10872 raft_consensus.cc:1240] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:22.804823 12184 raft_consensus.cc:2749] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:22.809887 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:22.834581 12207 raft_consensus.cc:2749] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:22.805440 10766 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:22.821203 10873 raft_consensus.cc:1240] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:22.825475 10841 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:16:22.872931 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 67568.2us
I20260709 13:16:22.846007 12187 raft_consensus.cc:2749] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:22.817525 12261 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:22.876068 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:22.876824 10873 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 2.
I20260709 13:16:22.877128 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:22.854542 12217 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:22.829459 10942 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 1.
I20260709 13:16:22.831508 10800 raft_consensus.cc:1240] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:22.832123 10916 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:22.833501 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:22.837951 11478 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:22.840553 12195 raft_consensus.cc:3055] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:22.842327 12264 consensus_queue.cc:1059] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:22.842744 12171 consensus_queue.cc:1059] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:16:22.884276 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 46463.4us
I20260709 13:16:22.795861 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:22.844126 10842 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:22.797106 10871 raft_consensus.cc:1240] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:22.847647 10875 raft_consensus.cc:2393] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 3: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:22.857077 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:22.853098 12065 consensus_queue.cc:1059] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 13:16:22.867923 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:22.869391 10799 raft_consensus.cc:2393] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 3: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:22.815034 10794 raft_consensus.cc:1240] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:22.832849 12170 raft_consensus.cc:2749] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:22.870970 12259 raft_consensus.cc:1081] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Signalling peer cb0428cc55984daba1ba63744a5102a2 to start an election
I20260709 13:16:22.873351 10797 raft_consensus.cc:1240] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:22.826215 10947 raft_consensus.cc:1240] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:16:22.874619 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:22.828805 10868 raft_consensus.cc:1240] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 1. Current term is 2. Ops: []
W20260709 13:16:22.875349 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 49947.4us
I20260709 13:16:22.875746 11507 raft_consensus.cc:1240] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 2 LEADER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:22.851229 12128 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
W20260709 13:16:22.879912 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 47870.9us
I20260709 13:16:22.912009 10916 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:22.881314 12127 consensus_queue.cc:1059] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:22.882055 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:22.883297 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:22.884719 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:22.876412 12070 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:22.872908 12221 consensus_queue.cc:260] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:22.882587 12195 raft_consensus.cc:740] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:22.882160 12213 raft_consensus.cc:2749] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:22.887018 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:22.890429 12264 raft_consensus.cc:3055] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:22.891003 12171 raft_consensus.cc:3055] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:16:22.891206 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 47152.7us
I20260709 13:16:22.896507 10766 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:22.881958 12158 raft_consensus.cc:2749] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:22.941929 12261 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:22.943885 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:22.945574 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:22.899165 10796 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 3.
I20260709 13:16:22.886062 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:22.877537 10873 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2778f463396643809ca1f6bef2755afc"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
 from {username='slave'} at 127.0.0.1:54950
I20260709 13:16:22.951006 10873 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:22.951573 10873 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:22.908464 10841 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:22.909819 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:16:22.952874 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 44498.3us
I20260709 13:16:22.953959 10841 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:22.954638 10794 raft_consensus.cc:1240] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:22.913247 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:22.913470 12239 consensus_queue.cc:1059] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:16:22.913247 12159 consensus_queue.cc:1059] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:22.916275 10793 raft_consensus.cc:2393] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 2: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:22.911207 12168 raft_consensus.cc:2804] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:22.917949 12128 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:22.916074 10945 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 2.
I20260709 13:16:22.900167 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:22.920621 12242 raft_consensus.cc:2749] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:22.920388 12224 consensus_queue.cc:1059] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:22.921521 10916 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:16:22.966269 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 44821.2us
I20260709 13:16:22.900298 12187 raft_consensus.cc:2804] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:22.932458 12171 raft_consensus.cc:740] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:22.972223 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:22.936383 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:22.937127 12078 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:16:22.937273 10766 consensus_peers.cc:597] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 -> Peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Couldn't send request to peer 12682eb70bc84c76879e04a8628ee586. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 695ff1b5a68e4111917d909909488d39. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:22.937405 12274 log.cc:927] Time spent T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:16:22.937704 12195 consensus_queue.cc:260] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:22.938967 12070 raft_consensus.cc:697] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:22.939285 12221 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:22.948220 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:22.960093 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:22.961799 10767 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:22.927788 12264 raft_consensus.cc:740] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:22.969516 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:22.908876 12232 raft_consensus.cc:3055] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:23.012207 12128 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:22.960050 12243 raft_consensus.cc:2749] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:23.010483 10795 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 2.
I20260709 13:16:22.969466 12207 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:23.021386 12232 raft_consensus.cc:740] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:23.022841 12232 consensus_queue.cc:260] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:23.021780 12264 consensus_queue.cc:260] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:23.008709 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:22.960345 12261 raft_consensus.cc:2749] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:23.029043 10801 raft_consensus.cc:2393] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 2: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:23.016388 12224 raft_consensus.cc:3055] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:16:23.050738 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 89040us
I20260709 13:16:22.976358 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:16:23.075443 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:23.077543 12243 consensus_queue.cc:1059] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:23.002027 12171 consensus_queue.cc:260] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:23.066165 12232 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:23.014706 10867 raft_consensus.cc:1240] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:22.978204 12168 raft_consensus.cc:697] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:16:23.080402 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 107063us
I20260709 13:16:23.083052 10916 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:22.965062 12190 consensus_queue.cc:1059] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.103s
I20260709 13:16:22.977543 12078 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:22.972527 12187 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:23.087858 10916 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:16:23.073715 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 95938.9us
W20260709 13:16:23.064484 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 88855.9us
I20260709 13:16:23.085237 10767 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:23.068436 12128 raft_consensus.cc:2749] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:23.065822 12224 raft_consensus.cc:740] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:16:23.130228 10730 debug-util.cc:398] Leaking SignalData structure 0x7b0800038a40 after lost signal to thread 9685
I20260709 13:16:23.049463 11509 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 3.
I20260709 13:16:23.086210 12064 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:16:23.158695 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 77313.4us
I20260709 13:16:23.095996 12207 raft_consensus.cc:2749] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:22.970192 12273 consensus_queue.cc:1059] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.196s
I20260709 13:16:23.013415 12267 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:23.148379 12187 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:23.145481 12078 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:23.167264 10766 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:23.083498 12159 consensus_queue.cc:1059] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.079s
W20260709 13:16:23.188819 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 219801us
I20260709 13:16:23.162621 12255 raft_consensus.cc:2749] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:16:23.214259 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 47085.4us
I20260709 13:16:23.210250 12126 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:16:23.213116 10730 debug-util.cc:398] Leaking SignalData structure 0x7b0800039120 after lost signal to thread 10733
W20260709 13:16:23.224880 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 54856.1us
I20260709 13:16:23.226483 11510 raft_consensus.cc:1240] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:23.230396 10767 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:23.238595 10948 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "741efe79e97646b396f0091dc10c745e"
dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:41014
I20260709 13:16:23.239404 10948 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:23.239849 10948 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:23.242249 10767 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:23.246367 12287 raft_consensus.cc:2749] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:23.247373 12287 raft_consensus.cc:2749] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:23.250365 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:23.250869 12224 consensus_queue.cc:260] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:23.252269 12224 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:23.234834 12168 consensus_queue.cc:237] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:23.253153 12172 raft_consensus.cc:3055] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:23.253634 12172 raft_consensus.cc:740] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:23.234617 12255 raft_consensus.cc:2804] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:23.255481 12255 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:23.255990 12255 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:16:23.233822 10730 debug-util.cc:398] Leaking SignalData structure 0x7b08000391e0 after lost signal to thread 10747
I20260709 13:16:23.260799 12236 raft_consensus.cc:3055] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:23.261318 12236 raft_consensus.cc:740] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:23.261705 12267 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:23.263458 12282 consensus_queue.cc:1059] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 13:16:23.254554 12172 consensus_queue.cc:260] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:23.268399 12172 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:23.270746 12249 raft_consensus.cc:3055] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:23.271250 12249 raft_consensus.cc:740] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:23.272230 12249 consensus_queue.cc:260] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:23.273618 12249 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:23.275903 10873 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:23.281850 10950 raft_consensus.cc:1240] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:23.244109 10767 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:23.287343 10873 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:23.245600 12159 raft_consensus.cc:2749] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:16:23.295792 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 85226.3us
I20260709 13:16:23.222397 12070 consensus_queue.cc:237] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:23.234400 12126 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:23.302502 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:16:23.304172 10730 debug-util.cc:398] Leaking SignalData structure 0x7b08002f2c60 after lost signal to thread 10825
I20260709 13:16:23.303969 12126 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:16:23.305567 10730 debug-util.cc:398] Leaking SignalData structure 0x7b0800363da0 after lost signal to thread 10900
W20260709 13:16:23.312711 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 85502.1us
I20260709 13:16:23.313704 10841 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:23.320608 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:23.321532 11507 raft_consensus.cc:2393] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 2: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
W20260709 13:16:23.308557 10730 debug-util.cc:398] Leaking SignalData structure 0x7b080035cbe0 after lost signal to thread 10974
I20260709 13:16:23.343566 12228 raft_consensus.cc:2749] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:23.345050 10944 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
W20260709 13:16:23.346585 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:23.347779 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 103796us
I20260709 13:16:23.349779 10767 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:23.352877 10767 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:23.355947 10946 raft_consensus.cc:1275] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:23.357427 10942 raft_consensus.cc:1240] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:23.357702 10767 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:23.358530 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:23.359309 12280 raft_consensus.cc:3055] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:23.361936 12280 raft_consensus.cc:740] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:23.362023 12160 consensus_queue.cc:1048] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:23.363029 12280 consensus_queue.cc:260] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:23.364666 12227 raft_consensus.cc:3055] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:23.365068 12227 raft_consensus.cc:740] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:23.365206 11477 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:23.365959 12227 consensus_queue.cc:260] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:23.366628 12170 consensus_queue.cc:1059] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:16:23.367215 12227 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:23.367314 12065 raft_consensus.cc:2749] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:23.367715 12170 raft_consensus.cc:2749] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:23.368425 12292 raft_consensus.cc:3055] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:23.368820 12294 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:23.368877 12292 raft_consensus.cc:740] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:23.373015 12015 raft_consensus.cc:2749] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:23.373816 12015 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:23.339881 12235 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:16:23.379887 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 62191us
I20260709 13:16:23.377701 10948 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:23.381253 10841 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:23.384222 10948 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:23.393697 12186 consensus_queue.cc:1059] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:23.370193 12292 consensus_queue.cc:260] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:23.396539 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:16:23.397013 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 56331.2us
I20260709 13:16:23.400267 10949 raft_consensus.cc:1240] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:23.403492 12158 raft_consensus.cc:2749] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:23.404016 10941 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:23.404479 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:23.409337 12187 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:23.409650 12255 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:23.411128 11507 raft_consensus.cc:1240] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:23.413265 12064 raft_consensus.cc:2749] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:23.413861 12292 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:23.415833 10841 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:16:23.397508 10730 debug-util.cc:398] Leaking SignalData structure 0x7b080031ef80 after lost signal to thread 11536
W20260709 13:16:23.423411 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:23.384544 12078 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:23.388540 12235 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:23.428712 12290 consensus_queue.cc:1059] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:23.429672 12235 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:23.429950 12290 raft_consensus.cc:3055] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:23.430399 12290 raft_consensus.cc:740] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:23.431339 12242 raft_consensus.cc:993] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2: : Instructing follower 468a8fd46e824f62b9c2c0d3532a6d26 to start an election
I20260709 13:16:23.431124 12282 consensus_queue.cc:1059] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:23.431849 12242 raft_consensus.cc:1081] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Signalling peer 468a8fd46e824f62b9c2c0d3532a6d26 to start an election
I20260709 13:16:23.432336 12065 raft_consensus.cc:3055] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:23.432753 12065 raft_consensus.cc:740] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:23.433666 12065 consensus_queue.cc:260] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:23.428097 10947 raft_consensus.cc:2955] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:23.437963 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:23.439231 11510 raft_consensus.cc:1275] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:23.425254 12064 raft_consensus.cc:2749] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:23.441155 12065 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:23.441630 12078 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:23.445391 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:23.445619 12255 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:23.431540 12290 consensus_queue.cc:260] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:23.449048 12290 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:23.448673 12064 consensus_queue.cc:1059] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:23.436741 10874 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
I20260709 13:16:23.450181 12184 raft_consensus.cc:3055] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:23.450542 12184 raft_consensus.cc:740] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:23.451686 12184 consensus_queue.cc:260] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:23.452270 10873 raft_consensus.cc:1275] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:23.455600 10945 raft_consensus.cc:1275] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:23.457461 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:23.459705 12128 consensus_queue.cc:1048] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:23.460777 10946 raft_consensus.cc:1240] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 1. Current term is 2. Ops: []
W20260709 13:16:23.461674 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 373915us
I20260709 13:16:23.462414 10916 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:23.453183 12184 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:23.468743 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:16:23.469403 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 110286us
W20260709 13:16:23.471351 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:23.472064 11478 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:16:23.473601 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:23.474107 10869 raft_consensus.cc:1240] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:23.473282 10841 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:23.479625 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:23.496171 10916 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:23.497201 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:23.497872 10916 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:16:23.493494 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:23.506989 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:23.470086 12293 raft_consensus.cc:2749] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:16:23.509354 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:23.491675 12158 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:23.469899 12187 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:23.514422 10767 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:23.488754 12064 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:23.492712 12235 raft_consensus.cc:2749] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:23.519825 10916 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:23.512455 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:23.522915 12126 raft_consensus.cc:2749] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:23.504506 12273 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:23.470283 12239 consensus_queue.cc:1048] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:16:23.517508 12294 raft_consensus.cc:697] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:23.526026 10869 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d32f5e230e464e61b23a41962708d120"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
 from {username='slave'} at 127.0.0.1:54950
I20260709 13:16:23.526347 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:23.508049 12286 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:23.522711 12064 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:23.514537 12227 raft_consensus.cc:2749] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:23.514739 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:23.492571 12172 consensus_queue.cc:1059] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 13:16:23.529366 10916 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:23.506963 12217 consensus_queue.cc:1059] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 13:16:23.528462 10869 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:23.529784 11478 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:23.529265 12273 raft_consensus.cc:697] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:23.540695 12217 raft_consensus.cc:3055] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:23.540910 12214 raft_consensus.cc:3055] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:23.538180 10869 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:23.518352 12164 consensus_queue.cc:1059] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 13:16:23.536599 12285 consensus_queue.cc:1059] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 13:16:23.547153 12126 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:23.547451 12285 raft_consensus.cc:3055] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:23.531663 12286 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:23.547866 12285 raft_consensus.cc:740] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:23.547962 12286 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:23.539697 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:23.539708 12195 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:23.541375 12214 raft_consensus.cc:740] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:23.553417 10916 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:23.544260 12235 raft_consensus.cc:2749] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:23.551357 10947 raft_consensus.cc:1275] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:23.551774 11507 raft_consensus.cc:2393] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 2: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:23.546557 12164 raft_consensus.cc:3055] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:16:23.552745 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:23.521719 12158 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:23.536660 12064 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:23.541050 12273 consensus_queue.cc:237] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:23.552599 12227 consensus_queue.cc:1059] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:16:23.560942 10869 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:23.541193 12217 raft_consensus.cc:740] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:16:23.564020 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:23.527845 12306 raft_consensus.cc:2749] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:23.564714 10916 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:23.562675 12172 raft_consensus.cc:2749] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:23.575771 10916 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:23.575088 12224 raft_consensus.cc:2749] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:23.577409 10796 raft_consensus.cc:1240] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:23.555912 12164 raft_consensus.cc:740] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:23.580402 10869 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:23.552878 12285 consensus_queue.cc:260] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:23.586081 12126 consensus_queue.cc:1059] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 13:16:23.555472 12293 consensus_queue.cc:1059] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 13:16:23.584052 12242 raft_consensus.cc:2749] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:23.593384 10867 raft_consensus.cc:1240] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:23.593789 12164 consensus_queue.cc:260] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:23.594442 12292 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:23.595273 12164 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:23.595384 12292 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:23.597833 10767 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:23.591914 12228 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:23.601315 12242 consensus_queue.cc:1059] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:23.607818 12158 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:16:23.609700 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:23.608927 10766 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:23.615119 10766 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:23.590022 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:23.624555 12208 raft_consensus.cc:2749] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:23.625813 12127 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:23.623277 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:23.613049 12228 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:23.613402 12190 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:23.633571 12213 raft_consensus.cc:3055] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:23.634835 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:23.702951 11504 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:23.638468 12306 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.602943 12217 consensus_queue.cc:260] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:23.639442 10916 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:23.639506 10842 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:23.640741 12292 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:23.643922 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:23.645659 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:23.657958 10873 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2.
I20260709 13:16:23.583670 12214 consensus_queue.cc:260] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:23.640761 12244 raft_consensus.cc:3055] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:23.688910 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:23.699708 11507 raft_consensus.cc:1275] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:23.702219 12213 raft_consensus.cc:740] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:23.728096 10798 raft_consensus.cc:1240] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:23.722894 12249 raft_consensus.cc:2749] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:23.702688 12286 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:23.730333 11508 raft_consensus.cc:2393] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 2: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
W20260709 13:16:23.705798 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 66424.7us
I20260709 13:16:23.711410 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:23.731792 10916 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:23.732335 10794 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:23.734148 12242 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:23.735199 10876 raft_consensus.cc:1240] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 1. Current term is 2. Ops: []
W20260709 13:16:23.736585 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 120282us
I20260709 13:16:23.700853 12070 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.590173 12294 consensus_queue.cc:237] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:23.714130 12190 raft_consensus.cc:697] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:23.695870 12297 consensus_queue.cc:1048] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 13:16:23.711670 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:23.623234 12287 consensus_queue.cc:1059] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 13:16:23.767589 11511 raft_consensus.cc:1240] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:23.768404 12236 consensus_queue.cc:260] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:23.769102 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:23.714546 12306 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:23.720556 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:23.772631 10797 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:23.769910 12236 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:23.722210 12303 consensus_queue.cc:1048] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:16:23.738238 10841 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:23.737993 10943 raft_consensus.cc:3055] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:23.743947 10794 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 3.
I20260709 13:16:23.747442 12217 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:23.748286 12214 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:23.749214 12280 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:23.752667 12190 consensus_queue.cc:237] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:23.753505 12285 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:23.755287 12228 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:23.756588 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:23.757176 12171 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:23.721952 12295 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.717844 12227 raft_consensus.cc:2804] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:23.728096 12213 consensus_queue.cc:260] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:23.759763 12264 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:23.730333 12286 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:16:23.721909 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 82497.5us
I20260709 13:16:23.761114 12070 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:23.726341 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:23.774168 12293 consensus_queue.cc:1059] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 13:16:23.785718 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:23.786181 11478 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:23.789194 10793 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 2.
I20260709 13:16:23.790410  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:23.784902 10841 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:23.737668 12244 raft_consensus.cc:740] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:23.821440 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:23.788240 10943 raft_consensus.cc:740] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:23.824036 10799 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:23.787446 12158 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:23.822444 12166 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.808068 12255 consensus_queue.cc:1048] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:16:23.824990 12227 raft_consensus.cc:697] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 2 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:23.831240 12292 raft_consensus.cc:2804] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:23.828414 12273 raft_consensus.cc:2764] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won
I20260709 13:16:23.835928 12213 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:23.836163 12158 raft_consensus.cc:697] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:23.829380 12295 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:23.855850 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:23.857853 12291 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.858445 12244 consensus_queue.cc:260] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:23.863046 12309 ts_tablet_manager.cc:933] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586: Initiating tablet copy from peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685)
W20260709 13:16:23.863281 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 43916.3us
I20260709 13:16:23.868635 10800 raft_consensus.cc:3055] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:23.869722 12273 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:23.875898 10916 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:23.878332 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:23.881410 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:23.883553 10800 raft_consensus.cc:740] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:23.841063 12166 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:23.885219 10944 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 1.
I20260709 13:16:23.888626 12291 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:23.890059 12166 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:23.890810 10797 raft_consensus.cc:2393] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 2: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:23.891041 12291 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:23.892294 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:23.892925 10944 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 3.
I20260709 13:16:23.893268 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:23.878470 12235 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.894316 10801 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:23.895720  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:23.873226 12292 raft_consensus.cc:697] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 2 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:23.872032 12306 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:23.887885 10766 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:23.899847 10800 consensus_queue.cc:260] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:23.886391 12273 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:23.894394 12235 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:23.904075 11477 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:23.942744 10800 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:23.906461 12293 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:23.944955 12311 log.cc:927] Time spent T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:16:23.907263 12070 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:23.899940 12064 consensus_queue.cc:1059] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 13:16:23.948686 11478 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:23.915673 10941 raft_consensus.cc:1240] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:23.930045 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:23.928033 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:23.935367 12227 consensus_queue.cc:237] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:23.940358 10799 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:23.943905 12239 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:23.944597 12293 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:23.965592 10842 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:23.950088 11510 raft_consensus.cc:2410] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:16:23.939540 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:23.893954 12126 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:23.950616 10950 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
I20260709 13:16:23.968899 10944 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 2.
I20260709 13:16:23.971117 11511 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 2.
I20260709 13:16:23.972018 10766 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:23.942658 12244 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:23.942495 12297 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:23.975391 10766 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:23.841288 12129 raft_consensus.cc:2955] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:23.872526 10943 consensus_queue.cc:260] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:23.989185 10943 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:23.991425 12186 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:23.991896 12186 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:23.992252 12186 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:16:23.979146 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:23.982990 12297 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:23.957024 12179 raft_consensus.cc:2749] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:23.964041 12239 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:23.995860 10842 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:24.006436 10842 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:23.999029 12249 raft_consensus.cc:2804] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:23.969949 12126 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:24.016237 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:24.029841 11511 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 2.
I20260709 13:16:24.022258 10800 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 2.
I20260709 13:16:24.007670 10842 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:23.994195 12295 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:24.032917 10916 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:24.035326 10799 raft_consensus.cc:1240] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:24.002774 12297 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:24.011173 12239 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:24.023542 12242 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.027446 12313 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:24.031592 12126 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:24.023305 12249 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:23.977959 12158 consensus_queue.cc:237] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:24.044919 12224 raft_consensus.cc:2804] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:24.038602 12187 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.027102 12232 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:16:24.044603 12078 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 13:16:24.044443 12282 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:24.028035 12284 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:24.076071 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:24.041320 11504 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 2.
I20260709 13:16:24.052487 12313 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:24.066213 12293 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:24.056747 10945 raft_consensus.cc:1275] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:24.057173 12249 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:24.117247 12282 raft_consensus.cc:697] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:24.042315 12292 consensus_queue.cc:237] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:24.061597 12232 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:24.120774 12284 raft_consensus.cc:697] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:24.075870 12224 raft_consensus.cc:697] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 2 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:24.115083 12313 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:24.054371 12242 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:24.164609 12309 tablet_copy_client.cc:323] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Beginning tablet copy session from remote peer at address 127.9.116.129:39685
I20260709 13:16:24.062629 12187 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:24.143380 12232 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:24.176666 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:24.176920 12282 consensus_queue.cc:237] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:24.180094 10873 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "12682eb70bc84c76879e04a8628ee586"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:16:24.181190 10873 raft_consensus.cc:606] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Received request to transfer leadership to TS 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:24.169210 10867 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 2.
I20260709 13:16:24.175943 12273 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:24.178084 12235 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:24.197690 10916 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:24.186725 12129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:39430 (request call id 10) took 2319 ms. Trace:
I20260709 13:16:24.198946 12224 consensus_queue.cc:237] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:24.198994 12159 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:24.211050 12242 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:24.212026 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:24.213579 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:24.216850 10799 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 2.
I20260709 13:16:24.216912 10868 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 1.
I20260709 13:16:24.218078 10916 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:24.229717 10766 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:24.229244 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:24.238144 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:24.239337 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:24.248540 10872 raft_consensus.cc:1275] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:24.240115 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:24.249903 12243 raft_consensus.cc:3055] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:24.250052 12321 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:24.250301 12243 raft_consensus.cc:740] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:24.237241 10868 raft_consensus.cc:2955] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:24.242197 10811 tablet_copy_service.cc:140] P e01ecd008edd4aa2b6b2b47efbc2ea87: Received BeginTabletCopySession request for tablet 695ff1b5a68e4111917d909909488d39 from peer 12682eb70bc84c76879e04a8628ee586 ({username='slave'} at 127.0.0.1:39442)
I20260709 13:16:24.247434 10871 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 2.
I20260709 13:16:24.251368 10767 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:24.232280 12179 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.219944 12159 raft_consensus.cc:697] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 2 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:16:24.222157 12317 log.cc:927] Time spent T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:16:24.258606 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:24.263178 10800 raft_consensus.cc:2393] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 2: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:24.219460 12126 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:24.243636 10867 raft_consensus.cc:1240] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:24.227200 12291 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.251935 12078 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:24.246579 12280 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.229115 12242 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:24.235222 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:24.217298 12326 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:24.250649 12321 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:24.219611 12186 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:24.289618 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:24.277164 12126 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:24.270648 12319 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:24.279976 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:16:24.294166 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 114768us
I20260709 13:16:24.280730 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:24.278968 10945 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2.
I20260709 13:16:24.278445 10811 tablet_copy_service.cc:161] P e01ecd008edd4aa2b6b2b47efbc2ea87: Beginning new tablet copy session on tablet 695ff1b5a68e4111917d909909488d39 from peer 12682eb70bc84c76879e04a8628ee586 at {username='slave'} at 127.0.0.1:39442: session id = 12682eb70bc84c76879e04a8628ee586-695ff1b5a68e4111917d909909488d39
I20260709 13:16:24.280462 12242 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:24.238171 10698 catalog_manager.cc:5697] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 1 to 2, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 2 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:24.284582 10915 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:24.276602 12187 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:24.259523 12306 raft_consensus.cc:2749] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:24.253790 12184 consensus_queue.cc:1059] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 13:16:24.256835 12297 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:24.229183 12327 consensus_queue.cc:1048] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.071s
I20260709 13:16:24.253921 12243 consensus_queue.cc:260] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:24.278754 12078 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:24.330652 10871 raft_consensus.cc:1275] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:24.219779 12284 consensus_queue.cc:237] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:24.299677 10876 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:16:24.262413 12179 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:24.311323 10800 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2.
I20260709 13:16:24.336285 11478 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:24.311717 12242 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:24.341475 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:24.342234 10947 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 3.
I20260709 13:16:24.343808 10799 raft_consensus.cc:2393] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 3: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:24.344746 11477 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:24.294488 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:24.295092 12326 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:24.331255 12236 consensus_meta.cc:287] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.544s	user 0.004s	sys 0.007s
I20260709 13:16:24.338423 12296 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:24.339188 12186 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:24.339836 12179 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:24.315168 12286 raft_consensus.cc:2749] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:24.281358 12291 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:24.215054 12129 rpcz_store.cc:276] 0709 13:16:21.865865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:21.925762 (+ 59897us) service_pool.cc:224] Handling call
0709 13:16:21.960286 (+ 34524us) log.cc:844] Serialized 506 byte log entry
0709 13:16:24.181125 (+2220839us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3972,"raft.run_cpu_time_us":521,"raft.run_wall_time_us":521,"thread_start_us":698087,"threads_started":1,"wal-append.queue_time_us":162507,"wal-append.run_cpu_time_us":2641,"wal-append.run_wall_time_us":1022504}
I20260709 13:16:24.333843 12078 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:24.340009 12321 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:24.332683 12126 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:24.395951 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:24.398591 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:24.363191 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:24.358371 12328 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.349162 12214 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:24.366988 12326 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:24.367605 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:24.333725 12064 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:24.377197 11477 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:24.331897 12319 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:24.442631 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 65550.8us
I20260709 13:16:24.420665 12217 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.444860 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:24.445178 12217 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:24.445645 10945 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 3.
I20260709 13:16:24.392488 12291 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:24.293381 12280 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:24.348855 12187 raft_consensus.cc:2804] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:24.401340 10947 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:24.446141 12313 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:24.410533 12328 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:24.366441 12296 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:24.460498 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:24.463495 11510 raft_consensus.cc:2393] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 2: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:24.449576 11511 raft_consensus.cc:1275] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:16:24.388833 12249 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:24.408178 11509 raft_consensus.cc:2393] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 2: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:24.466599 11477 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:24.411087 12214 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:24.474655 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:24.509618 10799 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:24.420469 10941 raft_consensus.cc:1275] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:16:24.479682 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 62828.1us
I20260709 13:16:24.482604 12264 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.482955 12296 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:24.483628 12313 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:24.485365 12297 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:24.485589 12213 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:24.486413 12259 consensus_queue.cc:1048] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:24.462957 12129 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.488013 10868 consensus_queue.cc:237] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:24.489866 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:24.490945 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:24.535624 10800 raft_consensus.cc:2393] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 2: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:24.381369 12243 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:24.544754 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:24.545343 12129 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:24.491263 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:24.547302 10950 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:24.547546 12129 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:24.548130 12168 consensus_queue.cc:1048] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:16:24.548743 10942 raft_consensus.cc:1275] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:16:24.491380 12217 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:24.493110 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:24.549680 12293 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.549808 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:24.550155 10797 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 3.
I20260709 13:16:24.550239 12293 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:24.552498 12293 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:24.516408 12164 consensus_queue.cc:1059] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 13:16:24.553934 12232 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:24.555398 12313 raft_consensus.cc:3055] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:24.555779 12313 raft_consensus.cc:740] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:16:24.556070 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:24.556849 12313 consensus_queue.cc:260] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:24.558058 12313 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:24.441567 11504 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:24.405064 12286 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.499069 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:24.506554 12291 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:24.565289 12291 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:24.565682 12291 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:24.533458 12213 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:24.507718 10793 raft_consensus.cc:1275] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:24.567492 10811 tablet_copy_source_session.cc:215] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:24.370296 12159 consensus_queue.cc:237] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:24.522971 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:24.411749 12126 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:24.477905 12187 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:24.516405 10842 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:16:24.532341 10943 consensus_meta.cc:287] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.543s	user 0.010s	sys 0.002s
I20260709 13:16:24.561065 12235 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:24.513373 12214 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:24.464128 12064 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:24.539278 10873 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2.
I20260709 13:16:24.516669 12190 consensus_queue.cc:1048] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 13:16:24.562747 10916 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:24.522615 12264 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:24.564857 10871 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:24.573559 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:24.596724 10947 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 4.
I20260709 13:16:24.597944 10870 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 3.
I20260709 13:16:24.443893 12195 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:24.576630 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:24.599272 10872 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 2.
I20260709 13:16:24.599182 12195 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:24.601945 12195 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:24.579607 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:24.602772 10874 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2.
I20260709 13:16:24.583706 12187 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:24.607394 12280 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:24.610764 12126 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:16:24.582125 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 65803.4us
I20260709 13:16:24.586141 10915 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:24.586272 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:24.621721 10766 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:24.622063 10869 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 3.
I20260709 13:16:24.584558 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:24.550477 10942 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
I20260709 13:16:24.590138 10875 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 2.
I20260709 13:16:24.596505 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:24.574368 12290 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:24.630538 10869 raft_consensus.cc:1275] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:24.580447 12328 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:24.639261 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:24.574612 12164 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:24.508898 12239 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:24.619452 10946 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:24.577903 12213 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:24.627784 10793 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 2.
I20260709 13:16:24.649173 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:24.588145 12065 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:24.574918 12286 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:24.583484 10943 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:24.587997 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:24.619511 12160 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.629967 12129 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:16:24.661577 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 137228us
I20260709 13:16:24.653733 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:24.652032 10842 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:24.630314 12217 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:24.661818 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:24.681849 11505 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:16:24.660753 12172 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 13:16:24.682123 12160 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:24.761960 11511 raft_consensus.cc:2393] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 2: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:24.690303 12309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 695ff1b5a68e4111917d909909488d39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:24.692967 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:24.762903 10947 raft_consensus.cc:2393] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 3: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:24.688160 10841 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:16:24.764253 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 76224.9us
I20260709 13:16:24.629388 12259 consensus_queue.cc:1048] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 13:16:24.652506 12065 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:24.706223 12064 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:24.695174 12129 raft_consensus.cc:697] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:24.772985 12129 consensus_queue.cc:237] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:24.662614 12306 consensus_queue.cc:1048] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 13:16:24.659278 12158 consensus_queue.cc:1048] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.049s
I20260709 13:16:24.658859 12328 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.683650 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:24.682046 10943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41020 (request call id 1509) took 1100 ms. Trace:
I20260709 13:16:24.650885 12290 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:24.695559 12179 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.655316 12164 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:24.728577 12286 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:24.730177 12264 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:24.732129 12244 consensus_meta.cc:287] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.736s	user 0.010s	sys 0.001s
I20260709 13:16:24.718986 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:24.734453 12242 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:24.717566 12078 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:24.737864 11504 raft_consensus.cc:2410] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:16:24.745900 10871 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
I20260709 13:16:24.625872 12289 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.756269 10950 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
W20260709 13:16:24.758069 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 106109us
I20260709 13:16:24.749042 12324 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.779750 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:24.783885 10943 rpcz_store.cc:276] 0709 13:16:23.581719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:23.582448 (+   729us) service_pool.cc:224] Handling call
0709 13:16:24.660869 (+1078421us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:24.786001 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:24.786520 12179 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:24.769587 12065 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:24.767717 12261 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:24.767716 12332 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:24.802767 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:24.777287 10946 raft_consensus.cc:1275] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:24.876907 12297 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.857148 10941 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:24.804095 10876 raft_consensus.cc:606] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Received request to transfer leadership to TS cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:24.803805 10767 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:24.860842 10949 raft_consensus.cc:2368] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 2. Current term is 3.
I20260709 13:16:24.776450 12328 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:24.779747 12184 consensus_queue.cc:1048] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:16:24.790391 12249 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:24.796392 10874 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
W20260709 13:16:24.878539 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 74824.8us
I20260709 13:16:24.879359 10876 raft_consensus.cc:614] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:24.805891 12171 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.805938 12285 raft_consensus.cc:2749] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:24.797963 12289 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:24.836648 12244 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.805867 12293 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.833155 12291 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:24.888849 12294 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:24.887164 12297 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:24.958567 12287 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.898916 12160 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:24.899123 12217 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:24.899616 12171 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:24.900543 12179 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:24.901299 12164 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:24.832827 12214 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:24.906731 10942 raft_consensus.cc:2393] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 3: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:24.961485 12244 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:24.981287 12319 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:24.982082 12244 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:24.913391 10797 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:24.915622 12327 raft_consensus.cc:993] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26: : Instructing follower 12682eb70bc84c76879e04a8628ee586 to start an election
I20260709 13:16:24.916832 12184 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:24.874187 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:24.983937 12327 raft_consensus.cc:1081] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Signalling peer 12682eb70bc84c76879e04a8628ee586 to start an election
I20260709 13:16:24.985113 12297 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:24.984938 12184 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:24.986548 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:24.987079 12184 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:24.879767 12232 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:24.893162 12239 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:24.872721 12187 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:24.988086 12127 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.988585 12127 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:24.989497 12187 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:24.838029 12324 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:24.990585 10947 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:24.991060 12290 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:24.991215 12127 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:24.991654 12324 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:24.992640 10796 raft_consensus.cc:2368] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 2. Current term is 3.
I20260709 13:16:24.991221 12329 raft_consensus.cc:3055] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:24.993796 12329 raft_consensus.cc:740] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:24.994656 12329 consensus_queue.cc:260] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:24.970504 12330 log.cc:927] Time spent T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:16:24.964280 12293 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:25.016203 12297 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.970220 12294 raft_consensus.cc:697] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:24.975674 12286 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.978749 12179 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.974349 12217 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:16:25.022670 10944 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.031s	sys 0.020s
I20260709 13:16:24.983222 12214 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:24.967456 12227 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.705s	user 0.004s	sys 0.030s
W20260709 13:16:25.026335 10944 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.657s	user 0.032s	sys 0.020s
W20260709 13:16:24.995788 12208 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.717s	user 0.012s	sys 0.034s
I20260709 13:16:24.969892 12287 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:25.016780 12297 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:25.022440 12172 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:16:25.028031 12227 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.766s	user 0.004s	sys 0.030s
I20260709 13:16:25.027062 12291 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:25.028362 12224 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.004s	sys 0.025s
W20260709 13:16:25.032289 12284 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.008s	sys 0.017s
I20260709 13:16:25.028077 12329 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:16:25.063378 12284 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.009s	sys 0.017s
I20260709 13:16:25.063622 12328 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:16:25.056259 12224 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.004s	sys 0.025s
I20260709 13:16:25.064445 12078 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:16:25.032178 12208 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.754s	user 0.012s	sys 0.034s
I20260709 13:16:25.024420 12286 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:25.049618 12242 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:25.027629 12179 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:25.097810 12171 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:25.051846 12172 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:25.100060 12287 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:25.106539 11505 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
I20260709 13:16:25.098764 12217 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:25.093509 12289 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:25.172305 12172 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:25.176756 12286 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:25.092816 12294 consensus_queue.cc:237] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:25.182142 12297 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:25.141574 12235 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:25.185660 12235 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:16:25.189126 12313 consensus_meta.cc:287] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.631s	user 0.002s	sys 0.008s
I20260709 13:16:25.192307 12217 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:25.194355 12179 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:25.203758 12293 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:25.205356 12261 raft_consensus.cc:2749] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:25.226699 12313 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:25.231949 12313 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:25.235329 12313 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:25.238713 12065 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:25.291424 10916 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.665s	user 0.018s	sys 0.021s
W20260709 13:16:25.291944 10916 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.018s	sys 0.021s
I20260709 13:16:25.268720 12313 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.284847 12127 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:25.292491 12159 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.698s	user 0.014s	sys 0.060s
W20260709 13:16:25.292300 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 666811us
W20260709 13:16:25.303761 11507 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.006s	sys 0.028s
W20260709 13:16:25.316125 11507 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.006s	sys 0.028s
W20260709 13:16:25.304085 10915 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.682s	user 0.023s	sys 0.034s
W20260709 13:16:25.316702 10915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.024s	sys 0.035s
W20260709 13:16:25.294595 10870 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.037s	sys 0.035s
W20260709 13:16:25.318400 10870 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.037s	sys 0.035s
W20260709 13:16:25.308454 10945 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.015s	sys 0.032s
W20260709 13:16:25.323248 10945 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.016s	sys 0.032s
I20260709 13:16:25.327198 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:25.293097 12293 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.334563 10949 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 2.
W20260709 13:16:25.337759 11477 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.012s	sys 0.063s
I20260709 13:16:25.337775 10947 raft_consensus.cc:1275] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:16:25.311048 12159 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.717s	user 0.014s	sys 0.061s
W20260709 13:16:25.338476 11477 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.013s	sys 0.063s
W20260709 13:16:25.337147 10948 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.823s	user 0.009s	sys 0.039s
W20260709 13:16:25.348016 10948 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.836s	user 0.010s	sys 0.039s
W20260709 13:16:25.308284 12292 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.864s	user 0.019s	sys 0.039s
I20260709 13:16:25.347261 10916 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:16:25.352944 10842 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.020s	sys 0.009s
W20260709 13:16:25.354658 10842 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.020s	sys 0.009s
I20260709 13:16:25.334266 12127 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:25.357368 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:25.357774 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:25.358148 10792 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 3.
I20260709 13:16:25.358366 10944 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:25.359299 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:25.359498 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:25.360071 10947 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 2.
W20260709 13:16:25.350508 12292 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.910s	user 0.020s	sys 0.040s
I20260709 13:16:25.370433 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:25.308102 12313 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:25.385331 10766 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:25.372444 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:25.389120 10950 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 1.
W20260709 13:16:25.389521 11478 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.020s	sys 0.017s
W20260709 13:16:25.390048 11478 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.020s	sys 0.017s
I20260709 13:16:25.390656 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:25.393844 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:25.394961 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:25.369609 10944 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 4.
I20260709 13:16:25.395735 10797 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:25.395888 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:25.396538 10799 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:25.395131 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:25.398969 10794 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 2.
I20260709 13:16:25.399135 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:25.399813 10800 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 1.
I20260709 13:16:25.400444 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:16:25.361172 11475 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.020s	sys 0.025s
W20260709 13:16:25.374006 12129 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.007s	sys 0.016s
I20260709 13:16:25.401140 10794 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 3.
W20260709 13:16:25.401388 10840 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.748s	user 0.005s	sys 0.021s
I20260709 13:16:25.360102 10943 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:25.335789 12235 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:25.401823 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:25.398343 11477 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:25.394577 10793 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 3.
I20260709 13:16:25.402455 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:25.451920 10873 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
I20260709 13:16:25.405426 10800 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:25.409744 10797 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 3.
I20260709 13:16:25.359393 12127 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:25.412128 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:25.386739 10766 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:16:25.362433 11476 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.004s	sys 0.018s
I20260709 13:16:25.338104 12293 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:25.423476 10840 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.771s	user 0.005s	sys 0.021s
I20260709 13:16:25.423159 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:25.450481 10799 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 4.
I20260709 13:16:25.372570 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:25.452340 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:25.411566 12289 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:25.453373 12331 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.009s	sys 0.010s
W20260709 13:16:25.704074 12331 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.906s	user 0.009s	sys 0.011s
W20260709 13:16:25.704165 12309 consensus_meta.cc:287] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.544s	user 0.011s	sys 0.000s
I20260709 13:16:25.705674 12309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:56416 (request call id 348) took 2155 ms. Trace:
I20260709 13:16:25.705943 12313 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:25.709462 10916 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:16:25.666090 10841 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.807s	user 0.036s	sys 0.024s
W20260709 13:16:25.711059 10841 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.852s	user 0.037s	sys 0.026s
I20260709 13:16:25.451104 10871 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 2.
I20260709 13:16:25.712428 10875 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:25.454147 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:16:25.419379 11475 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.020s	sys 0.026s
I20260709 13:16:25.731415 10916 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:25.423429 12213 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:25.717166 10947 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 4.
W20260709 13:16:25.720021 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 321745us
I20260709 13:16:25.762713 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:16:25.427481 12129 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.007s	sys 0.018s
I20260709 13:16:25.716708 10842 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:16:25.765745 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 49198.8us
I20260709 13:16:25.424117 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:25.741888 10916 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:25.452069 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:25.515753 12065 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:25.494638 12285 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.763649 11510 raft_consensus.cc:2393] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 2: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:25.423745 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:25.795780 11504 raft_consensus.cc:2368] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 2. Current term is 3.
I20260709 13:16:25.740751 12347 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:25.764849 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:25.796746 12347 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:25.797196 12347 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:25.719630 10869 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
I20260709 13:16:25.798771 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:25.799677 10869 raft_consensus.cc:2393] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 2: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:25.800516 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:25.801240 12292 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.801908 12292 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:25.803308 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:25.804358 12358 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:25.804512 10947 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 3.
I20260709 13:16:25.515857 12126 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.625804 12306 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.804936 12358 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:25.805682 12126 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:25.804612 12292 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:25.807082 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:25.419785 12287 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:25.768491 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:25.767159 12168 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:25.725740 12294 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.021s	sys 0.031s
W20260709 13:16:25.525638 12341 log.cc:927] Time spent T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.182s	user 0.000s	sys 0.001s
I20260709 13:16:25.425740 10946 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 3.
I20260709 13:16:25.625608 12286 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:25.615278 12339 log.cc:927] Time spent T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 13:16:25.720527 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:25.915894 10872 raft_consensus.cc:2410] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:16:25.725322 10941 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:25.765772 12343 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:25.781106 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:25.770498 12213 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:25.929536 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:25.930359 11503 raft_consensus.cc:2393] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 2: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:25.781236 12345 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.781885 11512 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8"
dest_uuid: "12682eb70bc84c76879e04a8628ee586"
 from {username='slave'} at 127.0.0.1:56410
I20260709 13:16:25.781141 10842 leader_election.cc:400] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 election: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 2. Current term is 3.
W20260709 13:16:25.742139 11476 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.971s	user 0.015s	sys 0.023s
I20260709 13:16:25.787369 10950 raft_consensus.cc:2393] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 3: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:25.755146 12179 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:25.790630 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:25.792151 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:25.800379 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:25.805719 12306 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:25.770131 10916 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:25.624898 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:25.808034 10944 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
I20260709 13:16:25.801831 10869 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
I20260709 13:16:25.814404 10841 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:16:25.770828 10839 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.931s	user 0.031s	sys 0.038s
I20260709 13:16:25.815903 10875 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
I20260709 13:16:25.824424 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:25.781944 12242 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:25.728502 12289 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:26.088229 12289 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:26.094105 12129 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:26.094985 12129 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:26.097870 12129 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:26.099860 10946 raft_consensus.cc:1275] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:26.103421 10946 raft_consensus.cc:1275] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:25.779089 12244 raft_consensus.cc:2804] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:25.800493 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:25.920478 11477 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:25.907395 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:25.926901 12286 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:25.927448 12343 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:25.927870 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:25.930258 12213 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:25.714890 12297 raft_consensus.cc:2764] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:16:25.706148 12309 rpcz_store.cc:276] 0709 13:16:23.553418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:23.557596 (+  4178us) service_pool.cc:224] Handling call
0709 13:16:23.887712 (+330116us) ts_tablet_manager.cc:934] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586: Initiating tablet copy from peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685)
0709 13:16:24.684251 (+796539us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:16:25.110653 (+426402us) tablet_metadata.cc:665] Metadata flushed
0709 13:16:25.110774 (+   121us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:16:25.704797 (+594023us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:16:25.705185 (+   388us) tablet_replica.cc:206] Creating consensus instance
0709 13:16:25.705515 (+   330us) ts_tablet_manager.cc:989] Replica registered.
0709 13:16:25.705585 (+    70us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":11957,"tablet-copy.queue_time_us":276329,"thread_start_us":307688,"threads_started":2}
W20260709 13:16:25.860087 12294 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.679s	user 0.022s	sys 0.031s
I20260709 13:16:26.000948 11511 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 3.
W20260709 13:16:25.959368 10876 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.023s	sys 0.066s
W20260709 13:16:26.147500 10876 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.735s	user 0.023s	sys 0.068s
I20260709 13:16:26.147783 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:26.003746 10842 leader_election.cc:403] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:16:26.146544 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:25.812732 12313 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:25.832799 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:26.075345 12242 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:26.060307 12345 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:26.075080 10871 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 3.
W20260709 13:16:26.010908 10945 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.024s	sys 0.023s
I20260709 13:16:26.066731 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:16:26.076133 10839 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.268s	user 0.033s	sys 0.040s
I20260709 13:16:26.077020 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:26.191264 10869 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:26.191737 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:16:26.191636 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 424992us
I20260709 13:16:26.193055 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:26.193307 10873 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 2.
I20260709 13:16:26.193605 10871 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:16:26.077247 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:26.194171 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:26.194469 10875 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:16:26.194762 10873 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:26.075779 10942 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
I20260709 13:16:26.078038 12306 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:26.078853 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:26.203873 10944 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2.
I20260709 13:16:26.078922 12358 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:26.081514 11512 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:26.206604 11512 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:26.206634 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:26.207589 10799 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 3.
I20260709 13:16:26.083806 12356 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:26.221133 10869 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 4.
I20260709 13:16:26.222574 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:26.223982 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:16:26.114804 10949 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.000s	sys 0.084s
W20260709 13:16:26.225071 10949 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.709s	user 0.000s	sys 0.084s
I20260709 13:16:26.144151 11508 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 3.
I20260709 13:16:26.226187 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:26.226687 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:26.227663 11508 raft_consensus.cc:2410] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:16:26.229568 12235 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:25.916301 12172 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:26.230856 12172 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:26.232636 11512 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:26.233033 12172 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:26.233630 12172 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:26.234118 12172 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:25.861173 12287 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:26.234618 11512 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:26.234923 10941 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 4.
I20260709 13:16:26.138655 10872 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2.
I20260709 13:16:26.236281 12287 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:26.236590 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:26.237275 10872 raft_consensus.cc:2393] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 2: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:26.237959 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:26.238467 10872 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 3.
I20260709 13:16:26.239558 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:26.240528 10872 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 3.
I20260709 13:16:26.241323 10875 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 4.
I20260709 13:16:26.139283 12342 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:25.815601 12285 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:26.123636 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 203225us
I20260709 13:16:26.148543 11511 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
I20260709 13:16:26.149258 11510 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:25.899385 12126 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:16:26.194907 10945 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.824s	user 0.025s	sys 0.023s
I20260709 13:16:26.195875 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:26.196244 10800 raft_consensus.cc:1275] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:26.335285 10794 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 3.
I20260709 13:16:26.337248 10794 raft_consensus.cc:1275] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:26.197870 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:26.016539 12350 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:26.144986 12292 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:26.205931 10801 raft_consensus.cc:1275] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:26.208200 10871 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 3.
I20260709 13:16:26.350140 10799 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8cab289810294719ac9686e40290a6ab"
dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
mode: GRACEFUL
new_leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:16:26.351007 10799 raft_consensus.cc:606] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Received request to transfer leadership to TS 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:26.213603 11478 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:26.199676 12313 raft_consensus.cc:697] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:26.219165 10873 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 4.
I20260709 13:16:25.837520 12065 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.241401 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:25.937572 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:26.142750 12244 raft_consensus.cc:697] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:26.242884 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:26.243986 12217 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:26.365473 10875 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2.
I20260709 13:16:25.900385 12168 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:26.206492 10874 raft_consensus.cc:2393] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 2: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:26.150058 12213 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:26.138438 12343 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:26.195940 12351 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.119196 12179 raft_consensus.cc:697] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 4 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:26.197139 10797 raft_consensus.cc:1240] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:26.128218 11506 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 2.
I20260709 13:16:26.338959 10793 raft_consensus.cc:2393] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 3: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:26.204753 12329 raft_consensus.cc:2749] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 2. Current term is 3.
I20260709 13:16:26.225155 10869 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:16:26.343533 11477 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:26.141624 12352 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:26.344565 12235 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:26.355041 12293 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:26.205597 12309 tablet_copy_client.cc:806] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:26.201629 12242 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:26.358855 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 145336us
I20260709 13:16:26.243422 12287 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.219884 12356 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:26.340253 11509 raft_consensus.cc:2368] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 3. Current term is 4.
I20260709 13:16:26.294396 12261 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.349578 12292 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:26.349452 12126 raft_consensus.cc:2804] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:26.381531 10950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41014 (request call id 1238) took 1007 ms. Trace:
I20260709 13:16:26.366497 12353 raft_consensus.cc:2804] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:26.403244 10868 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
I20260709 13:16:26.352706 12350 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:26.365391 12065 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:26.404175 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 60752.3us
I20260709 13:16:26.387171 12345 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:26.429476 10950 rpcz_store.cc:276] 0709 13:16:25.373839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:25.390213 (+ 16374us) service_pool.cc:224] Handling call
0709 13:16:26.381332 (+991119us) spinlock_profiling.cc:243] Waited 395 ms on lock 0x7b5400399c54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:26.381469 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":909119360}
W20260709 13:16:26.431702 11507 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.013s	user 0.061s	sys 0.080s
I20260709 13:16:26.433730 10870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1177) took 1810 ms. Trace:
I20260709 13:16:26.353580 12342 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:26.391139 12213 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:26.505743 10766 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.745s	user 0.041s	sys 0.122s
W20260709 13:16:26.439368 11507 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.020s	user 0.061s	sys 0.081s
I20260709 13:16:26.413359 12349 raft_consensus.cc:2804] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:26.418071 12356 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:16:26.515461 10766 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.755s	user 0.041s	sys 0.127s
I20260709 13:16:26.413451 12293 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:16:26.525502 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 1.13878e+06us
I20260709 13:16:26.394469 12351 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:26.428417 12292 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:16:26.418361 10796 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.060s	user 0.045s	sys 0.095s
I20260709 13:16:26.427625 12294 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.534364 10766 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:26.526160 12349 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:26.576587 12347 consensus_meta.cc:287] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.779s	user 0.009s	sys 0.003s
I20260709 13:16:26.576763 12297 consensus_queue.cc:1048] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:26.577344 12347 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:26.582538 11510 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
I20260709 13:16:26.408489 12352 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:26.428190 12126 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:26.435262 12353 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:26.497902 12306 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.409862 12235 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:26.430691 12261 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:26.437294 12309 tablet_copy_client.cc:670] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:26.517951 12286 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:26.426357 12287 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:26.593230 12126 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:26.438798 12313 consensus_queue.cc:237] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:26.584525 12294 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:26.439797 10870 rpcz_store.cc:276] 0709 13:16:24.623569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:24.624370 (+   801us) service_pool.cc:224] Handling call
0709 13:16:24.624732 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:25.318934 (+694202us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:25.318967 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:25.338462 (+ 19495us) log.cc:844] Serialized 26 byte log entry
0709 13:16:25.770822 (+432360us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:25.770872 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:26.014226 (+243354us) spinlock_profiling.cc:243] Waited 25.3 ms on lock 0x7b54003165d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:26.014308 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:26.417395 (+403087us) raft_consensus.cc:1685] finished
0709 13:16:26.417414 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:26.433635 (+ 16221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2094,"raft.queue_time_us":9439,"raft.run_cpu_time_us":1070,"raft.run_wall_time_us":29448,"spinlock_wait_cycles":58236672,"thread_start_us":1051137,"threads_started":2,"wal-append.queue_time_us":867823}
I20260709 13:16:26.554778 10945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 1531) took 1861 ms. Trace:
W20260709 13:16:26.615152 10841 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.013s	sys 0.006s
W20260709 13:16:26.615540 10841 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.013s	sys 0.006s
I20260709 13:16:26.556367 12350 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:26.523767 12244 consensus_queue.cc:237] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:26.638729 12217 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:26.639737 12217 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:26.523325 12179 consensus_queue.cc:237] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:26.639118 11507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56410 (request call id 328) took 1953 ms. Trace:
I20260709 13:16:26.643306 11507 rpcz_store.cc:276] 0709 13:16:24.686157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:24.686366 (+   209us) service_pool.cc:224] Handling call
0709 13:16:24.690973 (+  4607us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:25.335822 (+644849us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:25.335851 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:25.343264 (+  7413us) log.cc:844] Serialized 26 byte log entry
0709 13:16:26.515690 (+1172426us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:26.515757 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:26.515866 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:26.638864 (+122998us) raft_consensus.cc:1685] finished
0709 13:16:26.638888 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:26.639054 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2778,"raft.queue_time_us":280497,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":3662,"thread_start_us":1719446,"threads_started":2,"wal-append.queue_time_us":1249899}
I20260709 13:16:26.646394 12235 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:26.540360 12129 consensus_queue.cc:1048] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 13:16:26.557713 12172 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:26.558156 12285 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:26.600929 12306 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:26.559666 10795 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:26.560101 12065 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:26.561043 12342 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:26.562206 12213 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:26.540686 12293 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:26.562230 12168 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:26.568470 10949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 1229) took 1218 ms. Trace:
I20260709 13:16:26.572240 10876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1181) took 1200 ms. Trace:
I20260709 13:16:26.572413 12289 raft_consensus.cc:2804] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:16:26.537406 10767 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.806s	user 0.007s	sys 0.007s
W20260709 13:16:26.541391 10796 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.183s	user 0.046s	sys 0.095s
W20260709 13:16:26.588364 10948 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.237s	user 0.064s	sys 0.148s
W20260709 13:16:26.676352 10767 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.946s	user 0.007s	sys 0.007s
W20260709 13:16:26.676625 10948 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.326s	user 0.064s	sys 0.148s
W20260709 13:16:26.588769 12357 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.792s	user 0.005s	sys 0.009s
I20260709 13:16:26.576591 12349 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:26.598588 12353 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:16:26.619204 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 804852us
W20260709 13:16:26.619954 10915 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.687s	user 0.000s	sys 0.013s
W20260709 13:16:26.683377 10915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.750s	user 0.000s	sys 0.014s
I20260709 13:16:26.683689 10948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 1215) took 2323 ms. Trace:
I20260709 13:16:26.683892 10841 leader_election.cc:400] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Vote denied by peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553) with higher term. Message: Invalid argument: T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 2. Current term is 3.
I20260709 13:16:26.684204 10841 leader_election.cc:403] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:16:26.684092 10948 rpcz_store.cc:276] 0709 13:16:24.354263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:24.355386 (+  1123us) service_pool.cc:224] Handling call
0709 13:16:24.476539 (+121153us) spinlock_profiling.cc:243] Waited 1.33 ms on lock 0x7b5800112598. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd90a22 00007f9c86ca9c09 00007f9c85e03ee0 00007f9c85ecd0ab 00007f9c85ecd131 00007f9c85e8efc0 00007f9c85e79929 00007f9c85e793d2 00007f9c7f3f0bd4
0709 13:16:24.476769 (+   230us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:24.477036 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:16:25.350660 (+873624us) log.cc:844] Serialized 14 byte log entry
0709 13:16:26.677253 (+1326593us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:26.677339 (+    86us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:26.677403 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:26.677539 (+   136us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:26.677703 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":883461,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":12251,"raft.queue_time_us":834,"spinlock_wait_cycles":3053440,"thread_start_us":2162091,"threads_started":2}
W20260709 13:16:26.620384 10913 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.687s	user 0.000s	sys 0.013s
W20260709 13:16:26.687053 10913 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.754s	user 0.000s	sys 0.013s
I20260709 13:16:26.629328 10869 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 4.
I20260709 13:16:26.615402 10945 rpcz_store.cc:276] 0709 13:16:24.693194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:24.693464 (+   270us) service_pool.cc:224] Handling call
0709 13:16:24.693798 (+   334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:25.323777 (+629979us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:25.323823 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:25.341107 (+ 17284us) log.cc:844] Serialized 26 byte log entry
0709 13:16:26.335280 (+994173us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:26.335323 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:26.335391 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:26.554511 (+219120us) raft_consensus.cc:1685] finished
0709 13:16:26.554534 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:26.554713 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9963,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":123,"thread_start_us":1593135,"threads_started":2,"wal-append.queue_time_us":1171319}
W20260709 13:16:26.557125 10943 consensus_meta.cc:287] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 1.060s	user 0.012s	sys 0.000s
I20260709 13:16:26.670493 10949 rpcz_store.cc:276] 0709 13:16:25.350690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:25.352411 (+  1721us) service_pool.cc:224] Handling call
0709 13:16:25.352689 (+   278us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:25.353107 (+   418us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:25.353139 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:25.372826 (+ 19687us) log.cc:844] Serialized 26 byte log entry
0709 13:16:26.225635 (+852809us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:26.225726 (+    91us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:26.225799 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:26.568229 (+342430us) raft_consensus.cc:1685] finished
0709 13:16:26.568252 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:26.568406 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":178,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":7959,"thread_start_us":709567,"threads_started":1,"wal-append.queue_time_us":1139323}
I20260709 13:16:26.688967 10841 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:26.694195 10841 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:16:26.689723 10916 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.029s	sys 0.013s
I20260709 13:16:26.672170 10876 rpcz_store.cc:276] 0709 13:16:25.371995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:25.410893 (+ 38898us) service_pool.cc:224] Handling call
0709 13:16:25.411247 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:25.411615 (+   368us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:25.411650 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:25.412076 (+   426us) log.cc:844] Serialized 26 byte log entry
0709 13:16:26.148236 (+736160us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:16:26.148292 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:26.148382 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:26.572031 (+423649us) raft_consensus.cc:1685] finished
0709 13:16:26.572052 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:26.572178 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29514,"raft.queue_time_us":9931,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":27170,"spinlock_wait_cycles":55808,"thread_start_us":735835,"threads_started":1,"wal-append.queue_time_us":1130324}
W20260709 13:16:26.696900 10916 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.030s	sys 0.013s
W20260709 13:16:26.697396 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 927285us
I20260709 13:16:26.699997 10916 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:26.703677 12354 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:26.704200 12354 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:26.704635 12065 consensus_queue.cc:1048] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:16:26.679438 12357 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.883s	user 0.005s	sys 0.009s
I20260709 13:16:26.708271 10916 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:26.680866 12289 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:26.719142 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:26.719801 10868 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 2.
I20260709 13:16:26.718461 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:26.714097 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:26.716936 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:26.722415 10870 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
I20260709 13:16:26.701983 12168 raft_consensus.cc:2749] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553) with higher term. Message: Invalid argument: T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 2. Current term is 3.
W20260709 13:16:26.728178 10914 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.010s	sys 0.035s
W20260709 13:16:26.729482 10914 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.010s	sys 0.035s
W20260709 13:16:26.724870 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 317825us
I20260709 13:16:26.711294 10916 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:16:26.723929 10839 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.011s	sys 0.021s
I20260709 13:16:26.702196 12285 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:26.726606 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:26.704705 12354 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:26.692955 12172 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:26.731472 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:16:26.732208 10839 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.011s	sys 0.022s
I20260709 13:16:26.713948 12342 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:26.722621 12289 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:26.706488 12358 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:26.708400 12129 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.738149 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:26.759974 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:26.760593 10871 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2.
I20260709 13:16:26.760696 10948 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:26.701431 10943 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 4.
I20260709 13:16:26.747196 10797 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 3.
I20260709 13:16:26.711290 12368 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:26.718674 12213 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:26.731155 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:26.770058 10793 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:26.752002 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:26.719484 12367 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:26.733701 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:26.772166 10842 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:16:26.754236 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 317423us
I20260709 13:16:26.706934 12356 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:26.765563 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:26.780215 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:26.760654 10873 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 3.
I20260709 13:16:26.740036 10796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1195) took 1387 ms. Trace:
I20260709 13:16:26.773155 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:16:26.775516 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 205140us
I20260709 13:16:26.747689 12351 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:26.749396 12342 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:26.784940 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:26.783290 10766 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:26.785770 10800 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 2.
I20260709 13:16:26.785084 10799 raft_consensus.cc:1240] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:26.786707 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:26.787448 10800 raft_consensus.cc:2393] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 2: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:26.787487 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:26.788507 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:26.789685 11477 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:26.790501 11477 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:26.790555 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:26.790501 10916 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:26.794162 11510 raft_consensus.cc:1275] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:26.795929 11478 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:16:26.784475 12261 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:26.797894 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:26.798604 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:26.799391 10941 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 3.
I20260709 13:16:26.800936 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:26.801654 10873 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 2.
I20260709 13:16:26.784693 12347 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:26.803980 12294 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:26.777870 10943 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36714 (request call id 92) took 1412 ms (client timeout 1530 ms). Trace:
I20260709 13:16:26.804796 12294 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
W20260709 13:16:26.804920 10943 rpcz_store.cc:269] 0709 13:16:25.356883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:25.359238 (+  2355us) service_pool.cc:224] Handling call
0709 13:16:26.768760 (+1409522us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:26.805424 12294 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:26.754514 12358 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:26.758410 12329 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:26.804060 12356 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:26.756615 12224 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.801345 12343 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:26.808362 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:26.746315 12285 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:26.759590 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:26.810446 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:26.807076 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:26.831769 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:26.837054 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:26.837930 10801 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 3.
I20260709 13:16:26.838678 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:26.839234 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:26.812399 10916 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:26.812563 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:26.759335 12371 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:26.807397 12342 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:26.769354 12065 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:26.829234 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:26.804000 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:26.834275 10948 raft_consensus.cc:1275] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:26.841089 10875 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 2.
I20260709 13:16:26.752198 12172 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:26.769644 12352 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:26.798645 10946 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:26.844630 10871 raft_consensus.cc:2393] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 3: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:26.846271 12224 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:26.850836 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 78677.3us
I20260709 13:16:26.810354 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:26.832989 10793 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 3.
W20260709 13:16:26.858017 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 140653us
I20260709 13:16:26.855082 10797 raft_consensus.cc:1275] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:26.853983 10949 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2.
I20260709 13:16:26.812413 12358 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:26.867250 12370 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:26.875989 12370 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:26.878439 12345 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:26.808599 12367 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:26.879135 12345 raft_consensus.cc:697] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 4 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
W20260709 13:16:26.879889 11505 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56410 (request call id 329) took 1516 ms (client timeout 1846 ms). Trace:
W20260709 13:16:26.880365 11505 rpcz_store.cc:269] 0709 13:16:25.363066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:25.371068 (+  8002us) service_pool.cc:224] Handling call
0709 13:16:26.879635 (+1508567us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54005d66d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:26.879800 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2378251520}
I20260709 13:16:26.881366 12285 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:26.881973 12285 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:26.809620 12159 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:26.882928 12159 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:26.883490 12292 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:26.885143 12159 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:26.885509 12292 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:26.831889 12329 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:26.868248 12356 raft_consensus.cc:2804] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:26.813978 12286 raft_consensus.cc:2749] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:26.886878 12356 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:26.887349 12356 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:26.831358 12179 raft_consensus.cc:2804] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:26.887950 12179 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:26.888186 12329 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:26.888396 12179 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:26.868938 12375 consensus_queue.cc:1048] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 13:16:26.874891 12378 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:26.841773 10867 raft_consensus.cc:1240] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:16:26.890590 12378 raft_consensus.cc:697] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 4 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:26.842389 10798 raft_consensus.cc:1275] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:26.891167 10870 raft_consensus.cc:1240] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:26.891582 12378 consensus_queue.cc:237] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:26.845260 12347 raft_consensus.cc:2749] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:26.778314 12368 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:26.860574 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:16:26.871280 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 48998.8us
I20260709 13:16:26.766570 12372 consensus_queue.cc:1048] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.102s
I20260709 13:16:26.767562 12129 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:26.864527 12287 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:26.814244 12213 raft_consensus.cc:697] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 3 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:26.921885 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:26.887379 11511 raft_consensus.cc:1240] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:26.938401 12294 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:26.938527 10943 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 1.
I20260709 13:16:26.925559 12217 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:26.894810 11478 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
W20260709 13:16:26.951411 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 56695.2us
I20260709 13:16:26.951604 12235 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:26.952239 12235 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:26.952751 12235 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:26.953544 11478 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:16:26.895147 10841 leader_election.cc:400] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 election: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 2. Current term is 3.
W20260709 13:16:26.955374 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 60367.5us
W20260709 13:16:26.956437 10841 consensus_peers.cc:597] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Couldn't send request to peer 12682eb70bc84c76879e04a8628ee586. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2b82f18992434ed5ba58b805107060d4. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:26.867401 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 84237.9us
I20260709 13:16:26.958209 10766 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:26.882475 12285 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:26.961227 12285 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:26.868662 10942 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 1.
I20260709 13:16:26.916357 10800 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 1.
I20260709 13:16:26.914897 10945 raft_consensus.cc:1275] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:26.880302 12345 consensus_queue.cc:237] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:26.970350 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 107890us
W20260709 13:16:26.931411 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 58507.5us
I20260709 13:16:26.860224 10795 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
W20260709 13:16:26.926512 12242 consensus_meta.cc:287] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.505s	user 0.003s	sys 0.007s
I20260709 13:16:26.814863 12350 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.875389 12381 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:26.987206 12381 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:26.894896 12306 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:26.975673 10876 raft_consensus.cc:3055] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:26.849094 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:26.863924 12065 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:26.912894 12368 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:26.964859 12287 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:16:27.007628 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 50198.6us
I20260709 13:16:26.972750 10767 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:26.980057 12350 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:26.981376 12217 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:26.900756 10874 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 3.
I20260709 13:16:26.911360 10941 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2.
I20260709 13:16:26.920408 12172 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:26.878430 12370 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:26.964701 12235 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:26.996445 10876 raft_consensus.cc:740] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:26.918514 12224 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:26.938303 12343 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:26.909461 12313 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:27.001988 11508 raft_consensus.cc:2393] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 2: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:26.938474 12213 consensus_queue.cc:237] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:26.995615 12242 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:27.002557 12347 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:27.003943 10947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 1535) took 1291 ms. Trace:
I20260709 13:16:27.085050 10947 rpcz_store.cc:276] 0709 13:16:25.712862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:25.860648 (+147786us) service_pool.cc:224] Handling call
0709 13:16:25.860986 (+   338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:26.368827 (+507841us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:26.368866 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:26.376994 (+  8128us) log.cc:844] Serialized 26 byte log entry
0709 13:16:26.736155 (+359161us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:26.736212 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:26.746692 (+ 10480us) spinlock_profiling.cc:243] Waited 506 ms on lock 0x7b540018ecd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:26.746751 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:27.003537 (+256786us) raft_consensus.cc:1685] finished
0709 13:16:27.003564 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:27.003861 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":200900,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":125,"spinlock_wait_cycles":1163238272,"thread_start_us":317490,"threads_started":1,"wal-append.queue_time_us":596628}
I20260709 13:16:27.004529 10869 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:53748
W20260709 13:16:27.086812 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 122443us
I20260709 13:16:27.008769 10841 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:16:27.013371 10801 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 2.
W20260709 13:16:27.030900 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 58262.5us
I20260709 13:16:27.064523 10872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49290 (request call id 124) took 1113 ms. Trace:
I20260709 13:16:27.088831 10766 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:26.997694 12354 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:27.065750 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 217896us
I20260709 13:16:26.992025 12126 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:27.071835 10946 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 4.
I20260709 13:16:26.916114 12179 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:26.999191 12329 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:27.073582 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 97266.2us
I20260709 13:16:27.064299 10876 consensus_queue.cc:260] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:27.011138 12287 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:27.143513 12287 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:27.000198 12285 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:27.087553 10869 raft_consensus.cc:606] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Received request to transfer leadership to TS cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:27.161518 10869 raft_consensus.cc:614] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:26.935552 12293 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:27.090651 12347 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:27.065312 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 274871us
I20260709 13:16:27.088986 10872 rpcz_store.cc:276] 0709 13:16:25.951260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:26.241154 (+289894us) service_pool.cc:224] Handling call
0709 13:16:27.064326 (+823172us) spinlock_profiling.cc:243] Waited 491 ms on lock 0x7b540009bad4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:27.064452 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1128457472}
W20260709 13:16:27.094192 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 85548.8us
I20260709 13:16:27.078156 12313 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:26.791651 10796 rpcz_store.cc:276] 0709 13:16:25.352680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:25.354091 (+  1411us) service_pool.cc:224] Handling call
0709 13:16:25.357517 (+  3426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:25.357780 (+   263us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:25.357813 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:25.358333 (+   520us) log.cc:844] Serialized 26 byte log entry
0709 13:16:26.693569 (+1335236us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:26.707741 (+ 14172us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:26.712286 (+  4545us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:26.719234 (+  6948us) raft_consensus.cc:1685] finished
0709 13:16:26.722847 (+  3613us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:26.737241 (+ 14394us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11520,"thread_start_us":1320736,"threads_started":1,"wal-append.queue_time_us":1247804}
I20260709 13:16:27.076064 12387 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:27.080077 12168 consensus_queue.cc:1048] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 13:16:27.099834 12389 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:27.173157 12313 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:27.176484 12389 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:27.177021 12389 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:27.173403 12358 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:27.177929 12387 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:27.080284 12388 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.187044 12388 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:27.139235 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 48334.4us
I20260709 13:16:27.083792 12379 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:27.141171 10842 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:27.142330 10876 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:27.142725 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 53995.8us
I20260709 13:16:27.154004 12344 consensus_queue.cc:1048] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:27.194731 12388 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:27.169186 10841 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:16:27.197216 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 242409us
I20260709 13:16:27.166832 12285 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:27.170671 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 70909.2us
I20260709 13:16:27.184068 12358 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:27.201354 12395 raft_consensus.cc:993] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87: : Instructing follower 468a8fd46e824f62b9c2c0d3532a6d26 to start an election
I20260709 13:16:27.184577 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:27.186062 12354 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:27.186298 12126 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:27.171562 12261 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:27.186389 12387 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:27.163046 12294 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:27.197933 10841 leader_election.cc:400] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 3. Current term is 4.
W20260709 13:16:27.170289 11477 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.116.131:45553 timed out after 1.530s (SENT) [suppressed 9 similar messages]
I20260709 13:16:27.218134 10841 leader_election.cc:403] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:16:27.163756 12371 raft_consensus.cc:697] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 4 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:27.197485 12129 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:27.217267 10767 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:16:27.199196 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 58099.1us
I20260709 13:16:27.201398 10916 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:27.160907 12329 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:27.208509 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:27.213658 12395 raft_consensus.cc:1081] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Signalling peer 468a8fd46e824f62b9c2c0d3532a6d26 to start an election
I20260709 13:16:27.215485 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:27.202590 12388 raft_consensus.cc:2804] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:27.221578 10916 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:27.204514 12168 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:27.220299 10842 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:27.224114 11511 raft_consensus.cc:2393] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 3: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:27.220315 10767 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:27.224885 10841 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:27.171589 12349 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:27.203279 12396 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:27.231313 10767 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:27.171809 12353 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:27.233287 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:27.234083 10944 raft_consensus.cc:2393] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 4: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:27.229744 10842 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:27.218019 12294 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:27.236100 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:27.237141 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:27.239950 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:27.240785 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:27.240911 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:27.242473 10947 raft_consensus.cc:2393] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 3: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:27.243964 10950 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:27.204864 12374 consensus_queue.cc:1048] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:16:27.248041 12353 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:27.228595 12387 raft_consensus.cc:2749] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 3. Current term is 4.
I20260709 13:16:27.248134 12349 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:27.243476 10949 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:27.242274 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:27.234797 12358 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:27.241802 10916 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:27.226997 12126 raft_consensus.cc:2804] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:27.251787 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:27.226145 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:27.226542 12385 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:27.227550 12381 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:27.226797 10796 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740"
dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
mode: GRACEFUL
new_leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:16:27.228587 12388 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:27.243747 10948 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 3.
I20260709 13:16:27.234225 12396 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:27.262351 10916 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:27.245464 12168 raft_consensus.cc:697] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:27.249704 12386 raft_consensus.cc:2804] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:27.296731 12386 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:27.296849 12396 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:27.297159 12386 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:27.241407 12400 raft_consensus.cc:2749] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:27.263453 12126 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:27.263634 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1183) took 1866 ms. Trace:
I20260709 13:16:27.241321 12065 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:16:27.263974 12397 log.cc:927] Time spent T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:16:27.241120 12344 raft_consensus.cc:2749] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:27.236140 12350 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:27.263944 12347 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:27.266921 12285 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:27.267076 12297 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:27.267287 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:27.267513 12371 consensus_queue.cc:237] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:27.268803 12289 consensus_meta.cc:287] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.507s	user 0.010s	sys 0.002s
I20260709 13:16:27.268445 12385 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:27.227752 12356 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:27.270684 11504 raft_consensus.cc:2393] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 4: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:27.271935 12367 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:27.272341 12329 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:27.261037 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:27.274554 12389 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:27.257331 12306 raft_consensus.cc:2804] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:27.274616 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:27.219355 12261 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:27.288219 12342 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:27.281528 10698 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42506 (request call id 151) took 3324 ms. Trace:
I20260709 13:16:27.283192 12392 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:27.285494 12242 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:27.286573 12179 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:27.245039 12294 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:27.295751 10950 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
W20260709 13:16:27.302255 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 109675us
I20260709 13:16:27.260000 12353 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:27.261699 12358 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:27.263933 12374 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:27.304916 10916 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:27.304040 12343 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:27.276060 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:27.297173 12168 consensus_queue.cc:237] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:27.365847 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:27.642206 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:27.643021 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:27.282076 12404 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:16:27.641799 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 336935us
W20260709 13:16:27.658514 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 413978us
I20260709 13:16:27.659822 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:27.660816 10948 raft_consensus.cc:2410] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:16:27.661144 10841 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:27.663502 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:27.664290 10871 raft_consensus.cc:2393] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 2: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:27.665850 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:27.301724 12292 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:27.298028 12388 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:27.585894 10876 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2.
I20260709 13:16:27.585409 10949 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 3.
W20260709 13:16:27.454653 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 284348us
I20260709 13:16:27.651983 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:27.654534 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:27.313694 10874 raft_consensus.cc:1275] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:16:27.302156 10796 raft_consensus.cc:606] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Received request to transfer leadership to TS 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:27.301854 12126 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:27.667315 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:27.667873 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:16:27.494508 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 261918us
I20260709 13:16:27.668754 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:27.669431 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:27.671383 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:27.671927 11507 raft_consensus.cc:1240] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:27.655622 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:27.309937 12213 consensus_queue.cc:1059] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.435s
I20260709 13:16:27.762161 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:27.873924 10799 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 2.
I20260709 13:16:27.874122 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:27.673237 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:16:27.804715 10766 connection.cc:256] RPC call timeout handler was delayed by -1.7870650719999048s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1678s for a response
I20260709 13:16:27.725778 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:27.760448 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:27.873277 10797 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:27.877650 10947 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 1.
I20260709 13:16:27.760228 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:27.750886 10870 raft_consensus.cc:2393] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 4: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:27.879345 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:27.758273 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:27.881702 10801 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 1.
I20260709 13:16:27.329205 12379 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:27.877761 10800 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 1.
I20260709 13:16:27.877697 10868 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 1.
I20260709 13:16:27.881913 11512 raft_consensus.cc:2393] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 3: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:27.314507 12399 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:27.883562 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:27.931919 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:27.885056 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:27.925573 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:27.935061 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:27.935956 10950 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
I20260709 13:16:27.938099 10950 raft_consensus.cc:1240] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260709 13:16:27.877319 10766 connection.cc:256] RPC call timeout handler was delayed by -1.6181646059999366s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17120000000000002s for a response
I20260709 13:16:27.426326 12400 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:27.916939 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:27.877254 10871 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 3.
I20260709 13:16:27.804598 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:27.620764 12293 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:16:27.753520 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 87146.2us
I20260709 13:16:27.959465 10841 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:27.876057 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:27.877120 10876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49290 (request call id 128) took 1144 ms. Trace:
I20260709 13:16:27.311933 12297 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:27.618427 12394 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:27.530220 12235 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:27.645574 12374 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:27.961045 10876 rpcz_store.cc:276] 0709 13:16:26.733182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:26.733355 (+   173us) service_pool.cc:224] Handling call
0709 13:16:26.975528 (+242173us) spinlock_profiling.cc:243] Waited 86.5 ms on lock 0x7b540030eba0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80e2cdc0 00007f9c80eb8e18 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:27.877059 (+901531us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":198935424}
I20260709 13:16:27.657063 12358 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:27.657559 12306 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:27.661499 12285 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:27.758424 12390 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:16:27.872216 12402 log.cc:927] Time spent T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.397s	user 0.000s	sys 0.000s
I20260709 13:16:27.969168 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:28.005458 10876 raft_consensus.cc:2393] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 3: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:28.006640 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:28.007637 10876 raft_consensus.cc:2393] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 2: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:27.965138 11477 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:16:28.008713 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 43696.8us
I20260709 13:16:28.010202 11477 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:27.877378 10796 raft_consensus.cc:614] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:27.758096 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:27.760504 12352 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.659179 12392 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:27.980109 12347 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:28.023648 12352 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:28.024060 12347 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:28.025156 12392 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:27.645318 10698 rpcz_store.cc:276] 0709 13:16:23.957117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:23.969761 (+ 12644us) service_pool.cc:224] Handling call
0709 13:16:27.281428 (+3311667us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:16:24.769886 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:16:24.786522 (+ 16636us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:16:24.786557 (+    35us) write_op.cc:435] Acquired schema lock
0709 13:16:24.786578 (+    21us) tablet.cc:662] Decoding operations
0709 13:16:24.794730 (+  8152us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:16:24.794833 (+   103us) write_op.cc:641] Partition lock acquired for write op
0709 13:16:24.794866 (+    33us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:16:24.795078 (+   212us) tablet.cc:701] Row locks acquired
0709 13:16:24.795095 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:16:24.795242 (+   147us) write_op.cc:270] Start()
0709 13:16:24.795351 (+   109us) write_op.cc:276] Timestamp: P: 1783602984795200 usec, L: 0
0709 13:16:24.795373 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:16:24.798070 (+  2697us) log.cc:844] Serialized 536 byte log entry
0709 13:16:26.705380 (+1907310us) op_driver.cc:464] REPLICATION: finished
0709 13:16:26.913291 (+207911us) write_op.cc:301] APPLY: starting
0709 13:16:26.913378 (+    87us) tablet.cc:1367] starting BulkCheckPresence
0709 13:16:26.913483 (+   105us) tablet.cc:1370] finished BulkCheckPresence
0709 13:16:26.913608 (+   125us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:16:26.913875 (+   267us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:16:26.914044 (+   169us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:16:26.914091 (+    47us) write_op.cc:312] APPLY: finished
0709 13:16:26.914295 (+   204us) log.cc:844] Serialized 23 byte log entry
0709 13:16:26.914458 (+   163us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:26.914551 (+    93us) write_op.cc:454] Released schema lock
0709 13:16:26.914590 (+    39us) write_op.cc:341] FINISH: Updating metrics
0709 13:16:26.914978 (+   388us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:26.915007 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":11,"reported_tablets":9,"child_traces":[["op",{"apply.queue_time_us":207628,"apply.run_cpu_time_us":1665,"apply.run_wall_time_us":1665,"num_ops":1,"prepare.queue_time_us":420138,"prepare.run_cpu_time_us":24579,"prepare.run_wall_time_us":942446,"raft.queue_time_us":908690,"raft.run_cpu_time_us":13855,"raft.run_wall_time_us":160663,"replication_time_us":1909820,"thread_start_us":2221678,"threads_started":4,"wal-append.queue_time_us":982930,"wal-append.run_cpu_time_us":16810,"wal-append.run_wall_time_us":2150436}]]}
I20260709 13:16:28.025733 12352 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:27.428762 12287 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:28.026105 12347 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:27.973421 12293 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:28.028039 12287 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:28.028633 12287 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:27.971618 12329 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:28.029125 12287 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:28.029392 12329 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:27.672573 12354 raft_consensus.cc:2749] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:27.658524 12353 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:27.978257 10797 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 4.
I20260709 13:16:28.033106 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:27.760211 12356 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:27.759747 12261 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:27.313810 12289 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:27.767549 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:27.759945 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:27.767289 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:27.931457 12379 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:27.668308 10946 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:28.054936 10793 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:16:28.056840 12386 consensus_meta.cc:287] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.759s	user 0.009s	sys 0.000s
I20260709 13:16:28.057268 12386 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:27.965895 10841 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:16:28.059259 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 93409.7us
I20260709 13:16:28.060156 10841 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:28.061195 10841 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:16:27.763710 11504 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56410 (request call id 337) took 1950 ms (client timeout 2255 ms). Trace:
W20260709 13:16:28.062176 11504 rpcz_store.cc:269] 0709 13:16:25.813271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:25.823806 (+ 10535us) service_pool.cc:224] Handling call
0709 13:16:27.763458 (+1939652us) spinlock_profiling.cc:243] Waited 885 ms on lock 0x7b540068d8d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:27.763645 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2035698560}
I20260709 13:16:27.672830 10942 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:28.064177 12354 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:28.064726 12354 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:28.065191 12354 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:27.883715 12408 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:28.066545 12408 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:28.067549 12261 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:28.068265 12261 raft_consensus.cc:697] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:28.068774 12408 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:27.883298 12412 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:28.069164 10793 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 3.
I20260709 13:16:28.069938 12356 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:28.070470 12179 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:28.070604 12356 raft_consensus.cc:697] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:28.071061 12179 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:28.071533 12379 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:28.071889 12356 consensus_queue.cc:237] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:28.073385 12179 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:27.969162 10943 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.727s	user 0.008s	sys 0.027s
I20260709 13:16:27.982041 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:16:28.074142 10943 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.834s	user 0.008s	sys 0.027s
I20260709 13:16:27.978926 12394 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:28.012244 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:28.013053 12342 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:16:28.014569 12396 consensus_meta.cc:287] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.717s	user 0.008s	sys 0.003s
I20260709 13:16:28.075801 12396 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:28.076495 12394 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:28.077010 12394 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:28.077438 12394 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:28.077910 12396 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:27.301700 10867 rpcz_store.cc:276] 0709 13:16:25.397473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:25.419970 (+ 22497us) service_pool.cc:224] Handling call
0709 13:16:26.545261 (+1125291us) spinlock_profiling.cc:243] Waited 42.5 ms on lock 0x7b5800177098. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd90a22 00007f9c86ca9c09 00007f9c85e03ee0 00007f9c85ecd0ab 00007f9c85ecd131 00007f9c85e8efc0 00007f9c85e79929 00007f9c85e793d2 00007f9c7f3f0bd4
0709 13:16:26.569561 (+ 24300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:26.891016 (+321455us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:27.263332 (+372316us) spinlock_profiling.cc:243] Waited 238 ms on lock 0x7b54000f73d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:27.263566 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":644972160}
I20260709 13:16:27.981841 12306 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:28.079741 10798 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 2.
I20260709 13:16:27.954263 12292 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:28.069409 12261 consensus_queue.cc:237] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:28.084384 12292 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:27.758986 12065 consensus_queue.cc:1048] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.220s
W20260709 13:16:27.980635 12313 consensus_meta.cc:287] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.804s	user 0.011s	sys 0.000s
I20260709 13:16:28.085918 12313 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:28.087680 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:28.088011 12313 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:28.020277 12385 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:28.022514 12389 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:27.971858 12374 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:28.029426  9682 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:16:27.758297 12404 raft_consensus.cc:697] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:28.092083 12353 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:28.092461  9682 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:28.091821 12404 consensus_queue.cc:237] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:28.093078 11505 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 1.
I20260709 13:16:28.093982 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:27.967700 12297 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:27.963827 12399 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:27.656406 12343 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:28.095718 12343 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:28.096668 12399 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:28.031368 11477 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:28.033861 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:16:28.098101 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 66833.8us
I20260709 13:16:28.030709 10799 raft_consensus.cc:2368] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 2. Current term is 3.
I20260709 13:16:28.099045 10868 raft_consensus.cc:1275] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:16:28.099956 12287 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:28.043186 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:28.029960 12329 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:28.026660 12347 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:27.967557 12400 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:28.102913 12347 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:28.052878 12289 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:16:28.106761 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:28.029148 12293 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:28.069555 12412 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:27.981993 12285 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:28.108938 12285 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:28.109053 10875 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 3.
I20260709 13:16:28.030176 12390 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:28.094849 11503 raft_consensus.cc:2393] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 4: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:28.032413 12235 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:28.040381 12398 raft_consensus.cc:2804] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:28.065601 12358 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:28.112910 10915 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:16:28.159447 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 46658.4us
W20260709 13:16:28.119923 10842 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.869s	user 0.012s	sys 0.024s
W20260709 13:16:28.170496 10842 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.919s	user 0.013s	sys 0.024s
W20260709 13:16:28.171001 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 926524us
I20260709 13:16:27.690030 12344 consensus_queue.cc:1048] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.342s
W20260709 13:16:28.136562 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:28.139303 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 264778us
I20260709 13:16:28.175700 10767 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:28.023150 12294 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:28.132266 12353 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:28.207342 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:28.133168 12351 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:28.216444 10801 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 3.
I20260709 13:16:28.140261 12235 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:28.139340 12398 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:28.203263 12293 raft_consensus.cc:2804] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:16:28.358079 12414 log.cc:927] Time spent T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.335s	user 0.003s	sys 0.000s
W20260709 13:16:28.768342 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 504519us
I20260709 13:16:28.768373 11505 raft_consensus.cc:1240] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:28.783674 11511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56404 (request call id 361) took 1533 ms. Trace:
I20260709 13:16:28.355293 12306 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:28.566352 12416 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:28.264156 12065 consensus_queue.cc:1048] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.570s
I20260709 13:16:28.784740 10792 raft_consensus.cc:2368] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 3. Current term is 4.
I20260709 13:16:28.202378 12396 raft_consensus.cc:2804] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:28.784057 11511 rpcz_store.cc:276] 0709 13:16:27.250769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.251571 (+   802us) service_pool.cc:224] Handling call
0709 13:16:27.260781 (+  9210us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:27.265288 (+  4507us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:27.265319 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:27.275793 (+ 10474us) log.cc:844] Serialized 26 byte log entry
0709 13:16:27.645170 (+369377us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:27.645198 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:27.645244 (+    46us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:28.357524 (+712280us) spinlock_profiling.cc:243] Waited 9.47 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:16:28.783381 (+425857us) raft_consensus.cc:1685] finished
0709 13:16:28.783404 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:28.783603 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4086,"raft.queue_time_us":4312,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":99,"spinlock_wait_cycles":21771136,"thread_start_us":347268,"threads_started":1,"wal-append.queue_time_us":695913}
I20260709 13:16:28.785882 12396 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:16:28.637140 12417 log.cc:927] Time spent T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.321s	user 0.001s	sys 0.000s
I20260709 13:16:28.786469 12396 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:28.767768 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:28.355032 12394 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:28.349787 12354 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:28.645069 12386 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:28.790596 12420 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:28.488463 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:28.775835 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 600220us
I20260709 13:16:28.791409 12420 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:28.352525 12329 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:28.360806 12297 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:28.782573 12293 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:28.796893 12293 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:28.355788 12292 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:28.353749 12374 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:28.799041 12292 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:28.746695 10948 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.009s	sys 0.006s
W20260709 13:16:28.799998 10948 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.009s	sys 0.008s
I20260709 13:16:28.800143 11503 raft_consensus.cc:1240] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:28.801884 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:28.802999 11504 raft_consensus.cc:2393] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 4: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:28.803208 12329 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:28.604789 12371 consensus_queue.cc:1059] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.200s
I20260709 13:16:28.285861 12381 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:28.777871 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 200085us
I20260709 13:16:28.358496 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:28.314626 12351 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:28.781930 12398 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:16:28.429005 10913 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.013s	sys 0.008s
I20260709 13:16:28.794906 12416 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:28.827385 10913 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.957s	user 0.013s	sys 0.014s
I20260709 13:16:28.263355 12379 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:28.353749 12388 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:28.339232 12353 raft_consensus.cc:697] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 4 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:28.774971 12400 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:28.795363 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:28.566098 12242 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:16:28.846498 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 63035.7us
I20260709 13:16:28.805343 12235 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:28.824602 12294 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:28.354638 12126 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:28.575222 10950 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.003s	sys 0.019s
W20260709 13:16:28.854843 10950 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.888s	user 0.005s	sys 0.019s
I20260709 13:16:28.806826 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:28.263142 12389 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:28.791999 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:28.793850 12420 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:28.819085 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:28.863284 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:28.827821 10941 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 2.
I20260709 13:16:28.826975 10793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 1223) took 1137 ms. Trace:
I20260709 13:16:28.826177 10916 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:16:28.684556 12395 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.009s	sys 0.002s
I20260709 13:16:28.783115 12412 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:28.867106 10793 rpcz_store.cc:276] 0709 13:16:27.670993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.751089 (+ 80096us) service_pool.cc:224] Handling call
0709 13:16:27.762028 (+ 10939us) spinlock_profiling.cc:243] Waited 9.37 ms on lock 0x7b5800002798. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd90a22 00007f9c86ca9c09 00007f9c85e03ee0 00007f9c85ecd0ab 00007f9c85ecd131 00007f9c85e7b0a0 00007f9c85e7a984 00007f9c7f3f56d4 00007f9c7f3f5662
0709 13:16:28.807753 (+1045725us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b5400294d54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:28.808049 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":438071808}
I20260709 13:16:28.873754 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:28.874562 10800 raft_consensus.cc:2393] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 4: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:28.263660 12287 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:28.791043 12386 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:28.826651 12404 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:28.808109 12261 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:28.863934 10949 raft_consensus.cc:2376] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 4. Re-sending same reply.
I20260709 13:16:28.263976 10876 raft_consensus.cc:2393] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 4: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:28.861006 10794 raft_consensus.cc:1240] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:16:28.865952 10766 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:28.857252 12347 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:28.860651 12374 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:28.831239 12381 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:28.869102 11478 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:16:28.874684 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:16:28.917084 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 48081.7us
I20260709 13:16:28.853097 12358 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:28.852695 12371 consensus_queue.cc:1243] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer e01ecd008edd4aa2b6b2b47efbc2ea87 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:16:28.854408 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:28.851473 12242 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:28.868372 10916 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:16:28.923331 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 55031.1us
I20260709 13:16:28.924431 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:28.911253 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:16:28.911782 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 45952.1us
I20260709 13:16:28.861083 10869 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:28.926476 10766 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:16:28.878137 12395 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.821s	user 0.010s	sys 0.002s
I20260709 13:16:28.915333 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:28.861523 10943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 1555) took 1622 ms. Trace:
I20260709 13:16:28.860143 12379 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:28.918947 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:28.877687 12400 raft_consensus.cc:2749] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:28.858279 12424 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:28.934206 10916 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:28.934842 12371 raft_consensus.cc:2749] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:28.925103 10794 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 1.
I20260709 13:16:28.939699 10800 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:28.942176 12441 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:28.921753 12429 log.cc:927] Time spent T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 13:16:28.938640 12437 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:28.948819 10950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 1247) took 1298 ms. Trace:
I20260709 13:16:28.916518 12385 raft_consensus.cc:2749] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:28.913749 12390 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:28.869691 12389 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:28.957830 10949 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 1.
I20260709 13:16:28.961316 11507 raft_consensus.cc:1240] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:29.005108 10950 rpcz_store.cc:276] 0709 13:16:27.650019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.940123 (+290104us) service_pool.cc:224] Handling call
0709 13:16:27.940512 (+   389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:28.855236 (+914724us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:28.855271 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:28.857408 (+  2137us) log.cc:844] Serialized 26 byte log entry
0709 13:16:28.943131 (+ 85723us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:16:28.943186 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:28.943281 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:28.948531 (+  5250us) raft_consensus.cc:1685] finished
0709 13:16:28.948562 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:28.948736 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26111,"raft.queue_time_us":156325,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":79,"thread_start_us":973503,"threads_started":2,"wal-append.queue_time_us":85757}
I20260709 13:16:28.903105 12287 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:28.933626 12438 raft_consensus.cc:2804] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:28.909948 12386 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:29.026134 12126 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:16:29.046221 12434 log.cc:927] Time spent T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
I20260709 13:16:29.047086 12242 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:16:29.047374 12432 log.cc:927] Time spent T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 13:16:28.945844 10942 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:29.047807 12389 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:29.046268 12306 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:29.048915 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:29.050143 10942 raft_consensus.cc:2393] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 3: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:29.021744 12441 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:29.053215 12396 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:29.053900 12441 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:28.961792 10916 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:29.054713 12292 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:16:29.055280 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 93607.7us
I20260709 13:16:29.055989 12354 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:29.056694 10916 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:29.057993 10916 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:29.058363 12351 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:28.956631 12379 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:28.958871 12424 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:29.019663 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:29.061520 10950 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 3.
I20260709 13:16:29.062752 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:29.064284 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:28.916850 12404 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:29.047209 12407 log.cc:927] Time spent T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 1.028s	user 0.018s	sys 0.000s
I20260709 13:16:29.058005 12287 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:29.049197 12416 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:29.054237 12381 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:29.060727 12424 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:16:29.071450 12433 log.cc:927] Time spent T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.160s	user 0.000s	sys 0.001s
I20260709 13:16:29.073375 12388 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:29.008431 12347 raft_consensus.cc:2804] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:28.955502 12444 consensus_queue.cc:1048] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.119s
I20260709 13:16:29.074476 12347 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:29.074960 12347 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:29.076696 10941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41014 (request call id 1242) took 1428 ms. Trace:
I20260709 13:16:28.916582 12261 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:28.931020 12179 consensus_queue.cc:1048] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
W20260709 13:16:29.020516 12343 consensus_meta.cc:287] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.924s	user 0.013s	sys 0.000s
I20260709 13:16:29.067481 10950 raft_consensus.cc:2393] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 4: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
W20260709 13:16:29.079461 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 135333us
I20260709 13:16:29.068795 10948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 1258) took 1250 ms. Trace:
I20260709 13:16:29.080044 10948 rpcz_store.cc:276] 0709 13:16:27.818367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.958242 (+139875us) service_pool.cc:224] Handling call
0709 13:16:27.958636 (+   394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:28.104122 (+145486us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:28.104152 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:28.104620 (+   468us) log.cc:844] Serialized 26 byte log entry
0709 13:16:28.800913 (+696293us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:28.800971 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:29.047913 (+246942us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b54003aff54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:29.047959 (+    46us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:29.064064 (+ 16105us) raft_consensus.cc:1685] finished
0709 13:16:29.064094 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:29.068715 (+  4621us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":333985664,"thread_start_us":695483,"threads_started":1,"wal-append.queue_time_us":747418}
I20260709 13:16:29.060784 12379 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:29.083359 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:29.090065 10950 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2.
I20260709 13:16:28.924721 12353 consensus_queue.cc:237] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:29.091024 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:29.091601 10950 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:29.016196 12374 consensus_queue.cc:1048] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 13:16:29.098917 12404 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:29.068934 12428 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:29.089828 12343 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:29.100111 12428 raft_consensus.cc:697] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 2 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:16:29.101135 12418 log.cc:927] Time spent T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.635s	user 0.004s	sys 0.004s
I20260709 13:16:29.101828 12297 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:29.101418 12428 consensus_queue.cc:237] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:29.053215 12293 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:29.104701 12293 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:29.106113 12396 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:29.078850 12285 consensus_meta.cc:287] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.960s	user 0.009s	sys 0.004s
I20260709 13:16:29.066740 12371 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:29.016791 12437 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:29.113248 12285 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:29.117753 12261 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:29.118364 12437 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:29.119128 12386 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:29.113318 12371 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:29.119925 12371 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:29.101425 12343 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:29.122902 12285 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:29.121119 12398 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:29.135771 12438 raft_consensus.cc:697] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 2 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:29.137217 12438 consensus_queue.cc:237] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:29.161720 12387 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.141s	user 0.029s	sys 0.025s
W20260709 13:16:29.162351 12387 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.142s	user 0.029s	sys 0.025s
I20260709 13:16:29.196571  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:16:29.196341 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 403836us
I20260709 13:16:29.203274 10767 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:29.209947 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:16:29.217190 10792 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 1219) took 1551 ms (client timeout 1698 ms). Trace:
W20260709 13:16:29.219887 10798 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 1218) took 1554 ms (client timeout 1567 ms). Trace:
W20260709 13:16:29.220546 10798 rpcz_store.cc:269] 0709 13:16:27.665762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.672528 (+  6766us) service_pool.cc:224] Handling call
0709 13:16:29.219409 (+1546881us) spinlock_profiling.cc:243] Waited 30.1 ms on lock 0x7b54003be2d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:29.219685 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":69236096}
I20260709 13:16:29.221768 10767 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:16:29.222967 10799 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 1224) took 1549 ms (client timeout 1816 ms). Trace:
W20260709 13:16:29.223459 10799 rpcz_store.cc:269] 0709 13:16:27.673480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.916670 (+243190us) service_pool.cc:224] Handling call
0709 13:16:29.222510 (+1305840us) spinlock_profiling.cc:243] Waited 75.6 ms on lock 0x7b54000e9554. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:29.222798 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":173917056}
I20260709 13:16:29.229077 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:29.013424 10869 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 4.
I20260709 13:16:29.230984 10872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 132) took 2024 ms. Trace:
I20260709 13:16:29.234164 10872 rpcz_store.cc:276] 0709 13:16:27.206854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.207027 (+   173us) service_pool.cc:224] Handling call
0709 13:16:27.209521 (+  2494us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:27.209745 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:27.209774 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:27.214423 (+  4649us) log.cc:844] Serialized 26 byte log entry
0709 13:16:27.308769 (+ 94346us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:27.308795 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:27.308848 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:29.046509 (+1737661us) spinlock_profiling.cc:243] Waited 13.3 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:16:29.230228 (+183719us) raft_consensus.cc:1685] finished
0709 13:16:29.230274 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:29.230697 (+   423us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":30535040,"thread_start_us":93937,"threads_started":1,"wal-append.queue_time_us":768141}
I20260709 13:16:29.231726 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:16:29.232738 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:29.232795 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:16:29.235920 10869 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49290 (request call id 134) took 1984 ms (client timeout 2243 ms). Trace:
I20260709 13:16:29.236936 10873 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 3.
I20260709 13:16:29.238129 10795 raft_consensus.cc:2368] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 3. Current term is 4.
W20260709 13:16:29.237632 10869 rpcz_store.cc:269] 0709 13:16:27.251087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.251195 (+   108us) service_pool.cc:224] Handling call
0709 13:16:29.235382 (+1984187us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54000997d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:29.235628 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2805277056}
I20260709 13:16:29.241158 10876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 1225) took 1487 ms. Trace:
I20260709 13:16:29.241580 10876 rpcz_store.cc:276] 0709 13:16:27.753416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:28.032826 (+279410us) service_pool.cc:224] Handling call
0709 13:16:29.240618 (+1207792us) spinlock_profiling.cc:243] Waited 230 ms on lock 0x7b5400316ad4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:29.240952 (+   334us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":528903424}
W20260709 13:16:29.217914 10792 rpcz_store.cc:269] 0709 13:16:27.666069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.672355 (+  6286us) service_pool.cc:224] Handling call
0709 13:16:29.216552 (+1544197us) spinlock_profiling.cc:243] Waited 687 ms on lock 0x7b54000bccd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:29.216926 (+   374us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1579076864}
I20260709 13:16:29.250818 10873 raft_consensus.cc:1275] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:16:29.251304 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:29.252735 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:29.316125 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:29.319253 12386 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:29.337401 12412 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:29.337666 12297 raft_consensus.cc:2749] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:29.338289 12412 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:29.339411 12405 consensus_queue.cc:1059] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:16:29.341406 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:29.342092 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:29.337391 12387 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:29.354836 10874 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.752s	user 0.000s	sys 0.017s
I20260709 13:16:29.341010 12405 raft_consensus.cc:3055] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:29.359074 10872 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2.
I20260709 13:16:29.359367 10944 raft_consensus.cc:2393] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 4: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:29.077278 10941 rpcz_store.cc:276] 0709 13:16:27.648301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.654827 (+  6526us) service_pool.cc:224] Handling call
0709 13:16:29.076474 (+1421647us) spinlock_profiling.cc:243] Waited 785 ms on lock 0x7b5400173254. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:29.076614 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1804567296}
I20260709 13:16:29.350561 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:29.369343 12387 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:29.371439 10942 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:29.373924 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:29.375066 10948 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 3.
I20260709 13:16:29.375887 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:29.376250 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:29.377210 10799 raft_consensus.cc:2393] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 3: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:29.377177 10948 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:29.375804 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:29.384729 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:29.368136 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:29.391932 12344 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:29.392695 10944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41014 (request call id 1243) took 1744 ms. Trace:
I20260709 13:16:29.393138 12344 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:29.395064 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:29.396472 10946 raft_consensus.cc:2393] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 4: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
W20260709 13:16:29.368243 10874 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.889s	user 0.001s	sys 0.017s
I20260709 13:16:29.346643 12386 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:29.393354 10944 rpcz_store.cc:276] 0709 13:16:27.648609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.648767 (+   158us) service_pool.cc:224] Handling call
0709 13:16:29.392309 (+1743542us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b54000a96d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:29.392608 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3304972544}
W20260709 13:16:29.392093 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 334075us
I20260709 13:16:29.397346 11477 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:29.361912 12405 raft_consensus.cc:740] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:29.488188 10945 raft_consensus.cc:3055] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:29.364506 12395 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:29.499147 10945 raft_consensus.cc:740] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:28.939218 10943 rpcz_store.cc:276] 0709 13:16:27.236312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.237304 (+   992us) service_pool.cc:224] Handling call
0709 13:16:27.237860 (+   556us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:27.238309 (+   449us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:27.238343 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:27.239419 (+  1076us) log.cc:844] Serialized 506 byte log entry
0709 13:16:28.074587 (+835168us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:28.074651 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:28.074751 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:28.846381 (+771630us) raft_consensus.cc:1685] finished
0709 13:16:28.846401 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:28.858675 (+ 12274us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":834814,"threads_started":1,"wal-append.queue_time_us":941831}
I20260709 13:16:29.432647 10947 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:29.435372 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:29.417239 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:29.504987 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:29.469785 12448 ts_tablet_manager.cc:933] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586: Initiating tablet copy from peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:29.509430 12179 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:29.510293 12179 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:29.513700 12179 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:29.513751 12387 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:29.417829 12398 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:29.517789 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 536582us
W20260709 13:16:29.518311 12235 consensus_meta.cc:287] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.657s	user 0.006s	sys 0.008s
I20260709 13:16:29.515843 10875 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:29.519271 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:29.522367 12387 consensus_queue.cc:1059] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:29.523730 10841 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:29.524580 12412 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:29.525727 10916 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:29.528754 10944 raft_consensus.cc:1275] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:16:29.529139 11504 consensus_meta.cc:287] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.653s	user 0.010s	sys 0.005s
I20260709 13:16:29.529872 10874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1228) took 1670 ms. Trace:
I20260709 13:16:29.531433 10842 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:29.428167 10796 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:29.531927 10943 raft_consensus.cc:1275] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:16:29.447239 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 415237us
I20260709 13:16:29.501184 12395 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:29.541075 10942 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 3.
W20260709 13:16:29.541636 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 144689us
I20260709 13:16:29.392766 12396 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:29.543044 10797 raft_consensus.cc:2393] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 4: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:29.432971 12400 raft_consensus.cc:2749] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:29.551160 12343 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:29.563918 10799 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2.
I20260709 13:16:29.551299 12292 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:29.548934 12395 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:29.570995 12285 raft_consensus.cc:2804] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:29.571941 12285 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:29.572614 12285 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:29.571159 12443 raft_consensus.cc:2804] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:29.552346 12396 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:29.544387 12235 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:29.562000 11504 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 3.
I20260709 13:16:29.581702 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:29.587327 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:29.576442 12443 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:29.569547 12292 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:29.579062 12396 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:29.613550 12396 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:29.615361 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 49641.7us
I20260709 13:16:29.560506 12405 consensus_queue.cc:260] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:29.618989 10944 raft_consensus.cc:1275] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:16:29.614562 12065 raft_consensus.cc:2749] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:29.629261 12386 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:29.630756 12449 log.cc:927] Time spent T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.199s	user 0.002s	sys 0.003s
I20260709 13:16:29.632478 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:29.545401 10874 rpcz_store.cc:276] 0709 13:16:27.859994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:28.088659 (+228665us) service_pool.cc:224] Handling call
0709 13:16:28.088991 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:28.089243 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:28.089273 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:28.186542 (+ 97269us) log.cc:844] Serialized 26 byte log entry
0709 13:16:29.529295 (+1342753us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:29.529367 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:29.529482 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:29.529557 (+    75us) raft_consensus.cc:1685] finished
0709 13:16:29.529578 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:29.529795 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":1027614,"threads_started":1,"wal-append.queue_time_us":1205699}
I20260709 13:16:29.640631 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:29.641144 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:16:29.641635 10950 consensus_meta.cc:287] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.516s	user 0.008s	sys 0.007s
I20260709 13:16:29.642022 10950 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 4.
I20260709 13:16:29.642278 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:16:29.641260 12454 log.cc:927] Time spent T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
W20260709 13:16:29.642948 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 543277us
W20260709 13:16:29.643736 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 60414us
I20260709 13:16:29.645277 10842 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:16:29.647087 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 123590us
W20260709 13:16:29.647846 12459 log.cc:927] Time spent T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:16:29.648579 12358 raft_consensus.cc:2749] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:29.605216 12443 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:29.642019 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:29.631780 10916 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:29.650055 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:29.607054 12292 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:29.648434 10841 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:16:29.660360 12347 consensus_meta.cc:287] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.585s	user 0.011s	sys 0.008s
I20260709 13:16:29.665839 11477 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:29.667335 12405 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:29.670686 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:29.671193 12438 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:16:29.679090 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 103810us
I20260709 13:16:29.681066 10875 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 3.
I20260709 13:16:29.671098 12344 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:29.605221 12343 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:29.697252 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:29.697499 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:29.697916 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:29.698763 10795 raft_consensus.cc:2393] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 3: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
W20260709 13:16:29.699503 12458 log.cc:927] Time spent T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.187s	user 0.003s	sys 0.002s
I20260709 13:16:29.699733 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:29.700809 10795 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2.
I20260709 13:16:29.702037 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:29.702869 10795 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:29.705240 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:29.705862 10792 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:29.698200 10797 raft_consensus.cc:2393] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 4: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
W20260709 13:16:29.706841 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 63015.1us
I20260709 13:16:29.707032 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:29.671209 10945 consensus_queue.cc:260] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:29.709458 10799 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:29.700829 12400 raft_consensus.cc:2804] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:29.711290 12400 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:29.682508 12438 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:29.712155 12438 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:29.712281 12344 raft_consensus.cc:2749] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:29.682322 12347 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:29.713997 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:29.714807 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:29.715509 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:29.715883 11512 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 1.
I20260709 13:16:29.716382 11503 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 1.
I20260709 13:16:29.711730 12400 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:29.715040 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:29.750200 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:29.714990 11504 raft_consensus.cc:2393] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 5: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
W20260709 13:16:29.756696 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 61494.8us
I20260709 13:16:29.763285 10947 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 3.
I20260709 13:16:29.765100 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:16:29.771556 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 53076.1us
I20260709 13:16:29.696307 12358 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:29.775408 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:29.779692 10916 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:29.781167 10796 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
I20260709 13:16:29.782178 10916 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:29.782907 12437 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:29.783138 12428 raft_consensus.cc:2804] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:29.783694 12428 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:29.784214 12428 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:29.783746 12437 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:29.786296 12437 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:29.786939 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:29.787994 12412 raft_consensus.cc:3055] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:29.788463 12412 raft_consensus.cc:740] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:29.789826 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:29.789757 12412 consensus_queue.cc:260] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:29.790776 10794 raft_consensus.cc:2393] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 3: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:29.791842 10799 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2.
I20260709 13:16:29.788396 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:29.805332 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:29.806351 11508 raft_consensus.cc:2368] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 2. Current term is 3.
W20260709 13:16:29.811005 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 162727us
I20260709 13:16:29.813592 10841 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
W20260709 13:16:29.813791 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 95404.3us
I20260709 13:16:29.814486 12344 consensus_queue.cc:1059] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:29.814735 10841 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:29.782449 12395 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:29.822088 12353 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:29.822248 12344 raft_consensus.cc:3055] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:29.825560 10945 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:29.827721 10798 raft_consensus.cc:2393] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 4: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:29.822822 12344 raft_consensus.cc:740] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:29.822835 12353 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:29.830157 12344 consensus_queue.cc:260] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:29.811093 12374 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:29.834546 12344 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:29.819190 12387 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:29.815982 12347 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:29.837888 10800 raft_consensus.cc:2393] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 3: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:29.833433 12412 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:29.845041 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:29.835589 12235 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:29.822435 12289 consensus_queue.cc:1059] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 13:16:29.843144 12381 consensus_queue.cc:1059] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 13:16:29.847625 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:29.843261 12387 raft_consensus.cc:697] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:29.836385 12437 consensus_queue.cc:1059] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
W20260709 13:16:29.867885 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 52729us
I20260709 13:16:29.837090 12358 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:29.878922 12396 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:29.887066 10944 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 1.
I20260709 13:16:29.877316 12374 raft_consensus.cc:697] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 4 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:29.894726 12445 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:29.895270 12445 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:29.867161 12289 consensus_queue.cc:1243] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer 12682eb70bc84c76879e04a8628ee586 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:16:29.895628 12445 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:16:29.892467 12371 consensus_meta.cc:287] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.703s	user 0.009s	sys 0.009s
I20260709 13:16:29.899292 12371 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:29.899700 12400 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:29.901533 12371 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:29.901754 12400 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:29.904522 12438 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:29.885329 12448 tablet_copy_client.cc:323] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Beginning tablet copy session from remote peer at address 127.9.116.130:37895
I20260709 13:16:29.890511 10916 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:29.907356 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:29.907676 10841 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:29.906836 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:29.908948 10801 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
W20260709 13:16:29.909878 12465 log.cc:927] Time spent T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
I20260709 13:16:29.910465 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:29.913928 12395 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:29.914237 12405 raft_consensus.cc:3055] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:29.914734 12405 raft_consensus.cc:740] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:29.918530 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:29.919605 12065 raft_consensus.cc:3055] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:29.893594 11504 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:29.937397 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 46951.5us
I20260709 13:16:29.953922 10793 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:29.906562 12428 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:29.932394 12343 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:29.929375 12065 raft_consensus.cc:740] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:29.960807 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:29.967764 10944 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:29.903388 12443 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:29.928810 10795 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 3.
I20260709 13:16:29.936779 10867 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 1.
I20260709 13:16:29.978542 10794 raft_consensus.cc:1240] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:29.954727 12386 raft_consensus.cc:2804] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:16:29.980830 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 103093us
I20260709 13:16:29.964186 12347 raft_consensus.cc:2804] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:29.942370 12292 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:29.998064 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:29.980959 12386 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:30.010653 12342 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
W20260709 13:16:30.011711 12461 log.cc:927] Time spent T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.479s	user 0.007s	sys 0.004s
W20260709 13:16:30.013404 12469 log.cc:927] Time spent T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.095s	user 0.001s	sys 0.001s
I20260709 13:16:30.015116 10942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 1568) took 1005 ms. Trace:
I20260709 13:16:29.997292 10792 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 3.
I20260709 13:16:30.007000 12347 raft_consensus.cc:697] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 3 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:30.007361 12386 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:30.015221 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:30.042263 12374 consensus_queue.cc:237] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:30.048380 10794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 1583) took 1144 ms. Trace:
I20260709 13:16:30.016595 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:30.055653 10794 rpcz_store.cc:276] 0709 13:16:28.904224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:29.791567 (+887343us) service_pool.cc:224] Handling call
0709 13:16:29.978269 (+186702us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:29.980123 (+  1854us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:30.048167 (+ 68044us) spinlock_profiling.cc:243] Waited 182 ms on lock 0x7b5400387f50. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:30.048322 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":419330432}
I20260709 13:16:30.030233 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:30.031116 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:30.103482 10950 raft_consensus.cc:2393] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 4: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:30.015513 10942 rpcz_store.cc:276] 0709 13:16:29.010981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:29.550816 (+539835us) service_pool.cc:224] Handling call
0709 13:16:29.577746 (+ 26930us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:29.650304 (+ 72558us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:29.650336 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:29.675567 (+ 25231us) log.cc:844] Serialized 26 byte log entry
0709 13:16:29.815949 (+140382us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:29.815975 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:29.816020 (+    45us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:30.014860 (+198840us) raft_consensus.cc:1685] finished
0709 13:16:30.014877 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:30.015054 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1255,"raft.queue_time_us":20355,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":200,"thread_start_us":78723,"threads_started":1,"wal-append.queue_time_us":117077}
I20260709 13:16:30.038278 11478 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:16:30.018466 10842 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:16:30.103840 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 96136.3us
I20260709 13:16:30.133391 10948 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:30.043766 12353 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:30.140882 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:30.141682 10800 raft_consensus.cc:2393] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 3: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:30.018817 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:30.143034 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:30.028342 12443 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:30.143854 10943 raft_consensus.cc:1240] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:30.017036 12405 consensus_queue.cc:260] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:30.143874 10941 raft_consensus.cc:2393] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 2: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:30.145380 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:30.145706 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:30.146440 12404 raft_consensus.cc:3055] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:30.146728 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:30.147755 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:30.147992 12438 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:30.150323 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:30.147054 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:30.155189 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:30.024893 12428 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:16:30.019177 12464 log.cc:927] Time spent T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 13:16:30.028811 12343 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.023502 12358 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:30.047364 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:30.031867 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:30.049719 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:30.050912 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:30.148928 12353 consensus_queue.cc:1048] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 13:16:30.147970 10841 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:30.219632 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:30.061157 10916 leader_election.cc:400] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 election: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 2. Current term is 3.
W20260709 13:16:30.171241 12285 consensus_meta.cc:287] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.598s	user 0.019s	sys 0.003s
I20260709 13:16:30.046972 12387 consensus_queue.cc:237] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:30.223243 10945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 1241) took 2572 ms. Trace:
I20260709 13:16:30.171795 12235 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:30.209833 10941 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 4
W20260709 13:16:30.172226 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 134101us
I20260709 13:16:30.172860 12437 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:16:30.173753 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 155408us
I20260709 13:16:30.171826 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:30.029130 12400 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:30.189052 10793 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
I20260709 13:16:30.198228 10944 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
I20260709 13:16:30.201522 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:30.020310 12292 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:30.208416 11477 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:30.181103 12404 raft_consensus.cc:740] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:30.046761 10698 catalog_manager.cc:5697] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 1 to 3, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 3 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:30.220364 10916 leader_election.cc:403] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:16:30.227922 10943 raft_consensus.cc:2393] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 4: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:30.018657 12347 consensus_queue.cc:237] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:30.245672 12404 consensus_queue.cc:260] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:30.247108 12404 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:30.227515 12235 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:30.227965 10766 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:30.252471 10949 raft_consensus.cc:2368] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 3. Current term is 4.
W20260709 13:16:30.233070 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 85212us
I20260709 13:16:30.235702 10842 leader_election.cc:400] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 election: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 3. Current term is 4.
I20260709 13:16:30.217957 10915 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:30.223776 10945 rpcz_store.cc:276] 0709 13:16:27.647943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.650964 (+  3021us) service_pool.cc:224] Handling call
0709 13:16:27.651407 (+   443us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:29.487252 (+1835845us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b540030ffa0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80e2cdc0 00007f9c80eb71c0 00007f9c80eb8231 00007f9c80eb4d8f 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c
0709 13:16:29.939132 (+451880us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 13:16:29.939176 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:29.939238 (+    62us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:29.950094 (+ 10856us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:30.207670 (+257576us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b54001e2154. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:30.207743 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:30.219964 (+ 12221us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7041,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":6615,"spinlock_wait_cycles":3291549184}
W20260709 13:16:30.259666 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 269153us
W20260709 13:16:30.265396 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 116337us
I20260709 13:16:30.215956 12065 consensus_queue.cc:260] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:30.265527 12292 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:30.267295 12065 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:30.267699 12292 raft_consensus.cc:697] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 3 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:30.268167 11478 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:16:30.268721 12292 consensus_queue.cc:237] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:30.270237 12353 consensus_queue.cc:1048] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:30.274531 10948 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 3.
I20260709 13:16:30.267326 12428 raft_consensus.cc:2749] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 2. Current term is 3.
I20260709 13:16:30.229593 11504 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 2.
I20260709 13:16:30.221830 12343 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:30.278590 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 66119.4us
I20260709 13:16:30.257771 10698 catalog_manager.cc:5697] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 1 to 2, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 2 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:30.227451 12285 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:30.285166 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 63908.2us
I20260709 13:16:30.248528 10796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39442 (request call id 80) took 1022 ms. Trace:
I20260709 13:16:30.261467 12235 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:30.279103 12381 raft_consensus.cc:2749] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:30.291733 10842 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:30.280340 12428 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:30.328159 12428 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:30.328718 12428 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:30.331148 12347 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.290061 10796 rpcz_store.cc:276] 0709 13:16:29.226519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:29.782063 (+555544us) service_pool.cc:224] Handling call
0709 13:16:30.050697 (+268634us) spinlock_profiling.cc:243] Waited 5.01 ms on lock 0x7b5800002798. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd90a22 00007f9c86ca9c09 00007f9c85e03ee0 00007f9c85ecd0ab 00007f9c85ecd131 00007f9c85e7b0a0 00007f9c85e7a984 00007f9c7f3f56d4 00007f9c7f3f5662
0709 13:16:30.243237 (+192540us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11526016}
I20260709 13:16:30.331815 12347 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:30.334059 12347 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:30.295855 10841 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:16:30.335208 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 62696.2us
I20260709 13:16:30.336052 10841 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:30.337054 12477 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:30.337487 12477 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:30.337822 12477 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:30.292032 12451 raft_consensus.cc:2804] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:30.251942 12400 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:30.339871 12451 raft_consensus.cc:697] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 4 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:30.340405 10916 leader_election.cc:400] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 3. Current term is 4.
I20260709 13:16:30.340744 10916 leader_election.cc:403] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:16:30.341203 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 54075.1us
I20260709 13:16:30.341037 12451 consensus_queue.cc:237] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:30.342051 10916 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:30.295205 12368 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:30.346894 12443 raft_consensus.cc:2749] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:30.347311 12368 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:30.349538 12368 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:30.321663 12412 raft_consensus.cc:2955] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:30.350464 12374 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:30.351042 12374 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:30.351524 12374 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:30.323259 10766 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:30.356112 10766 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:30.357697 10766 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:30.327069 10915 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:30.349742 10841 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:30.359884 10915 leader_election.cc:400] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 election: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 2. Current term is 3.
I20260709 13:16:30.360212 10915 leader_election.cc:403] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
W20260709 13:16:30.360312 12473 log.cc:927] Time spent T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.129s	user 0.004s	sys 0.000s
I20260709 13:16:30.360802 12443 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:30.361351 12443 raft_consensus.cc:697] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:30.361964 12371 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:30.362552 12371 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:30.362964 12471 raft_consensus.cc:2804] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:30.363440 12471 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:30.363860 12471 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:30.364754 12371 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:30.365338 12371 raft_consensus.cc:2749] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 2. Current term is 3.
I20260709 13:16:30.367358 12313 raft_consensus.cc:2804] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:30.367820 12313 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:30.368201 12313 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:30.368280 12395 raft_consensus.cc:2749] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:30.368474 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:30.369118 12395 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:30.369300 10943 raft_consensus.cc:2368] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 3. Current term is 4.
I20260709 13:16:30.369515 12395 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:30.371707 12405 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:30.371932 12289 consensus_queue.cc:1059] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:30.283001 12358 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:30.396129 12343 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:30.396353 12358 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:30.292325 10698 catalog_manager.cc:5697] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 2 to 3, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 3 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:30.372109 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:16:30.397220 12445 consensus_meta.cc:287] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.501s	user 0.012s	sys 0.000s
I20260709 13:16:30.397631 12445 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:30.398154 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:30.369892 12395 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:30.399416 12376 raft_consensus.cc:2804] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:30.362344 12443 consensus_queue.cc:237] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:30.399848 12445 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:30.375417 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:30.405543 10945 raft_consensus.cc:2393] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 4: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:30.406718 10945 raft_consensus.cc:1240] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 1. Current term is 2. Ops: []
W20260709 13:16:30.401824 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 113661us
I20260709 13:16:30.377418 12396 raft_consensus.cc:3055] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:30.377625 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:30.397832 10698 catalog_manager.cc:5697] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 1 to 2, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 2 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:30.381430 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:30.408030 10947 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:30.382817 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:30.408468 10948 raft_consensus.cc:2393] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 3: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:30.408907 10944 raft_consensus.cc:2393] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 4: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
W20260709 13:16:30.385313 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:30.385452 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:30.385650 10946 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:30.392216 12400 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:30.393113 10949 raft_consensus.cc:1240] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:30.394556 12285 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:30.385342 12235 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:30.413363 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 177831us
I20260709 13:16:30.399958 12376 raft_consensus.cc:697] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:30.396849 12358 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:30.379734 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:30.420413 10950 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
I20260709 13:16:30.418377 10698 catalog_manager.cc:5697] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 1 to 2, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 2 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
W20260709 13:16:30.402371 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 135496us
I20260709 13:16:30.398443 12428 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:30.423863 10698 catalog_manager.cc:5697] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New cstate: current_term: 1 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:30.407955 12396 raft_consensus.cc:740] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:30.443357 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:30.444178 10950 raft_consensus.cc:2393] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 4: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:30.446305 12235 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:30.448422 10842 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:30.449806 12374 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:30.451104 12356 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:30.451555 12356 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:30.451297 12376 consensus_queue.cc:237] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:30.451946 12356 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:30.455407 12428 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:30.456729 10949 raft_consensus.cc:1275] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:30.459436 10842 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:30.460238 10766 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:30.464919 12386 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:30.465152 12396 consensus_queue.cc:260] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:30.436038 10698 catalog_manager.cc:5697] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 1 to 2, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 2 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:30.467136 12386 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:30.467012 10698 catalog_manager.cc:5697] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 1 to 2, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 2 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:30.467900 12404 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:30.468457 12404 raft_consensus.cc:697] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:30.466514 12396 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:30.479656 10946 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 3.
I20260709 13:16:30.468539 10698 catalog_manager.cc:5697] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 1 to 2, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 2 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:30.483559 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:30.485456 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:30.491355 10841 leader_election.cc:400] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 election: Vote denied by peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553) with higher term. Message: Invalid argument: T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 3. Current term is 4.
I20260709 13:16:30.491829 10841 leader_election.cc:403] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:16:30.485483 10698 catalog_manager.cc:5697] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 1 to 2, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 2 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:16:30.431025 12471 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:30.498531 10698 catalog_manager.cc:5697] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 2 to 3, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 3 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
W20260709 13:16:30.499980 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:30.500332 10698 catalog_manager.cc:5697] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 1 to 2, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 2 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:16:30.501255 10947 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
I20260709 13:16:30.502810 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:30.466966 12356 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:30.503525 10944 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:30.505239 12356 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:30.505029 12477 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:30.486320 12404 consensus_queue.cc:237] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:30.488605 12386 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:30.511173 12386 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:30.512135 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:30.512326 12471 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:30.512784 10867 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:30.513345 12386 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:30.514530 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:30.496104 12395 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:30.515204 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:30.515806 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:30.516137 10874 raft_consensus.cc:2393] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 4: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:30.516975 12395 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:30.517140 12481 consensus_queue.cc:1048] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:16:30.517306 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:30.518301 12374 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:30.518379 10869 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 1.
I20260709 13:16:30.488327 12235 consensus_queue.cc:1048] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:16:30.519810 10875 raft_consensus.cc:1275] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:30.522107 10949 raft_consensus.cc:1240] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:30.498775 12437 raft_consensus.cc:2804] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:30.523171 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:30.497994 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:30.524194 10948 raft_consensus.cc:1240] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:30.498154 11478 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
W20260709 13:16:30.525108 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 45161.8us
I20260709 13:16:30.525211 12387 raft_consensus.cc:2749] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553) with higher term. Message: Invalid argument: T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 3. Current term is 4.
I20260709 13:16:30.526417 10841 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:30.526535 11478 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:16:30.527480 10867 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 4.
I20260709 13:16:30.527702 12235 raft_consensus.cc:2804] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:30.528118 12235 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:30.528192 10841 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:30.528568 12235 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:30.497273 11477 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:30.533792 12065 raft_consensus.cc:2749] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:30.534188 12477 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:30.515278 10876 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 2.
W20260709 13:16:30.539680 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 45620.9us
W20260709 13:16:30.538544 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 72960.2us
I20260709 13:16:30.547386 10947 raft_consensus.cc:1240] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:30.550304 10944 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
I20260709 13:16:30.551326 10766 leader_election.cc:400] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Vote denied by peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553) with higher term. Message: Invalid argument: T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 3. Current term is 4.
I20260709 13:16:30.551481 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:30.552006 10766 leader_election.cc:403] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:16:30.544641 11477 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:30.547029 12405 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:30.556857  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:30.569515 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:30.539335 12437 raft_consensus.cc:697] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:30.583771 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:30.545137 12451 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:30.572876 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:30.584674 12368 raft_consensus.cc:2749] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553) with higher term. Message: Invalid argument: T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 3. Current term is 4.
I20260709 13:16:30.587201 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:30.545063 12477 raft_consensus.cc:2749] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:30.591173 10799 raft_consensus.cc:2393] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 4: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:30.534876 12065 consensus_queue.cc:1059] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 13:16:30.615459 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:30.554302 12405 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:30.618300 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 77371.6us
I20260709 13:16:30.618448 12292 raft_consensus.cc:2804] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:30.619169 12494 consensus_queue.cc:1059] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:16:30.624274 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:30.516219 12313 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:30.622809 12451 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:30.627269 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:30.652187 10798 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2.
I20260709 13:16:30.627820 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:30.625026 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:30.571058 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:30.634832 11477 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:30.663203 10915 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:30.635633 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:30.641372 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:30.640462 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:30.670915 10799 raft_consensus.cc:2376] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2. Re-sending same reply.
I20260709 13:16:30.669960 10801 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 1.
I20260709 13:16:30.674185 10800 raft_consensus.cc:2410] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:16:30.671773 10797 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 1.
I20260709 13:16:30.689000 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:30.690184 10793 raft_consensus.cc:2410] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:16:30.717759 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:30.718924 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:30.721926 10947 raft_consensus.cc:2393] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 3: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:30.719216 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:30.723898 10801 raft_consensus.cc:2393] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 4: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
W20260709 13:16:30.725116 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 265735us
I20260709 13:16:30.671429 10795 raft_consensus.cc:2393] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 4: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:30.729549 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:30.651829 12412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:49274 (request call id 10) took 6151 ms. Trace:
I20260709 13:16:30.668015 12451 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:30.727200 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 199108us
I20260709 13:16:30.643038 12292 raft_consensus.cc:697] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 3 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:30.744020 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:30.744611 12292 consensus_queue.cc:237] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:30.670073 10941 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 4.
I20260709 13:16:30.746815 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:30.748771 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:30.719853 12483 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:30.749509 10874 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 3.
I20260709 13:16:30.750388 10948 raft_consensus.cc:1240] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 3. Current term is 4. Ops: []
W20260709 13:16:30.751041 12489 log.cc:927] Time spent T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 13:16:30.752795 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:30.753068 12492 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.753604 12492 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:30.754335 10945 raft_consensus.cc:1240] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:30.755899 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:30.634935 12358 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:30.783787 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:30.737270 10841 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:30.746951 12347 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 3. Current term is 4.
I20260709 13:16:30.784893 12358 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:30.736811 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:30.785941 10950 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 2: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:30.756851 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:30.759198 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:30.787062 10799 raft_consensus.cc:2368] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 2. Current term is 3.
W20260709 13:16:30.773949 12490 log.cc:927] Time spent T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:16:30.729445 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:30.755267 10867 raft_consensus.cc:2393] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 3: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:30.730207 12374 raft_consensus.cc:2804] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:16:30.779942 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:30.758419 12437 consensus_queue.cc:237] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:30.731045 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:30.783592 12451 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:30.761649 10886 tablet_copy_service.cc:140] P 468a8fd46e824f62b9c2c0d3532a6d26: Received BeginTabletCopySession request for tablet 2b82f18992434ed5ba58b805107060d4 from peer 12682eb70bc84c76879e04a8628ee586 ({username='slave'} at 127.0.0.1:49290)
I20260709 13:16:30.733593 12412 rpcz_store.cc:276] 0709 13:16:24.484714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:24.486101 (+  1387us) service_pool.cc:224] Handling call
0709 13:16:24.560282 (+ 74181us) log.cc:844] Serialized 506 byte log entry
0709 13:16:25.098164 (+537882us) spinlock_profiling.cc:243] Waited 1.89 ms on lock 0x7b18003593e0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7dd42c0a 00007f9c87482219 00007f9c7ddcaa40 00007f9c7ddc54e4 00007f9c7ddc5455 00007f9c80e0eb0b 00007f9c80e93b63 00007f9c80ebb752 00007f9c85e7bd29
0709 13:16:30.632201 (+5534037us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":104351,"raft.queue_time_us":62196,"raft.run_cpu_time_us":2100,"raft.run_wall_time_us":109253,"spinlock_wait_cycles":4356480,"thread_start_us":479434,"threads_started":1,"wal-append.queue_time_us":783285,"wal-append.run_cpu_time_us":16145,"wal-append.run_wall_time_us":1584915}
I20260709 13:16:30.789211 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:30.789278 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:16:30.791256 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 57016.8us
I20260709 13:16:30.853929 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:30.855211 10800 raft_consensus.cc:2393] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 3: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:30.859421 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:16:30.793180 12491 log.cc:927] Time spent T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.113s	user 0.002s	sys 0.000s
I20260709 13:16:30.793445 12405 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:30.793591 12381 raft_consensus.cc:3055] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:16:30.869755 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 236986us
I20260709 13:16:30.787743 12443 raft_consensus.cc:3055] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:30.785943 12235 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:30.796271 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:30.797714 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:30.823410 12374 raft_consensus.cc:697] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 2 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:30.823894 10872 raft_consensus.cc:1240] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:30.825634 12343 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:30.828917 10916 connection.cc:256] RPC call timeout handler was delayed by -1.3753352950000135s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1567s for a response
I20260709 13:16:30.862219 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:30.838881 10794 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 3.
I20260709 13:16:30.864190 10797 raft_consensus.cc:2393] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 4: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:30.936515 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:30.866128 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:30.868178 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:30.871598 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:16:30.876132 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 138997us
I20260709 13:16:30.855288 12477 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:30.867102 12488 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:30.929491 10916 connection.cc:256] RPC call timeout handler was delayed by -1.2574352950000502s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1698s for a response
W20260709 13:16:30.945463 10916 connection.cc:256] RPC call timeout handler was delayed by -1.2034352949999629s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1758s for a response
W20260709 13:16:30.830674 12484 log.cc:927] Time spent T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.119s	user 0.000s	sys 0.003s
I20260709 13:16:30.867233 10886 tablet_copy_service.cc:161] P 468a8fd46e824f62b9c2c0d3532a6d26: Beginning new tablet copy session on tablet 2b82f18992434ed5ba58b805107060d4 from peer 12682eb70bc84c76879e04a8628ee586 at {username='slave'} at 127.0.0.1:49290: session id = 12682eb70bc84c76879e04a8628ee586-2b82f18992434ed5ba58b805107060d4
I20260709 13:16:30.937448 10873 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:30.937964 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:30.869835 12381 raft_consensus.cc:740] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:30.950824 10799 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:30.875243 12443 raft_consensus.cc:740] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:16:30.945835 10916 connection.cc:256] RPC call timeout handler was delayed by -1.1791352949999236s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1785s for a response
I20260709 13:16:30.917649 10793 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 3.
I20260709 13:16:30.947855 10801 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:30.865123 12483 raft_consensus.cc:697] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 3 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:30.875900 12492 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:30.942517 10798 raft_consensus.cc:2393] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 4: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:30.953680 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:30.956811 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:30.957653 10793 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:30.921542 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:16:30.965215 10916 connection.cc:256] RPC call timeout handler was delayed by -1.1512352950001059s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1816s for a response
W20260709 13:16:30.965641 10916 connection.cc:256] RPC call timeout handler was delayed by -1.1512352950001059s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1816s for a response
W20260709 13:16:30.966035 10916 connection.cc:256] RPC call timeout handler was delayed by -1.1233352950000608s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1847s for a response
I20260709 13:16:30.961149 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:30.968443 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:30.961534 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:30.960952 10841 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:30.953604 10943 pending_rounds.cc:85] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 1
I20260709 13:16:30.990942 10943 pending_rounds.cc:107] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:16:30.991298 10943 raft_consensus.cc:2889] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:30.993965 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:30.994947 10798 raft_consensus.cc:2410] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:16:30.995471 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:30.995985 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:30.996762 10798 raft_consensus.cc:2393] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 4: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:30.994668 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:30.934219 12343 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:31.020131 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:31.017539 10870 raft_consensus.cc:2368] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 3. Current term is 4.
W20260709 13:16:30.987915 11478 leader_election.cc:336] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Timed out: RequestConsensusVote RPC to 127.9.116.129:39685 timed out after 1.758s (SENT)
I20260709 13:16:31.030634 10873 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 4.
I20260709 13:16:30.953455 12371 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:31.032738 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:30.952003 12488 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:30.995007 12381 consensus_queue.cc:260] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:31.108301 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:31.061841 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:16:31.096408 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 145303us
I20260709 13:16:31.100425 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:31.093405 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:31.037007 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:31.110036 10944 raft_consensus.cc:1240] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:31.077610 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:31.128021 10799 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 2.
I20260709 13:16:31.128070 10870 raft_consensus.cc:2410] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:16:31.047899 10801 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 2.
I20260709 13:16:31.133948 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:31.130090 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:16:31.078078 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 88154.4us
I20260709 13:16:31.126925 10793 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
I20260709 13:16:31.132467 10949 raft_consensus.cc:1240] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:31.132221 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:31.133049 10872 raft_consensus.cc:2393] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 4: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:16:31.089787 12500 log.cc:927] Time spent T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 13:16:30.996764 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:31.151902 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:31.135666 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:16:31.090745 12497 log.cc:927] Time spent T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:16:31.153266 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:16:31.062265 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 124187us
I20260709 13:16:31.099506 10874 raft_consensus.cc:2368] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 2. Current term is 3.
I20260709 13:16:31.091540 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:31.062163 10797 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:31.139041 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:31.141075 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:31.151489 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:31.077843 12477 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:31.135654 10946 raft_consensus.cc:1240] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:31.152405 10798 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:31.152626 10793 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:16:31.038760 12387 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:31.098893 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:31.139344 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:31.096064 12343 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:31.094419 12451 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:31.156069 10766 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:31.164709 10841 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:31.153759 12381 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:31.180451 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:31.078323 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:31.170050 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:31.170095 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:16:31.174667 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:31.179591 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:31.189179 11512 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 3.
I20260709 13:16:31.066438 12371 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:31.165431 10948 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
I20260709 13:16:31.156985 12477 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:31.199064 12313 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:31.097151 12443 consensus_queue.cc:260] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:31.187417 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 55020.2us
I20260709 13:16:31.175455 11507 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 1.
I20260709 13:16:31.186844 10800 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 4.
W20260709 13:16:31.191900 12495 log.cc:927] Time spent T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
I20260709 13:16:31.195184 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:31.195384 10799 raft_consensus.cc:1240] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:16:31.182626 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:31.196135 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:31.207775 10841 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:31.201100 10872 raft_consensus.cc:2393] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 4: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:16:31.202889 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:31.205721 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:31.208662 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:31.166939 12344 consensus_queue.cc:1048] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 13:16:31.133471 12374 consensus_queue.cc:237] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:31.217274 10876 raft_consensus.cc:2393] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 3: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:16:31.222765 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 44581.3us
I20260709 13:16:31.225502 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:31.199545 12477 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:31.231153 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:16:31.237480 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:31.225544 10793 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 3.
I20260709 13:16:31.240672 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:31.182509 12387 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:31.243268 10793 raft_consensus.cc:1275] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:16:31.231012 12356 consensus_queue.cc:1059] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 13:16:31.217295 12368 raft_consensus.cc:2804] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:31.156682 12483 consensus_queue.cc:237] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:31.229830 12376 raft_consensus.cc:2804] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:31.233551 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:31.229669 12412 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:31.259997 12358 raft_consensus.cc:3055] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:31.260181 12412 raft_consensus.cc:697] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:31.260484 12358 raft_consensus.cc:740] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:31.263311 12358 consensus_queue.cc:260] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:31.264796 12358 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:31.266332 12371 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:16:31.268882 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:31.269153 12488 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:31.258069 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:31.234508 12235 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:31.263314 10792 raft_consensus.cc:1275] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:31.261248 11505 raft_consensus.cc:1240] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:31.276564 10798 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 3.
I20260709 13:16:31.250826 10793 raft_consensus.cc:1240] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:31.261231 12412 consensus_queue.cc:237] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:31.232169 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 43965us
I20260709 13:16:31.275790 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:31.224048 12494 consensus_queue.cc:1059] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:16:31.279574 10766 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:31.285614 10800 raft_consensus.cc:1240] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:31.204805 12313 consensus_queue.cc:1059] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:16:31.286890 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:16:31.300593 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 92894.9us
I20260709 13:16:31.300978 11512 raft_consensus.cc:2393] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 2: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:31.224370 12065 consensus_queue.cc:1059] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 13:16:31.219441 12499 consensus_queue.cc:1059] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 13:16:31.256124 11508 raft_consensus.cc:2393] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 4: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:31.245772 12386 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:31.297480 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:31.295982 10871 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 1.
I20260709 13:16:31.230777 12396 consensus_queue.cc:1059] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:16:31.248606 12443 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:31.328269 11477 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:31.305011 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:31.308266 11503 raft_consensus.cc:1240] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:31.335651 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:31.251802 10799 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c"
dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
mode: GRACEFUL
new_leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:16:31.309490 10948 raft_consensus.cc:1275] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:16:31.336519 10799 raft_consensus.cc:606] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Received request to transfer leadership to TS 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:31.336898 10799 raft_consensus.cc:614] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:31.309557 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:31.311378 10801 raft_consensus.cc:1240] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:16:31.337670 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:31.311873 10946 raft_consensus.cc:1275] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:16:31.329555 10916 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:31.322116 10793 raft_consensus.cc:1240] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:31.323072 11507 raft_consensus.cc:1275] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:16:31.283707 12368 raft_consensus.cc:697] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:31.293345 12376 raft_consensus.cc:697] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:31.335081 12356 raft_consensus.cc:3055] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:31.348918 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:31.326836 10942 raft_consensus.cc:1240] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 2 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:31.327387 12309 tablet_copy_client.cc:538] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:31.330837 11504 raft_consensus.cc:2393] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 2: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:31.332578 11478 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:31.309128 10876 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8cab289810294719ac9686e40290a6ab"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
 from {username='slave'} at 127.0.0.1:54948
I20260709 13:16:31.398356 10876 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:31.398744 10876 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:31.325249 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:31.347146 12471 raft_consensus.cc:2804] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:31.399523 10869 raft_consensus.cc:2393] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 3: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:31.347803 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:31.329797 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:31.347638 12368 consensus_queue.cc:237] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:31.342837 12235 raft_consensus.cc:2749] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:31.409762 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:31.410975 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:31.411118 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:31.411808 10874 raft_consensus.cc:2368] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 2. Current term is 3.
I20260709 13:16:31.412901 10874 raft_consensus.cc:1240] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:31.413636 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:31.387477 10943 raft_consensus.cc:1275] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:31.379739 12520 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:31.390588 12356 raft_consensus.cc:740] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:16:31.375700 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 46221.5us
I20260709 13:16:31.423020 10916 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:31.341563 12395 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:31.423668 12235 raft_consensus.cc:2749] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:31.424041 12395 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:31.424489 12395 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:31.325395 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:31.425765 10870 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 2.
I20260709 13:16:31.384253 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:31.389288 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:31.348316 12396 raft_consensus.cc:3055] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:31.424501 12371 raft_consensus.cc:2749] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:31.427184 12396 raft_consensus.cc:740] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:31.426652 10945 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:31.427775 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:31.428419 10869 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 2.
I20260709 13:16:31.428108 12396 consensus_queue.cc:260] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:31.387434 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:31.391726 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:31.391960 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:31.396809 10797 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 2.
I20260709 13:16:31.373399 12405 raft_consensus.cc:3055] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:31.325799 12386 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:31.404368 10941 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:31.349099 12376 consensus_queue.cc:237] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:31.400394 12471 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:31.408136 10868 raft_consensus.cc:1240] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:31.419869 10792 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2"
dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
mode: GRACEFUL
new_leader_uuid: "12682eb70bc84c76879e04a8628ee586"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:16:31.441201 11512 raft_consensus.cc:1275] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:16:31.441293 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 108817us
I20260709 13:16:31.441608 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:31.445135 10871 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:31.375317 10801 raft_consensus.cc:1240] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:31.411552 10872 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
I20260709 13:16:31.443702 10876 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:31.431044 10798 raft_consensus.cc:1240] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:31.439612 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:31.448808 10942 raft_consensus.cc:2393] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 3: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:31.436360 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:31.453182 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:31.458055 10868 raft_consensus.cc:2393] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 5: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:31.459103 10948 raft_consensus.cc:1240] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:31.463279 11478 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:31.436596 12405 raft_consensus.cc:740] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:31.440754 12471 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:31.475270 10946 raft_consensus.cc:1275] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:16:31.453756 12396 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:31.477416 10876 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:31.483445 10945 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 3.
I20260709 13:16:31.467674 12404 raft_consensus.cc:3055] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:31.429262 12504 consensus_queue.cc:1048] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 13:16:31.467994 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:31.485327 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:16:31.491286 12516 log.cc:927] Time spent T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 13:16:31.486572 11478 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:31.429435 12520 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:31.499683 11478 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:31.500074 10875 raft_consensus.cc:2410] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:16:31.500921 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:31.501138 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:31.501647 10869 raft_consensus.cc:2393] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 4: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:31.502203 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:31.502353 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:31.503571 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:31.440876 12508 consensus_queue.cc:1048] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 13:16:31.506042 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:31.495780 10867 raft_consensus.cc:2393] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 4: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:31.480923 12235 raft_consensus.cc:2749] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:31.483217 12387 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:31.519146 10869 raft_consensus.cc:1275] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:16:31.486215 12404 raft_consensus.cc:740] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:31.517131 12520 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:31.453178 12356 consensus_queue.cc:260] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:31.509023 12483 raft_consensus.cc:2749] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:31.533879 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:31.506456 12494 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:31.517879 12451 raft_consensus.cc:2804] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:31.550772 11478 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:31.551986 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:31.556878 10841 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:31.560664 10875 raft_consensus.cc:2393] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 3: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:31.533180 10874 raft_consensus.cc:2393] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 2: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:31.560763 10869 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
I20260709 13:16:31.525414 12493 raft_consensus.cc:3055] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:31.578226 10870 raft_consensus.cc:1275] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:31.579334 10916 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:31.581411 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:31.567736 12356 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:31.576447 10868 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 2.
I20260709 13:16:31.577762 12494 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:31.557927 12451 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:31.592445 10875 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 4.
I20260709 13:16:31.594463 11478 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:31.596064 12531 consensus_queue.cc:1048] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:16:31.603295 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 46516.3us
I20260709 13:16:31.578950 12235 raft_consensus.cc:2749] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:16:31.577875 12493 raft_consensus.cc:740] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:31.525357 12405 consensus_queue.cc:260] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:31.583809 12358 raft_consensus.cc:2749] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:31.555058 12289 consensus_queue.cc:1059] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:16:31.613186 10841 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:31.603045 12451 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:16:31.618074 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:31.606391 12374 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:31.608259 12386 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:31.625015 12376 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:31.624385 10841 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:16:31.625989 12376 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:31.626524 12376 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:31.633204 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 53996.4us
W20260709 13:16:31.633296 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:31.636921 10841 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:31.618651 12438 raft_consensus.cc:2749] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:16:31.638254 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 43873us
I20260709 13:16:31.640823 11478 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:16:31.647404 12522 log.cc:927] Time spent T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
I20260709 13:16:31.636739 12289 consensus_queue.cc:1243] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer cb0428cc55984daba1ba63744a5102a2 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:16:31.651718 12494 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:31.653568 12309 tablet_bootstrap.cc:492] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586: Bootstrap starting.
I20260709 13:16:31.623898 12374 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:31.670070 10945 raft_consensus.cc:1240] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:31.639528 12358 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:31.684451 12358 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:31.684662 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:31.684880 12358 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:31.685410 11503 raft_consensus.cc:2368] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 2. Current term is 3.
I20260709 13:16:31.686337 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:31.686496 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:31.687353 11508 raft_consensus.cc:1240] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:31.687613 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:31.677525 12520 consensus_queue.cc:1048] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:31.687987 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:31.679082 12527 consensus_queue.cc:1059] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:31.676440 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:31.689769 11508 raft_consensus.cc:1240] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 3 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:31.690203 11505 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 3.
I20260709 13:16:31.689846 12493 consensus_queue.cc:260] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:31.689838 12527 raft_consensus.cc:3055] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:31.691258 12493 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:31.691362 12527 raft_consensus.cc:740] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:31.680580 12530 consensus_queue.cc:1048] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:16:31.692560 12527 consensus_queue.cc:260] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:31.694412 12527 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:31.681175 12499 consensus_queue.cc:1059] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:31.682240 12353 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:31.695865 12353 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:31.682727 12508 raft_consensus.cc:2749] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:31.683842 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:31.699713 10841 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:31.700168 10945 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:31.667339 12374 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:31.696280 11510 raft_consensus.cc:1240] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:31.683080 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:31.654672 12386 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:31.694443 11508 raft_consensus.cc:1275] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:16:31.702613 10946 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:31.680243 11512 raft_consensus.cc:1240] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:31.705054 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:31.705394 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:31.706745 10875 raft_consensus.cc:1275] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:16:31.707835 12532 raft_consensus.cc:3055] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:31.716956 10872 raft_consensus.cc:1240] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:31.717129 12532 raft_consensus.cc:740] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:31.710320 10871 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 4.
I20260709 13:16:31.722642 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:16:31.757778 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 56165.7us
W20260709 13:16:31.738482 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:31.757205 10871 raft_consensus.cc:1275] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:31.759838 11511 raft_consensus.cc:1240] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:31.757141 10943 raft_consensus.cc:1275] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:16:31.748791 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:31.673321 10941 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 2.
I20260709 13:16:31.762667 10801 raft_consensus.cc:1275] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 13:16:31.763011 10800 raft_consensus.cc:2393] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 4: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:31.763159 10801 pending_rounds.cc:85] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 4
I20260709 13:16:31.763747 10801 pending_rounds.cc:107] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:16:31.764003 10801 raft_consensus.cc:2889] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:31.701761 11505 raft_consensus.cc:1240] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 3 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:31.765636 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:31.699976 12499 raft_consensus.cc:3055] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:31.767791 10949 raft_consensus.cc:1240] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:31.732455 10841 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:31.772845 11478 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:31.718580 10792 raft_consensus.cc:606] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Received request to transfer leadership to TS 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:31.720690 10886 tablet_copy_source_session.cc:215] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:31.781018 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:31.781674 10886 rpcz_store.cc:275] Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.0.0.1:49290 (request call id 149) took 1158 ms. Trace:
I20260709 13:16:31.778229 10799 raft_consensus.cc:1275] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:16:31.757241 12530 consensus_queue.cc:1059] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 13:16:31.683917 12503 consensus_queue.cc:1048] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.083s
I20260709 13:16:31.718590 12531 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:31.715901 12520 raft_consensus.cc:2804] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:31.770411 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:31.783479 10916 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:31.825346 11478 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:16:31.826535 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 94187.6us
I20260709 13:16:31.829908 10841 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:16:31.830587 10946 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:31.782289 10886 rpcz_store.cc:276] 0709 13:16:30.634725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:30.733208 (+ 98483us) service_pool.cc:224] Handling call
0709 13:16:31.710557 (+977349us) spinlock_profiling.cc:243] Waited 293 ms on lock 0x7b5400078354. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80ebf3f7 00007f9c85e55036 00007f9c85e5ea3e 00007f9c7d72227b 00007f9c85e5e9ba 00007f9c85e59acc 00007f9c85e54c7b 00007f9c85e498d1 00007f9c84efd4b4 00007f9c84efd442
0709 13:16:31.781549 (+ 70992us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":672933120}
I20260709 13:16:31.833366 10873 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 2.
I20260709 13:16:31.827322 10800 raft_consensus.cc:2376] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 3. Re-sending same reply.
I20260709 13:16:31.838055 12507 raft_consensus.cc:2749] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:31.839857 10943 raft_consensus.cc:1275] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:16:31.841513 12477 consensus_meta.cc:287] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.597s	user 0.009s	sys 0.008s
I20260709 13:16:31.841977 12477 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:31.774667 12499 raft_consensus.cc:740] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:31.792721 10792 raft_consensus.cc:614] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:31.864070 12547 raft_consensus.cc:2749] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:31.732684 12471 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:31.897918 12289 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:31.902233 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 141362us
I20260709 13:16:31.903731 10766 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:31.731477 12532 consensus_queue.cc:260] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:31.853658 12437 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:16:31.907316 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:31.896827 12344 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:31.930106 12520 raft_consensus.cc:697] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:31.849387 12395 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:31.934746 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 158311us
I20260709 13:16:31.857136 12494 consensus_queue.cc:1059] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 13:16:31.910120 12437 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:31.844547 12531 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:31.939430 12371 consensus_queue.cc:1048] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:31.906550 12343 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:31.946765 12343 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:31.899413 12065 consensus_queue.cc:1048] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 13:16:31.915771 12504 raft_consensus.cc:2749] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:31.922371 12353 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:31.896762 12443 consensus_queue.cc:1059] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:16:31.922371 12344 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:31.909147 12289 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:31.944679 10916 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:31.946170 12437 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:16:31.962491 12540 log.cc:927] Time spent T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:16:31.953296 12344 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:31.953649 12532 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:31.935837 12376 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:31.858814 12499 consensus_queue.cc:260] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:31.957413 12368 raft_consensus.cc:2804] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:31.979168 10916 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:31.992241 10941 raft_consensus.cc:1240] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:31.967813 12343 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:31.959081 12477 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:32.019193 12494 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:32.019867 12538 log.cc:927] Time spent T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.197s	user 0.001s	sys 0.000s
I20260709 13:16:32.019920 12494 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:32.016229 12471 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:31.968601 12371 raft_consensus.cc:2764] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:16:31.993136 12546 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:32.029994 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 204716us
I20260709 13:16:32.037320 11478 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:16:32.036237 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 72564.5us
I20260709 13:16:32.049045 10916 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:31.974606 12547 consensus_queue.cc:1048] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.054s
W20260709 13:16:32.051316 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 207164us
I20260709 13:16:32.027158 12368 raft_consensus.cc:697] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:16:32.056208 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 152604us
I20260709 13:16:32.058148 10944 raft_consensus.cc:1240] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:32.061451 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:32.061600 12371 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:32.053380 12343 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:32.054864 12292 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:31.992874 12539 consensus_queue.cc:1059] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.187s
I20260709 13:16:32.071455 10944 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 4.
I20260709 13:16:32.052022 12374 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:32.082769 12289 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:32.025731 12520 consensus_queue.cc:237] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:32.063438 12371 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:32.069380 12292 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:32.098713 10948 raft_consensus.cc:1240] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:32.036578 12546 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:32.094926 12395 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:32.101972 12539 consensus_queue.cc:1059] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:32.102308 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:32.102929 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:32.103289 12539 raft_consensus.cc:3055] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:32.103792 10876 raft_consensus.cc:2410] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:16:32.103775 12539 raft_consensus.cc:740] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:32.105015 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:32.104858 12539 consensus_queue.cc:260] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:32.105835 10872 raft_consensus.cc:2410] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:16:32.106257 12539 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:32.108095 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:32.108991 10876 raft_consensus.cc:2393] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 3: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:32.109835 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:32.110800 10874 raft_consensus.cc:2410] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 2 because replica has last-logged OpId of term: 1 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:16:32.111526 12386 raft_consensus.cc:2749] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:32.111654 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:16:32.112747 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 258209us
I20260709 13:16:32.113466 10841 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:32.114259 12412 raft_consensus.cc:2804] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:32.114758 12412 raft_consensus.cc:697] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:32.115711 12412 consensus_queue.cc:237] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:32.116449 12451 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:32.083982 12343 raft_consensus.cc:697] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 4 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:32.094288 12292 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:32.094907 12371 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:16:32.122030 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:32.125639 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:16:32.104051 12548 log.cc:927] Time spent T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 13:16:32.124591 10868 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2.
I20260709 13:16:32.098363 12550 raft_consensus.cc:3055] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:16:32.139396 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:32.118728 12451 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:32.152267 12358 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:32.153774 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:32.155591 12437 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:32.175938 12437 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:32.158496 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:32.159334 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:32.176970 10867 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 2.
I20260709 13:16:32.177403 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:32.178073 10875 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:32.178189 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:32.179024 10867 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
I20260709 13:16:32.180091 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:16:32.168375 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:32.168408 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:32.125648 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:32.131285 12368 consensus_queue.cc:237] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:32.135130 12550 raft_consensus.cc:740] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:32.188347 10867 raft_consensus.cc:2393] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 2: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:32.188688 12550 consensus_queue.cc:260] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:32.189316 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:32.190045 12550 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:32.172748 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:32.153831 12343 consensus_queue.cc:237] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:32.131388 12386 consensus_queue.cc:1048] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 13:16:32.167384 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:32.180518 12546 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:32.192061 10876 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 1.
I20260709 13:16:32.146821 10916 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:32.191201 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:32.173424 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:32.191537 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:32.190157 10867 raft_consensus.cc:2393] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 4: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
W20260709 13:16:32.196718 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:32.228873 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:32.233817 10794 raft_consensus.cc:1240] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:32.181710 10872 raft_consensus.cc:2368] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 3. Current term is 4.
I20260709 13:16:32.252080 10876 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 2.
I20260709 13:16:32.213181 10800 raft_consensus.cc:1240] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:32.253564 10799 raft_consensus.cc:1240] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 3. Current term is 4. Ops: []
W20260709 13:16:32.196913 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:32.239908 10767 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.116.130:37895 timed out after 2.248s (SENT) [suppressed 40 similar messages]
I20260709 13:16:32.262229 10875 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 4.
I20260709 13:16:32.263224 10767 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:32.264384 10876 raft_consensus.cc:1275] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:16:32.267392 10797 raft_consensus.cc:1275] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:32.240477 12374 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:32.154074 12344 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:32.183423 12405 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:32.237032 10795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 1608) took 1385 ms. Trace:
I20260709 13:16:32.212805 12477 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
W20260709 13:16:32.295403 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 152175us
I20260709 13:16:32.192586 12376 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:32.184497 12493 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:32.263906 12546 raft_consensus.cc:2749] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:32.192270 10874 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 2.
I20260709 13:16:32.342588 10916 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:32.344798 10871 raft_consensus.cc:1275] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:16:32.237246 12292 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:32.348388 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:32.350142 10869 raft_consensus.cc:2393] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 4: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:32.337409 12395 raft_consensus.cc:2749] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:32.356695 10916 leader_election.cc:400] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 election: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 3. Current term is 4.
I20260709 13:16:32.357039 10916 leader_election.cc:403] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:16:32.341943 12494 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:32.362505 10916 leader_election.cc:400] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 election: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 2. Current term is 3.
I20260709 13:16:32.364094 10916 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:32.365123 12471 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:32.369972 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:32.372768 12547 raft_consensus.cc:2749] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:32.373319 12519 raft_consensus.cc:3055] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:32.373772 12519 raft_consensus.cc:740] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:32.373636 12471 raft_consensus.cc:697] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 4 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:16:32.374495 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:32.370635 10876 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 4.
I20260709 13:16:32.380640 10870 raft_consensus.cc:1275] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:16:32.346762 12477 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:32.354183 10867 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:16:32.387354 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:32.394539 12552 raft_consensus.cc:3055] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:32.395023 12552 raft_consensus.cc:740] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:32.395864 12552 consensus_queue.cc:260] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:32.395880 12519 consensus_queue.cc:260] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:32.397584 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:32.424001 10794 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 4.
I20260709 13:16:32.395969 10799 raft_consensus.cc:1275] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:32.351552 12493 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:32.439297 12358 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:32.440071 12344 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:32.402688 10798 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2.
I20260709 13:16:32.376813 12404 consensus_queue.cc:260] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:32.441540 12404 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:32.375640 12546 raft_consensus.cc:2749] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 3. Current term is 4.
I20260709 13:16:32.443503 10801 raft_consensus.cc:1275] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:16:32.446952 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 388616us
I20260709 13:16:32.447036 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:32.447930 10794 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 3.
I20260709 13:16:32.339269 10795 rpcz_store.cc:276] 0709 13:16:30.851780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:30.960818 (+109038us) service_pool.cc:224] Handling call
0709 13:16:32.236789 (+1275971us) spinlock_profiling.cc:243] Waited 586 ms on lock 0x7b54000bccd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:32.236954 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1348364160}
W20260709 13:16:32.451509 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 60379us
I20260709 13:16:32.457827 12552 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:32.440317 12493 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:16:32.465904 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:32.466354 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:32.459074 10797 raft_consensus.cc:1275] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:16:32.459199 10767 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:32.446578 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:16:32.473322 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:32.468041 10767 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:32.466990 10794 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 3.
I20260709 13:16:32.401928 12395 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:32.466960 10842 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:32.468416 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:32.479417 12358 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:32.489642 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:32.490455 10797 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:32.492760 10842 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:32.497758 10842 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:32.465945 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:32.475332 12519 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:32.469551 12494 raft_consensus.cc:2749] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:32.475210 12376 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:32.503036 10767 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:16:32.499730 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 439742us
I20260709 13:16:32.504966 10916 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:32.506145 10842 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:32.508340 10842 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:32.465114 12532 consensus_queue.cc:1059] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 13:16:32.510470 10842 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:16:32.510869 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 47419.7us
I20260709 13:16:32.497354 12493 raft_consensus.cc:2749] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:32.499462 12344 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:32.526957 10797 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
I20260709 13:16:32.528384 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:32.536875 10766 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:32.537142 10916 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:32.515694 10871 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 3.
I20260709 13:16:32.514823 10842 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:32.538766 10767 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:32.521405 12344 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:32.511252 12235 consensus_queue.cc:1048] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:16:32.543519 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:32.512684 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:32.492431 12395 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:32.536224 12568 raft_consensus.cc:2749] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:32.471071 12471 consensus_queue.cc:237] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:32.558916 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:32.559391 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:32.559993 10949 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:32.545420 11511 raft_consensus.cc:1240] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:32.546154 10842 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:32.509374 12412 raft_consensus.cc:2749] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:16:32.547354 12546 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:32.512353 12443 raft_consensus.cc:2749] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:32.548909 10916 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:32.571727 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:32.508204 12343 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:32.547168 12492 consensus_queue.cc:1048] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:16:32.549217 12437 consensus_queue.cc:1048] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 13:16:32.550999 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:32.585192 10916 leader_election.cc:400] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 election: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 2. Current term is 3.
I20260709 13:16:32.585546 10916 leader_election.cc:403] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
W20260709 13:16:32.551589 12553 log.cc:927] Time spent T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.214s	user 0.001s	sys 0.000s
I20260709 13:16:32.552934 12565 raft_consensus.cc:2749] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:32.553144 12477 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:32.545305 12493 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:16:32.553768 12561 log.cc:927] Time spent T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
I20260709 13:16:32.510988 12503 consensus_queue.cc:1048] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:16:32.539366 12289 raft_consensus.cc:2749] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:32.525032 12527 raft_consensus.cc:2749] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:16:32.556272 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:32.503954 12358 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:32.555095 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:32.560201 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:32.562264 10767 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:32.563102 12574 consensus_queue.cc:1048] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:16:32.519559 12374 raft_consensus.cc:2764] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:16:32.510807 12376 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:32.615624 12358 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:32.616942 12376 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:32.554735 11504 raft_consensus.cc:1275] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:32.617266 12531 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:32.548456 12344 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:32.618599 10796 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2.
W20260709 13:16:32.618775 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 56598.1us
I20260709 13:16:32.620419 10767 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:32.565644 12563 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:32.567600 10795 raft_consensus.cc:1275] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:32.573796 12555 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:32.575505 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:32.622748 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:32.622824 12289 raft_consensus.cc:2764] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:16:32.622824 12493 consensus_queue.cc:1059] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:32.623369 10869 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 3.
I20260709 13:16:32.623695 12289 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:32.624074 12493 raft_consensus.cc:3055] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:32.624219 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:32.624480 12493 raft_consensus.cc:740] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:32.624883 10869 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 1.
I20260709 13:16:32.625511 12448 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b82f18992434ed5ba58b805107060d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:32.577881 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 1204) took 1424 ms. Trace:
I20260709 13:16:32.536780 12451 consensus_queue.cc:1048] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:16:32.576352 12569 consensus_queue.cc:1059] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.057s
I20260709 13:16:32.626125 10873 rpcz_store.cc:276] 0709 13:16:31.153772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:31.153940 (+   168us) service_pool.cc:224] Handling call
0709 13:16:32.577699 (+1423759us) spinlock_profiling.cc:243] Waited 644 ms on lock 0x7b54000983d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:32.577819 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1482272000}
I20260709 13:16:32.561331 12381 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:32.634945 12381 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:32.625386 12493 consensus_queue.cc:260] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:32.635438 12381 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:32.635908 12493 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:32.636327 12555 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:32.636756 12555 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:32.552073 12292 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:32.559471 12387 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:32.588620 11478 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:32.609395 10798 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 3.
I20260709 13:16:32.621002 10793 raft_consensus.cc:1275] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:16:32.621467 10767 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:32.564858 12313 consensus_queue.cc:1059] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 13:16:32.627002 10871 raft_consensus.cc:2393] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 4: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:32.621544 12563 raft_consensus.cc:697] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:32.599761 12539 raft_consensus.cc:2804] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:32.586973 10916 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:32.572183 12371 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:32.642402 10842 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:32.644794 10842 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:32.645560 10842 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:32.646083 10767 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:32.618630 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:32.648917 12573 raft_consensus.cc:2749] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 2. Current term is 3.
I20260709 13:16:32.649158 12477 raft_consensus.cc:2749] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:32.649381 12471 raft_consensus.cc:2749] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:32.589098 12343 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:32.653450 10875 raft_consensus.cc:1240] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:32.655056 10793 raft_consensus.cc:1275] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:16:32.655418 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 68504.5us
I20260709 13:16:32.657330 10916 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:32.642757 12539 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:32.647339 12443 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:32.658213 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:32.650004 12358 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:32.648944 12451 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:32.641974 12527 consensus_queue.cc:1048] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 13:16:32.670225 10948 raft_consensus.cc:1275] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:16:32.670512 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:32.662307 10798 raft_consensus.cc:1275] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 13:16:32.667723 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:32.663214 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:32.661536 10767 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:32.675068 10868 raft_consensus.cc:1240] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:32.644335 12376 consensus_queue.cc:1059] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 13:16:32.656458 12289 raft_consensus.cc:2749] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:32.642115 12387 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:32.668825 10916 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:32.676848 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:16:32.677843 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 89320.7us
I20260709 13:16:32.660234 12562 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:32.681432 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:32.691470 10942 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
I20260709 13:16:32.681529 12371 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:32.682348 12395 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:32.682605 12405 raft_consensus.cc:2749] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:32.683463 10799 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 3.
I20260709 13:16:32.663712 12539 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:32.660663 12488 consensus_queue.cc:1048] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 13:16:32.680382 10767 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:32.658703 12531 consensus_queue.cc:1059] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:16:32.684590 10867 raft_consensus.cc:606] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 LEADER]: Received request to transfer leadership to TS cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:32.654808 12565 consensus_queue.cc:1048] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:16:32.658595 12566 consensus_queue.cc:1048] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.087s
I20260709 13:16:32.662011 12374 raft_consensus.cc:2749] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:16:32.709179 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:32.688985 10793 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:32.695829 10766 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:32.692673 11478 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:16:32.698608 10767 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:32.699350 10916 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:32.700196 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:32.699998 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:32.713416 10868 raft_consensus.cc:2393] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 5: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:32.704350 10942 raft_consensus.cc:1240] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:32.715885 11478 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:32.674856 12404 consensus_queue.cc:1048] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 13:16:32.693401 12530 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:32.694168 12358 raft_consensus.cc:697] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:32.693720 12451 raft_consensus.cc:697] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 3 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:32.738579 10766 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:32.760883 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:32.725598 10916 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:32.702505 12573 raft_consensus.cc:2749] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:32.700523 12395 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:32.766484 10869 raft_consensus.cc:2393] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 4: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:16:32.769804 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 44307.5us
W20260709 13:16:32.774606 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 76058.6us
I20260709 13:16:32.775708 10767 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:32.720877 12508 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:32.722738 12503 raft_consensus.cc:2804] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:32.771718 12374 raft_consensus.cc:2749] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:32.792347 10793 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
I20260709 13:16:32.756915 12547 raft_consensus.cc:2749] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:16:32.778133 12550 consensus_meta.cc:287] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.588s	user 0.010s	sys 0.004s
I20260709 13:16:32.763228 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:32.784821 12396 raft_consensus.cc:2749] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:32.723210 12551 consensus_queue.cc:1048] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 13:16:32.805299 10916 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:16:32.806770 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 84826.9us
I20260709 13:16:32.814935 10916 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:32.815624 10842 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:32.815956 10916 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:32.766808 12494 raft_consensus.cc:2749] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:32.717893 12471 consensus_queue.cc:1059] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.086s
I20260709 13:16:32.778383 12531 raft_consensus.cc:2804] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:32.810130 10767 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:16:32.818907 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 80436.9us
I20260709 13:16:32.760583 12557 consensus_queue.cc:1059] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:16:32.821699 10916 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:32.794147 12508 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:32.701392 12563 consensus_queue.cc:237] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:32.784962 12376 consensus_queue.cc:1048] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 13:16:32.809206 12503 raft_consensus.cc:697] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 2 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:32.844125 12503 consensus_queue.cc:237] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:32.846002 10797 raft_consensus.cc:3055] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:32.846236 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:32.846442 10797 raft_consensus.cc:740] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:32.836773 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:32.851851 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:32.809841 12395 raft_consensus.cc:2804] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:32.842227 10945 raft_consensus.cc:1240] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:32.846899 10944 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:16:32.847524 10947 raft_consensus.cc:1275] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:16:32.824384 12531 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:32.824369 12374 raft_consensus.cc:2804] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:32.763542 12530 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:32.818414 12371 raft_consensus.cc:2749] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:32.852408 10949 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 4.
I20260709 13:16:32.867890 10797 consensus_queue.cc:260] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:32.852229 10767 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:32.820145 12552 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:32.865677 10947 raft_consensus.cc:1275] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:16:32.866961 10945 raft_consensus.cc:1275] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:16:32.845642 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:32.865046 12395 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
W20260709 13:16:32.873849 12576 log.cc:927] Time spent T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 13:16:32.874500 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:32.824570 12358 consensus_queue.cc:237] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:32.875883 12381 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:32.828557 12471 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:32.827504 12508 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:32.825366 12550 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:32.761965 12577 consensus_queue.cc:1048] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.095s
I20260709 13:16:32.909415 12387 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:32.902666 12471 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:32.910674 12404 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:32.911314 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:32.911417 12404 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:32.912539 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:32.913122 10794 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
I20260709 13:16:32.913609 12404 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:32.898497 10797 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:32.904068 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 51936.7us
I20260709 13:16:32.893972 12395 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:32.910897 12471 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:32.883625 12531 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:32.916085 10767 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:32.887512 12583 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:32.916635 10842 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:32.906546 12552 raft_consensus.cc:697] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:16:32.917835 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 65960.4us
I20260709 13:16:32.934482 10916 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:32.892761 12396 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:32.931488 10944 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
I20260709 13:16:32.887089 12344 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:32.935824 10797 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:32.835781 12451 consensus_queue.cc:237] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:32.907186 12405 raft_consensus.cc:2749] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:32.940409 10794 raft_consensus.cc:1275] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:32.943034 10767 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:32.948940 10767 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:16:32.950806 12581 log.cc:927] Time spent T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:16:32.903319 12532 consensus_queue.cc:1059] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 13:16:32.936813 10941 raft_consensus.cc:1275] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:16:32.954174 10873 raft_consensus.cc:1240] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:16:32.936133 12583 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:32.940845 12376 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:32.928874 12550 raft_consensus.cc:697] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 2 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:32.956331 10916 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:32.939100 12404 raft_consensus.cc:2749] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:16:32.952502 12569 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 13:16:32.954280 12289 raft_consensus.cc:2804] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:32.963271 10767 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:32.941738 12396 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:32.969353 12371 raft_consensus.cc:2804] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:32.973966 10767 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:32.957352 10874 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 2.
I20260709 13:16:32.959604 12376 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:32.985844 12404 raft_consensus.cc:2804] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:32.986282 12404 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:32.986689 12404 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:32.975003 12405 raft_consensus.cc:2749] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:32.976279 12494 raft_consensus.cc:3055] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:32.960011 12583 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:32.974339 12289 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:32.988968 12376 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:32.958264 12585 consensus_queue.cc:1059] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
I20260709 13:16:32.994958 12589 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:16:32.998713 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:32.997610 12580 log.cc:927] Time spent T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 13:16:33.001950 12589 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:33.003621 12589 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:33.023484 12585 consensus_queue.cc:1048] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:33.050009 12590 raft_consensus.cc:3055] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:33.050498 12590 raft_consensus.cc:740] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:33.050697 12344 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:33.051535 12590 consensus_queue.cc:260] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:33.052845 12590 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:32.993907 12494 raft_consensus.cc:740] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:32.997977 12235 raft_consensus.cc:3055] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:16:33.065505 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:33.001626 12289 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:32.983357 12552 consensus_queue.cc:237] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:33.068111 12492 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:33.078730 10946 raft_consensus.cc:1240] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:33.080310 10941 raft_consensus.cc:1275] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:16:33.065377 12235 raft_consensus.cc:740] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:33.089094 12597 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:33.089576 12589 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:33.083129 12492 raft_consensus.cc:697] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:33.110822 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:33.080650 12494 consensus_queue.cc:260] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:33.078863 12530 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:33.111701 10799 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:33.102037 12381 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:33.134157 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 162326us
I20260709 13:16:33.132323 12235 consensus_queue.cc:260] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:33.133047 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:33.136101 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:33.073464 12550 consensus_queue.cc:237] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:33.148500 10916 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:33.089907 12597 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:33.162463 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:33.137312 12396 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:16:33.148420 12555 consensus_meta.cc:287] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.511s	user 0.006s	sys 0.006s
I20260709 13:16:33.161746 12539 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:33.168123 10867 raft_consensus.cc:1240] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:33.168022 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:33.147086 12235 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:33.171319 12492 consensus_queue.cc:237] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:33.179190 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:33.180766 10942 raft_consensus.cc:1275] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:33.193717 12530 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:33.194456 12530 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:33.195807 12600 consensus_queue.cc:1048] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:33.195397 12374 raft_consensus.cc:697] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 2 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:33.197335 12577 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:33.197872 12577 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:33.180801 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:33.202732 10945 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
W20260709 13:16:33.181425 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:33.105727 12508 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:33.217856 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 394644us
W20260709 13:16:33.212693 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:33.220710 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:33.222540 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:33.225682 10944 raft_consensus.cc:1240] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:33.232575 10794 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30"
dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
mode: GRACEFUL
new_leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:16:33.233371 10794 raft_consensus.cc:606] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Received request to transfer leadership to TS 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:33.233771 10794 raft_consensus.cc:614] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:33.217046 10874 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:33.188189 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:33.199522 12530 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:33.241245 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:33.228739 12563 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:33.232951 12371 raft_consensus.cc:697] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 2 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:33.190557 12555 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:33.197965 12600 consensus_queue.cc:1048] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 13:16:33.230593 12599 raft_consensus.cc:2804] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:16:33.246117 12562 consensus_meta.cc:287] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.551s	user 0.011s	sys 0.000s
I20260709 13:16:33.241614 10841 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:33.198037 12374 consensus_queue.cc:237] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:33.265105 12599 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:33.199085 12603 consensus_queue.cc:1048] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 13:16:33.268374 12562 raft_consensus.cc:2749] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:33.265542 12344 raft_consensus.cc:2749] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:16:33.277495 12589 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:33.272370 12520 consensus_queue.cc:1048] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 13:16:33.262988 12563 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:33.270731 12599 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:33.298262 10841 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:16:33.414064 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 115910us
W20260709 13:16:33.416474 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 442551us
I20260709 13:16:33.274873 12387 consensus_queue.cc:1048] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.078s
W20260709 13:16:33.399089 12546 consensus_meta.cc:287] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.829s	user 0.009s	sys 0.004s
I20260709 13:16:33.439667 10767 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:33.337663 12574 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:33.333339 12539 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:33.328152 12602 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:33.340925 12448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:56410 (request call id 350) took 4372 ms. Trace:
I20260709 13:16:33.351769 12568 raft_consensus.cc:3055] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:16:33.477823 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:33.338680 12508 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:33.434345 10796 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39442 (request call id 116) took 1730 ms (client timeout 1641 ms). Trace:
I20260709 13:16:33.334128 12600 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:33.349915 12605 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:33.434638 12344 raft_consensus.cc:2804] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:33.300644 12527 consensus_queue.cc:1048] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.148s
I20260709 13:16:33.493906 12289 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:33.332723 12404 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:33.302305 12395 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:33.449105 12546 raft_consensus.cc:2749] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:16:33.559638 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 120084us
I20260709 13:16:33.510175 12568 raft_consensus.cc:740] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:33.353324 12376 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:33.563303 10916 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:33.516131 12577 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:33.535357 12605 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:33.565397 10767 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:33.522166 12597 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:33.434605 12371 consensus_queue.cc:237] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:33.496394 12583 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:33.523386 12539 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:33.575666 10916 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:33.512758 12358 consensus_queue.cc:1059] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.055s
I20260709 13:16:33.525763 12344 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:33.495627 12574 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:33.524791 12610 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:33.586766 10916 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:33.594848 12404 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:33.597441 12574 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:33.592450 12344 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:33.575217 12555 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:33.582324 12616 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:33.605396 12616 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:33.605798 12616 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:16:33.606544 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:33.582985 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 498158us
I20260709 13:16:33.608285 10916 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:33.589030 10767 leader_election.cc:400] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 3. Current term is 4.
I20260709 13:16:33.581801 12492 raft_consensus.cc:2764] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:16:33.520952 12602 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:33.610404 10799 raft_consensus.cc:2393] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 4: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
W20260709 13:16:33.514616 10796 rpcz_store.cc:269] 0709 13:16:31.703552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:31.705094 (+  1542us) service_pool.cc:224] Handling call
0709 13:16:33.412251 (+1707157us) spinlock_profiling.cc:243] Waited 873 ms on lock 0x7b5400295254. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:33.415295 (+  3044us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2007758592}
I20260709 13:16:33.614265 10767 leader_election.cc:403] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:16:33.572412 12562 consensus_queue.cc:1048] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 13:16:33.589035 12289 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:33.569646 12550 consensus_queue.cc:1048] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 13:16:33.559638 12600 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:33.607683 12590 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:33.610517 12597 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:33.592697 12539 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:33.652483 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 63556.4us
I20260709 13:16:33.653330 10767 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:33.640483 12614 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:33.637174 12483 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:33.637217 12376 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:33.620002 12566 consensus_queue.cc:1048] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:16:33.656016 12492 raft_consensus.cc:2749] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 3. Current term is 4.
I20260709 13:16:33.658998 11505 raft_consensus.cc:1240] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:16:33.660290 10874 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 3.
I20260709 13:16:33.661087 12483 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:33.675463 12395 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:33.676981 12483 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:33.661788 10698 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42968 (request call id 243) took 4817 ms. Trace:
I20260709 13:16:33.677021 10697 catalog_manager.cc:5697] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 1 to 2, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 2 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:33.658275 12590 raft_consensus.cc:697] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 3 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:33.680001 12437 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:33.681208 10948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 1305) took 1123 ms. Trace:
I20260709 13:16:33.681651 10948 rpcz_store.cc:276] 0709 13:16:32.558201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:32.558307 (+   106us) service_pool.cc:224] Handling call
0709 13:16:32.558521 (+   214us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:32.670646 (+112125us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:33.680924 (+1010278us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b54003f0054. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:33.681127 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":256073600}
I20260709 13:16:33.669726 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:16:33.682874 12608 log.cc:927] Time spent T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 13:16:33.679028 10697 catalog_manager.cc:5697] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 2 to 4, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 4 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:33.671656 12600 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:33.650871 12235 consensus_queue.cc:1059] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 13:16:33.650544 12612 consensus_queue.cc:1048] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:16:33.687765 12563 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:33.692541 12235 raft_consensus.cc:3055] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:33.692960 12235 raft_consensus.cc:740] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:33.693944 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:33.693845 12235 consensus_queue.cc:260] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:33.680744 12437 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:33.683053 11510 raft_consensus.cc:2393] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 4: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:33.670246 12443 raft_consensus.cc:3055] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:33.683930 12530 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:33.671094 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:33.637579 12574 consensus_queue.cc:1048] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:16:33.654881 10767 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:33.685434 10697 catalog_manager.cc:5697] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 1 to 2, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 2 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:33.657271 12289 raft_consensus.cc:2749] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:16:33.691602 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 46382.6us
I20260709 13:16:33.703243 10697 catalog_manager.cc:5697] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 1 to 2, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 2 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:33.720162 10916 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:33.721107 10697 catalog_manager.cc:5697] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 1 to 2, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 2 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:33.657511 12614 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:33.697698 12443 raft_consensus.cc:740] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:33.610328 12568 consensus_queue.cc:260] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:33.702625 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 47840.1us
I20260709 13:16:33.504101 12448 rpcz_store.cc:276] 0709 13:16:28.968710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:28.968886 (+   176us) service_pool.cc:224] Handling call
0709 13:16:29.541369 (+572483us) ts_tablet_manager.cc:934] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586: Initiating tablet copy from peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
0709 13:16:30.053130 (+511761us) spinlock_profiling.cc:243] Waited 22.6 ms on lock 0x7b4c000d05c0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7f181a20 00007f9c7f192a29 00007f9c7f19313f 00007f9c7f1572ab 00007f9c7f184f3c 00007f9c7f185461 00007f9c7f185768 00007f9c84f0b2eb 00007f9c85e30d3b 00007f9c85e2b008 00007f9c85e2a405
0709 13:16:32.509353 (+2456223us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:16:33.026235 (+516882us) tablet_metadata.cc:665] Metadata flushed
0709 13:16:33.026387 (+   152us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:16:33.339756 (+313369us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:16:33.340121 (+   365us) tablet_replica.cc:206] Creating consensus instance
0709 13:16:33.340567 (+   446us) ts_tablet_manager.cc:989] Replica registered.
0709 13:16:33.340777 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":118364,"spinlock_wait_cycles":51867520,"tablet-copy.queue_time_us":455250,"thread_start_us":500302,"threads_started":2}
W20260709 13:16:33.722721 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:33.723007 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:16:33.731273 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:33.678059 10698 rpcz_store.cc:276] 0709 13:16:28.844485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:28.844667 (+   182us) service_pool.cc:224] Handling call
0709 13:16:33.661699 (+4817032us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:16:31.155333 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:16:31.376607 (+221274us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:16:31.388870 (+ 12263us) write_op.cc:435] Acquired schema lock
0709 13:16:31.397040 (+  8170us) tablet.cc:662] Decoding operations
0709 13:16:31.624735 (+227695us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:16:31.642224 (+ 17489us) write_op.cc:641] Partition lock acquired for write op
0709 13:16:31.642261 (+    37us) tablet.cc:685] Acquiring locks for 12 operations
0709 13:16:31.645296 (+  3035us) tablet.cc:701] Row locks acquired
0709 13:16:31.645345 (+    49us) write_op.cc:260] PREPARE: finished
0709 13:16:31.645532 (+   187us) write_op.cc:270] Start()
0709 13:16:31.645701 (+   169us) write_op.cc:276] Timestamp: P: 1783602991645463 usec, L: 0
0709 13:16:31.645736 (+    35us) op_driver.cc:348] REPLICATION: starting
0709 13:16:31.646492 (+   756us) log.cc:844] Serialized 4848 byte log entry
0709 13:16:32.241094 (+594602us) op_driver.cc:464] REPLICATION: finished
0709 13:16:32.481602 (+240508us) write_op.cc:301] APPLY: starting
0709 13:16:32.481711 (+   109us) tablet.cc:1367] starting BulkCheckPresence
0709 13:16:32.482089 (+   378us) tablet.cc:1370] finished BulkCheckPresence
0709 13:16:32.482289 (+   200us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:16:32.484260 (+  1971us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:16:32.484684 (+   424us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=12
0709 13:16:32.484769 (+    85us) write_op.cc:312] APPLY: finished
0709 13:16:32.485304 (+   535us) log.cc:844] Serialized 89 byte log entry
0709 13:16:32.485585 (+   281us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:32.487382 (+  1797us) write_op.cc:454] Released schema lock
0709 13:16:32.509181 (+ 21799us) write_op.cc:341] FINISH: Updating metrics
0709 13:16:32.550572 (+ 41391us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:32.550600 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":17,"child_traces":[["op",{"apply.queue_time_us":240180,"apply.run_cpu_time_us":4957,"apply.run_wall_time_us":58167,"num_ops":12,"prepare.queue_time_us":324745,"prepare.run_cpu_time_us":107696,"prepare.run_wall_time_us":806598,"raft.queue_time_us":298884,"raft.run_cpu_time_us":25539,"raft.run_wall_time_us":240793,"replication_time_us":595176,"thread_start_us":731943,"threads_started":4,"wal-append.queue_time_us":292848,"wal-append.run_cpu_time_us":16890,"wal-append.run_wall_time_us":1566672}]]}
W20260709 13:16:33.739840 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:33.740516 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:33.742117 10798 raft_consensus.cc:1275] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:16:33.743732 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:33.720012 11504 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:33.745373 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:33.746850 10798 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:33.747078 10875 raft_consensus.cc:1275] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:33.722982 12405 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:33.726025 12614 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:33.749946 10876 raft_consensus.cc:1240] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:33.680081 12590 consensus_queue.cc:237] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:33.726097 12550 raft_consensus.cc:2804] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:33.736060 12235 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:33.756754 10872 raft_consensus.cc:1240] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:16:33.740190 12289 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:33.761425 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:33.704123 12530 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:33.755671 12448 tablet_copy_client.cc:806] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:33.754489 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:33.762831 12550 raft_consensus.cc:697] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 2 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:33.771642 12550 consensus_queue.cc:237] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:33.757249 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:33.752347 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:16:33.762408 12620 log.cc:927] Time spent T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.185s	user 0.000s	sys 0.001s
I20260709 13:16:33.770042 10798 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 3.
I20260709 13:16:33.756423 12568 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:33.781394 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:33.786258 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:33.786659 10872 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:16:33.767313 12531 consensus_meta.cc:287] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.828s	user 0.010s	sys 0.002s
I20260709 13:16:33.776316 12583 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:33.805351 11511 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 3.
W20260709 13:16:33.805652 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:33.783464 12448 tablet_copy_client.cc:670] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:33.782670 12563 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:33.763464 12289 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:33.788158 12343 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:16:33.809424 12309 log.cc:927] Time spent T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 13:16:33.805985 12499 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:33.809090 12371 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:33.808233 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:33.799649 12531 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:33.812939 12358 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:33.833448 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:16:33.828811 12471 consensus_meta.cc:287] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.888s	user 0.011s	sys 0.002s
I20260709 13:16:33.807315 12443 consensus_queue.cc:260] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:33.843410 12618 log.cc:927] Time spent T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:16:33.820282 12563 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:33.849498 12621 log.cc:927] Time spent T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 13:16:33.850198 12583 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:33.839821 12437 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:33.834993 12358 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:33.860316 12583 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:33.863834 12573 raft_consensus.cc:3055] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:33.870131 10793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 1641) took 1181 ms. Trace:
I20260709 13:16:33.870479 12573 raft_consensus.cc:740] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:33.871054 12539 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:33.871770 12573 consensus_queue.cc:260] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:33.873517 12573 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:33.873710 12616 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:33.865334 12614 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:33.867262 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:33.832182 12371 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:33.850237 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:33.870512 10793 rpcz_store.cc:276] 0709 13:16:32.689242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:32.806145 (+116903us) service_pool.cc:224] Handling call
0709 13:16:32.821348 (+ 15203us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:33.200498 (+379150us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:33.200531 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:33.203679 (+  3148us) log.cc:844] Serialized 26 byte log entry
0709 13:16:33.561713 (+358034us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:33.561768 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:33.561860 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:33.869876 (+308016us) raft_consensus.cc:1685] finished
0709 13:16:33.869895 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:33.870045 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":60555,"raft.queue_time_us":58673,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":13798,"thread_start_us":648472,"threads_started":2,"wal-append.queue_time_us":471762}
I20260709 13:16:33.879832 10871 raft_consensus.cc:2393] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 4: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:33.839257 12471 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:33.882213 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:33.883368 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:33.883565 12471 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:33.884478 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:33.884742 10946 raft_consensus.cc:1240] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:33.858613 12443 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:33.883683 10792 consensus_queue.cc:237] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:33.887041 11510 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 3.
I20260709 13:16:33.887406 12289 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:33.899302 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:33.901882 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:33.902104 11511 raft_consensus.cc:1240] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:33.899322 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:33.903221 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:33.902513 11510 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 5.
I20260709 13:16:33.904026 11503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56416 (request call id 400) took 1148 ms. Trace:
I20260709 13:16:33.904505 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:33.904417 11503 rpcz_store.cc:276] 0709 13:16:32.756036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:33.902941 (+1146905us) service_pool.cc:224] Handling call
0709 13:16:33.903968 (+  1027us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:33.905405 12531 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:33.906028 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:33.907534 12358 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:33.909114 12563 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:33.910741 12376 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:33.911252 12376 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:33.914440 12471 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:33.878224 12530 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:33.914973 12471 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:33.916173 12371 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:33.916954 12471 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:33.911496 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:33.918391 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:33.919689 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:33.920261 10943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 1621) took 1076 ms. Trace:
I20260709 13:16:33.920845 10943 rpcz_store.cc:276] 0709 13:16:32.843729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:32.847193 (+  3464us) service_pool.cc:224] Handling call
0709 13:16:32.847496 (+   303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:33.198715 (+351219us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:33.198756 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:33.229697 (+ 30941us) log.cc:844] Serialized 26 byte log entry
0709 13:16:33.533323 (+303626us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:16:33.559165 (+ 25842us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:33.563403 (+  4238us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:33.919960 (+356557us) raft_consensus.cc:1685] finished
0709 13:16:33.919988 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:33.920184 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":69053,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":19993,"thread_start_us":603752,"threads_started":2,"wal-append.queue_time_us":411544}
I20260709 13:16:33.926389 10871 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 4.
I20260709 13:16:33.929491 10797 raft_consensus.cc:1240] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:16:33.930608 10766 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:33.926919 10872 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
I20260709 13:16:33.933140 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:33.936384 10797 raft_consensus.cc:1240] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:33.936518 11478 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:33.941826 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:33.943081 10766 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:33.944020 11507 raft_consensus.cc:2393] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 3: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:33.944885 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:33.947048 11478 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:33.948247 10793 raft_consensus.cc:2393] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 6: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:33.900513 12602 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:33.959774 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:33.960590 12566 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:33.930984 12568 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:33.935279 12531 raft_consensus.cc:2749] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:33.920275 10874 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 5.
I20260709 13:16:33.963313 11478 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:33.938674 12292 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:16:33.998531 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 55550.4us
I20260709 13:16:34.000454 10766 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:16:33.947587 12599 consensus_meta.cc:287] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.506s	user 0.010s	sys 0.002s
I20260709 13:16:33.904668 12344 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:34.003654 12566 raft_consensus.cc:697] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 2 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:34.021375 12566 consensus_queue.cc:237] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:33.948383 12235 raft_consensus.cc:2764] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:16:34.016886 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:34.015874 12289 raft_consensus.cc:2749] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:16:34.043013 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 44315.8us
I20260709 13:16:34.027709 12616 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:34.017614 12569 raft_consensus.cc:2749] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:33.994374 12530 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:34.015756 12612 raft_consensus.cc:2804] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:34.002439 12568 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:34.068635 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:34.069623 10872 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:34.052423 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:34.073146 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:34.063709 10869 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 4.
I20260709 13:16:33.997299 12483 consensus_queue.cc:1048] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:16:34.074685 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:34.081754 10873 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 4.
I20260709 13:16:34.025734 12292 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:34.067646 12530 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:34.095031 12376 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:34.096153 12530 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:34.016319 12599 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:34.101842 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:34.086287 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:34.103503 11478 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:34.085121 11504 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:34.101883 12309 tablet_bootstrap.cc:492] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:34.125934 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:34.100239 12368 raft_consensus.cc:3055] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:16:34.130453 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:34.118273 10872 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 5.
I20260709 13:16:34.127537 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:34.110285 12568 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:34.149636 10950 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
I20260709 13:16:34.097039 12612 raft_consensus.cc:697] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 3 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:34.146540 10946 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 3.
I20260709 13:16:34.155297 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:16:34.104727 12638 log.cc:927] Time spent T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:16:34.109088 12483 raft_consensus.cc:2749] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:34.122220 10799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 1649) took 1066 ms. Trace:
I20260709 13:16:34.158715 10872 raft_consensus.cc:1240] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:34.108290 12599 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:34.113502 12614 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:34.145665 10874 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 2.
I20260709 13:16:34.121330 12405 raft_consensus.cc:2749] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:34.127236 12530 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:34.103859 12644 consensus_queue.cc:1059] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.074s
I20260709 13:16:34.170032 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:34.109856 12344 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:34.119837 12386 raft_consensus.cc:3055] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:34.151618 12616 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:34.174255 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:34.206460 10949 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 3.
I20260709 13:16:34.148037 12368 raft_consensus.cc:740] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:34.199488 12309 tablet_bootstrap.cc:492] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586: Bootstrap complete.
I20260709 13:16:34.105207 12574 consensus_queue.cc:1059] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
W20260709 13:16:34.211580 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 101600us
I20260709 13:16:34.105969 12557 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:34.208704 10946 raft_consensus.cc:1240] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:16:34.166996 10799 rpcz_store.cc:276] 0709 13:16:33.056182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:33.135780 (+ 79598us) service_pool.cc:224] Handling call
0709 13:16:34.121977 (+986197us) spinlock_profiling.cc:243] Waited 470 ms on lock 0x7b540039f8d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:34.122153 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1080180608}
I20260709 13:16:34.200138 10950 raft_consensus.cc:2410] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 6 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:16:34.205946 12634 log.cc:927] Time spent T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.262s	user 0.000s	sys 0.001s
I20260709 13:16:34.221987 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:34.202417 12386 raft_consensus.cc:740] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:34.427251 12386 consensus_queue.cc:260] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:34.225473 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:34.237102 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:34.429457 10945 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
I20260709 13:16:34.245678 10946 raft_consensus.cc:1240] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:34.224256 12557 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:34.432562 12557 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:34.209317 12616 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:34.252363 11512 raft_consensus.cc:1240] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:34.237483 12612 consensus_queue.cc:237] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:34.253005 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:34.435165 10944 raft_consensus.cc:1275] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:34.205672 12530 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:34.244455 12637 log.cc:927] Time spent T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
I20260709 13:16:34.241569 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:34.239828 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:34.251334 12309 ts_tablet_manager.cc:1403] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586: Time spent bootstrapping tablet: real 2.592s	user 0.033s	sys 0.030s
I20260709 13:16:34.244200 10941 raft_consensus.cc:1275] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:34.252599 11509 raft_consensus.cc:1240] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
W20260709 13:16:34.266177 10915 consensus_peers.cc:597] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 -> Peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Couldn't send request to peer 12682eb70bc84c76879e04a8628ee586. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:34.252380 12368 consensus_queue.cc:260] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:34.253494 12344 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:34.271919 12600 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:34.299347 12376 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:34.311750 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:34.434687 12616 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:34.441718 12386 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:34.306728 12648 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:34.444301 12648 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:34.305837 12292 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:34.331209 12356 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:34.447314 12376 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:34.447635 12356 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:34.449077 12530 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:34.401391 10869 raft_consensus.cc:1240] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 LEADER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:34.319968 12644 raft_consensus.cc:3055] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:35.676216 12644 raft_consensus.cc:740] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:35.676476 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 175) took 1657 ms. Trace:
I20260709 13:16:35.676968 10869 rpcz_store.cc:276] 0709 13:16:34.019550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:34.400848 (+381298us) service_pool.cc:224] Handling call
0709 13:16:34.401208 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:35.676153 (+1274945us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:35.676409 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:35.677330 12644 consensus_queue.cc:260] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:34.380529 12661 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:34.333258 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:36.722462 10868 raft_consensus.cc:1275] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:36.722827 10870 raft_consensus.cc:2393] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 2: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:36.723256 10868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1286) took 2450 ms. Trace:
I20260709 13:16:34.346681 10872 raft_consensus.cc:1275] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:16:34.364991 10875 raft_consensus.cc:1240] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:36.763067 10875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 170) took 2816 ms. Trace:
I20260709 13:16:36.763439 10875 rpcz_store.cc:276] 0709 13:16:33.946387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:34.364526 (+418139us) service_pool.cc:224] Handling call
0709 13:16:34.364815 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:36.762805 (+2397990us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:36.763009 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:36.764391 10875 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:16:34.457353 12648 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:34.383898 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:16:36.793934 10943 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36714 (request call id 153) took 2414 ms (client timeout 2632 ms). Trace:
W20260709 13:16:36.796478 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 2.69335e+06us
W20260709 13:16:34.384140 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:34.459020 12309 raft_consensus.cc:359] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:34.451151 12344 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:36.807590 12344 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:34.449546 12356 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:34.449510 12376 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:34.362226 12494 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:34.397511 12448 tablet_copy_client.cc:538] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:34.312978 12574 raft_consensus.cc:3055] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:36.820569 12574 raft_consensus.cc:740] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:36.821503 12574 consensus_queue.cc:260] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:34.453123 12600 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:36.825346 12600 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:34.395053 12605 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:34.422179 12617 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:36.827819 12617 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:36.828503 12448 tablet_bootstrap.cc:492] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586: Bootstrap starting.
I20260709 13:16:36.829948 12617 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:16:35.498831 10822 debug-util.cc:398] Leaking SignalData structure 0x7b080013b7c0 after lost signal to thread 9682
I20260709 13:16:35.634653 12292 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:35.636123 10873 raft_consensus.cc:1240] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:36.830804 12292 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:35.671382 10842 connection.cc:256] RPC call timeout handler was delayed by -1.225164869000082s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19940000000000002s for a response
W20260709 13:16:36.831909 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:36.832449 10842 connection.cc:256] RPC call timeout handler was delayed by -1.0697746999999254s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15610000000000002s for a response
W20260709 13:16:36.832854 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:36.832976 12292 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:35.644940 12583 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:35.678984 10869 raft_consensus.cc:1275] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:16:36.834055 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1283) took 2661 ms. Trace:
I20260709 13:16:35.678920 12644 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:36.834404 10869 rpcz_store.cc:276] 0709 13:16:34.172644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:35.678228 (+1505584us) service_pool.cc:224] Handling call
0709 13:16:35.678583 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:36.833777 (+1155194us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:36.833998 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:36.834964 12443 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:36.835776 10869 raft_consensus.cc:1240] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:36.836404 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 179) took 2525 ms. Trace:
I20260709 13:16:36.836691 10869 rpcz_store.cc:276] 0709 13:16:34.311192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:36.835271 (+2524079us) service_pool.cc:224] Handling call
0709 13:16:36.835613 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:36.836160 (+   547us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:36.836354 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:16:36.830611 10822 debug-util.cc:398] Leaking SignalData structure 0x7b080013b780 after lost signal to thread 9685
W20260709 13:16:36.838215 10822 debug-util.cc:398] Leaking SignalData structure 0x7b0800092d20 after lost signal to thread 10733
I20260709 13:16:36.417814 10874 raft_consensus.cc:1240] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:36.462040 10794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 1647) took 3524 ms. Trace:
I20260709 13:16:36.471786 12590 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:36.474061 12672 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:36.839059 10874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1285) took 2620 ms. Trace:
I20260709 13:16:36.839284 12344 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:36.481750 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:36.839143 12356 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:36.482810 12374 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:35.679936 12661 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:36.839761 10874 rpcz_store.cc:276] 0709 13:16:34.219209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:36.417058 (+2197849us) service_pool.cc:224] Handling call
0709 13:16:36.417532 (+   474us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:36.838765 (+421233us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:36.839004 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:36.839644 12672 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:36.840432 12374 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:36.842047 12661 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:36.842166 12672 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:35.975428 12483 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:36.484098 12471 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:36.528184 12599 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:36.842809 12483 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:36.843165 12590 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:36.843101 12471 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:36.845009 12483 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:36.494017 10949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 151) took 2281 ms. Trace:
I20260709 13:16:36.484508 10876 raft_consensus.cc:1240] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:36.838838 10794 rpcz_store.cc:276] 0709 13:16:32.937443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:33.240514 (+303071us) service_pool.cc:224] Handling call
0709 13:16:33.256122 (+ 15608us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:33.652339 (+396217us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:33.652386 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:33.652991 (+   605us) log.cc:844] Serialized 26 byte log entry
0709 13:16:33.723625 (+ 70634us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:16:33.723860 (+   235us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:36.461547 (+2737687us) spinlock_profiling.cc:243] Waited 394 ms on lock 0x7b54002125d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:36.461619 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:36.461668 (+    49us) raft_consensus.cc:1685] finished
0709 13:16:36.461682 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:36.461947 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1757,"raft.queue_time_us":3365,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":3033,"spinlock_wait_cycles":906010368,"thread_start_us":70296,"threads_started":1,"wal-append.queue_time_us":78518,"wal-append.run_cpu_time_us":35774,"wal-append.run_wall_time_us":2981876}
I20260709 13:16:36.523536 12371 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:36.846284 10876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 177) took 2621 ms. Trace:
W20260709 13:16:36.723362 10870 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49290 (request call id 178) took 2471 ms (client timeout 2632 ms). Trace:
W20260709 13:16:36.846901 10870 rpcz_store.cc:269] 0709 13:16:34.252354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:34.332951 (+ 80597us) service_pool.cc:224] Handling call
0709 13:16:36.723296 (+2390345us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:36.723654 10868 rpcz_store.cc:276] 0709 13:16:34.272573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:36.721733 (+2449160us) service_pool.cc:224] Handling call
0709 13:16:36.722095 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:36.722986 (+   891us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:36.723199 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:36.724633 12530 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:36.847733 12376 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:36.838954 12676 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:36.848148 12530 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:36.848457 12376 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:36.756129 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:36.839923 12344 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:36.850296 10942 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41014 (request call id 1311) took 2652 ms (client timeout 1535 ms). Trace:
I20260709 13:16:36.754914 11509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56416 (request call id 406) took 2864 ms. Trace:
I20260709 13:16:36.762188 10872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1275) took 3011 ms. Trace:
I20260709 13:16:36.764951 10875 raft_consensus.cc:606] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 LEADER]: Received request to transfer leadership to TS cb0428cc55984daba1ba63744a5102a2
W20260709 13:16:34.452178 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 186120us
W20260709 13:16:36.770526 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:36.875170 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:36.778896 12368 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:36.781387 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:36.783164 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:36.798667 12289 consensus_queue.cc:1048] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:36.880136 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:16:36.880941  9687 debug-util.cc:398] Leaking SignalData structure 0x7b080000fec0 after lost signal to thread 10767
I20260709 13:16:36.881325 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:16:36.881520  9687 kernel_stack_watchdog.cc:198] Thread 10767 stuck at /apache/dev/git/kudu/src/kudu/rpc/outbound_call.cc:307 for 1361ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
I20260709 13:16:36.882948 10950 raft_consensus.cc:2393] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 2: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:36.883795 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:36.802143 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:36.884510 10950 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 3.
I20260709 13:16:36.884639 10941 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:36.885429 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:36.885761 12590 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:36.804042 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:36.804219 12309 raft_consensus.cc:740] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Initialized, Role: LEARNER
I20260709 13:16:36.887023 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:36.886816 12309 consensus_queue.cc:260] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:36.888162 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:36.889575 12309 ts_tablet_manager.cc:1434] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586: Time spent starting tablet: real 2.438s	user 0.010s	sys 0.000s
W20260709 13:16:36.889941 12309 ts_tablet_manager.cc:1467] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586: Tablet startup took 2437ms
I20260709 13:16:36.891176 12683 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:36.891700 12683 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:36.890295 12309 ts_tablet_manager.cc:1469] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586: Trace:
0709 13:16:23.553418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:23.557596 (+  4178us) service_pool.cc:224] Handling call
0709 13:16:23.887712 (+330116us) ts_tablet_manager.cc:934] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586: Initiating tablet copy from peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685)
0709 13:16:24.684251 (+796539us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:16:25.110653 (+426402us) tablet_metadata.cc:665] Metadata flushed
0709 13:16:25.110774 (+   121us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:16:25.704797 (+594023us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:16:25.705185 (+   388us) tablet_replica.cc:206] Creating consensus instance
0709 13:16:25.705515 (+   330us) ts_tablet_manager.cc:989] Replica registered.
0709 13:16:25.705585 (+    70us) inbound_call.cc:177] Queueing success response
0709 13:16:31.653121 (+5947536us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:16:34.452028 (+2798907us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:16:34.455644 (+  3616us) tablet_replica.cc:261] Starting instrumentation
0709 13:16:34.456537 (+   893us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":11957,"raft.queue_time_us":3475,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":101,"tablet-copy.queue_time_us":276329,"tablet-download-pool.queue_time_us":2842,"tablet-download-pool.run_cpu_time_us":1858,"tablet-download-pool.run_wall_time_us":4719576,"thread_start_us":307688,"threads_started":2}
W20260709 13:16:36.806166 10766 connection.cc:256] RPC call timeout handler was delayed by -2.1937980620000417s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1535s for a response
W20260709 13:16:36.817340 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:36.893606 12471 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:16:36.894773 10766 connection.cc:256] RPC call timeout handler was delayed by -2.0056980619999649s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1744s for a response
I20260709 13:16:36.831184 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1269) took 3092 ms. Trace:
W20260709 13:16:36.895212 10841 connection.cc:256] RPC call timeout handler was delayed by -2.6214179429998694s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2102s for a response
I20260709 13:16:36.895368 10873 rpcz_store.cc:276] 0709 13:16:33.739081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:34.082429 (+343348us) service_pool.cc:224] Handling call
0709 13:16:35.635650 (+1553221us) spinlock_profiling.cc:243] Waited 2.02 ms on lock 0x7b5800177098. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd90a22 00007f9c86ca9c09 00007f9c85e03ee0 00007f9c85ecd0ab 00007f9c85ecd131 00007f9c85e8efc0 00007f9c85e79929 00007f9c85e793d2 00007f9c7f3f0bd4
0709 13:16:35.635916 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:36.830875 (+1194959us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:36.831120 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4636928}
I20260709 13:16:36.896107 12569 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:36.896188 12568 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:36.896552 12492 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:36.896682 12569 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:36.896831 12568 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:36.897140 12492 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:36.801858 11477 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:36.903674 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:16:36.904366 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 102606us
W20260709 13:16:36.838624 10822 debug-util.cc:398] Leaking SignalData structure 0x7b0800092f60 after lost signal to thread 10747
I20260709 13:16:36.846809 10876 rpcz_store.cc:276] 0709 13:16:34.225135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:36.483937 (+2258802us) service_pool.cc:224] Handling call
0709 13:16:36.484307 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:36.845972 (+361665us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:36.846216 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:16:36.708724 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:36.906334 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:36.712301 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:36.895203 10766 connection.cc:256] RPC call timeout handler was delayed by -2.0056980619999649s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1744s for a response
W20260709 13:16:36.907099 10916 connection.cc:256] RPC call timeout handler was delayed by -1.8715770240000893s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18560000000000001s for a response
W20260709 13:16:36.907824 10916 connection.cc:256] RPC call timeout handler was delayed by -1.4517168330000914s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1516s for a response
W20260709 13:16:36.908252 10916 connection.cc:256] RPC call timeout handler was delayed by -1.4517168330000914s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1516s for a response
W20260709 13:16:36.908624 10916 connection.cc:256] RPC call timeout handler was delayed by -1.4222587819999717s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1728s for a response
W20260709 13:16:36.908999 10916 connection.cc:256] RPC call timeout handler was delayed by -1.4222587819999717s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1728s for a response
W20260709 13:16:36.909358 10916 connection.cc:256] RPC call timeout handler was delayed by -1.390363071000138s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15400000000000003s for a response
W20260709 13:16:36.909713 10916 connection.cc:256] RPC call timeout handler was delayed by -1.390363071000138s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15400000000000003s for a response
W20260709 13:16:36.910077 10916 connection.cc:256] RPC call timeout handler was delayed by -1.3207941360001314s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1503s for a response
W20260709 13:16:36.910424 10916 connection.cc:256] RPC call timeout handler was delayed by -1.2881587819999822s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1877s for a response
I20260709 13:16:36.712312 12614 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:34.373572 10871 raft_consensus.cc:1275] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:36.999553 10871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 172) took 3036 ms. Trace:
I20260709 13:16:36.848714 12676 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:36.851373 12344 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:36.769193 12675 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:36.851099 10872 rpcz_store.cc:276] 0709 13:16:33.750413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:34.345800 (+595387us) service_pool.cc:224] Handling call
0709 13:16:34.346215 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:36.761848 (+2415633us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:36.762095 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:36.971959 12689 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:36.852030 11509 rpcz_store.cc:276] 0709 13:16:33.890855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:33.899264 (+  8409us) service_pool.cc:224] Handling call
0709 13:16:34.118079 (+218815us) spinlock_profiling.cc:243] Waited 21.1 ms on lock 0x7b5800003c98. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd90a22 00007f9c86ca9c09 00007f9c85e03ee0 00007f9c85ecd0ab 00007f9c85ecd131 00007f9c85e8efc0 00007f9c85e79929 00007f9c85e793d2 00007f9c7f3f0bd4
0709 13:16:34.118319 (+   240us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:34.441762 (+323443us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:36.754539 (+2312777us) spinlock_profiling.cc:243] Waited 88.9 ms on lock 0x7b5400484754. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:36.754817 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":253087360}
I20260709 13:16:36.852960  9682 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:16:36.850646 12530 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:36.845708 10949 rpcz_store.cc:276] 0709 13:16:34.212544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:34.212794 (+   250us) service_pool.cc:224] Handling call
0709 13:16:36.486958 (+2274164us) spinlock_profiling.cc:243] Waited 8.1 ms on lock 0x7b5800112598. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd90a22 00007f9c86ca9c09 00007f9c85e03ee0 00007f9c85ecd0ab 00007f9c85ecd131 00007f9c85e8efc0 00007f9c85e79929 00007f9c85e793d2 00007f9c7f3f0bd4
0709 13:16:36.487182 (+   224us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:36.487546 (+   364us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:36.487574 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:36.488047 (+   473us) log.cc:844] Serialized 26 byte log entry
0709 13:16:36.491868 (+  3821us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:36.491909 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:36.491976 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:36.493671 (+  1695us) raft_consensus.cc:1685] finished
0709 13:16:36.493696 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:36.493944 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":35775,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":106,"spinlock_wait_cycles":18625408,"thread_start_us":3414,"threads_started":1,"wal-append.queue_time_us":3752}
I20260709 13:16:36.846560 12371 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:36.853917 10946 raft_consensus.cc:1275] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:16:37.024220 10946 pending_rounds.cc:85] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 1
I20260709 13:16:37.024509 10946 pending_rounds.cc:107] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:16:37.024765 10946 raft_consensus.cc:2889] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:16:36.850832 10942 rpcz_store.cc:269] 0709 13:16:34.213796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:34.213958 (+   162us) service_pool.cc:224] Handling call
0709 13:16:36.755972 (+2542014us) spinlock_profiling.cc:243] Waited 5.59 ms on lock 0x7b5800112598. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd90a22 00007f9c86ca9c09 00007f9c85e03ee0 00007f9c85ecd0ab 00007f9c85ecd131 00007f9c85e7b0a0 00007f9c85e7a984 00007f9c7f3f56d4 00007f9c7f3f5662
0709 13:16:36.850214 (+ 94242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12855552}
I20260709 13:16:36.855261 12374 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:36.840147 12356 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:36.822522 12574 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:36.816313 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 173) took 2821 ms. Trace:
W20260709 13:16:37.031077 10841 connection.cc:256] RPC call timeout handler was delayed by -2.0355568259999473s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1736s for a response
I20260709 13:16:36.864787 10696 catalog_manager.cc:5697] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 2 to 3, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 3 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
W20260709 13:16:37.031589 10841 connection.cc:256] RPC call timeout handler was delayed by -2.009456825999905s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1765s for a response
W20260709 13:16:37.032006 10841 connection.cc:256] RPC call timeout handler was delayed by -1.9521852319999198s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15610000000000002s for a response
W20260709 13:16:37.032279 10841 connection.cc:256] RPC call timeout handler was delayed by -1.8951568259999476s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1892s for a response
W20260709 13:16:37.032521 10841 connection.cc:256] RPC call timeout handler was delayed by -1.8033568260000266s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19940000000000002s for a response
W20260709 13:16:37.032759 10841 connection.cc:256] RPC call timeout handler was delayed by -1.5927568259999134s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22280000000000003s for a response
W20260709 13:16:37.032987 10841 connection.cc:256] RPC call timeout handler was delayed by -1.5693568259998756s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22540000000000002s for a response
W20260709 13:16:37.033218 10841 connection.cc:256] RPC call timeout handler was delayed by -1.5678852319999805s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1988s for a response
W20260709 13:16:37.033460 10841 connection.cc:256] RPC call timeout handler was delayed by -1.3857568260000335s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24580000000000002s for a response
I20260709 13:16:36.869706 10697 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42984 (request call id 243) took 6128 ms. Trace:
I20260709 13:16:36.883422 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:36.885684 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:36.835454 12443 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:36.916345 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 2.39723e+06us
I20260709 13:16:36.918565 12688 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:36.915347 11477 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:37.038769 12688 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:36.923379 10916 connection.cc:256] RPC call timeout handler was delayed by -1.2089587820000816s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1965s for a response
W20260709 13:16:37.039768 10916 connection.cc:256] RPC call timeout handler was delayed by -1.1561521160001575s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1573s for a response
W20260709 13:16:37.040293 10916 connection.cc:256] RPC call timeout handler was delayed by -1.1468587819999811s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2034s for a response
I20260709 13:16:37.040419 12483 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:16:37.040733 10916 connection.cc:256] RPC call timeout handler was delayed by -1.1468587819999811s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2034s for a response
I20260709 13:16:37.040899 12483 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:37.041224 10916 connection.cc:256] RPC call timeout handler was delayed by -1.0418345479999971s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20779999999999998s for a response
I20260709 13:16:37.040088 10767 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:16:37.041947 10916 connection.cc:256] RPC call timeout handler was delayed by -1.0247345479999694s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2097s for a response
W20260709 13:16:36.794397 10943 rpcz_store.cc:269] 0709 13:16:34.380042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:34.383567 (+  3525us) service_pool.cc:224] Handling call
0709 13:16:36.793822 (+2410255us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:37.041355 12483 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:36.924928 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:37.045120 10792 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:37.047075 12691 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:37.047747 12376 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:37.047660 12691 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:37.048316 12376 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:37.048822 12376 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:37.049991 12691 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:36.928040 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:36.936295 10797 raft_consensus.cc:1240] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:37.023346 12689 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:37.052657 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:37.053540 10797 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:37.036089 12694 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:37.055289 10874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 181) took 2688 ms. Trace:
I20260709 13:16:37.055061 12694 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:36.943126 10915 connection.cc:256] RPC call timeout handler was delayed by -2.2923994660000062s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1877s for a response
I20260709 13:16:37.055742 10874 rpcz_store.cc:276] 0709 13:16:34.367219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:36.840834 (+2473615us) service_pool.cc:224] Handling call
0709 13:16:36.841174 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:36.841399 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:16:36.996128 (+154729us) log.cc:844] Serialized 14 byte log entry
0709 13:16:37.054843 (+ 58715us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:37.054910 (+    67us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:37.054962 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:37.055055 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:37.055219 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6545,"prepare.queue_time_us":154884,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":27,"raft.queue_time_us":20685,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":86,"thread_start_us":187699,"threads_started":2,"wal-append.queue_time_us":45449}
W20260709 13:16:37.057952 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:37.058326 10915 connection.cc:256] RPC call timeout handler was delayed by -2.2131994660001055s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1965s for a response
I20260709 13:16:36.950819 10794 raft_consensus.cc:1240] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:37.058918 12692 consensus_queue.cc:1059] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:37.059836 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:37.060051 12692 raft_consensus.cc:3055] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:37.060477 12692 raft_consensus.cc:740] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:37.060577 10794 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 2.
I20260709 13:16:37.061594 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:37.061349 12692 consensus_queue.cc:260] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:36.903602 12289 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:36.960453 10793 raft_consensus.cc:1240] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:16:36.916730 12662 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:36.916958 12644 raft_consensus.cc:2749] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:37.079289 12676 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:37.079411 12289 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:37.082022 12289 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:36.960850 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 46115.4us
I20260709 13:16:37.021751 12675 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:37.082557 12662 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:37.085075 12675 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:36.948143 12583 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:37.085678 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:37.086236 12583 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:37.087175 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:37.087585 11512 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:37.087924 11507 raft_consensus.cc:2393] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 3: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:36.949218 12617 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:37.088470 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:37.088811 12617 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:37.089457 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:36.952512 12395 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:37.091002 12617 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:36.950436 12471 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:36.954882 12531 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:37.091759 12471 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:37.090597 12395 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:37.091921 12531 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:36.955703 12396 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:36.955132 12313 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:37.092437 11505 raft_consensus.cc:1240] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:36.953886 12492 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:36.907114 10766 connection.cc:256] RPC call timeout handler was delayed by -1.9831980619999285s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1769s for a response
I20260709 13:16:36.970356 12568 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:16:36.970933 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:36.971340 12685 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:36.947460 12368 consensus_queue.cc:1048] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 13:16:36.960340 12569 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:36.971601 12683 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:36.951949 12614 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:36.969844 10822 debug-util.cc:398] Leaking SignalData structure 0x7b080028c540 after lost signal to thread 10825
I20260709 13:16:36.991770 12684 consensus_queue.cc:1048] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:36.843128 12599 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:37.000075 10871 rpcz_store.cc:276] 0709 13:16:33.962881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:34.372855 (+409974us) service_pool.cc:224] Handling call
0709 13:16:34.373221 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:36.999141 (+2625920us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:36.999470 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:37.008164 10842 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:37.010669  9682 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:37.011119 10796 raft_consensus.cc:1240] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:16:37.034428 10696 catalog_manager.cc:5697] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 1 to 2, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 2 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:37.034744 10945 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 3.
I20260709 13:16:37.124078 10696 catalog_manager.cc:5697] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:36.939312 10798 raft_consensus.cc:1275] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:16:37.078014 12605 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:37.126850 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:37.127588 10798 raft_consensus.cc:2393] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 4: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:37.127756 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:37.086501 11504 raft_consensus.cc:1240] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:36.955319 12405 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:37.094012 12395 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:37.094143 12313 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:36.956058 12600 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:16:37.106611 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 191392us
I20260709 13:16:37.140614 11512 raft_consensus.cc:1275] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:37.122047 12569 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:37.141933 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 101917us
I20260709 13:16:37.107312 10916 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:37.097635 10799 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:37.105870 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 71970.6us
W20260709 13:16:37.108453 10915 connection.cc:256] RPC call timeout handler was delayed by -1.2101580369999283s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1573s for a response
I20260709 13:16:37.110347 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:37.104526 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 180) took 2745 ms. Trace:
I20260709 13:16:37.145336 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:37.158587 10872 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 2.
I20260709 13:16:37.159492 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:16:37.160521 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 52731.7us
I20260709 13:16:37.111425 10792 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 5.
I20260709 13:16:37.041155 12688 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:37.114174 10766 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:37.034212 10697 rpcz_store.cc:276] 0709 13:16:30.741158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:30.741400 (+   242us) service_pool.cc:224] Handling call
0709 13:16:36.869612 (+6128212us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:16:34.037500 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:16:34.096189 (+ 58689us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:16:34.096227 (+    38us) write_op.cc:435] Acquired schema lock
0709 13:16:34.096246 (+    19us) tablet.cc:662] Decoding operations
0709 13:16:34.157420 (+ 61174us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:16:34.166551 (+  9131us) write_op.cc:641] Partition lock acquired for write op
0709 13:16:34.168525 (+  1974us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:16:34.243750 (+ 75225us) tablet.cc:701] Row locks acquired
0709 13:16:34.247124 (+  3374us) write_op.cc:260] PREPARE: finished
0709 13:16:34.260635 (+ 13511us) write_op.cc:270] Start()
0709 13:16:34.266314 (+  5679us) write_op.cc:276] Timestamp: P: 1783602994258763 usec, L: 0
0709 13:16:34.272234 (+  5920us) op_driver.cc:348] REPLICATION: starting
0709 13:16:34.321391 (+ 49157us) log.cc:844] Serialized 2074 byte log entry
0709 13:16:34.399363 (+ 77972us) op_driver.cc:464] REPLICATION: finished
0709 13:16:34.409575 (+ 10212us) write_op.cc:301] APPLY: starting
0709 13:16:34.409657 (+    82us) tablet.cc:1367] starting BulkCheckPresence
0709 13:16:34.409802 (+   145us) tablet.cc:1370] finished BulkCheckPresence
0709 13:16:34.409948 (+   146us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:16:34.410615 (+   667us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:16:34.410810 (+   195us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0709 13:16:34.410851 (+    41us) write_op.cc:312] APPLY: finished
0709 13:16:34.411401 (+   550us) log.cc:844] Serialized 47 byte log entry
0709 13:16:34.412197 (+   796us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:34.412326 (+   129us) write_op.cc:454] Released schema lock
0709 13:16:34.412371 (+    45us) write_op.cc:341] FINISH: Updating metrics
0709 13:16:34.412720 (+   349us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:34.412744 (+    24us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":320644,"reported_tablets":13,"child_traces":[["op",{"apply.queue_time_us":9987,"apply.run_cpu_time_us":2263,"apply.run_wall_time_us":3109,"num_ops":5,"prepare.queue_time_us":222583,"prepare.run_cpu_time_us":17128,"prepare.run_wall_time_us":345347,"raft.queue_time_us":18971,"raft.run_cpu_time_us":7131,"raft.run_wall_time_us":10176,"replication_time_us":126947,"thread_start_us":193273,"threads_started":4,"wal-append.queue_time_us":49405,"wal-append.run_cpu_time_us":11028,"wal-append.run_wall_time_us":1041336}]]}
I20260709 13:16:37.160773 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:37.163362 10794 raft_consensus.cc:2393] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 4: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:37.163617 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:37.118494 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:37.164263 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:37.164439 11510 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:37.119585 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:37.164928 10794 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:37.164885 11477 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:37.165647 10793 raft_consensus.cc:2393] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 3: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:37.120285 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:37.093871 12396 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:37.053148 12689 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:37.158279 10869 rpcz_store.cc:276] 0709 13:16:34.358968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:36.837393 (+2478425us) service_pool.cc:224] Handling call
0709 13:16:36.837697 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:36.837893 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:16:36.996933 (+159040us) log.cc:844] Serialized 14 byte log entry
0709 13:16:37.104054 (+107121us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:37.104131 (+    77us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:37.104189 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:37.104280 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:37.104443 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18905,"prepare.queue_time_us":158150,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":80661,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":6453,"thread_start_us":193522,"threads_started":2}
I20260709 13:16:37.167217 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:37.085176 12662 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:37.094588 12531 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:37.107841 12356 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:37.128484 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:37.136503 12676 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:37.063539 12494 raft_consensus.cc:2804] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:37.141616 12583 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:37.115810 12685 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:37.136637 12371 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:37.184257 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:37.107352 12692 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:37.064250 10941 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 3.
I20260709 13:16:37.136484 12683 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:37.195923 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:37.136158 12694 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:37.192489 10916 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:37.116295 12617 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:37.112367 12374 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:16:37.121807 10822 debug-util.cc:398] Leaking SignalData structure 0x7b08002bf160 after lost signal to thread 10826
I20260709 13:16:37.166996 12471 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:16:37.136929 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 128911us
I20260709 13:16:37.214740 10947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 150) took 3017 ms. Trace:
I20260709 13:16:37.163785 10873 raft_consensus.cc:1240] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:37.215377 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:37.216171 10868 raft_consensus.cc:2410] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:16:37.216200 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:37.203861 12371 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:37.217029 10873 raft_consensus.cc:2393] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 2: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:37.160158 10796 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:37.117444 10797 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 5.
I20260709 13:16:37.163885 11511 raft_consensus.cc:2410] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:16:37.183804 10795 raft_consensus.cc:1275] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:16:37.167418 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 53332.7us
I20260709 13:16:37.159695 12661 consensus_queue.cc:1059] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:16:37.167876 10875 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 3.
I20260709 13:16:37.165992 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:37.154626 12405 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:37.182202 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:37.234083 10871 raft_consensus.cc:2410] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:16:37.191373 10798 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:37.235356 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:37.236127 10798 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 5.
I20260709 13:16:37.237133 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:37.237813 10798 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:37.177945 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:37.137158 12687 raft_consensus.cc:2749] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:37.193027 10942 raft_consensus.cc:1240] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:37.194664 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:37.176774 10792 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:37.198223 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:37.199128 10841 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:37.199020 10916 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:37.196599 10870 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2.
I20260709 13:16:37.246462 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:37.136883 12599 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:37.201859 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:37.202346 10799 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 2.
W20260709 13:16:37.181236 12695 log.cc:927] Time spent T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:16:37.206367 10944 raft_consensus.cc:1240] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:37.264333 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:37.265390 10944 raft_consensus.cc:2393] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 4: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:37.169021 12573 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:37.115644 12590 consensus_queue.cc:1059] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 13:16:37.207103 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:37.166778 12671 consensus_queue.cc:1048] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:16:37.193126 12614 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:37.144485 12605 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:37.031469 10867 rpcz_store.cc:276] 0709 13:16:33.995358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:34.388653 (+393295us) service_pool.cc:224] Handling call
0709 13:16:34.389058 (+   405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:36.808307 (+2419249us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:36.808345 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:36.808390 (+    45us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:36.808424 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:36.816062 (+  7638us) spinlock_profiling.cc:243] Waited 2.42 s on lock 0x7b54000f78d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:36.816119 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:36.816253 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5563279744}
I20260709 13:16:37.210698 12624 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:37.179386 12396 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:37.187983 12313 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:37.263710 12688 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:37.213765 12443 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:37.214018 12448 tablet_bootstrap.cc:492] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:37.124217 12568 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:37.176960 12395 raft_consensus.cc:2764] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:16:37.220268 11510 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 6.
W20260709 13:16:37.220528 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 52508.7us
I20260709 13:16:37.314466 12573 raft_consensus.cc:697] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 3 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:37.326026 12405 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:37.325783 12573 consensus_queue.cc:237] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:37.169622 12569 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:37.326818 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:37.217051 12371 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:37.222702 12494 raft_consensus.cc:697] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:37.195997 12662 consensus_queue.cc:1059] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:16:37.183234 12376 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:37.223526 11478 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:37.180305 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:37.162520 12600 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:37.225543 10822 debug-util.cc:398] Leaking SignalData structure 0x7b08001d82c0 after lost signal to thread 10900
I20260709 13:16:37.202889 12697 consensus_queue.cc:1059] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 13:16:37.219487 12374 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:37.348467 12483 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:37.361217 10811 tablet_copy_service.cc:342] P e01ecd008edd4aa2b6b2b47efbc2ea87: Request end of tablet copy session 12682eb70bc84c76879e04a8628ee586-695ff1b5a68e4111917d909909488d39 received from {username='slave'} at 127.0.0.1:39442
I20260709 13:16:37.361727 10811 tablet_copy_service.cc:434] P e01ecd008edd4aa2b6b2b47efbc2ea87: ending tablet copy session 12682eb70bc84c76879e04a8628ee586-695ff1b5a68e4111917d909909488d39 on tablet 695ff1b5a68e4111917d909909488d39 with peer 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:37.362569 12374 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:37.215147 10947 rpcz_store.cc:276] 0709 13:16:34.197206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:34.197374 (+   168us) service_pool.cc:224] Handling call
0709 13:16:36.745065 (+2547691us) spinlock_profiling.cc:243] Waited 1.49 ms on lock 0x7b5800112598. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd90a22 00007f9c86ca9c09 00007f9c85e03ee0 00007f9c85ecd0ab 00007f9c85ecd131 00007f9c85e8efc0 00007f9c85e79929 00007f9c85e793d2 00007f9c7f3f0bd4
0709 13:16:36.745309 (+   244us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:36.836371 (+ 91062us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:36.836402 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:36.836883 (+   481us) log.cc:844] Serialized 26 byte log entry
0709 13:16:37.029242 (+192359us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:37.029283 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:37.214417 (+185134us) spinlock_profiling.cc:243] Waited 90.7 ms on lock 0x7b5400015ed4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:37.214469 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:37.214522 (+    53us) raft_consensus.cc:1685] finished
0709 13:16:37.214539 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:37.214677 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":53676,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":118,"spinlock_wait_cycles":212051584,"thread_start_us":191971,"threads_started":1,"wal-append.queue_time_us":341895}
I20260709 13:16:37.362991 12483 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:37.363656 12483 raft_consensus.cc:2749] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:16:37.121093 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:37.136293 12672 consensus_queue.cc:1059] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:16:37.230566 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:37.229245 10794 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 3.
I20260709 13:16:37.241668 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:37.243112 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:37.243199 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:37.244477 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:16:37.247308 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 48249.9us
I20260709 13:16:37.374984 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:16:37.247464 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 48506.4us
I20260709 13:16:37.246505 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:37.376163 10792 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:37.376281 10916 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:37.374267 12405 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:37.377621 12405 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:37.378396 12600 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:37.259497 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:37.379633 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:37.372133 12292 consensus_queue.cc:1059] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 13:16:37.380332 10943 raft_consensus.cc:1275] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:16:37.380731 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:37.279894 12685 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:37.384042 10795 raft_consensus.cc:1275] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:37.384100 12396 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:37.286011 10801 raft_consensus.cc:1275] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:37.384900 12396 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:37.385650 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:37.282413 10915 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:37.386718 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:37.386930 12672 consensus_queue.cc:1243] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer 468a8fd46e824f62b9c2c0d3532a6d26 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 13:16:37.387141 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 104827us
I20260709 13:16:37.387765 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:37.388921 10801 raft_consensus.cc:1240] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:37.390050 10801 raft_consensus.cc:1275] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:16:37.289717 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:37.391248 10950 raft_consensus.cc:2393] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 3: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:37.392091 12471 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:37.392282 11511 raft_consensus.cc:3055] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:37.291945 11507 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
I20260709 13:16:37.392817 10801 raft_consensus.cc:1240] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:37.204670 12683 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:37.197263 12676 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:37.306368 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:37.210918 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:16:37.312551 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:37.207726 12617 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:37.285521 12622 raft_consensus.cc:2804] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:37.398561 10801 raft_consensus.cc:1275] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:37.399010 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:37.399825 10875 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 4.
I20260709 13:16:37.318923 10949 raft_consensus.cc:2410] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:16:37.321043 12448 tablet_bootstrap.cc:492] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586: Bootstrap complete.
I20260709 13:16:37.317634 12688 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:37.332628 12371 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:37.332710 12569 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:37.300128 12644 consensus_queue.cc:1048] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:16:37.333483 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:37.336082 10799 raft_consensus.cc:1275] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:37.418612 10799 raft_consensus.cc:1275] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:16:37.419677 10799 raft_consensus.cc:1275] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:37.299996 12661 consensus_queue.cc:1059] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:16:37.349148 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:16:37.351724 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 128253us
I20260709 13:16:37.324113 12612 raft_consensus.cc:2749] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:37.331573 10796 raft_consensus.cc:1240] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:37.350045 12494 consensus_queue.cc:237] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:37.414995 10822 debug-util.cc:398] Leaking SignalData structure 0x7b080027f1c0 after lost signal to thread 10974
I20260709 13:16:37.364056 10947 raft_consensus.cc:1275] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:37.310457 12624 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:37.366899 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:37.375434 10841 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:37.376058 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:37.248085 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:37.379817 11503 raft_consensus.cc:2368] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 4. Current term is 5.
I20260709 13:16:37.380185 10945 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:16:37.328603 12568 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:37.381322 10794 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 4.
W20260709 13:16:37.283936 12698 log.cc:927] Time spent T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 13:16:37.284966 10793 raft_consensus.cc:1275] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:16:37.257084 10872 raft_consensus.cc:1275] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:37.476621 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:37.477402 10872 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:37.382182 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:37.382987 10916 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:37.479319 10797 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 4.
W20260709 13:16:37.479565 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 96674.6us
I20260709 13:16:37.383390 10943 raft_consensus.cc:1275] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:16:37.372401 12395 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:37.392751 11511 raft_consensus.cc:740] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:37.379653 12405 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:37.408857 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:37.409646 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:37.409763 12396 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:37.410288 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:37.402570 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:37.402417 10767 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:37.410539 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:37.514751 10870 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 2.
W20260709 13:16:37.514775 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 112445us
I20260709 13:16:37.400373 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:37.515738 10873 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
I20260709 13:16:37.407665 10792 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
I20260709 13:16:37.410597 10869 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 5.
I20260709 13:16:37.413093 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:37.416044 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:37.413039 12688 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:37.394549 12590 raft_consensus.cc:2749] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:37.422252 10946 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:37.360697 12697 raft_consensus.cc:3055] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:37.399302 12622 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:37.426731 10796 raft_consensus.cc:1240] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:37.549437 12569 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:37.227847 10766 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:37.411664 12605 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:37.410255 12676 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:16:37.430779 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 55428.3us
I20260709 13:16:37.582886 12289 raft_consensus.cc:2749] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:16:37.430840 10798 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 3.
I20260709 13:16:37.423185 12448 ts_tablet_manager.cc:1403] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586: Time spent bootstrapping tablet: real 0.595s	user 0.022s	sys 0.026s
I20260709 13:16:37.428715 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:37.600539 10941 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 4.
I20260709 13:16:37.439981 11505 raft_consensus.cc:2410] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:16:37.602811 12539 raft_consensus.cc:3055] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:37.603376 12539 raft_consensus.cc:740] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:37.440320 12617 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:37.413698 12313 consensus_queue.cc:1059] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 13:16:37.441025 12685 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:37.442209 12683 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:37.442430 12574 raft_consensus.cc:3055] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:37.412178 12599 consensus_queue.cc:1059] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 13:16:37.623162 12574 raft_consensus.cc:740] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:37.623855 12599 raft_consensus.cc:3055] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:37.624297 12599 raft_consensus.cc:740] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:37.624368 12574 consensus_queue.cc:260] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:37.625156 12599 consensus_queue.cc:260] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:37.625677 12574 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:37.626390 12599 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:37.429253 10822 debug-util.cc:398] Leaking SignalData structure 0x7b080025c940 after lost signal to thread 10975
I20260709 13:16:37.480590 10916 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:16:37.627431 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 146954us
I20260709 13:16:37.628360 10916 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:37.414217 12600 consensus_queue.cc:1048] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 13:16:37.636583 10946 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 6.
I20260709 13:16:37.492347 12662 raft_consensus.cc:3055] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:37.508699 10945 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 5.
I20260709 13:16:37.495563 11511 consensus_queue.cc:260] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:37.547115 12654 raft_consensus.cc:3055] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:37.536676 10801 raft_consensus.cc:1275] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:16:37.493996 12374 raft_consensus.cc:2804] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:37.513787 10868 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 2.
I20260709 13:16:37.564419 10872 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 5.
I20260709 13:16:37.499593 12471 raft_consensus.cc:697] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 5 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:37.426887 12701 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:37.585157 10841 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:37.537642 10874 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
I20260709 13:16:37.509521 12395 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:37.664878 11511 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:37.585448 12622 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:37.573892 12697 raft_consensus.cc:740] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:16:37.747785 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 162716us
I20260709 13:16:37.742727 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:37.748878 10841 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:37.665511 11478 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:16:37.750193 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:37.750108 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 84651us
W20260709 13:16:37.750649 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:37.751540 11478 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:16:37.679188 12709 log.cc:927] Time spent T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
I20260709 13:16:37.605777 12569 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:37.752671 12569 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:16:37.752913 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 525126us
I20260709 13:16:37.753859 12396 consensus_queue.cc:1048] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:37.627579 12356 raft_consensus.cc:993] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26: : Instructing follower cb0428cc55984daba1ba63744a5102a2 to start an election
I20260709 13:16:37.755668 10766 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:37.757041 10766 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:37.758286 12289 raft_consensus.cc:2749] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:37.762308 12289 consensus_queue.cc:1059] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:37.763339 12583 raft_consensus.cc:3055] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:37.763765 12583 raft_consensus.cc:740] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:37.763612 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:16:37.765048 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:37.737864 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 69583.9us
I20260709 13:16:37.767865 10766 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:37.762416 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:37.770134 10870 raft_consensus.cc:2410] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:16:37.627754 12685 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:37.779208 10797 raft_consensus.cc:1275] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:37.776858 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:37.773440 11477 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:37.781241 11507 raft_consensus.cc:2393] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 6: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:37.780416 10794 raft_consensus.cc:1240] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
W20260709 13:16:37.740422 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:37.783056 10793 raft_consensus.cc:1275] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:16:37.679809 12654 raft_consensus.cc:740] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:37.768532 10767 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:37.738441 12374 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:37.691505 12662 raft_consensus.cc:740] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:37.812105 12405 raft_consensus.cc:2804] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:37.812286 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:37.812677 12405 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:37.813133 12405 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:37.815394 12395 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:37.770116 12443 raft_consensus.cc:2804] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:37.787817 10841 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:37.789867 10766 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:37.792445 12483 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:37.785053 12683 raft_consensus.cc:1081] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 LEADER]: Signalling peer cb0428cc55984daba1ba63744a5102a2 to start an election
I20260709 13:16:37.680102 12590 raft_consensus.cc:2764] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:16:37.737977 12573 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:37.802230 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:37.765920 12617 consensus_queue.cc:1059] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
W20260709 13:16:37.818926 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 45565.5us
I20260709 13:16:37.820019 12371 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:37.802580 12356 consensus_queue.cc:1048] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 13:16:37.775760 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:37.828068 12483 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:37.815223 12471 consensus_queue.cc:237] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:37.604476 12539 consensus_queue.cc:260] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:37.826941 10841 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:37.828958 10766 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:16:37.903757 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 74883.8us
I20260709 13:16:37.904834 10766 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:37.797215 12448 raft_consensus.cc:359] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:37.764750 12583 consensus_queue.cc:260] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:37.828105 12694 raft_consensus.cc:2764] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:16:37.770545 12605 consensus_queue.cc:1048] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.052s
I20260709 13:16:37.775031 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:37.833854 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:37.747918 12616 consensus_queue.cc:1048] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.076s
I20260709 13:16:37.838964 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:37.839143 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:37.839773 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:37.840142 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:37.845965 10876 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 4.
I20260709 13:16:37.817117 12624 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:37.806504 12693 raft_consensus.cc:2804] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:37.897657 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:37.823060 12374 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:37.855916 11477 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:37.828768 12443 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:37.959141 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 103346us
I20260709 13:16:37.784093 12289 raft_consensus.cc:2749] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:37.769963 12701 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:37.882347  9687 debug-util.cc:398] Leaking SignalData structure 0x7b080000fee0 after lost signal to thread 10767
I20260709 13:16:37.875521 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:37.908751 12448 raft_consensus.cc:740] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Initialized, Role: LEARNER
I20260709 13:16:37.833472 12313 raft_consensus.cc:2749] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:37.916692 12568 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:37.786388 12685 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:37.852727 12396 raft_consensus.cc:2749] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:37.980332 12443 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:37.845423 12573 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:16:37.912580 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 143632us
I20260709 13:16:38.010133 10916 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
W20260709 13:16:37.912810 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 85945.7us
I20260709 13:16:37.831077 12405 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:37.922364 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:37.896798 12483 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:37.951736 10792 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
W20260709 13:16:37.918156 10822 debug-util.cc:398] Leaking SignalData structure 0x7b0800235c60 after lost signal to thread 11536
I20260709 13:16:37.952234 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:37.864748 12600 consensus_queue.cc:1048] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.098s
I20260709 13:16:37.953290 10797 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:38.023195 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:37.953950 10798 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 3.
I20260709 13:16:37.893821 12648 consensus_queue.cc:1048] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:16:37.920727 12683 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:37.775447 12612 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:37.916370 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:37.776968 11478 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:16:38.076009 12605 raft_consensus.cc:3055] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:37.911123 11511 raft_consensus.cc:1275] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:16:38.006219 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:37.955961 12693 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:37.981070 12583 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:38.014544 10841 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:37.898684 12654 consensus_queue.cc:260] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:37.995005 12395 raft_consensus.cc:2749] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:37.995623 12624 raft_consensus.cc:2749] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:38.020653 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:37.997133 12671 raft_consensus.cc:2749] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:16:38.028659 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 123881us
I20260709 13:16:38.012969 12573 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:37.908934 12662 consensus_queue.cc:260] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:38.109355 12719 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:38.117153 12662 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:38.111773 12693 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:38.117309 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:38.118037 10792 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
I20260709 13:16:38.008877 12289 raft_consensus.cc:2749] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:37.876353 12569 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:38.021649 10822 debug-util.cc:398] Leaking SignalData structure 0x7b080028bc20 after lost signal to thread 11537
I20260709 13:16:38.121001 12569 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:38.073789 10797 raft_consensus.cc:2393] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 5: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:38.122416 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:38.122957  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:38.100255 12405 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:38.106070 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 329156us
W20260709 13:16:38.004231 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:38.108326 10793 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:38.087776 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:16:38.110260 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 95805.8us
I20260709 13:16:38.098232 12720 raft_consensus.cc:2804] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:38.114367 10766 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:38.116712 12371 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:38.114862 12356 raft_consensus.cc:2749] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:38.070358 10796 raft_consensus.cc:2410] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 5 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:16:38.091511 12683 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:38.123118 10794 raft_consensus.cc:2393] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 5: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:38.131458 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:38.182106 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:38.182703 10792 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
I20260709 13:16:38.152868 10798 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:38.153309 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:38.154397 11478 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:16:38.111615 12672 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
W20260709 13:16:38.112927 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 102933us
I20260709 13:16:38.117235 12719 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:38.123195 10797 raft_consensus.cc:2393] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 3: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:38.107842 12605 raft_consensus.cc:740] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:38.190135 12605 consensus_queue.cc:260] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:38.191285 12688 consensus_meta.cc:287] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.610s	user 0.011s	sys 0.000s
I20260709 13:16:38.191779 12605 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:38.191743 12688 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:38.106928 12612 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:38.193049 12612 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:38.193955 12688 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:38.180030 12494 raft_consensus.cc:3055] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:38.194839 12494 raft_consensus.cc:740] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:38.195755 12494 consensus_queue.cc:260] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:38.196779 12494 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:38.168264 10841 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:38.174531 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:38.072491 12448 consensus_queue.cc:260] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:38.174955 10871 raft_consensus.cc:1240] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:38.201025 10841 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:16:38.156634 10801 raft_consensus.cc:2393] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 2: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
W20260709 13:16:38.201350 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:38.184387 10793 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
W20260709 13:16:38.184285 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 70000.2us
I20260709 13:16:38.149117 12654 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:38.186833 12683 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:16:38.139580 12722 log.cc:927] Time spent T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:16:38.187487 11478 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
W20260709 13:16:38.205040 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:38.248171 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:38.147827 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:38.156800 12720 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:38.212467 12443 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:38.123155 12313 consensus_queue.cc:1048] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.064s
W20260709 13:16:38.179986 12679 log.cc:927] Time spent T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 13:16:38.223958 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:38.284726 10871 raft_consensus.cc:2393] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 4: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:38.285882 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:38.224936 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:38.225483 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:38.287413 10873 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 4.
I20260709 13:16:38.287971 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:38.288589 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:38.290095 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:38.229081 10841 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:38.291397 12687 raft_consensus.cc:2764] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:16:38.292115 12573 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:38.249075 10949 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 2.
I20260709 13:16:38.249603 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:38.293835 10947 raft_consensus.cc:2393] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 5: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
W20260709 13:16:38.250113 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:38.250542 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:16:38.252224 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 64805.2us
I20260709 13:16:38.253535 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:38.259856 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:38.264282 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:16:38.199734 12724 log.cc:927] Time spent T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
W20260709 13:16:38.258270 12729 log.cc:927] Time spent T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
W20260709 13:16:38.267760 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:38.263192 12684 raft_consensus.cc:2804] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:16:38.268399 12730 log.cc:927] Time spent T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:16:38.268700 12483 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:38.375245 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:38.202921 12736 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:38.259877 12688 raft_consensus.cc:2749] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:38.180648 10842 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:38.377887 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:38.378600 10792 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 3.
W20260709 13:16:38.378731 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 203972us
I20260709 13:16:38.379563 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:38.379644 10842 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:38.380564 10842 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:38.377899 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:38.384943 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:16:38.385363 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:38.386337 10767 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:38.382110 10947 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 4.
I20260709 13:16:38.387919 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:16:38.292572 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 63573us
I20260709 13:16:38.210426 12622 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:38.200753 12719 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:38.294294 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:38.391286 10949 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
I20260709 13:16:38.251792 12614 consensus_queue.cc:1048] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.119s
I20260709 13:16:38.371133 10796 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 3.
I20260709 13:16:38.307071 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:38.305869 12648 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:38.266363 12720 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:38.218667 12723 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:38.373015 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:38.298228 10873 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:38.373082 11478 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
W20260709 13:16:38.373903 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:38.436077 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 63120us
I20260709 13:16:38.390064 12356 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:38.374121 12684 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:38.237718 10916 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.116.129:39685 timed out after 1.516s (SENT) [suppressed 14 similar messages]
W20260709 13:16:38.371959 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:38.393383 12687 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:16:38.375784 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:38.403041 12694 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:38.376049 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:38.403337 12648 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:38.371691 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:38.304683 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:16:38.389390 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:38.391388 10767 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:38.393523 10841 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:38.394079 10948 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 4.
I20260709 13:16:38.395318 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:38.395946 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:38.401029 12701 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:38.371189 10950 raft_consensus.cc:2393] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 2: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:38.412397 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:38.305126 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:38.440743 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:38.442022 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:38.442826 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:38.443608 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:38.435130 12723 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:38.395469 12569 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:38.315169 10795 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
I20260709 13:16:38.271473 12697 consensus_queue.cc:260] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:38.452572 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:38.404247 12736 raft_consensus.cc:697] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:38.376121 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:38.448608 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:38.454291 10944 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
I20260709 13:16:38.454545 10915 leader_election.cc:400] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 4. Current term is 5.
I20260709 13:16:38.460723 10766 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:38.469751 10873 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 6.
W20260709 13:16:38.526002 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 73508.7us
W20260709 13:16:38.526958 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 66333.4us
I20260709 13:16:38.527529 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:38.451308 12684 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:38.512503 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:38.512916 10799 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 4.
I20260709 13:16:38.513746 10942 raft_consensus.cc:2410] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 5 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:16:38.514199 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 122889us
W20260709 13:16:38.516911 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 123454us
I20260709 13:16:38.536566 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:38.537338 10799 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 3.
I20260709 13:16:38.536819 10841 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:38.512326 10793 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 3.
I20260709 13:16:38.437201 10943 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
I20260709 13:16:38.528208 10873 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 4.
I20260709 13:16:38.540922 11478 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:16:38.455716 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:38.530897 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:38.531507 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:38.446439 12725 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:38.455524 10941 raft_consensus.cc:1240] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:16:38.521574 12395 raft_consensus.cc:2764] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:16:38.454221 12662 consensus_queue.cc:1048] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.086s
I20260709 13:16:38.520481 10869 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 1.
I20260709 13:16:38.449052 12687 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:38.457871 12694 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:38.483125 12648 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:38.453451 12289 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:38.571321 12289 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:38.573544 12289 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:38.574209 12736 consensus_queue.cc:237] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:38.513015 12612 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:38.548247 12644 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:38.517280 12723 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:38.528982 10766 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:38.548993 10873 raft_consensus.cc:1275] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:38.515578 12569 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:38.537467 10767 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:16:38.582194 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 44807.7us
I20260709 13:16:38.583534 10767 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:38.407145 12683 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:38.536484 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:38.557142 12697 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:38.561771 10871 raft_consensus.cc:1275] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:38.540474 12313 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:38.522708 12688 consensus_queue.cc:1048] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 13:16:38.569687 12694 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:38.532127 12701 raft_consensus.cc:2749] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:38.565636 11504 raft_consensus.cc:2410] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:16:38.570536 12685 consensus_meta.cc:287] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.556s	user 0.009s	sys 0.002s
I20260709 13:16:38.567080 12687 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:38.586551 12569 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:38.597926 12289 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:38.603507 11510 raft_consensus.cc:2393] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 5: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
W20260709 13:16:38.587675 12745 log.cc:927] Time spent T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:16:38.575057 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:38.453704 12624 consensus_queue.cc:1048] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.092s
I20260709 13:16:38.607103 10875 raft_consensus.cc:1240] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:38.579852 12644 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:38.593736 12395 raft_consensus.cc:2804] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:16:38.601632 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 363924us
I20260709 13:16:38.623425 10945 raft_consensus.cc:1240] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 5 LEADER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 4. Current term is 5. Ops: []
W20260709 13:16:38.634501 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 97768.2us
W20260709 13:16:38.622221 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 81368.5us
I20260709 13:16:38.611119 12443 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:38.602587 12693 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:38.625677 12448 ts_tablet_manager.cc:1434] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586: Time spent starting tablet: real 1.041s	user 0.015s	sys 0.013s
I20260709 13:16:38.672238 10944 raft_consensus.cc:1275] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:16:38.626663 12573 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:38.624678 12289 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:38.603773 12313 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:38.611992 12685 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:38.695307 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:38.659835 12644 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:38.668490 12725 raft_consensus.cc:697] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 5 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:38.659955 12395 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:38.697909 10944 raft_consensus.cc:1275] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:38.627700 12483 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:38.662186 10795 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:16:38.699661 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:38.692840 12448 ts_tablet_manager.cc:1467] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586: Tablet startup took 1107ms
I20260709 13:16:38.700803 10870 raft_consensus.cc:1275] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:16:38.701220 12395 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:38.717944 10870 raft_consensus.cc:1240] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:38.696480 12662 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:38.696907 12289 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:38.724922 10876 raft_consensus.cc:1240] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:38.700301 12443 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:38.698908 12622 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:16:38.719421 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 188924us
W20260709 13:16:38.718106 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 324368us
I20260709 13:16:38.730679 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:38.731786 10766 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:38.720727 10870 raft_consensus.cc:1240] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:16:38.716517 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:16:38.715698 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 131037us
I20260709 13:16:38.728418 10868 raft_consensus.cc:1240] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:38.734002 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:38.733774 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:38.734678 10942 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 4.
I20260709 13:16:38.735152 10797 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 4.
I20260709 13:16:38.717216 12405 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:38.736281 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:38.717449 12749 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:38.736210 10766 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:38.740756 10797 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 2.
I20260709 13:16:38.727367 12662 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:38.722551 12755 raft_consensus.cc:2749] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:38.744269 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:38.778652 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:38.735746 12573 consensus_queue.cc:1059] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
W20260709 13:16:38.818361 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:38.820528 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:38.821149 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:38.736825 12614 consensus_queue.cc:1059] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.047s
I20260709 13:16:38.828831 10797 raft_consensus.cc:1240] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:38.742046 12622 consensus_queue.cc:1048] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
W20260709 13:16:38.834009 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 100749us
W20260709 13:16:38.785409 12744 log.cc:927] Time spent T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.205s	user 0.001s	sys 0.000s
I20260709 13:16:38.744678 12405 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:38.783856 12756 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:38.750303 12749 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:38.782230 12750 raft_consensus.cc:2749] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:16:38.816614 12683 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:16:38.835328 12742 log.cc:927] Time spent T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.260s	user 0.001s	sys 0.000s
I20260709 13:16:38.751207 12733 consensus_queue.cc:1059] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.087s
I20260709 13:16:38.788305 12471 consensus_queue.cc:1048] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.096s
W20260709 13:16:38.908246 10915 leader_election.cc:336] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Timed out: RequestConsensusVote RPC to 127.9.116.132:40555 timed out after 1.503s (SENT)
W20260709 13:16:38.841192 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 100579us
I20260709 13:16:38.841830 12758 raft_consensus.cc:2749] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:38.887934 12756 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:38.912925 11504 raft_consensus.cc:1275] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:16:38.909477 12443 raft_consensus.cc:697] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:38.950176 10798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 1681) took 1099 ms. Trace:
I20260709 13:16:38.950721 10798 rpcz_store.cc:276] 0709 13:16:37.850600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:38.184412 (+333812us) service_pool.cc:224] Handling call
0709 13:16:38.184730 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:38.188341 (+  3611us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:38.188372 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:38.189721 (+  1349us) log.cc:844] Serialized 26 byte log entry
0709 13:16:38.261200 (+ 71479us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:38.261258 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:38.261348 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:38.949883 (+688535us) raft_consensus.cc:1685] finished
0709 13:16:38.949912 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:38.950097 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3268,"raft.queue_time_us":4327,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":84,"thread_start_us":71101,"threads_started":1,"wal-append.queue_time_us":339779}
I20260709 13:16:38.817111 12725 consensus_queue.cc:237] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:38.963390 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 220017us
I20260709 13:16:38.923379 12756 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:16:38.984355 12746 log.cc:927] Time spent T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.211s	user 0.000s	sys 0.001s
I20260709 13:16:38.983888 12443 consensus_queue.cc:237] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:38.985199 12759 log.cc:927] Time spent T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
W20260709 13:16:38.986238 12356 consensus_meta.cc:287] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.537s	user 0.007s	sys 0.000s
W20260709 13:16:38.993342 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 151535us
I20260709 13:16:38.927629 12750 consensus_queue.cc:1048] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.056s
I20260709 13:16:38.932723 12685 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:38.923504 12683 consensus_queue.cc:1048] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.055s
I20260709 13:16:39.010622 12648 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:39.016409 12648 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:39.016628 12395 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:39.012696 10842 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:16:39.004057 12747 log.cc:927] Time spent T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.223s	user 0.000s	sys 0.001s
I20260709 13:16:38.921470 12754 consensus_queue.cc:1048] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.088s
W20260709 13:16:39.034403 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 300079us
I20260709 13:16:38.975838 12313 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:16:39.006949 12599 consensus_meta.cc:287] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 1.380s	user 0.016s	sys 0.000s
I20260709 13:16:38.989765 12662 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:38.987986 12733 consensus_queue.cc:1048] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.047s
I20260709 13:16:39.028168 12356 raft_consensus.cc:2749] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:38.988969 12684 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:39.039036 12574 consensus_meta.cc:287] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 1.413s	user 0.014s	sys 0.005s
W20260709 13:16:39.047868 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 311720us
I20260709 13:16:39.004817 12371 consensus_queue.cc:1059] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 13:16:39.053939 10766 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:39.006556 12590 consensus_queue.cc:1048] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
W20260709 13:16:39.057376 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 44824.1us
I20260709 13:16:39.057895 10766 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:39.004566 12761 consensus_queue.cc:1048] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.070s
W20260709 13:16:39.052343 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 47062.4us
I20260709 13:16:39.060307 10916 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:39.066264 10842 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:39.036257 12622 consensus_queue.cc:1048] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 13:16:39.069384 12771 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:39.071645 10916 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:39.062179 12736 raft_consensus.cc:2804] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:39.039863 12289 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:39.074348 10916 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:39.076843 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:16:39.077939 12762 log.cc:927] Time spent T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
I20260709 13:16:39.078327 10842 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:39.078986 12683 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:39.079667 10916 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:39.081221 10916 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:39.077098 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:16:39.071203 12605 consensus_meta.cc:287] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.879s	user 0.010s	sys 0.006s
I20260709 13:16:39.083076 10766 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:39.063205 12776 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:39.081991 12371 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:39.082219 12766 consensus_queue.cc:1048] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 13:16:39.092789 10842 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:39.096691 10766 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:39.092101 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:39.117167 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:39.075781 12736 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:39.117426 10798 raft_consensus.cc:2376] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 5. Re-sending same reply.
I20260709 13:16:39.163105 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:16:39.137945 12569 consensus_meta.cc:287] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.512s	user 0.007s	sys 0.000s
I20260709 13:16:39.152132 12405 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:39.146323 12612 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:16:39.132445 12494 consensus_meta.cc:287] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.935s	user 0.012s	sys 0.000s
W20260709 13:16:39.132323 12694 consensus_meta.cc:287] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.521s	user 0.010s	sys 0.000s
I20260709 13:16:39.129783 10795 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 6.
I20260709 13:16:39.127648 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:39.226816 10797 raft_consensus.cc:2368] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 4. Current term is 5.
I20260709 13:16:39.094317 12573 raft_consensus.cc:2804] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:39.228423 10797 raft_consensus.cc:1240] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 13:16:39.152263 12697 consensus_meta.cc:287] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.553s	user 0.006s	sys 0.005s
W20260709 13:16:39.102389 12726 log.cc:927] Time spent T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 1.023s	user 0.004s	sys 0.001s
W20260709 13:16:39.149233 10794 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 1678) took 1299 ms (client timeout 1561 ms). Trace:
W20260709 13:16:39.230279 10794 rpcz_store.cc:269] 0709 13:16:37.849773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:37.922050 (+ 72277us) service_pool.cc:224] Handling call
0709 13:16:39.149030 (+1226980us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b54000d9654. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:39.149175 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":252403072}
I20260709 13:16:39.230904 10800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 142) took 1899 ms. Trace:
W20260709 13:16:39.116313 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:39.148138 12749 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:39.138166 12539 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:39.231329 10800 rpcz_store.cc:276] 0709 13:16:37.331683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:37.510033 (+178350us) service_pool.cc:224] Handling call
0709 13:16:37.769001 (+258968us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:37.803235 (+ 34234us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:16:37.817080 (+ 13845us) log.cc:844] Serialized 14 byte log entry
0709 13:16:38.098210 (+281130us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:38.098263 (+    53us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:38.098291 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:39.230657 (+1132366us) spinlock_profiling.cc:243] Waited 34 ms on lock 0x7b540026b954. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:39.230704 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:39.230847 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":14159,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":10311,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":3033,"spinlock_wait_cycles":78202496,"thread_start_us":274908,"threads_started":1,"wal-append.queue_time_us":370057,"wal-append.run_cpu_time_us":1001,"wal-append.run_wall_time_us":1001024}
I20260709 13:16:39.209307 12605 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:39.130198 12289 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:39.127421 12773 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:39.116302 12776 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:39.254992 12773 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:39.255522 12773 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:16:39.128386 12654 consensus_meta.cc:287] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.875s	user 0.007s	sys 0.005s
W20260709 13:16:39.133581 12374 consensus_meta.cc:287] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 1.135s	user 0.008s	sys 0.004s
I20260709 13:16:39.085291 12771 raft_consensus.cc:697] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:39.127895 10766 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:16:39.156656 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 63907.6us
I20260709 13:16:39.209312 10792 raft_consensus.cc:2393] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 4: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:39.209241 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:39.093880 12750 raft_consensus.cc:2749] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:39.091908 12770 raft_consensus.cc:2804] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:16:39.105221 12723 consensus_meta.cc:287] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.508s	user 0.007s	sys 0.003s
I20260709 13:16:39.109067 12313 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:39.269816 11505 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 5.
I20260709 13:16:39.099208 12684 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:39.114347 12371 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:39.102237 12395 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:39.116086 12574 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:39.213879 12590 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:39.278656 12371 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:39.279173 12590 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:39.165248 12736 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:39.281565 12590 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:39.213587 12733 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:39.254354 10941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 1327) took 1007 ms. Trace:
I20260709 13:16:39.262068 10842 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:39.284087 12733 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:16:39.284683 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 1.32463e+06us
I20260709 13:16:39.285708 10842 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:39.286656 10875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 190) took 1864 ms. Trace:
I20260709 13:16:39.273828 12777 raft_consensus.cc:2764] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:16:39.287616 12777 raft_consensus.cc:2804] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:39.287186 10875 rpcz_store.cc:276] 0709 13:16:37.422240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:37.959162 (+536922us) service_pool.cc:224] Handling call
0709 13:16:37.988107 (+ 28945us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:38.661260 (+673153us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:39.286432 (+625172us) spinlock_profiling.cc:243] Waited 619 ms on lock 0x7b540009b0d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:39.286596 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1423115520}
I20260709 13:16:39.288134 12684 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:39.288414 12617 raft_consensus.cc:3055] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:39.288640 12684 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:39.288923 12617 raft_consensus.cc:740] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:39.289152 12684 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:39.289779 12617 consensus_queue.cc:260] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:39.288170 12777 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:39.266166 12770 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:39.291314 12777 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:39.291474 12770 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:16:39.262710 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 134879us
I20260709 13:16:39.262487 10798 raft_consensus.cc:2376] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 5. Re-sending same reply.
I20260709 13:16:39.293278 10842 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:39.235688 12573 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:39.294445 10766 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:39.270035 12313 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:39.091116 12775 raft_consensus.cc:2804] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:16:39.241101 12687 consensus_meta.cc:287] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.616s	user 0.008s	sys 0.004s
I20260709 13:16:39.214854 12569 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:39.261396 12644 consensus_meta.cc:287] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.544s	user 0.010s	sys 0.002s
I20260709 13:16:39.273933 12776 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:39.279115 12574 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:38.714835 12448 ts_tablet_manager.cc:1469] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586: Trace:
0709 13:16:28.968710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:28.968886 (+   176us) service_pool.cc:224] Handling call
0709 13:16:29.541369 (+572483us) ts_tablet_manager.cc:934] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586: Initiating tablet copy from peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
0709 13:16:30.053130 (+511761us) spinlock_profiling.cc:243] Waited 22.6 ms on lock 0x7b4c000d05c0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7f181a20 00007f9c7f192a29 00007f9c7f19313f 00007f9c7f1572ab 00007f9c7f184f3c 00007f9c7f185461 00007f9c7f185768 00007f9c84f0b2eb 00007f9c85e30d3b 00007f9c85e2b008 00007f9c85e2a405
0709 13:16:32.509353 (+2456223us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:16:33.026235 (+516882us) tablet_metadata.cc:665] Metadata flushed
0709 13:16:33.026387 (+   152us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:16:33.339756 (+313369us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:16:33.340121 (+   365us) tablet_replica.cc:206] Creating consensus instance
0709 13:16:33.340567 (+   446us) ts_tablet_manager.cc:989] Replica registered.
0709 13:16:33.340777 (+   210us) inbound_call.cc:177] Queueing success response
0709 13:16:36.828142 (+3487365us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:16:37.584319 (+756177us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:16:37.759335 (+175016us) tablet_replica.cc:261] Starting instrumentation
0709 13:16:37.760199 (+   864us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":118364,"mutex_wait_us":30227,"raft.queue_time_us":25267,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":40829,"spinlock_wait_cycles":51867520,"tablet-copy.queue_time_us":455250,"tablet-download-pool.queue_time_us":15311,"tablet-download-pool.run_cpu_time_us":2010,"tablet-download-pool.run_wall_time_us":564541,"thread_start_us":500302,"threads_started":2}
I20260709 13:16:39.284657 12733 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:39.290670 12617 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:39.303637 10871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1309) took 1022 ms. Trace:
I20260709 13:16:39.303550 10766 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:39.225946 12694 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:39.271875 12289 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:39.302973 12750 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:39.306078 12671 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:39.295147 10842 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:39.263988 12374 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:39.300587 12313 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:39.331892 10842 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:39.300470 12573 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:39.302932 12775 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:39.303212 12395 raft_consensus.cc:2749] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:39.291241 12771 consensus_queue.cc:237] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:39.337425 12697 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:39.330912 12405 consensus_queue.cc:1048] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
W20260709 13:16:39.339071 12751 log.cc:927] Time spent T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.415s	user 0.001s	sys 0.000s
I20260709 13:16:39.336318 12761 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:39.340682 12761 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:39.341041 12761 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:16:39.341902 12720 consensus_meta.cc:287] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.897s	user 0.012s	sys 0.000s
I20260709 13:16:39.330826 12750 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:39.343179 12750 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:39.343734 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:39.343950 12569 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:39.344592 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:39.344959 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:39.345256 11504 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 3.
I20260709 13:16:39.308636 12644 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:39.321631 10871 rpcz_store.cc:276] 0709 13:16:38.280843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:38.598526 (+317683us) service_pool.cc:224] Handling call
0709 13:16:38.608790 (+ 10264us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:38.754002 (+145212us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:38.754048 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:38.770614 (+ 16566us) log.cc:844] Serialized 26 byte log entry
0709 13:16:39.292697 (+522083us) spinlock_profiling.cc:243] Waited 1.07 ms on lock 0x7b1800324dc0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c80a5d4bc 00007f9c7ddab48b 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a550d1
0709 13:16:39.292855 (+   158us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:39.292890 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:39.292961 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:39.293000 (+    39us) raft_consensus.cc:1685] finished
0709 13:16:39.293011 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:39.303580 (+ 10569us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5576,"raft.queue_time_us":300353,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":10365,"spinlock_wait_cycles":2462592,"thread_start_us":300839,"threads_started":2,"wal-append.queue_time_us":214153}
I20260709 13:16:39.269819 12723 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:39.304796 12612 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:39.328089 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:39.328706 10766 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:39.284116 10941 rpcz_store.cc:276] 0709 13:16:38.246614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:38.258263 (+ 11649us) service_pool.cc:224] Handling call
0709 13:16:38.267920 (+  9657us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:38.550619 (+282699us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:39.254027 (+703408us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b54003f0054. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:39.254280 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":363612928}
I20260709 13:16:39.303421 12672 consensus_queue.cc:1048] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 13:16:39.330395 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:39.329293 12396 raft_consensus.cc:2749] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:39.356101 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:39.356913 11507 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:39.342315 12720 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:39.267645 12756 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:39.336683 10842 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:39.348910 10947 raft_consensus.cc:1240] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:16:39.329584 12289 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:39.370930 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:39.365114 12574 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:39.347787 12671 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:39.351327 10841 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:39.302877 12749 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:16:39.352329 10801 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 1684) took 1500 ms (client timeout 1710 ms). Trace:
I20260709 13:16:39.304525 12443 consensus_queue.cc:1048] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.027s
W20260709 13:16:39.353991 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 272825us
W20260709 13:16:39.310351 12719 consensus_meta.cc:287] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.898s	user 0.010s	sys 0.002s
I20260709 13:16:39.353585 12612 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:39.359236 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:39.355674 12569 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:39.361326 12599 raft_consensus.cc:3055] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:39.361938 12614 raft_consensus.cc:3055] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:39.347483 12775 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:39.372939 12688 raft_consensus.cc:2804] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:39.307304 12687 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:39.376008 10801 rpcz_store.cc:269] 0709 13:16:37.851494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:37.951962 (+100468us) service_pool.cc:224] Handling call
0709 13:16:39.352132 (+1400170us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b54003ccb54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:39.352276 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":294400768}
I20260709 13:16:39.380803 10916 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:39.380916 10842 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:39.373656 12374 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:39.359632 12644 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:39.388321 12599 raft_consensus.cc:740] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:39.373979 12289 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:39.388784 12614 raft_consensus.cc:740] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:39.419683 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:39.364395 12371 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:39.400723 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:39.401552 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:39.387279 12443 consensus_queue.cc:1243] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer 12682eb70bc84c76879e04a8628ee586 log is divergent from this leader: its last log entry 3.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:16:39.401604 10916 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:39.426383 10916 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:39.416205 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:39.427561 10801 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:39.380297 12766 raft_consensus.cc:2804] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:39.392819 12688 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:39.421463 10872 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:39.438071 10916 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:39.425004 12777 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:39.382754 12749 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:39.450163 11507 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
I20260709 13:16:39.449199 10916 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:39.453356 10947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 1340) took 1056 ms. Trace:
I20260709 13:16:39.457175 10916 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:39.456651 10947 rpcz_store.cc:276] 0709 13:16:38.397243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:38.397374 (+   131us) service_pool.cc:224] Handling call
0709 13:16:38.397692 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:39.373835 (+976143us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:39.452996 (+ 79161us) spinlock_profiling.cc:243] Waited 951 ms on lock 0x7b540004e054. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:39.453279 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2187043712}
I20260709 13:16:39.458635 10841 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:16:39.459185 11507 pending_rounds.cc:85] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586: Aborting all ops after (but not including) 4
I20260709 13:16:39.458992 12590 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:39.404129 12787 raft_consensus.cc:2749] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:39.459671 11507 pending_rounds.cc:107] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:16:39.483228 11507 raft_consensus.cc:2889] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:39.404596 12356 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:39.426167 12685 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:39.389163 12770 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:39.488989 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:39.431581 12766 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:39.465895 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:39.488989 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:39.460299 10916 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:39.489430 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:39.392555 12773 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:39.392282 12719 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:39.493490 10870 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:39.484616 12622 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:39.441624 12539 raft_consensus.cc:2749] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:39.496008 10916 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:39.529325 10916 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:39.530227 10916 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:39.496462 10873 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:39.437537 12471 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:39.438830 12688 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:39.485621 12356 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:39.437877 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:39.644491 10699 catalog_manager.cc:5697] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:39.696496 10699 catalog_manager.cc:5697] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:39.698105 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:39.649061 10696 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42956 (request call id 257) took 9988 ms. Trace:
I20260709 13:16:39.658389 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:39.459860 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:39.701310 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:39.702191 10795 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:39.702889 10874 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 4.
I20260709 13:16:39.702690 10699 catalog_manager.cc:5697] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 1 to 2, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 2 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:39.704713 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:39.706176 10874 raft_consensus.cc:2393] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 3: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:39.660449 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:39.663872 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:39.707721 10800 raft_consensus.cc:2393] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 4: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:39.708983 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:39.699484 10696 rpcz_store.cc:276] 0709 13:16:29.660628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:29.660990 (+   362us) service_pool.cc:224] Handling call
0709 13:16:39.648798 (+9987808us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:16:37.266843 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:16:37.267695 (+   852us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:16:37.267743 (+    48us) write_op.cc:435] Acquired schema lock
0709 13:16:37.267775 (+    32us) tablet.cc:662] Decoding operations
0709 13:16:37.268543 (+   768us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:16:37.268645 (+   102us) write_op.cc:641] Partition lock acquired for write op
0709 13:16:37.268686 (+    41us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:16:37.269138 (+   452us) tablet.cc:701] Row locks acquired
0709 13:16:37.269164 (+    26us) write_op.cc:260] PREPARE: finished
0709 13:16:37.269349 (+   185us) write_op.cc:270] Start()
0709 13:16:37.269500 (+   151us) write_op.cc:276] Timestamp: P: 1783602997269290 usec, L: 0
0709 13:16:37.269531 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 13:16:37.270469 (+   938us) log.cc:844] Serialized 1312 byte log entry
0709 13:16:37.895000 (+624531us) op_driver.cc:464] REPLICATION: finished
0709 13:16:38.198281 (+303281us) write_op.cc:301] APPLY: starting
0709 13:16:38.198366 (+    85us) tablet.cc:1367] starting BulkCheckPresence
0709 13:16:38.198545 (+   179us) tablet.cc:1370] finished BulkCheckPresence
0709 13:16:38.198691 (+   146us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:16:38.199193 (+   502us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:16:38.199441 (+   248us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 13:16:38.199487 (+    46us) write_op.cc:312] APPLY: finished
0709 13:16:38.199700 (+   213us) log.cc:844] Serialized 35 byte log entry
0709 13:16:38.206302 (+  6602us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:38.206399 (+    97us) write_op.cc:454] Released schema lock
0709 13:16:38.206442 (+    43us) write_op.cc:341] FINISH: Updating metrics
0709 13:16:38.206822 (+   380us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:38.206858 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1187006,"reported_tablets":15,"child_traces":[["op",{"apply.queue_time_us":294116,"apply.run_cpu_time_us":2125,"apply.run_wall_time_us":8484,"num_ops":3,"prepare.queue_time_us":77003,"prepare.run_cpu_time_us":17563,"prepare.run_wall_time_us":97443,"raft.queue_time_us":399825,"raft.run_cpu_time_us":30419,"raft.run_wall_time_us":271158,"replication_time_us":577720,"thread_start_us":503080,"threads_started":4,"wal-append.queue_time_us":93046,"wal-append.run_cpu_time_us":18534,"wal-append.run_wall_time_us":1843272}]]}
I20260709 13:16:39.528024 10886 tablet_copy_service.cc:342] P 468a8fd46e824f62b9c2c0d3532a6d26: Request end of tablet copy session 12682eb70bc84c76879e04a8628ee586-2b82f18992434ed5ba58b805107060d4 received from {username='slave'} at 127.0.0.1:49290
I20260709 13:16:39.711504 10886 tablet_copy_service.cc:434] P 468a8fd46e824f62b9c2c0d3532a6d26: ending tablet copy session 12682eb70bc84c76879e04a8628ee586-2b82f18992434ed5ba58b805107060d4 on tablet 2b82f18992434ed5ba58b805107060d4 with peer 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:39.497205 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:39.714362 10867 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 1.
I20260709 13:16:39.715682 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:39.702082 10792 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:39.717350 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:39.718024 10796 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 4.
I20260709 13:16:39.669550  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:39.719591 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:39.501200 10872 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 5.
I20260709 13:16:39.720413 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:39.720408 10796 raft_consensus.cc:2376] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 2. Re-sending same reply.
I20260709 13:16:39.721531 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:39.518590 10876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1295) took 2116 ms. Trace:
I20260709 13:16:39.723034 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:39.722977 10876 rpcz_store.cc:276] 0709 13:16:37.401818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:37.964633 (+562815us) service_pool.cc:224] Handling call
0709 13:16:38.000027 (+ 35394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:38.725334 (+725307us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:39.518308 (+792974us) spinlock_profiling.cc:243] Waited 725 ms on lock 0x7b54002f0654. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:39.518520 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1666716416}
I20260709 13:16:39.723969 10872 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 1.
I20260709 13:16:39.724560 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:39.725787 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:39.725996 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:39.727317 10872 raft_consensus.cc:2368] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 3. Current term is 4.
I20260709 13:16:39.728163 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:39.728384 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:39.728649 10876 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:39.729009 10796 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:39.729110 10795 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
I20260709 13:16:39.729806 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:39.730660 10872 raft_consensus.cc:2393] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 4: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:39.736155 10801 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
I20260709 13:16:39.743790 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:39.745363 10801 raft_consensus.cc:2410] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:16:39.749349 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:39.747311 10796 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 3.
W20260709 13:16:39.753743 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:39.533900 10916 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:16:39.523065 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:39.756587 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 222550us
W20260709 13:16:39.756930 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:39.758200 10916 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:39.520576 10945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 165) took 1443 ms. Trace:
I20260709 13:16:39.780777 12395 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:39.781102 10800 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:39.782047 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:16:39.783998 11505 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56404 (request call id 406) took 2010 ms (client timeout 1503 ms). Trace:
W20260709 13:16:39.784785 11505 rpcz_store.cc:269] 0709 13:16:37.773880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:37.777372 (+  3492us) service_pool.cc:224] Handling call
0709 13:16:39.783421 (+2006049us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54005fee54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:39.783692 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2866430848}
I20260709 13:16:39.706012 10699 catalog_manager.cc:5697] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:39.465663 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:39.559211 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:39.793124 11503 raft_consensus.cc:2393] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 5: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:39.795311 11510 raft_consensus.cc:1240] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:39.796038 10792 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 4.
I20260709 13:16:39.556866 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:39.797292 12683 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:39.569756 12289 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:39.798283 12683 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:39.542824 12356 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:39.536055 12371 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:39.438851 12374 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:39.483529 12443 raft_consensus.cc:2804] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:39.819801 10699 catalog_manager.cc:5697] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 1 to 3, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 3 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:39.538269 12471 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:39.585249 12569 raft_consensus.cc:2749] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:39.570685 12573 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:39.826064 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:39.827682 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:39.827809 12471 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:39.492637 12574 consensus_queue.cc:1059] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 13:16:39.464941 12590 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:39.828358 10794 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:39.829217 12590 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:39.520339 12622 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:39.831264 12622 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:39.832602 10873 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 6.
I20260709 13:16:39.832690 12471 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:39.833684 12471 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:39.834476 12683 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:39.448361 12599 consensus_queue.cc:260] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:39.836716 12471 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:39.837390 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:39.837575 10948 raft_consensus.cc:1240] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:16:39.837637 12799 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:39.566592 12612 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:39.497437 12614 consensus_queue.cc:260] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:39.492532 12672 consensus_queue.cc:1059] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:16:39.844774 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:39.844983 12614 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:39.845750 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:39.846246 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:39.572265 12684 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:39.493520 12685 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:39.575281 12687 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:39.582401 12688 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:39.583866 12693 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:39.854197 12289 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:39.855155 12289 consensus_queue.cc:1059] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:39.855618 10941 raft_consensus.cc:1240] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:16:39.855823 12569 consensus_queue.cc:1059] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:39.856266 12289 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:39.856491 10797 raft_consensus.cc:1240] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:16:39.856930 12569 raft_consensus.cc:3055] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:39.857323 12569 raft_consensus.cc:740] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:39.857676 12685 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:39.857421 12289 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:39.858088 10797 raft_consensus.cc:1240] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:16:39.482757 12694 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:39.859756 12755 raft_consensus.cc:3055] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:39.860198 12755 raft_consensus.cc:740] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:39.860451 12654 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:39.860620 12574 raft_consensus.cc:3055] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:39.861153 12574 raft_consensus.cc:740] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:39.861312 12654 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:39.861486 12755 consensus_queue.cc:260] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:39.828475 12573 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:39.583688 12719 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:39.863505 10874 raft_consensus.cc:1240] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:16:39.864084 12672 raft_consensus.cc:3055] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:39.864503 12672 raft_consensus.cc:740] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:39.865293 10874 raft_consensus.cc:1240] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:39.590090 12723 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:39.492259 12725 consensus_queue.cc:1059] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:16:39.865617 12672 consensus_queue.cc:260] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:39.562037 12720 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:39.448167 12736 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:39.867556 12725 raft_consensus.cc:3055] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:39.867995 12725 raft_consensus.cc:740] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:39.869208 12725 consensus_queue.cc:260] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:39.572531 12749 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:39.869935 12671 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:39.582373 12733 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:39.449941 12750 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:39.872665 12654 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:39.585704 12764 log.cc:927] Time spent T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.661s	user 0.003s	sys 0.000s
I20260709 13:16:39.493889 12766 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:39.558651 12770 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:39.875162 12750 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:39.567035 12756 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:39.564908 12777 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:39.558048 12761 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:39.536625 12779 raft_consensus.cc:2749] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:39.449491 12784 raft_consensus.cc:2749] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:39.537981 12783 raft_consensus.cc:2804] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:39.880069 12443 raft_consensus.cc:697] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:39.880295 12783 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:39.495507 12787 raft_consensus.cc:2749] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:39.880303 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1307) took 1616 ms. Trace:
I20260709 13:16:39.787153 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:39.881867 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:39.881300 10869 rpcz_store.cc:276] 0709 13:16:38.263621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:38.583860 (+320239us) service_pool.cc:224] Handling call
0709 13:16:38.606833 (+ 22973us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:38.698975 (+ 92142us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:38.699010 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:38.699471 (+   461us) log.cc:844] Serialized 26 byte log entry
0709 13:16:38.763837 (+ 64366us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:16:38.763893 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:38.763998 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:39.585111 (+821113us) spinlock_profiling.cc:243] Waited 2.2 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:16:39.879779 (+294668us) raft_consensus.cc:1685] finished
0709 13:16:39.879829 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:39.880124 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":248487,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":31205,"spinlock_wait_cycles":5070208,"thread_start_us":127706,"threads_started":2,"wal-append.queue_time_us":168865}
I20260709 13:16:39.781894 12395 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:39.884137 10869 raft_consensus.cc:1275] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:16:39.885008 12395 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:16:39.885419 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:39.780727 10945 rpcz_store.cc:276] 0709 13:16:38.076957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:38.077088 (+   131us) service_pool.cc:224] Handling call
0709 13:16:38.077485 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:38.661040 (+583555us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:39.520335 (+859295us) spinlock_profiling.cc:243] Waited 546 ms on lock 0x7b540039b054. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:39.520515 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1255039360}
I20260709 13:16:39.789690 12778 raft_consensus.cc:2749] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:39.790352 11511 raft_consensus.cc:1275] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:16:39.791198 12617 raft_consensus.cc:2749] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:39.795105 10842 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:39.493772 12313 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:39.890959 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:39.800400 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:39.808352 12508 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:39.806887 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 48963.8us
I20260709 13:16:39.797211 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:39.793154 12396 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:39.851639 10795 raft_consensus.cc:1240] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:16:39.819989 10947 raft_consensus.cc:1240] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:39.822016 12374 raft_consensus.cc:697] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 6 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:39.902995 12374 consensus_queue.cc:237] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:39.843818 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:39.905925 10942 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2.
I20260709 13:16:39.906553 12685 consensus_queue.cc:1048] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:39.907460 12777 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:39.908095 12777 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:39.908644 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:39.822324 10699 catalog_manager.cc:5697] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 3 to 4, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 4 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:39.846624 11512 raft_consensus.cc:2393] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 6: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:39.849692 10801 raft_consensus.cc:2393] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 4: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:39.840919 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:39.913554 10944 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:16:39.853605 10867 raft_consensus.cc:2393] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 3: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:39.853432 12583 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:39.853734 10868 raft_consensus.cc:2393] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 5: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:39.554466 12697 raft_consensus.cc:2749] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:39.865209 12776 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:39.869675 12736 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:39.536450 12739 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:39.959225 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:39.959435 12739 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:39.960180 12739 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:39.962248 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:39.578176 12773 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:39.963258 10943 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
I20260709 13:16:39.583535 12775 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:39.590026 12781 log.cc:927] Time spent T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.328s	user 0.000s	sys 0.003s
I20260709 13:16:39.881420 12443 consensus_queue.cc:237] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:39.788898 10876 raft_consensus.cc:2410] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:16:39.872721 12733 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:40.001610 10869 raft_consensus.cc:1240] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 2. Current term is 3. Ops: []
W20260709 13:16:39.894033 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:39.809062 12701 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:39.895565 10796 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 5.
I20260709 13:16:39.896494 12508 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:39.896639 12396 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:39.897258 10916 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:16:39.897529 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 102813us
I20260709 13:16:39.897961 11509 raft_consensus.cc:2393] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 5: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:39.898393 10793 raft_consensus.cc:1240] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:16:39.894702 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:39.899058 12754 raft_consensus.cc:2804] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:39.898319 10872 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:39.880832 12783 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:39.923884 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:39.924742 10797 raft_consensus.cc:1240] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:39.899819 12313 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:39.898330 12684 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:39.925810 10870 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 5.
I20260709 13:16:39.930693 10801 raft_consensus.cc:1240] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:39.862543 12574 consensus_queue.cc:260] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:39.932914 12755 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:39.858678 12569 consensus_queue.cc:260] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:39.944113 10794 raft_consensus.cc:2410] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:16:39.947638 12725 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:40.020851 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:39.949144 12617 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:39.940536 12583 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:39.949774 10767 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:39.951395 11478 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:16:39.951519 12289 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:39.949229 12599 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:39.956954 12590 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:40.034871 12590 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:39.958312 11511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56404 (request call id 415) took 1140 ms. Trace:
I20260709 13:16:39.958137 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:39.958571 10942 raft_consensus.cc:1275] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:16:39.872990 12688 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:40.049885 10950 raft_consensus.cc:1240] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:39.938709 12777 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:39.981335 10799 raft_consensus.cc:1240] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:16:39.982110 12775 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:39.895404 12799 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:39.992197 12795 log.cc:927] Time spent T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:16:39.979463 12800 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:39.965828 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:40.017768 10842 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:40.020290 12761 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:40.027055 10949 raft_consensus.cc:2410] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 7 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:16:40.027801 12672 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:40.035333 12617 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:40.043181 10948 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 4.
I20260709 13:16:40.044929 12574 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:40.045769 10797 raft_consensus.cc:1240] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:40.022192 12754 raft_consensus.cc:697] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:40.000469 12776 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:40.049630 12583 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:40.038034 11511 rpcz_store.cc:276] 0709 13:16:38.818107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:38.818291 (+   184us) service_pool.cc:224] Handling call
0709 13:16:39.789445 (+971154us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:39.887286 (+ 97841us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:39.957992 (+ 70706us) spinlock_profiling.cc:243] Waited 932 ms on lock 0x7b54005fee50. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:39.958222 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2144615680}
I20260709 13:16:40.036975 10795 raft_consensus.cc:1240] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:16:40.054644 12622 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:40.072199 12356 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:40.074409 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 177315us
I20260709 13:16:40.074463 10944 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 6.
I20260709 13:16:40.076225 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:16:40.081805 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 130547us
I20260709 13:16:40.081943 10945 raft_consensus.cc:2393] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 3: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
W20260709 13:16:40.083556 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 76820.7us
I20260709 13:16:40.085035 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:40.081822 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:40.087224 10872 raft_consensus.cc:2410] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:16:40.090310 10942 raft_consensus.cc:2393] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 4: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:40.093266 10873 raft_consensus.cc:1240] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:40.091708 10842 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:40.096364 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:40.102411 10916 leader_election.cc:400] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 4. Current term is 5.
I20260709 13:16:40.102638 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:16:40.103616 12805 log.cc:927] Time spent T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.099s	user 0.000s	sys 0.002s
I20260709 13:16:40.103636 10944 raft_consensus.cc:2393] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 4: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:40.103801 12723 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:40.104470 12723 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:40.103840 11478 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:16:40.104890 12723 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:40.105006 10916 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:40.106293 11478 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
W20260709 13:16:40.108346 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 69426.1us
I20260709 13:16:40.108719 10946 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:40.052772 12701 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:40.108371 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:40.109416 10949 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
I20260709 13:16:40.095268 12777 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:40.092697 12289 raft_consensus.cc:2749] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:40.128145 10949 raft_consensus.cc:1275] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:16:40.137274 10916 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:40.114638 12671 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:16:40.148829 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 232788us
I20260709 13:16:40.140550 12761 raft_consensus.cc:2804] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:40.129297 12777 raft_consensus.cc:697] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:40.089716 12800 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:40.133697 12776 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:40.151088 12733 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:40.139497 12768 consensus_queue.cc:1059] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:16:40.156967 10916 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:40.165190 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:40.165733 10949 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:40.123128 12590 consensus_queue.cc:1059] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 13:16:40.181551 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:40.162356 12539 raft_consensus.cc:2749] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:40.187108 12622 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:40.191406 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:40.192303 10944 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:40.160447 12671 raft_consensus.cc:697] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 6 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
W20260709 13:16:40.175680 12810 log.cc:927] Time spent T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:16:40.164863 12761 raft_consensus.cc:697] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:40.198017 12778 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:40.200634 12778 raft_consensus.cc:697] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 4 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:40.200510 12733 raft_consensus.cc:697] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 5 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:40.201435 12617 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:40.164552 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:40.180400 12356 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:40.160475 12804 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:40.170701 12798 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:40.209828 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:40.211426 10948 raft_consensus.cc:2393] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 3: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:40.211253 12396 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:40.211269 12671 consensus_queue.cc:237] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:40.187158 12809 log.cc:927] Time spent T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 13:16:40.221994 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:16:40.215652 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 47435.4us
W20260709 13:16:40.228091 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 74969.1us
I20260709 13:16:40.230027 12800 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:40.220827 12783 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:40.238612 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:40.267565 12783 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:40.224898 12798 raft_consensus.cc:697] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 5 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:40.270459 12798 consensus_queue.cc:237] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:40.275182 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 72115.3us
I20260709 13:16:40.276090 11478 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:40.277752 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:40.278566 12508 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:40.279376 12508 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:40.288643 10767 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:40.185504 12754 consensus_queue.cc:237] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:40.296258 10795 raft_consensus.cc:1275] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:40.297891 10916 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:40.298375 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:40.272399 10943 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
I20260709 13:16:40.279682 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:40.202245 12733 consensus_queue.cc:237] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:40.299764 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:40.201910 12778 consensus_queue.cc:237] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:40.303987 10792 raft_consensus.cc:2393] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 5: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
W20260709 13:16:40.304862 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:40.305310 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:40.281551 12804 raft_consensus.cc:697] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 5 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:40.282137 12508 raft_consensus.cc:697] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 4 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:40.310565 10948 raft_consensus.cc:2393] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 4: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:40.314548 11507 raft_consensus.cc:1275] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:16:40.316228 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:40.317018 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:16:40.322085 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:40.323299 12356 consensus_queue.cc:1059] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:40.324585 12508 consensus_queue.cc:237] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:40.316007 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:40.310114 11509 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 5.
I20260709 13:16:40.331110 10947 raft_consensus.cc:1240] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:40.313973 12807 raft_consensus.cc:3055] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:40.332252 12807 raft_consensus.cc:740] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:40.289498 12799 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:40.322366 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:40.308365 12776 raft_consensus.cc:2749] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:16:40.341058 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 67013.1us
I20260709 13:16:40.341656 10944 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 5.
I20260709 13:16:40.344746 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:40.338253 10945 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:40.345696 11511 raft_consensus.cc:2393] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 3: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:40.314692 12783 raft_consensus.cc:2804] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:40.347774 10793 raft_consensus.cc:2393] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 5: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:40.318071 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:40.349462 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:40.354247 10947 raft_consensus.cc:2393] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 6: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:40.355270 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:40.356307 10941 raft_consensus.cc:2410] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:16:40.356990 12783 raft_consensus.cc:697] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 2 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:40.358028 12783 consensus_queue.cc:237] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:40.359097 12617 raft_consensus.cc:3055] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:40.358935 12771 consensus_queue.cc:1059] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.008s
I20260709 13:16:40.359548 12617 raft_consensus.cc:740] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:40.289184 12777 consensus_queue.cc:237] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:40.360599 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:16:40.359786 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 77261.9us
I20260709 13:16:40.358686 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:40.372162 10801 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
I20260709 13:16:40.376705 10792 raft_consensus.cc:1275] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:16:40.380645 10766 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:40.383841 12617 consensus_queue.cc:260] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:40.385377 12617 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:40.386701 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:40.386953 10767 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:40.389981 12768 consensus_queue.cc:1059] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:40.392601 12755 raft_consensus.cc:2749] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:40.387390 12807 consensus_queue.cc:260] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:40.365667 12688 raft_consensus.cc:3055] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:40.394383 10950 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:40.365787 12761 consensus_queue.cc:237] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:40.397538 10947 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
I20260709 13:16:40.399139 12776 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:40.398919 12787 consensus_queue.cc:1059] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:40.399173 12396 consensus_queue.cc:1059] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:40.394413 12807 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:40.400597 10767 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:40.394511 12688 raft_consensus.cc:740] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:40.406106 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:40.416558 10794 raft_consensus.cc:2393] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 4: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:40.400359 12784 raft_consensus.cc:2955] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 2 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:40.420092 12787 raft_consensus.cc:3055] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:40.420619 12787 raft_consensus.cc:740] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:16:40.421005 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:40.354475 12701 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:40.427141 10946 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 4.
I20260709 13:16:40.434518 10799 raft_consensus.cc:1240] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:40.437189 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:40.411779 12776 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:40.405179 12574 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:40.440290 10766 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:40.438084 10947 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
I20260709 13:16:40.441560 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:40.449855 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:40.375861 12766 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:40.452757 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:40.398986 12539 consensus_queue.cc:1059] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:16:40.461915 12539 raft_consensus.cc:3055] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:40.462321 12539 raft_consensus.cc:740] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:40.463064 10948 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65"
dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:41020
I20260709 13:16:40.450953 10798 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:40.457679 10872 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:40.464864 10943 raft_consensus.cc:1240] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:16:40.466069 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:16:40.467782 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 67279.3us
I20260709 13:16:40.457744 10947 raft_consensus.cc:1240] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:40.460289 10944 raft_consensus.cc:1240] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:40.467190 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:16:40.449954 12813 log.cc:927] Time spent T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 13:16:40.463858 10948 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:40.471859 10948 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:40.457762 10842 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:40.444316 12776 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:40.474673 10872 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 4.
I20260709 13:16:40.468753 10943 raft_consensus.cc:2393] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 3: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
W20260709 13:16:40.477154 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 48876.8us
I20260709 13:16:40.479290 10767 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:40.451159 12672 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:40.453521 12701 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:40.491232 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:40.492146 10942 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:40.421640 12787 consensus_queue.cc:260] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:40.463330 12539 consensus_queue.cc:260] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:40.496213 12289 raft_consensus.cc:2749] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:16:40.497072 10874 raft_consensus.cc:1275] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:16:40.442291 12739 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:40.500319 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:40.500008 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:40.509244 10947 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:40.503887 10766 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:40.510682 10875 raft_consensus.cc:1240] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:40.487732 12684 raft_consensus.cc:2749] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:16:40.472148 12574 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:40.518618 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:40.517320 10767 leader_election.cc:400] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 3. Current term is 4.
I20260709 13:16:40.531281 10767 leader_election.cc:403] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
W20260709 13:16:40.532507 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:40.541141 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:40.487287 12396 consensus_queue.cc:1059] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
W20260709 13:16:40.544289 12818 log.cc:927] Time spent T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:16:40.544598 12773 raft_consensus.cc:2749] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:40.544982 10943 raft_consensus.cc:1240] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:16:40.534430 12583 raft_consensus.cc:2804] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:40.511996 12289 consensus_queue.cc:1048] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:16:40.490013 12688 consensus_queue.cc:260] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:40.547192 12787 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:40.508239 12701 raft_consensus.cc:697] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:40.562286 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:40.582643 10949 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 5.
I20260709 13:16:40.581918 10948 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:40.452023 12804 consensus_queue.cc:237] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:40.558715 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:40.562013 10945 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 4.
I20260709 13:16:40.575309 12688 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:40.519691 12672 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:40.560369 12539 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:40.605786 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:40.612479 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:16:40.617892 12824 log.cc:927] Time spent T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
W20260709 13:16:40.619995 12821 log.cc:927] Time spent T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
W20260709 13:16:40.621892 12819 log.cc:927] Time spent T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
W20260709 13:16:40.624655 12820 log.cc:927] Time spent T 00000000000000000000000000000000 P ae957c8fc66c4b6ba063d0c74cbf5484: Append to log took a long time: real 0.157s	user 0.002s	sys 0.000s
I20260709 13:16:40.627451 10872 raft_consensus.cc:2410] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:16:40.631522 12823 log.cc:927] Time spent T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.151s	user 0.000s	sys 0.001s
I20260709 13:16:40.662575 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:16:40.669839 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:40.671558 11507 raft_consensus.cc:1240] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:40.672318 10946 raft_consensus.cc:1240] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:40.673601 10871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49290 (request call id 197) took 1212 ms. Trace:
I20260709 13:16:40.674125 10871 rpcz_store.cc:276] 0709 13:16:39.461398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:39.464493 (+  3095us) service_pool.cc:224] Handling call
0709 13:16:40.673162 (+1208669us) spinlock_profiling.cc:243] Waited 912 ms on lock 0x7b540009bad4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:40.673371 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2098235648}
I20260709 13:16:40.677659 12569 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:40.679778 12701 consensus_queue.cc:237] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:40.683432 12654 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:40.684290 12654 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:40.684728 12754 raft_consensus.cc:3055] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:40.685354 12754 raft_consensus.cc:740] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:40.686566 12754 consensus_queue.cc:260] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:40.688616 12754 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:40.689352 10875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1300) took 1288 ms. Trace:
I20260709 13:16:40.674275 10946 raft_consensus.cc:1275] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:16:40.691062 10946 pending_rounds.cc:85] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 2
I20260709 13:16:40.691380 10946 pending_rounds.cc:107] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:16:40.691709 10946 raft_consensus.cc:2889] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:40.691995 12574 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:40.689754 10875 rpcz_store.cc:276] 0709 13:16:39.400683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:39.401446 (+   763us) service_pool.cc:224] Handling call
0709 13:16:39.401785 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:40.534603 (+1132818us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:40.689024 (+154421us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b5400097ed4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:40.689265 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2472194816}
I20260709 13:16:40.694955 12694 raft_consensus.cc:3055] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:40.695410 12694 raft_consensus.cc:740] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:16:40.696805 12725 consensus_meta.cc:287] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.675s	user 0.004s	sys 0.007s
I20260709 13:16:40.699474 12739 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:40.700126 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:40.701059 10874 raft_consensus.cc:2393] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 3: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:40.576124 12766 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:40.696326 12694 consensus_queue.cc:260] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:40.702728 12694 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:40.703851 12672 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:40.705714 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:40.707760 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:40.708206 12289 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:40.708777 12289 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:40.709151 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:40.712368 10870 raft_consensus.cc:2410] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:16:40.713377 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:40.716701 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:40.728579 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:40.729732 10949 raft_consensus.cc:2368] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 2. Current term is 3.
I20260709 13:16:40.733407 10842 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:40.741711 10942 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
I20260709 13:16:40.557397 12723 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:40.600262 12773 raft_consensus.cc:2749] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 3. Current term is 4.
W20260709 13:16:40.769295 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 220774us
I20260709 13:16:40.798491 10842 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:40.798853 10947 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
I20260709 13:16:40.757126 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:40.587735 12622 consensus_queue.cc:1048] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.144s
I20260709 13:16:40.800945 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:40.807945 10796 raft_consensus.cc:1240] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:16:40.723206 12289 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:40.815429 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:40.816255 10948 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:40.820580 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:40.821123 10869 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:40.757731 12771 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:40.827180 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:40.823128 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:40.820333 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:40.823683 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:40.825055 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:40.825744 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:16:40.849344 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 91648.3us
I20260709 13:16:40.851384 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:40.853278 10797 raft_consensus.cc:2393] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 4: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:40.828969 10801 raft_consensus.cc:2410] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:16:40.829483 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:40.855710 10796 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 4.
I20260709 13:16:40.857270 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:40.857354 12776 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:40.858338 11504 raft_consensus.cc:2410] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:16:40.859905 12826 log.cc:927] Time spent T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:16:40.860901 12654 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:40.861644 11509 raft_consensus.cc:2393] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 3: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:40.866561 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:40.769994 12688 consensus_queue.cc:1048] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.081s
I20260709 13:16:40.871131 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:40.871258 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:40.874737 10949 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 4.
I20260709 13:16:40.872582 10793 raft_consensus.cc:2376] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 6. Re-sending same reply.
I20260709 13:16:40.872015 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:40.827183 12771 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:40.849407 12684 raft_consensus.cc:2804] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:40.827343 12773 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:40.828091 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:40.863744 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:40.877324 10871 raft_consensus.cc:2393] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 5: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:40.878051 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:40.879724 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:40.879926 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:40.883701 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:40.894097 10801 raft_consensus.cc:2368] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 3. Current term is 4.
I20260709 13:16:40.884049 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:40.849442 12289 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:40.886350 12766 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:40.887190 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:40.821058 12807 consensus_queue.cc:1059] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 13:16:40.887306 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:40.883282 12771 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:40.904042 10797 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:40.885226 12684 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:40.887832 12761 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:40.914968 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:40.915658 11504 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 5.
I20260709 13:16:40.916721 12761 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:40.914702 12654 consensus_queue.cc:1059] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:16:40.934316 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:40.942205 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:40.952778 10794 raft_consensus.cc:2393] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 4: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:40.955878 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:40.901891 10943 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
I20260709 13:16:40.963351 10800 raft_consensus.cc:1275] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:16:40.921903 12684 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:40.969982 10766 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:40.904201 12374 consensus_queue.cc:1048] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.056s
I20260709 13:16:40.985523 10797 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 3.
W20260709 13:16:40.959374 12784 consensus_meta.cc:287] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.538s	user 0.008s	sys 0.013s
W20260709 13:16:40.977054 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 459896us
I20260709 13:16:40.958877 12787 consensus_queue.cc:1048] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:16:40.989550 10767 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:40.994911 10948 raft_consensus.cc:1240] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:16:40.998279 10798 raft_consensus.cc:2410] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:16:40.909142 12773 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:40.920487 12766 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:40.926911 12289 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:40.970558 12761 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:41.004842 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:41.010470 10794 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 3.
I20260709 13:16:41.006080 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:41.011785 11507 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 4.
I20260709 13:16:40.976027 12776 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:40.959802 12778 consensus_queue.cc:1048] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:16:41.016585 10873 raft_consensus.cc:1240] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 LEADER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:40.969250 12583 raft_consensus.cc:697] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:41.028138 12583 consensus_queue.cc:237] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:41.030315 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:41.031220 10948 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 3.
W20260709 13:16:41.031754 12828 log.cc:927] Time spent T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:16:41.032570 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:41.033416 10943 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
W20260709 13:16:41.031248 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 61382.5us
I20260709 13:16:41.035542 10766 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:41.038553 12773 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:41.039137 12773 raft_consensus.cc:2749] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:41.039984 12773 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:41.040371 12773 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:41.040666 12766 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:41.040778 12773 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:41.043056 12776 consensus_queue.cc:1048] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:41.043787 12776 consensus_queue.cc:1243] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer 12682eb70bc84c76879e04a8628ee586 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:16:41.039067 10766 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:41.046213 12831 raft_consensus.cc:2804] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:41.046609 12831 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:41.046993 12831 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:41.048285 12723 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:41.048281 10766 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:41.049690 10766 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:41.050881 12756 raft_consensus.cc:3055] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:41.051043 12776 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:41.051384 12756 raft_consensus.cc:740] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:41.051755 12776 raft_consensus.cc:697] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:41.057142 10916 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:41.020187 10874 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 4.
I20260709 13:16:41.020455 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:41.021948 10767 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:16:41.025200 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:41.063701 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:41.013563 12569 raft_consensus.cc:2749] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:41.055991 10868 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 5.
W20260709 13:16:41.064709 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:41.066399 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:16:41.025700 12617 consensus_meta.cc:287] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.640s	user 0.008s	sys 0.015s
I20260709 13:16:41.067370 10948 raft_consensus.cc:2410] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:16:41.069806 10915 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:41.080760 10915 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:16:41.082412 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:41.082979 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:41.083485 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:41.088601 10794 raft_consensus.cc:2955] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
W20260709 13:16:41.096740 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 74928.1us
I20260709 13:16:41.097872 10767 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:41.059916 12836 raft_consensus.cc:2804] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:41.070505 12374 raft_consensus.cc:2749] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:41.088114 12739 raft_consensus.cc:2804] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:41.110204 11478 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:41.110838 12289 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:16:41.114696 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 98557.4us
I20260709 13:16:41.115571 10916 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:41.117774 10869 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 5.
I20260709 13:16:41.118489 12838 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:41.088027 12787 consensus_queue.cc:1059] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:16:41.118991 12778 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:41.119393 12787 raft_consensus.cc:3055] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:41.119091 12838 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:41.119601 12778 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:41.119863 12787 raft_consensus.cc:740] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:41.120044 12778 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:41.120875 12787 consensus_queue.cc:260] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:41.121477 12838 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:41.108835 12739 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:41.122723 12739 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:41.122967 12836 raft_consensus.cc:697] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:41.124119 12836 consensus_queue.cc:237] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:41.100250 12396 raft_consensus.cc:2749] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:41.100456 12617 raft_consensus.cc:2764] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:16:41.125140 12684 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:41.114905 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:41.122092 12838 raft_consensus.cc:2749] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:41.079248 12756 consensus_queue.cc:260] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:41.130857 12756 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:41.131299 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:41.094501 12776 consensus_queue.cc:237] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:41.131987 10870 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 2.
I20260709 13:16:41.133551 12723 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:41.134135 12723 raft_consensus.cc:697] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 4 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:41.135377 10950 raft_consensus.cc:2410] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:16:41.134989 12723 consensus_queue.cc:237] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:41.135883 12831 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:41.136987 10841 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:41.137854 12843 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:41.137967 12831 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:41.138351 12843 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:41.138935 12843 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:41.138727 12672 consensus_queue.cc:1048] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:41.139556 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:41.140189 10948 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 5.
I20260709 13:16:41.141103 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:41.141470 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:41.141844 10948 raft_consensus.cc:2393] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 5: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:41.141930 11477 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:41.143743 12574 consensus_queue.cc:1048] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:41.145093 12574 raft_consensus.cc:2804] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:41.145483 12574 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:41.145790 12574 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:41.138399 10946 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
I20260709 13:16:41.145128 10793 raft_consensus.cc:1240] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:41.151484 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:41.152594 10948 raft_consensus.cc:2393] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 4: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:41.160728 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:41.161502 10870 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 5.
I20260709 13:16:41.201653 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:41.122063 12787 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:16:41.209255 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 51998us
I20260709 13:16:41.210369 11477 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:41.216907 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:41.217749 10870 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 3
W20260709 13:16:41.223126 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 107653us
I20260709 13:16:41.224177 11512 pending_rounds.cc:85] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586: Aborting all ops after (but not including) 4
I20260709 13:16:41.224603 11512 pending_rounds.cc:107] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 13:16:41.224895 11512 raft_consensus.cc:2889] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:41.227311 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:41.228113 10948 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:16:41.243057 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 145295us
I20260709 13:16:41.232198 12842 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:41.250923 10944 raft_consensus.cc:1240] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:41.253937 10767 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:16:41.257059 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:41.260807 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 59644.9us
W20260709 13:16:41.271745 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:41.271544 12842 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:41.280556 12771 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:41.246640 12773 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:41.285329 12773 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:41.292738 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:41.282406 12843 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:41.300655 12843 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:41.304418 12688 consensus_queue.cc:1059] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:41.288288 12766 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:41.305760 10916 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:41.304455 10841 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:16:41.312288 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 98113.1us
I20260709 13:16:41.316916 10870 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 3.
I20260709 13:16:41.268272 12671 raft_consensus.cc:2804] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:41.340979 10799 raft_consensus.cc:1240] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 LEADER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:41.359232 12684 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:41.341488 12671 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:41.361593 12671 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:41.344074 12766 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:41.343585 12622 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:41.375881 12622 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:41.376987 12766 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:41.377553 12688 raft_consensus.cc:2749] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:41.377732 12697 raft_consensus.cc:3055] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:41.378413 12374 raft_consensus.cc:2749] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:41.377971 12622 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:16:41.384987 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 80604.4us
W20260709 13:16:41.385924 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 80266us
I20260709 13:16:41.392398 10948 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 4.
I20260709 13:16:41.392874 11478 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:16:41.378439 12697 raft_consensus.cc:740] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:41.416208 11478 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
W20260709 13:16:41.436455 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 226185us
W20260709 13:16:41.436354 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 44394.5us
I20260709 13:16:41.441990 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:41.428512 12672 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:41.444897 10916 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:41.429855 12654 raft_consensus.cc:2804] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:41.478240 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:41.443746 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:41.481590 11505 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:16:41.481704 12846 log.cc:927] Time spent T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:16:41.476428 12688 consensus_queue.cc:1059] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 13:16:41.482968 12766 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:41.483532 12766 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:41.483969 12841 log.cc:927] Time spent T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:16:41.484216 12761 consensus_queue.cc:1059] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:16:41.484812 12855 log.cc:927] Time spent T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 13:16:41.485455 12843 raft_consensus.cc:3055] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:41.485862 12766 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:41.486567 12856 log.cc:927] Time spent T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:16:41.485884 12843 raft_consensus.cc:740] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:41.487769 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:41.487587 12843 consensus_queue.cc:260] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:41.488763 12843 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:41.489960 12739 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:41.480520 12672 raft_consensus.cc:697] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 3 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:41.492290 12858 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:41.480537 12845 log.cc:927] Time spent T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:16:41.420251 12574 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:41.495337 12766 consensus_queue.cc:1048] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:41.517927 12671 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:41.496806 12684 consensus_queue.cc:1059] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:41.519398 12684 raft_consensus.cc:3055] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:41.519760 12684 raft_consensus.cc:740] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:41.520385 12671 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:41.520591 12684 consensus_queue.cc:260] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:41.521957 12684 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:41.499357 10944 raft_consensus.cc:1275] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:16:41.501801 12842 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:41.524166 10944 raft_consensus.cc:1240] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:16:41.506601 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:41.444155 12771 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:41.477349 12617 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:16:41.513774 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 68954.3us
I20260709 13:16:41.541220 10949 raft_consensus.cc:1275] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:16:41.526365 10876 raft_consensus.cc:2393] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 4: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:41.549742 10916 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:41.551411 12852 raft_consensus.cc:2749] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:41.555213 10842 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:41.552759 10916 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:41.558028 11507 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 1 LEARNER]: Advancing to term 2
I20260709 13:16:41.567341 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:41.570179 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:41.571170 10797 raft_consensus.cc:2393] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 4: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:41.571211 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:16:41.574098 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 328922us
I20260709 13:16:41.575047 12773 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:41.580466 10766 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:41.580346 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:16:41.594120 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:41.606400 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:41.607869 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:41.608510 10949 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:41.513265 12858 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:41.621243 10841 leader_election.cc:400] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 election: Vote denied by peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553) with higher term. Message: Invalid argument: T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 2. Current term is 3.
I20260709 13:16:41.623205 10841 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:41.624261 12688 raft_consensus.cc:2749] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:41.623360 12858 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:41.624855 10841 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:41.629739 10871 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 4.
I20260709 13:16:41.560393 12771 raft_consensus.cc:2804] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:16:41.631856 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 51445.6us
I20260709 13:16:41.635818 10766 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:41.607520 12574 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:41.579818 12787 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:41.639859 10916 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:41.634424 12771 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:41.611824 12836 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:41.641810 10876 raft_consensus.cc:1275] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:16:41.644824 10871 raft_consensus.cc:1275] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:16:41.641434 10800 raft_consensus.cc:1275] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:41.645184 10766 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:41.636677 12688 raft_consensus.cc:2749] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:41.705435 10841 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:41.707026 10872 raft_consensus.cc:1240] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 2. Current term is 3. Ops: []
W20260709 13:16:41.707489 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 67691.1us
I20260709 13:16:41.638248 12833 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:41.642993 12617 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:41.644179 12583 raft_consensus.cc:2749] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:41.708760 10916 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:41.711057 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:41.712217 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:41.712965 10875 raft_consensus.cc:2410] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:16:41.717978 10949 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 6.
I20260709 13:16:41.646517 12854 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:41.647800 12771 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:41.648331 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:41.721776 10871 raft_consensus.cc:1240] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:16:41.723549 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:41.725363 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:41.725996 10796 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:41.725363 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:41.726889 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:41.728065 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:41.728821 10800 raft_consensus.cc:2393] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 6: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:41.727154 10944 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 3.
I20260709 13:16:41.730163 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:41.730525 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:41.730891 10944 raft_consensus.cc:2393] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 5: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:41.708799 12836 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:41.733880 12836 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:41.736003 12725 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:41.736758 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:41.736611 12725 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:41.731766 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:41.739990 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:41.740715 10944 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
W20260709 13:16:41.720153 12778 consensus_meta.cc:287] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.600s	user 0.012s	sys 0.000s
I20260709 13:16:41.627396 12672 consensus_queue.cc:237] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:41.710069 12787 raft_consensus.cc:697] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 5 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:41.646188 12775 consensus_queue.cc:1059] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.076s
I20260709 13:16:41.725167 12854 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:41.775301 10874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 1321) took 1049 ms. Trace:
I20260709 13:16:41.775746 10874 rpcz_store.cc:276] 0709 13:16:40.725614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:40.739019 (+ 13405us) service_pool.cc:224] Handling call
0709 13:16:41.775086 (+1036067us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b54003174d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:41.775221 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":285723904}
I20260709 13:16:41.776958 12854 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:41.777155 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:41.760151 11507 raft_consensus.cc:1275] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 2 LEARNER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:16:41.713296 12539 raft_consensus.cc:2749] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:41.713851 12833 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:41.713248 12701 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:41.764566 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:41.790392 10949 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 4.
I20260709 13:16:41.764840 11511 raft_consensus.cc:3055] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:41.765477 11512 raft_consensus.cc:1275] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:16:41.759553 12853 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:41.767241 10796 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 3.
I20260709 13:16:41.767436 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:41.714241 12289 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:41.795881 10950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41020 (request call id 1679) took 1264 ms. Trace:
I20260709 13:16:41.787179 12688 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:41.796121 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:41.796849 10945 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 3.
I20260709 13:16:41.796541 12688 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:41.797904 10945 raft_consensus.cc:1240] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:41.798238 10949 raft_consensus.cc:1275] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:16:41.798741 12688 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:41.793087 12574 consensus_queue.cc:1048] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 13:16:41.796288 10950 rpcz_store.cc:276] 0709 13:16:40.525964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:40.662181 (+136217us) service_pool.cc:224] Handling call
0709 13:16:41.786185 (+1124004us) spinlock_profiling.cc:243] Waited 471 ms on lock 0x7b54001f2f54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:41.789945 (+  3760us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1084278016}
I20260709 13:16:41.803303 10945 raft_consensus.cc:1240] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:41.720971 12684 raft_consensus.cc:2749] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:41.779794 10800 raft_consensus.cc:1275] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:16:41.739820 12725 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:41.740285 10943 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:41.742837 12756 raft_consensus.cc:2804] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:16:41.783170 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 136918us
I20260709 13:16:41.817059 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:41.818017 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:41.765375 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:41.818819 10875 raft_consensus.cc:2410] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:16:41.819620 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:41.820650 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:41.821381 10875 raft_consensus.cc:2393] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 5: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:41.822146 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:41.823531 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:41.792232 10793 raft_consensus.cc:1275] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 5. (term mismatch)
I20260709 13:16:41.791263 11511 raft_consensus.cc:740] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:41.824326 10875 raft_consensus.cc:2410] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:16:41.778534 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:41.790982 12833 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:41.721740 12617 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:41.799059 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:41.805037 11477 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:41.759007 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:16:41.813827 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 108441us
I20260709 13:16:41.795904 12289 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:41.795066 12831 raft_consensus.cc:3055] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:16:41.793848 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 51793.9us
I20260709 13:16:41.799952 12863 consensus_queue.cc:1059] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:16:41.794734 12853 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:41.824175 10793 pending_rounds.cc:85] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 4
I20260709 13:16:41.863999 10793 pending_rounds.cc:107] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:16:41.864391 10793 raft_consensus.cc:2889] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:41.829823 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:41.833261 10872 raft_consensus.cc:2410] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:16:41.828250 12756 raft_consensus.cc:697] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:41.813970 11505 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 6.
I20260709 13:16:41.836418 10916 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:41.828413 10947 raft_consensus.cc:1275] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:16:41.837096 11477 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:41.835615 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:41.837683 11511 consensus_queue.cc:260] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:41.838961 10874 raft_consensus.cc:2393] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 5: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:41.839159 12289 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:41.843133 11509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56404 (request call id 417) took 1240 ms. Trace:
I20260709 13:16:41.880587 11509 rpcz_store.cc:276] 0709 13:16:40.602693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:40.612038 (+  9345us) service_pool.cc:224] Handling call
0709 13:16:41.842753 (+1230715us) spinlock_profiling.cc:243] Waited 236 ms on lock 0x7b540021e654. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:41.843043 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":542814464}
I20260709 13:16:41.881807 12289 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:41.882321 12289 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:41.883893 12683 raft_consensus.cc:3055] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:41.884361 12683 raft_consensus.cc:740] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:41.884557 12289 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:41.815212 12701 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:41.885514 12683 consensus_queue.cc:260] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:41.887247 10797 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 4.
W20260709 13:16:41.845667 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:41.845504 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:41.846640 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:41.847966 12787 consensus_queue.cc:237] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:41.835496 12831 raft_consensus.cc:740] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:41.846956 12725 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:41.856084 12617 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:41.807363 12858 consensus_queue.cc:1048] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:16:41.863126 12865 raft_consensus.cc:2804] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:41.813827 12739 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:41.866981 12508 raft_consensus.cc:3055] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:41.867514 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:41.865290 12853 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:16:41.868899 12869 log.cc:927] Time spent T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:16:41.869419 12854 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:41.869561 12755 raft_consensus.cc:3055] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:41.773792 12778 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:41.876253 10945 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 4.
I20260709 13:16:41.877265 11477 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:41.877918 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:41.893641 12599 raft_consensus.cc:2749] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:41.895061 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:41.895635 10841 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:41.915452 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:41.916326 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:41.916680 10874 raft_consensus.cc:2393] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 4: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:41.898648 11478 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:16:41.899966 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 63641.3us
I20260709 13:16:41.902776 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:41.919857 11507 raft_consensus.cc:2393] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 5: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:41.920629 12289 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:41.904942 10872 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:41.907898 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:41.887988 12863 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:41.886958 12701 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:41.897753 12683 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:41.914414 10875 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 3.
I20260709 13:16:41.908735 12508 raft_consensus.cc:740] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:41.898372 11511 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:41.919466 12854 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:41.931277 10948 raft_consensus.cc:2393] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 3: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:41.905390 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:41.922760 10947 raft_consensus.cc:2393] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 4: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
W20260709 13:16:41.922534 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 45371.7us
I20260709 13:16:41.932107 10841 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:41.879693 12842 consensus_queue.cc:1048] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 13:16:41.927119 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:41.920811 12756 consensus_queue.cc:237] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:41.910717 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:41.932852 10876 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 5.
I20260709 13:16:41.933791 10841 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:41.904908 12775 consensus_queue.cc:1048] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.021s
I20260709 13:16:41.922039 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:41.955484 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:41.956218 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:41.956786 10876 raft_consensus.cc:2410] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:16:41.956928 10874 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 4.
I20260709 13:16:41.958447 10874 raft_consensus.cc:1240] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:16:41.959738 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 207) took 1476 ms. Trace:
I20260709 13:16:41.959964 12508 consensus_queue.cc:260] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:41.960235 10873 rpcz_store.cc:276] 0709 13:16:40.482976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:40.487663 (+  4687us) service_pool.cc:224] Handling call
0709 13:16:40.504793 (+ 17130us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:41.017887 (+513094us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:41.959498 (+941611us) spinlock_profiling.cc:243] Waited 511 ms on lock 0x7b54000f78d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:41.959677 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1176389376}
I20260709 13:16:41.961287 12508 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:41.961321 10946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41020 (request call id 1688) took 1254 ms. Trace:
I20260709 13:16:41.929625 10916 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:41.924012 12863 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:41.905506 12725 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:41.934407 12859 raft_consensus.cc:2749] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:41.943176 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:41.964174 10947 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:41.913719 12865 raft_consensus.cc:697] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 6 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:41.951434 11478 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:41.953835 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:41.882521 12697 consensus_queue.cc:260] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:41.913352 12755 raft_consensus.cc:740] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:41.962020 10946 rpcz_store.cc:276] 0709 13:16:40.706439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:40.708830 (+  2391us) service_pool.cc:224] Handling call
0709 13:16:41.961087 (+1252257us) spinlock_profiling.cc:243] Waited 370 ms on lock 0x7b54000a96d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:41.961243 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":851784960}
I20260709 13:16:41.953276 11511 raft_consensus.cc:1275] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:16:41.926493 12798 raft_consensus.cc:2804] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:41.979023 12599 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:41.981814 12798 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:41.982059 12599 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:41.984581 12599 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:41.985070 12697 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:16:41.985355 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 51608.4us
I20260709 13:16:41.987259 10874 raft_consensus.cc:1275] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:16:41.988665 10874 raft_consensus.cc:1240] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:41.992149 10872 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 6.
I20260709 13:16:41.994536 10841 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:41.921288 12289 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:41.975744 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 46171.4us
I20260709 13:16:41.999913 10916 leader_election.cc:400] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 election: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 3. Current term is 4.
I20260709 13:16:42.000329 10916 leader_election.cc:403] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:16:41.930634 12768 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:41.982470 12798 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:41.968385 12863 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:42.038626 10916 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:41.997421 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:42.040504 10872 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 1.
I20260709 13:16:42.001328 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:42.041754 10947 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 2.
I20260709 13:16:42.003212 10841 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:41.940783 12654 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:41.947724 12701 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:16:42.019256 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 67919us
I20260709 13:16:42.021603 12725 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:42.028868 12854 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:42.047185 10873 raft_consensus.cc:1240] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:42.047746 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:42.036119 12778 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:41.996691 12622 raft_consensus.cc:2749] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:16:42.063019 12723 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.811s	user 0.028s	sys 0.059s
W20260709 13:16:42.063409 12723 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.813s	user 0.028s	sys 0.059s
I20260709 13:16:42.064229 10942 raft_consensus.cc:1275] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
W20260709 13:16:42.064888 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:42.054566 10947 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:16:42.060406 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 57271.9us
I20260709 13:16:42.066905 12688 consensus_queue.cc:1059] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:42.067277 10841 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:42.067775 12766 consensus_queue.cc:1048] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:42.068078 12697 raft_consensus.cc:2749] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:42.054744 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:42.016820 12872 raft_consensus.cc:2764] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:16:42.067812 12688 raft_consensus.cc:3055] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:42.015928 12858 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:42.068958 10945 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
I20260709 13:16:42.069089 12688 raft_consensus.cc:740] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:42.069484 12858 raft_consensus.cc:697] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 5 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
W20260709 13:16:42.059969 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 127426us
I20260709 13:16:42.036760 12843 raft_consensus.cc:2804] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:16:42.073014 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:42.070314 10945 raft_consensus.cc:1240] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:42.068389 10841 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:42.074502 10868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 1327) took 1225 ms. Trace:
I20260709 13:16:42.056975 12654 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:42.070261 12688 consensus_queue.cc:260] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:42.080412 12688 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:42.083602 12289 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:16:42.084856 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:41.951609 12831 consensus_queue.cc:260] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:42.085597 10872 raft_consensus.cc:1275] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:16:42.085990 10872 pending_rounds.cc:85] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 1
I20260709 13:16:42.086123 12831 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:42.074914 10868 rpcz_store.cc:276] 0709 13:16:40.848922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:40.863404 (+ 14482us) service_pool.cc:224] Handling call
0709 13:16:42.074316 (+1210912us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b540006b3d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:42.074441 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":416980352}
I20260709 13:16:42.087848 12622 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:42.089766 12622 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:42.078846 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:42.086258 10872 pending_rounds.cc:107] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:16:42.091409 10872 raft_consensus.cc:2889] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:42.091928 12622 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:42.072270 12701 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:42.093519 12771 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:42.094280 12701 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:42.095134 10875 raft_consensus.cc:2393] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 4: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:42.095618 10870 raft_consensus.cc:1275] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:16:42.096804 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:16:42.095633 12773 consensus_meta.cc:287] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.520s	user 0.010s	sys 0.000s
I20260709 13:16:42.098101 12773 raft_consensus.cc:2749] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:42.098150 12771 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:42.072943 12843 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:42.099170 12843 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:42.099350 12654 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:42.054041 12739 raft_consensus.cc:2749] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 3. Current term is 4.
I20260709 13:16:42.105239 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:42.102947 10947 raft_consensus.cc:1275] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:16:42.108613 10766 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:42.061333 12804 consensus_queue.cc:1048] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.058s
I20260709 13:16:42.116533 12766 raft_consensus.cc:2804] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:42.117050 12766 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:42.117424 12766 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:42.118985 12838 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:42.088951 10798 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884"
dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
mode: GRACEFUL
new_leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:16:42.119593 12838 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:42.060811 12617 raft_consensus.cc:2804] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:42.071247 12768 raft_consensus.cc:2804] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:42.119961 10798 raft_consensus.cc:606] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Received request to transfer leadership to TS 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:16:42.122552 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:42.128732 10798 raft_consensus.cc:614] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:42.070333 12858 consensus_queue.cc:237] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:42.194453 10766 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:16:42.195654 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 127338us
I20260709 13:16:42.194314 10868 raft_consensus.cc:1275] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
W20260709 13:16:42.211086 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 44280.8us
I20260709 13:16:42.175540 10874 raft_consensus.cc:1275] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:16:42.220388 10767 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:42.176376 12683 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:42.221527 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:42.221750 12683 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:42.222347 12583 raft_consensus.cc:2749] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:42.197475 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:42.223381 10767 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:42.238795 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:42.239553 10870 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 5.
I20260709 13:16:42.253482 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:42.235919 10872 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 4.
I20260709 13:16:42.059062 12755 consensus_queue.cc:260] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:42.258522 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:42.261951 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:42.262288 10767 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:42.262993 10949 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
I20260709 13:16:42.266059 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:42.267179 10947 raft_consensus.cc:2393] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 3: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
W20260709 13:16:42.268239 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 195004us
I20260709 13:16:42.269729 10767 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:42.115644 12654 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:42.128052 12768 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:42.288900 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:42.290593 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:42.286172 11478 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:42.293530 11478 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:42.239706 12701 raft_consensus.cc:2804] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:42.299180 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:42.198920 12617 raft_consensus.cc:697] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 3 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:16:42.279999 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:42.280575 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:42.280874 12683 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:42.281888 12776 raft_consensus.cc:2749] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:42.282473 12739 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:42.312978 12739 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:42.282974 12838 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:42.283730 10942 raft_consensus.cc:1275] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:16:42.284058 10867 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 4.
I20260709 13:16:42.318759 12739 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:42.141973 12289 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:42.285483 10916 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:42.328398 12289 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:42.328811 12289 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:16:42.328902 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 44003.1us
I20260709 13:16:42.197834 12831 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:42.331875 12831 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:42.332391 12804 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:42.332834 12622 consensus_queue.cc:1048] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:42.224614 12684 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:42.193039 12842 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:42.340394 10916 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:42.274610 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:42.276264 12773 consensus_queue.cc:1048] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 13:16:42.129051 12833 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:42.306317 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:42.298038 12574 raft_consensus.cc:2749] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:42.287585 12784 raft_consensus.cc:1064] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2: attempting to promote NON_VOTER 12682eb70bc84c76879e04a8628ee586 to VOTER
I20260709 13:16:42.272347 12836 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:42.274132 12873 raft_consensus.cc:2749] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:42.276772 12854 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:42.311060 10941 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:42.321225 10799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 1724) took 1585 ms. Trace:
I20260709 13:16:42.324296 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:42.319013 10942 pending_rounds.cc:85] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 5
W20260709 13:16:42.320107 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:42.320183 10867 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 1325) took 1504 ms (client timeout 1563 ms). Trace:
W20260709 13:16:42.284870 12880 log.cc:927] Time spent T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:16:42.415176 12756 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:42.318598 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:42.296783 12768 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:42.415758 12756 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:42.416201 10945 raft_consensus.cc:2376] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4. Re-sending same reply.
I20260709 13:16:42.169847 10943 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 5.
I20260709 13:16:42.287022 12881 consensus_queue.cc:1059] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.100s
I20260709 13:16:42.342105 10949 raft_consensus.cc:2376] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4. Re-sending same reply.
I20260709 13:16:42.425976 12768 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:42.076808 12865 consensus_queue.cc:237] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:42.427738 10797 raft_consensus.cc:1275] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:16:42.428824 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:42.298895 12723 raft_consensus.cc:2749] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:42.299470 12701 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:42.429425 10797 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 3.
I20260709 13:16:42.429734 12701 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:42.430321 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:42.431135 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:42.431757 10793 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
I20260709 13:16:42.432660 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:42.432762 10842 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:42.300503 10947 raft_consensus.cc:2393] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 3: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:42.433849 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:42.352766 10841 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:42.332271 12755 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:42.372604 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:16:42.374029 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:42.388846 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:42.385690 10942 pending_rounds.cc:107] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:16:42.436645 10942 raft_consensus.cc:2889] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:42.384725 10799 rpcz_store.cc:276] 0709 13:16:40.758155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:40.759835 (+  1680us) service_pool.cc:224] Handling call
0709 13:16:40.816903 (+ 57068us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:41.343812 (+526909us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:42.320958 (+977146us) spinlock_profiling.cc:243] Waited 524 ms on lock 0x7b5400385c54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:42.321143 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1204678016}
I20260709 13:16:42.397226 10795 raft_consensus.cc:1240] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:42.437153 10950 raft_consensus.cc:2393] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 4: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:42.437875 10798 raft_consensus.cc:1275] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:16:42.435979 11512 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 4.
W20260709 13:16:42.392782 10867 rpcz_store.cc:269] 0709 13:16:40.815393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:40.827221 (+ 11828us) service_pool.cc:224] Handling call
0709 13:16:41.793349 (+966128us) spinlock_profiling.cc:243] Waited 5.86 ms on lock 0x7b5800177098. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd90a22 00007f9c86ca9c09 00007f9c85e03ee0 00007f9c85ecd0ab 00007f9c85ecd131 00007f9c85e7b0a0 00007f9c85e7a984 00007f9c7f3f56d4 00007f9c7f3f5662
0709 13:16:42.319931 (+526582us) spinlock_profiling.cc:243] Waited 205 ms on lock 0x7b54003165d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:42.320090 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":485743232}
I20260709 13:16:42.403378 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:42.403870 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:42.404623 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:42.405933 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:42.406620 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:42.407629 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:42.407799 10941 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
I20260709 13:16:42.304695 11478 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:42.438264 10797 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
I20260709 13:16:42.439180 10946 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
I20260709 13:16:42.440827 10792 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:42.439463 10796 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:42.297500 12852 consensus_queue.cc:1048] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.093s
W20260709 13:16:42.446336 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 176696us
I20260709 13:16:42.448999 10767 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:16:42.439314 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 236447us
W20260709 13:16:42.450986 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:42.384995 12739 raft_consensus.cc:2749] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:16:42.342693 12778 consensus_queue.cc:1048] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.095s
W20260709 13:16:42.456413 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 151850us
I20260709 13:16:42.457254 11478 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:42.460573 10767 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:42.462679 10767 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:42.463553 12583 raft_consensus.cc:2749] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:42.370553 12842 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:42.437619 12831 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:42.471822 10767 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:42.436879 12804 raft_consensus.cc:697] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 4 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:42.374275 12761 consensus_queue.cc:1048] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.079s
I20260709 13:16:42.479054 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:42.479401 10792 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 6.
I20260709 13:16:42.500897 10796 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 3.
I20260709 13:16:42.386564 12872 consensus_queue.cc:1048] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 13:16:42.471251 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:42.385339 12836 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:42.517526 10946 raft_consensus.cc:1275] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:16:42.516413 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:42.512130 10798 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2"
dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
mode: GRACEFUL
new_leader_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:16:42.539355 10798 raft_consensus.cc:606] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Received request to transfer leadership to TS cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:42.539860 10798 raft_consensus.cc:614] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:42.537050 10796 raft_consensus.cc:1240] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 5. Current term is 6. Ops: []
W20260709 13:16:42.530769 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:42.544443 10950 raft_consensus.cc:2393] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 5: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
W20260709 13:16:42.546289 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 113642us
I20260709 13:16:42.459362 12873 raft_consensus.cc:2749] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:16:42.556583 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 203921us
I20260709 13:16:42.503935 12775 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:42.516646 10795 raft_consensus.cc:1275] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:16:42.576262 12899 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:42.576821 12899 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:42.577155 12899 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:42.578773 12289 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:42.580376 12684 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:42.580991 12684 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:42.581148 12289 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:42.581954 12654 raft_consensus.cc:2749] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:42.568171 10800 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
I20260709 13:16:42.563907 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:42.584838 10792 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 3.
I20260709 13:16:42.572901 10841 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:42.466487 12599 consensus_queue.cc:1059] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.095s
I20260709 13:16:42.550218 12889 raft_consensus.cc:2749] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:42.571859 12775 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:42.602190 12775 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:42.603394 12893 raft_consensus.cc:3055] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:42.603928 12893 raft_consensus.cc:740] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:42.605044 12893 consensus_queue.cc:260] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:42.609680 12853 consensus_meta.cc:287] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.696s	user 0.008s	sys 0.000s
I20260709 13:16:42.611495 12761 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:42.612195 12761 raft_consensus.cc:697] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:42.613330 12761 consensus_queue.cc:237] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:42.615051 12654 raft_consensus.cc:2749] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:42.592535 12836 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:42.595845 12798 consensus_meta.cc:287] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.543s	user 0.005s	sys 0.005s
I20260709 13:16:42.596710 12768 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:42.539587 12831 consensus_queue.cc:1048] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.047s
I20260709 13:16:42.513290 12617 consensus_queue.cc:237] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
W20260709 13:16:42.599469 12863 consensus_meta.cc:287] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.540s	user 0.004s	sys 0.004s
I20260709 13:16:42.599586 12833 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:42.599982 12842 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:42.600595 12804 consensus_queue.cc:237] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:42.562216 12697 consensus_queue.cc:1048] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
W20260709 13:16:42.617835 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 248664us
I20260709 13:16:42.610124 12701 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:42.623600 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 71761.7us
I20260709 13:16:42.633571 10916 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:16:42.624394 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 167229us
W20260709 13:16:42.619571 12766 consensus_meta.cc:287] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.502s	user 0.008s	sys 0.002s
I20260709 13:16:42.641975 12766 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:42.644380 10916 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:42.645953 12778 consensus_queue.cc:1059] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:42.647065 12778 consensus_queue.cc:1059] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:42.647336 12895 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:42.648820 12778 raft_consensus.cc:3055] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:42.649163 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:42.649910 10867 raft_consensus.cc:2368] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 5. Current term is 6.
W20260709 13:16:42.650077 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:42.650923 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:42.649204 12778 raft_consensus.cc:740] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:42.653501 12778 consensus_queue.cc:260] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:42.656212 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:42.657017 10867 raft_consensus.cc:2393] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 4: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:42.656870 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:42.657568 12684 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:42.659547 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:42.631816 12768 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:42.662355 12768 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:42.662614 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:42.662756 12768 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:42.663130 10800 raft_consensus.cc:1240] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:42.663806 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:42.664527 10948 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 5.
I20260709 13:16:42.664846 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:42.663839 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:42.665865 10867 raft_consensus.cc:2393] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 3: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:42.665745 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:42.659547 12778 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:42.666715 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:42.629002 12898 consensus_queue.cc:1048] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.042s
W20260709 13:16:42.628815 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 126199us
I20260709 13:16:42.610258 12853 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:42.677326 11478 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:42.665757 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:42.675933 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:42.642097 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:42.679388 10792 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:42.680724 10871 raft_consensus.cc:1240] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 5. Current term is 6. Ops: []
W20260709 13:16:42.625391 12843 consensus_meta.cc:287] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.526s	user 0.009s	sys 0.002s
I20260709 13:16:42.685827 10916 leader_election.cc:400] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 5. Current term is 6.
I20260709 13:16:42.686313 10916 leader_election.cc:403] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:16:42.686136 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:42.687209 10873 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 4.
I20260709 13:16:42.692191 10798 raft_consensus.cc:1240] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:42.693742 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:42.694380 10796 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 4.
I20260709 13:16:42.622484 12798 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:42.693145 12739 raft_consensus.cc:2749] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 5. Current term is 6.
I20260709 13:16:42.690057 12843 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:42.699962 12783 consensus_queue.cc:1059] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:42.703917 12374 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:42.710623 11478 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:42.704979 12899 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:42.707224 10944 raft_consensus.cc:2393] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 5: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:42.654515 12775 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:42.644454 12766 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:42.722384 10916 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:16:42.747668 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:42.747768 10796 raft_consensus.cc:1240] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 5. Current term is 6. Ops: []
W20260709 13:16:42.748175 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 62694.9us
I20260709 13:16:42.748281 10767 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:42.736864 10792 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 6.
I20260709 13:16:42.690910 12894 raft_consensus.cc:2749] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:42.630213 12863 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:42.691113 12898 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:42.755499 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 44994.6us
I20260709 13:16:42.761880 11478 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:16:42.748903 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 179852us
I20260709 13:16:42.766376 10767 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:42.713078 12889 consensus_queue.cc:1048] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:16:42.771679 10766 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:42.755434 12739 raft_consensus.cc:3055] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:42.756956 12859 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:42.789458 10841 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:42.790525 11478 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:42.690074 12838 consensus_queue.cc:1048] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.077s
I20260709 13:16:42.755540 12783 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:42.804220 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:42.784473 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:42.793463 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:42.807394 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:42.808187 10870 raft_consensus.cc:2393] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 6: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:42.805578 10869 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 4.
I20260709 13:16:42.747285 12374 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:42.807180 10873 raft_consensus.cc:2393] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 3: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:42.776409 12836 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:42.779995 12583 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:42.805001 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:42.808779 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:42.818361 10799 raft_consensus.cc:2368] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 2. Current term is 3.
I20260709 13:16:42.779495 12289 raft_consensus.cc:2764] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:16:42.784508 12725 raft_consensus.cc:2749] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:42.799624 12853 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:42.787117 12739 raft_consensus.cc:740] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:42.781872 12833 consensus_queue.cc:1048] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 13:16:42.793154 12895 raft_consensus.cc:697] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 4 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:42.804360 12900 raft_consensus.cc:2749] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:42.819701 12859 raft_consensus.cc:697] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 3 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:42.851147 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:42.877866 10797 raft_consensus.cc:2393] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 5: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:42.878419 11507 raft_consensus.cc:1240] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:42.846572 12773 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:42.816975 12836 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:42.819679 12583 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:42.775063 12903 consensus_queue.cc:1048] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.060s
W20260709 13:16:42.856900 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 67984.7us
I20260709 13:16:42.858673 10916 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:42.821141 12766 raft_consensus.cc:2749] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:42.799959 12894 consensus_queue.cc:1048] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
W20260709 13:16:42.862653 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:42.863371 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:42.776929 12898 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:42.827158 12854 raft_consensus.cc:2749] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:42.813746 12783 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:42.894040 12701 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:42.819574 12672 consensus_queue.cc:1048] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:16:42.908003 12899 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:42.854651 10842 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:16:42.909250 12897 log.cc:927] Time spent T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 13:16:42.909509 12783 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:42.840631 12684 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:16:42.909775 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 55185.9us
I20260709 13:16:42.910357 12684 raft_consensus.cc:2749] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:42.910611 10842 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:42.911913 12684 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:16:42.876425 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:42.913241 11504 raft_consensus.cc:2368] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 5. Current term is 6.
I20260709 13:16:42.913558 10842 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:42.913714 12768 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:42.914465 12904 raft_consensus.cc:2804] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:42.915025 12904 raft_consensus.cc:697] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:42.915345 12775 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:42.915987 12904 consensus_queue.cc:237] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:42.917469 12843 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:42.917585 12898 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:42.918720 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:42.921680 12873 consensus_queue.cc:1048] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:42.921973 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:42.906214 10945 raft_consensus.cc:1275] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:16:42.906350 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 47801.3us
I20260709 13:16:42.905200 10841 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:42.882755 12622 raft_consensus.cc:2764] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:16:42.918913 11507 raft_consensus.cc:1240] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:42.926566 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:42.907496 12863 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:42.926954 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:42.925181 10842 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:42.864570 12853 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:42.935360 10870 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 4.
I20260709 13:16:42.936115 10945 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:42.937832 10842 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:42.940949 12374 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:42.940866 10916 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:42.942408 12771 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:42.942173 10841 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:42.943229 12899 consensus_queue.cc:1059] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:42.891597 12289 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:42.944409 12843 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:42.944975 12739 consensus_queue.cc:260] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:42.945376 12783 raft_consensus.cc:2749] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:42.946239 12739 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:42.944666 12289 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:42.894816 12773 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:42.947038 12783 consensus_queue.cc:1059] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:16:42.948473 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:42.949074 12783 raft_consensus.cc:3055] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:42.949865 12783 raft_consensus.cc:740] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:42.949213 10797 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 5.
I20260709 13:16:42.950631 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:42.951654 10916 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:42.951305 12859 consensus_queue.cc:237] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:42.904708 12583 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:42.894465 12836 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:42.942238 12842 consensus_queue.cc:1048] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:42.939834 10842 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:42.956548 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:42.962374 10873 raft_consensus.cc:2393] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 5: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:42.926224 12617 raft_consensus.cc:2749] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:42.967517 10870 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:16:42.965713 10766 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:42.931522 12443 raft_consensus.cc:2749] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:42.968243 10870 raft_consensus.cc:606] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Received request to transfer leadership to TS cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:42.969769 10870 raft_consensus.cc:614] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:42.969620 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:42.942946 12771 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:42.938943 12853 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:42.972370 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:42.975266 10873 raft_consensus.cc:1275] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:16:42.979691 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:42.958760 12899 raft_consensus.cc:3055] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:42.984936 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:42.960642 12599 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:42.959028 12843 raft_consensus.cc:697] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:42.986982 10841 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:42.939455 12672 consensus_queue.cc:1048] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:16:42.864326 12784 consensus_queue.cc:237] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 2, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:43.000244 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:43.000957 10869 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:42.985785 12863 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.003803 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:42.988631 12599 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:43.008203 12599 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:43.008607 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:16:43.010174 12910 log.cc:927] Time spent T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 13:16:42.971213 12539 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:42.974994 12853 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:43.011099 12539 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:43.013474 12539 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:42.975127 12771 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:42.988934 10915 leader_election.cc:400] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 5. Current term is 6.
I20260709 13:16:43.016271 10915 leader_election.cc:403] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:16:43.016878 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:43.017518 10873 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:42.961580 12798 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:43.019244 12778 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:42.986634 12899 raft_consensus.cc:740] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:43.022166 12899 consensus_queue.cc:260] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:43.023444 12899 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:43.018411 10915 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:42.993784 10766 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:42.997954 10945 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
W20260709 13:16:43.040802 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 47145.8us
I20260709 13:16:43.041565 10766 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:43.042023 12843 consensus_queue.cc:237] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:43.043391 10766 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:43.043727 12756 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:42.969525 12898 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:43.044625 10766 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:43.025579 12905 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.026651 12374 consensus_queue.cc:1059] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:16:43.027230 12599 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:43.004588 12903 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:43.029095 10873 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 4.
I20260709 13:16:43.029543 10797 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed"
dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
mode: GRACEFUL
new_leader_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:16:43.006057 12863 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:43.010473 10841 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:43.055956 10797 raft_consensus.cc:606] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Received request to transfer leadership to TS cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:42.982056 12766 raft_consensus.cc:2804] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:43.033149 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:43.056749 10797 raft_consensus.cc:614] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:43.056807 12872 raft_consensus.cc:2749] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:43.057073 12766 raft_consensus.cc:697] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:43.057240 10872 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 4.
I20260709 13:16:43.039840 12739 raft_consensus.cc:2749] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:43.040017 10796 raft_consensus.cc:1275] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:16:43.044273 12756 raft_consensus.cc:2749] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:42.951269 12783 consensus_queue.cc:260] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:43.053748 12905 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:43.044745 12701 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:16:43.032522 12755 consensus_meta.cc:287] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.594s	user 0.012s	sys 0.000s
W20260709 13:16:43.039439 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:43.084708 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:43.060117 10872 raft_consensus.cc:1240] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:43.044662 12898 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:43.067487 10868 raft_consensus.cc:1275] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:16:43.003358 12916 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.069340 10767 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:43.071827 10800 raft_consensus.cc:1240] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:16:43.033103 12919 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:43.052373 12889 raft_consensus.cc:2749] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:16:43.073772 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 133992us
I20260709 13:16:43.051134 12854 raft_consensus.cc:2804] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:16:43.077677 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 67305.1us
I20260709 13:16:43.108167 10841 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:43.078370 12617 consensus_queue.cc:1059] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:16:43.081595 10869 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 6.
I20260709 13:16:43.091328 10793 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 4.
I20260709 13:16:43.066195 12775 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.061046 12904 raft_consensus.cc:2749] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:43.108589 10842 leader_election.cc:400] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 election: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 2. Current term is 3.
I20260709 13:16:43.116029 10842 leader_election.cc:403] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:16:43.069963 12776 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:43.113086 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 66040.5us
I20260709 13:16:43.118743 10800 raft_consensus.cc:1240] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260709 13:16:43.109660 10766 consensus_peers.cc:597] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 -> Peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Couldn't send request to peer 12682eb70bc84c76879e04a8628ee586. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 171abdebfc4142cda86de477a5c89a7e. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:43.120539 10766 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:43.121995 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:43.124334 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:43.125087 10794 raft_consensus.cc:2393] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 5: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:43.127398 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:43.091804 12898 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:43.143361 10794 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:43.144594 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:43.107720 12854 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:43.112754 12768 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:43.151345 10872 raft_consensus.cc:2393] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 5: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:43.154865 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:43.155766 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:43.159389 10841 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:43.159708 10798 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
I20260709 13:16:43.126503 12697 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:43.117167 12918 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.159340 10796 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:43.166849 10794 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
I20260709 13:16:43.146555 12843 raft_consensus.cc:2749] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:42.999241 12895 consensus_queue.cc:237] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:43.155444 12854 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:43.118527 12775 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:43.116957 12583 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:43.178200 10796 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
I20260709 13:16:43.116616 12913 consensus_queue.cc:1048] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.056s
I20260709 13:16:43.188601 10798 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e41482a95776477784d9bd9dacfed231"
dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
mode: GRACEFUL
new_leader_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:16:43.192404 10798 raft_consensus.cc:606] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Received request to transfer leadership to TS cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:43.192792 10798 raft_consensus.cc:614] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:43.159242 12889 raft_consensus.cc:2749] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:16:43.174587 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 105376us
I20260709 13:16:43.193619 10767 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:43.151279 12776 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:43.109946 12766 consensus_queue.cc:237] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:43.179112 12787 raft_consensus.cc:3055] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:43.189935 12768 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:43.110044 12919 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:43.190363 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 47035.3us
I20260709 13:16:43.203143 10799 raft_consensus.cc:1240] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:43.210673 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:43.194567 12289 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:43.110189 12916 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:43.171548 12918 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:43.251971 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:43.292526 10870 raft_consensus.cc:1275] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:16:43.280756 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:43.289089 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:43.270828 11477 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:43.282943 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:43.264312 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:16:43.283174 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 89602.6us
I20260709 13:16:43.263556 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:43.317869 10798 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:16:43.321949 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 165459us
W20260709 13:16:43.353547 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 82857.5us
I20260709 13:16:43.220359 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:43.323676 10767 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:43.336685 10796 raft_consensus.cc:1275] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:16:43.361990 10870 raft_consensus.cc:1240] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 2. Current term is 3. Ops: []
W20260709 13:16:43.293509 12927 log.cc:927] Time spent T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 13:16:43.363758 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:43.366808 10945 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 4.
W20260709 13:16:43.294423 12929 log.cc:927] Time spent T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:16:43.368726 11477 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:43.203122 12773 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:43.200843 12787 raft_consensus.cc:740] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:43.376638 12940 raft_consensus.cc:2749] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:43.293587 12936 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:43.372239 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 48663.1us
I20260709 13:16:43.282261 12863 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:43.221755 12768 raft_consensus.cc:697] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:43.288573 12599 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:43.374444 10871 raft_consensus.cc:1275] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 3. (term mismatch)
I20260709 13:16:43.342260 12784 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.250774 12804 consensus_queue.cc:1048] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.126s
I20260709 13:16:43.371831 10841 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:43.295365 12905 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:43.382237 10767 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:43.290557 12934 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:43.293283 12853 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:43.385097 10796 raft_consensus.cc:1275] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:16:43.387634 10941 raft_consensus.cc:1275] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:16:43.362694 12671 raft_consensus.cc:2804] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:43.389235 10870 raft_consensus.cc:1275] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:16:43.366559 12894 consensus_queue.cc:1048] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:16:43.295185 12935 raft_consensus.cc:2749] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:16:43.409310 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:43.390810 10871 pending_rounds.cc:85] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 2
I20260709 13:16:43.419616 10871 pending_rounds.cc:107] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 2.3
I20260709 13:16:43.420030 10871 raft_consensus.cc:2889] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:43.420775 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:43.421424 10949 raft_consensus.cc:2410] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:16:43.422053 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:43.424696 10870 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 5.
I20260709 13:16:43.428907 10767 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:43.430392 12940 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:43.430938 12940 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:43.431396 12940 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:43.432304 12289 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.432855 12289 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:43.443403 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:43.399358 12934 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:43.401314 12773 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:16:43.451225 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:16:42.955168 12912 consensus_queue.cc:1175] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" responder_term: 6 status { last_received { term: 4 index: 3 } last_committed_idx: 3 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 5. Current term is 6. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:16:43.452005 10871 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 4.
I20260709 13:16:43.453505 10767 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:16:43.457767 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:43.460592 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 88837.3us
I20260709 13:16:43.461724 10841 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:43.441382 12843 raft_consensus.cc:2749] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:43.443928 12900 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:43.400540 12784 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:43.453840 12924 log.cc:927] Time spent T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.277s	user 0.001s	sys 0.000s
I20260709 13:16:43.466874 10841 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:43.466487 12905 raft_consensus.cc:2749] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:16:43.477290 12951 log.cc:927] Time spent T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 13:16:43.445995 12671 raft_consensus.cc:697] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 5 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:43.485719 10841 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
W20260709 13:16:43.486559 12928 log.cc:927] Time spent T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.226s	user 0.001s	sys 0.000s
I20260709 13:16:43.449443 12934 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:43.386053 12936 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:43.377125 12787 consensus_queue.cc:260] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:43.501855 12939 log.cc:927] Time spent T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 13:16:43.459702 12889 raft_consensus.cc:2804] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:16:43.504997 12932 log.cc:927] Time spent T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.230s	user 0.000s	sys 0.001s
I20260709 13:16:43.472216 12900 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:43.482746 12899 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.481707 12776 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:43.506762 12889 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:43.516556 12771 consensus_meta.cc:287] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.501s	user 0.010s	sys 0.000s
W20260709 13:16:43.536609 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:43.516252 12900 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:43.506580 12863 consensus_queue.cc:1059] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:16:43.488281 12919 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:43.533890 12787 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:16:43.517098 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 66725.8us
W20260709 13:16:43.549830 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 181177us
I20260709 13:16:43.525041 12899 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:43.555626 12671 consensus_queue.cc:237] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:43.536504 12889 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:43.508579 12768 consensus_queue.cc:237] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:43.540174 12771 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:43.617550 10801 raft_consensus.cc:1275] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:43.619340 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:43.572049 12936 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:43.580648 12918 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:43.625864 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:43.593310 12916 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:43.632083 12925 raft_consensus.cc:2764] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won
I20260709 13:16:43.624490 12853 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:43.625437 12872 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.658901 12872 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:43.647795 12778 consensus_meta.cc:287] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.628s	user 0.005s	sys 0.006s
W20260709 13:16:43.669165 12836 consensus_meta.cc:287] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.701s	user 0.008s	sys 0.003s
I20260709 13:16:43.669636 12872 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:43.658282 12854 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:43.686014 10798 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 6.
I20260709 13:16:43.650489 12881 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.705654 12881 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:43.711172 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:43.674633 12784 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:43.704054 10796 raft_consensus.cc:1240] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:43.695056 12962 ts_tablet_manager.cc:933] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586: Initiating tablet copy from peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685)
W20260709 13:16:43.683312 12697 consensus_meta.cc:287] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.514s	user 0.009s	sys 0.002s
I20260709 13:16:43.674714 12778 raft_consensus.cc:2749] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 5. Current term is 6.
I20260709 13:16:43.647588 12940 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:43.682057 12771 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:43.714444 12766 raft_consensus.cc:2804] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:43.718055 12872 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.738904 10873 raft_consensus.cc:1275] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:16:43.741001 12940 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:43.669658 12836 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:43.745939 10873 pending_rounds.cc:85] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 5
I20260709 13:16:43.748778 10948 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 6.
I20260709 13:16:43.754448 10873 pending_rounds.cc:107] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:16:43.754897 10873 raft_consensus.cc:2889] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:43.718642 12881 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:43.735512 12697 raft_consensus.cc:2749] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 2. Current term is 3.
I20260709 13:16:43.756309 12925 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.770295 12925 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:43.756372 12899 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:43.780269 12886 log.cc:927] Time spent T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 1.224s	user 0.003s	sys 0.003s
W20260709 13:16:43.780316 12923 log.cc:927] Time spent T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.659s	user 0.002s	sys 0.000s
I20260709 13:16:43.782020 10793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 1338) took 1132 ms. Trace:
I20260709 13:16:43.782765 10876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1339) took 2054 ms. Trace:
I20260709 13:16:43.783366 10876 rpcz_store.cc:276] 0709 13:16:41.728284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:41.975968 (+247684us) service_pool.cc:224] Handling call
0709 13:16:42.017578 (+ 41610us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:42.017820 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:42.017844 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:42.060237 (+ 42393us) log.cc:844] Serialized 26 byte log entry
0709 13:16:42.336199 (+275962us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:42.336261 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:42.336580 (+   319us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:43.773375 (+1436795us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:16:43.782487 (+  9112us) raft_consensus.cc:1685] finished
0709 13:16:43.782508 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:43.782709 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23808768,"thread_start_us":214657,"threads_started":1,"wal-append.queue_time_us":389043}
I20260709 13:16:43.767750 12961 raft_consensus.cc:3055] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:43.787031 12961 raft_consensus.cc:740] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:43.787179 12854 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:43.755069 12872 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:43.782456 10793 rpcz_store.cc:276] 0709 13:16:42.649898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:42.650564 (+   666us) service_pool.cc:224] Handling call
0709 13:16:43.781815 (+1131251us) spinlock_profiling.cc:243] Waited 440 ms on lock 0x7b5400211bd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:43.781938 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1010961280}
I20260709 13:16:43.791035 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:43.791103 10798 raft_consensus.cc:1240] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:16:43.796738 10801 raft_consensus.cc:1240] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:16:43.796813 10800 raft_consensus.cc:1275] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:16:43.797390 10874 raft_consensus.cc:1275] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:16:43.798074 10874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1343) took 1016 ms. Trace:
W20260709 13:16:43.801712 12898 consensus_meta.cc:287] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.642s	user 0.007s	sys 0.006s
I20260709 13:16:43.802227 10793 raft_consensus.cc:1240] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:43.808602 12671 consensus_queue.cc:1048] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:43.812196 12881 consensus_queue.cc:1059] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:16:43.788070 12961 consensus_queue.cc:260] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:43.812834 12854 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.820698 10869 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
I20260709 13:16:43.798398 10874 rpcz_store.cc:276] 0709 13:16:42.781942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:42.784980 (+  3038us) service_pool.cc:224] Handling call
0709 13:16:43.796943 (+1011963us) spinlock_profiling.cc:243] Waited 5.44 ms on lock 0x7b5800177098. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd90eee 0000000000375939 00007f9c85e041e0 00007f9c85ecd0c4 00007f9c85ecd131 00007f9c85e8efc0 00007f9c85e79929 00007f9c85e793d2 00007f9c7f3f0bd4
0709 13:16:43.797096 (+   153us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:43.797832 (+   736us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:43.798014 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12533376}
I20260709 13:16:43.835772 10867 raft_consensus.cc:1240] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:43.802121 12898 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:43.837344 12701 consensus_meta.cc:287] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.754s	user 0.013s	sys 0.000s
I20260709 13:16:43.837772 12701 raft_consensus.cc:2749] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:43.838546 12775 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:43.834388 12854 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:43.837822 12898 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:43.846444 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:43.851169 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:43.851800 10949 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:43.844785 12852 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:43.839599 12889 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:43.859076 10949 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 6.
I20260709 13:16:43.861723 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:43.865337 10941 raft_consensus.cc:2368] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 2. Current term is 3.
I20260709 13:16:43.865520 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:43.866070 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:43.866483 10941 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:43.862413 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:43.862725 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:43.833791 12671 consensus_queue.cc:1059] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 13:16:43.867731 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:43.846292 12778 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.869436 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:43.872200 10841 leader_election.cc:400] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 2 election: Vote denied by peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553) with higher term. Message: Invalid argument: T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 2. Current term is 3.
I20260709 13:16:43.874651 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:43.849617 12784 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.879988 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:43.881282 10949 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 3.
I20260709 13:16:43.881975 10947 raft_consensus.cc:2410] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 4 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 13:16:43.882165 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:43.882756 10949 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:43.884095 10841 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:43.873270 12925 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:43.855654 12852 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:43.830870 12902 consensus_queue.cc:1059] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:16:43.893508 10941 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
I20260709 13:16:43.863983 12872 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:43.895258 10841 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:43.891582 10945 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
I20260709 13:16:43.899189 10841 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:43.900147 10949 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
I20260709 13:16:43.901239 10841 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:43.902002 12968 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:43.902612 12968 raft_consensus.cc:697] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:16:43.904971 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:43.890311 12843 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:43.879678 12778 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:43.917565 12872 raft_consensus.cc:2804] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:43.918376 12872 raft_consensus.cc:697] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:43.919625 12872 consensus_queue.cc:237] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:43.907064 12775 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:43.921164 12775 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:43.923295 12775 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:43.924283 12843 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:43.924767 12843 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:43.888130 12784 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:43.929093 12784 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:16:43.930203 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:43.931576 10766 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:43.933068 10766 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:43.903604 12968 consensus_queue.cc:237] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:43.937834 12574 raft_consensus.cc:3055] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:43.938326 12574 raft_consensus.cc:740] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:43.940093 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:43.899923 12889 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:43.939463 10766 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:43.906081 12925 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:43.946311 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:43.951551 11510 raft_consensus.cc:1275] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 2 LEARNER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:16:43.938844 12913 raft_consensus.cc:2749] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:16:43.873615 12900 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:43.943734 12701 raft_consensus.cc:2804] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:43.950953 12940 raft_consensus.cc:2749] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:43.954370 12925 raft_consensus.cc:697] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:43.968170 11512 raft_consensus.cc:2393] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 5: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:43.961309 12962 tablet_copy_client.cc:323] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586: tablet copy: Beginning tablet copy session from remote peer at address 127.9.116.129:39685
I20260709 13:16:43.962208 12778 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:43.963536 12701 raft_consensus.cc:697] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:43.965592 11503 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 2 LEARNER]: Advancing to term 3
W20260709 13:16:44.000185 12970 log.cc:927] Time spent T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 13:16:43.958220 12934 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:44.006695 11478 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:44.012997 12701 consensus_queue.cc:237] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:44.014408 12863 raft_consensus.cc:2749] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:44.015270 12863 consensus_queue.cc:1048] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:16:44.016253 12971 log.cc:927] Time spent T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 13:16:44.014415 12289 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:44.014024 11478 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:43.994031 12622 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.038050 11478 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:16:43.997231 12854 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:44.045351 11478 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:16:44.044869 11509 raft_consensus.cc:1240] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:44.046587 12894 raft_consensus.cc:2804] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:44.047823 12622 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:44.049803 12798 consensus_queue.cc:1059] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 13:16:44.075408 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:44.076459 10949 raft_consensus.cc:2393] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 4: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:44.088904 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:44.096253 10800 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:44.089339 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:44.097651 10796 raft_consensus.cc:2393] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 3: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:44.085322 10944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41020 (request call id 1693) took 2382 ms. Trace:
I20260709 13:16:44.087460 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:44.102324 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:44.102521 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:44.103145 10870 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 5.
I20260709 13:16:44.103984 10876 raft_consensus.cc:2410] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:16:44.107189 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:44.108198 10870 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 3.
I20260709 13:16:44.066006 12671 raft_consensus.cc:2749] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:44.098363 10944 rpcz_store.cc:276] 0709 13:16:41.702882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:41.741349 (+ 38467us) service_pool.cc:224] Handling call
0709 13:16:41.894927 (+153578us) spinlock_profiling.cc:243] Waited 1.94 ms on lock 0x7b5800112598. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd90eee 0000000000375939 00007f9c85e041e0 00007f9c85ecd0c4 00007f9c85ecd131 00007f9c85e7b0a0 00007f9c85e7a984 00007f9c7f3f56d4 00007f9c7f3f5662
0709 13:16:44.085061 (+2190134us) spinlock_profiling.cc:243] Waited 791 ms on lock 0x7b54001e1c54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:44.085242 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1824103808}
I20260709 13:16:44.117707 10916 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:44.068346 12889 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:44.123867 10794 raft_consensus.cc:1240] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:44.125209 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:44.078249 12863 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:44.128186 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:44.051329 12925 consensus_queue.cc:237] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:44.131138 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:44.087741 12622 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:44.134370 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:44.083125 12934 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:44.148458 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:44.128537 10916 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:16:44.144428 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 99178.9us
I20260709 13:16:44.147675 10796 raft_consensus.cc:2410] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:16:44.138218 10794 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 5.
W20260709 13:16:44.181926 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 53453.2us
I20260709 13:16:44.182809 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:44.183168 10916 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:44.128826 12936 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:44.190937 10794 raft_consensus.cc:2393] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 5: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:44.122593 12895 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.179577 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:44.184003 10944 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
I20260709 13:16:44.207722 10794 raft_consensus.cc:1275] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:44.214545 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:44.215517 10796 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 3.
I20260709 13:16:44.147998 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:44.196578 10800 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 5.
I20260709 13:16:44.184801 12853 raft_consensus.cc:2749] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:16:44.186455 12900 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:44.219379 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:44.197395 12894 raft_consensus.cc:697] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 4 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:44.187944 12936 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:44.205791 12907 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:44.252094 10944 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
W20260709 13:16:44.260174 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:44.261428 12974 log.cc:927] Time spent T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
W20260709 13:16:44.275657 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 72030.3us
I20260709 13:16:44.253230 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:44.179315 12889 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:44.251163 12978 log.cc:927] Time spent T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
W20260709 13:16:44.298062 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 116244us
I20260709 13:16:44.209508 12930 consensus_queue.cc:1048] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.051s
I20260709 13:16:44.299516 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:44.259856 12936 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:44.259495 12907 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:44.209134 12895 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:44.302801 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:44.301600 11478 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:44.300539 12761 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.300531 12853 consensus_queue.cc:1048] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
W20260709 13:16:44.315671 12975 log.cc:927] Time spent T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:16:44.315544 12985 consensus_queue.cc:1048] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 13:16:44.319902 12761 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:44.321962 12985 consensus_queue.cc:1048] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:16:44.323136 12979 log.cc:927] Time spent T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 13:16:44.323745 12761 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:44.317430 12889 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:44.300531 12983 raft_consensus.cc:3055] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:44.325892 12983 raft_consensus.cc:740] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:44.326814 12983 consensus_queue.cc:260] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:44.327520 10942 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41014 (request call id 1389) took 1666 ms (client timeout 2108 ms). Trace:
I20260709 13:16:44.328157 12983 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:16:44.327972 10942 rpcz_store.cc:269] 0709 13:16:42.660921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:42.664602 (+  3681us) service_pool.cc:224] Handling call
0709 13:16:44.327306 (+1662704us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b54003d8e54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:44.327455 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2792416512}
I20260709 13:16:44.328415 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:44.329218 10949 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
I20260709 13:16:44.329265 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:44.330003 10942 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 6.
I20260709 13:16:44.331035 10841 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:16:44.332623 12976 log.cc:927] Time spent T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.236s	user 0.001s	sys 0.000s
I20260709 13:16:44.333020 12684 raft_consensus.cc:2804] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:44.333539 12684 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:44.333896 12684 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:44.332165 10841 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:44.307353 12907 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:44.344115  9682 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:16:44.318416 12900 raft_consensus.cc:2749] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:44.336162 12894 consensus_queue.cc:237] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:44.337555 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 1344) took 1216 ms. Trace:
I20260709 13:16:44.338239 12895 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:44.341665 12843 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:44.342716 10797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 173) took 1276 ms. Trace:
I20260709 13:16:44.348219 10869 rpcz_store.cc:276] 0709 13:16:43.120917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:43.121157 (+   240us) service_pool.cc:224] Handling call
0709 13:16:44.337384 (+1216227us) spinlock_profiling.cc:243] Waited 645 ms on lock 0x7b540037ad54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:44.337499 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1483522432}
I20260709 13:16:44.351225 10797 rpcz_store.cc:276] 0709 13:16:43.066690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:43.066836 (+   146us) service_pool.cc:224] Handling call
0709 13:16:43.067125 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:44.341454 (+1274329us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:44.341486 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:44.341533 (+    47us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:44.341569 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:44.342465 (+   896us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b54003cd054. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:44.342509 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:44.342654 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2930203520}
I20260709 13:16:44.346819 12899 raft_consensus.cc:2804] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:44.365870 12893 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:44.361280 12899 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:44.365513 12684 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:44.387351 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:44.373276 12899 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:16:44.394577 12915 log.cc:927] Time spent T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 1.439s	user 0.003s	sys 0.002s
W20260709 13:16:44.397614 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:44.398152 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:44.402712 10945 raft_consensus.cc:1275] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:16:44.404255 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 108511us
I20260709 13:16:44.385222 12992 consensus_queue.cc:1048] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.058s
I20260709 13:16:44.411494 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:44.412421 10947 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 2.
I20260709 13:16:44.412760 10916 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:44.412533 12904 consensus_queue.cc:1048] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 13:16:44.420522 10916 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:44.433471 10869 raft_consensus.cc:1275] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:16:44.442137 10916 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:44.443656 10916 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:44.444247 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:44.445452 10797 raft_consensus.cc:2410] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:16:44.455366 12983 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:44.456341 12983 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:44.459331 12983 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:44.460485 12617 raft_consensus.cc:2804] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:44.461084 12617 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:44.461499 12617 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:44.428818 12930 raft_consensus.cc:2804] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:44.483444 10947 raft_consensus.cc:1240] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 5. Current term is 6. Ops: []
W20260709 13:16:44.494454 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 50890.4us
I20260709 13:16:44.496014 10916 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:44.431540 12766 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:44.505007 12739 raft_consensus.cc:3055] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:44.507747 12904 consensus_queue.cc:1059] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:44.508639 12583 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:44.509155 12684 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:44.506567 10799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1349) took 1686 ms. Trace:
I20260709 13:16:44.528810 10799 rpcz_store.cc:276] 0709 13:16:42.820456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:42.820640 (+   184us) service_pool.cc:224] Handling call
0709 13:16:42.840857 (+ 20217us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:43.208371 (+367514us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:44.506291 (+1297920us) spinlock_profiling.cc:243] Waited 362 ms on lock 0x7b54000e9554. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:44.506494 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":832636800}
I20260709 13:16:44.523361 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:44.530006 10797 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:44.530587 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:44.524335 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:44.525265 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:44.535298 10794 raft_consensus.cc:2393] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 5: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:44.534473 10800 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
I20260709 13:16:44.489512 12930 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:44.527832 10916 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:44.548321 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:44.550194 10796 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:44.551934 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:44.526497 12766 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:44.541038 10797 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 6.
W20260709 13:16:44.555017 11503 consensus_meta.cc:287] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.525s	user 0.009s	sys 0.003s
I20260709 13:16:44.565220 10916 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:44.527248 12739 raft_consensus.cc:740] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:44.513689 12919 consensus_queue.cc:1048] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.057s
I20260709 13:16:44.491405 12936 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:44.554044 12930 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:44.573173 10916 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:44.570295 12739 consensus_queue.cc:260] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:44.513398 12907 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:44.578886 12994 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:44.553342 12889 consensus_queue.cc:1048] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 13:16:44.570618 12992 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:44.601296 12995 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:16:44.621619 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 314150us
I20260709 13:16:44.626529 12672 consensus_queue.cc:1059] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:44.627622 12672 consensus_queue.cc:1243] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer 468a8fd46e824f62b9c2c0d3532a6d26 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:16:44.628266 12989 raft_consensus.cc:3055] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:44.628710 12989 raft_consensus.cc:740] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:44.629129 12900 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:44.629696 12900 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:44.632887 10949 raft_consensus.cc:1275] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:16:44.633774 10949 pending_rounds.cc:85] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 1
I20260709 13:16:44.619592 12739 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:44.634011 10949 pending_rounds.cc:107] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:16:44.634348 10949 raft_consensus.cc:2889] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:44.633018 12919 consensus_queue.cc:1059] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
W20260709 13:16:44.632429 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:44.643980 12893 raft_consensus.cc:2749] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:44.644344 12919 consensus_queue.cc:1048] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:44.644942 12893 raft_consensus.cc:3055] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:44.645359 12893 raft_consensus.cc:740] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:44.645874 12783 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:44.615290 12994 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:44.574522 12761 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:44.637024 12784 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.660627 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:44.661564 11512 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
I20260709 13:16:44.583091 11503 raft_consensus.cc:1275] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 3 LEARNER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 3. (term mismatch)
I20260709 13:16:44.653892 12994 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:44.659366 13000 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:44.678702 11503 pending_rounds.cc:85] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586: Aborting all ops after (but not including) 2
I20260709 13:16:44.599795 12583 raft_consensus.cc:2955] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 LEADER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:44.701838 12936 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:44.646220 12893 consensus_queue.cc:260] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:44.695641 12872 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:44.687788 12889 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:44.629709 12989 consensus_queue.cc:260] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:44.672904 12784 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:44.668433 12761 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:44.707515 11503 pending_rounds.cc:107] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586: Aborting uncommitted NO_OP operation due to leader change: 2.3
I20260709 13:16:44.699859 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:16:44.719481 12863 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.515s	user 0.013s	sys 0.016s
I20260709 13:16:44.712888 13000 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.755718 12900 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:44.791456 13000 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:44.793733 13000 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:44.757040 12893 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:44.770659 11503 raft_consensus.cc:2889] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 3 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:44.765084 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:16:44.720357 11475 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.003s	sys 0.024s
I20260709 13:16:44.796778 12756 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:16:44.820231 12927 log.cc:927] Time spent T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 1.442s	user 0.005s	sys 0.000s
I20260709 13:16:44.724139 12872 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:44.822003 12766 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:44.744251 12889 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:44.823622 12617 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:44.824270 12761 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:44.777000 12863 raft_consensus.cc:697] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 6 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:44.764586 12925 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.798226 12989 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:44.805593 12907 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:44.757452 12899 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:44.859284 11478 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.116.130:37895 timed out after 1.593s (SENT) [suppressed 24 similar messages]
W20260709 13:16:44.829720 11475 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.003s	sys 0.025s
I20260709 13:16:44.836524 12756 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:44.896441 12756 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:44.896896 10943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 208) took 1046 ms. Trace:
W20260709 13:16:44.898226 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:44.899180 12852 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:44.897434 10943 rpcz_store.cc:276] 0709 13:16:43.850020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:43.856973 (+  6953us) service_pool.cc:224] Handling call
0709 13:16:43.888933 (+ 31960us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:43.942871 (+ 53938us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:16:43.995828 (+ 52957us) log.cc:844] Serialized 14 byte log entry
0709 13:16:44.055753 (+ 59925us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:44.061070 (+  5317us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:44.063650 (+  2580us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:44.895553 (+831903us) spinlock_profiling.cc:243] Waited 4.32 ms on lock 0x7b18002fe9c0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7dd962a6 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419
0709 13:16:44.896600 (+  1047us) spinlock_profiling.cc:243] Waited 22.7 ms on lock 0x7b54000177d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:44.896660 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:44.896820 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5793,"prepare.queue_time_us":17592,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":1537,"raft.queue_time_us":123,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":7039,"spinlock_wait_cycles":62041600,"thread_start_us":88628,"threads_started":2,"wal-append.queue_time_us":132856}
I20260709 13:16:44.901383 10801 raft_consensus.cc:1275] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:16:44.904098 12766 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:44.905462 10841 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:44.905860 10801 raft_consensus.cc:1275] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:16:44.908239 10801 raft_consensus.cc:1275] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:16:44.909035 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:44.909842 10945 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 6.
I20260709 13:16:44.920827 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:44.933418 10947 raft_consensus.cc:1275] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:16:44.933568 10950 raft_consensus.cc:1275] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:16:44.937690 10944 raft_consensus.cc:2393] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 5: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
W20260709 13:16:44.914685 12775 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.012s	sys 0.030s
I20260709 13:16:44.940840 12992 raft_consensus.cc:697] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 6 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:44.918977 12672 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:44.953997 12992 consensus_queue.cc:237] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:44.954331 12672 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:44.876350 12925 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:44.914067 12904 raft_consensus.cc:2749] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:16:44.935271 12900 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.922185 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:45.040310 11503 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Advancing to term 5
W20260709 13:16:44.948148 12775 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.012s	sys 0.030s
I20260709 13:16:45.045784 12925 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:44.941216 12508 consensus_queue.cc:1048] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.107s
W20260709 13:16:45.036871 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:44.939890 12766 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:16:45.120411 10842 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.007s	sys 0.009s
W20260709 13:16:45.120913 10842 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.007s	sys 0.009s
I20260709 13:16:44.950984 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:45.120690 12778 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.127497 12778 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:45.127921 12925 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:45.132130 10916 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.015s	sys 0.004s
W20260709 13:16:45.133061 10916 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.016s	sys 0.004s
W20260709 13:16:45.134016 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 553178us
I20260709 13:16:45.136920 10916 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:16:45.134059 12930 consensus_meta.cc:287] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.529s	user 0.008s	sys 0.004s
I20260709 13:16:45.153221 12778 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:45.153646 12775 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.154764 12672 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:45.155229 12997 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:45.156628 12872 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:45.159089 12907 consensus_queue.cc:1059] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:45.159621 13012 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.161496 12852 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:45.139289 12894 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:45.129475 13002 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:45.142943 12925 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:45.165768 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:45.197538 10801 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 6.
I20260709 13:16:45.201334 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:45.202049 10801 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:45.187894 12895 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.187883 12904 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
W20260709 13:16:45.221534 12921 log.cc:927] Time spent T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 2.064s	user 0.003s	sys 0.006s
I20260709 13:16:45.223258 10795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1342) took 2554 ms. Trace:
W20260709 13:16:45.223495 12926 log.cc:927] Time spent T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 1.957s	user 0.006s	sys 0.000s
I20260709 13:16:45.223709 10795 rpcz_store.cc:276] 0709 13:16:42.668676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:42.668837 (+   161us) service_pool.cc:224] Handling call
0709 13:16:42.669181 (+   344us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:42.669597 (+   416us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:42.669626 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:42.670305 (+   679us) log.cc:844] Serialized 52 byte log entry
0709 13:16:42.984698 (+314393us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:16:42.989309 (+  4611us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:43.000076 (+ 10767us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:45.220802 (+2220726us) spinlock_profiling.cc:243] Waited 28.8 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:16:45.222852 (+  2050us) raft_consensus.cc:1685] finished
0709 13:16:45.222875 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:45.223171 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14718,"raft.queue_time_us":8877,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":7125,"spinlock_wait_cycles":66274560,"thread_start_us":294591,"threads_started":1,"wal-append.queue_time_us":409491}
I20260709 13:16:45.229743 10801 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 4.
I20260709 13:16:45.182744 12775 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:45.107666 12900 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:44.950709 12863 consensus_queue.cc:237] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:45.184154 12778 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.141263 12766 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:45.209692 12583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:39430 (request call id 11) took 12548 ms. Trace:
I20260709 13:16:45.192519 12872 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:45.175587 12930 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:45.183569 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:45.199926 12894 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:45.248384 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:45.186453 10842 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:45.248303 12858 raft_consensus.cc:3055] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:45.203245 12889 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:45.248517 13016 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:45.281544 10792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 1753) took 2470 ms. Trace:
I20260709 13:16:45.312135 10792 rpcz_store.cc:276] 0709 13:16:42.810743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:42.810932 (+   189us) service_pool.cc:224] Handling call
0709 13:16:42.811401 (+   469us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:42.864952 (+ 53551us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:42.864997 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:42.875432 (+ 10435us) log.cc:844] Serialized 52 byte log entry
0709 13:16:43.115416 (+239984us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:43.115492 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:43.130829 (+ 15337us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:45.222978 (+2092149us) spinlock_profiling.cc:243] Waited 3.53 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:16:45.225694 (+  2716us) raft_consensus.cc:1685] finished
0709 13:16:45.225713 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:45.281439 (+ 55726us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6575,"raft.queue_time_us":44337,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":12112,"spinlock_wait_cycles":8129920,"thread_start_us":222034,"threads_started":2,"wal-append.queue_time_us":284979}
I20260709 13:16:45.195495 13012 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:45.315716 13012 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:45.225883 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:45.316546 10795 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 2.
I20260709 13:16:45.234838 12899 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:16:45.291484 12739 consensus_meta.cc:287] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.648s	user 0.013s	sys 0.000s
I20260709 13:16:45.199046 13002 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:45.296648 13015 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:45.233620 12904 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:45.303377 12925 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:45.304989 10946 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:45.232740 12993 raft_consensus.cc:2749] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:16:45.333499 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:45.334085 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:45.305801 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 119485us
I20260709 13:16:45.335412 10842 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:45.308393 12858 raft_consensus.cc:740] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:45.285035 12778 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:45.191111 12997 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:45.191533 12914 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.032s	user 0.018s	sys 0.048s
I20260709 13:16:45.246513 12617 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:45.214360 12895 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:45.311837 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 452598us
W20260709 13:16:45.320571 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 192431us
I20260709 13:16:45.296797 13014 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:16:45.298105 12995 consensus_meta.cc:287] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.696s	user 0.007s	sys 0.005s
I20260709 13:16:45.309214 12968 raft_consensus.cc:2749] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:16:45.307978 12872 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:45.203449 12852 raft_consensus.cc:697] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 5 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:16:45.306770 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 390690us
I20260709 13:16:45.346364 12899 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.374616 11477 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:45.341827 12925 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:16:45.342641 13013 log.cc:927] Time spent T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:16:45.371181 12995 raft_consensus.cc:2749] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:45.283704 12583 rpcz_store.cc:276] 0709 13:16:32.668235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:32.697653 (+ 29418us) service_pool.cc:224] Handling call
0709 13:16:33.882955 (+1185302us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b58003326d0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80e110aa 00007f9c80e93447 00007f9c80eb070a 00007f9c80eb176a 00007f9c80eb091b 00007f9c80ebc03c 00007f9c80ebb6b3 00007f9c85e7bd29 00007f9c7f3fa1d4 00007f9c7f3fa162
0709 13:16:33.995801 (+112846us) log.cc:844] Serialized 506 byte log entry
0709 13:16:34.408614 (+412813us) spinlock_profiling.cc:243] Waited 1.34 ms on lock 0x7b18003584e0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc83f3 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:16:36.471853 (+2063239us) spinlock_profiling.cc:243] Waited 43.8 ms on lock 0x7b5400306ea0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80e2e062 00007f9c80e0eeaa 00007f9c80e15d53 00007f9c80e15cd9 00007f9c80e15c69 00007f9c80e15c31 00007f9c80e14dfd 000000000038f354 000000000038f199 00007f9c7ddc8475
0709 13:16:36.764487 (+292634us) spinlock_profiling.cc:243] Waited 1.57 ms on lock 0x7b18003584e0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7dd42c0a 00007f9c87482219 00007f9c7ddcaa40 00007f9c7ddc54e4 00007f9c7ddc5455 00007f9c80e0eb0b 00007f9c80e93b63 00007f9c80ebb752 00007f9c85e7bd29
0709 13:16:45.209502 (+8445015us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3112257,"raft.queue_time_us":2700917,"raft.run_cpu_time_us":29404,"raft.run_wall_time_us":2068617,"spinlock_wait_cycles":469511808,"thread_start_us":65280,"threads_started":1,"wal-append.queue_time_us":124881,"wal-append.run_cpu_time_us":1866,"wal-append.run_wall_time_us":1001888}
I20260709 13:16:45.377393 12968 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.377962 12968 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:45.378652 12983 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:45.379173 12983 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:45.379665 12983 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:45.380491 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:45.380734 13021 consensus_queue.cc:1048] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:45.381171 10874 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:45.337951 10842 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:16:45.383864 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 46009.4us
W20260709 13:16:45.350720 12914 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.208s	user 0.018s	sys 0.049s
I20260709 13:16:45.368839 10946 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 4.
W20260709 13:16:45.386364 10946 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36714 (request call id 206) took 2121 ms (client timeout 2246 ms). Trace:
I20260709 13:16:45.387009 10943 raft_consensus.cc:1240] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:45.387101 12907 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.387852 12907 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:45.388455 12983 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:45.386729 10946 rpcz_store.cc:269] 0709 13:16:43.265434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:43.267815 (+  2381us) service_pool.cc:224] Handling call
0709 13:16:45.239386 (+1971571us) spinlock_profiling.cc:243] Waited 1.04 ms on lock 0x7b5800112598. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd90a22 00007f9c86ca9c09 00007f9c85e03ee0 00007f9c85ecd0ab 00007f9c85ecd131 00007f9c85e7b0a0 00007f9c85e7a984 00007f9c7f3f56d4 00007f9c7f3f5662
0709 13:16:45.386305 (+146919us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2388352}
I20260709 13:16:45.390305 10941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 1407) took 1354 ms. Trace:
I20260709 13:16:45.391124 12684 raft_consensus.cc:3055] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:45.391600 12684 raft_consensus.cc:740] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:16:45.392700 10948 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36714 (request call id 205) took 2127 ms (client timeout 1837 ms). Trace:
I20260709 13:16:45.392534 12684 consensus_queue.cc:260] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:45.393280 10948 rpcz_store.cc:269] 0709 13:16:43.265104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:43.267721 (+  2617us) service_pool.cc:224] Handling call
0709 13:16:45.392446 (+2124725us) spinlock_profiling.cc:243] Waited 416 ms on lock 0x7b54001707d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:45.392599 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":957014272}
I20260709 13:16:45.393904 12684 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:45.394902 12925 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:45.395318 12925 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:45.390674 10941 rpcz_store.cc:276] 0709 13:16:44.036400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:44.296859 (+260459us) service_pool.cc:224] Handling call
0709 13:16:44.297212 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:44.824776 (+527564us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:44.824810 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:44.825326 (+   516us) log.cc:844] Serialized 26 byte log entry
0709 13:16:45.228033 (+402707us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:45.228089 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:45.229499 (+  1410us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:45.390080 (+160581us) raft_consensus.cc:1685] finished
0709 13:16:45.390101 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:45.390237 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":37252,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":81,"thread_start_us":810092,"threads_started":2,"wal-append.queue_time_us":557925}
I20260709 13:16:45.398423 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:45.399587 12852 consensus_queue.cc:237] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:45.401894 12836 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:45.403694 12989 consensus_meta.cc:287] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.529s	user 0.011s	sys 0.000s
W20260709 13:16:45.407716 12994 consensus_meta.cc:287] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.701s	user 0.012s	sys 0.000s
W20260709 13:16:45.408390 13009 log.cc:927] Time spent T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 13:16:45.408838 12920 log.cc:927] Time spent T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 2.255s	user 0.000s	sys 0.008s
W20260709 13:16:45.407829 13011 log.cc:927] Time spent T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.224s	user 0.000s	sys 0.001s
I20260709 13:16:45.410705 13002 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:45.411131 11508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56404 (request call id 427) took 2584 ms. Trace:
I20260709 13:16:45.367950 13014 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:45.413066 11505 raft_consensus.cc:2410] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:16:45.416829 12783 consensus_meta.cc:287] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.763s	user 0.008s	sys 0.002s
I20260709 13:16:45.417344 12949 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.417881 12949 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:45.418563 11505 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56410 (request call id 407) took 1853 ms (client timeout 1619 ms). Trace:
I20260709 13:16:45.423663 12766 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:45.424293 12766 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.424646 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:45.408531 12994 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:45.427951 12994 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:45.428997 11503 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 5.
I20260709 13:16:45.429656 12836 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:45.430128 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:45.430341 12836 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:45.346642 13012 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:45.432578 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:45.450500 10945 raft_consensus.cc:2393] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 5: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:45.321440 13016 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:45.440618 10949 raft_consensus.cc:1275] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:45.429122 12583 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.443960 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:45.413250 12843 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:45.413712 13014 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:16:45.433688 12893 consensus_meta.cc:287] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.608s	user 0.011s	sys 0.000s
I20260709 13:16:45.368613 12739 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:45.429425 10766 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:45.451469 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:45.451469 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:16:45.419183 11505 rpcz_store.cc:269] 0709 13:16:43.565496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:43.582745 (+ 17249us) service_pool.cc:224] Handling call
0709 13:16:45.398186 (+1815441us) spinlock_profiling.cc:243] Waited 20.4 ms on lock 0x7b5800003c98. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd90a22 00007f9c86ca9c09 00007f9c85e03ee0 00007f9c85ecd0ab 00007f9c85ecd131 00007f9c85e7b0a0 00007f9c85e7a984 00007f9c7f3f56d4 00007f9c7f3f5662
0709 13:16:45.418272 (+ 20086us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b54004e5554. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:45.418467 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":71651328}
I20260709 13:16:45.416280 12968 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:45.461046 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:45.467581 10942 raft_consensus.cc:1275] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:16:45.414562 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:45.469630 10941 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:45.469743 10945 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 4.
I20260709 13:16:45.480182 11511 raft_consensus.cc:2410] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:16:45.480680 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:45.481418 10945 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 6.
I20260709 13:16:45.330550 13015 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:45.471371 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:45.471495 10766 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:45.483304 10947 raft_consensus.cc:2393] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 6: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:45.483655 10766 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:45.482574 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:45.487408 10945 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:45.482602 10797 raft_consensus.cc:2393] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 6: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:45.491379 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:45.491811 12894 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:45.492358 12997 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:16:45.479509 12903 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.336s	user 0.021s	sys 0.037s
W20260709 13:16:45.493115 12903 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.375s	user 0.021s	sys 0.037s
I20260709 13:16:45.493239 12904 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:45.493391 12903 raft_consensus.cc:697] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:45.493928 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:45.494910 11511 raft_consensus.cc:2393] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 5: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:45.495565 10792 raft_consensus.cc:1275] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:16:45.411530 11508 rpcz_store.cc:276] 0709 13:16:42.826488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:42.844363 (+ 17875us) service_pool.cc:224] Handling call
0709 13:16:42.844859 (+   496us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:42.849238 (+  4379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:42.849288 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:42.866909 (+ 17621us) log.cc:844] Serialized 26 byte log entry
0709 13:16:42.968492 (+101583us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:42.968587 (+    95us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:42.968659 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:45.408277 (+2439618us) spinlock_profiling.cc:243] Waited 31.1 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:16:45.410823 (+  2546us) raft_consensus.cc:1685] finished
0709 13:16:45.410845 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:45.411056 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1767,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":7583,"spinlock_wait_cycles":71539456,"thread_start_us":64613,"threads_started":1,"wal-append.queue_time_us":198531}
I20260709 13:16:45.496867 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:45.486151 10766 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:45.424731 12766 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:45.514452 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:45.458433 13002 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.465230 12914 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.464980 12889 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:45.519618 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:45.504707 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:45.527861 12766 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:45.477427 12739 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:45.506127 12756 raft_consensus.cc:2749] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:16:45.532629 12756 raft_consensus.cc:2804] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:45.533174 12756 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:45.533648 12756 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:45.534219 10945 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7.
I20260709 13:16:45.539512 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:45.530710 12739 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:45.543953 10941 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
I20260709 13:16:45.453418 12899 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:45.474769 13017 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.474715 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:45.494426 12930 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:45.456296 13012 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:45.468083 12583 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:45.504513 12853 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.474588 12838 raft_consensus.cc:2764] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:16:45.495066 12968 raft_consensus.cc:2749] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:45.455514 12858 consensus_queue.cc:260] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:45.473364 12865 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:45.547910 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:45.592836 11506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56410 (request call id 412) took 1746 ms. Trace:
I20260709 13:16:45.507184 12934 raft_consensus.cc:2804] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:45.517885 13024 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:45.574514 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:45.606228 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:16:45.649206 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:45.545436 12894 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:45.650173 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:45.606648 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:45.651116 10870 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:45.596722 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:45.593926 10943 raft_consensus.cc:2410] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:16:45.629067 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:16:45.631748 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 247156us
I20260709 13:16:45.659940 10842 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:45.577111 12775 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:45.631774 10947 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
I20260709 13:16:45.494453 12903 consensus_queue.cc:237] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:45.631338 11509 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 4.
I20260709 13:16:45.642771 12778 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:45.642977 12895 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:45.644101 13016 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:45.644989 12583 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:45.645269 12836 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:16:45.647044 13027 log.cc:927] Time spent T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:16:45.647329 13012 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:45.647819 13024 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:45.575304 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:45.700287 10948 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
I20260709 13:16:45.655650 10871 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 5.
I20260709 13:16:45.702529 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:45.595875 12766 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:45.493985 12904 consensus_queue.cc:1048] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.141s
I20260709 13:16:45.548147 12914 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:45.666934 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:45.669907 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:45.689306 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1367) took 1228 ms. Trace:
I20260709 13:16:45.672890 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:45.550407 13002 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:45.649343 12858 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:45.636063 12934 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:45.649071 12900 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:45.591854 13017 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:45.691111 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:45.626089 12853 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:45.746771 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:45.747299 10941 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
I20260709 13:16:45.747433 10947 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
W20260709 13:16:45.692297 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 185071us
I20260709 13:16:45.676232 12756 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:45.626437 12889 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:45.658340 10874 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 5.
W20260709 13:16:45.754935 10874 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 1360) took 1654 ms (client timeout 2116 ms). Trace:
W20260709 13:16:45.755422 10874 rpcz_store.cc:269] 0709 13:16:44.100865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:44.115581 (+ 14716us) service_pool.cc:224] Handling call
0709 13:16:45.380316 (+1264735us) spinlock_profiling.cc:243] Waited 4.23 ms on lock 0x7b5800177098. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd90a22 00007f9c86ca9c09 00007f9c85e03ee0 00007f9c85ecd0ab 00007f9c85ecd131 00007f9c85e7b0a0 00007f9c85e7a984 00007f9c7f3f56d4 00007f9c7f3f5662
0709 13:16:45.754863 (+374547us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9723136}
I20260709 13:16:45.669987 12872 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:45.759610 12914 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:45.759869 12961 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:45.703343 10871 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 6.
I20260709 13:16:45.635605 12865 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:45.708408 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:16:45.672518 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 311088us
W20260709 13:16:45.714036 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:45.714432 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:45.711548 12766 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:45.735128 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:45.772969 10875 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:45.658761 12894 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:45.739591 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:45.740883 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:45.777750 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:45.778223 12894 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:45.745788 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:45.778613 10792 raft_consensus.cc:2410] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:16:45.778954 10869 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:45.779660 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:45.780781 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:45.781761 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:45.691831 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:45.698866 13028 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:45.783231 10797 raft_consensus.cc:2393] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 5: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:45.783150 13028 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:45.783927 13028 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:45.783512 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:45.784901 10792 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
W20260709 13:16:45.695356 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 209291us
I20260709 13:16:45.786692 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:45.787634 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:45.788777 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:45.751830 10916 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:45.758724 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:45.765266 10801 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 6.
I20260709 13:16:45.729481 10870 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 5.
I20260709 13:16:45.743940 12934 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:45.790302 12783 raft_consensus.cc:2804] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:45.792742 10766 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:45.789537 10792 raft_consensus.cc:2393] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 5: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:45.802805 12783 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:45.803761 10801 raft_consensus.cc:1275] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:16:45.771795 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:45.790607 10874 raft_consensus.cc:2410] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:16:45.804275 12583 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:45.805399 12783 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:45.806946 12583 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:45.807027 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:45.807559 12583 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:45.807992 10797 raft_consensus.cc:2393] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 5: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:45.805693 10871 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 3.
I20260709 13:16:45.809725 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:45.809860 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:45.790174 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:45.810345 10871 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 3.
I20260709 13:16:45.812637 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:45.813418 10948 raft_consensus.cc:2393] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 5: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:45.790467 10916 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:45.814409 12930 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:45.817699 12930 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:45.820603 12930 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:45.814268 10868 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
I20260709 13:16:45.812471 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:45.830811 10948 raft_consensus.cc:2955] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:45.828339 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:45.838863 10945 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 5.
I20260709 13:16:45.829092 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:45.840425 10947 raft_consensus.cc:2410] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:16:45.829861 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:45.841461 10941 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
I20260709 13:16:45.829388 12783 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:45.831763 12836 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:45.832468 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:45.822314 10916 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:45.855082 12994 raft_consensus.cc:2804] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:45.855693 12994 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:45.856137 12994 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:45.831041 10874 raft_consensus.cc:2393] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 4: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:45.833424 12930 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.866616 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:45.830687 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:45.829738 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:45.826572 13028 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:45.876109 10916 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:45.878657 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:45.880729 10766 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:45.872812 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:45.882328 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:45.890615 10766 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:45.894450 12838 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:45.895031 12838 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:45.895411 12838 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:45.896200 13015 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:45.876613 10875 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 6.
I20260709 13:16:45.764803 12983 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:45.912835 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:45.915410 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:45.917884 10868 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:16:45.884565 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:45.888355 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:45.890136 10869 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 4.
I20260709 13:16:45.935035 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:45.934468 10870 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 5.
W20260709 13:16:45.918589 13030 log.cc:927] Time spent T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.186s	user 0.000s	sys 0.001s
I20260709 13:16:45.945204 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:45.948683 12889 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:45.952901 10871 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 4.
I20260709 13:16:45.916700 12775 raft_consensus.cc:2804] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:45.916355 12996 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:45.955121 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:45.917459 12739 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:45.959882 13024 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:45.934850 13035 log.cc:927] Time spent T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:16:45.878041 12930 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:45.972507 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:45.885020 12836 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:45.928013 12899 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:45.935503 13049 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:45.962610 12912 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.982856 10916 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:45.936283 12990 raft_consensus.cc:3055] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:45.964617 12775 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:46.008527 10799 raft_consensus.cc:1240] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:16:46.008410 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:45.966675 12996 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:45.981243 12778 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:46.044725 10945 raft_consensus.cc:1275] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:16:46.045267 10945 pending_rounds.cc:85] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 3
I20260709 13:16:46.045485 10945 pending_rounds.cc:107] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:16:46.045725 10945 raft_consensus.cc:2889] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:45.984010 12739 raft_consensus.cc:697] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 5 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:46.047394 10767 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:46.055081 10799 raft_consensus.cc:1275] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:16:46.014441 12775 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:46.062355 10842 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:46.011631 12990 raft_consensus.cc:740] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:46.014441 13039 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.015642 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:46.067175 10947 raft_consensus.cc:1275] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:16:46.072191 10942 raft_consensus.cc:1275] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:16:46.075631 10767 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:16:46.075796 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 93057.3us
I20260709 13:16:46.025225 12996 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:46.082902 10916 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:46.039649 12671 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.058873 13015 raft_consensus.cc:2804] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:46.093997 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:46.095497 10916 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:46.099056 10801 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 4.
I20260709 13:16:46.094962 12865 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:46.079762 12889 raft_consensus.cc:2749] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:46.076674 13051 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.016033 12912 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:46.016588 13049 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:46.048664 12574 consensus_queue.cc:260] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:46.109649 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:16:46.116386 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 54127.8us
I20260709 13:16:46.059310 12778 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:46.095702 13014 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.101238 13002 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:16:46.131069 13038 log.cc:927] Time spent T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.219s	user 0.001s	sys 0.000s
I20260709 13:16:45.633878 11506 rpcz_store.cc:276] 0709 13:16:43.623746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:43.625291 (+  1545us) service_pool.cc:224] Handling call
0709 13:16:43.698211 (+ 72920us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:43.713689 (+ 15478us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:43.713732 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:43.732179 (+ 18447us) log.cc:844] Serialized 26 byte log entry
0709 13:16:43.761988 (+ 29809us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:43.767944 (+  5956us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:45.342620 (+1574676us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b54005f0354. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:45.342682 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:45.342735 (+    53us) raft_consensus.cc:1685] finished
0709 13:16:45.342747 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:45.365720 (+ 22973us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1553,"raft.queue_time_us":137,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":118,"spinlock_wait_cycles":34474752,"thread_start_us":19747,"threads_started":1,"wal-append.queue_time_us":89712,"wal-append.run_cpu_time_us":3580,"wal-append.run_wall_time_us":1277515}
I20260709 13:16:46.150416 13028 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:46.151077 13055 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:46.151562 13055 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:46.151636 12778 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:46.152005 13055 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:46.107357 12994 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:46.153857 13014 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:46.155308 12994 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:46.155850 13014 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:16:46.135509 13042 log.cc:927] Time spent T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 13:16:46.156587 12930 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:46.134913 12739 consensus_queue.cc:237] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:46.127110 12768 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:46.136911 12836 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:46.130370 13017 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:46.140812 10842 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:46.105942 13032 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.106069 13029 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:46.079242 13039 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:46.124478 13043 log.cc:927] Time spent T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
I20260709 13:16:46.143790 10795 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 4.
I20260709 13:16:45.735891 10867 rpcz_store.cc:276] 0709 13:16:44.447621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:44.478373 (+ 30752us) service_pool.cc:224] Handling call
0709 13:16:44.486561 (+  8188us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:16:44.487172 (+   611us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:44.487222 (+    50us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:16:44.488205 (+   983us) log.cc:844] Serialized 78 byte log entry
0709 13:16:44.871435 (+383230us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:16:45.338423 (+466988us) log.cc:844] Serialized 14 byte log entry
0709 13:16:45.669057 (+330634us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:45.669193 (+   136us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:45.669267 (+    74us) raft_consensus.cc:1685] finished
0709 13:16:45.669285 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:45.675651 (+  6366us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18768,"prepare.queue_time_us":495147,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":40,"raft.queue_time_us":144101,"raft.run_cpu_time_us":354,"raft.run_wall_time_us":40396,"thread_start_us":1097249,"threads_started":3,"wal-append.queue_time_us":660664}
I20260709 13:16:46.127254 13015 raft_consensus.cc:697] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 4 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:46.144642 12574 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:46.145025 12907 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:46.167388 12768 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:46.146324 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:46.211146 10942 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 4.
I20260709 13:16:46.146993 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:46.212033 10947 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 5.
W20260709 13:16:46.148782 11509 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56416 (request call id 447) took 1472 ms (client timeout 1657 ms). Trace:
I20260709 13:16:46.148627 12838 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:46.153424 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:16:46.131597 13047 log.cc:927] Time spent T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 13:16:46.212806 10842 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:16:46.137902 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:46.120921 12853 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:46.169509 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:46.124318 12895 raft_consensus.cc:2804] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:46.102130 12671 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:46.131096 12766 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:46.252758 10801 raft_consensus.cc:1240] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:46.156730 12949 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:46.255292 12912 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:46.256006 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:46.199225 10766 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:46.203194 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:16:46.145684 13018 log.cc:927] Time spent T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.776s	user 0.002s	sys 0.000s
I20260709 13:16:46.261246 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:46.262573 11507 raft_consensus.cc:1240] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:46.264087 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:46.264830 11507 raft_consensus.cc:2376] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 5. Re-sending same reply.
I20260709 13:16:46.265985 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:46.213971 10868 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 5.
I20260709 13:16:46.205374 12836 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:46.213783 11509 rpcz_store.cc:269] 0709 13:16:44.676302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:44.877424 (+201122us) service_pool.cc:224] Handling call
0709 13:16:46.148579 (+1271155us) spinlock_profiling.cc:243] Waited 165 ms on lock 0x7b54004290d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:46.148710 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":380594048}
I20260709 13:16:46.216610 10944 raft_consensus.cc:1240] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:16:46.171082 12934 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:46.219705 10946 raft_consensus.cc:1240] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:16:46.261889 11508 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
I20260709 13:16:46.219586 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:46.185365 13028 raft_consensus.cc:2804] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:46.276489 13028 raft_consensus.cc:697] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:46.276896 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:46.277169 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:46.277433 13028 consensus_queue.cc:237] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:46.223225 10842 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:46.279534 10944 raft_consensus.cc:1275] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:16:46.280062 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 56897.8us
I20260709 13:16:46.224061 10794 raft_consensus.cc:1240] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:16:46.283731 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:46.284191 12899 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:46.284662 12899 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:46.284742 13046 consensus_queue.cc:1048] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:46.225374 10915 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:16:46.286279 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 60996.6us
I20260709 13:16:46.227541 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:46.286352 13046 consensus_queue.cc:1048] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:46.205168 13037 consensus_queue.cc:1048] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 13:16:46.127763 12990 consensus_queue.cc:260] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:46.207700 13029 raft_consensus.cc:697] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:46.288275 12990 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:46.288810 13029 consensus_queue.cc:237] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:46.289882 13010 log.cc:927] Time spent T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 1.034s	user 0.002s	sys 0.000s
W20260709 13:16:46.290189 12858 consensus_meta.cc:287] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.542s	user 0.009s	sys 0.001s
I20260709 13:16:46.236100 10797 raft_consensus.cc:2393] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 6: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:46.215126 12768 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:46.291322 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:46.292651 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:46.293686 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:46.294247 10801 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 4.
I20260709 13:16:46.207357 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:46.295063 10874 raft_consensus.cc:2376] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 6. Re-sending same reply.
I20260709 13:16:46.295979 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:46.296752 10874 raft_consensus.cc:2393] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 6: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:46.297562 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:46.298123 10874 raft_consensus.cc:2376] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 5. Re-sending same reply.
I20260709 13:16:46.223904 13052 consensus_queue.cc:1048] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 13:16:46.238878 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:46.241554 10798 raft_consensus.cc:1275] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:16:46.300127 10793 raft_consensus.cc:2368] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for earlier term 4. Current term is 5.
I20260709 13:16:46.300230 10767 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:46.239207 13049 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:46.243064 13015 consensus_queue.cc:237] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:46.303579 11512 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 6.
I20260709 13:16:46.188557 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:46.197698 13032 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:46.304332 11511 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 6.
I20260709 13:16:46.253350 12784 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:46.256704 10867 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 2.
I20260709 13:16:46.197273 12964 consensus_queue.cc:1048] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
W20260709 13:16:46.257666 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 58566.7us
I20260709 13:16:46.259223 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:46.268150 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:46.205968 12930 consensus_queue.cc:1048] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 13:16:46.326159 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:46.327389 10941 raft_consensus.cc:2410] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:16:46.272647 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:46.273559 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:46.273948 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:46.274159 12775 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:46.231226 12683 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:46.270314 10947 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 4.
I20260709 13:16:46.274940 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:46.275862 12671 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:46.284063 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:46.285043 12899 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:46.284931 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:46.306173 12903 raft_consensus.cc:2749] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:46.128139 13051 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:46.242652 13048 log.cc:927] Time spent T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:16:46.244236 10811 tablet_copy_service.cc:140] P e01ecd008edd4aa2b6b2b47efbc2ea87: Received BeginTabletCopySession request for tablet 171abdebfc4142cda86de477a5c89a7e from peer 12682eb70bc84c76879e04a8628ee586 ({username='slave'} at 127.0.0.1:39442)
I20260709 13:16:46.233070 12895 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:16:46.349516 12961 consensus_meta.cc:287] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.589s	user 0.008s	sys 0.003s
I20260709 13:16:46.226334 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:46.305838 10767 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:46.278122 12836 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:46.315487 12914 raft_consensus.cc:2804] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:46.317580 11504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56410 (request call id 408) took 2743 ms. Trace:
I20260709 13:16:46.318123 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:46.405469 11511 raft_consensus.cc:2376] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 4. Re-sending same reply.
I20260709 13:16:46.318207 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:46.406730 10793 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 2.
I20260709 13:16:46.318406 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:46.332643 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:46.333877 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:46.336447 11505 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 4.
W20260709 13:16:46.339032 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 226102us
I20260709 13:16:46.341389 10766 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:46.287067 10915 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:16:46.426020 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 139024us
I20260709 13:16:46.345369 10799 raft_consensus.cc:1240] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:16:46.337852 12683 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:46.306923 12949 raft_consensus.cc:2749] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:16:46.331635 11478 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:46.355512 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:16:46.430238 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 98686us
W20260709 13:16:46.342943 12925 consensus_meta.cc:287] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.947s	user 0.009s	sys 0.003s
W20260709 13:16:46.343315 12583 consensus_meta.cc:287] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.535s	user 0.011s	sys 0.000s
I20260709 13:16:46.359906 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:46.437244 10915 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:46.437781 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:46.244933 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:46.355669 12775 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:46.393366 12895 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:46.348846 13017 consensus_queue.cc:1059] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 13:16:46.408685 11503 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 5.
I20260709 13:16:46.436403 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:16:46.437889 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 253635us
I20260709 13:16:46.403354 10947 raft_consensus.cc:1275] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:16:46.409432 11506 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 2 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 2.
I20260709 13:16:46.447659 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:46.457465 10943 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 3.
I20260709 13:16:46.434193 10916 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:46.400341 10811 tablet_copy_service.cc:161] P e01ecd008edd4aa2b6b2b47efbc2ea87: Beginning new tablet copy session on tablet 171abdebfc4142cda86de477a5c89a7e from peer 12682eb70bc84c76879e04a8628ee586 at {username='slave'} at 127.0.0.1:39442: session id = 12682eb70bc84c76879e04a8628ee586-171abdebfc4142cda86de477a5c89a7e
I20260709 13:16:46.426254 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:46.443614 10941 raft_consensus.cc:1240] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:46.432753 12683 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:46.436542 12889 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:46.428287 10867 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 4.
I20260709 13:16:46.434209 13049 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:46.436800 12983 raft_consensus.cc:2804] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:46.439299 12912 raft_consensus.cc:2749] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:46.462502 10943 raft_consensus.cc:1240] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:46.482652 13037 consensus_queue.cc:1048] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:46.482977 13024 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.483572 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:46.483834 10841 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:46.484166 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:46.487000 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:46.447475 13056 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:46.493045 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:46.469827 12949 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:46.443171 12925 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:46.494446 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:46.504897 10950 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:46.496942 11507 raft_consensus.cc:2410] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:16:46.493938 10841 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:46.497207 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:46.497742 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:46.475793 12983 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:46.476969 10874 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:46.445466 12583 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:46.517719 11507 raft_consensus.cc:1240] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:46.478226 12775 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:46.493191 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:46.491189 10916 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:46.507824 10841 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:16:46.500398 12852 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.067s	user 0.002s	sys 0.015s
I20260709 13:16:46.507220 10947 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 6.
I20260709 13:16:46.506738 11508 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
I20260709 13:16:46.488323 12889 raft_consensus.cc:697] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:46.496482 13056 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:16:46.521929 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:46.496703 13016 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:46.498620 13032 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:46.521103 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:46.524216 10841 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:46.493796 12766 consensus_queue.cc:1048] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
W20260709 13:16:46.525333 12852 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.123s	user 0.005s	sys 0.016s
I20260709 13:16:46.505192 12949 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:46.528520 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:46.494997 12853 consensus_queue.cc:1048] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 13:16:46.515519 12983 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:46.524355 10916 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:46.556171 12783 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:46.556823 12783 raft_consensus.cc:2749] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:46.559095 10916 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:46.535142 10841 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:46.559993 12925 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:46.560688 12925 raft_consensus.cc:2749] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:46.531881 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:46.561992 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:46.562433 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:46.509095 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:46.563107 13060 raft_consensus.cc:2804] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:46.563570 13060 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:46.563655 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:46.564073 13060 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:46.550994 13032 raft_consensus.cc:2804] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:46.565115 13032 raft_consensus.cc:697] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:46.566049 13032 consensus_queue.cc:237] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:46.566650 10794 raft_consensus.cc:1240] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:46.560827 10841 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:46.531270 13016 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:46.568269 13016 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:46.569854 12949 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:46.404994 11504 rpcz_store.cc:276] 0709 13:16:43.574251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:43.585827 (+ 11576us) service_pool.cc:224] Handling call
0709 13:16:43.616174 (+ 30347us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:43.635435 (+ 19261us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:43.636878 (+  1443us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:44.511780 (+874902us) spinlock_profiling.cc:243] Waited 2.75 ms on lock 0x7b2c000130b8. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7f7aea11 00007f9c7f7be372 00007f9c80ef0a01 00007f9c80eb543a 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2
0709 13:16:44.512318 (+   538us) log.cc:844] Serialized 52 byte log entry
0709 13:16:44.949718 (+437400us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:44.950173 (+   455us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:44.956153 (+  5980us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:46.289331 (+1333178us) spinlock_profiling.cc:243] Waited 25.1 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:16:46.294642 (+  5311us) raft_consensus.cc:1685] finished
0709 13:16:46.294673 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:46.317510 (+ 22837us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":747,"raft.queue_time_us":8436,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":9778,"spinlock_wait_cycles":64113280,"thread_start_us":432060,"threads_started":1,"wal-append.queue_time_us":682827}
I20260709 13:16:46.571889 12766 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:46.572408 12766 raft_consensus.cc:697] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:46.564116 10795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 1380) took 1124 ms. Trace:
I20260709 13:16:46.573258 10795 rpcz_store.cc:276] 0709 13:16:45.439929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:45.451014 (+ 11085us) service_pool.cc:224] Handling call
0709 13:16:46.563942 (+1112928us) spinlock_profiling.cc:243] Waited 272 ms on lock 0x7b5400211bd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:46.564052 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":626634112}
I20260709 13:16:46.573335 12766 consensus_queue.cc:237] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:46.523277 12775 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:46.489678 13049 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:46.574990 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:46.575330 12775 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:46.514948 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:46.513270 12904 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:46.491287 12838 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:46.577679 12838 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.578158 12838 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:46.578401 12889 consensus_queue.cc:237] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:46.580302 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:46.582161 12836 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:46.586982 13057 log.cc:927] Time spent T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.250s	user 0.003s	sys 0.000s
I20260709 13:16:46.530763 13014 raft_consensus.cc:2749] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:46.552717 10943 raft_consensus.cc:1240] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 5 LEADER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:46.588842 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:46.589440 10876 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 6.
I20260709 13:16:46.590354 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:46.590696 13060 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:46.592865 13060 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:46.554893 11503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56404 (request call id 446) took 1122 ms. Trace:
I20260709 13:16:46.593014 12961 consensus_queue.cc:1048] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:16:46.593372 11503 rpcz_store.cc:276] 0709 13:16:45.432265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:45.504569 (+ 72304us) service_pool.cc:224] Handling call
0709 13:16:46.554654 (+1050085us) spinlock_profiling.cc:243] Waited 742 ms on lock 0x7b5400467654. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:46.554752 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1706342272}
W20260709 13:16:46.555310 10872 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49290 (request call id 227) took 3354 ms (client timeout 1593 ms). Trace:
I20260709 13:16:46.528182 13056 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:16:46.594601 10872 rpcz_store.cc:269] 0709 13:16:43.200490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:43.200688 (+   198us) service_pool.cc:224] Handling call
0709 13:16:46.555093 (+3354405us) spinlock_profiling.cc:243] Waited 2.35 s on lock 0x7b540009b0d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:46.555223 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5397024640}
I20260709 13:16:46.595728 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:46.536517 13052 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:16:46.504189 13024 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:46.596855 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:46.597419 10872 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 4.
I20260709 13:16:46.598237 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:46.598414 13024 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:46.587852 10841 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:46.599089 11508 raft_consensus.cc:2393] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 3: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:46.599191 13051 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:46.600011 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:46.600373 10841 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:46.601140 13051 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:46.601507 13051 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:46.601739 10869 raft_consensus.cc:2393] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 5: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:46.601835 13051 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:46.603434 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:46.603982 10869 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:16:46.606758 11512 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56416 (request call id 446) took 1952 ms (client timeout 1568 ms). Trace:
W20260709 13:16:46.608367 13044 log.cc:927] Time spent T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.599s	user 0.002s	sys 0.000s
I20260709 13:16:46.610404 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:46.614749 11511 raft_consensus.cc:2393] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 5: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:46.615348 12934 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:46.607434 10841 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:46.595856 13049 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:46.611501 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:46.633714 11510 raft_consensus.cc:2410] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:16:46.613063 11512 rpcz_store.cc:269] 0709 13:16:44.654878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:44.675622 (+ 20744us) service_pool.cc:224] Handling call
0709 13:16:46.008251 (+1332629us) spinlock_profiling.cc:243] Waited 1.62 ms on lock 0x7b5800003c98. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd90a22 00007f9c86ca9c09 00007f9c85e03ee0 00007f9c85ecd0ab 00007f9c85ecd131 00007f9c85e7b0a0 00007f9c85e7a984 00007f9c7f3f56d4 00007f9c7f3f5662
0709 13:16:46.606544 (+598293us) spinlock_profiling.cc:243] Waited 281 ms on lock 0x7b5400484754. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:46.606686 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":649966336}
I20260709 13:16:46.611425 10876 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:46.635223 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:46.548367 11477 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:46.630050 12756 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:46.630421 10841 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:46.631604 13034 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:16:46.638382 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 90132us
I20260709 13:16:46.651609 10950 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 4.
W20260709 13:16:46.650666 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 309357us
I20260709 13:16:46.629136 10869 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 5.
I20260709 13:16:46.656477 10766 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:46.656913 10841 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:46.631486 12899 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:46.582036 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:46.635290 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:46.595798 12583 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:46.669561 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:46.670413 11512 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 5.
I20260709 13:16:46.672827 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:46.673712 10872 raft_consensus.cc:2368] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for earlier term 4. Current term is 5.
I20260709 13:16:46.668174 10841 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:46.675870 10841 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:46.611315 12838 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:46.664708 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:46.665954 11511 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 6.
I20260709 13:16:46.695871 10869 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 4.
I20260709 13:16:46.651823 13024 raft_consensus.cc:2749] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:16:46.683771 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:46.679709  9682 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:46.699460 10943 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 3.
W20260709 13:16:46.697589 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 391768us
I20260709 13:16:46.679023 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:46.685312 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:46.653364 13034 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:46.558625 12996 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:46.667058 13049 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:46.805887 12683 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:46.698355 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:16:46.702804 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 70287.6us
I20260709 13:16:46.670157 12934 raft_consensus.cc:2749] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:46.807396 10916 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:46.807906 10841 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:46.672376 12925 raft_consensus.cc:2749] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:46.702279 10950 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
I20260709 13:16:46.711374 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:46.703526 11477 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:46.704972 11478 leader_election.cc:400] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for earlier term 4. Current term is 5.
I20260709 13:16:46.829419 11478 leader_election.cc:403] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:16:46.679447 12865 consensus_queue.cc:1048] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 13:16:46.660553 13012 consensus_queue.cc:1048] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.051s
I20260709 13:16:46.613384 13051 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:46.661579 10766 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:46.670886 10811 tablet_copy_source_session.cc:215] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:46.697064 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:46.864300 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:16:46.864605 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 203103us
I20260709 13:16:46.610677 13056 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:46.827739 10874 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 4.
I20260709 13:16:46.833320 10876 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 4.
I20260709 13:16:46.664024 12688 consensus_queue.cc:1048] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.089s
W20260709 13:16:46.839365 13036 log.cc:927] Time spent T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.916s	user 0.013s	sys 0.009s
I20260709 13:16:46.707561 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:46.661345 12872 consensus_queue.cc:1048] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.051s
I20260709 13:16:46.696009 12904 raft_consensus.cc:2804] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:16:46.847826 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 144363us
I20260709 13:16:46.698438 12798 consensus_queue.cc:1048] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.161s
I20260709 13:16:46.681028 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:46.892095 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:46.838263 12949 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:46.865731 10766 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:46.699558 13046 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:46.869443 10767 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
W20260709 13:16:46.933380 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 64068.9us
I20260709 13:16:46.807308 13034 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:46.873334 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:46.808157 12990 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:46.873315 10699 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42506 (request call id 152) took 15164 ms (client timeout 15000 ms). Trace:
I20260709 13:16:46.678969 12574 consensus_queue.cc:1059] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.179s
I20260709 13:16:46.679123 12961 consensus_queue.cc:1059] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.173s
W20260709 13:16:46.848579 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 143793us
W20260709 13:16:46.846735 12768 consensus_meta.cc:287] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.555s	user 0.007s	sys 0.003s
I20260709 13:16:46.713054 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:46.887537 12914 raft_consensus.cc:697] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:46.984349 12914 consensus_queue.cc:237] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:46.984908 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:46.888562 12684 raft_consensus.cc:2749] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:46.853294 10942 raft_consensus.cc:1240] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:46.847070 13015 raft_consensus.cc:2749] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:16:46.894856 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:46.989213 10875 raft_consensus.cc:2393] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 4: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:46.702191 12671 consensus_queue.cc:1059] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.174s
I20260709 13:16:46.711014 12858 consensus_queue.cc:1048] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.159s
I20260709 13:16:46.894759 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:46.902310 10947 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 6.
I20260709 13:16:46.837944 12895 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:46.864117 12881 raft_consensus.cc:2804] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:46.853166 13071 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:46.908413 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:46.912365 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1369) took 1311 ms. Trace:
I20260709 13:16:46.944769 13029 raft_consensus.cc:2804] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:47.001969 10873 rpcz_store.cc:276] 0709 13:16:45.601482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:45.601949 (+   467us) service_pool.cc:224] Handling call
0709 13:16:45.637006 (+ 35057us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:45.637277 (+   271us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:16:45.674612 (+ 37335us) log.cc:844] Serialized 14 byte log entry
0709 13:16:46.248133 (+573521us) spinlock_profiling.cc:243] Waited 1.34 ms on lock 0x7b18002a2bc0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a5523a 00007f9c80ebfa2a
0709 13:16:46.513611 (+265478us) log.cc:844] Serialized 14 byte log entry
0709 13:16:46.874858 (+361247us) spinlock_profiling.cc:243] Waited 8.43 ms on lock 0x7b18002a2bc0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc83f3 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:16:46.910498 (+ 35640us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:46.910578 (+    80us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:46.910647 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:46.910742 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:46.912290 (+  1548us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2994,"prepare.queue_time_us":20209,"prepare.run_cpu_time_us":104,"prepare.run_wall_time_us":2892,"raft.queue_time_us":9509,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":20898,"spinlock_wait_cycles":29492736,"thread_start_us":661632,"threads_started":2,"wal-append.queue_time_us":1042463}
W20260709 13:16:46.922194 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 56581.1us
I20260709 13:16:46.875541 12934 raft_consensus.cc:2749] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:46.934728 10767 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:16:47.005611 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 71419us
I20260709 13:16:46.896270 12904 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:46.945009 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:46.947397 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:46.939270 13073 raft_consensus.cc:2804] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:46.951885 12683 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:46.952191 12912 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:47.012451 10798 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 4.
W20260709 13:16:46.953222 13067 log.cc:927] Time spent T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 13:16:46.947114 13028 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:46.953975 13066 log.cc:927] Time spent T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:16:46.957079 13056 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:46.916813 13052 raft_consensus.cc:2749] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:16:46.958344 13063 log.cc:927] Time spent T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.150s	user 0.000s	sys 0.000s
I20260709 13:16:46.961262 12949 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:46.961853 11509 raft_consensus.cc:2393] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 5: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:46.963066 12899 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:46.966604 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:47.024106 12961 raft_consensus.cc:3055] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:46.968984 12775 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:46.971028 12996 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:16:46.972100 13055 consensus_meta.cc:287] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.820s	user 0.009s	sys 0.004s
I20260709 13:16:46.975222 12983 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:46.976079 12990 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:46.932893 13046 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:46.980314 10792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:39430 (request call id 12) took 1101 ms. Trace:
I20260709 13:16:46.980755 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:46.981298 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:46.917757 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:46.991848 10916 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:46.992561 10875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 1373) took 1042 ms. Trace:
W20260709 13:16:46.999886 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 192058us
I20260709 13:16:47.074893 10875 rpcz_store.cc:276] 0709 13:16:45.954655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:45.967428 (+ 12773us) service_pool.cc:224] Handling call
0709 13:16:46.894664 (+927236us) spinlock_profiling.cc:243] Waited 2.59 ms on lock 0x7b5800177098. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd90a22 00007f9c86ca9c09 00007f9c85e03ee0 00007f9c85ecd0ab 00007f9c85ecd131 00007f9c85e7b0a0 00007f9c85e7a984 00007f9c7f3f56d4 00007f9c7f3f5662
0709 13:16:46.992493 (+ 97829us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5965568}
I20260709 13:16:47.073256 10792 rpcz_store.cc:276] 0709 13:16:45.879061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:45.881140 (+  2079us) service_pool.cc:224] Handling call
0709 13:16:46.979985 (+1098845us) spinlock_profiling.cc:243] Waited 649 ms on lock 0x7b54002125d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80ebb735 00007f9c85e7bd29 00007f9c7f3fa1d4 00007f9c7f3fa162 00007f9c7f3fa091 00007f9c7f3fa00c 00007f9c7f3f92c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:46.980218 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1491585408}
I20260709 13:16:47.075939 10841 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:46.863195 13024 raft_consensus.cc:2749] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for earlier term 4. Current term is 5.
W20260709 13:16:47.078400 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 86672.8us
I20260709 13:16:47.000735 10915 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:47.008728 10767 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:47.079469 10916 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:47.005518 10766 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:47.013069 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:16:47.080107 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 71476.8us
I20260709 13:16:47.022281 11478 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:46.992367 12783 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:16:47.000535 13065 log.cc:927] Time spent T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 13:16:47.001595 12881 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:47.006117 13029 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:47.025350 12904 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:47.050334 10871 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:46.961306 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:47.031252 12865 raft_consensus.cc:3055] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:47.024770 13074 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:47.057092 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:47.050084 13016 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:46.993794 12768 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:47.099902 12768 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:47.038408 13073 raft_consensus.cc:697] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:47.101239 13073 consensus_queue.cc:237] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:47.101783 12990 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:47.044344 13049 raft_consensus.cc:2749] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:16:47.103048 11537 heartbeater.cc:646] Failed to heartbeat to 127.9.116.190:36007 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.9.116.190:36007 timed out after 15.000s (SENT)
I20260709 13:16:47.091193 12889 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:47.103780 12889 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:47.104250 12889 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:47.104909 12983 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:46.938318 10699 rpcz_store.cc:269] 0709 13:16:31.709257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:31.710939 (+  1682us) service_pool.cc:224] Handling call
0709 13:16:32.187350 (+476411us) spinlock_profiling.cc:243] Waited 198 ms on lock 0x7b54002fce54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1f090 00007f9c80ebdf3a 00007f9c86c741cc 00007f9c86d7237e 00007f9c852332a4 00007f9c85233232 00007f9c85233161 00007f9c852330dc 00007f9c85232392 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:46.873198 (+14685848us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:16:40.134721 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:16:40.135604 (+   883us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:16:40.135662 (+    58us) write_op.cc:435] Acquired schema lock
0709 13:16:40.135686 (+    24us) tablet.cc:662] Decoding operations
0709 13:16:40.163466 (+ 27780us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:16:40.182763 (+ 19297us) write_op.cc:641] Partition lock acquired for write op
0709 13:16:40.186525 (+  3762us) tablet.cc:685] Acquiring locks for 6 operations
0709 13:16:40.229761 (+ 43236us) tablet.cc:701] Row locks acquired
0709 13:16:40.229817 (+    56us) write_op.cc:260] PREPARE: finished
0709 13:16:40.230009 (+   192us) write_op.cc:270] Start()
0709 13:16:40.230193 (+   184us) write_op.cc:276] Timestamp: P: 1783603000229926 usec, L: 0
0709 13:16:40.230242 (+    49us) op_driver.cc:348] REPLICATION: starting
0709 13:16:40.235864 (+  5622us) log.cc:844] Serialized 2452 byte log entry
0709 13:16:41.021325 (+785461us) op_driver.cc:464] REPLICATION: finished
0709 13:16:41.286401 (+265076us) write_op.cc:301] APPLY: starting
0709 13:16:41.286542 (+   141us) tablet.cc:1367] starting BulkCheckPresence
0709 13:16:41.286845 (+   303us) tablet.cc:1370] finished BulkCheckPresence
0709 13:16:41.287011 (+   166us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:16:41.291636 (+  4625us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:16:41.291956 (+   320us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=6
0709 13:16:41.292020 (+    64us) write_op.cc:312] APPLY: finished
0709 13:16:41.292323 (+   303us) log.cc:844] Serialized 53 byte log entry
0709 13:16:41.292540 (+   217us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:41.292649 (+   109us) write_op.cc:454] Released schema lock
0709 13:16:41.292715 (+    66us) write_op.cc:341] FINISH: Updating metrics
0709 13:16:41.340556 (+ 47841us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:41.344134 (+  3578us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":203,"reported_tablets":13,"spinlock_wait_cycles":455265664,"child_traces":[["op",{"apply.queue_time_us":264721,"apply.run_cpu_time_us":3728,"apply.run_wall_time_us":15465,"num_ops":6,"prepare.queue_time_us":24149,"prepare.run_cpu_time_us":50888,"prepare.run_wall_time_us":181213,"raft.queue_time_us":392233,"raft.run_cpu_time_us":16853,"raft.run_wall_time_us":202570,"replication_time_us":790895,"thread_start_us":557045,"threads_started":4,"wal-append.queue_time_us":191046,"wal-append.run_cpu_time_us":35499,"wal-append.run_wall_time_us":1866039}]]}
I20260709 13:16:47.087397 12783 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:47.087494 12893 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:47.089112 12881 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:47.107326 12893 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:47.107656 12783 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:47.107892 12893 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:46.952330 13051 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:47.081012 10767 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:47.109215 10870 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 4: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:47.052842 12961 raft_consensus.cc:740] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:47.042018 13071 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:47.090137 13029 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:47.082132 12843 consensus_queue.cc:1059] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
W20260709 13:16:47.083364 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 77921.7us
I20260709 13:16:47.111577 12775 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:16:47.115407 10747 auto_rebalancer.cc:629] Failed to schedule move for tablet 8fd9828f212b47c19244738ee2a27a95: Illegal state: Replica e01ecd008edd4aa2b6b2b47efbc2ea87 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 5 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:16:47.119634 12895 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:47.085084 12934 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:47.111580 12843 raft_consensus.cc:3055] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:47.041045 13028 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:47.123754 12843 raft_consensus.cc:740] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:47.124502 10873 raft_consensus.cc:1275] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:16:47.124993 10873 pending_rounds.cc:85] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 3
I20260709 13:16:47.125221 10873 pending_rounds.cc:107] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:16:47.124596 12843 consensus_queue.cc:260] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:47.125440 10873 raft_consensus.cc:2889] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:47.125753 13028 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:47.125885 12843 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:47.126137 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1383) took 1373 ms. Trace:
I20260709 13:16:47.092976 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:47.126860 10875 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 5.
I20260709 13:16:47.127846 10842 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:47.128139 10875 raft_consensus.cc:1240] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:47.128804 10875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1366) took 1279 ms. Trace:
I20260709 13:16:47.080614 13046 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:16:47.129509 13068 log.cc:927] Time spent T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.226s	user 0.000s	sys 0.001s
I20260709 13:16:47.078012 10841 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:47.110327 12961 consensus_queue.cc:260] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:47.035560 12912 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:47.131206 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:16:47.131574 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 53647.4us
I20260709 13:16:47.132390 13031 raft_consensus.cc:2749] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:16:47.132573 12912 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:47.133096 13031 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:47.133188 10868 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 6.
I20260709 13:16:47.133507 13031 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:47.133914 13031 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:47.134122 12961 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:47.080297 12964 raft_consensus.cc:2749] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:47.084661 10916 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:47.091830 13074 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:47.083858 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 83211.1us
W20260709 13:16:47.136153 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 51574.6us
I20260709 13:16:47.137300 10869 raft_consensus.cc:1240] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:47.129185 10842 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:47.138142 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1364) took 1297 ms. Trace:
I20260709 13:16:47.138521 10869 rpcz_store.cc:276] 0709 13:16:45.842997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.136886 (+1293889us) service_pool.cc:224] Handling call
0709 13:16:47.137150 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:47.137868 (+   718us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:47.138085 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:47.129166 10875 rpcz_store.cc:276] 0709 13:16:45.850994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.127598 (+1276604us) service_pool.cc:224] Handling call
0709 13:16:47.127964 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:47.128557 (+   593us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:47.128748 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:47.137296 10916 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:47.126506 10873 rpcz_store.cc:276] 0709 13:16:45.753099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.123826 (+1370727us) service_pool.cc:224] Handling call
0709 13:16:47.124169 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:47.125886 (+  1717us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:47.126060 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:47.150669 10915 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:47.153584 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:47.154321 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:47.155032 10944 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:47.156380 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:47.159495 13039 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:47.087581 12949 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:47.163288 12949 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:47.163632 10949 raft_consensus.cc:1275] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 6. (term mismatch)
I20260709 13:16:47.163735 12949 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:47.164127 10949 pending_rounds.cc:85] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 5
I20260709 13:16:47.164410 10949 pending_rounds.cc:107] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 13:16:47.164671 10949 raft_consensus.cc:2889] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:47.136065 13074 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:47.167201 12964 raft_consensus.cc:2804] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:47.167701 12964 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:47.167800 13071 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:47.168182 12964 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:47.081897 12996 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:47.170599 12996 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:47.171119 12996 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:47.172567 13016 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:16:47.090719 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:47.178900 10916 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:16:47.180123 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 99194.4us
I20260709 13:16:47.190237 10767 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:47.073494 13055 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:47.191695 10876 raft_consensus.cc:2393] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 7: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:47.174757 12912 raft_consensus.cc:2804] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:47.192162 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:47.195523 10875 raft_consensus.cc:1240] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:47.093442 12865 raft_consensus.cc:740] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:47.196527 10766 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:47.197086 10875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1387) took 1245 ms. Trace:
I20260709 13:16:47.197471 10875 rpcz_store.cc:276] 0709 13:16:45.951365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.140129 (+1188764us) service_pool.cc:224] Handling call
0709 13:16:47.195361 (+ 55232us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:47.195937 (+   576us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:47.197019 (+  1082us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:47.197427 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:47.198660 10950 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:47.199447 10873 raft_consensus.cc:1275] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:47.199980 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:47.199955 10916 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:47.203166 10949 raft_consensus.cc:2410] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 6 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:16:47.179976 12895 raft_consensus.cc:2804] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:47.203096 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1389) took 1108 ms. Trace:
I20260709 13:16:47.209892 10873 rpcz_store.cc:276] 0709 13:16:46.094311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.174743 (+1080432us) service_pool.cc:224] Handling call
0709 13:16:47.199130 (+ 24387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:47.199813 (+   683us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:47.203038 (+  3225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:47.187520 12783 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:47.192952 12775 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:47.186673 12671 consensus_queue.cc:1243] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer cb0428cc55984daba1ba63744a5102a2 log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:16:47.213310 10875 raft_consensus.cc:1240] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 13:16:47.191109 12838 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:47.193365 12934 raft_consensus.cc:697] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:47.228785 10875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 233) took 1027 ms. Trace:
I20260709 13:16:47.229195 10875 rpcz_store.cc:276] 0709 13:16:46.201659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.198437 (+996778us) service_pool.cc:224] Handling call
0709 13:16:47.213030 (+ 14593us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:47.228495 (+ 15465us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:47.228727 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:47.221561 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1365) took 1376 ms. Trace:
I20260709 13:16:47.230942 10867 rpcz_store.cc:276] 0709 13:16:45.847341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.105581 (+1258240us) service_pool.cc:224] Handling call
0709 13:16:47.105908 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:47.106137 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:16:47.106467 (+   330us) log.cc:844] Serialized 14 byte log entry
0709 13:16:47.215527 (+109060us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:47.221108 (+  5581us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:47.221184 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:47.221290 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:47.221483 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9568,"prepare.queue_time_us":638,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":20856,"thread_start_us":79425,"threads_started":1,"wal-append.queue_time_us":106269}
I20260709 13:16:47.198475 13071 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:47.213104 12783 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:47.237831 10867 raft_consensus.cc:1275] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:16:47.241983 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 238) took 1032 ms. Trace:
W20260709 13:16:47.238116 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 47970.5us
I20260709 13:16:47.194417 12912 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:47.200979 12756 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.242386 10867 rpcz_store.cc:276] 0709 13:16:46.209478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.233762 (+1024284us) service_pool.cc:224] Handling call
0709 13:16:47.234568 (+   806us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:47.241725 (+  7157us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:47.241928 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:47.241127 10873 raft_consensus.cc:1240] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 LEADER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:16:47.206169 12930 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:47.206054 13056 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:47.219079 12583 raft_consensus.cc:2804] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:47.246263 10875 raft_consensus.cc:1240] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:16:47.248163 10766 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:16:47.292366 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 44289.6us
I20260709 13:16:47.292850 10875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 237) took 1087 ms. Trace:
I20260709 13:16:47.293159 10766 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:47.249354 10767 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:47.239475 10871 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 6.
I20260709 13:16:47.253894 10950 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 5.
I20260709 13:16:47.257617 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 235) took 1055 ms. Trace:
I20260709 13:16:47.239223 12836 raft_consensus.cc:2804] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:47.234943 12895 raft_consensus.cc:697] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 4 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:47.299702 10944 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 6.
I20260709 13:16:47.202543 13016 consensus_queue.cc:1059] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:16:47.201021 12768 consensus_queue.cc:1059] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260709 13:16:47.219784 13039 consensus_queue.cc:1048] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260709 13:16:47.257090 12912 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:47.294986 10871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 1386) took 1344 ms. Trace:
I20260709 13:16:47.312086 10871 rpcz_store.cc:276] 0709 13:16:45.950973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:45.972468 (+ 21495us) service_pool.cc:224] Handling call
0709 13:16:46.905340 (+932872us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b5800177098. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd90a22 00007f9c86ca9c09 00007f9c85e03ee0 00007f9c85ecd0ab 00007f9c85ecd131 00007f9c85e7b0a0 00007f9c85e7a984 00007f9c7f3f56d4 00007f9c7f3f5662
0709 13:16:46.966453 (+ 61113us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x7b5800177098. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90e9c 00007f9c7dd90a5b 00007f9c86ca9c09 00007f9c85e03ee0 00007f9c85ecd0ab 00007f9c85ecd131 00007f9c85e7b0a0 00007f9c85e7a984 00007f9c7f3f56d4 00007f9c7f3f5662
0709 13:16:47.294923 (+328470us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":52197888}
I20260709 13:16:47.303260 10873 rpcz_store.cc:276] 0709 13:16:46.202171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.211321 (+1009150us) service_pool.cc:224] Handling call
0709 13:16:47.240880 (+ 29559us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:47.257339 (+ 16459us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:47.257560 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:47.295267 10874 raft_consensus.cc:1240] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:47.243492 12783 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:47.293277 10875 rpcz_store.cc:276] 0709 13:16:46.205244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.230155 (+1024911us) service_pool.cc:224] Handling call
0709 13:16:47.246029 (+ 15874us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:47.291580 (+ 45551us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:47.292782 (+  1202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:47.240568 12838 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:47.318162 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:47.320672 10871 raft_consensus.cc:1240] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:16:47.311031 12990 raft_consensus.cc:2749] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:47.293517 13028 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.291543 12583 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:47.291590 12930 raft_consensus.cc:697] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 4 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:47.331825 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:47.321746 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:47.324666 10871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 242) took 1094 ms. Trace:
I20260709 13:16:47.283380 13056 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:47.328969 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:47.243602 13071 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:47.332716 10871 rpcz_store.cc:276] 0709 13:16:46.229826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.320139 (+1090313us) service_pool.cc:224] Handling call
0709 13:16:47.320453 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:47.324318 (+  3865us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:47.324593 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:47.259472 12756 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:47.335675 12930 consensus_queue.cc:237] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:47.336549 12993 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:47.336475 12881 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:47.337415 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1369) took 1475 ms. Trace:
I20260709 13:16:47.337929 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:47.337759 12893 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:47.338339 13055 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:47.340584 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:47.349995 12583 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:47.348048 10797 raft_consensus.cc:1275] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:16:47.330303 13097 log.cc:927] Time spent T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:16:47.259792 12865 consensus_queue.cc:260] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:47.325131 12775 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:47.362551 10869 rpcz_store.cc:276] 0709 13:16:45.884034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.139416 (+1255382us) service_pool.cc:224] Handling call
0709 13:16:47.139756 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:47.193414 (+ 53658us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:47.193445 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:47.201078 (+  7633us) log.cc:844] Serialized 26 byte log entry
0709 13:16:47.244544 (+ 43466us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:47.286166 (+ 41622us) log.cc:844] Serialized 14 byte log entry
0709 13:16:47.286386 (+   220us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:47.286460 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:47.337217 (+ 50757us) raft_consensus.cc:1685] finished
0709 13:16:47.337237 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:47.337359 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51424,"prepare.queue_time_us":46185,"raft.queue_time_us":55221,"raft.run_cpu_time_us":262,"raft.run_wall_time_us":19846,"spinlock_wait_cycles":16413952,"thread_start_us":37743,"threads_started":1,"wal-append.queue_time_us":118813}
I20260709 13:16:47.383602 12865 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:47.324525 12836 raft_consensus.cc:697] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:47.385658 12836 consensus_queue.cc:237] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:47.386909 12895 consensus_queue.cc:237] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:47.322103 13099 raft_consensus.cc:2749] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:47.368907 12881 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:47.370512 12756 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:47.316414 12889 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:47.365180 12993 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:47.371860 13046 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:47.331286 12914 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:47.374022 10800 raft_consensus.cc:2393] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 4: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:47.346521 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:16:47.375182 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 125897us
I20260709 13:16:47.321771 13100 consensus_queue.cc:1059] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
W20260709 13:16:47.411057 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 116706us
I20260709 13:16:47.436484 10766 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:47.393671 12949 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:47.362413 12994 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:47.350548 13028 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:47.375626 13002 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:47.326246 12934 consensus_queue.cc:237] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:47.447543 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:47.448464 10793 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:16:47.424574 12768 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.393668 13031 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:47.434890 10794 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:47.329749 13034 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
W20260709 13:16:47.438424 13095 log.cc:927] Time spent T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
I20260709 13:16:47.453646 10875 raft_consensus.cc:1240] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:47.434451 12914 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:47.436354 13039 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:47.443612 13017 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.480201 10868 raft_consensus.cc:1240] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:47.486501 10794 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 5.
I20260709 13:16:47.499675 10873 raft_consensus.cc:1240] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 LEADER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 6. Current term is 7. Ops: []
W20260709 13:16:47.509282 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 54184.2us
I20260709 13:16:47.489560 10798 raft_consensus.cc:1275] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:16:47.515610 10766 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:47.525722 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:47.529527 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:47.471634 12914 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:47.450546 12994 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:47.534408 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:16:47.488665 13092 log.cc:927] Time spent T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.246s	user 0.001s	sys 0.000s
I20260709 13:16:47.538237 10766 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:16:47.538859 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 90057.2us
I20260709 13:16:47.512842 12930 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:47.461352 12768 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:47.542122 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:47.454203 13002 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:47.546546 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:47.550411 10798 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:47.508775 12843 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.547744 10766 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:47.531869 13099 raft_consensus.cc:2749] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:47.459520 13101 consensus_queue.cc:1059] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.077s
I20260709 13:16:47.561487 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:47.488440 13039 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:47.509223 13017 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:47.512229 13103 consensus_queue.cc:1059] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.061s
I20260709 13:16:47.569044 10796 raft_consensus.cc:1240] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 5. Current term is 6. Ops: [3.2-5.3]
I20260709 13:16:47.573071 10793 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 4.
I20260709 13:16:47.574620 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:47.575592 10916 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:47.585884 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:47.585961 10947 raft_consensus.cc:1240] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 4 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:16:47.589561 10950 raft_consensus.cc:1240] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:47.596572 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:47.610368 10798 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 7.
I20260709 13:16:47.545748 12964 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:47.630256 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:47.640040 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:47.640803 10796 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:47.587189 13015 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:47.558064 12756 consensus_queue.cc:1048] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 13:16:47.655939 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:47.554152 12930 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:47.593586 13109 raft_consensus.cc:2749] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:47.557345 12843 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:47.598277 13071 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:47.668330 10871 raft_consensus.cc:1240] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:47.592922 13032 consensus_queue.cc:1059] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.067s
I20260709 13:16:47.632915 13113 raft_consensus.cc:2749] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:16:47.681838 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:47.600677 12895 consensus_queue.cc:1048] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.071s
I20260709 13:16:47.700906 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:47.702005 10800 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 4: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:47.664419 12889 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:47.670094 13073 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:47.671401 13055 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.679251 13046 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:47.680140 12949 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:47.704263 13034 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:47.733912  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:47.714890 13031 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:16:47.743464 13029 consensus_meta.cc:287] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.632s	user 0.012s	sys 0.000s
W20260709 13:16:47.754492 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 153157us
I20260709 13:16:47.727161 13073 raft_consensus.cc:697] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:47.758337 10916 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:47.769878 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:47.726855 13055 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:47.755380 12994 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:16:47.773046 13111 log.cc:927] Time spent T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 13:16:47.779390 13002 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:47.780234 13073 consensus_queue.cc:237] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:47.783124 12996 consensus_meta.cc:287] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.611s	user 0.009s	sys 0.001s
I20260709 13:16:47.806085 12996 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:47.783537 13110 log.cc:927] Time spent T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 13:16:47.765049 13029 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:47.807866 10798 raft_consensus.cc:1240] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 4. Current term is 5. Ops: [2.2-4.3]
I20260709 13:16:47.809020 13017 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:47.809409 13029 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:16:47.791356 13114 log.cc:927] Time spent T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:16:47.809790 10798 raft_consensus.cc:1240] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:16:47.791715 12895 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:47.760797 12768 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:47.792558 12574 raft_consensus.cc:3055] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:47.815269 12574 raft_consensus.cc:740] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:47.816118 12574 consensus_queue.cc:260] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:47.816314 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:47.817297 12574 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:47.817610 12583 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:47.817966 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:47.818857 10801 raft_consensus.cc:2393] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 6: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:47.819886 12930 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:47.820881 12843 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:47.825590 12964 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:47.826512 10870 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 6.
W20260709 13:16:47.827339 12961 consensus_meta.cc:287] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.693s	user 0.008s	sys 0.004s
I20260709 13:16:47.827720 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:16:47.828483 12783 consensus_meta.cc:287] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.504s	user 0.009s	sys 0.001s
I20260709 13:16:47.829023 12993 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:16:47.826506 12904 consensus_meta.cc:287] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.710s	user 0.011s	sys 0.000s
I20260709 13:16:47.830368 12904 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:47.831051 12996 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:16:47.804381 13108 log.cc:927] Time spent T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 13:16:47.833446 12904 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:47.828547 10870 raft_consensus.cc:2410] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 13:16:47.829025 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 692163us
I20260709 13:16:47.831578 10872 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 3.
I20260709 13:16:47.836345 13102 raft_consensus.cc:3055] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:47.836777 13102 raft_consensus.cc:740] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:47.837502 10872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49290 (request call id 249) took 1171 ms. Trace:
I20260709 13:16:47.837930 10841 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:47.837636 13102 consensus_queue.cc:260] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:47.839370 13102 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:47.801299 13055 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:47.845127 13124 raft_consensus.cc:2749] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:47.848326 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:47.849182 10870 raft_consensus.cc:2410] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 7 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:16:47.850267 10870 raft_consensus.cc:1275] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:16:47.851344 10870 raft_consensus.cc:1240] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:47.853703 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:47.854019 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:47.854329 10943 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:47.855079 12583 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:47.770182 10916 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:47.771405 10869 raft_consensus.cc:2393] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 4: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:47.837885 10872 rpcz_store.cc:276] 0709 13:16:46.666603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.696641 (+ 30038us) service_pool.cc:224] Handling call
0709 13:16:47.837344 (+1140703us) spinlock_profiling.cc:243] Waited 959 ms on lock 0x7b54001095d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:47.837444 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2205798656}
I20260709 13:16:47.856786 10869 raft_consensus.cc:1240] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:47.855713 10949 raft_consensus.cc:1240] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 5. Current term is 6. Ops: [3.2-5.3]
I20260709 13:16:47.857995 10869 raft_consensus.cc:1275] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:16:47.860126 12838 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:47.860858 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 243) took 1623 ms. Trace:
I20260709 13:16:47.861238 10873 rpcz_store.cc:276] 0709 13:16:46.237398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.321059 (+1083661us) service_pool.cc:224] Handling call
0709 13:16:47.333818 (+ 12759us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:47.513608 (+179790us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:47.860644 (+347036us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b54000997d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:47.860807 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":380839296}
I20260709 13:16:47.862627 12583 consensus_queue.cc:1059] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:47.863548 13073 consensus_queue.cc:1059] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:47.864362 13073 raft_consensus.cc:3055] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:47.864706 13073 raft_consensus.cc:740] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:47.857065 10872 raft_consensus.cc:1275] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:16:47.865432 13073 consensus_queue.cc:260] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:47.871093 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:47.872067 10943 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 6.
I20260709 13:16:47.812501 12912 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:47.876677 10949 raft_consensus.cc:1240] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:16:47.878038 12798 consensus_queue.cc:1059] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:16:47.879055 12798 raft_consensus.cc:3055] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:16:47.882678 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 715858us
I20260709 13:16:47.879372 10946 raft_consensus.cc:1275] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:16:47.880016 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:47.892062 10950 raft_consensus.cc:2410] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:16:47.886202 10875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 247) took 1363 ms. Trace:
W20260709 13:16:47.882558 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 860353us
I20260709 13:16:47.892675 10875 rpcz_store.cc:276] 0709 13:16:46.522438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.323423 (+800985us) service_pool.cc:224] Handling call
0709 13:16:47.331695 (+  8272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:47.455693 (+123998us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:47.875365 (+419672us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b5400109ad4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:47.886107 (+ 10742us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":279991680}
I20260709 13:16:47.893467 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:47.893781 11478 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:47.879205 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:47.895388 10948 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:16:47.897082 10949 raft_consensus.cc:1240] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:16:47.829124 12783 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:47.898852 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:47.899024 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:47.899482 10942 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 6.
I20260709 13:16:47.897850 10796 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7.
I20260709 13:16:47.891078 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:47.894244 10943 raft_consensus.cc:2393] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 7: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
W20260709 13:16:47.902624 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 132496us
I20260709 13:16:47.906909 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:47.827415 12895 raft_consensus.cc:2749] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:47.911204 10947 raft_consensus.cc:1240] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:16:47.911970 10946 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 4: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:47.898288 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:47.913136 10949 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:47.909188 10916 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:16:47.879575 13074 consensus_meta.cc:287] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.713s	user 0.007s	sys 0.005s
I20260709 13:16:47.866926 13073 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:47.917213 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:16:47.917915 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:47.920433 10841 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:47.924573 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:47.879465 12798 raft_consensus.cc:740] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
W20260709 13:16:47.945354 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 51702.8us
I20260709 13:16:47.946702 11478 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:47.947438 12930 consensus_queue.cc:1048] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:47.947894 12912 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:16:47.924081 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:47.949270 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:47.949558 12961 raft_consensus.cc:2749] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:16:47.945030 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 55678.7us
I20260709 13:16:47.950400 11477 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:47.951292 11477 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:47.952728 13015 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:47.876240 12583 consensus_queue.cc:1243] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer cb0428cc55984daba1ba63744a5102a2 log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:16:47.953855 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:47.954038 12574 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:47.895037 13072 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.955225 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:47.957746 10869 raft_consensus.cc:2368] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 6. Current term is 7.
I20260709 13:16:47.958166 11478 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:47.927973 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:47.959458 10872 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 5.
W20260709 13:16:47.952910 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:47.923698 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:47.963001 10948 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7.
I20260709 13:16:47.963922 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:47.967767 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:47.954543 12574 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:47.946693 12996 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:47.969967 12574 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:47.882398 13024 raft_consensus.cc:2749] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:48.007306 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:47.954483 12858 raft_consensus.cc:2804] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:48.016139 10948 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 3.
I20260709 13:16:47.973026 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:16:47.971033 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 50676.4us
I20260709 13:16:48.022029 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:48.030236 10949 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 6.
I20260709 13:16:48.034000 10871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 248) took 1449 ms. Trace:
I20260709 13:16:48.034457 10871 rpcz_store.cc:276] 0709 13:16:46.584954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.352201 (+767247us) service_pool.cc:224] Handling call
0709 13:16:47.352479 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:47.668950 (+316471us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:48.018548 (+349598us) spinlock_profiling.cc:243] Waited 316 ms on lock 0x7b5400077954. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:48.033936 (+ 15388us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":725836544}
I20260709 13:16:47.917488 13014 raft_consensus.cc:2804] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:47.972090 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:47.973580 10916 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:47.974399 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:47.974296 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:47.916239 13055 consensus_queue.cc:1048] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.100s
I20260709 13:16:47.972867 12671 raft_consensus.cc:2749] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:16:48.037856 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:48.009253 12964 raft_consensus.cc:2804] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:16:48.038516 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 65009.9us
I20260709 13:16:48.045640 11508 raft_consensus.cc:2368] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 3. Current term is 4.
I20260709 13:16:48.046012 10841 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:47.970376 13096 raft_consensus.cc:2749] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:48.043303 10800 raft_consensus.cc:2393] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 5: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:47.971728 11478 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:48.007246 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:48.011606 12996 raft_consensus.cc:697] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 7 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:48.007723 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:48.035465 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:47.945297 13074 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:48.048725 10916 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:48.032436 12756 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:47.973006 12994 consensus_queue.cc:1059] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.072s
I20260709 13:16:48.055948 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:48.044205 12990 raft_consensus.cc:2804] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:48.037209 12858 raft_consensus.cc:697] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 5 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
W20260709 13:16:48.063022 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 90874.3us
I20260709 13:16:48.049489 12964 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:48.064471 10800 raft_consensus.cc:2393] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 3: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:48.078219 10916 leader_election.cc:400] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 6. Current term is 7.
I20260709 13:16:48.078585 10916 leader_election.cc:403] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:16:48.078620 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:48.036681 13112 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:48.080417 10793 raft_consensus.cc:2393] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 5: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:48.081745 10800 raft_consensus.cc:2368] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 6. Current term is 7.
I20260709 13:16:48.082993 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:48.084015 10800 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
I20260709 13:16:48.085438 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:48.086964 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:48.088550 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:48.092836 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:48.093267 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:48.094053 12949 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 6. Current term is 7.
I20260709 13:16:48.062661 13096 raft_consensus.cc:2804] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:48.094249 11508 raft_consensus.cc:1275] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:16:48.094722 13096 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:48.094094 11507 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
I20260709 13:16:48.059865 13014 raft_consensus.cc:697] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 5 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:47.974984 13072 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:48.096822 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 50897.6us
I20260709 13:16:48.046880 13134 consensus_queue.cc:1059] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 13:16:48.100068 11507 raft_consensus.cc:1240] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:48.057004 13002 consensus_queue.cc:1048] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 13:16:48.095173 13096 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:48.058578 12993 consensus_queue.cc:1048] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:16:48.080528 12990 raft_consensus.cc:697] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 6 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:48.059299 13060 consensus_queue.cc:1059] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:16:48.077248 13086 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:48.095467 12912 raft_consensus.cc:2749] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:48.081549 12671 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:48.111231 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:48.101948 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:48.066210 12756 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:48.094893 12684 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:48.095991 12881 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:48.116381 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:48.117551 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:48.081993 12964 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:48.126403 11509 raft_consensus.cc:2376] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 5. Re-sending same reply.
I20260709 13:16:48.128247 13052 consensus_queue.cc:1048] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:48.153550 11512 raft_consensus.cc:1275] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:16:48.128154 13014 consensus_queue.cc:237] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:48.129349 13056 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:48.131649 12574 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:48.163007 10799 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 6.
I20260709 13:16:48.132455 13039 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:48.095703 13112 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:48.171191 11506 raft_consensus.cc:2393] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 7: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:48.133785 13071 raft_consensus.cc:3055] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:48.135129 13072 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:48.175172 13071 raft_consensus.cc:740] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:48.135221 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:48.135675 13032 consensus_queue.cc:1059] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:48.137562 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:48.139184 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:16:48.143163 13106 log.cc:927] Time spent T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.561s	user 0.003s	sys 0.000s
I20260709 13:16:48.146946 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:48.147509 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:48.063593 12798 consensus_queue.cc:260] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:48.109400 12783 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:48.152176  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:48.160544 12684 raft_consensus.cc:697] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:48.168123 13096 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:48.172962 10870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1405) took 1195 ms. Trace:
I20260709 13:16:48.117472 13134 consensus_queue.cc:1059] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 13:16:48.124442 13086 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:48.144904 12671 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:48.115278 12996 consensus_queue.cc:237] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:48.176767 12990 consensus_queue.cc:237] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:48.152336 12858 consensus_queue.cc:237] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:48.177388 13112 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:48.363830 13002 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:48.234441 12914 consensus_meta.cc:287] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.691s	user 0.012s	sys 0.000s
I20260709 13:16:48.427615 13002 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:48.428030 12914 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:48.177997 13071 consensus_queue.cc:260] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:48.429352 10794 raft_consensus.cc:2393] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 6: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:48.151925 12903 raft_consensus.cc:3055] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:48.359367 13051 raft_consensus.cc:3055] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:48.431109 12903 raft_consensus.cc:740] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:48.431218 13051 raft_consensus.cc:740] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:48.362416 12893 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:48.387460 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:48.432382 13051 consensus_queue.cc:260] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:48.433904 13051 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:48.382570 13088 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:48.434773 13088 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:48.436650 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:48.436898 10800 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 3.
I20260709 13:16:48.438027 13088 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:48.357182 13072 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:48.441167 13072 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:48.441597 10876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 1404) took 1470 ms. Trace:
I20260709 13:16:48.441682 10867 raft_consensus.cc:1240] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:48.442025 10876 rpcz_store.cc:276] 0709 13:16:46.971136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.012795 (+ 41659us) service_pool.cc:224] Handling call
0709 13:16:48.441400 (+1428605us) spinlock_profiling.cc:243] Waited 99.7 ms on lock 0x7b54002f08d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:48.441541 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":229278976}
I20260709 13:16:48.357820 12930 raft_consensus.cc:3055] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:48.442919 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 239) took 2229 ms. Trace:
I20260709 13:16:48.443229 12930 raft_consensus.cc:740] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:48.443289 10867 rpcz_store.cc:276] 0709 13:16:46.213153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.248851 (+1035698us) service_pool.cc:224] Handling call
0709 13:16:48.441512 (+1192661us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:48.442169 (+   657us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:48.442718 (+   549us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b54002f08d0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:48.442863 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2742159616}
I20260709 13:16:48.444541 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:48.387962 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:48.444717 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:48.445307 10876 raft_consensus.cc:2393] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 7: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:48.445362 10949 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 6.
I20260709 13:16:48.445642 12768 raft_consensus.cc:3055] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:48.446069 12768 raft_consensus.cc:740] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:48.446219 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:16:48.447011 13090 log.cc:927] Time spent T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 1.191s	user 0.003s	sys 0.000s
I20260709 13:16:48.447213 10876 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:48.448227 10795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 199) took 1592 ms. Trace:
I20260709 13:16:48.448589 10795 rpcz_store.cc:276] 0709 13:16:46.856067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.871185 (+ 15118us) service_pool.cc:224] Handling call
0709 13:16:46.931786 (+ 60601us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:46.981940 (+ 50154us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:46.981975 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:46.982535 (+   560us) log.cc:844] Serialized 26 byte log entry
0709 13:16:47.080071 (+ 97536us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:47.082547 (+  2476us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:47.090748 (+  8201us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:48.446401 (+1355653us) spinlock_profiling.cc:243] Waited 3.17 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:16:48.447896 (+  1495us) raft_consensus.cc:1685] finished
0709 13:16:48.447933 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:48.448166 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7287040,"thread_start_us":91135,"threads_started":1,"wal-append.queue_time_us":221141}
I20260709 13:16:48.452524 12893 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:48.453063 12893 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:48.455596 12893 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:48.456135 10873 raft_consensus.cc:1275] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:16:48.457126 12895 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:48.457185 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:48.457752 12895 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:48.458410 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:16:48.466871 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:48.468219 13080 log.cc:927] Time spent T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 1.210s	user 0.001s	sys 0.001s
I20260709 13:16:48.444227 12930 consensus_queue.cc:260] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:48.469555 12930 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:48.470250 10945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 1769) took 1779 ms. Trace:
I20260709 13:16:48.460213 12895 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:48.460108 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:48.471143 10948 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:48.470640 10945 rpcz_store.cc:276] 0709 13:16:46.691000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.691179 (+   179us) service_pool.cc:224] Handling call
0709 13:16:46.691587 (+   408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:46.702123 (+ 10536us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:46.702156 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:46.702638 (+   482us) log.cc:844] Serialized 26 byte log entry
0709 13:16:46.991236 (+288598us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:46.991275 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:46.991343 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:48.467537 (+1476194us) spinlock_profiling.cc:243] Waited 6.49 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:16:48.469950 (+  2413us) raft_consensus.cc:1685] finished
0709 13:16:48.469972 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:48.470190 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10090,"raft.queue_time_us":10968,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":141,"spinlock_wait_cycles":14937344,"thread_start_us":288282,"threads_started":1,"wal-append.queue_time_us":503908}
I20260709 13:16:48.472913 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:48.206488 10796 raft_consensus.cc:1240] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:48.460800 12843 raft_consensus.cc:3055] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:48.461561 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:48.461937 10944 raft_consensus.cc:1240] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:48.485534 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:48.462839 12756 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:48.439580 12798 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:48.463980 12903 consensus_queue.cc:260] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:48.447014 12768 consensus_queue.cc:260] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:48.440054 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 51432.7us
W20260709 13:16:48.494341 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:48.262182 12962 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 171abdebfc4142cda86de477a5c89a7e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:16:48.178146 13104 log.cc:927] Time spent T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.507s	user 0.001s	sys 0.000s
I20260709 13:16:48.470400 13074 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:48.263478 12865 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:48.255805 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 154284us
I20260709 13:16:48.438068 10800 raft_consensus.cc:1240] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:48.478340 10948 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 4.
I20260709 13:16:48.444825 13086 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:48.524166 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:48.473455 10945 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:48.427264 10942 raft_consensus.cc:1240] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:48.468108 10795 raft_consensus.cc:1275] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:16:48.525415 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:48.525997 10942 raft_consensus.cc:1240] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:48.427775 10870 rpcz_store.cc:276] 0709 13:16:46.977564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.851984 (+874420us) service_pool.cc:224] Handling call
0709 13:16:47.852234 (+   250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:47.852528 (+   294us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:16:47.859117 (+  6589us) log.cc:844] Serialized 14 byte log entry
0709 13:16:48.172566 (+313449us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:48.172622 (+    56us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:48.172670 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:48.172736 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:48.172887 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2923,"prepare.queue_time_us":6150,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":26033,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":138,"thread_start_us":310099,"threads_started":2,"wal-append.queue_time_us":181070}
I20260709 13:16:48.492014 10876 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 6.
I20260709 13:16:48.480352 12843 raft_consensus.cc:740] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:48.497022 11508 raft_consensus.cc:1275] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:16:48.502336 13096 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:48.499434 10766 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:48.492311 12783 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:48.501709 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:48.535931 10944 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 3.
I20260709 13:16:48.493674 11509 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 2 LEARNER]: Advancing to term 4
I20260709 13:16:48.152976 12881 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:48.515515 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:48.518131 10797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1402) took 1140 ms. Trace:
I20260709 13:16:48.313923 13060 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:48.264155 12574 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:48.527333 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:48.468073 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:48.481468 13002 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:48.554960 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:48.527330 10942 raft_consensus.cc:1275] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 2. (index mismatch)
I20260709 13:16:48.529549 11477 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:16:48.533527 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:48.501325 13071 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:48.533686 11508 raft_consensus.cc:1275] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 13:16:48.498175 11511 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:48.565464 11508 pending_rounds.cc:85] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586: Aborting all ops after (but not including) 4
I20260709 13:16:48.548775 10841 leader_election.cc:400] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 election: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 3. Current term is 4.
I20260709 13:16:48.544773 10945 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 7.
I20260709 13:16:48.539781 10797 rpcz_store.cc:276] 0709 13:16:47.377568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.377688 (+   120us) service_pool.cc:224] Handling call
0709 13:16:47.409779 (+ 32091us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:47.409975 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:16:47.439624 (+ 29649us) log.cc:844] Serialized 14 byte log entry
0709 13:16:48.440360 (+1000736us) spinlock_profiling.cc:243] Waited 3.65 ms on lock 0x7b18003d7560. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c80a5d4bc 00007f9c7ddab48b 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc5455 00007f9c80ead060 00007f9c80ebf716 00007f9c80ed0287
0709 13:16:48.517357 (+ 76997us) log.cc:844] Serialized 14 byte log entry
0709 13:16:48.517638 (+   281us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:48.517719 (+    81us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:48.517791 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:48.517893 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:48.518056 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":29945,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":103985,"raft.run_cpu_time_us":288,"raft.run_wall_time_us":19057,"spinlock_wait_cycles":8406272,"thread_start_us":293947,"threads_started":3,"wal-append.queue_time_us":225043}
I20260709 13:16:48.565969 10841 leader_election.cc:403] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:16:48.557160 10942 raft_consensus.cc:1240] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:48.565826 11508 pending_rounds.cc:107] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:16:48.567003 10841 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:48.567063 10945 raft_consensus.cc:1275] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:16:48.567916 10841 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:48.436669 12684 consensus_queue.cc:237] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:48.537077 12768 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:48.535118 12903 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:48.549738 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:48.570807 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:48.566609 11508 raft_consensus.cc:2889] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:48.597431 10870 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 5.
I20260709 13:16:48.539088 12756 raft_consensus.cc:2804] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:48.594660 10944 raft_consensus.cc:1275] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 13:16:48.595429 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 65959.8us
I20260709 13:16:48.599247 11511 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 4.
I20260709 13:16:48.593160 10841 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:48.599313 11477 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:48.527557 12865 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:48.535147 13086 consensus_queue.cc:1059] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 13:16:48.603497 10797 raft_consensus.cc:1240] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 5. Current term is 6. Ops: [3.6-5.7]
I20260709 13:16:48.631635 11508 raft_consensus.cc:1240] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 4. Current term is 5. Ops: []
W20260709 13:16:48.592844 13111 log.cc:927] Time spent T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
W20260709 13:16:48.592257 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:48.604086 10841 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:48.666499 11507 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 4.
I20260709 13:16:48.546515 12783 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:48.598600 13134 raft_consensus.cc:2749] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:16:48.674996 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 71020.7us
I20260709 13:16:48.593357 13031 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:48.593604 13124 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:48.656654 12990 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:48.599006 12756 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:48.557363 13060 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:48.592039 13074 consensus_queue.cc:1059] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.081s
I20260709 13:16:48.658842 13136 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:48.688213 10841 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:48.666828 13039 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:48.658303 13088 raft_consensus.cc:2804] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:48.600420 12838 consensus_queue.cc:1048] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.083s
I20260709 13:16:48.660569 12961 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:48.596458 13117 consensus_queue.cc:1059] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.083s
I20260709 13:16:48.667120 13143 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:48.602623 13096 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:48.681355 13052 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:48.669298 13112 consensus_queue.cc:1059] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 13:16:48.687037 12756 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:48.600754 12843 consensus_queue.cc:260] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:48.696511 13046 raft_consensus.cc:2749] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:16:48.691104 13031 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:48.704124 12983 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:48.692505 12990 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:48.708668 13088 raft_consensus.cc:697] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 4 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:48.698325 12904 consensus_queue.cc:1048] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
W20260709 13:16:48.729771 13139 log.cc:927] Time spent T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.160s	user 0.000s	sys 0.001s
I20260709 13:16:48.696651 13142 consensus_queue.cc:1059] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.045s
W20260709 13:16:48.743288 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 47637.2us
I20260709 13:16:48.721318 13052 raft_consensus.cc:697] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:48.727492 13134 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:48.730691 13031 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:48.737154 12983 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:48.700978 13039 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:48.703850 12961 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:48.712906 13096 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:48.754247 13152 consensus_queue.cc:1048] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:16:48.755028 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 155785us
I20260709 13:16:48.758019 13117 consensus_queue.cc:1243] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer cb0428cc55984daba1ba63744a5102a2 log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:16:48.706888 13143 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:48.750708 12964 consensus_meta.cc:287] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.593s	user 0.010s	sys 0.000s
I20260709 13:16:48.762387 11477 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:48.745086 12671 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:48.747390 12843 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:48.774618 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:48.775522 10949 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 5.
I20260709 13:16:48.758354 13136 raft_consensus.cc:697] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:48.759168 12983 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:48.759403 13082 raft_consensus.cc:3055] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:16:48.799342 13145 log.cc:927] Time spent T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 13:16:48.771436 12881 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:48.801463 10870 raft_consensus.cc:1240] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:48.776177 12574 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:48.803165 12574 raft_consensus.cc:697] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 7 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:48.803258 13060 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:48.776968 13140 consensus_queue.cc:1048] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:16:48.759785 13134 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:48.771950 12964 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:48.811941 10867 raft_consensus.cc:2393] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 5: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:48.815091 10870 raft_consensus.cc:1275] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:16:48.777710 11477 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:48.763233 12838 raft_consensus.cc:3055] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:48.763741 12865 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:48.820232 10873 raft_consensus.cc:2410] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 13:16:48.828230 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 50654.4us
I20260709 13:16:48.794693 13136 consensus_queue.cc:237] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:48.951543 12838 raft_consensus.cc:740] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:48.804080 12574 consensus_queue.cc:237] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:48.838343 13141 log.cc:927] Time spent T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.238s	user 0.001s	sys 0.000s
I20260709 13:16:48.952330 12990 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:48.793442 13153 consensus_queue.cc:1059] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.165s
I20260709 13:16:48.789407 13082 raft_consensus.cc:740] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:48.953277 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:48.953684 13147 raft_consensus.cc:3055] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:48.885268 13086 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:48.932490 13124 raft_consensus.cc:2749] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 3. Current term is 4.
W20260709 13:16:48.958289 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:48.871398 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:48.960651 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:48.983120 10799 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 1412) took 1458 ms (client timeout 1646 ms). Trace:
W20260709 13:16:49.014428 10799 rpcz_store.cc:269] 0709 13:16:47.524850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.534033 (+  9183us) service_pool.cc:224] Handling call
0709 13:16:48.982758 (+1448725us) spinlock_profiling.cc:243] Waited 626 ms on lock 0x7b54000bccd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:48.983041 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1439269632}
W20260709 13:16:48.979869 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:48.812853 12964 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:48.808104 13096 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:48.814459 11509 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 4 LEARNER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 4.
I20260709 13:16:48.810588 13134 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:49.029485 10876 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 4.
I20260709 13:16:48.980129 12783 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:48.983918 13147 raft_consensus.cc:740] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:49.061210 13147 consensus_queue.cc:260] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:49.062316 13147 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:49.010689 13086 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:49.017386 11504 raft_consensus.cc:1240] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:48.823249 13088 consensus_queue.cc:237] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:49.032184 13142 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:48.895803 13016 raft_consensus.cc:3055] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:49.039266 13082 consensus_queue.cc:260] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:49.082466 10874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1367) took 3228 ms. Trace:
I20260709 13:16:49.082921 10874 rpcz_store.cc:276] 0709 13:16:45.855844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.132107 (+1276263us) service_pool.cc:224] Handling call
0709 13:16:47.295016 (+162909us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:47.313575 (+ 18559us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:49.082212 (+1768637us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b540006b3d0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:49.082395 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":373893504}
I20260709 13:16:49.042870 10949 raft_consensus.cc:1240] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:48.989356 12983 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:49.047371 10872 raft_consensus.cc:1240] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:49.018469 12838 consensus_queue.cc:260] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:49.049075 13122 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:49.050765 13039 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:49.051069 12961 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:49.052426 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:49.091526 10943 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 5.
W20260709 13:16:49.053686 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:49.044904 12798 consensus_meta.cc:287] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.546s	user 0.010s	sys 0.000s
I20260709 13:16:48.823109 13052 consensus_queue.cc:237] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:49.065306 13086 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:16:49.108721 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 334782us
I20260709 13:16:49.093406 11477 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:49.083899 13016 raft_consensus.cc:740] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:16:49.116246 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.117153 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:16:49.116706 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.118845 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:49.136258 10949 raft_consensus.cc:2393] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 5: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:49.114058 12838 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:49.121002 12574 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:49.139186 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:49.089040 13142 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:49.140775 10949 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 4.
I20260709 13:16:49.094650 13122 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:49.134325 12756 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:49.144409 13154 log.cc:927] Time spent T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
I20260709 13:16:49.127444 13082 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:49.154088 11477 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:49.160544 13028 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:49.180939 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:49.138358 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:49.140429 12574 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:49.139393 13143 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:49.186348 13159 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:49.189456 13159 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:49.189898 13159 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:49.197312 11505 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7.
I20260709 13:16:49.198203 12756 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:49.211938 12983 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:49.214092 12912 raft_consensus.cc:1064] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2: attempting to promote NON_VOTER 12682eb70bc84c76879e04a8628ee586 to VOTER
I20260709 13:16:49.214928 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:49.215621 10946 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:49.161476 13163 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:49.212056 10794 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 1409) took 1688 ms (client timeout 2129 ms). Trace:
W20260709 13:16:49.221565 10794 rpcz_store.cc:269] 0709 13:16:47.523954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.529183 (+  5229us) service_pool.cc:224] Handling call
0709 13:16:49.211836 (+1682653us) spinlock_profiling.cc:243] Waited 899 ms on lock 0x7b540039f8d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:49.211988 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2067040512}
W20260709 13:16:49.188167 12903 consensus_meta.cc:287] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.586s	user 0.011s	sys 0.004s
W20260709 13:16:49.219676 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.228524 10842 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:49.195657 12574 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:49.229547 12684 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:49.246692 12962 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:56416 (request call id 443) took 5950 ms. Trace:
I20260709 13:16:49.249848 10842 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:49.253232 10946 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 4.
I20260709 13:16:49.254588 11477 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:49.230872 12768 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:49.221132 13049 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:49.179708 13016 consensus_queue.cc:260] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:49.258729 10842 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:16:49.253608 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.244748 13032 raft_consensus.cc:2749] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:49.229540 13134 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:16:49.262277 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 43833.7us
I20260709 13:16:49.225929 13028 consensus_queue.cc:1059] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
W20260709 13:16:49.266660 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.253902 12684 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:49.276132 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.272513 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:49.280112 10946 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
I20260709 13:16:49.260396 13143 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:49.243363 13163 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:49.285934 10946 raft_consensus.cc:1240] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:49.268106 13136 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:49.292011 10841 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:49.288084 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:49.288780 10874 raft_consensus.cc:1240] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:16:49.293527 10872 raft_consensus.cc:2393] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 3: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:16:49.282407 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.271879 13101 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:49.272217 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:49.270370 12684 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:49.247386 12962 rpcz_store.cc:276] 0709 13:16:43.295835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:43.298595 (+  2760us) service_pool.cc:224] Handling call
0709 13:16:43.724699 (+426104us) ts_tablet_manager.cc:934] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586: Initiating tablet copy from peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685)
0709 13:16:48.009465 (+4284766us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:16:49.049596 (+1040131us) tablet_metadata.cc:665] Metadata flushed
0709 13:16:49.049752 (+   156us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:16:49.245568 (+195816us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:16:49.246025 (+   457us) tablet_replica.cc:206] Creating consensus instance
0709 13:16:49.246471 (+   446us) ts_tablet_manager.cc:989] Replica registered.
0709 13:16:49.246566 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":25399,"tablet-copy.queue_time_us":332835,"thread_start_us":419531,"threads_started":2}
I20260709 13:16:49.296633 13152 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:49.302464 12962 tablet_copy_client.cc:806] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:49.302336 13152 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:49.303236 12962 tablet_copy_client.cc:670] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:49.303381 13100 raft_consensus.cc:3055] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:49.303880 13100 raft_consensus.cc:740] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:49.304790 13100 consensus_queue.cc:260] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:49.294981 13157 consensus_queue.cc:1059] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 13:16:49.307022 13122 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:49.307286 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:49.307765 13028 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:49.308193 13028 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:49.308499 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:49.308708 10868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1368) took 3450 ms. Trace:
I20260709 13:16:49.309276 10942 raft_consensus.cc:1275] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:16:49.300719 13101 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:49.297341 13143 raft_consensus.cc:697] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 4 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:49.284318 13016 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:49.306099 13100 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:49.310761 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:16:49.320467 13165 log.cc:927] Time spent T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 13:16:49.275117 12912 consensus_queue.cc:237] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:49.313378 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:49.247020 13159 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:49.309078 10868 rpcz_store.cc:276] 0709 13:16:45.860067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.135145 (+1275078us) service_pool.cc:224] Handling call
0709 13:16:47.135421 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:47.480706 (+345285us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:49.308443 (+1827737us) spinlock_profiling.cc:243] Waited 345 ms on lock 0x7b540037b254. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:49.308622 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":792425856}
I20260709 13:16:49.331626 10942 raft_consensus.cc:2410] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:16:49.332893 10874 raft_consensus.cc:2393] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 5: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:49.314446 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:49.348959 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:49.260407 12768 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:49.349439 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:49.354310 10949 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
I20260709 13:16:49.314344 10944 raft_consensus.cc:1240] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:49.314582 10950 raft_consensus.cc:1240] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:49.263679 13049 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:49.356176 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:49.360968 10950 raft_consensus.cc:1240] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:16:49.352373 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:49.362483 10872 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 5.
I20260709 13:16:49.353061 10876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49290 (request call id 250) took 1281 ms. Trace:
I20260709 13:16:49.363333 10876 rpcz_store.cc:276] 0709 13:16:48.071842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:48.570480 (+498638us) service_pool.cc:224] Handling call
0709 13:16:49.352865 (+782385us) spinlock_profiling.cc:243] Waited 423 ms on lock 0x7b54001095d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:49.352997 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":973891456}
W20260709 13:16:49.367455 10793 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 1807) took 1801 ms (client timeout 2318 ms). Trace:
I20260709 13:16:49.367599 10800 raft_consensus.cc:1240] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 4. Current term is 5. Ops: []
W20260709 13:16:49.368005 10793 rpcz_store.cc:269] 0709 13:16:47.566310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.657296 (+ 90986us) service_pool.cc:224] Handling call
0709 13:16:49.367210 (+1709914us) spinlock_profiling.cc:243] Waited 397 ms on lock 0x7b5400211bd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:49.367389 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":913213184}
I20260709 13:16:49.267865 13169 raft_consensus.cc:2749] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:16:49.355302 10871 raft_consensus.cc:1240] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:49.355688 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:49.308517 13028 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:49.328214 12994 raft_consensus.cc:3055] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:49.374402 10798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 1799) took 1832 ms. Trace:
I20260709 13:16:49.311841 13101 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:49.379930 13049 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:49.380223 12768 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:49.313017 13163 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:49.382150 10870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1423) took 1146 ms. Trace:
I20260709 13:16:49.373986 12994 raft_consensus.cc:740] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:16:49.383363 13166 log.cc:927] Time spent T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 13:16:49.383971 12994 consensus_queue.cc:260] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:49.385139 12994 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:49.313638 13143 consensus_queue.cc:237] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:49.374606 13153 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:49.388170 13153 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:49.389551 13101 consensus_queue.cc:1059] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:16:49.394981 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:49.396476 13101 consensus_queue.cc:1059] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:49.397119 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:49.397256 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:49.397261 13101 consensus_queue.cc:1243] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer cb0428cc55984daba1ba63744a5102a2 log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:16:49.398087 11511 raft_consensus.cc:2393] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 7: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:49.399101 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:49.400635 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:49.401201 11511 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:16:49.403296 13031 consensus_meta.cc:287] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.639s	user 0.009s	sys 0.008s
I20260709 13:16:49.403852 13031 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:49.406230 13031 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:49.350739 13100 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:49.407224 13100 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:49.409694 13100 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:49.359990 13073 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:49.411232 13073 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:49.414077 13073 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:49.397421 12838 raft_consensus.cc:3055] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:49.277487 13134 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:49.376641 10798 rpcz_store.cc:276] 0709 13:16:47.541756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.885572 (+343816us) service_pool.cc:224] Handling call
0709 13:16:48.983882 (+1098310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:48.984258 (+   376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:48.984285 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:48.984710 (+   425us) log.cc:844] Serialized 26 byte log entry
0709 13:16:49.110313 (+125603us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:49.110358 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:49.110432 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:49.273228 (+162796us) raft_consensus.cc:1685] finished
0709 13:16:49.273256 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:49.361813 (+ 88557us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54000bccd0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:49.374292 (+ 12479us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":652,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":6833,"spinlock_wait_cycles":2431727872,"thread_start_us":77564,"threads_started":1,"wal-append.queue_time_us":160681}
W20260709 13:16:49.428881 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.415310 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:49.415797 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:16:49.416350 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.355330 10841 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:16:49.431583 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 76324.1us
I20260709 13:16:49.382570 10870 rpcz_store.cc:276] 0709 13:16:48.235993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:48.925479 (+689486us) service_pool.cc:224] Handling call
0709 13:16:48.984410 (+ 58931us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:49.013826 (+ 29416us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:49.013869 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:49.014437 (+   568us) log.cc:844] Serialized 52 byte log entry
0709 13:16:49.381493 (+367056us) spinlock_profiling.cc:243] Waited 26.5 ms on lock 0x7b18003c6400. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c80a5d4bc 00007f9c7ddab48b 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a550d1
0709 13:16:49.381716 (+   223us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:49.381762 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:49.381834 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:49.381875 (+    41us) raft_consensus.cc:1685] finished
0709 13:16:49.381885 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:49.382077 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29044,"raft.queue_time_us":32580,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":1014,"spinlock_wait_cycles":62873728,"thread_start_us":115521,"threads_started":1,"wal-append.queue_time_us":115250}
I20260709 13:16:49.401212 13153 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:49.398396 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:49.434474 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:49.435938 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:49.437109 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:49.437783 10945 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
W20260709 13:16:49.438828 10766 leader_election.cc:336] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Timed out: RequestConsensusVote RPC to 127.9.116.132:40555 timed out after 2.221s (SENT)
W20260709 13:16:49.420066 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.439653 10841 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:49.442116 10942 raft_consensus.cc:2393] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 7: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:49.444628 11478 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:49.374102 13122 consensus_queue.cc:1048] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.076s
I20260709 13:16:49.449858 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:49.452775 10944 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
I20260709 13:16:49.453919 11478 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:49.376852 12903 consensus_queue.cc:1059] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 13:16:49.419538 12838 raft_consensus.cc:740] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:49.456498 11478 leader_election.cc:400] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for earlier term 4. Current term is 5.
I20260709 13:16:49.449168 11511 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 7.
I20260709 13:16:49.457048 13088 raft_consensus.cc:2749] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:49.440676 13046 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:49.468883 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:49.453099 13169 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:49.451077 13142 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:49.477161 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:49.482928 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:49.487407 10871 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 6.
I20260709 13:16:49.485015 10915 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:49.461552 13024 raft_consensus.cc:2749] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:49.492321 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:49.498304 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:16:49.515180 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:49.528044 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.547574 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:49.548547 10870 raft_consensus.cc:2393] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 7: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:16:49.549381 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 64511.1us
I20260709 13:16:49.550523 10766 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:16:49.552325 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.467525 13046 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:49.560377 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:49.476799 13151 raft_consensus.cc:3055] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:49.475957 13169 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:49.565313 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:49.566720 10767 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:49.566500 10841 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:49.564796 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:49.575591 10872 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 4.
I20260709 13:16:49.579439 10948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 240) took 1116 ms. Trace:
I20260709 13:16:49.579988 10948 rpcz_store.cc:276] 0709 13:16:48.464331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:48.528044 (+ 63713us) service_pool.cc:224] Handling call
0709 13:16:48.531365 (+  3321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:48.699710 (+168345us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:48.699750 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:48.699790 (+    40us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:48.699819 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:49.579096 (+879277us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b5400015ed4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:49.579157 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:49.579366 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":386641280}
W20260709 13:16:49.554826 13074 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.847s	user 0.018s	sys 0.020s
I20260709 13:16:49.471364 13133 consensus_queue.cc:1059] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.096s
I20260709 13:16:49.556586 13100 raft_consensus.cc:2749] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:16:49.584354 11508 pending_rounds.cc:85] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586: Aborting all ops after (but not including) 6
I20260709 13:16:49.476627 12798 consensus_queue.cc:1059] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.095s
W20260709 13:16:49.568177 13174 log.cc:927] Time spent T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 13:16:49.561390 13055 raft_consensus.cc:2749] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:16:49.563046 13014 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.299s	user 0.034s	sys 0.083s
I20260709 13:16:49.562013 12994 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:49.490710 13157 consensus_queue.cc:1059] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.099s
I20260709 13:16:49.579648 13049 raft_consensus.cc:2749] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:16:49.600744 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.567402 13046 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:49.568068 12756 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:49.590694 11508 pending_rounds.cc:107] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:16:49.567965 13151 raft_consensus.cc:740] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
W20260709 13:16:49.596887 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.573086 13039 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:49.572919 13101 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:16:49.586851 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 255874us
I20260709 13:16:49.578121 13169 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:16:49.585839 13074 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.881s	user 0.019s	sys 0.020s
I20260709 13:16:49.603395 11508 raft_consensus.cc:2889] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:49.565644 12996 consensus_queue.cc:1048] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
W20260709 13:16:49.598506 13014 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.334s	user 0.034s	sys 0.083s
I20260709 13:16:49.600586 13159 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:49.612700 11510 raft_consensus.cc:1275] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:16:49.610070 13101 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:49.601334 13100 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:49.610294 13039 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
W20260709 13:16:49.619585 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.634024 11510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56416 (request call id 470) took 1509 ms. Trace:
I20260709 13:16:49.569132 12838 consensus_queue.cc:260] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:49.638863 11510 rpcz_store.cc:276] 0709 13:16:48.124384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:49.612045 (+1487661us) service_pool.cc:224] Handling call
0709 13:16:49.612379 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:49.632949 (+ 20570us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:49.633939 (+   990us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:49.643905 10916 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:49.600201 12994 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:49.633903 13086 consensus_meta.cc:287] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.511s	user 0.011s	sys 0.000s
I20260709 13:16:49.632308 13074 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:16:49.692575 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 132725us
I20260709 13:16:49.620980 13073 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:49.608876 12756 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:49.701407 11512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56416 (request call id 469) took 1581 ms. Trace:
W20260709 13:16:49.701412 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 134773us
I20260709 13:16:49.642015 13014 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:49.713660 11512 rpcz_store.cc:276] 0709 13:16:48.119648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:48.120183 (+   535us) service_pool.cc:224] Handling call
0709 13:16:48.153112 (+ 32929us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:48.154027 (+   915us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:49.701147 (+1547120us) spinlock_profiling.cc:243] Waited 25.8 ms on lock 0x7b54001d06d0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:49.701330 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":59308928}
I20260709 13:16:49.631573 12903 consensus_queue.cc:1048] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.074s
I20260709 13:16:49.702255 10915 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:49.714574 10767 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:49.694830 13074 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
W20260709 13:16:49.732017 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.641579 13100 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:49.738646 10767 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:16:49.744314 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 306318us
I20260709 13:16:49.694717 13086 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:49.735069 12996 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:49.729521 13169 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:49.750681 12994 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:49.751623 13186 raft_consensus.cc:2749] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:49.733567 13074 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:16:49.753342 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.StartTabletCopy -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 200170us
I20260709 13:16:49.723049 13073 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:49.691243 13151 consensus_queue.cc:260] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:49.736308 13176 raft_consensus.cc:2749] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:16:49.758903 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.723377 13014 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:49.771878 12756 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:49.773268 13147 consensus_meta.cc:287] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.711s	user 0.006s	sys 0.007s
I20260709 13:16:49.773567 12994 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:49.773605 13014 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:49.777678 12994 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:49.779492 13039 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:49.779858 12994 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:49.781551 13169 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:49.760537 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 116728us
I20260709 13:16:49.763296 12783 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:49.763574 13186 consensus_queue.cc:1048] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:16:49.763944 13179 log.cc:927] Time spent T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 13:16:49.765019 13086 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:49.765530 11509 raft_consensus.cc:1240] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 13:16:49.767172 12914 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:49.798882 13028 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:49.767298 13072 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:49.767747 11503 raft_consensus.cc:2368] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 5. Current term is 6.
I20260709 13:16:49.802569 12756 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:49.803557 10868 raft_consensus.cc:2393] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 6: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:16:49.769284 13177 log.cc:927] Time spent T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
I20260709 13:16:49.804461 13028 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:49.804520 13046 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:49.806636 10794 raft_consensus.cc:1240] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 4. Current term is 5. Ops: []
W20260709 13:16:49.769107 12574 consensus_meta.cc:287] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.516s	user 0.008s	sys 0.000s
I20260709 13:16:49.784687 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1424) took 1345 ms. Trace:
I20260709 13:16:49.788038 13039 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:49.791471 11506 raft_consensus.cc:2410] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:16:49.791810 12783 raft_consensus.cc:697] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 7 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:49.818048 13169 consensus_queue.cc:1059] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:49.818460 12783 consensus_queue.cc:237] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:49.819483 13052 consensus_queue.cc:1059] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:16:49.793363 13074 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:49.794486 13101 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:49.795835 13034 raft_consensus.cc:3055] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:49.754813 13105 consensus_queue.cc:1048] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:16:49.796941 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:49.830550 10801 raft_consensus.cc:2393] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 3: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:49.796888 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:49.803192 12756 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:49.833016 10872 raft_consensus.cc:1240] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:49.835494 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:49.837841 10875 raft_consensus.cc:2393] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 5: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:49.786347 10916 leader_election.cc:400] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 6. Current term is 7.
I20260709 13:16:49.813813 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:49.838600 10946 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:16:49.838732 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 52593.3us
I20260709 13:16:49.840067 10916 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:49.809098 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:49.840864 10794 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
I20260709 13:16:49.841809 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:49.782689 12996 raft_consensus.cc:2749] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:49.815722 10799 raft_consensus.cc:1275] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:16:49.844174 10799 pending_rounds.cc:85] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 3
I20260709 13:16:49.844534 10799 pending_rounds.cc:107] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:16:49.844831 10799 raft_consensus.cc:2889] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:49.816977 10793 raft_consensus.cc:1240] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:16:49.817158 10796 raft_consensus.cc:1240] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 LEADER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 5. Current term is 6. Ops: [3.2-5.3]
I20260709 13:16:49.812760 10867 rpcz_store.cc:276] 0709 13:16:48.439478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:48.953477 (+513999us) service_pool.cc:224] Handling call
0709 13:16:48.953808 (+   331us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:48.954185 (+   377us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:48.954216 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:49.311933 (+357717us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b2c00029f28. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7f7aea11 00007f9c7f7be372 00007f9c80ef0a01 00007f9c80eb543a 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2
0709 13:16:49.312443 (+   510us) log.cc:844] Serialized 52 byte log entry
0709 13:16:49.586410 (+273967us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:49.586466 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:49.586565 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:49.784371 (+197806us) raft_consensus.cc:1685] finished
0709 13:16:49.784395 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:49.784620 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4064,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":9214,"spinlock_wait_cycles":30035840,"thread_start_us":273502,"threads_started":1,"wal-append.queue_time_us":427767}
I20260709 13:16:49.846379 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:49.848109 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:16:49.822324 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.848775 10799 raft_consensus.cc:2410] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:16:49.823269 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:49.849673 10948 raft_consensus.cc:2410] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 5 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 13:16:49.823945 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:49.828384 10944 raft_consensus.cc:1240] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 4. Current term is 5. Ops: [3.6-4.7]
I20260709 13:16:49.828926 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:49.829322 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:49.829980 13034 raft_consensus.cc:740] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:49.832854 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:49.822909 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:49.857478 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:49.858388 10798 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
I20260709 13:16:49.823450 13039 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:49.861660 10799 raft_consensus.cc:1275] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:16:49.862047 10799 pending_rounds.cc:85] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 1
I20260709 13:16:49.853447 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:49.862360 10799 pending_rounds.cc:107] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:16:49.862604 10799 raft_consensus.cc:2889] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:49.864619 10946 raft_consensus.cc:2410] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 6 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:16:49.853672 10916 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:49.855161 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:49.866492 10801 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
W20260709 13:16:49.870004 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 125738us
I20260709 13:16:49.821856 12574 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:49.857479 12994 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:49.827168 13016 consensus_queue.cc:1048] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 13:16:49.879735 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:49.880635 10872 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 6.
I20260709 13:16:49.879910 10916 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:49.883531 10916 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:49.884801 12990 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:49.886409 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:49.887562 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:49.888242 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:49.888505 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:49.889057 10794 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 5.
I20260709 13:16:49.889635 10799 raft_consensus.cc:1275] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:49.890100 10794 raft_consensus.cc:1275] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:16:49.888263 10797 raft_consensus.cc:2393] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 5: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:49.892567 10916 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:49.893465 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:49.895459 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:49.875790 12996 raft_consensus.cc:2764] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:16:49.897591 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:49.898216 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:49.873260 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:49.898381 10799 raft_consensus.cc:2393] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 5: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:49.899317 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:49.900843 13074 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:49.901273 11504 raft_consensus.cc:2393] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 6: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:49.893036 13184 raft_consensus.cc:2749] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:16:49.903445 12756 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:49.903878 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:49.904497 10795 raft_consensus.cc:2410] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:16:49.906360 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:49.907996 10801 raft_consensus.cc:1275] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:16:49.908977 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:49.909476 13142 raft_consensus.cc:3055] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:49.909793 11508 raft_consensus.cc:2393] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 6: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:49.909909 13142 raft_consensus.cc:740] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:49.910979 13142 consensus_queue.cc:260] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:49.912017 13142 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:49.912446 13024 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:49.913084 13024 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:49.914566 13123 raft_consensus.cc:3055] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:49.915028 13123 raft_consensus.cc:740] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:49.908066 12574 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:49.872417 10767 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:16:49.916671 10766 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.116.132:40555 timed out after 1.553s (SENT) [suppressed 34 similar messages]
I20260709 13:16:49.916904 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:49.923528 10872 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 5.
I20260709 13:16:49.923689 10948 raft_consensus.cc:1240] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:16:49.916918 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:49.917886 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:49.918879 10916 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:49.919849 13123 consensus_queue.cc:260] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:49.915541 13046 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:49.924489 11508 raft_consensus.cc:2376] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 4. Re-sending same reply.
W20260709 13:16:49.926813 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 54446.3us
I20260709 13:16:49.934767 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:49.935894 10872 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:49.937747 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:49.944988 11478 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:16:49.947419 11478 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
W20260709 13:16:49.947768 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.943790 10915 leader_election.cc:400] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 5. Current term is 6.
I20260709 13:16:49.949965 10915 leader_election.cc:403] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:16:49.950549 10916 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:49.953955 10872 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 6.
I20260709 13:16:49.955019 13105 consensus_queue.cc:1059] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:49.956346 10766 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:49.957546 10766 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:49.955111 12756 consensus_queue.cc:1048] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:16:49.925874 13101 raft_consensus.cc:2749] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:49.958982 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:49.960259 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:49.960440 10916 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:49.925879 13192 raft_consensus.cc:2749] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:49.962106 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:49.962656 10946 raft_consensus.cc:1240] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 7 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:16:49.975097 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:49.957382 13105 consensus_queue.cc:1048] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:49.963835 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:49.958138 13185 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:49.961300 12993 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:49.879308 13034 consensus_queue.cc:260] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:49.964190 13039 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:49.964643 12996 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:49.961513 13184 raft_consensus.cc:2749] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:49.965438 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:49.994751 10801 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 3.
I20260709 13:16:49.968619 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:49.968950 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:49.973778 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:49.976343 10793 raft_consensus.cc:1240] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:49.976790 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:49.996894 10872 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 5.
I20260709 13:16:49.962023 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:49.998190 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:50.003521 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:50.004185 10872 raft_consensus.cc:2393] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 6: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:49.931059 12990 consensus_queue.cc:1048] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 13:16:49.978329 11510 raft_consensus.cc:2393] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 5: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:50.005121 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:50.005909 10872 raft_consensus.cc:2393] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 6: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:50.006115 13024 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:49.951524 13122 consensus_queue.cc:1059] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 13:16:49.975371 12912 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:50.007661 12962 tablet_copy_client.cc:538] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:50.007735 12912 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:50.008347 12912 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:50.009143 13185 raft_consensus.cc:2749] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:49.974202 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:50.013031 13151 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:50.014881 10795 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:50.016353 11510 raft_consensus.cc:1240] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 5 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:50.016582 13034 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:50.016721 10766 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:50.017114 10876 raft_consensus.cc:1275] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:16:50.018394 10876 raft_consensus.cc:1275] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:16:49.971571 13074 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:49.983363 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:49.959255 13086 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:16:49.999765 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:50.003094 12996 raft_consensus.cc:697] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 6 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:50.004766 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:50.042196 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:49.959070 13159 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:50.021984 11478 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:50.043054 10801 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 4.
I20260709 13:16:50.006631 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:50.044639 11478 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:50.045595 10872 raft_consensus.cc:2393] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 6: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:50.047240 10872 raft_consensus.cc:1240] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:16:50.034574 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:50.052986 12996 consensus_queue.cc:237] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:50.054934 13147 raft_consensus.cc:2804] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:49.978523 13101 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:50.055579 13147 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:50.059902 13024 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:50.061879 12798 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:50.062364 13157 consensus_queue.cc:1048] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:50.018922 12783 consensus_queue.cc:1048] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 13:16:50.063738 12983 raft_consensus.cc:3055] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:49.979264 13055 consensus_queue.cc:1059] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.074s
I20260709 13:16:50.032217 13105 raft_consensus.cc:2749] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:50.040086 13086 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:50.070418 13133 consensus_queue.cc:1048] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 13:16:50.073769 13055 raft_consensus.cc:3055] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:50.073998 13086 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:50.074304 13055 raft_consensus.cc:740] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:49.997515 12574 consensus_queue.cc:1059] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.084s
I20260709 13:16:50.049952 13159 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:50.050341 13074 raft_consensus.cc:697] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:50.072094 12983 raft_consensus.cc:740] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:50.068784 13147 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:50.072028 13088 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:50.076233 13159 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:50.071961 13024 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:50.090444 12574 raft_consensus.cc:3055] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:50.090905 12574 raft_consensus.cc:740] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:50.091768 12994 raft_consensus.cc:2749] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:50.091959 13024 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:50.091706 12574 consensus_queue.cc:260] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:50.092941 12574 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:50.069363 13101 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:50.155506 13112 raft_consensus.cc:3055] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:50.156060 13112 raft_consensus.cc:740] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:50.157037 13112 consensus_queue.cc:260] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:50.155169 12983 consensus_queue.cc:260] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:50.158275 13112 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:50.099377 13055 consensus_queue.cc:260] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:50.171653 13055 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:50.172329 13101 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:50.174480 13074 consensus_queue.cc:237] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:50.088693 13088 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:50.180692 13147 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:50.184491 13147 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:50.186702 13159 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:50.188848 13159 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:50.227627 10794 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:50.229471 10766 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:50.023015 10796 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 6.
W20260709 13:16:50.243398 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 268852us
I20260709 13:16:50.246114 10916 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:50.247484 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:50.250407 10916 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:50.252439 10916 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:50.262588 11503 raft_consensus.cc:1240] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:16:50.265202 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:50.266645 11504 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 6.
W20260709 13:16:50.271413 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:50.280881 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:50.285455 10792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1398) took 2946 ms. Trace:
I20260709 13:16:50.165766 12962 tablet_bootstrap.cc:492] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586: Bootstrap starting.
I20260709 13:16:50.289860 10800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1423) took 2356 ms. Trace:
I20260709 13:16:50.290305 10800 rpcz_store.cc:276] 0709 13:16:47.933484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:48.523908 (+590424us) service_pool.cc:224] Handling call
0709 13:16:49.367384 (+843476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:49.368231 (+   847us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:50.289468 (+921237us) spinlock_profiling.cc:243] Waited 843 ms on lock 0x7b5400211bd0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:50.289684 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1938892160}
I20260709 13:16:50.293665 12983 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:50.293743 12994 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:50.294836 12994 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:50.297850 12994 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:50.299993 12996 raft_consensus.cc:2749] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:16:50.305531 13016 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:50.309299 13055 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:50.309448 12574 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:50.286015 10792 rpcz_store.cc:276] 0709 13:16:47.338633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.339254 (+   621us) service_pool.cc:224] Handling call
0709 13:16:47.339603 (+   349us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:47.431220 (+ 91617us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:47.431259 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:47.438277 (+  7018us) log.cc:844] Serialized 52 byte log entry
0709 13:16:49.121274 (+1682997us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b18003d7740. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a550d1 00007f9c80e70a0f
0709 13:16:49.140050 (+ 18776us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:16:49.140077 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:50.284727 (+1144650us) spinlock_profiling.cc:243] Waited 33.3 ms on lock 0x7b54000bc7d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:50.284846 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:50.284917 (+    71us) raft_consensus.cc:1685] finished
0709 13:16:50.284930 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:50.285240 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28625,"raft.queue_time_us":50821,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":12105,"spinlock_wait_cycles":100953088,"thread_start_us":76467,"threads_started":2,"wal-append.queue_time_us":1753983}
I20260709 13:16:50.319785 12914 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:50.320114 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:50.322073 12993 raft_consensus.cc:2749] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 5. Current term is 6.
I20260709 13:16:50.286027 12912 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:50.327265 13039 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:50.334343 10948 raft_consensus.cc:1275] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:16:50.302488 12996 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:50.341171 10794 raft_consensus.cc:1275] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:50.342156 12996 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:50.311919 13016 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:50.360318 13101 raft_consensus.cc:2749] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:16:50.364521 11507 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56416 (request call id 465) took 2249 ms (client timeout 1626 ms). Trace:
I20260709 13:16:50.364743 11506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56410 (request call id 449) took 1247 ms. Trace:
I20260709 13:16:50.356930 12996 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:16:50.368376 11505 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56410 (request call id 436) took 2406 ms (client timeout 2066 ms). Trace:
I20260709 13:16:50.369802 12768 raft_consensus.cc:3055] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:50.371196 10947 raft_consensus.cc:1240] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:50.373312 12912 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:16:50.365061 11507 rpcz_store.cc:269] 0709 13:16:48.115849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:48.116128 (+   279us) service_pool.cc:224] Handling call
0709 13:16:50.364112 (+2247984us) spinlock_profiling.cc:243] Waited 425 ms on lock 0x7b54004e5554. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:50.364285 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":976683264}
I20260709 13:16:50.375540 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:50.360971 13088 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:50.361016 13016 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:50.382409 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:50.378882 12768 raft_consensus.cc:740] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:50.382879 10944 raft_consensus.cc:1275] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:16:50.383157 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:50.383740 13105 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:50.383934 10948 raft_consensus.cc:1275] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:16:50.383630 13147 consensus_queue.cc:1048] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:16:50.377465 11505 rpcz_store.cc:269] 0709 13:16:47.961771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.971724 (+  9953us) service_pool.cc:224] Handling call
0709 13:16:50.367930 (+2396206us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400527954. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:50.368135 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2411362560}
I20260709 13:16:50.385231 13115 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:50.366964 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:50.377404 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:50.355108 12574 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:50.382510 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:50.399462 13147 consensus_queue.cc:1048] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:16:50.399919 13209 consensus_queue.cc:1048] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:16:50.392953 13039 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:50.401582 13101 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:50.402421 13101 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:50.405210 13101 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:50.405519 13039 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:50.366045 11506 rpcz_store.cc:276] 0709 13:16:49.117054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:49.122256 (+  5202us) service_pool.cc:224] Handling call
0709 13:16:50.364293 (+1242037us) spinlock_profiling.cc:243] Waited 596 ms on lock 0x7b54004d3fd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:50.364569 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1371246848}
I20260709 13:16:50.408825 12768 consensus_queue.cc:260] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:50.413736 12768 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:50.367398 12914 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:50.420454 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:50.424916 13155 raft_consensus.cc:3055] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:50.425621 13155 raft_consensus.cc:740] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:50.427016 10795 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 5.
W20260709 13:16:50.422880 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 170741us
I20260709 13:16:50.434599 13115 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:50.435117 13155 consensus_queue.cc:260] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:50.436865 13155 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:50.437326 11477 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:50.437567 13115 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:50.438277 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:50.390722 13105 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:50.439930 11477 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:50.440709 13088 raft_consensus.cc:2749] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:50.441860 13105 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:50.447556 13157 raft_consensus.cc:2749] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:16:50.453809 13032 raft_consensus.cc:3055] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:16:50.454068 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 92862.9us
I20260709 13:16:50.454458 13032 raft_consensus.cc:740] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:50.455816 13032 consensus_queue.cc:260] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:50.457136 13032 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:50.457827 12914 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:50.465307 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:50.466091 11506 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 7.
I20260709 13:16:50.471763 10949 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 6.
W20260709 13:16:50.480394 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:50.495357 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:50.495992 13046 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:50.496511 11506 raft_consensus.cc:2393] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 7: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:50.496896 13046 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
W20260709 13:16:50.497783 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:50.497685 13034 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:50.475684 12574 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:50.500547 13046 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:50.509769 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 70030.7us
I20260709 13:16:50.498445 13034 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:50.504976 13074 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:50.516997 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:50.521677 13152 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:50.517102 12798 raft_consensus.cc:2749] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:50.523662 11506 raft_consensus.cc:1275] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 3 LEARNER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:16:50.524380 11478 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:50.524211 12798 consensus_queue.cc:1048] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:50.526278 12798 raft_consensus.cc:2804] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:50.527187 12798 raft_consensus.cc:697] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 5 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:50.528411 12798 consensus_queue.cc:237] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:50.512351 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:50.533140 11505 raft_consensus.cc:2376] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 4 LEARNER]: Leader election vote request: Already granted yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 4. Re-sending same reply.
I20260709 13:16:50.531822 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:50.522567 13086 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:50.536103 11512 raft_consensus.cc:1240] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:16:50.543815 12996 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:50.546386 12996 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:50.546913 13115 consensus_queue.cc:1059] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:16:50.548218 13086 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:50.540985 13074 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:50.549829 13073 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:50.550863 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:50.551608 10949 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 6.
I20260709 13:16:50.550451 13074 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:50.561823 10873 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 1411) took 2692 ms (client timeout 2125 ms). Trace:
I20260709 13:16:50.562737 13134 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:50.562929 11477 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:50.563230 10868 raft_consensus.cc:1275] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:16:50.564486 11477 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:50.565097 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:50.566015 10869 raft_consensus.cc:1275] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:16:50.566428 10869 pending_rounds.cc:85] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 3
I20260709 13:16:50.566735 10869 pending_rounds.cc:107] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:16:50.567103 10869 raft_consensus.cc:2889] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:50.567811 13133 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:50.568125 13034 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:50.568405 13133 raft_consensus.cc:697] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 6 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:50.569212 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1432) took 1246 ms. Trace:
I20260709 13:16:50.569636 10869 rpcz_store.cc:276] 0709 13:16:49.323036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:49.357154 (+ 34118us) service_pool.cc:224] Handling call
0709 13:16:50.565344 (+1208190us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:50.567745 (+  2401us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:50.568941 (+  1196us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b5400078d50. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:50.569104 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2777876736}
I20260709 13:16:50.571336 12574 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:50.571924 12574 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:50.572383 12574 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:16:50.562337 10873 rpcz_store.cc:269] 0709 13:16:47.868891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:48.466885 (+597994us) service_pool.cc:224] Handling call
0709 13:16:50.561298 (+2094413us) spinlock_profiling.cc:243] Waited 265 ms on lock 0x7b5400078d54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:50.561553 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":610479872}
I20260709 13:16:50.579694 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:50.580395 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:50.580602 11505 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 2.
I20260709 13:16:50.569563 13133 consensus_queue.cc:237] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:50.592408 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:50.596342 10793 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 4.
I20260709 13:16:50.599328 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:50.600427 10794 raft_consensus.cc:2368] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 5. Current term is 6.
I20260709 13:16:50.603874 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:50.606756 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:50.609515 10794 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650"
dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
mode: GRACEFUL
new_leader_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:16:50.610329 13123 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:50.610509 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:50.611235 10944 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 6.
I20260709 13:16:50.611738 10869 raft_consensus.cc:1275] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:16:50.612155 10869 pending_rounds.cc:85] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 2
I20260709 13:16:50.612438 10869 pending_rounds.cc:107] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:16:50.612701 10869 raft_consensus.cc:2889] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:50.614176 13100 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:50.614765 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:50.615875 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:50.616284 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:50.610352 10794 raft_consensus.cc:606] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Received request to transfer leadership to TS cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:50.618332 10794 raft_consensus.cc:614] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:50.611781 13055 raft_consensus.cc:2749] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:50.619889 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:50.620592 10945 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 6.
I20260709 13:16:50.623064 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:50.624217 10868 raft_consensus.cc:2410] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 5 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:16:50.624907 10800 raft_consensus.cc:1275] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:16:50.625514 10797 raft_consensus.cc:1275] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:16:50.625917 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:50.625463 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:50.626412 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:50.626746 10800 raft_consensus.cc:2410] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 7 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:16:50.627265 10793 raft_consensus.cc:2393] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 6: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:50.623927 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:50.631330 10794 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 7.
I20260709 13:16:50.630503 10950 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 6.
W20260709 13:16:50.644883 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:50.647363 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:50.648406 10945 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 5.
I20260709 13:16:50.650693 10875 raft_consensus.cc:1275] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:16:50.651230 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:50.651836 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:50.652176 10876 raft_consensus.cc:2393] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 6: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
W20260709 13:16:50.652434 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:50.655988 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:50.659484 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:50.658488 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:50.666052 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:50.668141 10794 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:50.670925 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:50.672231 10946 raft_consensus.cc:2410] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:16:50.673466 13184 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:50.674708 13184 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:50.643714 12574 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:50.686265 10794 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 6.
I20260709 13:16:50.695374 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:50.696417 10870 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:50.698289 10869 raft_consensus.cc:2393] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 7: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:50.699015 10795 raft_consensus.cc:1240] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:50.699519 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:50.700217 10869 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:50.701956 13055 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:16:50.703492 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:50.708060 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:50.709353 10874 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 1399) took 1430 ms (client timeout 1717 ms). Trace:
W20260709 13:16:50.709784 10874 rpcz_store.cc:269] 0709 13:16:49.278378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:49.308237 (+ 29859us) service_pool.cc:224] Handling call
0709 13:16:50.708911 (+1400674us) spinlock_profiling.cc:243] Waited 23.5 ms on lock 0x7b540009a6d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:50.709175 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":54026496}
I20260709 13:16:50.695358 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:50.710963 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:16:50.695928 13218 log.cc:927] Time spent T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:16:50.721642 10766 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:50.729019 13212 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:50.729796 13212 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:50.729058 12775 consensus_queue.cc:1059] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 13:16:50.732676 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:50.733479 12775 raft_consensus.cc:3055] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:50.734022 12775 raft_consensus.cc:740] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:50.736479 10948 raft_consensus.cc:1275] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:16:50.737624 10874 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:50.738162 10945 raft_consensus.cc:1275] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:16:50.702735 13055 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:50.733621 11509 raft_consensus.cc:2368] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 4. Current term is 5.
I20260709 13:16:50.732697 13212 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:50.763852 13055 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:50.763463 12775 consensus_queue.cc:260] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:50.765228 12775 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:50.777287 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:50.775338 10766 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:50.780572 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:16:50.781814 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:50.752914 13074 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:50.783759 13074 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:50.784204 13074 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:50.786152 11503 raft_consensus.cc:1240] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:50.787994 10915 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:50.788195 12996 consensus_queue.cc:1048] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:50.790813 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:50.791067 13212 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:50.791828 11509 raft_consensus.cc:2393] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 6: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:50.791795 13212 raft_consensus.cc:697] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 4 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:50.793020 13212 consensus_queue.cc:237] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:50.794826 13184 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:50.797097 10766 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:16:50.799307 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:50.799880 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:50.800076 10766 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:50.800612 10871 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:50.802326 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:16:50.790825 13143 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.070s	user 0.022s	sys 0.010s
I20260709 13:16:50.803165 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:50.803894 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:50.813606 10944 raft_consensus.cc:2393] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 6: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:50.804505 13032 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:50.804765 13034 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:50.805472 10841 connection.cc:256] RPC call timeout handler was delayed by -1.0222779979999359s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2066s for a response
I20260709 13:16:50.805649 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:50.806427 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:50.806910 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:50.807267 10767 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:50.807929 10945 raft_consensus.cc:2393] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 5: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:50.809124 12838 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:50.819597 10946 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
I20260709 13:16:50.811229 10873 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 4.
I20260709 13:16:50.809620 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:50.828418 10874 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 6.
I20260709 13:16:50.832054 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:50.832867 10873 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:16:50.834209 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:50.835526 10841 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:50.836452 13134 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:50.837083 13134 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:50.838948 13046 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:50.839403 12574 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:50.839640 13046 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:50.840734 13122 raft_consensus.cc:2804] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:50.841209 13122 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:50.841547 13122 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:50.842017 13046 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:50.842937 10949 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:50.833056 10947 raft_consensus.cc:1275] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:16:50.845902 10867 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 7.
I20260709 13:16:50.846683 10869 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 6.
W20260709 13:16:50.812302 13143 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.226s	user 0.023s	sys 0.010s
I20260709 13:16:50.859109 11512 raft_consensus.cc:1240] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 5 LEADER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:50.815582 13105 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:50.866824 10870 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 5.
I20260709 13:16:50.842701 13046 raft_consensus.cc:2804] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:50.885100 10874 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:53748
W20260709 13:16:50.885653 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:50.880194 11503 raft_consensus.cc:2393] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 7: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:50.865582 13155 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:50.902454 11478 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:50.823175 13032 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:50.872148 13105 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:50.907706 10766 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:50.846941 13034 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:50.893198 13115 raft_consensus.cc:2749] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:16:50.914565 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:50.910550 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 143392us
I20260709 13:16:50.911535 13105 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:50.901542 13016 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:50.902074 13155 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:50.929677 10944 raft_consensus.cc:1240] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:50.894461 13046 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:50.917469 12990 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:50.930804 12574 raft_consensus.cc:2749] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:50.904027 12798 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:50.942425 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:50.930593 12903 raft_consensus.cc:2749] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:50.945591 13134 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:50.931145 12684 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:50.929561 11509 raft_consensus.cc:1275] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
W20260709 13:16:50.986877 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 294118us
W20260709 13:16:51.008641 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 111197us
I20260709 13:16:51.000787 13155 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:51.000315 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:51.000264 13046 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:51.029208 10767 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:51.013935 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:51.058547 10944 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 8
W20260709 13:16:51.058722 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 160813us
I20260709 13:16:51.064906 10949 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 6.
I20260709 13:16:51.074456 11477 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:51.007057 13016 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:51.057480 13034 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:51.099879 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:51.101523 10766 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:16:51.102488 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 114746us
W20260709 13:16:51.094242 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 229164us
I20260709 13:16:51.051254 12798 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:51.107127 10874 raft_consensus.cc:606] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Received request to transfer leadership to TS cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:51.098744 13073 raft_consensus.cc:2749] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:16:51.112545 10874 raft_consensus.cc:614] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:51.109612 11478 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:51.110594 13086 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:51.073920 12990 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:51.124375 10871 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 5.
I20260709 13:16:51.126603 12768 consensus_queue.cc:1048] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:51.116256 12903 raft_consensus.cc:2804] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:51.120491 10699 catalog_manager.cc:5697] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:51.100767 12684 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:51.186237 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:51.110927 12996 consensus_queue.cc:1048] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
W20260709 13:16:51.190434 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 58621.3us
I20260709 13:16:51.192965 10767 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:51.110064 13055 consensus_queue.cc:1048] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:16:51.142619 12903 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:51.133188 13074 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:51.203766 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 100390us
I20260709 13:16:51.205478 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:51.206419 10794 raft_consensus.cc:2410] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 13:16:51.208134 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 175670us
I20260709 13:16:51.198858 13032 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:51.206202 12903 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:51.203815 13253 raft_consensus.cc:2804] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:16:51.212471 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 58585.9us
I20260709 13:16:51.213018 12775 raft_consensus.cc:2749] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:51.225762 10699 catalog_manager.cc:5697] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 2 to 4, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 4 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:51.218009 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:16:51.218766 13123 consensus_meta.cc:287] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.597s	user 0.015s	sys 0.000s
I20260709 13:16:51.218613 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:51.214392 10873 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 7.
I20260709 13:16:51.235494 10793 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 3.
I20260709 13:16:51.242185 10796 raft_consensus.cc:1240] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:16:51.240774 10800 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
I20260709 13:16:51.243582 10699 catalog_manager.cc:5697] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 3 to 4, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 4 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:51.240976 10943 raft_consensus.cc:1275] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:16:51.247061 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 145632us
I20260709 13:16:51.242991 10916 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:16:51.246862 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:51.256690 11504 raft_consensus.cc:1275] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:16:51.257752 13248 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:51.258181 10795 raft_consensus.cc:1240] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:51.256860 10801 raft_consensus.cc:1275] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 7. (term mismatch)
I20260709 13:16:51.258984 10801 pending_rounds.cc:85] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 6
I20260709 13:16:51.259305 10801 pending_rounds.cc:107] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 13:16:51.259729 10801 raft_consensus.cc:2889] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:51.259181 10699 catalog_manager.cc:5697] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:51.258358 13248 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:51.260388 13060 raft_consensus.cc:2749] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:51.242542 13186 consensus_queue.cc:1059] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.150s
I20260709 13:16:51.259868 13253 raft_consensus.cc:697] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 6 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:51.261477 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:51.261641 10699 catalog_manager.cc:5697] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 3 to 4, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 4 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:51.262243 13253 consensus_queue.cc:237] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:51.265267 13246 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:51.276146 13246 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:51.277848 13240 log.cc:927] Time spent T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
W20260709 13:16:51.271756 13232 log.cc:927] Time spent T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.339s	user 0.002s	sys 0.000s
I20260709 13:16:51.271881 10841 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:51.272047 12775 consensus_queue.cc:1059] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:16:51.279747 12775 raft_consensus.cc:3055] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:51.280099 12775 raft_consensus.cc:740] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:51.280943 12775 consensus_queue.cc:260] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:51.281685 13052 raft_consensus.cc:2804] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:51.282114 13052 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:51.282291 12775 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:51.273267 13186 raft_consensus.cc:3055] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:51.283176 13186 raft_consensus.cc:740] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:51.283987 13242 consensus_queue.cc:1048] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:16:51.284196 13186 consensus_queue.cc:260] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:51.282517 13052 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:51.285583 13186 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:16:51.274132 13228 log.cc:927] Time spent T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.339s	user 0.000s	sys 0.002s
I20260709 13:16:51.278530 13246 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:51.289402 13074 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:51.274250 10766 leader_election.cc:400] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 4. Current term is 5.
W20260709 13:16:51.278978 13234 log.cc:927] Time spent T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.170s	user 0.000s	sys 0.001s
I20260709 13:16:51.291858 10766 leader_election.cc:403] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:16:51.293270 12838 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:51.294322 12838 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:51.295069 13074 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:51.295603 12990 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:51.296669 12838 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:51.299547 13016 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:51.301051 13238 log.cc:927] Time spent T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
I20260709 13:16:51.304116 12684 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:51.306003 13100 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:51.306720 13100 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:51.310953 13243 consensus_queue.cc:1048] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.115s
I20260709 13:16:51.302098 10699 catalog_manager.cc:5697] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 1 to 3, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 3 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
W20260709 13:16:51.313710 13241 log.cc:927] Time spent T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:16:51.319860 10699 catalog_manager.cc:5697] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 2 to 6, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 6 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
W20260709 13:16:51.333369 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:51.260474 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 48396.7us
I20260709 13:16:51.336244 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:51.341461 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:51.342618 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:51.343276 10945 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:16:51.273288 13239 log.cc:927] Time spent T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 13:16:51.354367 13074 raft_consensus.cc:2749] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 4. Current term is 5.
I20260709 13:16:51.357067 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:51.359022 10873 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 5.
I20260709 13:16:51.361413 13123 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:51.362123 13123 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:51.362380 13052 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:51.365157 13052 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:51.361442 13247 log.cc:927] Time spent T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 13:16:51.369475 13123 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:51.354929 10699 catalog_manager.cc:5697] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:16:51.367990 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:51.371726 10699 catalog_manager.cc:5697] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 3 to 5, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 5 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:16:51.380584 12962 tablet_bootstrap.cc:492] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:51.384729 13157 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:51.390069 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:51.398288 10874 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
I20260709 13:16:51.396601 10944 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 8.
I20260709 13:16:51.406153 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:51.407034 10875 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:51.408344 10916 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:51.405208 13086 raft_consensus.cc:697] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 6 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:51.398885 10868 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 4.
I20260709 13:16:51.397675 13157 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:51.428907 10874 raft_consensus.cc:1240] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 6. Current term is 7. Ops: [5.5-6.6]
I20260709 13:16:51.429114 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:51.431722 10868 raft_consensus.cc:2410] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 6 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:16:51.439044 10793 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8cab289810294719ac9686e40290a6ab"
dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
mode: GRACEFUL
new_leader_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:16:51.440034 10793 raft_consensus.cc:606] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Received request to transfer leadership to TS cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:51.440466 10793 raft_consensus.cc:614] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:51.439122 10873 raft_consensus.cc:1275] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:16:51.411950 13123 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
W20260709 13:16:51.447156 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 58734us
I20260709 13:16:51.374449 10699 catalog_manager.cc:5697] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 1 to 2, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 2 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:51.422644 13055 raft_consensus.cc:2749] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:51.453483 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:51.454195 10868 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 5.
I20260709 13:16:51.431064 13074 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:51.462538 10873 raft_consensus.cc:1275] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 2. (term mismatch)
I20260709 13:16:51.462945 10873 pending_rounds.cc:85] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 1
I20260709 13:16:51.466310 10873 pending_rounds.cc:107] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:16:51.466604 10873 raft_consensus.cc:2889] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:16:51.467510 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 251226us
I20260709 13:16:51.469377 10945 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 7.
W20260709 13:16:51.465843 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 184664us
I20260709 13:16:51.471040 10873 pending_rounds.cc:107] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 13:16:51.466990 10699 catalog_manager.cc:5697] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 1 to 4, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 4 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:51.472523 10873 raft_consensus.cc:2889] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:51.475678 10766 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:16:51.476902 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 353461us
I20260709 13:16:51.456318 13123 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:51.482100 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:51.478250 10699 catalog_manager.cc:5697] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 4 to 5, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 5 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:51.481678 10841 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:51.485260 10793 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309"
dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
mode: GRACEFUL
new_leader_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:16:51.485936 10793 raft_consensus.cc:606] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Received request to transfer leadership to TS cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:51.486349 10793 raft_consensus.cc:614] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:51.481751 13256 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:51.487325 13256 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:51.489709 13256 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:51.489816 10841 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:51.479395 12775 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:51.490706 12775 raft_consensus.cc:697] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:51.490885 13246 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:51.491799 12775 consensus_queue.cc:237] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:51.492106 13134 consensus_queue.cc:1059] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:51.491529 10841 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:51.481348 12962 tablet_bootstrap.cc:492] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586: Bootstrap complete.
I20260709 13:16:51.496937 10767 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:51.498571 10868 raft_consensus.cc:1240] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 5. Current term is 6. Ops: [3.6-5.7]
I20260709 13:16:51.496997 10873 raft_consensus.cc:2393] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 6: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:51.491596 13246 raft_consensus.cc:697] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:51.499739 10699 catalog_manager.cc:5697] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 2 to 5, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 5 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:51.496676 13034 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:51.470274 13086 consensus_queue.cc:237] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:51.479482 13074 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:51.509372 13246 consensus_queue.cc:237] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:51.510349 10793 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740"
dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
mode: GRACEFUL
new_leader_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:53748
W20260709 13:16:51.511935 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:51.513401 13186 raft_consensus.cc:2749] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:51.515413 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:16:51.516402 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 45805us
I20260709 13:16:51.517223 10793 raft_consensus.cc:606] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Received request to transfer leadership to TS cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:51.517789 10793 raft_consensus.cc:614] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:51.520108 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:16:51.523268 13237 log.cc:927] Time spent T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.325s	user 0.002s	sys 0.000s
I20260709 13:16:51.522990 12990 consensus_queue.cc:1048] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:51.523926 13055 consensus_queue.cc:1048] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:16:51.527714 13229 log.cc:927] Time spent T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.593s	user 0.000s	sys 0.005s
I20260709 13:16:51.529881 10801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 211) took 1564 ms. Trace:
I20260709 13:16:51.520733 10945 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 6.
I20260709 13:16:51.532361 10767 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:51.530270 10801 rpcz_store.cc:276] 0709 13:16:49.965132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:50.044136 (+ 79004us) service_pool.cc:224] Handling call
0709 13:16:50.044525 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:51.260190 (+1215665us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:51.529578 (+269388us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b5400386654. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:51.529815 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2761360128}
I20260709 13:16:51.526769 12798 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:51.508507 13100 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:51.528188 13155 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:51.541662 10766 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:51.542667 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:51.533686 13258 raft_consensus.cc:3055] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:51.520180 13243 raft_consensus.cc:2749] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:16:51.533281 13034 raft_consensus.cc:697] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:51.534539 12962 ts_tablet_manager.cc:1403] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586: Time spent bootstrapping tablet: real 1.439s	user 0.035s	sys 0.022s
I20260709 13:16:51.539008 10699 catalog_manager.cc:5697] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 1 to 4, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 4 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
W20260709 13:16:51.553527 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 62129.5us
I20260709 13:16:51.541430 13250 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:51.555896 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:51.556665 10873 raft_consensus.cc:2393] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 5: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:51.548650 10767 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:51.540963 13186 raft_consensus.cc:2804] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:51.552922  9682 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:16:51.548146 13100 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:51.557947 13186 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:51.563167  9682 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:51.566205 12903 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:51.580374 13105 consensus_meta.cc:287] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.567s	user 0.007s	sys 0.004s
W20260709 13:16:51.580551 13122 consensus_meta.cc:287] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.739s	user 0.018s	sys 0.001s
I20260709 13:16:51.580770 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:51.580845 13105 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:51.581177 13122 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:51.581900 12903 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:51.566978 13157 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:51.569998 13074 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:51.572219 13123 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:51.573014 13034 consensus_queue.cc:237] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:51.583454 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:51.574747 13250 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:51.585490 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:51.586081 10944 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:51.575274 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:51.576817 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:51.587147 10869 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 7.
I20260709 13:16:51.587251 13250 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:51.577538 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:51.588039 10874 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 5.
I20260709 13:16:51.588156 10798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 1829) took 1764 ms. Trace:
I20260709 13:16:51.588596 10798 rpcz_store.cc:276] 0709 13:16:49.824228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:49.859159 (+ 34931us) service_pool.cc:224] Handling call
0709 13:16:49.875143 (+ 15984us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:51.308369 (+1433226us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:51.308417 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:51.308475 (+    58us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:51.308521 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:51.587818 (+279297us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b54000bc2d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:51.587864 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:51.588004 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":545,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":94,"spinlock_wait_cycles":3295152512}
I20260709 13:16:51.590061 10916 leader_election.cc:400] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 5. Current term is 6.
I20260709 13:16:51.591094 10916 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:51.593089 12996 raft_consensus.cc:2804] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:51.593487 12996 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:51.593817 12996 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:51.596064 12962 raft_consensus.cc:359] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 5 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:51.597069 12962 raft_consensus.cc:740] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 5 LEARNER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Initialized, Role: LEARNER
I20260709 13:16:51.597570 13248 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:51.597744 12962 consensus_queue.cc:260] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:51.598994 13055 consensus_queue.cc:1048] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:51.599329 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:51.600489 12903 raft_consensus.cc:2749] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:16:51.601547 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 53062.8us
I20260709 13:16:51.602306 10767 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:51.603485 10767 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:51.578775 13046 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:51.580066 13186 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:51.603593 13123 raft_consensus.cc:2749] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:51.548419 13258 raft_consensus.cc:740] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:16:51.606979 13244 log.cc:927] Time spent T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.560s	user 0.005s	sys 0.000s
I20260709 13:16:51.606827 13258 consensus_queue.cc:260] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:51.608989 13258 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:51.611593 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:51.611748 13186 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:51.612318 10943 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 5.
I20260709 13:16:51.613929 13186 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:51.614264 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:51.616698 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:51.618379 10869 raft_consensus.cc:1275] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:16:51.580265 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:51.619680 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:51.620004 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:51.650337 11477 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:51.622893 13122 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:51.623310 13100 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:51.651867 11477 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:51.652407 13100 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:51.652685 12983 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:51.624418 10875 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 7.
I20260709 13:16:51.653286 12983 raft_consensus.cc:697] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 8 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:51.654532 13142 raft_consensus.cc:2804] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:51.654948 13142 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:51.579118 13155 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:51.630970 10916 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:16:51.675774 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 44888.6us
W20260709 13:16:51.676252 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:51.677170 13155 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:51.677435 10916 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:51.631549 13184 consensus_queue.cc:1048] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:51.677992 13155 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:51.635298 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:51.638353 10944 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 6.
I20260709 13:16:51.643389 10699 catalog_manager.cc:5697] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: config changed from index -1 to 2, term changed from 1 to 3, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to <none>, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New cstate: current_term: 3 committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:51.680737 13100 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:51.681592 13060 raft_consensus.cc:2749] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:51.681143 10699 catalog_manager.cc:5697] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 1 to 5, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 5 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:51.681998 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:51.682340 13060 raft_consensus.cc:2764] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:16:51.683182 13100 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:51.646898 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 91548.4us
I20260709 13:16:51.682668 10944 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 5.
I20260709 13:16:51.684064 10916 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:51.684167 10841 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:51.683804 10699 catalog_manager.cc:5697] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:51.685952 12768 raft_consensus.cc:2749] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:16:51.650435 10871 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 7.
I20260709 13:16:51.685997 10699 catalog_manager.cc:5697] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 1 to 4. New cstate: current_term: 4 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:51.687510 10950 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 7.
I20260709 13:16:51.620707 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:51.651796 13157 consensus_queue.cc:1048] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:16:51.646493 12838 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:51.694761 12838 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:51.695026 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:51.695369 12838 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:51.696038 11511 raft_consensus.cc:2410] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:16:51.698972 10876 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 3.
I20260709 13:16:51.655310 13142 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:51.702551 10941 raft_consensus.cc:1240] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260709 13:16:51.703208 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:51.704085 10944 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 6.
I20260709 13:16:51.705133 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:51.706633 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:51.708194 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:16:51.710987 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 107573us
I20260709 13:16:51.659139 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:51.711946 10767 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:51.713043 10873 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 5.
I20260709 13:16:51.713228 13123 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:51.659399 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:51.713860 13123 raft_consensus.cc:697] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:51.715695 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:51.716629 10950 raft_consensus.cc:2410] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:16:51.716555 13123 consensus_queue.cc:237] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:51.662935 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 58488.4us
I20260709 13:16:51.720136 10842 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:51.721637 10842 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:51.721824 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:51.722463 13032 raft_consensus.cc:2749] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:51.723321 12768 raft_consensus.cc:2749] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:16:51.667554 13259 log.cc:927] Time spent T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 13:16:51.669234 13105 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:51.670614 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:51.683108 10766 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:51.654282 12983 consensus_queue.cc:237] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:51.727746 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 44713.2us
I20260709 13:16:51.727910 12768 consensus_queue.cc:1048] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:51.728883 13115 raft_consensus.cc:2804] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:51.727749 10947 raft_consensus.cc:2393] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 8: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:51.729482 13115 raft_consensus.cc:697] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:51.687781 10699 catalog_manager.cc:5697] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 2 to 5, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 5 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:51.730474 13115 consensus_queue.cc:237] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:51.730638 10699 catalog_manager.cc:5697] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 2 to 3, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 3 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:51.731351 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:51.731972 10944 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:51.732313 10699 catalog_manager.cc:5697] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 1 to 2, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 2 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:51.733357 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:51.645063 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:51.688081 13016 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:51.704080 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:51.729068 10766 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:51.660619 13122 consensus_queue.cc:1048] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:51.684933 12990 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:51.624291 13046 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:51.734453 10945 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 7.
I20260709 13:16:51.684823 13155 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:51.735632 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 46403.8us
I20260709 13:16:51.736291 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:16:51.737599 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 132574us
I20260709 13:16:51.747098 10950 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:51.738911 10841 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:51.748771 10944 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 7.
I20260709 13:16:51.750638 10948 raft_consensus.cc:1240] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:16:51.752297 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:16:51.753945 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:51.753932 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:51.750691 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:51.758431 13016 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:51.759588 11478 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:16:51.760268 10766 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:51.760982 13016 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:51.761327 11478 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:16:51.738305 12768 consensus_queue.cc:1059] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 13:16:51.737358 13074 raft_consensus.cc:2804] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:51.754743 11511 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 6.
I20260709 13:16:51.772714 10842 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:51.772992 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:51.773733 10944 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
I20260709 13:16:51.775487 13250 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:51.777797 13074 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:51.777995 13257 consensus_queue.cc:1048] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:16:51.778549 13074 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:51.779488 13073 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:51.780026 13073 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:51.780642 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:51.781450 10794 raft_consensus.cc:2410] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:16:51.781634 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:51.782176 13073 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:51.782444 10799 raft_consensus.cc:2393] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 3: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:51.783041 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:51.783521 10797 raft_consensus.cc:1275] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:16:51.783620 10801 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 7.
I20260709 13:16:51.783751 10799 raft_consensus.cc:1275] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:16:51.782266 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:51.786152 10794 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 3.
I20260709 13:16:51.763487 11478 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:16:51.788405 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:51.789448 10792 raft_consensus.cc:2393] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 8: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:51.793423 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:51.793836 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:51.753229 12990 raft_consensus.cc:697] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 6 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:51.799877 10875 raft_consensus.cc:1240] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:51.799139 11478 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:16:51.804086 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:51.771286 13032 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:51.742513 13060 consensus_queue.cc:1059] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 13:16:51.753163 12996 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:51.787946 13073 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:51.826874 11477 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:16:51.836427 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:51.791216 13133 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:51.858597 10794 raft_consensus.cc:1275] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:16:51.853822 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:51.808687 10874 raft_consensus.cc:1275] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:16:51.863281 11477 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:51.866484 11507 raft_consensus.cc:1275] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 4 LEARNER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:16:51.801465 13046 raft_consensus.cc:3055] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:51.802464 12768 raft_consensus.cc:2749] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:51.880712 10946 raft_consensus.cc:2393] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 5: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
W20260709 13:16:51.868584 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 120219us
I20260709 13:16:51.807451 13086 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:51.869478 10841 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:51.869637 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:51.798494 13250 raft_consensus.cc:697] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 7 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:51.866945 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:51.885891 10950 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 6.
I20260709 13:16:51.842571 13032 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:51.843330 13253 raft_consensus.cc:2804] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:51.887679 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:51.867280 11504 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 5 LEARNER]: Advancing to term 6
I20260709 13:16:51.909695 10868 raft_consensus.cc:1275] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:16:51.867585 13055 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:51.915060 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:51.918686 10797 raft_consensus.cc:2393] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 7: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:51.864586 13100 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:51.876717 12574 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:51.875442 13046 raft_consensus.cc:740] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:51.881640 13073 raft_consensus.cc:697] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:51.886752 13086 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:51.879145 13133 raft_consensus.cc:697] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 5 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:51.952798 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:16:51.953794 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:51.901202 13253 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:51.940681 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 71456.8us
I20260709 13:16:51.938692 10841 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:51.901610 13032 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:51.937668 10873 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 5.
I20260709 13:16:51.960639 10841 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:51.961652 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:51.965111 10801 raft_consensus.cc:2410] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:16:51.936990 12574 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:51.944512 13246 raft_consensus.cc:2749] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:51.959141 13086 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:51.974310 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:51.952630 11503 raft_consensus.cc:1240] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 13:16:52.020432 13133 consensus_queue.cc:237] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:51.962880 13253 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:51.972563 12768 raft_consensus.cc:2804] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:51.974753 13243 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:51.972748 12574 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:51.958492 13100 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:51.940124 12990 consensus_queue.cc:237] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:52.055667 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:52.056972 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:52.028581 12996 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:52.064163 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:51.971418 13055 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:52.075261 10766 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:51.973680 13034 consensus_queue.cc:1059] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.075s
I20260709 13:16:52.083388 10766 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:52.041882 13115 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:52.095952 10792 raft_consensus.cc:1275] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:16:52.097756 10874 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 5.
I20260709 13:16:52.064289 12775 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:52.080633 12962 ts_tablet_manager.cc:1434] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586: Time spent starting tablet: real 0.522s	user 0.009s	sys 0.000s
I20260709 13:16:52.052237 12798 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:52.051828 12768 raft_consensus.cc:697] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:52.088701 13186 raft_consensus.cc:2749] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:52.098552 13157 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:52.098280 13105 raft_consensus.cc:2749] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:52.112859 13258 raft_consensus.cc:3055] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:52.127588 13258 raft_consensus.cc:740] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:16:52.128023 13267 log.cc:927] Time spent T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.248s	user 0.000s	sys 0.001s
I20260709 13:16:52.128610 13258 consensus_queue.cc:260] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
W20260709 13:16:52.108605 13264 log.cc:927] Time spent T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
W20260709 13:16:52.121385 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 221486us
I20260709 13:16:52.101441 13184 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:52.101855 13060 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:52.100951 12996 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:52.095288 13115 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:52.048069 13250 consensus_queue.cc:237] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:52.094575 13142 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:52.053328 13046 consensus_queue.cc:260] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:52.109519 12962 ts_tablet_manager.cc:1467] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586: Tablet startup took 551ms
I20260709 13:16:52.131479 10916 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:16:52.135599 10767 leader_election.cc:336] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Timed out: RequestConsensusVote RPC to 127.9.116.130:37895 timed out after 1.717s (SENT)
I20260709 13:16:52.147828 11504 raft_consensus.cc:1275] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 6 LEARNER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:16:52.144774 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:52.150043 10767 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:16:52.153571 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:52.110085 12775 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:52.161852 10842 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:52.119571 13257 consensus_queue.cc:1048] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:16:52.119240 12798 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:52.167480 10792 raft_consensus.cc:1240] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:16:52.160486 13258 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:52.177970 10793 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 5.
I20260709 13:16:52.131234 13157 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:52.085758 13073 consensus_queue.cc:237] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:52.145356 13184 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:52.167616 13212 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:52.190447 10800 raft_consensus.cc:1240] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 2. Current term is 3. Ops: []
W20260709 13:16:52.193584 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 58004us
I20260709 13:16:52.178188 13105 raft_consensus.cc:2749] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:52.148758 13060 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:52.187465 13262 log.cc:927] Time spent T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.241s	user 0.001s	sys 0.000s
I20260709 13:16:52.161653 12996 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:52.221998 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 90583.3us
I20260709 13:16:52.223558 10916 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:52.225924 13157 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:52.226107 13246 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:52.226634 13246 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:52.196122 13280 raft_consensus.cc:2749] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:52.169327 12768 consensus_queue.cc:237] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:52.211493 13055 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:52.238502 10916 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:16:52.242246 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 145752us
W20260709 13:16:52.235106 13261 log.cc:927] Time spent T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.243s	user 0.001s	sys 0.000s
I20260709 13:16:52.221463 13196 consensus_queue.cc:1048] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.154s
I20260709 13:16:52.280648 11478 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:52.241128 13186 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:52.263855 13100 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:52.269805 13248 raft_consensus.cc:2749] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:52.239305 13055 consensus_queue.cc:1048] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 13:16:52.271167 13275 consensus_queue.cc:1048] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.144s
I20260709 13:16:52.295224 13283 raft_consensus.cc:2749] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:52.235913 13212 consensus_queue.cc:1048] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.058s
I20260709 13:16:52.315678 10945 raft_consensus.cc:1275] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:16:52.319626 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 165897us
W20260709 13:16:52.313076 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 188012us
I20260709 13:16:52.295619 12775 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:52.298938 12838 consensus_meta.cc:287] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.603s	user 0.010s	sys 0.003s
I20260709 13:16:52.295706 13274 raft_consensus.cc:2749] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:52.336256 10842 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:52.339764 10945 raft_consensus.cc:1275] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:16:52.320220 13034 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:52.296080 13186 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:52.332520 12983 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:52.325197 13142 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:52.332755 13143 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:52.351897 13115 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:52.378182 11507 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 6.
W20260709 13:16:52.384363 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:52.420368 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:52.333847 12838 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:52.383402 13143 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:52.435460 13074 consensus_meta.cc:287] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.656s	user 0.006s	sys 0.006s
I20260709 13:16:52.436086 10796 raft_consensus.cc:1275] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:16:52.447108 10796 pending_rounds.cc:85] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 6
I20260709 13:16:52.447427 10796 pending_rounds.cc:107] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:16:52.447722 10796 raft_consensus.cc:2889] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:52.361336 13286 consensus_queue.cc:1048] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.104s
I20260709 13:16:52.356715 13287 consensus_queue.cc:1048] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.099s
I20260709 13:16:52.365240 13034 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:52.439024 13184 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:16:52.441057 13278 log.cc:927] Time spent T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.245s	user 0.001s	sys 0.000s
W20260709 13:16:52.443423 13284 log.cc:927] Time spent T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:16:52.445030 10793 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 6.
I20260709 13:16:52.449234 10916 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:52.379546 12798 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:52.452698 13279 log.cc:927] Time spent T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.229s	user 0.001s	sys 0.000s
I20260709 13:16:52.476233 10798 raft_consensus.cc:2368] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 5. Current term is 6.
I20260709 13:16:52.441425 13060 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:16:52.478437 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 111049us
W20260709 13:16:52.479549 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 150488us
I20260709 13:16:52.483740 10916 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:16:52.150548 12962 ts_tablet_manager.cc:1469] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586: Trace:
0709 13:16:43.295835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:43.298595 (+  2760us) service_pool.cc:224] Handling call
0709 13:16:43.724699 (+426104us) ts_tablet_manager.cc:934] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586: Initiating tablet copy from peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685)
0709 13:16:48.009465 (+4284766us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:16:49.049596 (+1040131us) tablet_metadata.cc:665] Metadata flushed
0709 13:16:49.049752 (+   156us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:16:49.245568 (+195816us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:16:49.246025 (+   457us) tablet_replica.cc:206] Creating consensus instance
0709 13:16:49.246471 (+   446us) ts_tablet_manager.cc:989] Replica registered.
0709 13:16:49.246566 (+    95us) inbound_call.cc:177] Queueing success response
0709 13:16:50.095040 (+848474us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:16:51.589326 (+1494286us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:16:51.589975 (+   649us) tablet_replica.cc:261] Starting instrumentation
0709 13:16:51.590606 (+   631us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":25399,"raft.queue_time_us":50361,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":116,"tablet-copy.queue_time_us":332835,"tablet-download-pool.queue_time_us":911,"tablet-download-pool.run_cpu_time_us":1274,"tablet-download-pool.run_wall_time_us":702780,"thread_start_us":419531,"threads_started":2}
I20260709 13:16:52.478286 13274 raft_consensus.cc:2749] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:52.454761 13074 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:52.497380 10842 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:52.527776 10916 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:52.529776 12990 raft_consensus.cc:2764] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won
I20260709 13:16:52.532270 13046 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:52.532451 13286 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:52.532991 13286 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:52.500607 13060 raft_consensus.cc:2749] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:16:52.503949 13246 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:52.556706 13275 consensus_queue.cc:1059] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 13:16:52.526666 13301 raft_consensus.cc:2764] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
W20260709 13:16:52.553097 13032 consensus_meta.cc:287] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.535s	user 0.011s	sys 0.001s
I20260709 13:16:52.554291 13289 raft_consensus.cc:2749] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:52.551994 13290 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:52.569751 13186 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:52.581863 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 84554.1us
I20260709 13:16:52.601945  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:52.610402 10842 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:52.568648 13184 consensus_queue.cc:1048] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 13:16:52.559396 13274 consensus_queue.cc:1048] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 13:16:52.614616 10842 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:52.615238 13289 raft_consensus.cc:2749] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:16:52.622141 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:52.580451 13032 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:52.630026 13301 raft_consensus.cc:2749] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:16:52.634358 13297 log.cc:927] Time spent T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:16:52.635253 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:52.643159 10868 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:52.608531 12838 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:52.640180 10842 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:52.612978 12996 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:16:52.645429 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 84378.5us
I20260709 13:16:52.623937 13046 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:52.647574 10869 raft_consensus.cc:1275] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 6. (term mismatch)
I20260709 13:16:52.649588 13286 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:52.657241 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:52.657994 10872 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 5.
I20260709 13:16:52.659420 10875 raft_consensus.cc:2393] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 7: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:52.650261 10869 pending_rounds.cc:85] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 5
I20260709 13:16:52.649236 10916 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:52.660065 10869 pending_rounds.cc:107] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:16:52.660434 10869 raft_consensus.cc:2889] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:52.667469 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:52.606528 13290 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:52.674376 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:52.670817 10871 raft_consensus.cc:1275] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:16:52.721280 10916 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:52.693979 10870 raft_consensus.cc:2393] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 6: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:52.682308 10872 raft_consensus.cc:2393] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 5: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:52.669430 13275 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:52.724407 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:52.673324 11478 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:16:52.659250 13301 raft_consensus.cc:2749] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:16:52.728914 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 56042.7us
I20260709 13:16:52.730719 10842 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:16:52.726927 13086 consensus_meta.cc:287] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.698s	user 0.013s	sys 0.000s
I20260709 13:16:52.721968 13143 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:16:52.734208 12574 consensus_meta.cc:287] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.656s	user 0.011s	sys 0.001s
I20260709 13:16:52.725835 13243 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:52.735249 12574 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:52.668565 13046 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:52.737363 10916 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:52.689908 13286 consensus_queue.cc:1048] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 13:16:52.757741 12574 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:52.758332 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:52.767863 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:52.768797 10869 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 3.
I20260709 13:16:52.737365 13283 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:52.736138 13275 raft_consensus.cc:697] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 5 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:52.736405 13074 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:52.737689 13243 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:16:52.779230 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 86832.8us
W20260709 13:16:52.779632 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:52.783631 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:52.785987 13046 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:52.764397 12990 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:16:52.786844 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 49604.4us
W20260709 13:16:52.773355 13253 consensus_meta.cc:287] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.733s	user 0.008s	sys 0.005s
I20260709 13:16:52.790218 12574 consensus_queue.cc:1048] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:16:52.736059 13086 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:52.786572 12994 raft_consensus.cc:3055] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:52.756837 13060 consensus_queue.cc:1048] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 13:16:52.771862 13258 raft_consensus.cc:2749] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:16:52.775611 13283 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:52.735958 13256 consensus_queue.cc:1048] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:16:52.809057 10842 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:52.810081 12994 raft_consensus.cc:740] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:52.825168 10842 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:16:52.824580 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 45121.1us
I20260709 13:16:52.816494 13032 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:52.812872 13060 consensus_queue.cc:1243] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer e01ecd008edd4aa2b6b2b47efbc2ea87 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:16:52.818396 13287 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:52.805836 12996 consensus_queue.cc:1048] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.018s
I20260709 13:16:52.835527 10867 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 5.
I20260709 13:16:52.816815 13283 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:16:52.847072 13311 log.cc:927] Time spent T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 13:16:52.846908 13275 consensus_queue.cc:237] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:52.821727 13286 raft_consensus.cc:2749] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:52.811048 13253 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:52.841761 10946 raft_consensus.cc:1275] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 13:16:52.853435 13310 log.cc:927] Time spent T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 13:16:52.840869 13184 consensus_queue.cc:1059] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 13:16:52.854365 13253 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:52.856885 13287 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:52.857359 13287 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:52.842211 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:52.823873 13307 consensus_queue.cc:1059] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 13:16:52.843636 12994 consensus_queue.cc:260] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:52.863952 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:52.840962 13301 raft_consensus.cc:2804] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:52.870097 12994 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:52.858899 13274 consensus_queue.cc:1048] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.012s
I20260709 13:16:52.873122 13290 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:52.873672 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:52.874600 13301 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:52.875109 13301 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:52.875988 13184 raft_consensus.cc:3055] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:52.876389 13184 raft_consensus.cc:740] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:52.878770 12775 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:52.877209 13184 consensus_queue.cc:260] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:52.880695 13184 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:52.879388 12775 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:52.879674 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:52.874595 10868 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 6.
I20260709 13:16:52.884591 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:52.893071 11478 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:16:52.896591 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:52.897238 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:52.898090 10950 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:52.870560 13086 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:52.897390 10945 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 6.
W20260709 13:16:52.906270 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 89040.8us
I20260709 13:16:52.912663 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:52.915181 11477 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:52.913800 10916 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:52.911630 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:52.920780 10916 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:52.927081 10943 raft_consensus.cc:1275] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:16:52.908286 11478 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:52.933100 10950 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 6.
I20260709 13:16:52.936232 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:52.936761 10949 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:52.906546 13314 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:52.912143 13253 raft_consensus.cc:2804] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:52.911803 13143 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:52.917150 13060 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:52.935303 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:52.946655 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:16:52.935021 12983 consensus_meta.cc:287] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.558s	user 0.011s	sys 0.000s
I20260709 13:16:52.935971 12798 raft_consensus.cc:2804] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:52.952344 11508 raft_consensus.cc:2410] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:16:52.955942 10767 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:52.904208 13324 consensus_queue.cc:1059] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 13:16:52.958914 11505 raft_consensus.cc:1240] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:52.961838 12994 raft_consensus.cc:2804] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:52.962301 12994 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:52.950989 13143 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:52.965471 13143 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:52.965967 12798 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:52.966419 12798 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:52.958902 13115 raft_consensus.cc:2749] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:16:52.967585 13318 log.cc:927] Time spent T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 13:16:52.969668 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:52.953306 13086 raft_consensus.cc:2749] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:52.970331 10876 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 4.
I20260709 13:16:52.957836 13060 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:52.971444 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:16:52.972105 13322 log.cc:927] Time spent T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 13:16:52.972339 10876 raft_consensus.cc:2410] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:16:52.947635 13314 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:52.972998 13060 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:52.973232 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:52.974387 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:52.974632 13314 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:52.975297 10949 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 8.
I20260709 13:16:52.912290 13288 consensus_queue.cc:1048] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.040s
W20260709 13:16:52.976521 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 68306.5us
W20260709 13:16:52.978879 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 58137.3us
I20260709 13:16:52.979075 13253 raft_consensus.cc:697] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 6 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:52.981593 13253 consensus_queue.cc:237] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:52.982000 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:52.982831 10867 raft_consensus.cc:2393] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 6: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:52.976468 12983 raft_consensus.cc:2749] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:16:52.984031 13273 log.cc:927] Time spent T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.835s	user 0.001s	sys 0.004s
I20260709 13:16:52.962638 12994 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:52.986596 10916 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:52.926079 13133 consensus_queue.cc:1048] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:16:52.992892 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:52.975940 12990 raft_consensus.cc:2749] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:52.994745 13074 consensus_queue.cc:1048] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:16:52.996731 10767 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:53.003149 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:53.004671 13324 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:16:53.004983 13325 log.cc:927] Time spent T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 13:16:53.006855 13243 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:53.009938 13115 consensus_queue.cc:1048] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:16:53.010923 13319 log.cc:927] Time spent T 00000000000000000000000000000000 P ae957c8fc66c4b6ba063d0c74cbf5484: Append to log took a long time: real 0.155s	user 0.002s	sys 0.001s
I20260709 13:16:53.022192 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:53.023397 12775 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:53.023348 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:53.018139 13324 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:53.024492 13324 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:53.011632 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:52.976260 13055 raft_consensus.cc:2749] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:53.016428 10867 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 6.
I20260709 13:16:53.018222 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:53.019135 10767 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:53.029731 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:53.030871 10869 raft_consensus.cc:2376] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 7. Re-sending same reply.
I20260709 13:16:53.024156 13287 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:53.020797 12838 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:53.015702 10916 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:53.043254 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:53.048035 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:53.049301 10767 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:53.050405 10876 raft_consensus.cc:2410] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:16:53.051183 10946 raft_consensus.cc:1240] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:53.053356 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:53.054246 10869 raft_consensus.cc:2393] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 6: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:53.058323 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:53.058910 10767 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:53.056974 13055 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:53.060750 13243 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:53.062057 10876 raft_consensus.cc:2410] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:16:53.063189 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:53.064960 10869 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:53.066587 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:53.061079 13055 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:53.068207 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 142877us
I20260709 13:16:53.071985 10874 raft_consensus.cc:2410] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:16:53.055698 12775 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:53.076165 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:53.014392 13143 consensus_queue.cc:1048] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:16:53.078295 10767 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:53.082731 10767 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:53.083359 10874 raft_consensus.cc:2393] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 6: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:53.078312 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:53.064394 13115 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:53.085685 11477 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:53.087320 10767 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:53.089073 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:53.089661 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:53.071898 13074 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:53.073794 13055 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:53.093416 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:53.097162 10874 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 6.
I20260709 13:16:53.098479 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:53.099370 10869 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 6.
I20260709 13:16:53.099378 10874 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 6.
I20260709 13:16:53.100275 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:53.104609 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:53.084199 12775 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:53.089956 13115 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:53.088032 13122 raft_consensus.cc:2749] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:53.109537 10874 raft_consensus.cc:1275] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:16:53.092118 13155 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:53.085072 13046 consensus_queue.cc:1059] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:16:53.109053 13283 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:53.087854 13133 consensus_queue.cc:1048] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:16:53.116314 10767 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:53.122622 10876 raft_consensus.cc:1275] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 2. (index mismatch)
I20260709 13:16:53.123754 10767 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:16:53.113284 10798 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 1450) took 1363 ms (client timeout 1708 ms). Trace:
I20260709 13:16:53.120208 10874 raft_consensus.cc:1275] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:16:53.116024 12990 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:53.125166 10874 pending_rounds.cc:85] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 3
I20260709 13:16:53.125572 10874 pending_rounds.cc:107] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:16:53.125841 10874 raft_consensus.cc:2889] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:16:53.135744 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:53.125072 10798 rpcz_store.cc:269] 0709 13:16:51.750294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:51.750439 (+   145us) service_pool.cc:224] Handling call
0709 13:16:53.113034 (+1362595us) spinlock_profiling.cc:243] Waited 715 ms on lock 0x7b54002125d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:53.113193 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1645613952}
I20260709 13:16:53.136881 10916 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:53.110549 12775 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:53.111080 13115 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:53.113822 13186 raft_consensus.cc:2749] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:53.110225 13337 raft_consensus.cc:2749] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:16:53.150121 10876 raft_consensus.cc:1275] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:16:53.151886 10767 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:53.156348 10767 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:53.141943 13122 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:53.162015 10876 raft_consensus.cc:1275] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:16:53.151381 13074 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:53.158751 13155 raft_consensus.cc:2749] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:16:53.144977 12990 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:53.165504 13122 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:53.196420 13287 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:53.172148 13073 raft_consensus.cc:2749] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:53.191075 10767 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:53.209144 13155 consensus_queue.cc:1048] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:16:53.210047 13328 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:53.210598 13328 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:53.211819 12775 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:53.212785 13328 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:53.213328 13155 consensus_queue.cc:1048] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:16:53.214383 13326 log.cc:927] Time spent T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 13:16:53.215422 13155 consensus_queue.cc:1048] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:53.188071 13122 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:53.186452 13074 raft_consensus.cc:697] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:16:53.220655 13335 log.cc:927] Time spent T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 13:16:53.221776 10798 raft_consensus.cc:1275] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:16:53.216814 13186 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:16:53.231869 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 92472.1us
I20260709 13:16:53.241281 13340 consensus_queue.cc:1048] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:16:53.242662 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:53.255539 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:53.204107 12798 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:53.256311 10876 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 6.
I20260709 13:16:53.233824 13060 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:53.255999 13046 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:53.272012 13046 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:53.273775 10811 tablet_copy_service.cc:342] P e01ecd008edd4aa2b6b2b47efbc2ea87: Request end of tablet copy session 12682eb70bc84c76879e04a8628ee586-171abdebfc4142cda86de477a5c89a7e received from {username='slave'} at 127.0.0.1:39442
I20260709 13:16:53.274327 13046 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:53.274825 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:53.275094 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:53.275467 10795 raft_consensus.cc:2376] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 5. Re-sending same reply.
I20260709 13:16:53.275660 10798 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 4.
I20260709 13:16:53.277606 10797 raft_consensus.cc:1275] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:16:53.282141 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:53.283141 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:53.222194 12994 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:53.269601 10916 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:53.245314 13186 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:53.311414 11478 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:53.329145 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:53.330332 10799 raft_consensus.cc:2393] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 8: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:53.332401 12838 raft_consensus.cc:2749] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:53.333197 12838 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:53.333724 12838 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:53.335358 12775 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:53.164749 13275 raft_consensus.cc:2955] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 2 to 4, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:53.336011 13301 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:53.339162 13292 consensus_queue.cc:1048] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.118s
I20260709 13:16:53.339015 13074 consensus_queue.cc:237] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:53.340469 10870 raft_consensus.cc:1240] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 3. Current term is 4. Ops: []
W20260709 13:16:53.342015 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 83418.1us
I20260709 13:16:53.369742 10767 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:53.274375 10811 tablet_copy_service.cc:434] P e01ecd008edd4aa2b6b2b47efbc2ea87: ending tablet copy session 12682eb70bc84c76879e04a8628ee586-171abdebfc4142cda86de477a5c89a7e on tablet 171abdebfc4142cda86de477a5c89a7e with peer 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:53.311285 13186 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:16:53.371344 13349 log.cc:927] Time spent T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 13:16:53.414800 13196 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:53.415426 13196 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:53.417073 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:53.418269 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:16:53.420154 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 50536.6us
I20260709 13:16:53.422004 10767 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:53.424579 10868 raft_consensus.cc:2376] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 6. Re-sending same reply.
I20260709 13:16:53.425904 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:16:53.430397 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 119099us
I20260709 13:16:53.434222 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:53.409373 13212 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:53.371411 13250 consensus_queue.cc:1048] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:16:53.430475 12775 raft_consensus.cc:2749] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:53.461544 12838 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:53.434222 13290 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:53.463872 13287 consensus_queue.cc:1048] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:53.465368 13306 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:53.465826 13306 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:53.466192 13306 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:53.446391 13073 raft_consensus.cc:2749] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:53.452643 13122 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:53.457705 13184 consensus_meta.cc:287] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.565s	user 0.010s	sys 0.001s
I20260709 13:16:53.458938 13115 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
W20260709 13:16:53.459975 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:53.460210 13055 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:53.438961 13283 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:53.471035 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:53.450668 13212 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:53.462424 13290 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:53.482383 13212 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:53.522882 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:53.523586 10869 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:53.513140 13301 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:53.527700 13312 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:53.528391 13312 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:53.551009 13286 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:53.791208 13301 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.313812 13286 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:54.314626 13301 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:53.784062 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:53.605759 10941 raft_consensus.cc:1275] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:16:54.092592 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 278) took 1469 ms. Trace:
I20260709 13:16:53.669421 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:54.125005 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:53.608186 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:53.606756 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:53.607134 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:16:54.326928 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 657228us
I20260709 13:16:53.608443 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:53.606384 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:54.330495 10947 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 5.
I20260709 13:16:53.521289 13122 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:54.333922 10875 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 1455) took 2459 ms (client timeout 1548 ms). Trace:
I20260709 13:16:54.334649 10946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 1844) took 1432 ms. Trace:
I20260709 13:16:54.321213 10873 rpcz_store.cc:276] 0709 13:16:52.623002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:52.623134 (+   132us) service_pool.cc:224] Handling call
0709 13:16:52.623433 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:52.771959 (+148526us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:16:52.772253 (+   294us) log.cc:844] Serialized 14 byte log entry
0709 13:16:52.814089 (+ 41836us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:52.814305 (+   216us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:52.814375 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:54.092226 (+1277851us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b540006c2d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:54.092303 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:54.092504 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18226,"prepare.queue_time_us":2775,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":55,"raft.queue_time_us":117,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":3622,"spinlock_wait_cycles":341068032,"thread_start_us":41435,"threads_started":1,"wal-append.queue_time_us":57459,"wal-append.run_cpu_time_us":1163,"wal-append.run_wall_time_us":1014630}
I20260709 13:16:53.640626 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:53.607789 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:16:54.356962 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:53.850620 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:54.263476 13347 log.cc:927] Time spent T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.849s	user 0.003s	sys 0.000s
W20260709 13:16:54.134888 13352 log.cc:927] Time spent T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.744s	user 0.001s	sys 0.001s
W20260709 13:16:54.364830 10875 rpcz_store.cc:269] 0709 13:16:51.859733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:51.873557 (+ 13824us) service_pool.cc:224] Handling call
0709 13:16:54.318654 (+2445097us) spinlock_profiling.cc:243] Waited 761 ms on lock 0x7b54000f6ed4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:54.318777 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1751400704}
W20260709 13:16:54.151656 13364 log.cc:927] Time spent T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.465s	user 0.001s	sys 0.001s
W20260709 13:16:54.091307 13353 log.cc:927] Time spent T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.748s	user 0.003s	sys 0.000s
W20260709 13:16:54.363744 10869 consensus_meta.cc:287] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.832s	user 0.008s	sys 0.002s
I20260709 13:16:54.365310 10946 rpcz_store.cc:276] 0709 13:16:52.895692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:52.895841 (+   149us) service_pool.cc:224] Handling call
0709 13:16:52.896132 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:53.053031 (+156899us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:54.197890 (+1144859us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b5400399c54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:54.328531 (+130641us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":355997952}
I20260709 13:16:54.371198 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:54.371762 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:16:53.938867 13331 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.751s	user 0.005s	sys 0.025s
W20260709 13:16:53.592639 13357 log.cc:927] Time spent T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:16:54.366192 11504 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
I20260709 13:16:54.373828 10869 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 7.
I20260709 13:16:54.374392 10941 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:16:53.803984 13134 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:54.338840 13344 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.741s	user 0.008s	sys 0.024s
I20260709 13:16:53.520009 13324 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:54.393114 10868 raft_consensus.cc:1275] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 6. (term mismatch)
I20260709 13:16:54.393298 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:53.936370 13327 raft_consensus.cc:3055] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:53.600255 12990 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:53.932047 13060 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:54.340891 13290 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:54.371625 13328 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.372829 13302 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:53.592756 13358 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:53.600796 13034 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:54.392182 13286 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:54.393664 10868 pending_rounds.cc:85] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 5
I20260709 13:16:54.431646 10868 pending_rounds.cc:107] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 6.6
I20260709 13:16:54.432044 10868 raft_consensus.cc:2889] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:16:53.593968 13296 log.cc:927] Time spent T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 1.068s	user 0.007s	sys 0.000s
I20260709 13:16:54.410032 13060 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:54.396299 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:53.934011 12798 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:54.412437 13355 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.396873 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:54.442152 10873 raft_consensus.cc:2393] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 4: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:54.443055 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:54.397070 13074 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.415263 13105 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:54.391144 13134 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:54.436759 13340 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:54.447854 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:54.459079 13074 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:54.459501 13340 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:54.436589 13313 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.557s	user 0.016s	sys 0.038s
W20260709 13:16:54.461266 13313 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.582s	user 0.017s	sys 0.040s
I20260709 13:16:54.461357 13074 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:54.461593 13373 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:54.462121 13373 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:54.422801 13321 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:53.603516 13115 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:54.409163 10914 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.011s	sys 0.012s
I20260709 13:16:54.463266 13321 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:54.463882 10914 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.071s	user 0.011s	sys 0.013s
I20260709 13:16:54.464417 13373 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:54.465435 13321 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:54.368882 13312 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:54.355538 13257 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:54.466950 13257 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:54.468231 13212 consensus_meta.cc:287] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.948s	user 0.007s	sys 0.003s
I20260709 13:16:54.468662 13212 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:54.469828 13257 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:53.592432 13055 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:54.407467 13274 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.410912 12574 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:54.147629 13280 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:16:54.390270 13331 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.206s	user 0.006s	sys 0.060s
W20260709 13:16:54.387802 13344 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.790s	user 0.009s	sys 0.024s
I20260709 13:16:53.666532 13306 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:54.388676 13289 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:53.930521 13032 raft_consensus.cc:3055] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:54.433837 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:54.396482 13327 raft_consensus.cc:740] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:54.436964 10868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1467) took 1011 ms. Trace:
I20260709 13:16:54.394178 10869 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:54.438479 10875 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:54.494445 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:54.417735 13328 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:53.606225 13186 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:54.455222 10794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1463) took 2422 ms. Trace:
I20260709 13:16:54.477651 10941 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 7.
I20260709 13:16:54.479952 11477 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:54.500695 10876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1481) took 1245 ms. Trace:
I20260709 13:16:54.480739 12574 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:54.483630 10949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 298) took 1395 ms. Trace:
I20260709 13:16:54.483932 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:54.430514 13152 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:54.502494 10949 rpcz_store.cc:276] 0709 13:16:53.088734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:53.088903 (+   169us) service_pool.cc:224] Handling call
0709 13:16:53.103458 (+ 14555us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:53.103759 (+   301us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:53.103784 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:53.114146 (+ 10362us) log.cc:844] Serialized 26 byte log entry
0709 13:16:53.227698 (+113552us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:53.227746 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:53.227820 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:54.262681 (+1034861us) spinlock_profiling.cc:243] Waited 25 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:16:54.471966 (+209285us) raft_consensus.cc:1685] finished
0709 13:16:54.471987 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:54.483557 (+ 11570us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":81640,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":19346,"spinlock_wait_cycles":57405312,"thread_start_us":104985,"threads_started":1,"wal-append.queue_time_us":275980}
I20260709 13:16:54.488576 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:54.499980 10794 rpcz_store.cc:276] 0709 13:16:52.032767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:52.225872 (+193105us) service_pool.cc:224] Handling call
0709 13:16:52.226415 (+   543us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:52.226803 (+   388us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:52.226828 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:52.235482 (+  8654us) log.cc:844] Serialized 564 byte log entry
0709 13:16:52.382561 (+147079us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:52.382619 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:52.382729 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:53.593098 (+1210369us) spinlock_profiling.cc:243] Waited 18.5 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:16:54.454859 (+861761us) raft_consensus.cc:1685] finished
0709 13:16:54.454883 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:54.455136 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42637568,"thread_start_us":111118,"threads_started":1,"wal-append.queue_time_us":206482}
I20260709 13:16:54.505978 11509 raft_consensus.cc:1240] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:54.426757 13302 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:54.449154 13122 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.491346 11504 raft_consensus.cc:2393] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 4: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:54.461390 13340 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:54.415484 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:54.548214 10948 raft_consensus.cc:2393] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 6: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:54.495393 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:54.492306 10868 rpcz_store.cc:276] 0709 13:16:53.422020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:54.336747 (+914727us) service_pool.cc:224] Handling call
0709 13:16:54.376295 (+ 39548us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:54.432686 (+ 56391us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:54.432979 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:54.492216 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:54.548813 10947 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:54.549105 10799 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:54.511049 10869 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 8.
I20260709 13:16:54.514082 10875 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 4.
I20260709 13:16:54.470594 13212 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:54.507606 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:54.518967 13134 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:54.527329 13152 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:54.554458 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:54.481040 13105 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:54.499971 13055 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.556746 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:54.557137 13055 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:54.555670 10794 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 3.
I20260709 13:16:54.487476 13373 consensus_queue.cc:1059] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 13:16:54.532153 12990 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:54.532557 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:54.567430 10796 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:54.492368 13289 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:54.535499 10950 raft_consensus.cc:1275] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:16:54.534986 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:54.501106 10876 rpcz_store.cc:276] 0709 13:16:53.255199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:53.256992 (+  1793us) service_pool.cc:224] Handling call
0709 13:16:53.257276 (+   284us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:53.277050 (+ 19774us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:53.277080 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:53.277494 (+   414us) log.cc:844] Serialized 26 byte log entry
0709 13:16:53.416852 (+139358us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:53.416892 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:54.094302 (+677410us) spinlock_profiling.cc:243] Waited 19.4 ms on lock 0x7b540006bdd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:54.094361 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:54.493233 (+398872us) raft_consensus.cc:1685] finished
0709 13:16:54.493269 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:54.500637 (+  7368us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1023,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":84,"spinlock_wait_cycles":44557056,"thread_start_us":46506,"threads_started":1,"wal-append.queue_time_us":200532}
I20260709 13:16:54.479192 13355 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:54.537199 10767 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:54.501909 13350 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:54.541801 12994 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:54.542359 13324 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:54.543217 13196 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:54.545280 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:54.545322 13258 raft_consensus.cc:3055] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:54.490842 13032 raft_consensus.cc:740] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:54.500195 12996 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.553217 12574 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:54.441983 13358 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:54.555100 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:54.529664 13186 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:54.484882 13274 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:54.556640 13341 log.cc:927] Time spent T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 1.252s	user 0.000s	sys 0.006s
I20260709 13:16:54.531692 13257 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:54.559842 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:54.489996 13280 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:54.563149 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:54.533675 13306 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:54.549299 13122 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:54.576459 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:54.645327 10942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 1843) took 1750 ms. Trace:
I20260709 13:16:54.809299 10943 raft_consensus.cc:1275] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:16:54.902406 11508 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:54.859417 10947 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7.
I20260709 13:16:54.597296 10792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 234) took 1374 ms. Trace:
I20260709 13:16:54.771710 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:54.729882  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:54.588456 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:54.849575 10950 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7.
I20260709 13:16:54.905562 10949 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7.
I20260709 13:16:54.602160 10799 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 4.
I20260709 13:16:54.778324 10948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41014 (request call id 1510) took 1172 ms. Trace:
W20260709 13:16:54.809029 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:54.908031 10796 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 7.
I20260709 13:16:54.696923 10873 raft_consensus.cc:2410] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 6 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:16:54.907112 10948 rpcz_store.cc:276] 0709 13:16:53.606207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:54.334888 (+728681us) service_pool.cc:224] Handling call
0709 13:16:54.778025 (+443137us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b5400174654. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:54.778237 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":400043392}
I20260709 13:16:54.904382 10792 rpcz_store.cc:276] 0709 13:16:53.222988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:53.223097 (+   109us) service_pool.cc:224] Handling call
0709 13:16:53.233090 (+  9993us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:53.367610 (+134520us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:16:53.510181 (+142571us) log.cc:844] Serialized 14 byte log entry
0709 13:16:54.368656 (+858475us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:54.369154 (+   498us) log.cc:844] Serialized 26 byte log entry
0709 13:16:54.369305 (+   151us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:54.371170 (+  1865us) log.cc:844] Serialized 14 byte log entry
0709 13:16:54.552212 (+181042us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:54.552319 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:54.596934 (+ 44615us) raft_consensus.cc:1685] finished
0709 13:16:54.596971 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:54.597197 (+   226us) inbound_call.cc:177] Queueing success response
0709 13:16:54.692548 (+ 95351us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x7b18003f5620. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc8522 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
Metrics: {"mutex_wait_us":717901,"prepare.queue_time_us":164782,"prepare.run_cpu_time_us":105,"prepare.run_wall_time_us":3891,"raft.queue_time_us":821146,"raft.run_cpu_time_us":349,"raft.run_wall_time_us":33766,"spinlock_wait_cycles":31060096,"thread_start_us":528559,"threads_started":4,"wal-append.queue_time_us":987978}
I20260709 13:16:54.916633 10945 raft_consensus.cc:2393] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 4: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:54.552333 13290 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:54.609138 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 46062.8us
I20260709 13:16:54.902333 10942 rpcz_store.cc:276] 0709 13:16:52.895392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:52.895532 (+   140us) service_pool.cc:224] Handling call
0709 13:16:52.915233 (+ 19701us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:52.915550 (+   317us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:52.915579 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:52.941627 (+ 26048us) log.cc:844] Serialized 26 byte log entry
0709 13:16:54.268628 (+1327001us) spinlock_profiling.cc:243] Waited 18.3 ms on lock 0x7b18004623a0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a550d1 00007f9c80e70a0f
0709 13:16:54.491931 (+223303us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:54.491969 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:54.492031 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:54.644842 (+152811us) raft_consensus.cc:1685] finished
0709 13:16:54.644877 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:54.645231 (+   354us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":26576,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":9452,"spinlock_wait_cycles":42099968,"thread_start_us":164444,"threads_started":1,"wal-append.queue_time_us":1684474}
I20260709 13:16:54.950138 10876 raft_consensus.cc:1275] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:16:55.455961 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:55.461099 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:16:55.255275 10766 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.854s	user 0.011s	sys 0.006s
I20260709 13:16:55.132079 13289 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:55.348623 11508 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7.
I20260709 13:16:54.779448 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:54.978982 13324 consensus_queue.cc:1059] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.847s
W20260709 13:16:55.758523 13386 log.cc:927] Time spent T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.155s	user 0.000s	sys 0.002s
W20260709 13:16:55.826877 11505 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56410 (request call id 473) took 1438 ms (client timeout 1715 ms). Trace:
I20260709 13:16:55.805804 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:55.285499 13274 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:54.809145 13337 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:55.026827 13152 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:54.762498 13328 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:55.045863 11504 raft_consensus.cc:2955] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 2 to 4, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
W20260709 13:16:55.455251 10874 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.058s	user 0.009s	sys 0.012s
W20260709 13:16:55.830544 10874 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.433s	user 0.018s	sys 0.022s
I20260709 13:16:54.976056 13290 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:55.346942 10871 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.941s	user 0.001s	sys 0.017s
W20260709 13:16:55.831663 10871 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.425s	user 0.006s	sys 0.043s
I20260709 13:16:54.950614 13375 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:55.832549 10871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1463) took 2544 ms. Trace:
I20260709 13:16:55.737421 13397 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:55.831529 11507 raft_consensus.cc:1240] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:55.225570 12994 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:55.833083 13375 raft_consensus.cc:697] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:55.787534 13380 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.926358 13350 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:55.839154 13397 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:55.261505 13344 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.727s	user 0.009s	sys 0.003s
W20260709 13:16:55.539762 13331 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.038s	user 0.011s	sys 0.017s
I20260709 13:16:55.522300 13280 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:16:55.843050 13344 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.327s	user 0.017s	sys 0.021s
W20260709 13:16:55.843159 13331 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.344s	user 0.017s	sys 0.027s
I20260709 13:16:55.843510 13280 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:16:55.340011 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:55.776114 13395 log.cc:927] Time spent T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
I20260709 13:16:54.927340 12996 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:55.844183 13280 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:55.011010 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:55.845799 11511 raft_consensus.cc:2393] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 5: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
W20260709 13:16:55.354918 10764 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.855s	user 0.000s	sys 0.017s
I20260709 13:16:55.846290 13280 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:55.020403 13032 consensus_queue.cc:260] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:54.598313 13105 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:55.625253 13358 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:54.640092 13321 raft_consensus.cc:2955] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 4 LEADER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:54.977898 13306 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.689535 13034 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
W20260709 13:16:55.467973 10801 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.004s	sys 0.017s
W20260709 13:16:55.849808 10801 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.019s	user 0.008s	sys 0.023s
I20260709 13:16:54.580996 12990 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:54.962225 10916 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:16:55.255095 10870 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.824s	user 0.008s	sys 0.008s
W20260709 13:16:55.852865 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 890699us
W20260709 13:16:55.532039 10798 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.013s	user 0.005s	sys 0.022s
W20260709 13:16:55.853758 10798 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.335s	user 0.017s	sys 0.029s
I20260709 13:16:55.854557 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:55.846743 13397 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:55.261492 10872 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.449s	user 0.000s	sys 0.018s
W20260709 13:16:55.856830 10872 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.052s	user 0.011s	sys 0.040s
I20260709 13:16:55.839728 13375 consensus_queue.cc:237] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:55.857821 10872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1458) took 2577 ms. Trace:
I20260709 13:16:54.736266 13301 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:54.950067 13257 raft_consensus.cc:697] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 8 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:54.660540 13286 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:55.841672 12994 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:55.791581 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 212235us
I20260709 13:16:55.839367 13380 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:55.064893 13122 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:16:55.825464 10766 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.424s	user 0.027s	sys 0.022s
I20260709 13:16:54.620651 13134 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:55.853106 10870 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.422s	user 0.008s	sys 0.035s
W20260709 13:16:55.826570 11508 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56410 (request call id 472) took 1449 ms (client timeout 1759 ms). Trace:
I20260709 13:16:55.935645 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:55.853925 12990 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:55.009442 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:16:55.285074 10795 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.763s	user 0.003s	sys 0.011s
I20260709 13:16:55.935662 10870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1466) took 2514 ms. Trace:
W20260709 13:16:55.936803 10795 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.415s	user 0.008s	sys 0.024s
I20260709 13:16:55.937918 10795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 245) took 1431 ms. Trace:
I20260709 13:16:55.821606 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:55.041330 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:55.007627 13055 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:16:55.255019 13288 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.735s	user 0.001s	sys 0.016s
I20260709 13:16:55.855808 10798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 246) took 1338 ms. Trace:
W20260709 13:16:55.827656 11505 rpcz_store.cc:269] 0709 13:16:54.388367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:54.394009 (+  5642us) service_pool.cc:224] Handling call
0709 13:16:54.779166 (+385157us) spinlock_profiling.cc:243] Waited 4.03 ms on lock 0x7b5800003c98. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd90a22 00007f9c86ca9c09 00007f9c85e03ee0 00007f9c85ecd0ab 00007f9c85ecd131 00007f9c85e7b0a0 00007f9c85e7a984 00007f9c7f3f56d4 00007f9c7f3f5662
0709 13:16:55.826814 (+1047648us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9276672}
W20260709 13:16:54.926968 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 389791us
I20260709 13:16:55.860352 10874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1462) took 2572 ms. Trace:
I20260709 13:16:55.828337 13337 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:55.261586 10842 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.004s	sys 0.008s
W20260709 13:16:55.370957 10867 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.831s	user 0.002s	sys 0.015s
W20260709 13:16:55.522735 10765 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.132s	user 0.001s	sys 0.018s
W20260709 13:16:55.380492 10800 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.883s	user 0.006s	sys 0.009s
W20260709 13:16:55.846809 10764 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.447s	user 0.004s	sys 0.039s
I20260709 13:16:55.303335 13355 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:55.868785 10801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 250) took 1335 ms. Trace:
I20260709 13:16:55.841394 13350 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:55.858294 10872 rpcz_store.cc:276] 0709 13:16:53.280631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:53.482227 (+201596us) service_pool.cc:224] Handling call
0709 13:16:53.484370 (+  2143us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:53.488051 (+  3681us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:53.488105 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:53.501183 (+ 13078us) log.cc:844] Serialized 26 byte log entry
0709 13:16:53.679971 (+178788us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:16:53.804933 (+124962us) log.cc:844] Serialized 14 byte log entry
0709 13:16:55.857346 (+2052413us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:55.857452 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:55.857525 (+    73us) raft_consensus.cc:1685] finished
0709 13:16:55.857546 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:55.857719 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":125479,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":310983,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":3651,"thread_start_us":2207456,"threads_started":2,"wal-append.queue_time_us":793925,"wal-append.run_cpu_time_us":2086,"wal-append.run_wall_time_us":1028963}
I20260709 13:16:55.833037 10871 rpcz_store.cc:276] 0709 13:16:53.288370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:53.917558 (+629188us) service_pool.cc:224] Handling call
0709 13:16:53.918040 (+   482us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:54.341380 (+423340us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:54.341412 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:54.341870 (+   458us) log.cc:844] Serialized 26 byte log entry
0709 13:16:55.832223 (+1490353us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:55.832263 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:55.832331 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:55.832368 (+    37us) raft_consensus.cc:1685] finished
0709 13:16:55.832379 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:55.832492 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":197692,"raft.run_cpu_time_us":384,"raft.run_wall_time_us":4011,"thread_start_us":1843876,"threads_started":2,"wal-append.queue_time_us":1405635}
I20260709 13:16:54.598737 13212 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.827808 13302 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:55.062350 11503 raft_consensus.cc:1240] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:54.771652 13348 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:16:55.644972 13060 consensus_meta.cc:287] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 1.166s	user 0.014s	sys 0.000s
I20260709 13:16:55.917184 11511 raft_consensus.cc:1240] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:16:54.901620 13258 raft_consensus.cc:740] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:54.693758 13312 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:55.855268 13331 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:55.919332 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:55.848590 13306 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:55.833181 13290 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:55.916007 13407 log.cc:927] Time spent T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:16:55.903203 13328 raft_consensus.cc:2749] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:55.846302 12996 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:55.925390 10948 raft_consensus.cc:1275] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:16:55.928006 10876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 294) took 1516 ms. Trace:
I20260709 13:16:55.849442 13034 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:55.938432 10795 rpcz_store.cc:276] 0709 13:16:54.506675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:54.520207 (+ 13532us) service_pool.cc:224] Handling call
0709 13:16:54.520566 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:54.520956 (+   390us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:16:54.520988 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:54.521505 (+   517us) log.cc:844] Serialized 26 byte log entry
0709 13:16:55.937440 (+1415935us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:55.937496 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:55.937593 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:55.937655 (+    62us) raft_consensus.cc:1685] finished
0709 13:16:55.937673 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:55.937841 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":359,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":102,"thread_start_us":1415459,"threads_started":1,"wal-append.queue_time_us":1078648}
W20260709 13:16:55.935838 11508 rpcz_store.cc:269] 0709 13:16:54.377586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:54.392104 (+ 14518us) service_pool.cc:224] Handling call
0709 13:16:54.545109 (+153005us) spinlock_profiling.cc:243] Waited 3.73 ms on lock 0x7b5800003c98. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd90a22 00007f9c86ca9c09 00007f9c85e03ee0 00007f9c85ecd0ab 00007f9c85ecd131 00007f9c85e7b0a0 00007f9c85e7a984 00007f9c7f3f56d4 00007f9c7f3f5662
0709 13:16:55.826497 (+1281388us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8570240}
I20260709 13:16:55.937170 10870 rpcz_store.cc:276] 0709 13:16:53.421728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:54.328833 (+907105us) service_pool.cc:224] Handling call
0709 13:16:54.329177 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:54.329593 (+   416us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:54.329621 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:54.369001 (+ 39380us) log.cc:844] Serialized 26 byte log entry
0709 13:16:55.935325 (+1566324us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:55.935365 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:55.935433 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:55.935473 (+    40us) raft_consensus.cc:1685] finished
0709 13:16:55.935485 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:55.935603 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":91,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":38164,"thread_start_us":1503992,"threads_started":1,"wal-append.queue_time_us":1211596}
I20260709 13:16:55.949213 10874 rpcz_store.cc:276] 0709 13:16:53.288071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:53.552279 (+264208us) service_pool.cc:224] Handling call
0709 13:16:53.552716 (+   437us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:53.553147 (+   431us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:53.553187 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:54.328541 (+775354us) log.cc:844] Serialized 26 byte log entry
0709 13:16:55.830997 (+1502456us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:55.831043 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:55.831131 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:55.860118 (+ 28987us) raft_consensus.cc:1685] finished
0709 13:16:55.860140 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:55.860283 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":611,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":27975,"thread_start_us":1433826,"threads_started":1,"wal-append.queue_time_us":1483585}
W20260709 13:16:55.371196 10868 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.004s	sys 0.013s
I20260709 13:16:55.945174 10798 rpcz_store.cc:276] 0709 13:16:54.517508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:54.517662 (+   154us) service_pool.cc:224] Handling call
0709 13:16:54.518038 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:54.518439 (+   401us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:54.518466 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:54.518944 (+   478us) log.cc:844] Serialized 26 byte log entry
0709 13:16:55.855409 (+1336465us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:55.855452 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:55.855517 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:55.855564 (+    47us) raft_consensus.cc:1685] finished
0709 13:16:55.855578 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:55.855742 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2641,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":101,"thread_start_us":1335756,"threads_started":1,"wal-append.queue_time_us":1322699}
W20260709 13:16:55.954063 10842 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.310s	user 0.017s	sys 0.014s
I20260709 13:16:55.946861 13186 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:56.007902 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:56.008108 11512 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
W20260709 13:16:56.008826 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 1.36483e+06us
I20260709 13:16:55.966915 13403 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
W20260709 13:16:55.954425 10867 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.415s	user 0.009s	sys 0.022s
I20260709 13:16:55.966748 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:55.970042 10875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 289) took 1597 ms. Trace:
I20260709 13:16:55.954936 13289 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:55.952363 13385 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:54.581250 13327 consensus_queue.cc:260] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:55.944166 13286 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:55.950088 13288 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.430s	user 0.013s	sys 0.030s
I20260709 13:16:55.972361 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:55.991313 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:55.979655 10801 rpcz_store.cc:276] 0709 13:16:54.533540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:54.552586 (+ 19046us) service_pool.cc:224] Handling call
0709 13:16:54.568293 (+ 15707us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:54.623737 (+ 55444us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:54.623784 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:54.668659 (+ 44875us) log.cc:844] Serialized 52 byte log entry
0709 13:16:55.850302 (+1181643us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:55.850645 (+   343us) log.cc:844] Serialized 14 byte log entry
0709 13:16:55.851004 (+   359us) log.cc:844] Serialized 14 byte log entry
0709 13:16:55.851168 (+   164us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:55.851244 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:55.868533 (+ 17289us) raft_consensus.cc:1685] finished
0709 13:16:55.868555 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:55.868719 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3417,"prepare.queue_time_us":618,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":61,"raft.queue_time_us":112377,"raft.run_cpu_time_us":416,"raft.run_wall_time_us":84535,"spinlock_wait_cycles":3603840,"thread_start_us":1019452,"threads_started":1,"wal-append.queue_time_us":1102408}
I20260709 13:16:55.955560 13105 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:55.951370 13381 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:55.943946 13301 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:55.967304 13134 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:55.954502 10765 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.564s	user 0.002s	sys 0.019s
I20260709 13:16:55.946823 13397 consensus_queue.cc:1059] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
W20260709 13:16:55.979790 10800 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.482s	user 0.014s	sys 0.011s
I20260709 13:16:56.002928 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:16:56.147737 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:56.002496 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:56.122709 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:56.217501 10796 raft_consensus.cc:2410] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:16:56.157884 10869 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:56.218801 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:56.219707 10874 raft_consensus.cc:2410] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 7 because replica has last-logged OpId of term: 6 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 13:16:56.220118 10868 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.396s	user 0.012s	sys 0.028s
I20260709 13:16:56.110558 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:56.193125 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:56.224565 10792 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
I20260709 13:16:56.082520 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:56.127992 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:56.231565 13289 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:56.234128 13289 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:56.225795 10871 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:56.238473 10875 rpcz_store.cc:276] 0709 13:16:54.372380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:54.807972 (+435592us) service_pool.cc:224] Handling call
0709 13:16:54.808442 (+   470us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:55.260405 (+451963us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:55.260453 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:55.261080 (+   627us) log.cc:844] Serialized 26 byte log entry
0709 13:16:55.928890 (+667810us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:55.928948 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:55.929023 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:55.969781 (+ 40758us) raft_consensus.cc:1685] finished
0709 13:16:55.969810 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:55.969978 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":852907,"spinlock_wait_cycles":32875904,"thread_start_us":525335,"threads_started":2,"wal-append.queue_time_us":662456}
I20260709 13:16:56.235450 10870 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 5.
I20260709 13:16:56.214179 10876 rpcz_store.cc:276] 0709 13:16:54.411166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:54.974415 (+563249us) service_pool.cc:224] Handling call
0709 13:16:55.454477 (+480062us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:55.454853 (+   376us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:16:55.791961 (+337108us) log.cc:844] Serialized 14 byte log entry
0709 13:16:55.927493 (+135532us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:55.927573 (+    80us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:55.927660 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:55.927766 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:55.927928 (+   162us) inbound_call.cc:177] Queueing success response
0709 13:16:56.149284 (+221356us) spinlock_profiling.cc:243] Waited 14.1 ms on lock 0x7b18000a3780. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7dd962a6 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc83f3 00007f9c7ddca7c1
Metrics: {"prepare.queue_time_us":373917,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":1260,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":14815,"spinlock_wait_cycles":53876480,"thread_start_us":382717,"threads_started":2,"wal-append.queue_time_us":137423}
I20260709 13:16:55.966684 13383 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:56.014802 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 974737us
I20260709 13:16:56.163095 10841 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:56.273053 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:16:56.284444 10842 connection.cc:256] RPC call timeout handler was delayed by -1.3136684389999118s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20320000000000002s for a response
W20260709 13:16:56.285871 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:56.287271 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 124270us
I20260709 13:16:56.243600 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:16:55.996757 10916 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.116.129:39685 timed out after 1.708s (SENT) [suppressed 16 similar messages]
W20260709 13:16:56.290256 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 293536us
I20260709 13:16:56.265966 10871 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 6.
I20260709 13:16:55.989181 13122 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:16:56.276630 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 308878us
I20260709 13:16:56.280660 11478 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:56.295925 10766 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:56.002723 12994 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:56.081382 13290 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:56.343508 10868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 287) took 2020 ms. Trace:
W20260709 13:16:56.343245 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 47442.1us
I20260709 13:16:56.104460 13344 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:56.366073 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:56.301553 10950 raft_consensus.cc:1240] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:56.301563 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:56.242662 11503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56404 (request call id 507) took 1145 ms. Trace:
I20260709 13:16:56.305095 10869 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 7.
I20260709 13:16:56.153290 13274 raft_consensus.cc:3055] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:56.369742 10947 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 6.
I20260709 13:16:56.370724 13415 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:56.301872 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:16:56.151378 13402 log.cc:927] Time spent T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
I20260709 13:16:56.010430 11506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56404 (request call id 505) took 1013 ms. Trace:
I20260709 13:16:56.147302 13032 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:55.997362 13411 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:56.402172 13411 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:56.403208 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:56.404510 13411 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:56.391407 11506 rpcz_store.cc:276] 0709 13:16:54.997384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:54.998062 (+   678us) service_pool.cc:224] Handling call
0709 13:16:56.002872 (+1004810us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:56.388738 13350 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:56.406241 13350 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:56.407765 13337 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:56.381479 13327 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:56.409495 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:56.410487 10792 raft_consensus.cc:2393] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 6: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:56.411586 10801 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:16:56.367578 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:56.376678 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:56.377435 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:56.404523 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:55.994378 13355 consensus_queue.cc:1048] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.353s
I20260709 13:16:56.271579 13105 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:56.374751 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:16:56.308971 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:56.380487 10916 connection.cc:256] RPC call timeout handler was delayed by -1.7636348629998793s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15880000000000002s for a response
W20260709 13:16:56.424062 10916 connection.cc:256] RPC call timeout handler was delayed by -1.7087348630000179s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16490000000000002s for a response
W20260709 13:16:56.424655 10916 connection.cc:256] RPC call timeout handler was delayed by -1.2519700149998698s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1511s for a response
W20260709 13:16:56.425050 10916 connection.cc:256] RPC call timeout handler was delayed by -1.1209457329998713s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1917s for a response
W20260709 13:16:56.385330 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 97006.1us
I20260709 13:16:56.344833 13373 raft_consensus.cc:2749] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:56.081001 13331 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:56.083561 13060 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:56.342862 13289 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:56.005180 13212 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:56.085465 13186 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:56.273726 13385 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:56.397923 13306 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:56.369331 13290 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:56.377110 13344 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:56.398814 13134 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:56.399977 13034 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:56.401436 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:56.408664 13350 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:56.364523 10868 rpcz_store.cc:276] 0709 13:16:54.323293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:54.561644 (+238351us) service_pool.cc:224] Handling call
0709 13:16:54.599497 (+ 37853us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:54.603519 (+  4022us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:54.603556 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:54.684769 (+ 81213us) log.cc:844] Serialized 52 byte log entry
0709 13:16:56.220521 (+1535752us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:56.292967 (+ 72446us) log.cc:844] Serialized 14 byte log entry
0709 13:16:56.343139 (+ 50172us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:56.343229 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:56.343270 (+    41us) raft_consensus.cc:1685] finished
0709 13:16:56.343283 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:56.343439 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53531,"prepare.queue_time_us":74493,"raft.queue_time_us":21151,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":37949,"spinlock_wait_cycles":576256,"thread_start_us":1395969,"threads_started":1,"wal-append.queue_time_us":915112}
I20260709 13:16:56.415370 10794 raft_consensus.cc:2410] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:16:56.414333 10945 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
I20260709 13:16:56.423288 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:56.428277 10841 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:56.376523 13274 raft_consensus.cc:740] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:56.273501 13312 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:56.380143 12994 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:56.460238 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:16:56.460623 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 348080us
W20260709 13:16:56.461146 10916 connection.cc:256] RPC call timeout handler was delayed by -1.033734862999836s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2399s for a response
W20260709 13:16:56.453718 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 173139us
I20260709 13:16:56.386445 13414 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:16:56.454200 10842 connection.cc:256] RPC call timeout handler was delayed by -1.2551684389998172s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2097s for a response
I20260709 13:16:56.460381 10800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 243) took 1974 ms. Trace:
I20260709 13:16:56.271790 13301 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:56.082444 13257 consensus_queue.cc:237] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:56.346868 13122 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:56.395198 13055 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:56.277360 13258 consensus_queue.cc:260] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:56.085235 13348 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:56.297207 13383 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:56.277213 13403 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:56.295641 13381 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:56.609205 10875 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:16:56.574997 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 486320us
I20260709 13:16:56.451534 13375 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:16:56.522914 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 157993us
I20260709 13:16:56.379758 13415 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:56.606993 10793 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
W20260709 13:16:56.611864 10842 connection.cc:256] RPC call timeout handler was delayed by -1.1858684389999326s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2174s for a response
I20260709 13:16:56.523432 10941 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7.
I20260709 13:16:56.612654 11478 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:56.489243 10943 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:56.614706 11504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56404 (request call id 506) took 1588 ms. Trace:
I20260709 13:16:56.616187 10795 raft_consensus.cc:2393] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 6: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:56.616576 10766 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:16:56.611305 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 183101us
I20260709 13:16:56.452858 13328 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:16:56.621294 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:56.690943 13290 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:56.623595 13337 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:56.691401 11504 rpcz_store.cc:276] 0709 13:16:55.026266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.026491 (+   225us) service_pool.cc:224] Handling call
0709 13:16:55.045415 (+ 18924us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:55.045710 (+   295us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:16:55.871032 (+825322us) log.cc:844] Serialized 14 byte log entry
0709 13:16:56.614228 (+743196us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:56.614291 (+    63us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:56.614374 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:56.614463 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:56.614623 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":104228,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":25113,"thread_start_us":694846,"threads_started":2,"wal-append.queue_time_us":566291}
I20260709 13:16:56.694410 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:56.690361 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:56.613108 10800 rpcz_store.cc:276] 0709 13:16:54.486334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:54.486469 (+   135us) service_pool.cc:224] Handling call
0709 13:16:54.497349 (+ 10880us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:54.497540 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:16:56.218068 (+1720528us) log.cc:844] Serialized 14 byte log entry
0709 13:16:56.459518 (+241450us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:56.459590 (+    72us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:56.459680 (+    90us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:56.459788 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:56.460314 (+   526us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":195052,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":59674,"thread_start_us":1961126,"threads_started":3,"wal-append.queue_time_us":251050}
I20260709 13:16:56.701049 10795 raft_consensus.cc:2393] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 6: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:56.701810 10794 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 7.
I20260709 13:16:56.702724 10792 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 3.
I20260709 13:16:56.378824 11503 rpcz_store.cc:276] 0709 13:16:55.039606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.041090 (+  1484us) service_pool.cc:224] Handling call
0709 13:16:55.062046 (+ 20956us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:56.083422 (+1021376us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:56.185325 (+101903us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:56.711653 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:56.712108 10875 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 7.
I20260709 13:16:56.715638 10801 raft_consensus.cc:1275] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:16:56.716157 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:56.716967 10875 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 7.
I20260709 13:16:56.718020 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:16:56.714941 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:56.713982 10872 raft_consensus.cc:2410] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 13:16:56.719385 10842 connection.cc:256] RPC call timeout handler was delayed by -1.1507684389998758s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22130000000000002s for a response
I20260709 13:16:56.720468 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:16:56.719880 10842 connection.cc:256] RPC call timeout handler was delayed by -1.1453684389998671s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2219s for a response
I20260709 13:16:56.714946 10797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 244) took 2223 ms. Trace:
I20260709 13:16:56.621084 13152 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:16:56.733424 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 119229us
I20260709 13:16:56.718712 10875 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:56.777871 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:56.723009 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:56.696167 13380 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:56.621093 13055 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:56.800063 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 187491us
W20260709 13:16:56.781111 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:56.792070 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 96882.7us
I20260709 13:16:56.621364 12994 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:56.489641 13373 consensus_queue.cc:1048] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.235s
I20260709 13:16:56.777108 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:56.482450 13289 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:56.706975 13286 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:56.776590 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:56.807845 10945 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7.
I20260709 13:16:56.721077 10868 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 4.
I20260709 13:16:56.795498 10801 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 6.
I20260709 13:16:56.722534 10797 rpcz_store.cc:276] 0709 13:16:54.496125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:54.496275 (+   150us) service_pool.cc:224] Handling call
0709 13:16:54.515355 (+ 19080us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:55.453399 (+938044us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:55.453430 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:55.453900 (+   470us) log.cc:844] Serialized 26 byte log entry
0709 13:16:55.793051 (+339151us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:55.793109 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:55.875041 (+ 81932us) spinlock_profiling.cc:243] Waited 445 ms on lock 0x7b54000d7354. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:55.875110 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:56.522634 (+647524us) raft_consensus.cc:1685] finished
0709 13:16:56.522662 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:56.714827 (+192165us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":506936,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":32191,"spinlock_wait_cycles":1023501056,"thread_start_us":185855,"threads_started":1,"wal-append.queue_time_us":413172}
W20260709 13:16:56.722124 10842 connection.cc:256] RPC call timeout handler was delayed by -1.1300684389998423s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22360000000000002s for a response
W20260709 13:16:56.810468 10842 connection.cc:256] RPC call timeout handler was delayed by -1.0479249610000352s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1715s for a response
W20260709 13:16:56.803620 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:56.805277 10875 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 4.
I20260709 13:16:56.701637 13328 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:56.706883 13134 consensus_queue.cc:1048] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.087s
I20260709 13:16:56.814304 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:56.816607 10875 raft_consensus.cc:2410] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:16:56.621627 13414 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:56.828509 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:56.793710 13152 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:16:56.846169 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:56.874776 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:56.808432 13290 raft_consensus.cc:2749] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:16:56.878924 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:16:56.879683 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 258125us
I20260709 13:16:56.955675 10870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1487) took 1025 ms. Trace:
I20260709 13:16:56.951753 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:56.883769 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:56.919613 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:16:56.950980 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 167394us
I20260709 13:16:56.995296 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:57.013363 10801 raft_consensus.cc:2393] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 4: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
W20260709 13:16:56.997386 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 380854us
I20260709 13:16:57.015033 10766 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:57.015833 10841 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:16:56.998608 10793 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:57.004103 13060 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:56.998356 11478 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:16:57.021238 11478 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:57.022279 13430 raft_consensus.cc:2804] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:16:56.994923 13416 log.cc:927] Time spent T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.392s	user 0.000s	sys 0.001s
I20260709 13:16:56.998050 10870 rpcz_store.cc:276] 0709 13:16:55.930255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.954523 (+1024268us) service_pool.cc:224] Handling call
0709 13:16:56.954949 (+   426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:56.955194 (+   245us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:56.955234 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:56.955292 (+    58us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:56.955359 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:56.955432 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:56.955581 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:57.025965 10870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1489) took 1077 ms. Trace:
W20260709 13:16:57.027136 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:57.026479 10870 rpcz_store.cc:276] 0709 13:16:55.948431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:57.024866 (+1076435us) service_pool.cc:224] Handling call
0709 13:16:57.025263 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:57.025498 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:57.025539 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:57.025599 (+    60us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:57.025662 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:57.025732 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:57.025892 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:57.030407 10870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1492) took 1079 ms. Trace:
I20260709 13:16:57.031031 10870 rpcz_store.cc:276] 0709 13:16:55.950696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:57.029240 (+1078544us) service_pool.cc:224] Handling call
0709 13:16:57.029662 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:57.029912 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:57.029950 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:57.030010 (+    60us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:57.030070 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:57.030146 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:57.030326 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:57.035774 11478 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:16:57.007793 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 55317.3us
I20260709 13:16:57.037832 10841 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:56.876263 13152 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:57.040560 10841 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:57.042752 13290 raft_consensus.cc:2749] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:57.047269 10870 raft_consensus.cc:1240] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:57.049085 10870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1493) took 1098 ms. Trace:
I20260709 13:16:57.050499 10870 rpcz_store.cc:276] 0709 13:16:55.951080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:57.033382 (+1082302us) service_pool.cc:224] Handling call
0709 13:16:57.046517 (+ 13135us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:57.048815 (+  2298us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:57.049027 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:57.063076 10793 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
I20260709 13:16:57.071159 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:57.071301 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:57.072114 10945 raft_consensus.cc:2410] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:16:56.988540 13427 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:56.995500 13105 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:57.045383 13032 consensus_meta.cc:287] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.647s	user 0.009s	sys 0.000s
I20260709 13:16:57.041360 13034 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:57.054697 10870 raft_consensus.cc:2955] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:56.987845 13426 raft_consensus.cc:2749] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:16:57.048539 13134 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:16:57.086310 11478 connection.cc:256] RPC call timeout handler was delayed by -1.1866444330000832s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1748s for a response
I20260709 13:16:57.091777 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:56.932257 13424 consensus_queue.cc:1059] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.352s
W20260709 13:16:57.091928 11478 connection.cc:256] RPC call timeout handler was delayed by -1.0426444330000777s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1908s for a response
I20260709 13:16:57.039390 13430 raft_consensus.cc:697] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 4 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:57.096575 13312 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:57.096845 10801 raft_consensus.cc:1275] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:16:56.987829 13423 consensus_queue.cc:1048] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.263s
I20260709 13:16:57.086529 13034 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:57.099939 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:57.104303 13034 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:57.104629 10800 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
I20260709 13:16:57.105706 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:57.106406 10800 raft_consensus.cc:2393] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 7: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:57.107218 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:57.107827 10800 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
I20260709 13:16:57.107456 13430 consensus_queue.cc:237] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:57.098567 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:57.110280 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:57.111371 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:16:57.091114 13428 log.cc:927] Time spent T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 13:16:57.098150 13105 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:57.087256 13280 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:57.108785 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:57.112571 13105 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:57.113195 10800 raft_consensus.cc:2393] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 6: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:57.112911 13280 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:57.114039 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:57.107883 10875 raft_consensus.cc:1240] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:57.112341 10797 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
I20260709 13:16:56.987861 13431 consensus_queue.cc:1048] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.563s
I20260709 13:16:57.101951 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:57.103327 10798 raft_consensus.cc:1240] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 LEADER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:16:57.076805 13419 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:57.101092 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1464) took 3798 ms. Trace:
I20260709 13:16:57.091516 13306 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:57.114578 10800 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
I20260709 13:16:57.083465 13290 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:57.111692 13438 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:57.103894 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:57.152841 10867 rpcz_store.cc:276] 0709 13:16:53.303136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:54.093401 (+790265us) service_pool.cc:224] Handling call
0709 13:16:54.093850 (+   449us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:54.491087 (+397237us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:54.491118 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:54.492734 (+  1616us) log.cc:844] Serialized 26 byte log entry
0709 13:16:56.220992 (+1728258us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:56.221034 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:57.100680 (+879646us) spinlock_profiling.cc:243] Waited 397 ms on lock 0x7b5400077454. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:57.100740 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:57.100786 (+    46us) raft_consensus.cc:1685] finished
0709 13:16:57.100800 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:57.101022 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":585,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":7929,"spinlock_wait_cycles":912338304,"thread_start_us":1681317,"threads_started":1,"wal-append.queue_time_us":1112174,"wal-append.run_cpu_time_us":5619,"wal-append.run_wall_time_us":1258042}
I20260709 13:16:57.156936 10795 raft_consensus.cc:2393] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 7: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:57.135799 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1485) took 1273 ms. Trace:
I20260709 13:16:57.047286 13432 consensus_queue.cc:1048] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:16:57.161118 10950 raft_consensus.cc:1240] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:57.095793 11478 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:16:57.104751 10869 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 1472) took 2568 ms (client timeout 1658 ms). Trace:
I20260709 13:16:57.072891 13212 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:57.121810 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:57.111310 13385 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:57.084545 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 69585.2us
I20260709 13:16:57.214401 10766 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:57.156482 13438 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:56.987807 13420 consensus_queue.cc:1048] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.292s
I20260709 13:16:57.217707 13420 raft_consensus.cc:2749] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:16:57.114722 13105 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:57.084534 13288 raft_consensus.cc:2749] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:57.120239 13122 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:57.103453 13348 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:57.098942 13055 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:57.154563 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:57.154887 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:57.157752 10873 rpcz_store.cc:276] 0709 13:16:55.862357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.862510 (+   153us) service_pool.cc:224] Handling call
0709 13:16:55.862859 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:55.863114 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:16:55.919261 (+ 56147us) log.cc:844] Serialized 14 byte log entry
0709 13:16:56.795031 (+875770us) log.cc:844] Serialized 14 byte log entry
0709 13:16:57.135212 (+340181us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:57.135272 (+    60us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:57.135345 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:57.135432 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:57.135725 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13142,"prepare.queue_time_us":26061,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":1780,"raft.queue_time_us":47983,"raft.run_cpu_time_us":7822,"raft.run_wall_time_us":149803,"spinlock_wait_cycles":27062656,"thread_start_us":1214715,"threads_started":4,"wal-append.queue_time_us":105572}
I20260709 13:16:57.241186 10798 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 5: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:57.099367 13403 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:56.968739 13274 consensus_queue.cc:260] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:57.090408 13134 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:57.208400 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:57.213356 13415 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:57.216441 13438 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:57.152109 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:16:57.163158 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 156601us
W20260709 13:16:57.212246 10869 rpcz_store.cc:269] 0709 13:16:54.536147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:54.559586 (+ 23439us) service_pool.cc:224] Handling call
0709 13:16:57.104513 (+2544927us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54000f64d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:57.104649 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3010207488}
I20260709 13:16:57.229171 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:16:57.248371 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 169365us
I20260709 13:16:57.252951 10800 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
I20260709 13:16:57.097959 13437 raft_consensus.cc:2749] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:57.234371 10766 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:57.164000 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:16:57.236075 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 140341us
W20260709 13:16:57.239652 10943 consensus_meta.cc:287] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.549s	user 0.010s	sys 0.000s
I20260709 13:16:57.241623 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:57.209474 13409 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:57.089258 13427 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:57.253225 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:57.155462 13419 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:57.157706 13327 consensus_meta.cc:287] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.748s	user 0.007s	sys 0.003s
I20260709 13:16:57.251809 10801 raft_consensus.cc:2393] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 8: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:57.255242 10795 raft_consensus.cc:2393] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 5: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:57.255465 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:57.255501 10943 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 7.
I20260709 13:16:57.257361 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:57.255712 10873 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 7.
I20260709 13:16:57.155047 13331 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:57.277837 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:57.284322 11508 raft_consensus.cc:2376] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7. Re-sending same reply.
I20260709 13:16:57.280448 10869 raft_consensus.cc:2410] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:16:57.278923 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:57.281906 10873 raft_consensus.cc:1275] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:16:57.282604 10793 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 6.
W20260709 13:16:57.286211 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:57.287194 11512 raft_consensus.cc:1240] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:57.285979 10948 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 3.
I20260709 13:16:57.245498 13086 raft_consensus.cc:2804] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:57.289727 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:57.289877 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:57.256415 13409 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:57.291554 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:57.293777 11507 raft_consensus.cc:2393] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 6: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
W20260709 13:16:57.249866 13425 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.007s	sys 0.003s
I20260709 13:16:57.297032 10798 raft_consensus.cc:1240] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:57.304092 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:57.304982 13415 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:16:57.305292 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 71001us
I20260709 13:16:57.161624 13306 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:57.306330 10766 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:57.307318 13437 raft_consensus.cc:2749] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:57.282435 10916 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:57.298732 10800 raft_consensus.cc:2410] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:16:57.299558 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 43829.8us
I20260709 13:16:57.210807 13432 consensus_queue.cc:1243] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer 468a8fd46e824f62b9c2c0d3532a6d26 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:16:57.302597 10795 raft_consensus.cc:1240] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:16:57.297850 10797 raft_consensus.cc:2410] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:16:57.307911 10766 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:57.257467 13344 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:57.210590 13290 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:57.285465 13274 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:57.293159 13086 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:57.317422 10916 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:57.309964 11478 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:57.312884 10801 raft_consensus.cc:1275] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 6. (term mismatch)
I20260709 13:16:57.251387 13381 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:57.324756 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:57.325430 10798 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 3.
I20260709 13:16:57.326309 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:57.302351 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:57.326902 10798 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 6.
I20260709 13:16:57.327136 13452 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:57.314082 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:57.332788 13409 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:57.322713 10801 pending_rounds.cc:85] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 5
I20260709 13:16:57.323288 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:57.337055 10801 pending_rounds.cc:107] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 6.6
I20260709 13:16:57.337396 10795 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 5.
I20260709 13:16:57.323801 10766 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:57.256340 13134 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:16:57.345844 13337 consensus_meta.cc:287] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.654s	user 0.007s	sys 0.002s
I20260709 13:16:57.346321 13337 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:57.347602 13421 raft_consensus.cc:2804] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:57.348115 13280 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:57.349545 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:57.337445 10801 raft_consensus.cc:2889] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:57.343575 13290 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:57.343665 13034 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:57.353235 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:57.355501 13452 raft_consensus.cc:697] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 6 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:57.361405 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:57.361524 13452 consensus_queue.cc:237] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:57.362026 13034 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:16:57.344877 13321 consensus_meta.cc:287] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 1.496s	user 0.008s	sys 0.006s
I20260709 13:16:57.362591 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:57.357569 10766 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:57.350756 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:57.360183 13439 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:57.355868 13427 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:57.353267 10699 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42968 (request call id 244) took 8573 ms. Trace:
I20260709 13:16:57.364399 10797 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 4.
I20260709 13:16:57.356814 10946 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
I20260709 13:16:57.366604 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:57.367000 10766 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:57.370076 11504 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
I20260709 13:16:57.369990 10766 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:57.325484 13086 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:57.336757 13437 raft_consensus.cc:2749] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:16:57.372843 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:57.367347 10795 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 6.
I20260709 13:16:57.374396 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:57.313331 13331 consensus_queue.cc:1059] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:16:57.380555 11478 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:57.336627 13414 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:57.380256 10797 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 5.
I20260709 13:16:57.387449 13447 raft_consensus.cc:2804] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:57.388016 13447 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:57.388414 13447 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:57.389377 10950 raft_consensus.cc:1275] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 2. (term mismatch)
I20260709 13:16:57.389518 13331 raft_consensus.cc:3055] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:57.389779 10950 pending_rounds.cc:85] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 1
I20260709 13:16:57.389884 13331 raft_consensus.cc:740] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:57.390089 10950 pending_rounds.cc:107] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:16:57.370535 13444 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:57.390981 10950 raft_consensus.cc:2889] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:57.391244 13444 raft_consensus.cc:697] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:57.392283 13383 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:57.392154 13444 consensus_queue.cc:237] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:57.375015 13439 raft_consensus.cc:697] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 5 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:57.364506 10699 rpcz_store.cc:276] 0709 13:16:48.779635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:48.779846 (+   211us) service_pool.cc:224] Handling call
0709 13:16:57.353172 (+8573326us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:16:52.535622 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:16:52.536235 (+   613us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:16:52.536292 (+    57us) write_op.cc:435] Acquired schema lock
0709 13:16:52.536327 (+    35us) tablet.cc:662] Decoding operations
0709 13:16:52.619842 (+ 83515us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:16:52.619916 (+    74us) write_op.cc:641] Partition lock acquired for write op
0709 13:16:52.619946 (+    30us) tablet.cc:685] Acquiring locks for 21 operations
0709 13:16:52.654453 (+ 34507us) tablet.cc:701] Row locks acquired
0709 13:16:52.654508 (+    55us) write_op.cc:260] PREPARE: finished
0709 13:16:52.654760 (+   252us) write_op.cc:270] Start()
0709 13:16:52.655026 (+   266us) write_op.cc:276] Timestamp: P: 1783603012654671 usec, L: 0
0709 13:16:52.655060 (+    34us) op_driver.cc:348] REPLICATION: starting
0709 13:16:52.657475 (+  2415us) log.cc:844] Serialized 8345 byte log entry
0709 13:16:53.113407 (+455932us) op_driver.cc:464] REPLICATION: finished
0709 13:16:53.216531 (+103124us) write_op.cc:301] APPLY: starting
0709 13:16:53.216611 (+    80us) tablet.cc:1367] starting BulkCheckPresence
0709 13:16:53.219289 (+  2678us) tablet.cc:1370] finished BulkCheckPresence
0709 13:16:53.220556 (+  1267us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:16:53.567417 (+346861us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:16:54.397262 (+829845us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=21
0709 13:16:54.409690 (+ 12428us) write_op.cc:312] APPLY: finished
0709 13:16:54.541101 (+131411us) log.cc:844] Serialized 145 byte log entry
0709 13:16:54.961467 (+420366us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:54.995006 (+ 33539us) write_op.cc:454] Released schema lock
0709 13:16:55.013557 (+ 18551us) write_op.cc:341] FINISH: Updating metrics
0709 13:16:55.617147 (+603590us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:55.623330 (+  6183us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1340,"reported_tablets":39,"child_traces":[["op",{"apply.queue_time_us":102806,"apply.run_cpu_time_us":68851,"apply.run_wall_time_us":2357374,"num_ops":21,"prepare.queue_time_us":363052,"prepare.run_cpu_time_us":66990,"prepare.run_wall_time_us":249052,"raft.queue_time_us":90875,"raft.run_cpu_time_us":34576,"raft.run_wall_time_us":63779,"replication_time_us":458162,"thread_start_us":738164,"threads_started":5,"wal-append.queue_time_us":1203313,"wal-append.run_cpu_time_us":45048,"wal-append.run_wall_time_us":2310588}]]}
I20260709 13:16:57.394273 13439 consensus_queue.cc:237] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:57.395126 13331 consensus_queue.cc:260] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:57.396384 13331 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:57.392529 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:57.397850 10945 raft_consensus.cc:1240] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:57.309855 13419 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:57.434935 13302 raft_consensus.cc:3055] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:57.435441 13302 raft_consensus.cc:740] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:57.436219 13302 consensus_queue.cc:260] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:57.365665 13421 raft_consensus.cc:697] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 7 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:57.437247 13302 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:57.438223 13421 consensus_queue.cc:237] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:57.440439 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:57.441052 10947 raft_consensus.cc:2376] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7. Re-sending same reply.
W20260709 13:16:57.442162 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:57.443149 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:57.444392 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:57.373997 13381 raft_consensus.cc:2749] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:16:57.437223 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:57.383989 13429 raft_consensus.cc:2749] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:57.471154 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:16:57.472860 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 92384.8us
I20260709 13:16:57.474050 11478 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:57.415659 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:57.476879 10801 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:57.483753 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:57.483847 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:57.485688 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:57.487380 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:57.488945 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:57.490368 10795 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 7.
I20260709 13:16:57.492189 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:57.493513 10947 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
I20260709 13:16:57.493552 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:57.494299 10950 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 5.
I20260709 13:16:57.492599 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:57.495436 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:57.484750 10916 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:57.499915 10916 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:57.501343 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:57.502867 11478 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:57.482139 13412 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:57.513594 10947 raft_consensus.cc:1275] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:16:57.509188 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:57.512573 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:57.510826 10870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1498) took 1365 ms. Trace:
I20260709 13:16:57.510967 11478 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:57.509212 13327 raft_consensus.cc:2749] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:57.499368 13423 raft_consensus.cc:2749] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:16:57.522401 10795 raft_consensus.cc:2393] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 7: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:57.517730 10950 raft_consensus.cc:2410] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:16:57.516584 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:57.545917 10947 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 5.
I20260709 13:16:57.516206 10870 rpcz_store.cc:276] 0709 13:16:56.145457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:57.053817 (+908360us) service_pool.cc:224] Handling call
0709 13:16:57.054229 (+   412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:57.054538 (+   309us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:16:57.233454 (+178916us) log.cc:844] Serialized 14 byte log entry
0709 13:16:57.510220 (+276766us) log.cc:844] Serialized 14 byte log entry
0709 13:16:57.510439 (+   219us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:57.510499 (+    60us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:57.510553 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:57.510617 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:57.510748 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":2030,"prepare.run_wall_time_us":39,"raft.queue_time_us":165,"raft.run_wall_time_us":5253,"spinlock_wait_cycles":22417152,"thread_start_us":275121,"threads_started":1}
I20260709 13:16:57.515139 13055 raft_consensus.cc:2749] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:57.538616 10916 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:57.517143 13430 raft_consensus.cc:2804] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:57.531812 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:57.553615 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:57.513628 13412 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
W20260709 13:16:57.555886 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 45079.7us
I20260709 13:16:57.557294 10766 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:57.557418 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:57.558683 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:57.559527 10795 raft_consensus.cc:2368] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 5. Current term is 6.
I20260709 13:16:57.559702 10801 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 6.
I20260709 13:16:57.560449 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:57.508432 13450 consensus_queue.cc:1048] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:16:57.564733 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:57.564424 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:57.566939 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:57.623975 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:16:57.631197 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:57.635677 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:57.568219 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:57.567164 10947 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 4.
I20260709 13:16:57.565469 10766 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:57.591135 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:16:57.654665 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 89284.4us
I20260709 13:16:57.655292 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:57.569244 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:57.553512 13430 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:57.568514 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:57.561887 13412 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:57.556959 13350 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:57.689169 10766 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:57.591447 10916 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:57.689138 10797 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 6.
I20260709 13:16:57.562426 13426 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:57.682994 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:57.698091 10947 raft_consensus.cc:2393] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 4: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:57.592219 11511 raft_consensus.cc:2955] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:57.646085 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:16:57.626338 13455 log.cc:927] Time spent T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.146s	user 0.003s	sys 0.000s
I20260709 13:16:57.567665 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:57.693823 10868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1486) took 1764 ms. Trace:
I20260709 13:16:57.567893 13134 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:57.573674 13425 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:57.695685 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:57.718462 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:16:57.691802 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 100477us
I20260709 13:16:57.602164 13436 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:57.719421 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:57.719857 10916 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:57.719741 13436 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:57.720674 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:57.703204 11510 raft_consensus.cc:2393] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 5: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:57.683063 13375 raft_consensus.cc:2749] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:57.723877 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:57.715561 10766 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:57.688925 13430 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:57.689980 13350 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:57.563746 13381 consensus_queue.cc:1048] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.139s
I20260709 13:16:57.617935 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:57.592206 13358 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:16:57.721679 10916 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:57.701967 13439 raft_consensus.cc:2804] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:57.692492 13411 consensus_queue.cc:1059] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 6.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 13:16:57.704208 13426 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:57.778597 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:57.746585 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:57.862413 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:57.862825 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:16:57.850999 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 111378us
I20260709 13:16:57.873642 10766 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:16:57.808290 13457 log.cc:927] Time spent T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
I20260709 13:16:57.801754 13436 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:57.875422 10795 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 7.
I20260709 13:16:57.875779 13258 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:57.929234 10799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 1859) took 1416 ms. Trace:
I20260709 13:16:57.806799 13446 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:57.929816 13446 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:57.930361 13446 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:16:57.931481 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:57.874601 13331 raft_consensus.cc:2804] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:57.932037 13331 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:57.932474 13331 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:57.844358 13301 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:57.816299 13458 log.cc:927] Time spent T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.160s	user 0.000s	sys 0.001s
I20260709 13:16:57.703110 10868 rpcz_store.cc:276] 0709 13:16:55.929664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.919407 (+989743us) service_pool.cc:224] Handling call
0709 13:16:57.040832 (+121425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:57.041052 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:16:57.075329 (+ 34277us) log.cc:844] Serialized 14 byte log entry
0709 13:16:57.445092 (+369763us) spinlock_profiling.cc:243] Waited 8.63 ms on lock 0x7b18002eb820. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a5523a 00007f9c80ebfa2a
0709 13:16:57.693378 (+248286us) log.cc:844] Serialized 14 byte log entry
0709 13:16:57.693572 (+   194us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:57.693608 (+    36us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:57.693644 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:57.693689 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:57.693769 (+    80us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":33851,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":4526,"raft.queue_time_us":3424,"raft.run_cpu_time_us":249,"raft.run_wall_time_us":12111,"spinlock_wait_cycles":19851520,"thread_start_us":242799,"threads_started":2,"wal-append.queue_time_us":563568}
I20260709 13:16:57.936046 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:57.936228 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:57.936734 10946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41020 (request call id 1871) took 1224 ms. Trace:
I20260709 13:16:57.937134 10946 rpcz_store.cc:276] 0709 13:16:56.712325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.833084 (+120759us) service_pool.cc:224] Handling call
0709 13:16:57.936572 (+1103488us) spinlock_profiling.cc:243] Waited 337 ms on lock 0x7b5400399754. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:57.936674 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":774992640}
I20260709 13:16:57.717434 13383 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:57.938112 13383 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:57.938561 13383 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:57.941844 13446 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:57.942421 13328 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:57.840118 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:57.748986 13350 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:16:57.840073 13152 consensus_meta.cc:287] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.768s	user 0.012s	sys 0.007s
W20260709 13:16:57.878146 13410 log.cc:927] Time spent T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 1.500s	user 0.004s	sys 0.000s
I20260709 13:16:57.810379 13375 consensus_queue.cc:1048] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.069s
I20260709 13:16:57.891029 13426 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:57.873812 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 152190us
I20260709 13:16:57.874863 13439 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:57.746258 13134 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:57.738971 13450 raft_consensus.cc:2749] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:57.876681 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:16:57.884155 10948 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36714 (request call id 312) took 1802 ms (client timeout 1790 ms). Trace:
W20260709 13:16:57.900461 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:57.903769 13419 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
W20260709 13:16:57.904937 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:57.905400 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:57.982497 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:57.983139 10871 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 7.
I20260709 13:16:57.984431 10767 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:57.983433 11510 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 7.
I20260709 13:16:57.906682 10942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 311) took 1916 ms. Trace:
W20260709 13:16:57.907945 10792 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 1858) took 1485 ms (client timeout 1869 ms). Trace:
I20260709 13:16:57.909420 10876 raft_consensus.cc:1240] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:57.910677 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:57.906762 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:57.914018 13186 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:16:57.915306 10872 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 1503) took 1768 ms (client timeout 1700 ms). Trace:
W20260709 13:16:57.991225 10872 rpcz_store.cc:269] 0709 13:16:56.147179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.242298 (+ 95119us) service_pool.cc:224] Handling call
0709 13:16:57.915086 (+1672788us) spinlock_profiling.cc:243] Waited 425 ms on lock 0x7b54000f69d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:57.915217 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":976867584}
I20260709 13:16:57.919960 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:57.921396 13032 raft_consensus.cc:3055] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:16:57.922261 13086 consensus_meta.cc:287] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.522s	user 0.010s	sys 0.006s
W20260709 13:16:57.926467 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 52909.9us
W20260709 13:16:57.891271 13467 log.cc:927] Time spent T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:16:57.927224 13358 raft_consensus.cc:3055] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:57.927330 13381 raft_consensus.cc:2804] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:58.010982 13381 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:58.011493 13381 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:57.779556 13253 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.012643 13446 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:58.012777 13253 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:58.014968 13253 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:57.929018 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:58.015166 10874 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 8.
I20260709 13:16:58.015758 10794 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 4.
I20260709 13:16:58.016813 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:58.018069 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:57.937300 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:57.849340 13447 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:57.998971 13032 raft_consensus.cc:740] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:58.019148 10868 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 6.
I20260709 13:16:58.019734 13032 consensus_queue.cc:260] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:58.020587 13447 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:16:57.731125 13461 log.cc:927] Time spent T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 13:16:58.020968 13032 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:58.022003 12994 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:58.024189 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:58.024886 10794 raft_consensus.cc:2376] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4. Re-sending same reply.
I20260709 13:16:57.946923 10799 rpcz_store.cc:276] 0709 13:16:56.513422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.513577 (+   155us) service_pool.cc:224] Handling call
0709 13:16:56.528563 (+ 14986us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:56.796708 (+268145us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:16:57.249175 (+452467us) log.cc:844] Serialized 14 byte log entry
0709 13:16:57.717007 (+467832us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:57.744847 (+ 27840us) log.cc:844] Serialized 26 byte log entry
0709 13:16:57.745200 (+   353us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:57.745507 (+   307us) log.cc:844] Serialized 14 byte log entry
0709 13:16:57.893314 (+147807us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:57.928836 (+ 35522us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b5400294854. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:57.928885 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:57.929010 (+   125us) raft_consensus.cc:1685] finished
0709 13:16:57.929027 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:57.929170 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":555489,"prepare.queue_time_us":452307,"prepare.run_cpu_time_us":148,"prepare.run_wall_time_us":20526,"raft.queue_time_us":445864,"raft.run_cpu_time_us":477,"raft.run_wall_time_us":453635,"spinlock_wait_cycles":410829440,"thread_start_us":909914,"threads_started":3,"wal-append.queue_time_us":566842}
I20260709 13:16:58.004604 13358 raft_consensus.cc:740] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:58.031666 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:58.032313 10799 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 8.
I20260709 13:16:58.032187 13358 consensus_queue.cc:260] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.2, Last appended by leader: 2, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:58.033488 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:57.949431 10800 raft_consensus.cc:2376] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 6. Re-sending same reply.
I20260709 13:16:57.962332 13451 consensus_queue.cc:1059] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:16:57.834069 13463 log.cc:927] Time spent T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 13:16:58.035264 13451 raft_consensus.cc:3055] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:57.999743 13426 raft_consensus.cc:2749] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:58.035696 13451 raft_consensus.cc:740] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:57.970300 10916 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:58.037000 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:16:57.975530 10948 rpcz_store.cc:269] 0709 13:16:56.082179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.084126 (+  1947us) service_pool.cc:224] Handling call
0709 13:16:57.883906 (+1799780us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b5400018bd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:57.884036 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2437875456}
I20260709 13:16:57.975557 10801 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 6.
I20260709 13:16:58.037567 10875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1488) took 2089 ms. Trace:
I20260709 13:16:58.038138 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:58.038292 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:58.038772 10871 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 4.
I20260709 13:16:58.039497 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:58.039700 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:57.987267 10869 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:58.040149 10801 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 3.
W20260709 13:16:57.987016 10792 rpcz_store.cc:269] 0709 13:16:56.422707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.423046 (+   339us) service_pool.cc:224] Handling call
0709 13:16:57.907767 (+1484721us) spinlock_profiling.cc:243] Waited 90.9 ms on lock 0x7b54003bd3d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:57.907883 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":209055616}
I20260709 13:16:57.913041 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:58.037966 10875 rpcz_store.cc:276] 0709 13:16:55.947777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.995564 (+1047787us) service_pool.cc:224] Handling call
0709 13:16:57.107687 (+112123us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:57.115677 (+  7990us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:58.037359 (+921682us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b54000f64d0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:58.037505 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":257021312}
I20260709 13:16:58.041378 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:58.041944 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:16:58.042181 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 71946.4us
I20260709 13:16:58.042712 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:58.043462 10916 leader_election.cc:400] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 5. Current term is 6.
I20260709 13:16:58.043531 10792 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 6.
I20260709 13:16:58.044859 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:57.989137 10870 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 6.
I20260709 13:16:57.995460 10915 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:58.000903 10766 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:57.977964 13439 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:57.985257 13409 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.020074 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:57.804491 11506 raft_consensus.cc:2393] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 8: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:58.025977 13412 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:57.943850 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:57.944514 10798 pending_rounds.cc:85] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 3
I20260709 13:16:58.061704 10798 pending_rounds.cc:107] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:16:58.061806 10793 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 5: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:58.062043 10798 raft_consensus.cc:2889] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:57.945152 10949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 314) took 1642 ms. Trace:
I20260709 13:16:58.027872 13302 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.063112 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:58.064795 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:58.033561 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:58.002494 13301 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:57.838461 13422 log.cc:927] Time spent T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 1.313s	user 0.000s	sys 0.002s
I20260709 13:16:58.036672 13451 consensus_queue.cc:260] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:58.039844 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:58.088891 10800 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7.
I20260709 13:16:58.089807 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:57.985653 10767 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:57.718570 13425 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:58.040366 10941 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41020 (request call id 1861) took 1740 ms (client timeout 1847 ms). Trace:
I20260709 13:16:58.047477 10871 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 7.
I20260709 13:16:57.986217 10942 rpcz_store.cc:276] 0709 13:16:55.990814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.995090 (+  4276us) service_pool.cc:224] Handling call
0709 13:16:56.001542 (+  6452us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:56.708365 (+706823us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:16:57.052733 (+344368us) log.cc:844] Serialized 14 byte log entry
0709 13:16:57.162588 (+109855us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:57.162657 (+    69us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:57.162744 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:57.906465 (+743721us) spinlock_profiling.cc:243] Waited 707 ms on lock 0x7b54003aff54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:57.906509 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:57.906630 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":426838,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":56,"raft.queue_time_us":89492,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":20241,"spinlock_wait_cycles":1625026432,"thread_start_us":369454,"threads_started":2,"wal-append.queue_time_us":82839}
I20260709 13:16:58.016566 13383 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:58.045671 10916 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:58.041144 10873 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 6.
W20260709 13:16:58.095640 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 121625us
I20260709 13:16:58.096243 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:58.053814 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:58.097599 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:58.097873 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:58.098682 10794 raft_consensus.cc:2393] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 4: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:58.099530 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:58.100193 10794 raft_consensus.cc:2393] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 6: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:58.100912 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:58.101429 10794 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 6.
I20260709 13:16:58.102195 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:16:58.054685 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 59345.5us
I20260709 13:16:58.045512 10792 raft_consensus.cc:2376] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5. Re-sending same reply.
I20260709 13:16:58.062863 10949 rpcz_store.cc:276] 0709 13:16:56.302410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.394161 (+ 91751us) service_pool.cc:224] Handling call
0709 13:16:56.460447 (+ 66286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:56.933212 (+472765us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:56.933258 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:56.933317 (+    59us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:56.933379 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:57.944937 (+1011558us) spinlock_profiling.cc:243] Waited 472 ms on lock 0x7b54000177d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:57.944982 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:57.945096 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1086668544}
I20260709 13:16:58.070042 10796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 1850) took 1701 ms. Trace:
I20260709 13:16:57.965642 13152 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:58.077679 10867 raft_consensus.cc:1275] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:16:58.066144 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:16:58.082569 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:58.085728 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:16:58.094825 10941 rpcz_store.cc:269] 0709 13:16:56.299817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.304194 (+  4377us) service_pool.cc:224] Handling call
0709 13:16:58.040215 (+1736021us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b54001e1254. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:58.040309 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":251082624}
W20260709 13:16:58.096228 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:58.108296 10793 raft_consensus.cc:2393] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 6: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:58.109144 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:16:58.114602 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 69030.9us
I20260709 13:16:58.114768 10793 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650"
dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
mode: GRACEFUL
new_leader_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:16:58.106046 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:58.115504 10916 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:58.107874 10867 pending_rounds.cc:85] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 4
I20260709 13:16:58.116109 10867 pending_rounds.cc:107] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 2.5
W20260709 13:16:58.104475 10874 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 1504) took 1957 ms (client timeout 1914 ms). Trace:
I20260709 13:16:58.116528 10867 raft_consensus.cc:2889] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:58.115578 10793 raft_consensus.cc:606] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Received request to transfer leadership to TS cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:58.108590 10875 raft_consensus.cc:2393] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 8: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:58.112107 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:58.117228 10793 raft_consensus.cc:614] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:58.109090 10801 raft_consensus.cc:2410] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:16:58.117591 10942 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:16:58.116652 10874 rpcz_store.cc:269] 0709 13:16:56.147493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.243232 (+ 95739us) service_pool.cc:224] Handling call
0709 13:16:58.104242 (+1861010us) spinlock_profiling.cc:243] Waited 1.72 s on lock 0x7b54002f08d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:58.104384 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3964325248}
I20260709 13:16:58.111847 10842 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:58.119027 10874 raft_consensus.cc:1275] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:16:58.104342 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:58.124583 10792 raft_consensus.cc:1240] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:58.126313 10792 raft_consensus.cc:1240] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 4. Current term is 5. Ops: [2.4-4.5]
I20260709 13:16:58.127704 10792 raft_consensus.cc:1275] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:16:58.106559 10796 rpcz_store.cc:276] 0709 13:16:56.368441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.381690 (+ 13249us) service_pool.cc:224] Handling call
0709 13:16:56.396719 (+ 15029us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:56.727317 (+330598us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:56.783206 (+ 55889us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:56.783284 (+    78us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:56.783366 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:58.069706 (+1286340us) spinlock_profiling.cc:243] Waited 325 ms on lock 0x7b54000d8254. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:58.069771 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:58.069951 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":746450816}
W20260709 13:16:58.129534 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 128734us
I20260709 13:16:58.130147 10916 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:58.052807 13344 raft_consensus.cc:2749] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:58.131318 10916 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:58.095258 10869 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 7.
I20260709 13:16:58.135787 10868 raft_consensus.cc:1275] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:16:58.214740 10871 raft_consensus.cc:1240] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:58.212833 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:16:58.215616 10792 raft_consensus.cc:1240] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:58.117869 10841 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:58.082293 12994 raft_consensus.cc:2804] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:16:58.102733 13462 log.cc:927] Time spent T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.362s	user 0.002s	sys 0.000s
I20260709 13:16:58.217792 10795 raft_consensus.cc:2410] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:16:58.223435 10793 raft_consensus.cc:1240] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 3. Current term is 4. Ops: []
W20260709 13:16:58.219126 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 233531us
I20260709 13:16:58.231905 10767 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:16:58.218354 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 100588us
I20260709 13:16:58.109197 13425 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:58.221405 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:16:58.229285 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 98036.3us
W20260709 13:16:58.239061 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 141361us
I20260709 13:16:58.239284 10916 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:58.229862 11504 raft_consensus.cc:1275] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:16:58.098688 10696 catalog_manager.cc:5697] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: config changed from index -1 to 4, term changed from 3 to 6, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130), NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New cstate: current_term: 6 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:16:58.241127 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:58.007516 13086 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:58.223634 11508 raft_consensus.cc:2393] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 8: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:58.106405 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:58.108107 13383 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:58.246165 10916 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:58.241218 10696 catalog_manager.cc:5697] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 2 to 5, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 5 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:16:58.247975 10696 catalog_manager.cc:5697] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 1 to 4, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 4 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:16:58.252633 10916 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:58.211171 13414 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:58.250453 11478 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:58.209498 13152 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:58.082211 13409 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:58.258895 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:58.258878 11478 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:58.092708 13302 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:58.263211 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:58.226176 13186 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:58.269208 11504 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:58.217986 12996 raft_consensus.cc:2749] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:58.092919 13301 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:58.217947 13380 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.236318 13060 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:58.264822 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:58.249593 10696 catalog_manager.cc:5697] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 1 to 4, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 4 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:58.266072 10841 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:58.220682 13212 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:16:58.220867 13134 raft_consensus.cc:2749] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:16:58.223784 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 110853us
I20260709 13:16:58.231087 13451 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:58.223963 12994 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:58.226217 13397 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:58.328727 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:58.329073 13397 raft_consensus.cc:697] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:58.330021 13397 consensus_queue.cc:237] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:58.225067 13430 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:58.334218 13380 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:58.262908 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:58.339968 13412 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:58.342049 13306 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:58.344081 10868 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:16:58.344295 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:58.344761 10868 raft_consensus.cc:606] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Received request to transfer leadership to TS cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:58.345117 10868 raft_consensus.cc:614] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:58.346372 10793 raft_consensus.cc:1240] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:58.347254 13152 consensus_queue.cc:1048] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:16:58.349155 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:58.271584 11512 raft_consensus.cc:1240] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 3. Current term is 4. Ops: []
W20260709 13:16:58.310809 10943 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36714 (request call id 304) took 2474 ms (client timeout 2163 ms). Trace:
I20260709 13:16:58.258095 13450 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:58.259572 13344 raft_consensus.cc:2804] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:16:58.298619 13462 log.cc:927] Time spent T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:16:58.259428 13414 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:58.325724 13425 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:58.327773 12994 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:58.328274 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:16:58.352635 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:58.306042 10795 raft_consensus.cc:1240] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:58.279712 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:58.352787 10696 catalog_manager.cc:5697] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 1 to 2, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 2 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:58.222294 13331 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:58.350008 10943 rpcz_store.cc:269] 0709 13:16:55.804870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.821216 (+ 16346us) service_pool.cc:224] Handling call
0709 13:16:58.278703 (+2457487us) spinlock_profiling.cc:243] Waited 549 ms on lock 0x7b54002a53d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:58.278867 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1263038720}
I20260709 13:16:58.356314 13450 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:58.313501 13253 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:58.217068 13474 raft_consensus.cc:2804] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:58.257928 13447 consensus_queue.cc:1048] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.117s
I20260709 13:16:58.368103 11504 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 6.
W20260709 13:16:58.369625 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 56289.1us
I20260709 13:16:58.440936 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:58.474481 11509 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 7.
I20260709 13:16:58.475251 10842 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:58.442430 10868 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:16:58.406253 13328 raft_consensus.cc:2804] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:58.313318 13337 raft_consensus.cc:2804] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:58.486096 10868 raft_consensus.cc:606] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Received request to transfer leadership to TS cb0428cc55984daba1ba63744a5102a2
W20260709 13:16:58.371659 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 105681us
I20260709 13:16:58.273442 13186 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:58.497357 10868 raft_consensus.cc:614] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:58.300405 13060 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:58.272774 13355 raft_consensus.cc:2749] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:58.498373 10842 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:58.509660 10841 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:58.510011 10842 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:58.491568 10696 catalog_manager.cc:5697] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 2 to 5, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 5 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:16:58.507447 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:58.513129 10696 catalog_manager.cc:5697] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 1 to 6. New cstate: current_term: 6 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:58.516927 10842 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:58.486687 13474 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:58.518149 13474 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:58.374050 13344 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:58.483398 13450 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:58.377120 13430 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:58.623535 10696 catalog_manager.cc:5697] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 2 to 5, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 5 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:16:58.626101 10941 raft_consensus.cc:2393] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 8: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:58.375069 13258 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:58.624249 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:58.522245 13034 raft_consensus.cc:2749] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:16:58.507318 13060 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:58.407377 13086 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:58.451897 13301 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:58.515796 13186 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:58.635938 10696 catalog_manager.cc:5697] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 2 to 4, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 4 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:58.630558 10793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1513) took 1035 ms. Trace:
I20260709 13:16:58.624825 13344 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:58.641952 10696 catalog_manager.cc:5697] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:58.655972 10696 catalog_manager.cc:5697] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 1 to 4, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 4 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:16:58.657763 10696 catalog_manager.cc:5697] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 5 to 6, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 6 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:58.651574 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:58.627045 13436 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.659355 10943 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:58.664834 10766 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:16:58.667744 10766 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:58.660164 10696 catalog_manager.cc:5697] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 2 to 3, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 3 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
W20260709 13:16:58.669646 10942 consensus_meta.cc:287] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.551s	user 0.006s	sys 0.000s
I20260709 13:16:58.670064 10942 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7.
I20260709 13:16:58.669039 10696 catalog_manager.cc:5697] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 5 to 6, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 6 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:16:58.675556 10696 catalog_manager.cc:5697] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:16:58.683125 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:58.688102 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:58.690366 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:16:58.691993 13484 log.cc:927] Time spent T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:16:58.694092 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:58.694728 10948 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 7.
I20260709 13:16:58.695632 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:58.696123 10948 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:58.681511 13473 raft_consensus.cc:2749] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:58.704423 10794 raft_consensus.cc:1240] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:58.707546 13328 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:58.708087 13328 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:58.712404 13425 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:58.713064 13425 consensus_queue.cc:1059] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:16:58.745038 10870 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.663s	user 0.009s	sys 0.010s
W20260709 13:16:58.745652 10870 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.009s	sys 0.010s
W20260709 13:16:58.747299 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 494706us
I20260709 13:16:58.772018 10916 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:58.373697 13414 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:58.432744 13409 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:58.790894 13212 consensus_queue.cc:1048] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:16:58.799921 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 132234us
W20260709 13:16:58.794338 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:58.776603 12996 raft_consensus.cc:2749] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:16:58.800812 10766 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:16:58.801797 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 284942us
I20260709 13:16:58.483673 13419 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:58.432261 13289 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:58.507534 13152 raft_consensus.cc:2804] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:58.496837 13337 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:58.515527 13134 raft_consensus.cc:2749] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:58.523624 13427 raft_consensus.cc:2804] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:58.776998 13375 raft_consensus.cc:2749] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:16:58.838119 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 327074us
I20260709 13:16:58.839483 10841 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:58.822556 13447 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:58.832563 12996 consensus_queue.cc:1059] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:16:58.805099 10943 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 6.
I20260709 13:16:58.833778 13383 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:58.833300 13473 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:58.670920 13436 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:58.845674 10841 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:58.826735 13337 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:58.827064 13427 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:58.873577 13438 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:58.875190 13397 raft_consensus.cc:2749] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:16:58.865418 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:58.820672 13152 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:58.829097 13289 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:58.901866 13152 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:58.902117 13289 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:58.886623 10841 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:58.884735 13427 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:58.826134 13409 consensus_queue.cc:1059] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 13:16:58.927281 13409 raft_consensus.cc:3055] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:58.927651 13409 raft_consensus.cc:740] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:58.928411 13409 consensus_queue.cc:260] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:58.929581 13409 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:58.929827 13186 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:58.819620 13485 consensus_queue.cc:1059] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.109s
I20260709 13:16:58.951342 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:58.872831 13134 consensus_queue.cc:1048] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 13:16:58.882077 13383 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:58.914463 13350 consensus_meta.cc:287] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.948s	user 0.003s	sys 0.014s
I20260709 13:16:58.915468 13328 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:58.918059 13490 log.cc:927] Time spent T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 13:16:58.921900 13436 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:16:58.923645 13439 consensus_meta.cc:287] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.842s	user 0.012s	sys 0.006s
I20260709 13:16:58.901902 13212 raft_consensus.cc:3055] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:58.923990 13302 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:16:58.949949 13451 consensus_meta.cc:287] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.604s	user 0.007s	sys 0.007s
W20260709 13:16:58.965159 10946 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.696s	user 0.017s	sys 0.030s
I20260709 13:16:58.951748 10792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 263) took 1109 ms. Trace:
I20260709 13:16:58.902658 13289 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:58.896099 13419 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:16:58.956457 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 153667us
I20260709 13:16:58.926707 13493 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:58.883301 13473 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:58.964263 10801 raft_consensus.cc:1240] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
W20260709 13:16:58.965638 10946 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.017s	sys 0.030s
I20260709 13:16:58.965826 10792 rpcz_store.cc:276] 0709 13:16:57.842074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:58.712650 (+870576us) service_pool.cc:224] Handling call
0709 13:16:58.712983 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:58.713187 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:16:58.713497 (+   310us) log.cc:844] Serialized 14 byte log entry
0709 13:16:58.951251 (+237754us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:58.951336 (+    85us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:58.951413 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:58.951509 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:58.951671 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6729,"prepare.queue_time_us":59148,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":18812,"raft.queue_time_us":468,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":1820,"thread_start_us":236917,"threads_started":1}
I20260709 13:16:58.974558 10842 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:58.954851 13331 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:58.985795 10797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 264) took 1073 ms. Trace:
I20260709 13:16:58.959759 13350 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:58.993464 13350 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:58.995038 10947 raft_consensus.cc:1240] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:58.997056 10792 raft_consensus.cc:1275] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:16:58.986209 10797 rpcz_store.cc:276] 0709 13:16:57.912660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:58.978053 (+1065393us) service_pool.cc:224] Handling call
0709 13:16:58.985282 (+  7229us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:58.985465 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:58.985489 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:58.985531 (+    42us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:58.985573 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:58.985625 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:58.985741 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:59.009008 10842 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:58.997705 10792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 265) took 1077 ms. Trace:
I20260709 13:16:58.965368 13212 raft_consensus.cc:740] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:59.010567 10792 rpcz_store.cc:276] 0709 13:16:57.920140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:58.980758 (+1060618us) service_pool.cc:224] Handling call
0709 13:16:58.996752 (+ 15994us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:58.997463 (+   711us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:58.997649 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:58.934628 13438 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:58.976735 13419 raft_consensus.cc:697] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:59.025036 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:58.984465 13485 raft_consensus.cc:3055] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:59.007109 13397 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.972056 13439 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:59.033246 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:16:59.032068 13381 consensus_meta.cc:287] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 1.020s	user 0.012s	sys 0.005s
W20260709 13:16:59.038403 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 230391us
W20260709 13:16:59.032637 13032 consensus_meta.cc:287] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 1.011s	user 0.007s	sys 0.014s
I20260709 13:16:59.041002 10941 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:59.032121 13134 raft_consensus.cc:2804] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:59.046518 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:59.032407 13493 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:59.054364 10796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1495) took 1558 ms. Trace:
I20260709 13:16:59.057219 10796 rpcz_store.cc:276] 0709 13:16:57.496398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:58.216856 (+720458us) service_pool.cc:224] Handling call
0709 13:16:58.217192 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:58.217402 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:16:58.221030 (+  3628us) log.cc:844] Serialized 14 byte log entry
0709 13:16:58.829484 (+608454us) log.cc:844] Serialized 14 byte log entry
0709 13:16:59.038062 (+208578us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:59.038123 (+    61us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:59.041392 (+  3269us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:59.054112 (+ 12720us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:59.054287 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":124700,"prepare.queue_time_us":133085,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":5713,"raft.queue_time_us":173284,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":34689,"spinlock_wait_cycles":6289280,"thread_start_us":725147,"threads_started":3,"wal-append.queue_time_us":411073}
I20260709 13:16:59.040454 10795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 259) took 1342 ms. Trace:
I20260709 13:16:59.030123 13485 raft_consensus.cc:740] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:16:59.070631 10948 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 4.
I20260709 13:16:59.072628 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:59.073740 10944 raft_consensus.cc:1240] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:59.074155 10870 raft_consensus.cc:1240] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:58.984413 10794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 262) took 1177 ms. Trace:
I20260709 13:16:59.080544 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:59.081511 10870 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:59.083050 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:59.078142 10794 rpcz_store.cc:276] 0709 13:16:57.806921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:58.705390 (+898469us) service_pool.cc:224] Handling call
0709 13:16:58.705659 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:58.799078 (+ 93419us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:16:58.835176 (+ 36098us) log.cc:844] Serialized 14 byte log entry
0709 13:16:58.977782 (+142606us) log.cc:844] Serialized 14 byte log entry
0709 13:16:58.977936 (+   154us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:58.977985 (+    49us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:58.984139 (+  6154us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:58.984221 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:58.984353 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":44057,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":1781,"raft.queue_time_us":190,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":10733,"spinlock_wait_cycles":22226304,"thread_start_us":140958,"threads_started":1,"wal-append.queue_time_us":240461}
I20260709 13:16:59.069435 10795 rpcz_store.cc:276] 0709 13:16:57.697815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:58.376018 (+678203us) service_pool.cc:224] Handling call
0709 13:16:58.420738 (+ 44720us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:58.421006 (+   268us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:16:58.451515 (+ 30509us) log.cc:844] Serialized 14 byte log entry
0709 13:16:59.040044 (+588529us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:59.040105 (+    61us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:59.040167 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:59.040255 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:59.040393 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20011,"prepare.queue_time_us":100748,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":27278,"spinlock_wait_cycles":6506752,"thread_start_us":581869,"threads_started":2,"wal-append.queue_time_us":495431}
W20260709 13:16:59.089536 12994 consensus_meta.cc:287] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.682s	user 0.005s	sys 0.005s
I20260709 13:16:59.090030 12994 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:59.091171 10943 raft_consensus.cc:1240] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:59.090941 13419 consensus_queue.cc:237] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 6.2, Last appended by leader: 2, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:59.026969 13337 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:16:59.086437 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 267991us
I20260709 13:16:59.099958 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:59.068008 13134 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:16:59.034137 13397 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:59.101446 10948 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7.
I20260709 13:16:59.105502 13152 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:59.067203 13493 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
W20260709 13:16:59.120247 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 111339us
I20260709 13:16:59.122948 10842 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:59.121688 10941 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
I20260709 13:16:59.051941 13381 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:59.125819 10842 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:16:59.050067 10916 connection.cc:256] RPC call timeout handler was delayed by -1.4667705199999546s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19140000000000001s for a response
W20260709 13:16:59.126667 10916 connection.cc:256] RPC call timeout handler was delayed by -1.334470519999968s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2061s for a response
I20260709 13:16:59.108412 13258 raft_consensus.cc:2749] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:59.132287 10868 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 8.
I20260709 13:16:59.133109 10867 raft_consensus.cc:1240] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:59.110935 13134 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:59.131806 10841 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:59.135850 10842 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:59.139904 13439 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:59.142903 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:59.053328 13212 consensus_queue.cc:260] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:59.149987 10797 raft_consensus.cc:1240] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:59.150107 10842 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:59.152063 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:59.117640 13493 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:59.125888 12994 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:59.127995 13438 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:59.156071 11508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56416 (request call id 498) took 1436 ms. Trace:
I20260709 13:16:59.157698 11508 rpcz_store.cc:276] 0709 13:16:57.719199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:57.721801 (+  2602us) service_pool.cc:224] Handling call
0709 13:16:59.155873 (+1434072us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b540046d2d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:59.156016 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":552445184}
I20260709 13:16:59.156718 11507 raft_consensus.cc:1240] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:59.132418 13302 raft_consensus.cc:2749] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:59.161595 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:59.146268 13032 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:59.180332 13337 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:59.181914 13289 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:59.156100 13485 consensus_queue.cc:260] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:59.183487 13302 raft_consensus.cc:2749] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:59.157757 13258 raft_consensus.cc:2749] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:59.184944 13473 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:16:59.185247 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:59.186280 13380 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:59.150715 13383 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:59.155433 13501 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:59.136534 13186 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:59.184278 13289 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:59.164942 10842 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:59.188444 13397 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:59.141412 13451 consensus_queue.cc:1048] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 13:16:59.188609 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 305) took 1245 ms. Trace:
I20260709 13:16:59.302717 10943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 1876) took 1230 ms. Trace:
I20260709 13:16:59.260273 10874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 311) took 1315 ms. Trace:
I20260709 13:16:59.303910 10943 rpcz_store.cc:276] 0709 13:16:58.072270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:58.841742 (+769472us) service_pool.cc:224] Handling call
0709 13:16:58.842046 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:59.091623 (+249577us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:59.302358 (+210735us) spinlock_profiling.cc:243] Waited 249 ms on lock 0x7b5400017cd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:59.302623 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":572446592}
I20260709 13:16:59.303606 10873 rpcz_store.cc:276] 0709 13:16:57.943523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:58.096807 (+153284us) service_pool.cc:224] Handling call
0709 13:16:59.187961 (+1091154us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b5800177098. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd90eee 0000000000375939 00007f9c85e041e0 00007f9c85ecd0c4 00007f9c85ecd131 00007f9c85e8efc0 00007f9c85e79929 00007f9c85e793d2 00007f9c7f3f0bd4
0709 13:16:59.188113 (+   152us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:59.188295 (+   182us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:59.188318 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:59.188356 (+    38us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:59.188393 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:59.188443 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:59.188555 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":30063488}
I20260709 13:16:59.305573 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:59.304561 10874 rpcz_store.cc:276] 0709 13:16:57.945184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:58.120545 (+175361us) service_pool.cc:224] Handling call
0709 13:16:58.120791 (+   246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:58.120975 (+   184us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:16:58.121234 (+   259us) log.cc:844] Serialized 14 byte log entry
0709 13:16:58.995399 (+874165us) spinlock_profiling.cc:243] Waited 150 ms on lock 0x7b1800364c00. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a5523a 00007f9c80ebfa2a
0709 13:16:59.259886 (+264487us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:59.259942 (+    56us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:59.260002 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:59.260073 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:59.260201 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4343,"prepare.queue_time_us":152,"prepare.run_cpu_time_us":107,"prepare.run_wall_time_us":10951,"raft.queue_time_us":36489,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":124,"spinlock_wait_cycles":343996160,"thread_start_us":259432,"threads_started":2,"wal-append.queue_time_us":1134689}
I20260709 13:16:59.306561 10794 raft_consensus.cc:1275] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:16:59.220456 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:16:59.265532 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 138290us
I20260709 13:16:59.306912 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:16:59.308341 10916 connection.cc:256] RPC call timeout handler was delayed by -1.2363705200000368s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.217s for a response
W20260709 13:16:59.308815 10916 connection.cc:256] RPC call timeout handler was delayed by -1.2300705200000266s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2177s for a response
I20260709 13:16:59.308995 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:16:59.309412 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:59.309723 10943 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7.
I20260709 13:16:59.266450 10870 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 8.
I20260709 13:16:59.310655 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:59.314477 10801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 256) took 1988 ms. Trace:
I20260709 13:16:59.314924 10801 rpcz_store.cc:276] 0709 13:16:57.326010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:58.209227 (+883217us) service_pool.cc:224] Handling call
0709 13:16:58.237781 (+ 28554us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:58.980472 (+742691us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:59.314243 (+333771us) spinlock_profiling.cc:243] Waited 726 ms on lock 0x7b54001f8bd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:59.314421 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1669911808}
I20260709 13:16:59.316172 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:59.316439 10947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 320) took 1383 ms. Trace:
I20260709 13:16:59.316891 10947 rpcz_store.cc:276] 0709 13:16:57.933050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:57.937084 (+  4034us) service_pool.cc:224] Handling call
0709 13:16:57.937346 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:58.995576 (+1058230us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:59.316175 (+320599us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54001e3554. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:59.316380 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2431908096}
I20260709 13:16:59.318044 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:59.319115 10766 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:59.318629 10947 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7.
I20260709 13:16:59.319957 10943 raft_consensus.cc:2410] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 9 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:16:59.225203 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:59.258606 10797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 266) took 1338 ms. Trace:
I20260709 13:16:59.321223 10797 rpcz_store.cc:276] 0709 13:16:57.920676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.002020 (+1081344us) service_pool.cc:224] Handling call
0709 13:16:59.023199 (+ 21179us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:59.150428 (+127229us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:59.258354 (+107926us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b5400386654. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:59.258543 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":291107584}
W20260709 13:16:59.233238 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:59.323350 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:59.323906 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:16:59.253392 13060 consensus_meta.cc:287] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.597s	user 0.013s	sys 0.006s
I20260709 13:16:59.324918 10794 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:59.325323 10916 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:59.327246 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:16:59.327646 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 162785us
I20260709 13:16:59.328271 10800 raft_consensus.cc:2393] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 4: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:16:59.265372 13337 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:59.331712 10948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 1885) took 1237 ms. Trace:
W20260709 13:16:59.221484 13253 consensus_meta.cc:287] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.725s	user 0.007s	sys 0.016s
I20260709 13:16:59.329917 10947 raft_consensus.cc:1240] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:59.330941 10916 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:16:59.252352 13474 consensus_meta.cc:287] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.733s	user 0.011s	sys 0.009s
I20260709 13:16:59.336831 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:59.307971 13435 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:59.333164 10943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 1886) took 1222 ms. Trace:
I20260709 13:16:59.331889 10870 raft_consensus.cc:1275] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:16:59.343767 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:59.329605 10842 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:59.344892 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:16:59.255610 13498 log.cc:927] Time spent T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.208s	user 0.000s	sys 0.000s
I20260709 13:16:59.334224 10947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 1877) took 1261 ms. Trace:
I20260709 13:16:59.346149 10842 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:59.347405 10947 rpcz_store.cc:276] 0709 13:16:58.072559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.329458 (+1256899us) service_pool.cc:224] Handling call
0709 13:16:59.329762 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:59.333972 (+  4210us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:59.334173 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:59.345963 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:59.348779 10842 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:59.349011 10947 raft_consensus.cc:1275] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:16:59.349643 10947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 326) took 1237 ms. Trace:
I20260709 13:16:59.350632 10947 rpcz_store.cc:276] 0709 13:16:58.112367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.348436 (+1236069us) service_pool.cc:224] Handling call
0709 13:16:59.348696 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:59.349353 (+   657us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:59.349592 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:59.328890 13405 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:59.351503 13405 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:59.352495 10798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1473) took 2030 ms. Trace:
I20260709 13:16:59.352747 13032 raft_consensus.cc:2749] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:59.353878 13405 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:59.334540 13443 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:59.339776 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:59.356112 10871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 306) took 1412 ms. Trace:
I20260709 13:16:59.343670 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:59.356338 13435 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:59.356528 10871 rpcz_store.cc:276] 0709 13:16:57.943939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:58.096855 (+152916us) service_pool.cc:224] Handling call
0709 13:16:58.214579 (+117724us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:58.215167 (+   588us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:59.355939 (+1140772us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b540009b0d0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:59.356056 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":270111616}
I20260709 13:16:59.356156 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:59.358105 11507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56404 (request call id 520) took 1381 ms. Trace:
I20260709 13:16:59.345069 10874 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:59.184311 13485 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:59.344774 10943 rpcz_store.cc:276] 0709 13:16:58.111193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.332407 (+1221214us) service_pool.cc:224] Handling call
0709 13:16:59.332688 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:59.332867 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:59.332897 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:59.332934 (+    37us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:59.332975 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:59.333024 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:59.333114 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:59.331913 13086 consensus_queue.cc:1048] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 13:16:59.342391 13134 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:59.381435 10943 raft_consensus.cc:1240] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 6. Current term is 7. Ops: [6.2-6.2]
I20260709 13:16:59.382102 10943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 1534) took 1153 ms. Trace:
I20260709 13:16:59.352895 10798 rpcz_store.cc:276] 0709 13:16:57.322548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:57.327757 (+  5209us) service_pool.cc:224] Handling call
0709 13:16:57.944311 (+616554us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:58.219003 (+274692us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:58.219035 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:58.219591 (+   556us) log.cc:844] Serialized 52 byte log entry
0709 13:16:58.485339 (+265748us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:58.485402 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:58.485509 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:58.729300 (+243791us) raft_consensus.cc:1685] finished
0709 13:16:58.729329 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:59.352259 (+622930us) spinlock_profiling.cc:243] Waited 616 ms on lock 0x7b54000bbdd0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:59.352436 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":181976,"raft.queue_time_us":156294,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":9181,"spinlock_wait_cycles":1417222656,"thread_start_us":213678,"threads_started":1,"wal-append.queue_time_us":229420}
I20260709 13:16:59.179838 13152 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:59.333514 10916 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:59.381920 11510 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 6 LEARNER]: Advancing to term 7
I20260709 13:16:59.358415 10792 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:16:59.342298 13450 consensus_meta.cc:287] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.701s	user 0.007s	sys 0.003s
I20260709 13:16:59.375042 10842 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:59.376304 13381 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:59.343730 13438 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:59.335028 13288 consensus_queue.cc:1059] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 13:16:59.376740 13355 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:59.376907 13471 log.cc:927] Time spent T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 1.150s	user 0.000s	sys 0.006s
I20260709 13:16:59.377846 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:59.379390 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:59.419857 10794 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 4.
I20260709 13:16:59.343247 13253 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:59.380722 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:59.332129 10948 rpcz_store.cc:276] 0709 13:16:58.094695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.330849 (+1236154us) service_pool.cc:224] Handling call
0709 13:16:59.331105 (+   256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:59.331288 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:59.331316 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:59.331362 (+    46us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:59.331401 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:59.331449 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:59.331541 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:59.382194 10947 raft_consensus.cc:1275] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:16:59.358970 13380 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:59.320317 13501 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:59.394758 13344 consensus_meta.cc:287] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.724s	user 0.000s	sys 0.008s
I20260709 13:16:59.386330 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:59.382450 13032 raft_consensus.cc:2749] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:59.345625 13444 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:16:59.366715 13424 raft_consensus.cc:2749] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:59.337116 13503 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:59.396144 13306 raft_consensus.cc:3055] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:59.434705 13306 raft_consensus.cc:740] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:59.419492 13450 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:59.435632 13306 consensus_queue.cc:260] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:59.436666 13306 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:59.436923 13450 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:59.437726 10942 raft_consensus.cc:1240] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:16:59.337248 13499 raft_consensus.cc:2764] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:16:59.438282 13328 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:59.396160 13427 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:59.439261 10873 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 4.
I20260709 13:16:59.439868 10942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 1875) took 1368 ms. Trace:
I20260709 13:16:59.440225 10942 rpcz_store.cc:276] 0709 13:16:58.071946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:58.700021 (+628075us) service_pool.cc:224] Handling call
0709 13:16:58.700520 (+   499us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:59.438114 (+737594us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:59.439602 (+  1488us) spinlock_profiling.cc:243] Waited 737 ms on lock 0x7b5400173254. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:59.439804 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1694819584}
I20260709 13:16:59.440994 13427 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:16:59.376914 13414 consensus_meta.cc:287] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.553s	user 0.009s	sys 0.000s
I20260709 13:16:59.441906 10942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 1536) took 1168 ms. Trace:
I20260709 13:16:59.441727 13414 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:59.443897 13414 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:16:59.444114 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:59.445696 10766 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:59.446890 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:59.447088 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:59.447279 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:59.447984 11503 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7.
I20260709 13:16:59.448874 13437 raft_consensus.cc:2749] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:59.389794 13134 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:59.407984 10944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 325) took 1473 ms. Trace:
I20260709 13:16:59.450196 10944 rpcz_store.cc:276] 0709 13:16:57.934492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:58.038525 (+104033us) service_pool.cc:224] Handling call
0709 13:16:58.038775 (+   250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:59.077349 (+1038574us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:59.407753 (+330404us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54000168d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:59.407925 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2379676416}
I20260709 13:16:59.450960 13444 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:59.451403 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:16:59.451949 10944 raft_consensus.cc:2376] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 4. Re-sending same reply.
I20260709 13:16:59.452898 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:59.399820 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:16:59.453675 10798 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:59.446918 10766 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:59.397558 10874 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 7.
I20260709 13:16:59.418401 11510 raft_consensus.cc:1275] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 7 LEARNER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:16:59.332149 13060 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:59.421895 10947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 1533) took 1199 ms. Trace:
I20260709 13:16:59.459049 10947 rpcz_store.cc:276] 0709 13:16:58.222349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.381699 (+1159350us) service_pool.cc:224] Handling call
0709 13:16:59.381982 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:59.421654 (+ 39672us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:59.421843 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:59.459841 10766 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:16:59.460716 10841 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:59.422150 10948 raft_consensus.cc:1275] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:16:59.402825 10792 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 8.
I20260709 13:16:59.382551 10943 rpcz_store.cc:276] 0709 13:16:58.228444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.380872 (+1152428us) service_pool.cc:224] Handling call
0709 13:16:59.381205 (+   333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:59.381855 (+   650us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:59.382050 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:59.461771 10948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 328) took 1191 ms. Trace:
I20260709 13:16:59.462096 10948 rpcz_store.cc:276] 0709 13:16:58.270641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.421582 (+1150941us) service_pool.cc:224] Handling call
0709 13:16:59.421854 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:59.461497 (+ 39643us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:59.461708 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:59.462433 10943 raft_consensus.cc:1275] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:16:59.426132 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 51164.8us
I20260709 13:16:59.463773 10842 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:59.464856 10842 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:59.358610 13435 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:59.428467 13515 raft_consensus.cc:2749] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:59.337847 13337 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:59.430487 13413 consensus_queue.cc:1048] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:59.419209 13355 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:59.426858 13501 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:16:59.438894 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:59.403515 10871 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:16:59.453186 13444 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:59.455435 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:16:59.504289 10871 raft_consensus.cc:606] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 LEADER]: Received request to transfer leadership to TS cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:59.504649 11504 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
I20260709 13:16:59.455641 11477 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:16:59.505218 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 44578.7us
W20260709 13:16:59.505870 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 50286.9us
I20260709 13:16:59.424721 13425 consensus_queue.cc:1048] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:16:59.442266 10942 rpcz_store.cc:276] 0709 13:16:58.273155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.441077 (+1167922us) service_pool.cc:224] Handling call
0709 13:16:59.441391 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:59.441572 (+   181us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:59.441599 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:59.441636 (+    37us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:59.441700 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:59.441750 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:59.441855 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:59.512030 11508 raft_consensus.cc:2393] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 5: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:16:59.410701 10800 raft_consensus.cc:1275] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:16:59.506942 11477 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:59.358510 11507 rpcz_store.cc:276] 0709 13:16:57.976663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:57.976850 (+   187us) service_pool.cc:224] Handling call
0709 13:16:59.156527 (+1179677us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:59.158846 (+  2319us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:59.357925 (+199079us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b540046d2d0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:16:59.358039 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2704986368}
I20260709 13:16:59.465853 10943 raft_consensus.cc:1240] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:59.515004 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:16:59.515583 11504 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:59.467204 10947 raft_consensus.cc:1240] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:59.469878 10842 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:59.349436 13474 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:59.516770 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 46958.7us
I20260709 13:16:59.518007 13474 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:59.424026 13380 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:16:59.518350 10842 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:59.518720 13380 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:59.519498 13032 raft_consensus.cc:2749] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:59.520274 13032 raft_consensus.cc:2804] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:59.520744 13032 raft_consensus.cc:697] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:59.521613 13032 consensus_queue.cc:237] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:59.410638 13288 consensus_queue.cc:1243] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer e01ecd008edd4aa2b6b2b47efbc2ea87 log is divergent from this leader: its last log entry 6.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:16:59.507330 10948 raft_consensus.cc:1240] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:59.407233 13438 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:59.524593 13438 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:59.524576 10842 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:59.419273 13493 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:59.525641 13424 raft_consensus.cc:2749] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:59.526543 10842 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:59.527112 13493 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:59.527369 13424 raft_consensus.cc:2749] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:59.528048 13253 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:59.528326 13451 raft_consensus.cc:2804] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:59.528501 13253 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:59.529150 13253 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:59.528940 13451 raft_consensus.cc:697] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 4 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:59.531613 13451 consensus_queue.cc:237] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:59.514192 10841 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:16:59.527748 10842 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:59.537706 11504 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
I20260709 13:16:59.454914 13419 raft_consensus.cc:2764] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won
I20260709 13:16:59.540292 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:59.534458 13503 raft_consensus.cc:2749] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:59.471045 13427 raft_consensus.cc:2749] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:16:59.540498 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 111418us
I20260709 13:16:59.559794 11506 raft_consensus.cc:1240] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:59.511091 13515 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:59.566231 10841 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:59.568521 10841 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:59.468566 13472 raft_consensus.cc:2749] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:59.547195 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:59.569701 13515 raft_consensus.cc:697] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:59.570775 13515 consensus_queue.cc:237] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:59.571661 13427 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:59.470404 13375 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:59.454994 13344 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:59.540665 10842 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:16:59.554906 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:59.555569 13497 log.cc:927] Time spent T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.445s	user 0.000s	sys 0.002s
I20260709 13:16:59.556409 13337 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:59.557320 10950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 1532) took 1999 ms. Trace:
W20260709 13:16:59.583396 10767 leader_election.cc:336] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Timed out: RequestConsensusVote RPC to 127.9.116.130:37895 timed out after 1.658s (SENT)
I20260709 13:16:59.584235 10767 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:59.569816 10942 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:59.572310 13427 raft_consensus.cc:697] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:59.583140 10950 rpcz_store.cc:276] 0709 13:16:57.558040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:57.695769 (+137729us) service_pool.cc:224] Handling call
0709 13:16:57.696095 (+   326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:57.700303 (+  4208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:57.700344 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:57.706127 (+  5783us) log.cc:844] Serialized 26 byte log entry
0709 13:16:57.954088 (+247961us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:57.973398 (+ 19310us) log.cc:844] Serialized 14 byte log entry
0709 13:16:57.973780 (+   382us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:57.973853 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:59.103144 (+1129291us) spinlock_profiling.cc:243] Waited 1.41 ms on lock 0x7b18003d19e0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc83f3 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:16:59.376347 (+273203us) spinlock_profiling.cc:243] Waited 5.23 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:16:59.557102 (+180755us) raft_consensus.cc:1685] finished
0709 13:16:59.557125 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:59.557260 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38663,"prepare.queue_time_us":16731,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":6113,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":12004,"spinlock_wait_cycles":15282688,"thread_start_us":223921,"threads_started":1,"wal-append.queue_time_us":320055}
I20260709 13:16:59.575881 13375 raft_consensus.cc:697] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:16:59.575966 13425 consensus_queue.cc:1059] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:59.576958 10949 raft_consensus.cc:2393] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 8: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:59.589332 10841 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:59.558511 13060 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:16:59.542953 13493 raft_consensus.cc:2749] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:59.543154 13424 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:59.540740 13290 consensus_queue.cc:1048] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:16:59.579763 10797 raft_consensus.cc:1240] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:16:59.475756 13134 raft_consensus.cc:2764] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
W20260709 13:16:59.586844 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 46299.5us
I20260709 13:16:59.590138 10796 raft_consensus.cc:1240] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:59.577889 13344 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:16:59.597069 10942 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
I20260709 13:16:59.582988 13034 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:59.602146 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:59.602679 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:59.602959 10871 raft_consensus.cc:2393] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 4: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:59.603461 10874 raft_consensus.cc:2393] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 6: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:59.603714 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:59.604624 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:59.605225 10874 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 5.
I20260709 13:16:59.606906 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:59.608187 10874 raft_consensus.cc:2368] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 4. Current term is 5.
I20260709 13:16:59.587808 13503 raft_consensus.cc:2749] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:16:59.590126 13289 raft_consensus.cc:2749] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:59.610720 10841 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:59.613602 10871 raft_consensus.cc:1240] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:59.620811 10871 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d32f5e230e464e61b23a41962708d120"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:16:59.621649 10871 raft_consensus.cc:606] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Received request to transfer leadership to TS cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:59.622013 10871 raft_consensus.cc:614] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:59.596870 13424 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:59.623728 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:59.625613 10871 raft_consensus.cc:1275] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:16:59.626075 10873 raft_consensus.cc:2376] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 7. Re-sending same reply.
I20260709 13:16:59.620795 10841 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:16:59.595074 13152 consensus_queue.cc:1059] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:16:59.630357 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:59.609352 13032 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:59.635921 10946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 322) took 1702 ms. Trace:
I20260709 13:16:59.597553 13380 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:59.636389 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:59.639006 13289 consensus_queue.cc:1059] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:59.647128 13032 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:59.648459 13032 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:59.646486 10946 rpcz_store.cc:276] 0709 13:16:57.933594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:57.949005 (+ 15411us) service_pool.cc:224] Handling call
0709 13:16:57.967316 (+ 18311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:57.967845 (+   529us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:16:58.035964 (+ 68119us) spinlock_profiling.cc:243] Waited 1.73 ms on lock 0x7b18004629a0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7dd42c0a 00007f9c87482219 00007f9c7ddcaa40 00007f9c7ddc54e4 00007f9c7ddc5455 00007f9c848ad3c7 00007f9c80ebf9a7 00007f9c80ecf0bf 00007f9c80ecf038
0709 13:16:58.221938 (+185974us) log.cc:844] Serialized 14 byte log entry
0709 13:16:59.121068 (+899130us) log.cc:844] Serialized 14 byte log entry
0709 13:16:59.126368 (+  5300us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:59.127269 (+   901us) log.cc:844] Serialized 26 byte log entry
0709 13:16:59.151336 (+ 24067us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:59.151393 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:59.151497 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:59.466109 (+314612us) spinlock_profiling.cc:243] Waited 6.28 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:16:59.635688 (+169579us) raft_consensus.cc:1685] finished
0709 13:16:59.635715 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:59.635858 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3911,"prepare.queue_time_us":229564,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":74,"raft.queue_time_us":15699,"raft.run_cpu_time_us":337,"raft.run_wall_time_us":13442,"spinlock_wait_cycles":18427520,"thread_start_us":1035976,"threads_started":3,"wal-append.queue_time_us":801629}
I20260709 13:16:59.649937 13289 raft_consensus.cc:3055] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:59.650363 13289 raft_consensus.cc:740] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:59.651224 13289 consensus_queue.cc:260] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:59.652509 13289 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:59.629972 13424 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:59.639803 13375 consensus_queue.cc:237] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:59.642225 10873 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "12682eb70bc84c76879e04a8628ee586"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:16:59.656077 10873 raft_consensus.cc:606] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Received request to transfer leadership to TS 12682eb70bc84c76879e04a8628ee586
I20260709 13:16:59.656476 10873 raft_consensus.cc:614] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:59.657895  9682 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
W20260709 13:16:59.643632 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 76187.7us
I20260709 13:16:59.659371 13355 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:59.659847  9682 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:59.660791 10798 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 5.
W20260709 13:16:59.627854 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 66978.6us
I20260709 13:16:59.621887 13493 raft_consensus.cc:2749] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:59.642084 13134 raft_consensus.cc:2749] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:59.640995 13427 consensus_queue.cc:237] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.5, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:59.673538 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:59.674398 13032 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:59.674556 13253 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:59.675953 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:59.677851 10872 raft_consensus.cc:1240] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:59.684598 13258 consensus_queue.cc:1048] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:16:59.712563 13034 raft_consensus.cc:2749] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:16:59.759613 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 95862.9us
W20260709 13:16:59.759958 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:59.685482 13302 consensus_queue.cc:1059] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.058s
W20260709 13:16:59.760660 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 101039us
W20260709 13:16:59.761636 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 89101.5us
I20260709 13:16:59.765676 11478 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:16:59.768826 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:16:59.769438 10868 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:59.763432 13523 consensus_queue.cc:1048] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 13:16:59.770805 13302 raft_consensus.cc:3055] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:59.771291 13302 raft_consensus.cc:740] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:59.772359 10876 raft_consensus.cc:1240] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:16:59.772226 13302 consensus_queue.cc:260] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:16:59.763741 13451 consensus_queue.cc:1048] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 13:16:59.773559 13302 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:59.774088 13152 raft_consensus.cc:993] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26: : Instructing follower cb0428cc55984daba1ba63744a5102a2 to start an election
I20260709 13:16:59.774564 13152 raft_consensus.cc:1081] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 LEADER]: Signalling peer cb0428cc55984daba1ba63744a5102a2 to start an election
I20260709 13:16:59.775065 13519 consensus_queue.cc:1059] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:16:59.776034 13519 raft_consensus.cc:3055] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:59.778695 11478 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:16:59.776420 13519 raft_consensus.cc:740] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:59.782773 10841 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:59.785748 13413 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:59.786455 13413 raft_consensus.cc:697] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 7 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:16:59.787443 13253 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:59.787882 13134 consensus_queue.cc:1059] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:59.788869 13375 consensus_queue.cc:1048] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:16:59.790724 13212 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:59.796468 10868 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 8.
I20260709 13:16:59.799690 10873 raft_consensus.cc:2410] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:16:59.799757 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:59.800680 10872 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 4.
I20260709 13:16:59.802183 10950 raft_consensus.cc:1240] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:59.802201 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:59.803301 10942 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:59.805913 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:59.806243 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:16:59.806849 10947 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:59.801512 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:59.809628 10868 raft_consensus.cc:2393] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 6: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:59.813740 10942 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 8.
I20260709 13:16:59.814448 10943 raft_consensus.cc:1240] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:16:59.819432 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:59.781723 13380 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:16:59.821130 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:59.828219 10870 raft_consensus.cc:2393] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 6: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:16:59.820212 10876 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 8.
I20260709 13:16:59.798803 13523 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:59.830994 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:59.837616 10872 raft_consensus.cc:2393] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 6: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:59.809096 13134 raft_consensus.cc:2804] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:59.815814 10950 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0"
dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:41020
I20260709 13:16:59.844048 10950 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:59.844587 10950 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:59.834501 10871 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 6.
I20260709 13:16:59.839933 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:59.777184 13501 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:59.892673 10947 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
I20260709 13:16:59.883180 10870 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 5.
I20260709 13:16:59.806744 13451 consensus_queue.cc:1048] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:16:59.868040 10950 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:16:59.893743 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:59.896927 11512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56404 (request call id 518) took 2202 ms. Trace:
I20260709 13:16:59.898846 10841 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:16:59.897505 11512 rpcz_store.cc:276] 0709 13:16:57.694319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:57.694456 (+   137us) service_pool.cc:224] Handling call
0709 13:16:57.714840 (+ 20384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:58.349753 (+634913us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:59.896395 (+1546642us) spinlock_profiling.cc:243] Waited 556 ms on lock 0x7b54005014d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:16:59.896847 (+   452us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1279841664}
I20260709 13:16:59.904811 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:16:59.810767 13253 consensus_queue.cc:1059] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.083s
I20260709 13:16:59.818459 13289 consensus_queue.cc:1059] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 13:16:59.920934 10870 raft_consensus.cc:1240] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:59.921952 10875 raft_consensus.cc:2393] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 9: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:16:59.895431 13375 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:59.936683 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:59.787436 13413 consensus_queue.cc:237] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:16:59.835425 13523 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:16:58.642575 10793 rpcz_store.cc:276] 0709 13:16:57.595007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:58.375296 (+780289us) service_pool.cc:224] Handling call
0709 13:16:58.414276 (+ 38980us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:58.414510 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:16:58.431267 (+ 16757us) log.cc:844] Serialized 14 byte log entry
0709 13:16:58.629911 (+198644us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:58.629984 (+    73us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:16:58.630063 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:58.630153 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:58.630459 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":36806,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":2346,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":5298,"thread_start_us":198157,"threads_started":1}
I20260709 13:16:59.928331 13380 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:59.907226 13288 consensus_queue.cc:1059] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
W20260709 13:17:00.055893 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:00.046417 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 229319us
I20260709 13:17:00.024986 13511 raft_consensus.cc:3055] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:00.040803 13523 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:00.097093 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:00.110162 10950 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:00.108176 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:00.127486 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 228784us
I20260709 13:17:00.132261 10947 raft_consensus.cc:1275] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260709 13:17:00.133700 10947 raft_consensus.cc:1240] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:00.134488 11477 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:00.096298 13511 raft_consensus.cc:740] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:16:59.993527 13375 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:16:59.920137 13519 consensus_queue.cc:260] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:00.138621 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:00.141628 10870 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 8.
I20260709 13:17:00.141901 10941 raft_consensus.cc:1275] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:17:00.128548 13380 raft_consensus.cc:697] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:00.148968 10870 raft_consensus.cc:1240] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:00.152257 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:00.153910 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:00.155062 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:00.156385 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:00.157514 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:00.158016 10793 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:00.159648 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:00.159952 13397 raft_consensus.cc:3055] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:00.160372 13397 raft_consensus.cc:740] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:00.160877 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:00.161487 10795 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:00.161155 13397 consensus_queue.cc:260] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:00.162379 13397 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:00.162487 13501 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:00.111460 13424 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:00.163473 13515 consensus_queue.cc:1048] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:00.163861 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:00.164394 10800 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:00.164819 13424 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:00.165067 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:00.165655 10950 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:17:00.166841 13375 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:00.175382 13511 consensus_queue.cc:260] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:00.180430 10793 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
I20260709 13:17:00.183789 13288 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:00.184629 13288 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:00.185622 13519 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:00.198593 10794 raft_consensus.cc:1240] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 LEADER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:00.214056 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:00.225502 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:00.226311 10941 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:00.230978 10950 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:17:00.236603 10841 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:00.189471 10800 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 6.
I20260709 13:17:00.257079 10795 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
I20260709 13:17:00.260371 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:00.191207 13289 consensus_queue.cc:1048] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.108s
I20260709 13:17:00.268254 10800 raft_consensus.cc:1240] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 4. Current term is 5. Ops: []
W20260709 13:17:00.266657 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 363393us
I20260709 13:17:00.192334 13501 consensus_queue.cc:1059] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 13:17:00.257555 13424 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:00.285228 10941 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
I20260709 13:17:00.288411 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:00.289171 11512 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:00.292529 10841 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:17:00.286166 13302 consensus_meta.cc:287] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.512s	user 0.013s	sys 0.000s
I20260709 13:17:00.301644 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:00.306567 10867 raft_consensus.cc:2410] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 6 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 13:17:00.266320 13534 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:00.315828 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:00.327178 10870 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 6.
I20260709 13:17:00.295822 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:00.344488 11506 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:00.297434 13424 raft_consensus.cc:697] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:00.345861 13424 consensus_queue.cc:237] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:00.350108 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:00.344075 13380 consensus_queue.cc:237] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:00.351490 11504 raft_consensus.cc:1275] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:17:00.288144 13134 raft_consensus.cc:697] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:00.391899 10944 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 4.
I20260709 13:17:00.354492 13534 raft_consensus.cc:697] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 8 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:00.305649 13536 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:17:00.405314 11476 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.006s	sys 0.011s
I20260709 13:17:00.412465 11504 raft_consensus.cc:1240] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 4. Current term is 5. Ops: []
W20260709 13:17:00.423559 11476 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.006s	sys 0.011s
W20260709 13:17:00.424571 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:00.425047 10839 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.008s	sys 0.009s
I20260709 13:17:00.385980 13535 consensus_queue.cc:1048] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.018s
W20260709 13:17:00.416868 11478 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.014s	sys 0.002s
I20260709 13:17:00.403965 13536 raft_consensus.cc:697] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:17:00.428413 11478 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.015s	sys 0.002s
W20260709 13:17:00.425580 10839 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.008s	sys 0.009s
I20260709 13:17:00.439029 11512 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:17:00.442047 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:00.433885 10874 raft_consensus.cc:1240] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
W20260709 13:17:00.431260 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:00.433288 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:00.447764 11507 raft_consensus.cc:1240] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 7 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:00.448637 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:00.449381 11505 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 6.
I20260709 13:17:00.451063 10945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 1522) took 2957 ms. Trace:
I20260709 13:17:00.451485 10945 rpcz_store.cc:276] 0709 13:16:57.493846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:57.495650 (+  1804us) service_pool.cc:224] Handling call
0709 13:16:57.496036 (+   386us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:57.496432 (+   396us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:57.496461 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:58.997088 (+1500627us) spinlock_profiling.cc:243] Waited 2.04 ms on lock 0x7b1800314da0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7dd962a6 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7f7aea11 00007f9c7f7be372 00007f9c80ef0a01 00007f9c80eb543a
0709 13:16:59.022981 (+ 25893us) log.cc:844] Serialized 52 byte log entry
0709 13:16:59.298617 (+275636us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:59.334284 (+ 35667us) log.cc:844] Serialized 14 byte log entry
0709 13:16:59.435200 (+100916us) spinlock_profiling.cc:243] Waited 22.4 ms on lock 0x7b1800314da0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc8522 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:17:00.450503 (+1015303us) spinlock_profiling.cc:243] Waited 5.91 ms on lock 0x7b1800314da0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7dd42c0a 00007f9c87482219 00007f9c7ddcaa40 00007f9c7ddc54e4 00007f9c7ddc5455 00007f9c80ead060 00007f9c80ebf716 00007f9c80ed0287 00007f9c80ed0208
0709 13:17:00.450662 (+   159us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:00.450745 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:00.450816 (+    71us) raft_consensus.cc:1685] finished
0709 13:17:00.450831 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:00.450984 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3180,"prepare.queue_time_us":14603,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":385,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":9916,"spinlock_wait_cycles":71668480,"thread_start_us":258642,"threads_started":1,"wal-append.queue_time_us":315573,"wal-append.run_cpu_time_us":2195,"wal-append.run_wall_time_us":1027969}
I20260709 13:17:00.454087 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:00.455878 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:00.456480 11511 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:00.459587 11506 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
I20260709 13:17:00.429997 13134 consensus_queue.cc:237] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:00.461668 13421 raft_consensus.cc:3055] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:00.462082 13421 raft_consensus.cc:740] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
W20260709 13:17:00.463785 10949 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41014 (request call id 1535) took 2233 ms (client timeout 1759 ms). Trace:
W20260709 13:17:00.464190 10949 rpcz_store.cc:269] 0709 13:16:58.229946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:58.264593 (+ 34647us) service_pool.cc:224] Handling call
0709 13:17:00.463575 (+2198982us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54001e1254. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:00.463705 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2905173760}
I20260709 13:17:00.466320 10875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49290 (request call id 315) took 1130 ms. Trace:
I20260709 13:17:00.466701 10875 rpcz_store.cc:276] 0709 13:16:59.336219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.336575 (+   356us) service_pool.cc:224] Handling call
0709 13:17:00.466129 (+1129554us) spinlock_profiling.cc:243] Waited 584 ms on lock 0x7b54002f08d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:00.466259 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1343506304}
I20260709 13:17:00.468844 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:00.435081 13539 consensus_queue.cc:1059] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:17:00.474669 13443 raft_consensus.cc:697] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 7 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:17:00.475368 13533 log.cc:927] Time spent T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:17:00.475984 13443 consensus_queue.cc:237] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.2, Last appended by leader: 2, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:00.477756 10842 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:00.479434 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 1.14591e+06us
I20260709 13:17:00.480417 10916 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:00.483048 13274 raft_consensus.cc:2804] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:17:00.483484 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:00.484354 13450 raft_consensus.cc:2804] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:00.485438 13414 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:00.485889 10916 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:00.488785 10842 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:00.490130 10945 raft_consensus.cc:2393] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 8: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:00.502544 11511 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 8.
I20260709 13:17:00.508761 10842 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:00.444640 13288 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:00.519991 10842 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:00.491415 13535 raft_consensus.cc:2749] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:17:00.483516 13274 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:00.518567 13539 raft_consensus.cc:2749] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:17:00.527319 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 1.08183e+06us
I20260709 13:17:00.529791 10842 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:00.530184 10947 raft_consensus.cc:1240] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:00.530597 10915 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:00.530946 10842 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:00.536125 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 51323.8us
I20260709 13:17:00.537307 10947 raft_consensus.cc:1240] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:17:00.537751 10945 raft_consensus.cc:1240] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:00.543092 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:00.490281 13474 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:17:00.549141 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 103857us
I20260709 13:17:00.549894 10916 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:00.550381 10947 raft_consensus.cc:1275] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:17:00.551064 10916 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:00.554059 11478 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:00.563768 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:00.566615 10947 raft_consensus.cc:2393] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 6: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:00.570950 10916 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:00.525041 13274 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:00.573048 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:00.574757 10945 raft_consensus.cc:2393] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 8: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:00.533747 13535 raft_consensus.cc:2749] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:00.548725 13501 raft_consensus.cc:2804] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:00.547714 13499 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:00.547962 13060 raft_consensus.cc:2764] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:17:00.522418 13545 raft_consensus.cc:2749] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:17:00.490063 13414 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:00.539898 13424 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:00.563705 13409 raft_consensus.cc:2804] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:00.527482 13405 consensus_queue.cc:1048] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 13:17:00.435261 13534 consensus_queue.cc:237] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:00.585078 13350 raft_consensus.cc:2764] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:17:00.552387 13397 consensus_queue.cc:1059] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:17:00.524382 13523 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:00.572860 12996 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:00.590895 13501 raft_consensus.cc:697] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:00.603997 12996 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:00.553493 13485 consensus_queue.cc:1059] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:17:00.563902 13439 consensus_queue.cc:1059] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 13:17:00.607751 13485 raft_consensus.cc:3055] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:00.608035 13439 raft_consensus.cc:3055] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:00.608152 13485 raft_consensus.cc:740] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:00.608500 13439 raft_consensus.cc:740] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:00.609078 13485 consensus_queue.cc:260] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:00.609258 13439 consensus_queue.cc:260] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:00.610340 13439 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:00.570993 13524 consensus_queue.cc:1059] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:17:00.593380 13409 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:00.612073 13409 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:00.598750 13424 raft_consensus.cc:697] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:00.614796 13424 consensus_queue.cc:237] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:00.616261 13421 consensus_queue.cc:260] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:00.600999 13414 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:00.601609 13032 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:00.593163 13413 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:00.621832 13034 raft_consensus.cc:3055] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:00.622301 13034 raft_consensus.cc:740] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:00.621961 13413 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:00.624675 13358 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:00.619254 13501 consensus_queue.cc:237] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:00.593667 13560 raft_consensus.cc:2749] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:17:00.586907 13450 raft_consensus.cc:697] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 4 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:00.525705 13536 consensus_queue.cc:237] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:00.669749 13450 consensus_queue.cc:237] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:00.562520 13474 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:00.606725 13523 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:00.677163 13485 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:00.677202 13511 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:00.678342 13474 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:00.708534  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:00.726451 10797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1514) took 1390 ms. Trace:
I20260709 13:17:00.727011 10797 rpcz_store.cc:276] 0709 13:16:59.335572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.337001 (+  1429us) service_pool.cc:224] Handling call
0709 13:16:59.337315 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:59.599411 (+262096us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:00.725663 (+1126252us) spinlock_profiling.cc:243] Waited 242 ms on lock 0x7b540024efd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:00.726237 (+   574us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":556999424}
W20260709 13:17:00.731823 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:00.737217 10798 raft_consensus.cc:3055] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:00.742053 10798 raft_consensus.cc:740] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:00.745235 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 1491) took 1071 ms. Trace:
I20260709 13:17:00.739579 10797 raft_consensus.cc:1275] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:17:00.741312 10793 raft_consensus.cc:3055] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:00.604493 12996 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:00.790140 13034 consensus_queue.cc:260] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:00.605394 13337 raft_consensus.cc:3055] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:00.798112 13301 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:00.798136 13337 raft_consensus.cc:740] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:00.799248 13301 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:00.746321 10793 raft_consensus.cc:740] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:00.800472 13337 consensus_queue.cc:260] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:00.801563 10793 consensus_queue.cc:260] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.2, Last appended by leader: 2, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:00.802860 13337 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:00.751446 10916 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:00.803205 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:00.805642 13405 raft_consensus.cc:2764] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
I20260709 13:17:00.806509 13405 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:00.807297 13405 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:00.811031 10943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 1903) took 1004 ms. Trace:
I20260709 13:17:00.811533 10943 rpcz_store.cc:276] 0709 13:16:59.806024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.806347 (+   323us) service_pool.cc:224] Handling call
0709 13:16:59.814204 (+  7857us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:59.814855 (+   651us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:00.810526 (+995671us) spinlock_profiling.cc:243] Waited 7.54 ms on lock 0x7b540039b050. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:00.810750 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17342720}
I20260709 13:17:00.813860 10948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 337) took 1368 ms. Trace:
I20260709 13:17:00.755261 13212 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:00.814316 10948 rpcz_store.cc:276] 0709 13:16:59.445369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.465295 (+ 19926us) service_pool.cc:224] Handling call
0709 13:16:59.465542 (+   247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:59.523122 (+ 57580us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:00.813186 (+1290064us) spinlock_profiling.cc:243] Waited 41.6 ms on lock 0x7b54003aff54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:00.813604 (+   418us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":95598336}
I20260709 13:17:00.815083 13212 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:00.817778 13212 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:17:00.819707 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 240145us
I20260709 13:17:00.821085 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:00.821391 10916 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:00.755784 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:00.763648 11504 raft_consensus.cc:1240] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:00.765165 10795 raft_consensus.cc:1240] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 4. Current term is 5. Ops: []
W20260709 13:17:00.770280 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:00.824671 13531 consensus_queue.cc:1048] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:00.777503 10874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1485) took 1164 ms. Trace:
I20260709 13:17:00.803102 10793 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:00.779656 13421 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:00.840437 13375 consensus_queue.cc:1048] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:00.841554 10941 raft_consensus.cc:1240] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:00.838450 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:00.826776 10874 rpcz_store.cc:276] 0709 13:16:59.612409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.622288 (+  9879us) service_pool.cc:224] Handling call
0709 13:16:59.622578 (+   290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:00.444912 (+822334us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:00.776611 (+331699us) spinlock_profiling.cc:243] Waited 811 ms on lock 0x7b540037b254. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:00.777169 (+   558us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1865100160}
I20260709 13:17:00.842942 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:00.843109 10795 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 6.
I20260709 13:17:00.844123 10947 raft_consensus.cc:2410] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 6 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 13:17:00.847276 13419 consensus_queue.cc:1059] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:00.850201 13427 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:00.851118 13427 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:00.854435 13397 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:00.855355 13397 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:00.858176 13539 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:00.859081 13473 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:00.859733 13545 raft_consensus.cc:3055] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:00.860291 13545 raft_consensus.cc:740] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:00.859978 13473 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:00.860885 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:00.861898 10948 raft_consensus.cc:2393] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 8: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:00.861446 13545 consensus_queue.cc:260] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:00.781010 13258 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:00.588798 13253 consensus_queue.cc:1048] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 13:17:00.744351 10798 consensus_queue.cc:260] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:17:00.745779 10873 rpcz_store.cc:276] 0709 13:16:59.673143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.673276 (+   133us) service_pool.cc:224] Handling call
0709 13:17:00.744548 (+1071272us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b540006b3d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:00.744900 (+   352us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":260584704}
I20260709 13:17:00.837288 13474 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:00.810456 13405 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:00.817778 10944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36714 (request call id 331) took 1374 ms. Trace:
I20260709 13:17:00.838482 13086 raft_consensus.cc:3055] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:00.892588 13212 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:00.865144 13301 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:00.862768 13473 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:00.897957 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:00.909050 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:00.901460 10944 rpcz_store.cc:276] 0709 13:16:59.443685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.452676 (+  8991us) service_pool.cc:224] Handling call
0709 13:17:00.817347 (+1364671us) spinlock_profiling.cc:243] Waited 836 ms on lock 0x7b54000ab9d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:00.817576 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1922813696}
I20260709 13:17:00.917848 10945 raft_consensus.cc:2410] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:17:00.911646 10767 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:00.918650 13212 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:00.919853 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 365735us
I20260709 13:17:00.921459 13212 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:00.921238 13409 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:00.923096 13258 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:00.923985 13413 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:00.924868 13450 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:00.925839 13450 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:00.930122 13450 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:00.935055 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 113866us
I20260709 13:17:00.939348 10876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1502) took 1109 ms. Trace:
I20260709 13:17:00.939929 10876 rpcz_store.cc:276] 0709 13:16:59.829901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.830072 (+   171us) service_pool.cc:224] Handling call
0709 13:16:59.830392 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:59.830591 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:16:59.893777 (+ 63186us) log.cc:844] Serialized 14 byte log entry
0709 13:17:00.819409 (+925632us) spinlock_profiling.cc:243] Waited 1.87 ms on lock 0x7b1800324820. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a5523a 00007f9c80ebfa2a
0709 13:17:00.938625 (+119216us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:00.938714 (+    89us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:00.938844 (+   130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:00.938941 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:00.939135 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24064,"prepare.queue_time_us":62896,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":6806,"spinlock_wait_cycles":4311296,"thread_start_us":70489,"threads_started":1}
I20260709 13:17:00.942920 13443 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:00.946314 10793 raft_consensus.cc:1275] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 6 index: 2. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260709 13:17:00.946880 10842 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:17:00.947885 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 83656.2us
I20260709 13:17:00.950567 13134 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:00.888543 13539 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:00.907400 13086 raft_consensus.cc:740] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:00.900175 13545 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:00.963145 10943 raft_consensus.cc:2393] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 7: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:00.955801 11511 raft_consensus.cc:1240] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 8 LEADER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:00.968071 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:00.982918 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:00.962183 11508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56416 (request call id 511) took 1592 ms. Trace:
I20260709 13:17:00.988933 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:00.907708 13531 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:00.962636 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:00.951136 13134 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:00.955421 11505 raft_consensus.cc:1275] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:17:00.983875 11508 rpcz_store.cc:276] 0709 13:16:59.371648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.371775 (+   127us) service_pool.cc:224] Handling call
0709 13:17:00.961747 (+1589972us) spinlock_profiling.cc:243] Waited 88.4 ms on lock 0x7b54001f90d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:00.961970 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":203415424}
I20260709 13:17:00.954903 13429 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:00.960901 13397 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:01.022050 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:01.023041 10946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 338) took 1178 ms. Trace:
I20260709 13:17:00.957197 13539 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:00.958702 13328 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:00.951588 13473 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:01.031853 10946 rpcz_store.cc:276] 0709 13:16:59.800935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.801773 (+   838us) service_pool.cc:224] Handling call
0709 13:16:59.818008 (+ 16235us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:00.522195 (+704187us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:00.522226 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:00.541394 (+ 19168us) log.cc:844] Serialized 26 byte log entry
0709 13:17:00.923124 (+381730us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:00.923229 (+   105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:00.923360 (+   131us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:00.976750 (+ 53390us) raft_consensus.cc:1685] finished
0709 13:17:00.976772 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:00.979376 (+  2604us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":740890,"raft.queue_time_us":704271,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":23405,"thread_start_us":318346,"threads_started":1,"wal-append.queue_time_us":367401}
I20260709 13:17:01.048224 13424 consensus_queue.cc:1048] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:01.049098 10792 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7.
I20260709 13:17:00.961241 13358 consensus_queue.cc:1059] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.074s
I20260709 13:17:01.050978 10795 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
I20260709 13:17:00.950731 13274 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:01.062367 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:01.062147 13328 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:00.953773 13535 consensus_queue.cc:1059] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.099s
W20260709 13:17:01.063324 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 68506.5us
I20260709 13:17:01.060276 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:01.065127 10793 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 5: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:01.063400 10798 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:01.063556 12996 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:01.067150 13535 raft_consensus.cc:3055] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:01.067675 13535 raft_consensus.cc:740] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:01.069142 12996 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:01.068974 13535 consensus_queue.cc:260] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:01.070370 13535 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:01.070482 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:01.070825 13152 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:01.071143 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:01.070958 13473 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:01.071529 13152 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:01.071832 10942 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
I20260709 13:17:01.071772 13350 consensus_queue.cc:1059] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.078s
I20260709 13:17:01.073045 13331 raft_consensus.cc:3055] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:01.073467 13331 raft_consensus.cc:740] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:01.073998 13152 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:01.075049 13105 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:01.074730 13331 consensus_queue.cc:260] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:01.075569 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:01.076421 10945 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 3.
I20260709 13:17:01.063939 13429 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:01.017355 13337 consensus_queue.cc:1059] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 13:17:01.079120 10799 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 7.
I20260709 13:17:01.031997 13258 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:01.080073 13337 raft_consensus.cc:3055] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:01.080520 13337 raft_consensus.cc:740] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:01.081554 13337 consensus_queue.cc:260] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:00.964087 13086 consensus_queue.cc:260] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:00.962894 13474 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:01.085686 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:01.070662 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:01.137825 10944 raft_consensus.cc:1240] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:17:01.250017 13375 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:01.075130 13519 consensus_meta.cc:287] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.878s	user 0.018s	sys 0.032s
I20260709 13:17:01.251638 13086 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:01.255832 13375 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:01.247000 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 180374us
I20260709 13:17:01.258440 13375 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:01.252118 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:01.263811 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 404974us
I20260709 13:17:01.201119 11477 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:01.151070 10916 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
W20260709 13:17:01.193130 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:01.280809 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:01.273691 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:01.200306 10950 raft_consensus.cc:1275] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:17:01.147351 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:01.032109 13134 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:01.292114 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 258322us
I20260709 13:17:01.293013 10916 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:01.294149 13350 raft_consensus.cc:2804] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:01.294687 13350 raft_consensus.cc:697] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 8 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:01.296056 13350 consensus_queue.cc:237] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:01.301229 13105 raft_consensus.cc:2749] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:01.138269 10876 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 5.
I20260709 13:17:01.180948 10766 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:01.285921 10942 raft_consensus.cc:1240] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:01.248016 13540 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:01.249778 13443 raft_consensus.cc:2749] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:01.265234 13331 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:01.266729 13337 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:01.239341 13405 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:01.340729 10767 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:01.332320 12996 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:17:01.344960 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 311020us
I20260709 13:17:01.340003 13383 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:01.181695 13531 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:01.347466 11478 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:17:01.086341 13413 consensus_queue.cc:1175] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" responder_term: 8 status { last_received { term: 8 index: 7 } last_committed_idx: 7 last_received_current_leader { term: 8 index: 7 } }
I20260709 13:17:01.338104 13473 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:01.347322 13539 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:01.347656 12996 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:01.354573 13383 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:01.371420 13570 raft_consensus.cc:2804] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:01.372116 13570 raft_consensus.cc:697] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 5 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:01.372797 13383 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:01.366894 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:01.373271 13570 consensus_queue.cc:237] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:01.360682 13152 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:01.375617 13152 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:01.376713 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:01.377887 13152 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:01.379415 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:01.380105 10870 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 6.
I20260709 13:17:01.381237 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:01.382113 10870 raft_consensus.cc:2410] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 13:17:01.382200 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 204821us
W20260709 13:17:01.383293 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 253543us
I20260709 13:17:01.384222 10872 raft_consensus.cc:1240] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:01.385843 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:01.367075 13253 raft_consensus.cc:2804] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:01.387010 13253 raft_consensus.cc:697] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 4 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
W20260709 13:17:01.387158 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 67911.5us
I20260709 13:17:01.388077 13253 consensus_queue.cc:237] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:01.389107 10767 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:01.367805 13501 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:01.391258 13501 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:01.393285 13501 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:01.355237 13450 consensus_queue.cc:1059] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 13:17:01.394939 13539 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:01.395972 13450 raft_consensus.cc:3055] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:01.396405 13450 raft_consensus.cc:740] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:01.397521 13539 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:01.397581 13450 consensus_queue.cc:260] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:01.367944 10797 raft_consensus.cc:3055] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:01.399402 10797 raft_consensus.cc:740] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:01.370371 13385 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:01.368397 13405 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
W20260709 13:17:01.367175 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:01.403690 10869 raft_consensus.cc:2410] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 13:17:01.404981 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:01.355028 13105 raft_consensus.cc:2749] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:17:01.408558 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:01.404568 11509 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 6.
I20260709 13:17:01.410211 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:01.690968 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 364481us
I20260709 13:17:01.467810 10948 raft_consensus.cc:1275] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 13:17:01.501858 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:01.673393 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 264794us
W20260709 13:17:01.680406 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:01.484041 10798 raft_consensus.cc:1275] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
W20260709 13:17:01.699509 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 308688us
I20260709 13:17:01.400036 12996 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:01.609730 10792 raft_consensus.cc:1240] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:01.694571 10798 pending_rounds.cc:85] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 5
W20260709 13:17:01.431191 10915 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.116.132:40555 timed out after 1.854s (SENT) [suppressed 47 similar messages]
I20260709 13:17:01.431804 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:01.431348 10797 consensus_queue.cc:260] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:01.410416 13383 raft_consensus.cc:2749] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:01.726572 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:01.706857 10798 pending_rounds.cc:107] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 13:17:01.735105 10798 raft_consensus.cc:2889] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:01.416530 13375 raft_consensus.cc:2804] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:01.701319 13450 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:01.542078 13421 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:01.698536 13274 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:17:01.771623 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 69771.2us
I20260709 13:17:01.571532 13105 consensus_queue.cc:1048] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.165s
I20260709 13:17:01.688272 13580 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:01.712738 13474 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:01.405454 13358 consensus_queue.cc:1048] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.333s
I20260709 13:17:01.790051 10793 raft_consensus.cc:2393] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 6: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:01.367082 13540 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:01.766117 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:01.795547 10798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 1922) took 1060 ms. Trace:
W20260709 13:17:01.779629 13086 consensus_meta.cc:287] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.516s	user 0.011s	sys 0.000s
I20260709 13:17:01.801601 10798 rpcz_store.cc:276] 0709 13:17:00.734996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:00.735791 (+   795us) service_pool.cc:224] Handling call
0709 13:17:00.736722 (+   931us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:01.795233 (+1058511us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:01.795473 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:01.769444 13375 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:01.612109 13473 consensus_queue.cc:1059] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.189s
I20260709 13:17:01.515110 13385 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:01.831441 10800 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:01.842092 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:01.843797 13559 log.cc:927] Time spent T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:17:01.844507 13511 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:01.845266 13511 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:01.847072 13474 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:01.847187 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:01.847656 13474 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:01.848100 10801 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:01.849888 13474 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:01.793083 13380 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:01.821880 13383 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:01.794248 13301 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:01.856328 13531 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:01.866176 13474 raft_consensus.cc:3055] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:17:01.877944 13134 consensus_meta.cc:287] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.528s	user 0.009s	sys 0.002s
I20260709 13:17:01.823312 13375 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:01.824590 13421 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:01.882017 13405 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:17:01.906333 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 139754us
I20260709 13:17:01.855758 13580 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:01.903074 13474 raft_consensus.cc:740] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:01.958145 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:01.963531 10842 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:17:01.965250 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:01.965565 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:01.971161 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 539989us
W20260709 13:17:01.973290 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:01.973747 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:01.974148 11504 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 7 LEARNER]: Advancing to term 8
I20260709 13:17:01.975050 13493 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:01.975474 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:01.941715 13152 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:01.877754 13380 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:01.976986 10842 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:01.980151 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:01.903182 13383 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:02.006351 13383 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:02.006999 13443 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.007555 13443 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:02.009569 13443 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:02.010449 13596 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:01.992686 11505 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 6.
I20260709 13:17:02.011137 13596 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:02.011319 10874 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:02.011610 13596 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:02.013237 13385 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:02.013010 12996 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:02.013692 13380 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:01.958691 13590 consensus_queue.cc:1059] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.141s
I20260709 13:17:02.014907 13598 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.015462 13598 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:02.017712 13598 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:01.961833 11503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56416 (request call id 515) took 1520 ms. Trace:
I20260709 13:17:02.019797 13540 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:01.994426 13421 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:01.909205 13134 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:02.000191 13580 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:02.020860 13580 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:02.021222 13580 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:02.021607 13580 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:01.958465 13588 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.022099 13134 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:02.003115 10800 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 7.
I20260709 13:17:02.023114 10867 raft_consensus.cc:1240] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:02.022650 13588 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:02.024438 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 321) took 1057 ms. Trace:
I20260709 13:17:02.024926 13152 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:02.025547 13588 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:02.003783 13576 consensus_queue.cc:1048] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:17:01.962296 13595 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:02.027585 13152 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:02.027536 13595 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:02.028680 13405 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.029321 13405 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:02.029894 13595 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:02.024824 10873 rpcz_store.cc:276] 0709 13:17:00.967483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:00.979414 (+ 11931us) service_pool.cc:224] Handling call
0709 13:17:01.025436 (+ 46022us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:17:01.188802 (+163366us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:01.188932 (+   130us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:17:01.259420 (+ 70488us) log.cc:844] Serialized 78 byte log entry
0709 13:17:01.415226 (+155806us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:01.502159 (+ 86933us) log.cc:844] Serialized 14 byte log entry
0709 13:17:01.680138 (+177979us) log.cc:844] Serialized 14 byte log entry
0709 13:17:01.819797 (+139659us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:01.819917 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:02.024185 (+204268us) raft_consensus.cc:1685] finished
0709 13:17:02.024204 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:02.024371 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":251046,"prepare.queue_time_us":86702,"prepare.run_cpu_time_us":132,"prepare.run_wall_time_us":3379,"raft.queue_time_us":314841,"raft.run_cpu_time_us":488,"raft.run_wall_time_us":21084,"spinlock_wait_cycles":679680,"thread_start_us":362117,"threads_started":3,"wal-append.queue_time_us":701862}
I20260709 13:17:02.032384 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:02.033797 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:02.034142 13328 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:02.019871 10794 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:02.025465 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:02.036166 10867 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 4.
W20260709 13:17:02.037957 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 48103us
I20260709 13:17:01.995589 11508 raft_consensus.cc:1275] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
W20260709 13:17:01.995414 13439 consensus_meta.cc:287] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 1.385s	user 0.013s	sys 0.009s
I20260709 13:17:02.037279 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:02.046157 10876 consensus_meta.cc:287] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.669s	user 0.008s	sys 0.004s
I20260709 13:17:02.050266 10876 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 8.
I20260709 13:17:02.051592 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:02.053122 10867 raft_consensus.cc:1240] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:02.034515 10873 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 6.
I20260709 13:17:01.959301 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:02.071717 10794 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 9.
I20260709 13:17:02.010541 13604 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:02.088035 11509 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56404 (request call id 527) took 1645 ms (client timeout 2018 ms). Trace:
I20260709 13:17:02.088322 13604 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:02.088531 11509 rpcz_store.cc:269] 0709 13:17:00.442268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:00.468559 (+ 26291us) service_pool.cc:224] Handling call
0709 13:17:02.087833 (+1619274us) spinlock_profiling.cc:243] Waited 768 ms on lock 0x7b54004290d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:02.087971 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1765312384}
I20260709 13:17:01.923024 13301 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:02.046766 13535 consensus_meta.cc:287] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.927s	user 0.008s	sys 0.004s
I20260709 13:17:02.113106 10875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 325) took 1143 ms. Trace:
I20260709 13:17:02.100860 10797 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:02.121651 10944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 1552) took 1133 ms. Trace:
I20260709 13:17:02.133930 10944 rpcz_store.cc:276] 0709 13:17:00.988707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:00.988867 (+   160us) service_pool.cc:224] Handling call
0709 13:17:01.031215 (+ 42348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:01.255425 (+224210us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:02.121362 (+865937us) spinlock_profiling.cc:243] Waited 55.1 ms on lock 0x7b540004db54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:02.121588 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":126693120}
I20260709 13:17:02.074329 10767 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:02.006831 13493 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:02.076484 13511 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:02.077288 11505 raft_consensus.cc:1275] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:17:02.077849 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:02.080762 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:02.085989 10948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 1916) took 1036 ms. Trace:
I20260709 13:17:02.320822 10948 rpcz_store.cc:276] 0709 13:17:01.049304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.260836 (+211532us) service_pool.cc:224] Handling call
0709 13:17:01.333565 (+ 72729us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:01.692624 (+359059us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:02.085723 (+393099us) spinlock_profiling.cc:243] Waited 98.3 ms on lock 0x7b54001e1754. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:02.085922 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":226198272}
I20260709 13:17:02.087596 12996 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:02.091203 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:02.340294 11511 raft_consensus.cc:2393] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 5: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:02.075412 10872 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 7.
I20260709 13:17:02.090728 13604 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:02.132699 10875 rpcz_store.cc:276] 0709 13:17:00.969164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.017102 (+ 47938us) service_pool.cc:224] Handling call
0709 13:17:01.203113 (+186011us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:01.344923 (+141810us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:01.344954 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:01.362516 (+ 17562us) log.cc:844] Serialized 26 byte log entry
0709 13:17:01.681636 (+319120us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:01.681686 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:01.681767 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:02.112685 (+430918us) raft_consensus.cc:1685] finished
0709 13:17:02.112716 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:02.112907 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47486,"raft.queue_time_us":83115,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":3307,"spinlock_wait_cycles":175360,"thread_start_us":271551,"threads_started":1,"wal-append.queue_time_us":706325}
I20260709 13:17:02.101998 13474 consensus_queue.cc:260] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:02.161341 13337 consensus_meta.cc:287] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.817s	user 0.010s	sys 0.000s
W20260709 13:17:02.067185 10916 connection.cc:256] RPC call timeout handler was delayed by -1.8563655220000328s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2146s for a response
I20260709 13:17:02.257321 13405 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:02.097450 10915 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:02.095003 11506 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
I20260709 13:17:02.019156 11503 rpcz_store.cc:276] 0709 13:17:00.441257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:00.443507 (+  2250us) service_pool.cc:224] Handling call
0709 13:17:00.450557 (+  7050us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:01.080203 (+629646us) spinlock_profiling.cc:243] Waited 1.44 ms on lock 0x7b180032b720. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c80a5d4bc 00007f9c7ddab48b 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc5455 00007f9c80ead060 00007f9c80eadbfb 00007f9c80eb864b
0709 13:17:01.080496 (+   293us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:01.080534 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:01.081198 (+   664us) log.cc:844] Serialized 26 byte log entry
0709 13:17:01.286484 (+205286us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:01.315087 (+ 28603us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:01.337806 (+ 22719us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:01.961315 (+623509us) raft_consensus.cc:1685] finished
0709 13:17:01.961342 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:01.961620 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":78551,"raft.queue_time_us":85025,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":14590,"spinlock_wait_cycles":3302272,"thread_start_us":222403,"threads_started":2,"wal-append.queue_time_us":787947}
I20260709 13:17:02.176661 13301 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:02.116266 13545 consensus_meta.cc:287] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 1.152s	user 0.013s	sys 0.000s
I20260709 13:17:02.260396 11504 raft_consensus.cc:1275] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 8 LEARNER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 13:17:02.283957 13375 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:02.353147 13443 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:02.353576 13534 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.353736 13443 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:02.354297 13599 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.354120 13534 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:02.355934 13443 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:17:02.356549 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 259179us
I20260709 13:17:02.286060 10801 raft_consensus.cc:2410] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:17:02.092307 13409 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:02.110549 10871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 324) took 1141 ms. Trace:
I20260709 13:17:02.358343 13405 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:02.160080 10950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 1920) took 1109 ms. Trace:
I20260709 13:17:02.358996 13405 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:02.091333 13427 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:02.359338 10868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 323) took 1363 ms. Trace:
I20260709 13:17:02.359500 13405 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:02.360508 13535 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.358425 10871 rpcz_store.cc:276] 0709 13:17:00.968697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.020324 (+ 51627us) service_pool.cc:224] Handling call
0709 13:17:01.076983 (+ 56659us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:01.077509 (+   526us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:01.077537 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:01.350612 (+273075us) log.cc:844] Serialized 52 byte log entry
0709 13:17:01.767738 (+417126us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:01.824457 (+ 56719us) log.cc:844] Serialized 14 byte log entry
0709 13:17:01.854527 (+ 30070us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:01.854612 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:02.110211 (+255599us) raft_consensus.cc:1685] finished
0709 13:17:02.110230 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:02.110380 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29747,"prepare.queue_time_us":60245,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":12739,"raft.queue_time_us":74628,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":126,"spinlock_wait_cycles":6620416,"thread_start_us":351455,"threads_started":1,"wal-append.queue_time_us":746384}
I20260709 13:17:02.361070 13535 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:02.359246 10950 rpcz_store.cc:276] 0709 13:17:01.050767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.317885 (+267118us) service_pool.cc:224] Handling call
0709 13:17:01.350313 (+ 32428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:01.398452 (+ 48139us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:17:01.537049 (+138597us) log.cc:844] Serialized 14 byte log entry
0709 13:17:01.662619 (+125570us) log.cc:844] Serialized 14 byte log entry
0709 13:17:02.124454 (+461835us) spinlock_profiling.cc:243] Waited 2.3 ms on lock 0x7b180041c680. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7dd42c0a 00007f9c87482219 00007f9c80a6bd70 00007f9c80a5f293 00007f9c80a54cea 00007f9c80a5523a 00007f9c80ebfa2a 00007f9c80ecf0bf 00007f9c80ecf038
0709 13:17:02.159695 (+ 35241us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:02.159757 (+    62us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:02.159807 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:02.159874 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:02.160010 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16384,"prepare.queue_time_us":611180,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":90,"raft.queue_time_us":75041,"raft.run_cpu_time_us":323,"raft.run_wall_time_us":31177,"spinlock_wait_cycles":5298048,"thread_start_us":128707,"threads_started":1}
W20260709 13:17:02.105484 10799 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 1510) took 4511 ms (client timeout 2146 ms). Trace:
W20260709 13:17:02.287065 13331 consensus_meta.cc:287] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.953s	user 0.009s	sys 0.005s
I20260709 13:17:02.119441 13429 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:02.166931 10941 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7.
I20260709 13:17:02.270758 13596 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:02.279834 10797 raft_consensus.cc:1275] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 13:17:02.075603 10874 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 8.
W20260709 13:17:02.284832 11478 connection.cc:256] RPC call timeout handler was delayed by -1.3104471400001785s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1524s for a response
I20260709 13:17:02.347764 10876 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 8.
I20260709 13:17:02.350863 11504 pending_rounds.cc:85] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586: Aborting all ops after (but not including) 5
I20260709 13:17:02.351908 13474 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:02.352649 13493 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:02.272549 13580 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:02.354997 13328 consensus_queue.cc:1059] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:02.354916 13599 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:02.352638 13600 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:02.350804 13413 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.369335 11504 pending_rounds.cc:107] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 13:17:02.381417 11504 raft_consensus.cc:2889] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 8 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:02.359742 10868 rpcz_store.cc:276] 0709 13:17:00.968331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:00.978746 (+ 10415us) service_pool.cc:224] Handling call
0709 13:17:01.032761 (+ 54015us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:01.368278 (+335517us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:01.368318 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:01.368388 (+    70us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:01.368427 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:02.331150 (+962723us) spinlock_profiling.cc:243] Waited 329 ms on lock 0x7b540006c2d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:02.331221 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:02.331428 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":756780416}
I20260709 13:17:02.372103 10941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41020 (request call id 1914) took 1323 ms. Trace:
I20260709 13:17:02.368547 10797 pending_rounds.cc:85] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 3
I20260709 13:17:02.355022 13375 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:02.383154 10797 pending_rounds.cc:107] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 13:17:02.385593 10797 raft_consensus.cc:2889] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:17:02.383760 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 416448us
W20260709 13:17:02.364199 10799 rpcz_store.cc:269] 0709 13:16:57.594188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:58.033207 (+439019us) service_pool.cc:224] Handling call
0709 13:17:02.105207 (+4072000us) spinlock_profiling.cc:243] Waited 3.04 s on lock 0x7b54002116d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:02.105371 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7002248576}
W20260709 13:17:02.384330 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:02.383035 10941 rpcz_store.cc:276] 0709 13:17:01.048571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.059108 (+ 10537us) service_pool.cc:224] Handling call
0709 13:17:02.371919 (+1312811us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54000a96d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:02.372040 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2537198976}
I20260709 13:17:02.370023 13380 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.388270 11478 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:02.388447 10797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 1930) took 1022 ms. Trace:
I20260709 13:17:02.392527 10797 rpcz_store.cc:276] 0709 13:17:01.366150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.366293 (+   143us) service_pool.cc:224] Handling call
0709 13:17:01.367783 (+  1490us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:02.061934 (+694151us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b58007334d0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80e110aa 00007f9c80e93447 00007f9c80eabfe7 00007f9c80ead7de 00007f9c80eb7b49 00007f9c80eb8416 00007f9c80eb4d8f 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4
0709 13:17:02.388163 (+326229us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:02.388389 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":246169856}
W20260709 13:17:02.388597 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:02.394002 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:02.382298 12996 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.384677 13493 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:02.417655 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:02.418817 10868 raft_consensus.cc:2410] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
W20260709 13:17:02.420794 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 67696.7us
I20260709 13:17:02.401435 13600 raft_consensus.cc:697] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:02.421362 10767 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:02.385206 13413 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:02.403623 13337 raft_consensus.cc:2749] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:02.418609 13603 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.402684 13380 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:02.430688 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 58010.4us
I20260709 13:17:02.412518 12996 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:02.438401 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:02.440179 10875 raft_consensus.cc:1240] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:02.442234 10868 raft_consensus.cc:2393] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 8: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:02.444695 13429 raft_consensus.cc:2804] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:02.445262 13429 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:02.445670 13429 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:02.433760 13606 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:02.456204 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 50597.8us
I20260709 13:17:02.458899 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:02.459237 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:02.459673 10945 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 4.
I20260709 13:17:02.459945 10868 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:02.461340 10941 raft_consensus.cc:1240] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:02.461314 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:02.457444 11478 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:17:02.428901 13493 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:02.467360 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:02.468278 10945 raft_consensus.cc:2393] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 5: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:02.468482 13608 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:17:02.468938 13446 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 13:17:02.469125 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:02.469787 10944 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 3.
I20260709 13:17:02.470064 13534 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:02.471200 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:02.471638 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:02.471889 11505 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 7.
I20260709 13:17:02.472512 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:02.473037 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:02.473343 10949 raft_consensus.cc:2393] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 9: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:02.474231 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:02.474862 10942 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 7.
W20260709 13:17:02.475476 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:02.477454 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:02.483111 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:02.483952 11505 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 5.
I20260709 13:17:02.486502 13596 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:02.489406 13599 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:02.489843 13429 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:02.462092 13603 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:02.492888 13599 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.493462 13599 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:02.495479 13599 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:17:02.493863 13610 log.cc:927] Time spent T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:17:02.495806 10871 raft_consensus.cc:1240] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 7. Current term is 8. Ops: [5.5-7.6]
I20260709 13:17:02.499135 11477 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:02.464140 13606 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:02.500056 13580 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:02.500080 13375 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.500730 13612 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:02.500841 13375 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:02.501394 13612 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:02.501827 13413 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:02.503569 13413 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:02.503978 13413 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:02.468964 13608 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:02.506353 13385 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.506425 10871 raft_consensus.cc:1275] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 13:17:02.506219 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:02.506223 13608 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:02.509335 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:02.512846 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:02.513779 10944 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 5.
I20260709 13:17:02.514007 13606 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:02.515308 10945 raft_consensus.cc:1275] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:17:02.512879 10950 raft_consensus.cc:1240] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:02.506989 13385 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:02.518270 10870 raft_consensus.cc:3055] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:02.519038 13385 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:02.518652 10870 raft_consensus.cc:740] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:02.504815 13413 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:02.532240 10868 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 5.
I20260709 13:17:02.530054 10767 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:02.526531 10766 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:02.529290 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:02.524772 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:17:02.528632 10764 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.009s	sys 0.026s
W20260709 13:17:02.540154 10764 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.736s	user 0.010s	sys 0.027s
I20260709 13:17:02.546484 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:02.547971 10945 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 5.
I20260709 13:17:02.524184 13493 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:02.550222 10950 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 5.
I20260709 13:17:02.551728 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:02.551589 13600 consensus_queue.cc:237] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:02.552516 11505 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 9.
I20260709 13:17:02.554273 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:02.554878 10868 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:02.559897 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:02.560250 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:02.561751 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:02.562911 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:02.563638 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:02.564384 11509 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 5.
I20260709 13:17:02.566506 10876 raft_consensus.cc:1275] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:17:02.563469 11505 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 5.
I20260709 13:17:02.567915 10870 consensus_queue.cc:260] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:02.576531 10870 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:02.578120 13603 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:02.581679 11511 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 9.
I20260709 13:17:02.603458 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:02.604353 13446 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:02.604781 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:02.604997 13446 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:02.605427 11512 raft_consensus.cc:2376] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 8. Re-sending same reply.
I20260709 13:17:02.606264 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:02.607160 13446 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:02.612592 13446 raft_consensus.cc:2804] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:02.603597 13034 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:02.614867 13375 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:02.529307 13590 consensus_queue.cc:1059] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 13:17:02.615887 13590 raft_consensus.cc:3055] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:02.616225 13590 raft_consensus.cc:740] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:02.617179 13590 consensus_queue.cc:260] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:02.611060 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:02.618472 13590 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:02.607338 11507 raft_consensus.cc:1217] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[4.5-5.6]   Dedup: 4.5->[5.6-5.6]
I20260709 13:17:02.619572 11507 raft_consensus.cc:1240] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 5. Current term is 6. Ops: [4.5-5.6]
I20260709 13:17:02.567034 10876 pending_rounds.cc:85] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 7
I20260709 13:17:02.621043 10876 pending_rounds.cc:107] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:17:02.621310 10876 raft_consensus.cc:2889] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:02.623032 10868 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 8.
W20260709 13:17:02.623749 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:02.627964 11508 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 7.
I20260709 13:17:02.628573 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:02.568732 13429 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:02.631512 10916 leader_election.cc:336] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Timed out: RequestConsensusVote RPC to 127.9.116.129:39685 timed out after 2.146s (SENT)
I20260709 13:17:02.631652 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:02.636444 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:02.637144 11509 raft_consensus.cc:2376] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 6. Re-sending same reply.
I20260709 13:17:02.637362 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:02.637964 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:02.638504 11509 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 8.
I20260709 13:17:02.631743 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:02.639168 10944 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 8.
I20260709 13:17:02.639568 10947 raft_consensus.cc:1275] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:17:02.640056 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:02.640512 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:02.641151 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:02.632855 10945 raft_consensus.cc:1240] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:02.641916 10947 raft_consensus.cc:1275] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:17:02.633415 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:02.639609 11504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56410 (request call id 504) took 1686 ms. Trace:
I20260709 13:17:02.665408 10945 raft_consensus.cc:1275] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 6. (term mismatch)
I20260709 13:17:02.665259 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:02.664811 10948 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 7.
I20260709 13:17:02.649876 10870 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 7.
I20260709 13:17:02.565411 13588 raft_consensus.cc:2749] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:02.665783 11504 rpcz_store.cc:276] 0709 13:17:00.953129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:00.955846 (+  2717us) service_pool.cc:224] Handling call
0709 13:17:01.963073 (+1007227us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:02.382155 (+419082us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:02.639411 (+257256us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54006dd150. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:02.639551 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2315535232}
I20260709 13:17:02.668864 10871 raft_consensus.cc:1240] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:02.629082 13405 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:02.620606 13446 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:02.579864 13385 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.664902 10949 raft_consensus.cc:1275] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:17:02.671168 10944 raft_consensus.cc:2393] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 7: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:02.671449 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:02.665972 10945 pending_rounds.cc:85] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 5
I20260709 13:17:02.673678 10945 pending_rounds.cc:107] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 13:17:02.674046 10945 raft_consensus.cc:2889] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:02.643935 10915 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:02.543895 13557 consensus_queue.cc:1048] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.099s
W20260709 13:17:02.675853 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 169967us
I20260709 13:17:02.676225 10947 raft_consensus.cc:2393] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 8: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:02.678287 10950 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 9.
I20260709 13:17:02.676673 10767 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:02.677320 13446 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:02.682958 10915 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:02.685079 10876 raft_consensus.cc:1275] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:17:02.687970 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:02.678069 13429 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:02.723788 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:02.729876 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:02.684518 13350 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:02.742422 10799 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:02.690865 13034 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:02.743825 13034 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:02.742543 13350 raft_consensus.cc:697] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:17:02.748436 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 71851.2us
I20260709 13:17:02.749333 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:02.749898 10867 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:02.751034 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:02.752771 10797 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:02.754244 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:17:02.755000 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 227894us
I20260709 13:17:02.745939 13034 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:02.757010 10800 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 4.
I20260709 13:17:02.751543 13606 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:02.758278 13350 consensus_queue.cc:237] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:02.759545 10841 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:17:02.761732 13289 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.765631 11511 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 5.
I20260709 13:17:02.762213 13429 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:02.770841 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 139369us
I20260709 13:17:02.680107 13385 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:02.774828 10949 raft_consensus.cc:1275] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:17:02.742344 13485 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:02.776870 10766 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:02.778275 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:17:02.778985 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:02.779397 10799 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 8.
I20260709 13:17:02.775306 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:02.781031 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:02.784376 10797 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 7.
I20260709 13:17:02.747279 13588 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:02.789199 13588 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:02.789844 13588 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:02.790613 13578 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.774621 13580 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:02.791501 13580 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:02.792021 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:02.792788 10800 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 6.
I20260709 13:17:02.793943 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:02.794359 13613 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.746403 13493 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.795239 13289 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:02.796027 13493 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:02.797612 13289 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:02.798455 13397 consensus_queue.cc:1048] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 13:17:02.758311 13606 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:02.803303 13258 consensus_queue.cc:1048] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:02.804836 13258 consensus_queue.cc:1048] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:02.804934 13606 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:02.805686 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:02.805908 13606 consensus_queue.cc:1048] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:02.806610 10793 raft_consensus.cc:2410] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:17:02.807662 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:02.807909 13614 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.808542 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:02.809338 13439 consensus_queue.cc:1059] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:02.792111 13580 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:02.810371 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:02.809738 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:02.803372 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:02.805070 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:02.775897 10946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 350) took 1728 ms. Trace:
I20260709 13:17:02.821326 10916 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:02.788305 10841 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:02.821002 10946 rpcz_store.cc:276] 0709 13:17:01.047477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.049383 (+  1906us) service_pool.cc:224] Handling call
0709 13:17:01.049812 (+   429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:01.050290 (+   478us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:01.050324 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:02.441390 (+1391066us) spinlock_profiling.cc:243] Waited 1.4 ms on lock 0x7b2c0015d218. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7f7aea11 00007f9c7f7be372 00007f9c80ef0a01 00007f9c80eb543a 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2
0709 13:17:02.457128 (+ 15738us) log.cc:844] Serialized 26 byte log entry
0709 13:17:02.628878 (+171750us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:02.628975 (+    97us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:02.629126 (+   151us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:02.762900 (+133774us) raft_consensus.cc:1685] finished
0709 13:17:02.762919 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:02.775833 (+ 12914us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":571,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":182,"spinlock_wait_cycles":3214208,"thread_start_us":132428,"threads_started":1,"wal-append.queue_time_us":206215}
I20260709 13:17:02.762451 13274 raft_consensus.cc:2804] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:02.819686 10795 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 8.
I20260709 13:17:02.823658 10950 raft_consensus.cc:1275] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:17:02.823722 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 46937.2us
I20260709 13:17:02.826365 10798 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 5.
I20260709 13:17:02.826908 13590 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:02.827363 13443 raft_consensus.cc:3055] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:02.831073 10796 raft_consensus.cc:1275] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:17:02.819038 10801 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 5.
I20260709 13:17:02.830315 13413 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:02.785940 13577 consensus_queue.cc:1048] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.242s
I20260709 13:17:02.908018 10916 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:03.164531 13620 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:03.165166 13620 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:17:03.165517 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:03.166436 13413 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:03.167227 11503 raft_consensus.cc:2393] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 5: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:03.167445 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:02.924007 13212 consensus_queue.cc:1059] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.244s
I20260709 13:17:03.183835 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:02.907980 13600 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:03.168574 13385 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:03.168744 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 380581us
I20260709 13:17:03.009645 13539 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:02.836181 10793 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 5.
I20260709 13:17:02.925065 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:02.949298 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:02.810496 13439 raft_consensus.cc:3055] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:03.197324 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:02.988890 10874 raft_consensus.cc:1275] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:17:03.170476 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:02.992175 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:02.795143 13613 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:02.786640 13608 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:03.163374 12996 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:03.197485 10876 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 6.
I20260709 13:17:02.791340 13578 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:03.172533 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:03.028687 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:03.204598 10871 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:03.205093 10948 raft_consensus.cc:1275] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:17:02.987949 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:03.149034 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:03.055209 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:02.923748 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:17:02.833273 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:02.938773 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:02.987182 10794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 1526) took 1339 ms. Trace:
I20260709 13:17:03.180549 10801 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7.
I20260709 13:17:03.055609 13515 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:02.808759 13331 consensus_queue.cc:1059] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.394s
I20260709 13:17:02.808554 13614 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:03.178534 13443 raft_consensus.cc:740] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:03.181136 10766 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:03.185346 10868 raft_consensus.cc:1275] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 6 index: 2. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260709 13:17:03.186851 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:03.168419 13590 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:02.810766 13493 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:03.192903 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:03.195210 13606 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:03.195366 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:03.336916 10798 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
I20260709 13:17:03.335031 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:03.195619 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:02.950166 13599 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:03.191128 10949 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7.
I20260709 13:17:03.175612 13328 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.948838 10767 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:03.208003 10799 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 8.
I20260709 13:17:03.343698 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:03.236650 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:03.251480 13620 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:03.221422 13274 raft_consensus.cc:697] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 8 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:03.206348 13413 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:03.211529 13380 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:03.251194 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:03.360181 10943 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:03.324963 10946 raft_consensus.cc:2393] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 5: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:03.246958 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:03.260416 13350 raft_consensus.cc:3055] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:17:03.226473 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 318509us
W20260709 13:17:03.329196 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 149711us
W20260709 13:17:03.267288 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:03.327143 10797 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 7.
I20260709 13:17:03.362978 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:03.328632 13289 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:03.331602 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:03.363565 10799 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:03.209352 10841 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:03.364919 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 155661us
W20260709 13:17:03.332680 10867 consensus_meta.cc:287] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.570s	user 0.009s	sys 0.000s
I20260709 13:17:03.366567 10867 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 5.
I20260709 13:17:03.213668 13545 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:17:03.333851 11508 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.011s	sys 0.022s
I20260709 13:17:03.325798 10794 rpcz_store.cc:276] 0709 13:17:01.611052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.689782 (+ 78730us) service_pool.cc:224] Handling call
0709 13:17:02.950040 (+1260258us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b54000e8b54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:02.950166 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":408248448}
I20260709 13:17:02.818892 13485 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:03.386469 13493 consensus_queue.cc:1048] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
W20260709 13:17:03.395972 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 571099us
I20260709 13:17:03.202642 13539 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:03.396855 13539 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:03.397082 10767 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:03.397477 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:03.398136 10794 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
I20260709 13:17:03.399272 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:03.399879 10794 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7.
I20260709 13:17:03.362350 13413 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:03.400682 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:03.401788 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:03.402210 13413 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:03.397109 13485 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:03.402446 10794 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 7.
I20260709 13:17:03.361727 13331 consensus_queue.cc:1243] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer 468a8fd46e824f62b9c2c0d3532a6d26 log is divergent from this leader: its last log entry 6.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:17:03.403688 13274 consensus_queue.cc:237] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:03.404588 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:03.405671 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:03.406328 10794 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
I20260709 13:17:03.407181 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:03.407805 10794 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:03.393520 13424 consensus_queue.cc:1048] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 13:17:03.340397 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:03.413239 10948 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:17:03.338564 10950 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
W20260709 13:17:03.339673 13446 consensus_meta.cc:287] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.615s	user 0.005s	sys 0.005s
I20260709 13:17:03.221284 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:03.346367 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:03.350970 10941 raft_consensus.cc:2393] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 9: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:03.207082 10875 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 7.
I20260709 13:17:03.352810 10842 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:03.243067 13405 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:03.327996 13590 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:03.431537 10799 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 6.
I20260709 13:17:03.432754 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:03.433710 10799 raft_consensus.cc:2410] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:17:03.202595 13439 raft_consensus.cc:740] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:03.213970 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:03.414254 13629 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:03.434549 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:03.212885 13397 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:03.435535 13439 consensus_queue.cc:260] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:03.390848 13545 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:03.436415 13397 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:03.209380 13613 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:03.359480 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:03.217051 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:03.184244 13450 raft_consensus.cc:2749] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:03.345630 13328 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:03.367949 13385 raft_consensus.cc:2804] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:03.368542 13620 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:03.369302 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:03.371853 10795 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 6.
I20260709 13:17:03.443912 10948 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 5.
I20260709 13:17:03.199592 13600 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:03.245772 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:03.378243 10871 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 9.
I20260709 13:17:03.379733 10915 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:03.446297 10873 raft_consensus.cc:1240] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:03.341908 13429 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:03.344775 13599 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:03.375391 10916 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:03.384986 10841 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:03.362251 13350 raft_consensus.cc:740] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:03.224606 13580 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:03.447592 10797 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
I20260709 13:17:03.383756 13289 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:03.407351 12996 raft_consensus.cc:3055] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:03.374855 13614 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:03.420135 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 67394us
I20260709 13:17:03.339447 13515 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:03.421375 13424 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:03.421240 13446 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:03.471028 10697 catalog_manager.cc:5697] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 4 to 5, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 5 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:03.472652 13446 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:03.473431 13289 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:03.474515 10792 raft_consensus.cc:2393] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 5: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:03.464570 12996 raft_consensus.cc:740] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:03.475728 12996 consensus_queue.cc:260] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:03.426280 10794 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
I20260709 13:17:03.426285 13413 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:03.477448 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:03.478107 10795 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:03.229116 13612 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:03.435294 13629 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:03.220628 13578 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:03.478746 13612 raft_consensus.cc:2749] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:03.480314 10794 raft_consensus.cc:1240] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:03.480511 13629 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:03.482053 10794 raft_consensus.cc:1275] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:17:03.436818 13439 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:03.211556 13535 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:03.392014 13588 consensus_meta.cc:287] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.601s	user 0.012s	sys 0.000s
I20260709 13:17:03.382982 13380 raft_consensus.cc:2749] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:03.343947 13606 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:03.437568 10799 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 6.
I20260709 13:17:03.441190 10793 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
I20260709 13:17:03.401664 13608 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:03.372586 13443 consensus_queue.cc:260] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:03.394420 11508 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.011s	sys 0.023s
W20260709 13:17:03.564620 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:03.460724 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 81063.1us
W20260709 13:17:03.460546 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 75686.1us
I20260709 13:17:03.476929 12996 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:03.477495 13413 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:03.428260 10943 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
I20260709 13:17:03.462347 13577 raft_consensus.cc:2749] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:03.483323 10842 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:17:03.571671 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 88443.7us
I20260709 13:17:03.572500 10842 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:03.473963 10697 catalog_manager.cc:5697] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 2 to 5, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 5 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:03.483904 13424 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:03.494827 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:17:03.565567 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:03.577673 10798 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
I20260709 13:17:03.483414 10794 raft_consensus.cc:1275] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:17:03.466435 13613 raft_consensus.cc:2749] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:17:03.579146 10696 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42956 (request call id 258) took 11612 ms (client timeout 15000 ms). Trace:
I20260709 13:17:03.564291 10795 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7.
I20260709 13:17:03.574121 10697 catalog_manager.cc:5697] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 6 to 7, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 7 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:03.569139 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:03.568735 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:03.567193 10841 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:03.491112 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 128554us
W20260709 13:17:03.491025 13421 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.681s	user 0.014s	sys 0.030s
I20260709 13:17:03.576056 10915 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:03.459934 13599 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:03.584383 10796 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
I20260709 13:17:03.505924 13614 raft_consensus.cc:2804] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:03.579941 10842 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:03.610478 10915 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:03.496402 13289 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:03.604179 10916 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:03.616683 10801 raft_consensus.cc:1240] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:03.611886 10697 catalog_manager.cc:5697] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 5 to 6, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 6 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:03.573958 13545 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:03.582234 13380 raft_consensus.cc:2804] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:03.628793 10798 raft_consensus.cc:1275] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:17:03.576763 13578 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:03.625166 10916 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:17:03.600260 13421 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.790s	user 0.015s	sys 0.030s
I20260709 13:17:03.565333 13588 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:03.581218 13443 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:03.569509 13535 consensus_queue.cc:1048] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
I20260709 13:17:03.493927 13429 consensus_queue.cc:1059] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.119s
I20260709 13:17:03.572242 13493 consensus_queue.cc:1048] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:17:03.608851 13577 raft_consensus.cc:2749] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:03.580804 13424 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:03.667865 10800 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 8.
I20260709 13:17:03.636770 10916 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:03.620198 10915 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:03.663735 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:03.655445 13539 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:03.673563 10869 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 5.
I20260709 13:17:03.673826 13413 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:03.674908 13515 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:03.621933 13557 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:03.675554 13557 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:03.632484 13629 raft_consensus.cc:2804] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:03.675981 13557 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:03.664347 13606 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:03.676667 13629 raft_consensus.cc:697] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 8 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:03.677043 13485 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:03.677572 13485 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:03.677915 13629 consensus_queue.cc:237] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:03.628115 13613 raft_consensus.cc:2804] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:03.632346 13614 raft_consensus.cc:697] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 7 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:03.675710 13577 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:03.632227 13380 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:03.628932 13289 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:03.591220 10797 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
I20260709 13:17:03.651880 13331 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:03.625897 13590 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:03.633486 13578 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:03.665231 13612 raft_consensus.cc:3055] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:17:03.702359 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 65701.8us
I20260709 13:17:03.604573 13350 consensus_queue.cc:260] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:03.642699 13545 consensus_queue.cc:1048] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.060s
W20260709 13:17:03.727017 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 106911us
W20260709 13:17:03.739203 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:03.737219 10916 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:03.698316 13577 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:03.705034 13380 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:03.675566 13539 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:03.719928 13450 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:17:03.740988 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 364126us
I20260709 13:17:03.720651 13409 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:03.718410 13331 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:03.721942 13534 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.029s	sys 0.038s
I20260709 13:17:03.720837 13612 raft_consensus.cc:740] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:17:03.777504 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 380494us
I20260709 13:17:03.721823 13578 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:03.753300 13577 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:03.754573 13397 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:03.777068 13606 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:03.795073 13534 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.029s	sys 0.039s
I20260709 13:17:03.779070 13421 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:03.810235 10767 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:03.791810 13331 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:03.794461 13580 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:03.794188 13600 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:03.827592 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:03.579624 10696 rpcz_store.cc:269] 0709 13:16:51.758979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:51.759273 (+   294us) service_pool.cc:224] Handling call
0709 13:17:03.370677 (+11611404us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:16:59.211625 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:16:59.250335 (+ 38710us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:16:59.250373 (+    38us) write_op.cc:435] Acquired schema lock
0709 13:16:59.250385 (+    12us) tablet.cc:662] Decoding operations
0709 13:16:59.253322 (+  2937us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:16:59.253398 (+    76us) write_op.cc:641] Partition lock acquired for write op
0709 13:16:59.253443 (+    45us) tablet.cc:685] Acquiring locks for 15 operations
0709 13:16:59.312325 (+ 58882us) tablet.cc:701] Row locks acquired
0709 13:16:59.312347 (+    22us) write_op.cc:260] PREPARE: finished
0709 13:16:59.312507 (+   160us) write_op.cc:270] Start()
0709 13:16:59.312628 (+   121us) write_op.cc:276] Timestamp: P: 1783603019312461 usec, L: 0
0709 13:16:59.312651 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:16:59.313399 (+   748us) log.cc:844] Serialized 5997 byte log entry
0709 13:16:59.599832 (+286433us) spinlock_profiling.cc:243] Waited 3.82 ms on lock 0x7b18002a8440. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc83f3 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:16:59.674658 (+ 74826us) op_driver.cc:464] REPLICATION: finished
0709 13:16:59.775568 (+100910us) write_op.cc:301] APPLY: starting
0709 13:16:59.775652 (+    84us) tablet.cc:1367] starting BulkCheckPresence
0709 13:16:59.776020 (+   368us) tablet.cc:1370] finished BulkCheckPresence
0709 13:16:59.776044 (+    24us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:00.023336 (+247292us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:00.023690 (+   354us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=15
0709 13:17:00.023742 (+    52us) write_op.cc:312] APPLY: finished
0709 13:17:00.024131 (+   389us) log.cc:844] Serialized 107 byte log entry
0709 13:17:00.024315 (+   184us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:00.024479 (+   164us) write_op.cc:454] Released schema lock
0709 13:17:00.024536 (+    57us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:00.041434 (+ 16898us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:00.046525 (+  5091us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":19154,"reported_tablets":43,"spinlock_wait_cycles":53632,"child_traces":[["op",{"apply.queue_time_us":100663,"apply.run_cpu_time_us":5393,"apply.run_wall_time_us":249423,"num_ops":15,"prepare.queue_time_us":223603,"prepare.run_cpu_time_us":24781,"prepare.run_wall_time_us":271650,"raft.queue_time_us":40139,"raft.run_cpu_time_us":19635,"raft.run_wall_time_us":86837,"replication_time_us":361863,"spinlock_wait_cycles":8780032,"thread_start_us":375081,"threads_started":4,"wal-append.queue_time_us":152983,"wal-append.run_cpu_time_us":8496,"wal-append.run_wall_time_us":1650451}]]}
I20260709 13:17:03.833305 13274 raft_consensus.cc:2804] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:03.836162 11478 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:03.838460 13485 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:03.842161 11477 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.004s	sys 0.020s
I20260709 13:17:03.847405 11508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56410 (request call id 516) took 1077 ms. Trace:
W20260709 13:17:03.839028 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 253896us
W20260709 13:17:03.879918 11477 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.004s	sys 0.020s
I20260709 13:17:03.821437 13606 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:03.889312 10800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 1532) took 1092 ms. Trace:
I20260709 13:17:03.890920 10841 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:03.819671 13421 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:03.843576 13274 raft_consensus.cc:697] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 7 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:03.895359 10841 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:03.891675 13588 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:03.907756 11477 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:03.898041 10767 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:03.894174 10800 rpcz_store.cc:276] 0709 13:17:02.796893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:02.825220 (+ 28327us) service_pool.cc:224] Handling call
0709 13:17:03.889081 (+1063861us) spinlock_profiling.cc:243] Waited 461 ms on lock 0x7b5400294854. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:03.889235 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1059190144}
I20260709 13:17:03.904142 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:03.841542 13635 consensus_queue.cc:1059] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 13:17:03.882303 13413 raft_consensus.cc:2749] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:03.919458 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:03.927898 10943 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
I20260709 13:17:03.919920 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:03.932955 10941 raft_consensus.cc:2393] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 7: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:03.835223 13614 consensus_queue.cc:237] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:03.892055 13600 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:17:03.941335 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 364360us
I20260709 13:17:03.848598 13637 consensus_queue.cc:1059] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.077s
I20260709 13:17:03.906829 13539 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:03.911146 13493 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:03.898643 13034 consensus_queue.cc:1048] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
W20260709 13:17:03.945899 13630 log.cc:927] Time spent T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 13:17:03.931593 10697 catalog_manager.cc:5697] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:03.893657 13580 consensus_queue.cc:1048] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 13:17:03.916442 13534 consensus_queue.cc:1059] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:17:03.948539 13539 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:03.949793 13621 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:03.962105 13633 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:03.956104 10767 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:03.947633 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 226281us
I20260709 13:17:03.879972 11508 rpcz_store.cc:276] 0709 13:17:02.769924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:02.770056 (+   132us) service_pool.cc:224] Handling call
0709 13:17:02.770335 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:02.770530 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:17:03.564769 (+794239us) log.cc:844] Serialized 14 byte log entry
0709 13:17:03.847032 (+282263us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:03.847086 (+    54us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:03.847141 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:03.847221 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:03.847344 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":239314,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":128,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":28434,"thread_start_us":1075781,"threads_started":2}
I20260709 13:17:03.942013 13588 raft_consensus.cc:2749] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:17:03.958315 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:03.944362 13600 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:03.970561 13493 raft_consensus.cc:697] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:03.961416 13539 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:03.962157 13621 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:03.964479 13328 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:03.950340 13409 raft_consensus.cc:697] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 8 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:03.980844 13289 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:03.986722 10948 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
W20260709 13:17:04.020205 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 82470.7us
I20260709 13:17:04.019855 13493 consensus_queue.cc:237] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:04.021064 10767 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:03.987044 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 151573us
I20260709 13:17:03.943245 13613 raft_consensus.cc:697] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 6 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:03.958324 10842 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:03.923662 13634 consensus_queue.cc:1048] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
W20260709 13:17:03.981781 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 67347us
I20260709 13:17:04.029924 11477 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:03.962770 13633 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:03.899044 13612 consensus_queue.cc:260] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:04.029803 10916 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:03.999394 13600 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:03.929399 13274 consensus_queue.cc:237] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:04.041231 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:04.058054 10841 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:04.037595 11477 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:04.056998 10916 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:04.026412 13328 consensus_queue.cc:1048] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 13:17:04.057732 13646 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:04.064435 13654 log.cc:927] Time spent T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:17:04.022639 13534 raft_consensus.cc:3055] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:04.065412 13534 raft_consensus.cc:740] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:04.066375 13534 consensus_queue.cc:260] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:04.067001 13647 raft_consensus.cc:2749] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:17:04.067307 13650 log.cc:927] Time spent T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:17:04.067673 13534 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:04.068212 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 109996us
W20260709 13:17:04.010591 13439 consensus_meta.cc:287] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.515s	user 0.009s	sys 0.012s
I20260709 13:17:04.069000 10842 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:04.070060 13545 raft_consensus.cc:2749] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:04.070828 13545 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:04.071345 13545 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:04.071722 13545 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:04.075382 13413 consensus_queue.cc:1059] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:17:04.076289 13413 raft_consensus.cc:3055] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:04.076629 13413 raft_consensus.cc:740] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:04.077486 13413 consensus_queue.cc:260] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:04.079694 13429 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:04.080150 13429 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:04.080500 13429 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:17:04.081481 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 44003.3us
I20260709 13:17:04.081983 13534 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:04.082221 11477 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:04.082623 13534 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:04.084388 13421 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:04.021082 13621 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:04.087133 13421 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:04.087527 13421 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:04.005115 13450 raft_consensus.cc:697] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 5 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:04.092041 10916 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:04.085815 13600 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:04.088420 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 299118us
I20260709 13:17:04.069867 10697 catalog_manager.cc:5697] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 2 to 4, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 4 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:17:04.064594 13646 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:04.069888 10842 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:04.085538 13613 consensus_queue.cc:237] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:04.092530 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 70160.9us
I20260709 13:17:04.094619 10916 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
W20260709 13:17:04.099944 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:04.101235 10842 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:04.103938 10842 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:04.078670 13413 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:17:04.115134 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:04.097095 10697 catalog_manager.cc:5697] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
W20260709 13:17:04.158447 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 136208us
I20260709 13:17:04.036317 13633 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:04.051927 13608 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:04.107002 13328 raft_consensus.cc:2804] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:04.171540 13608 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:04.171957 13608 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:04.059391 13385 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:04.174338 13385 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:04.175268 13450 consensus_queue.cc:237] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:04.089701 13588 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:04.149333 13034 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:17:04.163846 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 59953.4us
W20260709 13:17:04.179054 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:04.179555 10766 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:04.179802 10842 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:04.092068 13642 log.cc:927] Time spent T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 13:17:04.180790 13350 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:04.181383 10766 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:04.165496 10798 raft_consensus.cc:1275] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 6 index: 2. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260709 13:17:04.094353 13421 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:04.111378 13557 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:04.092429 13534 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:04.093163 13629 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:04.183609 10792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1542) took 1383 ms. Trace:
I20260709 13:17:04.191663 10766 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:04.195995 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:04.198285 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:04.095429 13637 consensus_queue.cc:1243] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer 12682eb70bc84c76879e04a8628ee586 log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:17:04.198553 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:04.199072 10871 raft_consensus.cc:2393] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 7: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:04.052237 13405 consensus_queue.cc:1048] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.113s
I20260709 13:17:04.207325 10767 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:04.192566 10842 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:17:04.244752 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 52253.1us
I20260709 13:17:04.245443 10842 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:04.243973 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:04.246675 10798 raft_consensus.cc:1275] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:17:04.164976 13443 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:04.200531 10697 catalog_manager.cc:5697] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 1 to 2, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 2 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:17:04.254087 10799 raft_consensus.cc:1240] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:04.195225 10792 rpcz_store.cc:276] 0709 13:17:02.800126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:03.475983 (+675857us) service_pool.cc:224] Handling call
0709 13:17:03.476329 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:03.476580 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:17:03.809709 (+333129us) log.cc:844] Serialized 14 byte log entry
0709 13:17:03.995192 (+185483us) log.cc:844] Serialized 14 byte log entry
0709 13:17:04.084164 (+ 88972us) spinlock_profiling.cc:243] Waited 15.4 ms on lock 0x7b18003cca00. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc8522 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:17:04.183274 (+ 99110us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:04.183326 (+    52us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:04.183377 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:04.183450 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:04.183557 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37454,"prepare.queue_time_us":484101,"prepare.run_cpu_time_us":108,"prepare.run_wall_time_us":11806,"raft.queue_time_us":135168,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":4258,"spinlock_wait_cycles":71766016,"thread_start_us":629236,"threads_started":3,"wal-append.queue_time_us":187984}
I20260709 13:17:04.107568 13647 raft_consensus.cc:2749] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:04.256798 13647 raft_consensus.cc:2749] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:04.258188 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:04.259065 13600 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:04.094161 13653 consensus_queue.cc:1048] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.171s
W20260709 13:17:04.257542 13620 consensus_meta.cc:287] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.810s	user 0.009s	sys 0.008s
I20260709 13:17:04.264592 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:04.282570 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 180489us
I20260709 13:17:04.276175 10800 raft_consensus.cc:1240] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260709 13:17:04.282722 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:04.283277 11478 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:17:04.283286 10943 raft_consensus.cc:2410] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 8 because replica has last-logged OpId of term: 7 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 2.
I20260709 13:17:04.283862 10875 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 8.
I20260709 13:17:04.276482 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:04.284657 10796 raft_consensus.cc:2393] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 5: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:04.279716 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:17:04.281654 10916 connection.cc:256] RPC call timeout handler was delayed by -1.3813421280001421s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1535s for a response
W20260709 13:17:04.285392 10916 connection.cc:256] RPC call timeout handler was delayed by -1.2895421279999937s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1637s for a response
I20260709 13:17:04.287648 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:04.287798 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:04.289004 10766 connection.cc:256] RPC call timeout handler was delayed by -1.072625822999953s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1603s for a response
I20260709 13:17:04.190121 13397 raft_consensus.cc:2749] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:17:04.285815 10916 connection.cc:256] RPC call timeout handler was delayed by -1.240042128000141s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16920000000000002s for a response
W20260709 13:17:04.296734 10916 connection.cc:256] RPC call timeout handler was delayed by -1.1734421280000333s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1766s for a response
W20260709 13:17:04.297215 10916 connection.cc:256] RPC call timeout handler was delayed by -1.1662421280000217s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1774s for a response
W20260709 13:17:04.297614 10916 connection.cc:256] RPC call timeout handler was delayed by -1.1572421280000071s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1784s for a response
W20260709 13:17:04.298026 10916 connection.cc:256] RPC call timeout handler was delayed by -1.1527421279999999s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1789s for a response
W20260709 13:17:04.298413 10916 connection.cc:256] RPC call timeout handler was delayed by -1.0357421280000381s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19190000000000002s for a response
I20260709 13:17:04.199707 13289 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:04.190654 13034 raft_consensus.cc:697] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:04.190865 13328 raft_consensus.cc:697] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 5 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:04.311900 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:04.190580 13588 raft_consensus.cc:2764] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:17:04.189973 13535 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:04.200208 13629 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:04.248243 13612 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:04.302994 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:04.326195 10800 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 5.
I20260709 13:17:04.199978 13621 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:04.294584 10697 catalog_manager.cc:5697] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 1 to 4, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 4 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:17:04.251929 13515 raft_consensus.cc:2804] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:04.288763 13443 raft_consensus.cc:697] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:04.322433 10876 raft_consensus.cc:1240] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:04.291141 13620 raft_consensus.cc:2749] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:04.256698 13646 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:04.338119 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 54914.6us
I20260709 13:17:04.191164 13545 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:04.275839 13665 raft_consensus.cc:2764] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:17:04.295277 13600 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:04.316504 13653 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:04.263290 13608 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:04.339293 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:04.351330 10943 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 7.
I20260709 13:17:04.331080 13621 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:04.352383 13621 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:04.353772 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:04.311058 13606 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:04.340078 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:04.320184 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:04.307600 13289 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:04.351735 13639 consensus_queue.cc:1059] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.098s
W20260709 13:17:04.345214 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 152948us
I20260709 13:17:04.325587 13629 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:04.251488 13409 consensus_queue.cc:237] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:04.366834 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 279179us
I20260709 13:17:04.332551 13515 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:04.362155 10942 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 5.
I20260709 13:17:04.368525 10916 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:17:04.379518 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:04.379729 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:04.380626 10945 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 8.
I20260709 13:17:04.382349 11477 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:04.328572 13535 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:04.345763 13666 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:04.396045 10916 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:04.371268 13289 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:04.326653 13634 consensus_queue.cc:1048] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:17:04.406677 10767 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:04.405844 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 160449us
W20260709 13:17:04.409839 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:04.415103 10842 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:04.419292 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:04.403597 13034 consensus_queue.cc:237] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:04.420022 10945 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 5.
I20260709 13:17:04.422878 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:04.423310 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:04.424525 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:04.425241 10945 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:04.404227 13606 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:04.427263 13606 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:04.429275 13606 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:04.384824 13580 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:04.430681 13580 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:04.432883 13580 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:04.433045 10915 leader_election.cc:336] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Timed out: RequestConsensusVote RPC to 127.9.116.132:40555 timed out after 1.692s (SENT)
W20260709 13:17:04.434113 13599 consensus_meta.cc:287] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.777s	user 0.011s	sys 0.000s
I20260709 13:17:04.434337 13413 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:04.434547 13599 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:04.434705 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:04.381436 13515 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:04.440614 10766 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:04.443646 10766 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:04.444420 10766 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:04.445235 10766 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:17:04.437312 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:04.451548 10945 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 5.
I20260709 13:17:04.368640 13600 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:04.455466 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 59549.3us
I20260709 13:17:04.463577 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:04.463784 10801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 1947) took 1417 ms. Trace:
I20260709 13:17:04.398001 13405 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:04.464284 10916 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:17:04.466701 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 47971.3us
I20260709 13:17:04.464315 10801 rpcz_store.cc:276] 0709 13:17:03.046596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:03.627170 (+580574us) service_pool.cc:224] Handling call
0709 13:17:03.653808 (+ 26638us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:03.654099 (+   291us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:17:03.847008 (+192909us) log.cc:844] Serialized 14 byte log entry
0709 13:17:04.440217 (+593209us) log.cc:844] Serialized 14 byte log entry
0709 13:17:04.463426 (+ 23209us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:04.463477 (+    51us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:04.463524 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:04.463591 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:04.463724 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31564,"prepare.queue_time_us":288075,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":1983,"raft.queue_time_us":88162,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":97,"thread_start_us":766895,"threads_started":3,"wal-append.queue_time_us":500129}
I20260709 13:17:04.469415 10766 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:04.471015 10949 raft_consensus.cc:2393] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 5: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:04.430370 13385 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:04.446281 13331 consensus_meta.cc:287] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.611s	user 0.010s	sys 0.000s
I20260709 13:17:04.475144 10842 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
W20260709 13:17:04.478461 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:04.519018 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 166755us
I20260709 13:17:04.507337 10842 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:04.522049 10842 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:04.369726 13653 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:04.506649 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 106871us
I20260709 13:17:04.523638 10842 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:04.522328 13606 raft_consensus.cc:2804] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:04.453390 13614 raft_consensus.cc:2749] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:04.528142 10916 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:04.472481 13439 raft_consensus.cc:2804] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:04.468039 13580 raft_consensus.cc:2804] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:04.542814 10916 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:17:04.473310 13577 consensus_meta.cc:287] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.663s	user 0.007s	sys 0.002s
I20260709 13:17:04.463572 13664 raft_consensus.cc:2804] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:04.472824 13663 raft_consensus.cc:2804] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:04.525916 13671 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:04.403439 13674 consensus_queue.cc:1048] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.130s
I20260709 13:17:04.552035 10942 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 8.
I20260709 13:17:04.553205 10916 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:04.446414 13413 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:04.535408 13380 consensus_meta.cc:287] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.745s	user 0.008s	sys 0.002s
I20260709 13:17:04.526726 13405 raft_consensus.cc:697] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 7 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:04.524524 13331 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:04.463909 13328 consensus_queue.cc:237] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:04.439656 13646 consensus_queue.cc:1048] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.114s
I20260709 13:17:04.529606 13493 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:04.452653 13620 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:04.452486 13665 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:17:04.562182 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:04.577970 10870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 339) took 1384 ms. Trace:
I20260709 13:17:04.575228 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:04.576624 10794 raft_consensus.cc:1240] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:04.575989 10799 raft_consensus.cc:1240] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 LEADER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
W20260709 13:17:04.570504 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:04.534983 13606 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:04.469755 13666 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:04.542570 13034 raft_consensus.cc:2804] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:04.584144 10916 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:04.557792 13485 raft_consensus.cc:2804] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:04.546096 13668 raft_consensus.cc:2804] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:04.545661 13557 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:04.597566 10870 rpcz_store.cc:276] 0709 13:17:03.194041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:03.339641 (+145600us) service_pool.cc:224] Handling call
0709 13:17:03.362481 (+ 22840us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:03.436978 (+ 74497us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:17:03.786690 (+349712us) log.cc:844] Serialized 14 byte log entry
0709 13:17:03.923084 (+136394us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:03.923154 (+    70us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:03.923225 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:04.436109 (+512884us) spinlock_profiling.cc:243] Waited 7.25 ms on lock 0x7b1800364a80. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7dd962a6 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419
0709 13:17:04.577725 (+141616us) spinlock_profiling.cc:243] Waited 74.3 ms on lock 0x7b5400077954. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:04.577769 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:04.577909 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2880,"prepare.queue_time_us":383490,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":44,"raft.queue_time_us":102,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":10323,"spinlock_wait_cycles":187488384,"thread_start_us":485167,"threads_started":2,"wal-append.queue_time_us":157626}
I20260709 13:17:04.596582 13620 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:04.551034 13274 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:04.615374 10795 raft_consensus.cc:1240] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 13:17:04.571604 13539 consensus_meta.cc:287] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.549s	user 0.008s	sys 0.000s
I20260709 13:17:04.547008 13580 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:04.565673 13659 log.cc:927] Time spent T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.182s	user 0.000s	sys 0.000s
I20260709 13:17:04.552287 13577 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:04.565074 13670 log.cc:927] Time spent T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 13:17:04.551386 13663 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:04.546460 13439 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:04.526494 13678 raft_consensus.cc:2804] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:04.550311 13676 consensus_queue.cc:1048] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.100s
I20260709 13:17:04.568194 13600 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:04.627351 10916 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:04.548785 13664 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:04.637717 10793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 1937) took 1594 ms. Trace:
I20260709 13:17:04.638954 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:04.627283 10943 raft_consensus.cc:2368] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 8. Current term is 9.
I20260709 13:17:04.567118 13608 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:04.569408 13385 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:04.600278 10796 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 5.
W20260709 13:17:04.646124 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 105197us
W20260709 13:17:04.646474 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 265383us
I20260709 13:17:04.619660 13606 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:04.527302 13443 consensus_queue.cc:237] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:04.653889 10794 raft_consensus.cc:1240] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:04.632453 13034 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:04.614333 13666 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:04.673801 13664 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:04.573026 12996 consensus_meta.cc:287] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.998s	user 0.013s	sys 0.004s
I20260709 13:17:04.609103 13665 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:04.643724 13485 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:04.645845 13635 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.697s	user 0.012s	sys 0.008s
I20260709 13:17:04.577073 13599 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:04.554018 13647 consensus_queue.cc:1048] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.089s
I20260709 13:17:04.654902 13580 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:04.596012 13380 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:04.715240 13666 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:17:04.734143 13350 consensus_meta.cc:287] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.535s	user 0.007s	sys 0.000s
I20260709 13:17:04.734804 13535 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:04.668565 10916 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:04.735883 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 67382us
I20260709 13:17:04.737821 12996 raft_consensus.cc:2749] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:04.650625 13620 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:04.671777 10842 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:04.649928 13274 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:04.669404 10793 rpcz_store.cc:276] 0709 13:17:03.043227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:03.581795 (+538568us) service_pool.cc:224] Handling call
0709 13:17:03.613697 (+ 31902us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:03.614155 (+   458us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:03.614206 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:03.626599 (+ 12393us) log.cc:844] Serialized 26 byte log entry
0709 13:17:03.844215 (+217616us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:04.073000 (+228785us) spinlock_profiling.cc:243] Waited 13.4 ms on lock 0x7b18003335e0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc83f3 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:17:04.632955 (+559955us) spinlock_profiling.cc:243] Waited 14 ms on lock 0x7b18003335e0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c80a5d4bc 00007f9c7ddab48b 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc5455 00007f9c848ad3c7 00007f9c80ebf9a7 00007f9c80ecf0bf
0709 13:17:04.636914 (+  3959us) log.cc:844] Serialized 14 byte log entry
0709 13:17:04.637253 (+   339us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:04.637375 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:04.637437 (+    62us) raft_consensus.cc:1685] finished
0709 13:17:04.637455 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:04.637637 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":110579,"prepare.run_wall_time_us":4093,"raft.queue_time_us":11677,"raft.run_cpu_time_us":355,"raft.run_wall_time_us":18983,"spinlock_wait_cycles":71496960,"thread_start_us":256282,"threads_started":2,"wal-append.queue_time_us":316974}
I20260709 13:17:04.678220  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:04.619067 13614 consensus_queue.cc:1048] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.043s
I20260709 13:17:04.672904 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:04.751974 10795 raft_consensus.cc:2393] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 5: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:04.655347 13545 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:04.664815 13439 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:04.700239 10796 raft_consensus.cc:1240] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:04.663684 13678 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:04.760931 13678 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:04.753314 13274 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:04.673667 13557 raft_consensus.cc:2804] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:17:04.655676 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 134930us
I20260709 13:17:04.657374 13539 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:04.713310 13653 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:04.713759 13657 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:04.714185 13651 raft_consensus.cc:2804] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:04.714548 13677 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:17:04.721095 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:04.728827 13590 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:04.799634 13655 consensus_queue.cc:1048] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
W20260709 13:17:04.728976 13424 consensus_meta.cc:287] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 1.011s	user 0.007s	sys 0.004s
I20260709 13:17:04.699020 13646 consensus_queue.cc:1048] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 13:17:04.716020 13665 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:04.732860 13413 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:04.734978 10946 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 6.
W20260709 13:17:04.713757 13429 consensus_meta.cc:287] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.633s	user 0.009s	sys 0.000s
I20260709 13:17:04.702554 13668 raft_consensus.cc:697] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:04.710515 13034 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:17:04.750494 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:04.717655 13635 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.769s	user 0.012s	sys 0.008s
W20260709 13:17:04.754034 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 82353.2us
I20260709 13:17:04.716116 13485 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:04.661551 13663 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:04.765729 11478 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:04.716979 13493 raft_consensus.cc:697] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:04.766536 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:04.734804 13380 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:04.781404 13539 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:04.773675 13557 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:04.783201 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:17:04.846194 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 80572.5us
I20260709 13:17:04.846668 10801 raft_consensus.cc:1240] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:04.788375 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:17:04.788530 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 50420.1us
W20260709 13:17:04.789021 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:04.757695 12996 consensus_queue.cc:1048] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 13:17:04.790371 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:04.850528 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:04.850839 10949 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 8.
I20260709 13:17:04.851158 11504 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:04.792817 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:04.792608 13629 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:04.793695 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:04.761608 13535 raft_consensus.cc:2804] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:04.793937 10944 raft_consensus.cc:1240] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:17:04.809294 13677 raft_consensus.cc:697] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 5 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:04.794544 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:04.714208 13405 consensus_queue.cc:237] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:04.792822 13289 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:04.795001 13331 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:04.804018 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:17:04.800904 13534 consensus_meta.cc:287] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.549s	user 0.009s	sys 0.002s
I20260709 13:17:04.819455 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:04.783205 13657 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:04.817554 13621 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:04.823879 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:04.783025 13651 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:04.818369 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:04.831652 10797 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 5.
I20260709 13:17:04.833948 10794 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 5.
I20260709 13:17:04.845820 10799 raft_consensus.cc:1275] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:17:04.846737 11512 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7.
I20260709 13:17:04.816092 13577 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:04.809872 13424 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:04.864710 10947 raft_consensus.cc:2393] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 9: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:04.816830 13429 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:05.005151 10875 raft_consensus.cc:1240] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 LEADER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:04.866164 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:04.901926 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:05.015764 10766 leader_election.cc:400] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 election: Vote denied by peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553) with higher term. Message: Invalid argument: T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 8. Current term is 9.
W20260709 13:17:05.008955 11478 connection.cc:256] RPC call timeout handler was delayed by -1.5562924179998845s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15800000000000003s for a response
I20260709 13:17:04.923781 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:05.016328 10766 leader_election.cc:403] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:17:04.876224 11504 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 7.
I20260709 13:17:05.016912 10798 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 7.
I20260709 13:17:04.983057 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:04.913801 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:05.013597 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:04.937798 10948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41020 (request call id 1945) took 1034 ms. Trace:
I20260709 13:17:05.026826 10948 rpcz_store.cc:276] 0709 13:17:03.903652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:03.903833 (+   181us) service_pool.cc:224] Handling call
0709 13:17:04.937607 (+1033774us) spinlock_profiling.cc:243] Waited 76.1 ms on lock 0x7b54001e2b54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:04.937741 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":174955264}
W20260709 13:17:05.027961 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:05.028759 10796 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 5.
I20260709 13:17:05.024497 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:05.026127 10873 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:04.943882 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:04.946293 10941 raft_consensus.cc:1240] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 6. Current term is 7. Ops: []
W20260709 13:17:04.922350 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:05.036705 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:04.903476 13689 log.cc:927] Time spent T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:17:05.015913 10792 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 9.
W20260709 13:17:05.041218 13633 consensus_meta.cc:287] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.841s	user 0.007s	sys 0.002s
I20260709 13:17:05.041814 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:05.043511 10800 raft_consensus.cc:2393] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 5: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:05.018971 10766 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:04.939245 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:05.041716 13633 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:05.032454 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:05.046329 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:05.047479 10915 leader_election.cc:336] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Timed out: RequestConsensusVote RPC to 127.9.116.132:40555 timed out after 1.774s (SENT)
W20260709 13:17:04.987238 13680 log.cc:927] Time spent T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.279s	user 0.002s	sys 0.000s
I20260709 13:17:05.030859 10944 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
I20260709 13:17:04.938059 10869 raft_consensus.cc:1240] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 6. Current term is 8. Ops: []
I20260709 13:17:05.045492 10801 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 8.
W20260709 13:17:05.046101 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 147560us
I20260709 13:17:05.069052 10916 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:05.069273 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:05.029667 10943 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 6.
I20260709 13:17:05.047901 10796 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:04.999944 13380 raft_consensus.cc:2955] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:17:04.937294 13289 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:04.865999 13413 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:05.070170 10876 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 8.
W20260709 13:17:04.985378 13681 log.cc:927] Time spent T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.417s	user 0.003s	sys 0.000s
I20260709 13:17:05.071238 10793 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 5.
I20260709 13:17:05.076951 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:05.107004 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:05.048074 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:05.108682 10875 raft_consensus.cc:1240] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 7. Current term is 8. Ops: [7.3-7.3]
I20260709 13:17:04.849805 13557 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:05.074424 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:05.025769 13612 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:05.111945 10950 raft_consensus.cc:1240] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 5 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 4. Current term is 5. Ops: []
W20260709 13:17:04.899504 13421 consensus_meta.cc:287] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.659s	user 0.010s	sys 0.000s
I20260709 13:17:04.866797 13535 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:04.904338 13515 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:05.047257 13350 consensus_queue.cc:1059] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.188s
I20260709 13:17:04.886570 10870 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:17:05.014078 13493 consensus_queue.cc:237] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:05.953799 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 906286us
W20260709 13:17:05.954710 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:05.957474 13710 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:05.957815 13450 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:05.958546 10872 raft_consensus.cc:2368] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 5. Current term is 6.
W20260709 13:17:05.064711 13578 consensus_meta.cc:287] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 1.266s	user 0.008s	sys 0.004s
W20260709 13:17:05.950999 13034 consensus_meta.cc:287] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 1.118s	user 0.011s	sys 0.000s
I20260709 13:17:05.959653 10946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41020 (request call id 1966) took 1054 ms. Trace:
I20260709 13:17:05.959141 13578 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:05.958314 13450 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:05.896051 13696 log.cc:927] Time spent T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 13:17:05.961412 11506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56410 (request call id 525) took 1178 ms. Trace:
I20260709 13:17:05.961921 13450 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:05.962101 13578 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:05.959465 13034 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:05.958221 13710 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:05.581521 10842 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.758s	user 0.023s	sys 0.017s
I20260709 13:17:05.961792 11506 rpcz_store.cc:276] 0709 13:17:04.782925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:04.785986 (+  3061us) service_pool.cc:224] Handling call
0709 13:17:04.799201 (+ 13215us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:04.812205 (+ 13004us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:04.812241 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:04.812833 (+   592us) log.cc:844] Serialized 26 byte log entry
0709 13:17:04.857409 (+ 44576us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:04.857473 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:04.857580 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:05.961215 (+1103635us) raft_consensus.cc:1685] finished
0709 13:17:05.961234 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:05.961349 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25417,"raft.queue_time_us":18480,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":7847,"thread_start_us":44009,"threads_started":1,"wal-append.queue_time_us":896850}
I20260709 13:17:05.017683 13651 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:05.299189 13666 consensus_meta.cc:287] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.565s	user 0.006s	sys 0.004s
I20260709 13:17:06.006949 13666 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:17:06.009727 13666 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:06.012856 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:06.013556 10875 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:05.552670 13485 consensus_meta.cc:287] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.691s	user 0.008s	sys 0.001s
I20260709 13:17:05.965080 11511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56404 (request call id 546) took 1117 ms. Trace:
W20260709 13:17:05.836666 13606 consensus_meta.cc:287] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 1.134s	user 0.009s	sys 0.000s
I20260709 13:17:05.965734 13535 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:05.619447 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:05.729784 13274 consensus_meta.cc:287] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.963s	user 0.003s	sys 0.006s
I20260709 13:17:05.412420 13621 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:05.025239 13657 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:06.095563 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:17:05.110677 13671 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.000s	sys 0.014s
W20260709 13:17:06.098265 13671 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.532s	user 0.005s	sys 0.058s
I20260709 13:17:06.098634 13671 raft_consensus.cc:697] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 5 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
W20260709 13:17:05.763389 13439 consensus_meta.cc:287] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.976s	user 0.007s	sys 0.003s
I20260709 13:17:06.099618 13439 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:06.099658 13671 consensus_queue.cc:237] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:06.101701 10945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 369) took 1311 ms. Trace:
W20260709 13:17:05.718508 13678 consensus_meta.cc:287] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.955s	user 0.010s	sys 0.000s
W20260709 13:17:05.229096 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 212323us
W20260709 13:17:05.737243 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 289389us
I20260709 13:17:06.108273 10841 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:05.971849 10870 raft_consensus.cc:606] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Received request to transfer leadership to TS e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:06.109544 10870 raft_consensus.cc:614] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:06.110173 10870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:53748 (request call id 24) took 1224 ms. Trace:
I20260709 13:17:06.110545 10870 rpcz_store.cc:276] 0709 13:17:04.886162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:04.886379 (+   217us) service_pool.cc:224] Handling call
0709 13:17:06.110077 (+1223698us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:05.179447 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:06.111810 10948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41020 (request call id 1968) took 1206 ms. Trace:
I20260709 13:17:06.112216 10948 rpcz_store.cc:276] 0709 13:17:04.908202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:05.106032 (+197830us) service_pool.cc:224] Handling call
0709 13:17:06.111735 (+1005703us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:05.717346 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:17:05.170648 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:05.880167 10796 consensus_meta.cc:287] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.744s	user 0.010s	sys 0.000s
I20260709 13:17:06.114744 10796 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 6.
I20260709 13:17:06.115407 10796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39442 (request call id 298) took 1086 ms. Trace:
I20260709 13:17:05.064323 13588 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:05.960125 10946 rpcz_store.cc:276] 0709 13:17:04.904777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:05.016605 (+111828us) service_pool.cc:224] Handling call
0709 13:17:05.959588 (+942983us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:05.976231 10842 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.152s	user 0.023s	sys 0.040s
I20260709 13:17:06.116748 10799 raft_consensus.cc:2410] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:17:05.179772 10800 raft_consensus.cc:1275] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 13:17:06.116773 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 1.29294e+06us
I20260709 13:17:06.117489 10800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1557) took 1043 ms. Trace:
I20260709 13:17:06.117816 10800 rpcz_store.cc:276] 0709 13:17:05.074314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:05.074600 (+   286us) service_pool.cc:224] Handling call
0709 13:17:05.127470 (+ 52870us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:06.117252 (+989782us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:06.117439 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:06.095444 13485 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:05.982000 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:06.122812 11508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56404 (request call id 547) took 1274 ms. Trace:
I20260709 13:17:06.123200 11508 rpcz_store.cc:276] 0709 13:17:04.848245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:05.679242 (+830997us) service_pool.cc:224] Handling call
0709 13:17:06.122720 (+443478us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:05.982159 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:06.115779 10796 rpcz_store.cc:276] 0709 13:17:05.029389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:05.046073 (+ 16684us) service_pool.cc:224] Handling call
0709 13:17:06.115335 (+1069262us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:06.102236 10945 rpcz_store.cc:276] 0709 13:17:04.790575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:04.801032 (+ 10457us) service_pool.cc:224] Handling call
0709 13:17:04.811865 (+ 10833us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:04.812376 (+   511us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:04.812418 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:04.813018 (+   600us) log.cc:844] Serialized 26 byte log entry
0709 13:17:05.953375 (+1140357us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:05.953417 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:05.953485 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:06.101373 (+147888us) raft_consensus.cc:1685] finished
0709 13:17:06.101405 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:06.101600 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1069,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":10784,"thread_start_us":295159,"threads_started":1,"wal-append.queue_time_us":1284558}
I20260709 13:17:05.982000 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:06.126029 10801 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:05.656978 10868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1560) took 1374 ms. Trace:
I20260709 13:17:06.126609 10868 rpcz_store.cc:276] 0709 13:17:04.282332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:04.282449 (+   117us) service_pool.cc:224] Handling call
0709 13:17:04.315988 (+ 33539us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:04.334323 (+ 18335us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:04.334371 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:04.345115 (+ 10744us) log.cc:844] Serialized 26 byte log entry
0709 13:17:04.550590 (+205475us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:04.550637 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:04.550710 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:04.986479 (+435769us) spinlock_profiling.cc:243] Waited 21 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:17:05.614336 (+627857us) raft_consensus.cc:1685] finished
0709 13:17:05.614367 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:05.656856 (+ 42489us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":714,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":17887,"spinlock_wait_cycles":48287616,"thread_start_us":170040,"threads_started":1,"wal-append.queue_time_us":290212}
W20260709 13:17:05.923650 13665 consensus_meta.cc:287] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 1.094s	user 0.009s	sys 0.003s
W20260709 13:17:05.874208 13620 consensus_meta.cc:287] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 1.113s	user 0.003s	sys 0.009s
I20260709 13:17:05.132229 13413 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:05.750846 13580 consensus_meta.cc:287] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 1.015s	user 0.010s	sys 0.001s
I20260709 13:17:06.000481 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:06.001542 13710 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:06.001796 13350 raft_consensus.cc:3055] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:06.002496 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:06.158280 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:05.975546 13421 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:06.004125 13515 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:06.006057 13034 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:05.856117 10873 consensus_meta.cc:287] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.808s	user 0.009s	sys 0.000s
I20260709 13:17:06.168895 10873 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 6.
I20260709 13:17:06.169528 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49290 (request call id 341) took 1165 ms. Trace:
I20260709 13:17:06.169972 10873 rpcz_store.cc:276] 0709 13:17:05.004108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:05.004248 (+   140us) service_pool.cc:224] Handling call
0709 13:17:06.169441 (+1165193us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:05.997399 10916 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:17:06.171258 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 173968us
I20260709 13:17:06.173167 10916 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:06.004036 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:06.175153 10944 raft_consensus.cc:2393] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 9: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:06.057930 10875 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 7.
I20260709 13:17:06.177053 10875 raft_consensus.cc:1275] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:17:05.105821 13614 raft_consensus.cc:2804] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:05.890306 13633 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:06.059243 10867 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.012s	sys 0.033s
I20260709 13:17:06.057796 11511 rpcz_store.cc:276] 0709 13:17:04.847972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:05.048021 (+200049us) service_pool.cc:224] Handling call
0709 13:17:05.965014 (+916993us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:06.058513 13578 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:05.511700 13539 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:05.407313 13638 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:05.682343 13590 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:06.059458 13634 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.086s	user 0.023s	sys 0.029s
W20260709 13:17:05.994994 13663 consensus_meta.cc:287] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 1.133s	user 0.006s	sys 0.006s
I20260709 13:17:06.090988 13606 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:05.407838 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:05.406060 13424 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:05.064039 13534 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:06.139775 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:06.232110 10796 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 6.
W20260709 13:17:06.147497 10766 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.003s	sys 0.030s
I20260709 13:17:06.147986 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:06.232746 10796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39442 (request call id 299) took 1126 ms. Trace:
I20260709 13:17:06.233016 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1549) took 2035 ms. Trace:
I20260709 13:17:06.233174 10945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41020 (request call id 1971) took 1215 ms. Trace:
I20260709 13:17:06.233124 10796 rpcz_store.cc:276] 0709 13:17:05.106340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:06.139440 (+1033100us) service_pool.cc:224] Handling call
0709 13:17:06.232684 (+ 93244us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:06.233737 10945 rpcz_store.cc:276] 0709 13:17:05.018035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:06.125275 (+1107240us) service_pool.cc:224] Handling call
0709 13:17:06.233118 (+107843us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:05.237025 13664 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:06.151188 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:06.234256 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:17:06.232760 10766 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.691s	user 0.003s	sys 0.030s
I20260709 13:17:04.905167 13668 consensus_queue.cc:237] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:06.234863 10948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41020 (request call id 1954) took 1368 ms. Trace:
I20260709 13:17:06.235208 10948 rpcz_store.cc:276] 0709 13:17:04.866096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:06.135535 (+1269439us) service_pool.cc:224] Handling call
0709 13:17:06.234777 (+ 99242us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:06.152236 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:06.236356 10868 raft_consensus.cc:2393] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 9: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:06.153611 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:06.237398 10946 raft_consensus.cc:2393] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 7: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:06.159101 11508 raft_consensus.cc:2393] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 6: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:06.160296 10870 raft_consensus.cc:1275] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:17:06.182067 10867 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.779s	user 0.014s	sys 0.039s
I20260709 13:17:06.194331 13666 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.139110 13588 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:06.150169 13413 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:06.150949 13350 raft_consensus.cc:740] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:17:06.254582 10872 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 1555) took 2010 ms (client timeout 1884 ms). Trace:
I20260709 13:17:06.255939 10870 raft_consensus.cc:1275] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:17:06.255973 10801 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 9.
W20260709 13:17:06.193380 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 235704us
I20260709 13:17:06.257836 11505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56410 (request call id 523) took 1493 ms. Trace:
I20260709 13:17:06.258170 10801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 1556) took 1185 ms. Trace:
I20260709 13:17:06.137483 13678 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:05.419158 13677 consensus_queue.cc:237] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:06.144979 13665 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:06.222419 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:06.135785 13274 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:17:06.226333 11506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56404 (request call id 549) took 1377 ms. Trace:
I20260709 13:17:06.179819 13700 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.181339 13621 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.181962 13614 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:06.150447 13620 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:06.186060 13450 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:06.232162 10950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 1567) took 1978 ms. Trace:
I20260709 13:17:06.185503 10944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41020 (request call id 1969) took 1211 ms. Trace:
I20260709 13:17:06.164893 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:06.243311 10796 raft_consensus.cc:2393] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 5: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:06.280299 10944 rpcz_store.cc:276] 0709 13:17:04.973838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:06.003605 (+1029767us) service_pool.cc:224] Handling call
0709 13:17:06.185423 (+181818us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:06.280519 10800 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 6.
I20260709 13:17:06.236658 10766 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:06.247081 11508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56410 (request call id 531) took 1011 ms. Trace:
I20260709 13:17:06.162957 13580 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:06.194191 13655 raft_consensus.cc:2749] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:17:06.222741 13634 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.250s	user 0.023s	sys 0.032s
I20260709 13:17:06.233431 10869 rpcz_store.cc:276] 0709 13:17:04.197113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:04.197250 (+   137us) service_pool.cc:224] Handling call
0709 13:17:04.265434 (+ 68184us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:05.066283 (+800849us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:06.232805 (+1166522us) spinlock_profiling.cc:243] Waited 672 ms on lock 0x7b540006bdd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:06.232959 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1546417920}
I20260709 13:17:06.216946 13515 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.258868 10801 rpcz_store.cc:276] 0709 13:17:05.073289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:05.073630 (+   341us) service_pool.cc:224] Handling call
0709 13:17:06.258111 (+1184481us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":58752}
I20260709 13:17:06.291836 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:06.292502 10801 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 6.
I20260709 13:17:06.223840 13034 raft_consensus.cc:2804] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:06.260788 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:06.228123 13663 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:06.225816 13631 raft_consensus.cc:3055] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:17:06.261816 10915 leader_election.cc:336] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Timed out: RequestConsensusVote RPC to 127.9.116.132:40555 timed out after 1.637s (SENT)
I20260709 13:17:06.225333 13446 raft_consensus.cc:3055] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:06.264351 10946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41020 (request call id 1963) took 1360 ms. Trace:
I20260709 13:17:06.267397 11511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56410 (request call id 529) took 1369 ms. Trace:
I20260709 13:17:06.195484 13578 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:06.253911 13666 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:06.164161 13710 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
W20260709 13:17:06.269150 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 160982us
I20260709 13:17:06.279839 13409 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.270077 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:06.275228 11507 raft_consensus.cc:2368] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 7. Current term is 8.
I20260709 13:17:06.247529 13639 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.305305 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:06.315706 10944 raft_consensus.cc:2393] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 7: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:06.316223 10944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41020 (request call id 1957) took 1413 ms. Trace:
I20260709 13:17:06.316751 10944 rpcz_store.cc:276] 0709 13:17:04.902443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:06.292044 (+1389601us) service_pool.cc:224] Handling call
0709 13:17:06.316152 (+ 24108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:06.278007 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:06.217638 13539 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:06.284155 13424 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.319746 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:06.279385 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:06.280182 10950 rpcz_store.cc:276] 0709 13:17:04.254080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:04.254434 (+   354us) service_pool.cc:224] Handling call
0709 13:17:04.303552 (+ 49118us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:05.969341 (+1665789us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:06.231918 (+262577us) spinlock_profiling.cc:243] Waited 805 ms on lock 0x7b540004e054. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:06.232100 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1851669888}
I20260709 13:17:06.282097 11508 rpcz_store.cc:276] 0709 13:17:05.235446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:06.139320 (+903874us) service_pool.cc:224] Handling call
0709 13:17:06.247012 (+107692us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:06.252663 13588 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:06.258461 11505 rpcz_store.cc:276] 0709 13:17:04.764298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:04.764415 (+   117us) service_pool.cc:224] Handling call
0709 13:17:04.771186 (+  6771us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:04.771413 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:17:04.852667 (+ 81254us) log.cc:844] Serialized 14 byte log entry
0709 13:17:05.046129 (+193462us) log.cc:844] Serialized 14 byte log entry
0709 13:17:06.257211 (+1211082us) spinlock_profiling.cc:243] Waited 17.9 ms on lock 0x7b180004ba60. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7dd42c0a 00007f9c87482219 00007f9c7ddcaa40 00007f9c7ddc54e4 00007f9c7ddc5455 00007f9c80ead060 00007f9c80ebf716 00007f9c80ed0287 00007f9c80ed0208
0709 13:17:06.257395 (+   184us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:06.257453 (+    58us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:17:06.257525 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:06.257610 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:06.257751 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1637,"prepare.queue_time_us":130696,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":71,"raft.queue_time_us":1972,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":9030,"spinlock_wait_cycles":41054976,"thread_start_us":273436,"threads_started":2,"wal-append.queue_time_us":534545}
I20260709 13:17:06.283629 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:06.323032 10868 raft_consensus.cc:2368] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 5. Current term is 6.
I20260709 13:17:06.268203 13653 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.324048 13653 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:06.324458 13709 log.cc:927] Time spent T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 13:17:06.325546 13706 log.cc:927] Time spent T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:17:06.319942 13328 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.327363 13328 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:06.326800 10871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 348) took 1292 ms. Trace:
W20260709 13:17:06.329504 13702 log.cc:927] Time spent T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:17:06.329650 13424 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:06.332477 10941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 1576) took 1910 ms. Trace:
I20260709 13:17:06.332892 10941 rpcz_store.cc:276] 0709 13:17:04.422126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:04.422245 (+   119us) service_pool.cc:224] Handling call
0709 13:17:04.422583 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:05.045878 (+623295us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:06.332234 (+1286356us) spinlock_profiling.cc:243] Waited 523 ms on lock 0x7b54002a58d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:06.332416 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1203822848}
I20260709 13:17:06.334023 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:06.266424 13380 consensus_meta.cc:287] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.858s	user 0.009s	sys 0.003s
I20260709 13:17:06.334566 13639 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:06.285861 10874 raft_consensus.cc:1240] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:06.285000 13450 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:06.288220 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:06.287184 13350 consensus_queue.cc:260] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:06.271232 13331 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.229559 13638 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:06.289891 13535 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:06.302011 13446 raft_consensus.cc:740] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:06.294556 10870 raft_consensus.cc:2368] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 8. Current term is 9.
I20260709 13:17:06.294962 10792 raft_consensus.cc:2393] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 7: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
W20260709 13:17:06.255023 10872 rpcz_store.cc:269] 0709 13:17:04.244226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:04.244389 (+   163us) service_pool.cc:224] Handling call
0709 13:17:06.254366 (+2009977us) spinlock_profiling.cc:243] Waited 1.67 s on lock 0x7b540006c7d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:06.254515 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3840970624}
I20260709 13:17:06.291198 13439 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:06.300189 13515 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:06.275324 12996 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.273784 13651 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:06.280331 13614 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:06.294157 11503 raft_consensus.cc:2410] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:17:06.302160 10946 rpcz_store.cc:276] 0709 13:17:04.903963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:06.138092 (+1234129us) service_pool.cc:224] Handling call
0709 13:17:06.264264 (+126172us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:06.269446 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:06.302416 11511 rpcz_store.cc:276] 0709 13:17:04.897756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:06.191713 (+1293957us) service_pool.cc:224] Handling call
0709 13:17:06.267339 (+ 75626us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:06.303599 10876 raft_consensus.cc:1240] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
W20260709 13:17:06.306998 10799 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 1559) took 1230 ms (client timeout 1524 ms). Trace:
I20260709 13:17:06.307178 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:06.294057 13034 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:06.319654 13710 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:06.322873 10869 raft_consensus.cc:2393] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 9: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:06.334760 10941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41020 (request call id 1948) took 1543 ms. Trace:
I20260709 13:17:06.329104 10871 rpcz_store.cc:276] 0709 13:17:05.034328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:05.034465 (+   137us) service_pool.cc:224] Handling call
0709 13:17:05.034982 (+   517us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:05.035411 (+   429us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:05.035452 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:05.036034 (+   582us) log.cc:844] Serialized 26 byte log entry
0709 13:17:05.234560 (+198526us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:05.234608 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:05.234704 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:06.326563 (+1091859us) raft_consensus.cc:1685] finished
0709 13:17:06.326585 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:06.326719 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3647,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":78,"spinlock_wait_cycles":17152,"thread_start_us":198133,"threads_started":1,"wal-append.queue_time_us":1191040}
I20260709 13:17:06.335732 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 346) took 1302 ms. Trace:
I20260709 13:17:06.336431 11477 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:06.336314 10800 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:06.340830 10875 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:06.340977 11510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56410 (request call id 521) took 1602 ms. Trace:
I20260709 13:17:06.272918 11506 rpcz_store.cc:276] 0709 13:17:04.848866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:06.002206 (+1153340us) service_pool.cc:224] Handling call
0709 13:17:06.223490 (+221284us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:06.341679 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:06.342180 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:06.343890 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:06.298532 13631 raft_consensus.cc:740] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:06.225731 13590 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:06.291437 13621 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:06.313750 13695 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:06.346104 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:06.312707 13405 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:06.284533 13700 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:06.439471 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 202869us
I20260709 13:17:06.348344 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:17:06.633790 10766 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.116.132:40555 timed out after 1.568s (SENT) [suppressed 17 similar messages]
I20260709 13:17:06.634555 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:06.635897 10870 raft_consensus.cc:2393] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 6: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:17:06.496603 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:06.637074 10950 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:06.339430 13450 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:06.638651 10867 rpcz_store.cc:276] 0709 13:17:05.033678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:05.033786 (+   108us) service_pool.cc:224] Handling call
0709 13:17:05.068767 (+ 34981us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:05.112935 (+ 44168us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:05.112969 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:05.127346 (+ 14377us) log.cc:844] Serialized 52 byte log entry
0709 13:17:06.238311 (+1110965us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:06.238346 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:06.238406 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:06.335525 (+ 97119us) raft_consensus.cc:1685] finished
0709 13:17:06.335542 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:06.335677 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":20915,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":88154,"thread_start_us":835135,"threads_started":1,"wal-append.queue_time_us":902196}
I20260709 13:17:06.384315 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:06.646330 11505 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
I20260709 13:17:06.631634 10945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41020 (request call id 1970) took 1633 ms. Trace:
I20260709 13:17:06.647050 10945 rpcz_store.cc:276] 0709 13:17:04.998036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:06.314364 (+1316328us) service_pool.cc:224] Handling call
0709 13:17:06.631567 (+317203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:06.636310 10941 rpcz_store.cc:276] 0709 13:17:04.791472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:06.333764 (+1542292us) service_pool.cc:224] Handling call
0709 13:17:06.334701 (+   937us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:06.639020 10799 rpcz_store.cc:269] 0709 13:17:05.076452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:05.076623 (+   171us) service_pool.cc:224] Handling call
0709 13:17:06.306701 (+1230078us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b54000d9654. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:06.306923 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":346922368}
I20260709 13:17:06.329514 13328 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:06.639873 10916 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:06.348455 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:06.649035 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 312665us
I20260709 13:17:06.326805 13653 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:06.634842 11510 rpcz_store.cc:276] 0709 13:17:04.743826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:04.757636 (+ 13810us) service_pool.cc:224] Handling call
0709 13:17:04.771764 (+ 14128us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:04.784846 (+ 13082us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:04.784896 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:05.970233 (+1185337us) spinlock_profiling.cc:243] Waited 1.31 ms on lock 0x7b2c000130b8. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7f7aea11 00007f9c7f7be372 00007f9c80ef0a01 00007f9c80eb543a 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2
0709 13:17:05.992057 (+ 21824us) log.cc:844] Serialized 26 byte log entry
0709 13:17:06.272157 (+280100us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:06.272207 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:06.272296 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:06.335594 (+ 63298us) raft_consensus.cc:1685] finished
0709 13:17:06.335610 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:06.340894 (+  5284us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12711,"raft.queue_time_us":12774,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":9352,"spinlock_wait_cycles":3012992,"thread_start_us":279752,"threads_started":1,"wal-append.queue_time_us":335112}
W20260709 13:17:06.690894 11478 connection.cc:256] RPC call timeout handler was delayed by -1.4406466330001422s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2137s for a response
W20260709 13:17:06.675396 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:06.715514 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 75720.9us
I20260709 13:17:06.716344 10868 raft_consensus.cc:2393] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 7: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:06.649389 11505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56410 (request call id 533) took 1056 ms. Trace:
I20260709 13:17:06.718024 11505 rpcz_store.cc:276] 0709 13:17:05.593158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:06.343367 (+750209us) service_pool.cc:224] Handling call
0709 13:17:06.649334 (+305967us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:06.719725 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:06.721715 10799 raft_consensus.cc:2393] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 9: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:06.720106 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:06.720395 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:06.721058 10944 raft_consensus.cc:1240] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:17:06.722672 11511 raft_consensus.cc:2393] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 7: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:06.722751 11508 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 6.
I20260709 13:17:06.731643 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:06.732265 10873 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:06.732095 10944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 1950) took 1940 ms. Trace:
I20260709 13:17:06.732831 10944 rpcz_store.cc:276] 0709 13:17:04.791947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:06.645455 (+1853508us) service_pool.cc:224] Handling call
0709 13:17:06.720833 (+ 75378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:06.731810 (+ 10977us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:06.732037 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:06.733860 11511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56410 (request call id 530) took 1625 ms. Trace:
I20260709 13:17:06.734297 11511 rpcz_store.cc:276] 0709 13:17:05.108004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:06.645401 (+1537397us) service_pool.cc:224] Handling call
0709 13:17:06.733775 (+ 88374us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:06.732007 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:06.736001 10801 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:06.594632 13721 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.386915 13350 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:06.742422 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:06.746493 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:06.746322 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:06.528764 13717 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.386039 13703 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.373767 13485 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:06.627746 13678 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:06.626768 13665 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:06.702900 13534 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:06.384915 13606 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:06.773181 10799 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:06.340168 13557 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:06.694416 13690 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.987s	user 0.003s	sys 0.007s
I20260709 13:17:06.866163 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:06.336771 13409 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:06.788431 10872 raft_consensus.cc:2393] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 8: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:06.789471 10868 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:17:06.719305 13439 consensus_queue.cc:1048] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.102s
I20260709 13:17:06.345690 13331 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:06.811143 10800 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:17:06.810448 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:06.637719 13664 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:06.399219 13713 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.561784 13723 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:06.645345 13034 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:06.849318 10875 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 7.
W20260709 13:17:06.731408 11503 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56404 (request call id 545) took 1871 ms (client timeout 1962 ms). Trace:
I20260709 13:17:06.746272 13580 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:06.644892 12996 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:06.955572 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 321803us
I20260709 13:17:06.785333 13677 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:07.029656 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:07.041790 10873 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 6.
I20260709 13:17:07.043071 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:17:07.040568 10766 leader_election.cc:336] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Timed out: RequestConsensusVote RPC to 127.9.116.132:40555 timed out after 1.603s (SENT)
I20260709 13:17:07.043834 10873 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:07.066985 11505 raft_consensus.cc:2368] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 5. Current term is 6.
W20260709 13:17:07.061787 11512 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56404 (request call id 550) took 2212 ms (client timeout 1903 ms). Trace:
W20260709 13:17:07.076879 11512 rpcz_store.cc:269] 0709 13:17:04.849099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:07.039393 (+2190294us) service_pool.cc:211] Skipping call since client already timed out
0709 13:17:07.061700 (+ 22307us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:17:06.865547 13690 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.211s	user 0.004s	sys 0.008s
I20260709 13:17:07.069130 13663 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:06.820709 13274 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:17:07.062207 13612 consensus_meta.cc:287] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 1.099s	user 0.012s	sys 0.000s
I20260709 13:17:06.955047 10868 raft_consensus.cc:606] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Received request to transfer leadership to TS e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:06.656389 13635 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.529049 13657 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:07.036783 10948 raft_consensus.cc:2393] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 8: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:07.014351 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:07.105886 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:07.105927 10868 raft_consensus.cc:614] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:07.106652 10872 raft_consensus.cc:2393] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 7: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:07.107033 10800 raft_consensus.cc:2393] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 8: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:06.691396 13695 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:07.038887 13704 log.cc:927] Time spent T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.738s	user 0.001s	sys 0.001s
I20260709 13:17:06.647485 13405 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:07.102959 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 310907us
I20260709 13:17:07.107592 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:07.108172 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:06.953485 10801 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 6.
I20260709 13:17:07.126005 10946 raft_consensus.cc:1275] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:17:06.995832 13678 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:07.061836 13723 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:07.077490 13539 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:07.139133 10872 raft_consensus.cc:1240] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:17:07.168684 10873 raft_consensus.cc:2410] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:17:07.169070 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:07.139879 10840 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.007s	sys 0.035s
I20260709 13:17:06.957170 13725 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:17:07.129339 11476 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.015s	sys 0.019s
I20260709 13:17:07.063004 13677 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:06.813943 13703 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:07.100471 13634 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.713s	user 0.093s	sys 0.186s
I20260709 13:17:07.151499 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:06.814141 13717 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:07.137907 10799 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 6.
I20260709 13:17:07.069679 13713 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:07.024600 13724 consensus_queue.cc:1059] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.254s
I20260709 13:17:06.868600 13446 consensus_queue.cc:260] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:07.078745 13666 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:07.078547 13736 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.781416 13721 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:07.135545 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:07.084605 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:06.868965 13631 consensus_queue.cc:260] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:07.361831 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:17:07.315277 10797 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39442 (request call id 284) took 2742 ms (client timeout 1501 ms). Trace:
W20260709 13:17:07.374908 10840 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.838s	user 0.022s	sys 0.074s
I20260709 13:17:07.263993 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:07.375170 10797 rpcz_store.cc:269] 0709 13:17:04.572731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:04.596969 (+ 24238us) service_pool.cc:224] Handling call
0709 13:17:07.315066 (+2718097us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b5400387a54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:07.315187 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":336397184}
W20260709 13:17:07.243992 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:07.169807 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:17:07.037647 11503 rpcz_store.cc:269] 0709 13:17:04.847735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:05.290476 (+442741us) service_pool.cc:224] Handling call
0709 13:17:06.719170 (+1428694us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:07.307801 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:07.143199 13638 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:07.376355 10872 raft_consensus.cc:2393] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 6: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:07.852615 13647 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:07.856132 13647 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:07.374650 13634 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.029s	user 0.095s	sys 0.202s
I20260709 13:17:07.858080 13534 raft_consensus.cc:3055] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:07.355255 13664 consensus_queue.cc:1059] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.503s
I20260709 13:17:07.858807 13534 raft_consensus.cc:740] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:07.859012 13664 consensus_queue.cc:1243] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer cb0428cc55984daba1ba63744a5102a2 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:17:07.859673 13747 raft_consensus.cc:3055] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:07.375218 10799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 1982) took 1211 ms. Trace:
I20260709 13:17:07.860304 13747 raft_consensus.cc:740] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:07.859825 13534 consensus_queue.cc:260] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:07.861199 13677 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:07.861599 13534 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:07.251041 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 112753us
W20260709 13:17:07.862744 11506 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56416 (request call id 529) took 1439 ms (client timeout 1603 ms). Trace:
I20260709 13:17:07.861589 13747 consensus_queue.cc:260] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:07.863241 11506 rpcz_store.cc:269] 0709 13:17:06.423179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:06.780800 (+357621us) service_pool.cc:224] Handling call
0709 13:17:07.862678 (+1081878us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:07.863545 13747 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:07.863982 10916 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:17:07.277441 13708 log.cc:927] Time spent T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.865s	user 0.001s	sys 0.001s
I20260709 13:17:07.865346 13743 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:07.858546 13647 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:07.167819 13723 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:07.403071 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:07.551662 13578 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:07.464918 13413 consensus_meta.cc:287] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 1.195s	user 0.009s	sys 0.002s
W20260709 13:17:07.609589 10950 consensus_meta.cc:287] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.967s	user 0.008s	sys 0.004s
I20260709 13:17:07.894122 10950 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
W20260709 13:17:07.894634 10950 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41020 (request call id 1967) took 2989 ms (client timeout 2955 ms). Trace:
W20260709 13:17:07.894985 10950 rpcz_store.cc:269] 0709 13:17:04.905021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:06.322071 (+1417050us) service_pool.cc:224] Handling call
0709 13:17:07.894557 (+1572486us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:07.170987 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:07.896258 10801 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 1974) took 1755 ms (client timeout 2019 ms). Trace:
W20260709 13:17:07.896762 10801 rpcz_store.cc:269] 0709 13:17:06.140488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:07.140146 (+999658us) service_pool.cc:224] Handling call
0709 13:17:07.896184 (+756038us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:07.402383 10793 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39442 (request call id 290) took 2618 ms (client timeout 1743 ms). Trace:
I20260709 13:17:07.208011 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:07.141553 13612 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553) with higher term. Message: Invalid argument: T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 8. Current term is 9.
I20260709 13:17:07.170601 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:17:07.249998 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:07.901347 11505 raft_consensus.cc:2368] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 5. Current term is 6.
I20260709 13:17:07.427688 10942 raft_consensus.cc:2393] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 6: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:07.403699 13657 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:07.217314 10947 raft_consensus.cc:2368] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 5. Current term is 6.
I20260709 13:17:07.529840 10874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1562) took 2436 ms. Trace:
I20260709 13:17:07.550276 13429 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:07.855026 13424 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:07.904912 10874 rpcz_store.cc:276] 0709 13:17:05.093737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:05.093871 (+   134us) service_pool.cc:224] Handling call
0709 13:17:05.106511 (+ 12640us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:06.335900 (+1229389us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:07.529538 (+1193638us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b540009a1d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:07.529768 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2711592704}
I20260709 13:17:07.543602 10876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1564) took 2430 ms. Trace:
W20260709 13:17:07.403604 11504 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56404 (request call id 544) took 2556 ms (client timeout 1774 ms). Trace:
I20260709 13:17:07.871914 10800 raft_consensus.cc:2393] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 6: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:07.399250 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:07.712707 10875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 1570) took 1433 ms. Trace:
I20260709 13:17:07.907609 10875 rpcz_store.cc:276] 0709 13:17:06.278958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:06.279131 (+   173us) service_pool.cc:224] Handling call
0709 13:17:07.712488 (+1433357us) spinlock_profiling.cc:243] Waited 19 ms on lock 0x7b5400316ad4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:07.712625 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":43796736}
I20260709 13:17:07.547578 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:07.875098 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:07.376794 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:07.529954 10792 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 1558) took 2454 ms (client timeout 1816 ms). Trace:
I20260709 13:17:07.139493 13663 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:07.914690 11510 raft_consensus.cc:2410] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:17:07.851442 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:07.860567 10799 rpcz_store.cc:276] 0709 13:17:06.163981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:06.746194 (+582213us) service_pool.cc:224] Handling call
0709 13:17:07.375155 (+628961us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:07.916007 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:07.916311 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:07.916586 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:07.878455 11512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56416 (request call id 524) took 1456 ms. Trace:
I20260709 13:17:07.872259 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:07.876071 10842 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:07.917929 10796 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 5: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:07.918905 11512 rpcz_store.cc:276] 0709 13:17:06.421664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:07.088726 (+667062us) service_pool.cc:224] Handling call
0709 13:17:07.878386 (+789660us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:07.274739 13614 consensus_meta.cc:287] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.582s	user 0.009s	sys 0.000s
W20260709 13:17:07.910774 10792 rpcz_store.cc:269] 0709 13:17:05.075368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:05.075529 (+   161us) service_pool.cc:224] Handling call
0709 13:17:07.529747 (+2454218us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b54002125d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:07.529876 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":387354240}
I20260709 13:17:07.884481 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:07.866364 13713 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:07.279809 13727 log.cc:927] Time spent T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.209s	user 0.000s	sys 0.000s
W20260709 13:17:07.818563 13274 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.000s	sys 0.011s
W20260709 13:17:07.544188 13588 consensus_meta.cc:287] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 1.201s	user 0.010s	sys 0.000s
I20260709 13:17:07.904660 13721 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:07.934279 10794 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 9.
W20260709 13:17:07.212260 13731 log.cc:927] Time spent T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.176s	user 0.000s	sys 0.000s
W20260709 13:17:07.548483 13746 log.cc:927] Time spent T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.155s	user 0.000s	sys 0.000s
W20260709 13:17:07.207710 13729 log.cc:927] Time spent T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 13:17:07.898324 10793 rpcz_store.cc:269] 0709 13:17:04.783673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:04.783906 (+   233us) service_pool.cc:224] Handling call
0709 13:17:07.402155 (+2618249us) spinlock_profiling.cc:243] Waited 35.4 ms on lock 0x7b5400295254. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:07.402288 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":81459456}
I20260709 13:17:07.923386 13424 raft_consensus.cc:2804] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:17:07.403034 13719 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.010s	sys 0.018s
I20260709 13:17:07.850088 13446 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:07.557096 13629 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:07.355801 13651 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:07.207417 13710 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:07.973474 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:07.974378 10792 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:07.561131 13421 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:07.206646 13515 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:07.573112 13331 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:07.276924 13590 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:07.306231 13405 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:07.270713 13665 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:07.279016 13409 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:07.136849 13485 consensus_queue.cc:1059] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 6.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.758s
I20260709 13:17:07.209864 13695 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:07.902711 11505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56416 (request call id 526) took 1480 ms. Trace:
I20260709 13:17:07.999040 11505 rpcz_store.cc:276] 0709 13:17:06.422303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:07.138887 (+716584us) service_pool.cc:224] Handling call
0709 13:17:07.902618 (+763731us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:08.000087 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:08.000797 11505 raft_consensus.cc:2393] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 6: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:08.002152 11505 raft_consensus.cc:1275] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:17:07.504447 12996 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:07.498137 13557 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:07.543814 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:07.208370 13700 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:07.907312 10873 raft_consensus.cc:2393] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 8: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:08.004154 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:08.004235 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:08.004938 10873 raft_consensus.cc:2368] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 5. Current term is 6.
I20260709 13:17:08.002730 11505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56416 (request call id 530) took 1474 ms. Trace:
I20260709 13:17:08.005483 11505 rpcz_store.cc:276] 0709 13:17:06.528311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.001497 (+1473186us) service_pool.cc:224] Handling call
0709 13:17:08.001809 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:08.002471 (+   662us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:08.002651 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:08.005981 10873 raft_consensus.cc:1217] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Deduplicated request from leader. Original: 2.5->[5.6-6.7]   Dedup: 5.6->[6.7-6.7]
I20260709 13:17:08.006522 10873 raft_consensus.cc:1240] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 6. Current term is 7. Ops: [5.6-6.7]
I20260709 13:17:07.743109 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:08.007344 10867 raft_consensus.cc:2393] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 9: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:08.008731 10867 raft_consensus.cc:1240] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
W20260709 13:17:07.906311 11504 rpcz_store.cc:269] 0709 13:17:04.847460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:07.403409 (+2555949us) service_pool.cc:211] Skipping call since client already timed out
0709 13:17:07.403516 (+   107us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:17:08.011636 11504 raft_consensus.cc:3055] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:08.012141 11504 raft_consensus.cc:740] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
W20260709 13:17:07.911998 10800 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 1965) took 1787 ms (client timeout 2024 ms). Trace:
I20260709 13:17:07.925093 13614 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:07.911473 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:17:07.917352 10801 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 1565) took 1669 ms (client timeout 2130 ms). Trace:
I20260709 13:17:07.918984 11510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56410 (request call id 532) took 2406 ms. Trace:
I20260709 13:17:08.025068 11510 rpcz_store.cc:276] 0709 13:17:05.513086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:07.402954 (+1889868us) service_pool.cc:224] Handling call
0709 13:17:07.918908 (+515954us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:07.917371 10799 raft_consensus.cc:2393] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 6: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
W20260709 13:17:07.884086 11507 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56404 (request call id 543) took 3032 ms (client timeout 1692 ms). Trace:
W20260709 13:17:07.901803 13668 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.006s	sys 0.017s
I20260709 13:17:07.918507 10842 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:07.879747 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:07.155926 13678 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:07.910370 10797 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:07.936590 13588 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:07.922703 10870 raft_consensus.cc:2368] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 8. Current term is 9.
W20260709 13:17:07.918545 10796 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 1964) took 1794 ms (client timeout 2222 ms). Trace:
W20260709 13:17:08.026432 11507 rpcz_store.cc:269] 0709 13:17:04.847140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:04.849632 (+  2492us) service_pool.cc:224] Handling call
0709 13:17:07.867810 (+3018178us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b5400467654. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:07.879271 (+ 11461us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2884296064}
W20260709 13:17:08.028312 10796 rpcz_store.cc:269] 0709 13:17:06.124551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:07.851110 (+1726559us) service_pool.cc:224] Handling call
0709 13:17:07.918469 (+ 67359us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:07.923759 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:07.148401 13635 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:07.881068 13443 consensus_queue.cc:1048] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:17:07.880719 13439 consensus_queue.cc:1048] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:17:08.033764 10873 raft_consensus.cc:1240] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:07.925733 10767 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:07.460223 13730 consensus_queue.cc:1059] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.539s
I20260709 13:17:07.902288 13664 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:07.140616 13535 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:07.927851 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:08.030059 13731 log.cc:927] Time spent T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:17:07.901651 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:17:07.922765 13450 consensus_meta.cc:287] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 1.229s	user 0.013s	sys 0.000s
I20260709 13:17:07.916797 13612 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:07.916980 13638 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:07.885880 13736 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:07.928658 13429 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:07.207492 13621 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:07.927117 13274 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.001s	sys 0.012s
I20260709 13:17:08.045132 13590 raft_consensus.cc:2749] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:07.922292 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:07.904452 13413 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:07.938467 13034 consensus_meta.cc:287] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.854s	user 0.011s	sys 0.000s
I20260709 13:17:07.963171 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:07.963828 13719 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.345s	user 0.018s	sys 0.025s
W20260709 13:17:07.980592 11476 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.460s	user 0.015s	sys 0.033s
I20260709 13:17:07.920256 13663 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:07.963651 13424 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:07.929641 13655 consensus_queue.cc:1059] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:17:07.995703 10941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 1956) took 3093 ms. Trace:
I20260709 13:17:07.997407 13665 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:08.013146 10800 rpcz_store.cc:269] 0709 13:17:06.124813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:07.135196 (+1010383us) service_pool.cc:224] Handling call
0709 13:17:07.911895 (+776699us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:08.083649 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:08.084408 10800 raft_consensus.cc:2393] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 7: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
W20260709 13:17:08.084775 10800 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 1966) took 1959 ms (client timeout 2503 ms). Trace:
W20260709 13:17:08.085086 10800 rpcz_store.cc:269] 0709 13:17:06.125028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.083408 (+1958380us) service_pool.cc:224] Handling call
0709 13:17:08.084723 (+  1315us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:07.912708 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:08.018400 11504 consensus_queue.cc:260] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:08.025007 10801 rpcz_store.cc:269] 0709 13:17:06.248423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:07.903126 (+1654703us) service_pool.cc:224] Handling call
0709 13:17:07.917297 (+ 14171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:07.921667 10869 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 8.
W20260709 13:17:08.027830 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 109388us
W20260709 13:17:08.026476 10799 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 1962) took 1902 ms (client timeout 2330 ms). Trace:
W20260709 13:17:08.032387 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 156583us
I20260709 13:17:08.033609 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:07.906129 10876 rpcz_store.cc:276] 0709 13:17:05.113060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:06.158408 (+1045348us) service_pool.cc:224] Handling call
0709 13:17:06.186762 (+ 28354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:06.633339 (+446577us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:07.543310 (+909971us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b5400340154. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:07.543540 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":268102400}
W20260709 13:17:08.035554 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 109896us
I20260709 13:17:08.037142 10795 raft_consensus.cc:1240] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 LEADER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:08.037819 10943 raft_consensus.cc:2393] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 6: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:08.045909 10868 raft_consensus.cc:2393] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 6: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:07.935496 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:08.011958 13703 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:08.053900 10797 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 5.
I20260709 13:17:08.055554 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:17:08.058393 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 175089us
I20260709 13:17:08.032886 13493 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:17:08.064076 10948 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41014 (request call id 1582) took 2108 ms (client timeout 1584 ms). Trace:
I20260709 13:17:07.957597 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:08.067513 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:08.044812 13409 raft_consensus.cc:3055] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:08.143256 10796 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:17:08.045409 13725 raft_consensus.cc:697] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:08.086841 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:08.072574 13743 raft_consensus.cc:697] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 7 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:17:08.078080 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:08.066418 13421 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:08.197595 13421 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:08.199486 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:08.199586 13723 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:08.200210 10950 raft_consensus.cc:2393] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 8: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:08.087589 11509 raft_consensus.cc:2368] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 5. Current term is 6.
W20260709 13:17:08.142714 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 68243.8us
I20260709 13:17:08.068631 10798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1561) took 2979 ms. Trace:
I20260709 13:17:08.200917 10798 rpcz_store.cc:276] 0709 13:17:05.085089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:05.085274 (+   185us) service_pool.cc:224] Handling call
0709 13:17:05.085748 (+   474us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:05.128041 (+ 42293us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:05.128086 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:06.305221 (+1177135us) spinlock_profiling.cc:243] Waited 3.28 ms on lock 0x7b2c000014c8. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7f7aea11 00007f9c7f7be372 00007f9c80ef0a01 00007f9c80eb543a 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2
0709 13:17:06.313728 (+  8507us) log.cc:844] Serialized 52 byte log entry
0709 13:17:06.639185 (+325457us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:07.064568 (+425383us) log.cc:844] Serialized 14 byte log entry
0709 13:17:07.263774 (+199206us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:07.263860 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:08.057920 (+794060us) raft_consensus.cc:1685] finished
0709 13:17:08.064517 (+  6597us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:08.064659 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":571969,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":164626,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":89693,"spinlock_wait_cycles":7554560,"thread_start_us":841287,"threads_started":3,"wal-append.queue_time_us":483257}
W20260709 13:17:08.203609 11509 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56410 (request call id 528) took 3370 ms (client timeout 2222 ms). Trace:
I20260709 13:17:08.203845 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:08.204741 10798 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 5.
I20260709 13:17:08.205338 10798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 1574) took 1481 ms. Trace:
I20260709 13:17:08.205703 10798 rpcz_store.cc:276] 0709 13:17:06.724319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.203536 (+1479217us) service_pool.cc:224] Handling call
0709 13:17:08.205270 (+  1734us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:08.055348 13664 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:08.143915 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:08.208920 10800 raft_consensus.cc:2393] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 9: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
W20260709 13:17:08.209617 10799 rpcz_store.cc:269] 0709 13:17:06.124084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:07.915986 (+1791902us) service_pool.cc:224] Handling call
0709 13:17:08.026416 (+110430us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:08.210582 10948 rpcz_store.cc:269] 0709 13:17:05.955221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:06.251087 (+295866us) service_pool.cc:224] Handling call
0709 13:17:08.063862 (+1812775us) spinlock_profiling.cc:243] Waited 387 ms on lock 0x7b54002a53d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:08.063999 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":889178880}
W20260709 13:17:08.204026 11509 rpcz_store.cc:269] 0709 13:17:04.832913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:04.834481 (+  1568us) service_pool.cc:224] Handling call
0709 13:17:08.203410 (+3368929us) spinlock_profiling.cc:243] Waited 3.07 s on lock 0x7b54007e1b54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:08.203539 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7063761152}
I20260709 13:17:08.212103 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:08.081666 10941 rpcz_store.cc:276] 0709 13:17:04.902172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:06.688619 (+1786447us) service_pool.cc:224] Handling call
0709 13:17:06.740447 (+ 51828us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:06.750876 (+ 10429us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:06.750912 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:06.812453 (+ 61541us) log.cc:844] Serialized 26 byte log entry
0709 13:17:07.545855 (+733402us) spinlock_profiling.cc:243] Waited 6.46 ms on lock 0x7b180029a3a0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a550d1 00007f9c80e70a0f
0709 13:17:07.893695 (+347840us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:07.893737 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:07.893808 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:07.995357 (+101549us) raft_consensus.cc:1685] finished
0709 13:17:07.995376 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:07.995632 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33324,"raft.queue_time_us":201871,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":63248,"spinlock_wait_cycles":14853504,"thread_start_us":347642,"threads_started":1,"wal-append.queue_time_us":1037425}
I20260709 13:17:08.076516 13034 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:08.145678 13749 log.cc:927] Time spent T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 13:17:08.067431 13638 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:08.146673 13755 log.cc:927] Time spent T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:17:08.150830 13620 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:08.151364 13717 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:08.155589 13678 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:08.156730 13635 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:08.065506 13612 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:08.062999 13450 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:08.158363 13736 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:08.159543 13350 consensus_meta.cc:287] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 1.338s	user 0.012s	sys 0.000s
I20260709 13:17:08.067137 13429 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:08.260061 13429 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:08.159747 13614 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:17:08.160151 13727 log.cc:927] Time spent T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:17:08.143303 13639 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:08.143510 13409 raft_consensus.cc:740] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:08.263916 13409 consensus_queue.cc:260] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:08.265693 13409 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:08.161154 13752 log.cc:927] Time spent T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
W20260709 13:17:08.166384 10794 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 1560) took 3083 ms (client timeout 1794 ms). Trace:
W20260709 13:17:08.268184 10794 rpcz_store.cc:269] 0709 13:17:05.083173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:05.083369 (+   196us) service_pool.cc:224] Handling call
0709 13:17:08.166202 (+3082833us) spinlock_profiling.cc:243] Waited 1.79 s on lock 0x7b5400294854. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:08.166309 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4107297664}
I20260709 13:17:08.269788 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:08.270483 10794 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:17:08.168084 10947 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41020 (request call id 1955) took 3266 ms (client timeout 2222 ms). Trace:
W20260709 13:17:08.270972 10947 rpcz_store.cc:269] 0709 13:17:04.901820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:06.069573 (+1167753us) service_pool.cc:224] Handling call
0709 13:17:08.167864 (+2098291us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b5400173754. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:08.167987 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2475048704}
I20260709 13:17:08.272358 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:08.273361 10947 raft_consensus.cc:2393] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 6: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:08.169108 10792 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 6.
W20260709 13:17:08.275154 10792 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 1573) took 1555 ms (client timeout 1756 ms). Trace:
W20260709 13:17:08.275631 10792 rpcz_store.cc:269] 0709 13:17:06.720221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:07.957259 (+1237038us) service_pool.cc:224] Handling call
0709 13:17:08.275059 (+317800us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:08.170955 13759 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:08.171108 13331 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:08.276854 13759 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:08.277282 13331 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:08.171731 13655 raft_consensus.cc:3055] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:08.278827 13655 raft_consensus.cc:740] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:08.279750 13655 consensus_queue.cc:260] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:08.177757 10949 raft_consensus.cc:2393] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 9: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:08.281250 10949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41014 (request call id 1587) took 1200 ms. Trace:
I20260709 13:17:08.176898 13413 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:08.178596 13588 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:08.181063 13750 log.cc:927] Time spent T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 13:17:08.177269 13493 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:08.184319 13725 consensus_queue.cc:237] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:08.074546 10793 raft_consensus.cc:2393] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 9: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
W20260709 13:17:08.185129 10797 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 1572) took 1467 ms (client timeout 1619 ms). Trace:
I20260709 13:17:08.188334 13663 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:08.144223 13634 consensus_queue.cc:1059] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.102s
I20260709 13:17:08.320786 13034 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:08.322050 10793 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39442 (request call id 303) took 2563 ms (client timeout 2771 ms). Trace:
I20260709 13:17:08.244333 12996 consensus_queue.cc:1059] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.084s
W20260709 13:17:08.322525 10793 rpcz_store.cc:269] 0709 13:17:05.759239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:07.962872 (+2203633us) service_pool.cc:224] Handling call
0709 13:17:08.321983 (+359111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:08.323120 10944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 1958) took 3420 ms. Trace:
I20260709 13:17:08.189404 10943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41014 (request call id 1589) took 1104 ms. Trace:
I20260709 13:17:08.190536 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:08.192713 10795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 289) took 3434 ms. Trace:
I20260709 13:17:08.193707 13665 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:08.183853 10946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 1947) took 3392 ms. Trace:
I20260709 13:17:08.078334 11503 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:17:08.196571 13631 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:08.200059 13421 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:08.057485 13666 consensus_queue.cc:1048] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 13:17:08.215246 10796 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
I20260709 13:17:08.216248 11511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56410 (request call id 535) took 2128 ms. Trace:
I20260709 13:17:08.239434 13534 consensus_queue.cc:1059] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.130s
I20260709 13:17:08.215646 10842 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:08.219996 10798 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:08.231177 10800 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 1576) took 1483 ms (client timeout 1910 ms). Trace:
I20260709 13:17:08.235080 10916 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:08.236101 10841 connection.cc:256] RPC call timeout handler was delayed by -1.8407204259999617s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2503s for a response
I20260709 13:17:08.237841 10767 leader_election.cc:400] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 5. Current term is 6.
I20260709 13:17:08.238947 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:08.080888 13424 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:08.240063 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:08.240826 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:08.253917 10871 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 7.
I20260709 13:17:08.255921 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:08.274520 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:08.281656 10949 rpcz_store.cc:276] 0709 13:17:07.080800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.033306 (+952506us) service_pool.cc:224] Handling call
0709 13:17:08.281188 (+247882us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:08.316767 13493 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:08.317889 10797 rpcz_store.cc:269] 0709 13:17:06.717494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:07.376443 (+658949us) service_pool.cc:224] Handling call
0709 13:17:08.185069 (+808626us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:08.177677 13677 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:08.253085 13633 consensus_queue.cc:1059] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.139s
I20260709 13:17:08.534148 11504 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:08.544334 13678 raft_consensus.cc:2804] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:08.544579 10950 raft_consensus.cc:2393] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 5: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:08.544934 13678 raft_consensus.cc:697] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:08.545169 10950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41014 (request call id 1593) took 1235 ms. Trace:
W20260709 13:17:08.545287 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 310261us
W20260709 13:17:08.543990 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:08.545636 10950 rpcz_store.cc:276] 0709 13:17:07.309287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.229676 (+920389us) service_pool.cc:224] Handling call
0709 13:17:08.545112 (+315436us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:08.545866 13678 consensus_queue.cc:237] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:08.546890 10916 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:08.544513 10796 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 1976) took 2403 ms (client timeout 2445 ms). Trace:
I20260709 13:17:08.546941 13629 raft_consensus.cc:2749] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:08.564543 10794 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
W20260709 13:17:08.547631 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 332082us
I20260709 13:17:08.566370 10916 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:08.566893 10842 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:08.567830 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:08.568192 10916 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:08.567870 13614 raft_consensus.cc:2749] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:08.571071 13736 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:08.571502 13736 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:08.571843 13736 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:08.548069 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:08.549047 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:08.575757 10916 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:08.549477 13634 raft_consensus.cc:3055] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:08.576671 13634 raft_consensus.cc:740] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:08.323463 10944 rpcz_store.cc:276] 0709 13:17:04.902686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:06.733649 (+1830963us) service_pool.cc:224] Handling call
0709 13:17:06.779747 (+ 46098us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:07.077440 (+297693us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:07.077470 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:07.079377 (+  1907us) log.cc:844] Serialized 26 byte log entry
0709 13:17:07.376774 (+297397us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:07.964176 (+587402us) log.cc:844] Serialized 14 byte log entry
0709 13:17:07.964966 (+   790us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:07.965038 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:08.169461 (+204423us) spinlock_profiling.cc:243] Waited 8.71 ms on lock 0x7b180029a340. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7dd42c0a 00007f9c87482219 00007f9c84d8958f 00007f9c7ddc8571 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd
0709 13:17:08.180509 (+ 11048us) spinlock_profiling.cc:243] Waited 2.61 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:17:08.322918 (+142409us) raft_consensus.cc:1685] finished
0709 13:17:08.322937 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:08.323068 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17744,"prepare.queue_time_us":313105,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":71,"raft.queue_time_us":347016,"raft.run_cpu_time_us":277,"raft.run_wall_time_us":10239,"spinlock_wait_cycles":26016896,"thread_start_us":921698,"threads_started":3,"wal-append.queue_time_us":804024}
I20260709 13:17:08.565217 13450 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:08.578056 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:08.550019 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:08.578771 10797 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:08.579286 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:08.579924 10792 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 6.
W20260709 13:17:08.549746 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:08.550379 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:08.323514 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:08.580955 10949 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 4.
I20260709 13:17:08.581761 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:08.581902 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:08.582280 10949 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:08.582585 10793 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7.
I20260709 13:17:08.550088 13655 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:17:08.583585 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:08.584683 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:08.551671 10941 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 8.
I20260709 13:17:08.596184 10793 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:08.596529 10941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41014 (request call id 1591) took 1287 ms. Trace:
I20260709 13:17:08.552187 10799 raft_consensus.cc:2393] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 9: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:08.596748 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:08.596941 10941 rpcz_store.cc:276] 0709 13:17:07.308652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.234330 (+925678us) service_pool.cc:224] Handling call
0709 13:17:08.596472 (+362142us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:08.548177 10948 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 4.
I20260709 13:17:08.400301 13743 consensus_queue.cc:237] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:08.551084 10841 connection.cc:256] RPC call timeout handler was delayed by -1.7956972750000659s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2765s for a response
W20260709 13:17:08.600157 10841 connection.cc:256] RPC call timeout handler was delayed by -1.7956972750000659s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2765s for a response
W20260709 13:17:08.600520 10841 connection.cc:256] RPC call timeout handler was delayed by -1.7657631600000059s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18680000000000002s for a response
W20260709 13:17:08.600796 10841 connection.cc:256] RPC call timeout handler was delayed by -1.7298554320000221s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1988s for a response
I20260709 13:17:08.600690 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:08.601351 10841 connection.cc:256] RPC call timeout handler was delayed by -1.6577631600000586s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1988s for a response
I20260709 13:17:08.595295 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:08.602141 10841 connection.cc:256] RPC call timeout handler was delayed by -1.5872817449999275s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20190000000000002s for a response
I20260709 13:17:08.602171 10876 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 7.
I20260709 13:17:08.556231 10943 rpcz_store.cc:276] 0709 13:17:07.084659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:07.147359 (+ 62700us) service_pool.cc:224] Handling call
0709 13:17:08.189231 (+1041872us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b54000159d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:08.189351 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":319482240}
W20260709 13:17:08.327050 10915 leader_election.cc:336] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Timed out: RequestConsensusVote RPC to 127.9.116.132:40555 timed out after 1.588s (SENT)
I20260709 13:17:08.603405 10915 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:08.557758 10942 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 8.
I20260709 13:17:08.604175 10942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41014 (request call id 1595) took 1253 ms. Trace:
I20260709 13:17:08.558660 11505 raft_consensus.cc:1240] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 8. Current term is 9. Ops: []
W20260709 13:17:08.549512 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 313773us
I20260709 13:17:08.605469 10767 leader_election.cc:400] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 8. Current term is 9.
I20260709 13:17:08.605763 10767 leader_election.cc:403] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:17:08.560400 13677 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:08.551198 10795 rpcz_store.cc:276] 0709 13:17:04.758616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:04.760266 (+  1650us) service_pool.cc:224] Handling call
0709 13:17:04.771214 (+ 10948us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:08.143241 (+3372027us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:08.192488 (+ 49247us) spinlock_profiling.cc:243] Waited 3.27 s on lock 0x7b5400294d54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:08.192652 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7510398720}
W20260709 13:17:08.610396 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 283359us
W20260709 13:17:08.551004 10800 rpcz_store.cc:269] 0709 13:17:06.747546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.143660 (+1396114us) service_pool.cc:224] Handling call
0709 13:17:08.231117 (+ 87457us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:08.548082 10796 rpcz_store.cc:269] 0709 13:17:06.141078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.033395 (+1892317us) service_pool.cc:224] Handling call
0709 13:17:08.544447 (+511052us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:08.577677 13634 consensus_queue.cc:260] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:08.578261 13034 raft_consensus.cc:2804] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:17:08.580410 10792 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 1961) took 2456 ms (client timeout 3104 ms). Trace:
W20260709 13:17:08.615736 10792 rpcz_store.cc:269] 0709 13:17:06.123823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.276463 (+2152640us) service_pool.cc:224] Handling call
0709 13:17:08.580354 (+303891us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:08.616684 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:08.617380 10792 raft_consensus.cc:2393] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 6: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:08.617846 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:08.618202 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:08.585348 13638 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:08.548221 11511 rpcz_store.cc:276] 0709 13:17:06.087958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:07.443694 (+1355736us) service_pool.cc:224] Handling call
0709 13:17:08.173473 (+729779us) spinlock_profiling.cc:243] Waited 8.66 ms on lock 0x7b5800003c98. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd90a22 00007f9c86ca9c09 00007f9c85e03ee0 00007f9c85ecd0ab 00007f9c85ecd131 00007f9c85e8efc0 00007f9c85e79929 00007f9c85e793d2 00007f9c7f3f0bd4
0709 13:17:08.173734 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:08.201192 (+ 27458us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:08.201229 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:08.201270 (+    41us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:08.201325 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:08.202827 (+  1502us) spinlock_profiling.cc:243] Waited 27.2 ms on lock 0x7b540021e654. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:08.202897 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:08.216169 (+ 13272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":82532096}
I20260709 13:17:08.621587 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:08.551517 10947 raft_consensus.cc:2393] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 9: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:08.323408 12996 raft_consensus.cc:3055] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:08.626493 10873 raft_consensus.cc:1240] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:08.589310 13450 raft_consensus.cc:2804] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:08.587061 10842 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:17:08.585700 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 339657us
W20260709 13:17:08.589397 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 239163us
W20260709 13:17:08.318168 13749 log.cc:927] Time spent T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:17:08.324142 10801 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 6.
I20260709 13:17:08.551055 13759 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:08.575146 13614 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:08.594295 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:17:08.565464 10794 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 1981) took 2408 ms (client timeout 3012 ms). Trace:
W20260709 13:17:08.671693 10794 rpcz_store.cc:269] 0709 13:17:06.157471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.269431 (+2111960us) service_pool.cc:224] Handling call
0709 13:17:08.565394 (+295963us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:08.595498 10949 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 9.
I20260709 13:17:08.598454 10948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41014 (request call id 1586) took 1518 ms. Trace:
I20260709 13:17:08.678596 10948 rpcz_store.cc:276] 0709 13:17:07.080503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.211594 (+1131091us) service_pool.cc:224] Handling call
0709 13:17:08.598380 (+386786us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:08.596232 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:08.607074 10799 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 1970) took 2467 ms (client timeout 2955 ms). Trace:
W20260709 13:17:08.679703 10799 rpcz_store.cc:269] 0709 13:17:06.139233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.210479 (+2071246us) service_pool.cc:224] Handling call
0709 13:17:08.607011 (+396532us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:08.607899 10797 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
I20260709 13:17:08.608078 10798 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
I20260709 13:17:08.680748 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:17:08.681288 10798 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 1973) took 2541 ms (client timeout 2792 ms). Trace:
W20260709 13:17:08.681766 10798 rpcz_store.cc:269] 0709 13:17:06.140191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.206676 (+2066485us) service_pool.cc:224] Handling call
0709 13:17:08.681217 (+474541us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:08.604498 10942 rpcz_store.cc:276] 0709 13:17:07.351247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.190271 (+839024us) service_pool.cc:224] Handling call
0709 13:17:08.604117 (+413846us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:08.613742 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:08.684405 10796 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:08.684924 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:08.622006 13274 raft_consensus.cc:2749] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:08.614935 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:08.619318 10792 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 6.
I20260709 13:17:08.612583 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:08.622599 10944 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 6.
I20260709 13:17:08.622663 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:08.622469 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:08.625679 12996 raft_consensus.cc:740] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:08.623715 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:08.695377 10867 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:08.550014 10946 rpcz_store.cc:276] 0709 13:17:04.791205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:06.645282 (+1854077us) service_pool.cc:224] Handling call
0709 13:17:06.719779 (+ 74497us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:07.139432 (+419653us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:08.183610 (+1044178us) spinlock_profiling.cc:243] Waited 370 ms on lock 0x7b54000aafd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:08.183787 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":850187008}
I20260709 13:17:08.629714 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:08.630285 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:08.696617 10916 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:08.640142 10876 raft_consensus.cc:1275] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:17:08.641978 10767 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:08.642208 10793 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7.
I20260709 13:17:08.699672 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:08.645820 11505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56410 (request call id 537) took 1067 ms. Trace:
I20260709 13:17:08.700546 11505 rpcz_store.cc:276] 0709 13:17:07.578720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.006426 (+427706us) service_pool.cc:224] Handling call
0709 13:17:08.006718 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:08.605050 (+598332us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:08.645566 (+ 40516us) spinlock_profiling.cc:243] Waited 552 ms on lock 0x7b54001d1354. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:08.645756 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1268970240}
I20260709 13:17:08.636708 10766 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:08.649451 11478 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:17:08.701974 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:08.702464 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 53115.9us
I20260709 13:17:08.703404 11478 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:08.610447 11504 raft_consensus.cc:1275] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:17:08.706367 11504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56410 (request call id 539) took 1127 ms. Trace:
I20260709 13:17:08.706837 11504 rpcz_store.cc:276] 0709 13:17:07.579386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.011012 (+431626us) service_pool.cc:224] Handling call
0709 13:17:08.011384 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:08.522016 (+510632us) spinlock_profiling.cc:243] Waited 225 ms on lock 0x7b54006c0aa0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80e2aa1d 00007f9c80e111a4 00007f9c80e93447 00007f9c80eabfe7 00007f9c80ead7de 00007f9c80eb7b49 00007f9c80eb8416 00007f9c80eb4d8f 00007f9c80eb483e 00007f9c85e79419
0709 13:17:08.706025 (+184009us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:08.706296 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":518478592}
I20260709 13:17:08.628903 13657 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:08.691161 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:08.691735 10800 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:08.710274 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 73679.2us
I20260709 13:17:08.683454 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:08.711930 10947 raft_consensus.cc:2393] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 9: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:08.692610 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:08.698658 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 56765.1us
I20260709 13:17:08.713042 10794 raft_consensus.cc:2393] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 6: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
W20260709 13:17:08.599750 13763 log.cc:927] Time spent T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.356s	user 0.000s	sys 0.002s
I20260709 13:17:08.715327 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:08.652619 13638 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:08.652375 13663 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:08.684496 13450 raft_consensus.cc:697] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 5 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:08.692747 10869 raft_consensus.cc:2368] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 5. Current term is 6.
I20260709 13:17:08.714926 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:08.625766 13665 raft_consensus.cc:2749] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 8. Current term is 9.
W20260709 13:17:08.716254 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:08.716886 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:08.718993 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:08.724331 10873 raft_consensus.cc:1275] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:17:08.713258 10767 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:08.722532 10801 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 1963) took 2598 ms (client timeout 2547 ms). Trace:
I20260709 13:17:08.724588 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:08.726969 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:08.726114 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:08.727516 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:08.727814 11478 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:08.710289 10949 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 7.
I20260709 13:17:08.728425 10800 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
I20260709 13:17:08.731101 10947 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:17:08.740787 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:08.740891 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:17:08.732481 10801 rpcz_store.cc:269] 0709 13:17:06.124327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.086491 (+1962164us) service_pool.cc:224] Handling call
0709 13:17:08.722302 (+635811us) spinlock_profiling.cc:243] Waited 99 ms on lock 0x7b54001aa4d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:08.722443 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":227588992}
I20260709 13:17:08.733897 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:08.741631 10946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41014 (request call id 1588) took 1657 ms. Trace:
I20260709 13:17:08.742774 10916 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:08.742969 10950 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:08.743120 10946 rpcz_store.cc:276] 0709 13:17:07.084402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.717843 (+1633441us) service_pool.cc:224] Handling call
0709 13:17:08.741574 (+ 23731us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:08.735841 10799 raft_consensus.cc:2393] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 8: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:08.743714 10867 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 6.
I20260709 13:17:08.736141 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:08.745149 10871 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:17:08.716349 13746 log.cc:927] Time spent T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:17:08.684000 13534 consensus_queue.cc:1048] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:17:08.709285 13614 raft_consensus.cc:697] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:17:08.822242 10841 connection.cc:256] RPC call timeout handler was delayed by -1.4905972750000274s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3104s for a response
I20260709 13:17:08.830333 10950 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 8.
I20260709 13:17:08.715216 13639 raft_consensus.cc:2749] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:17:08.745791 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 107119us
W20260709 13:17:08.928097 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:08.773748 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:08.736493 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:17:08.891664 10841 connection.cc:256] RPC call timeout handler was delayed by -1.3321972749999986s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.328s for a response
I20260709 13:17:08.772444 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:08.931416 10841 connection.cc:256] RPC call timeout handler was delayed by -1.0416386500000954s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2445s for a response
I20260709 13:17:08.932231 10841 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:08.735569 10873 raft_consensus.cc:1275] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:17:08.720590 13331 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:08.720324 13635 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:08.719681 13429 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:08.715994 13759 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:08.933678 10874 raft_consensus.cc:1240] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:08.732311 10767 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:17:08.993957 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 261744us
I20260709 13:17:08.995026 10767 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:08.745301 10796 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 6.
I20260709 13:17:08.996935 10796 raft_consensus.cc:1240] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:08.997742 10796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 300) took 3701 ms. Trace:
I20260709 13:17:08.729230 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:08.998205 10796 rpcz_store.cc:276] 0709 13:17:05.296851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.996300 (+3699449us) service_pool.cc:224] Handling call
0709 13:17:08.996707 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:08.997425 (+   718us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:08.997675 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:08.820640 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:08.772428 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:09.001482 10867 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 6.
I20260709 13:17:08.728799 13350 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:08.819429 10800 raft_consensus.cc:1240] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:17:08.741004 13620 raft_consensus.cc:2749] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:08.720106 13677 raft_consensus.cc:2804] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:08.928843 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:09.051797 10799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 1967) took 2926 ms. Trace:
I20260709 13:17:08.740572 13663 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:08.740146 13638 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:08.943838 10941 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 6: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:08.946204 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:08.948930 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:08.929738 13274 consensus_queue.cc:1059] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.111s
I20260709 13:17:08.950187 10801 raft_consensus.cc:1240] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 7. Current term is 8. Ops: [7.3-7.3]
I20260709 13:17:09.061383 10801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1563) took 3972 ms. Trace:
I20260709 13:17:09.061844 10801 rpcz_store.cc:276] 0709 13:17:05.089317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.949491 (+3860174us) service_pool.cc:224] Handling call
0709 13:17:08.949920 (+   429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:09.061000 (+111080us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:09.061307 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:08.953148 10915 leader_election.cc:336] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Timed out: RequestConsensusVote RPC to 127.9.116.132:40555 timed out after 1.962s (SENT)
I20260709 13:17:09.063071 10915 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:09.061862 10869 raft_consensus.cc:2393] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 8: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:17:08.902524 13749 log.cc:927] Time spent T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 13:17:08.729111 13710 raft_consensus.cc:3055] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:09.093163 13710 raft_consensus.cc:740] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:08.736265 13439 consensus_queue.cc:1059] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.225s
I20260709 13:17:08.961279 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:08.824254 13725 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:09.095239 13725 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:09.095634 13725 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:09.095710 13612 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:09.095963 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:09.096349 13612 raft_consensus.cc:2749] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:09.097167 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:09.097769 10946 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 6.
I20260709 13:17:08.962394 10873 raft_consensus.cc:1240] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:09.098937 13760 raft_consensus.cc:2749] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:09.099684 10947 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 6.
I20260709 13:17:08.983232 13450 consensus_queue.cc:237] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:08.745545 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:09.104290 10795 raft_consensus.cc:2393] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 8: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:09.104465 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:08.899093 13665 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:08.721669 13657 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:09.085847 13493 consensus_meta.cc:287] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.531s	user 0.006s	sys 0.001s
I20260709 13:17:09.020942 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:08.727095 13034 raft_consensus.cc:697] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:09.047116 10800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1562) took 3958 ms. Trace:
I20260709 13:17:09.114965 10800 rpcz_store.cc:276] 0709 13:17:05.088304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.818870 (+3730566us) service_pool.cc:224] Handling call
0709 13:17:08.819183 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:09.046823 (+227640us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:09.047052 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:08.818961 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:09.117660 10944 raft_consensus.cc:2393] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 8: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:08.960309 13678 raft_consensus.cc:2804] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:08.991392 13633 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:09.052219 10799 rpcz_store.cc:276] 0709 13:17:06.125254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.821727 (+2696473us) service_pool.cc:224] Handling call
0709 13:17:09.051735 (+230008us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:08.938289 10916 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:08.959295 13429 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:08.738433 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:09.057652 10871 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 9.
I20260709 13:17:09.002797 13635 raft_consensus.cc:697] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 6 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:09.051657 13639 raft_consensus.cc:2749] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:09.070940 13614 consensus_queue.cc:237] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:09.051429 13331 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:09.052889 13677 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:09.075191 13274 raft_consensus.cc:3055] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:17:09.076352 13736 consensus_meta.cc:287] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.504s	user 0.009s	sys 0.000s
I20260709 13:17:09.076839 13663 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:09.077741 10796 raft_consensus.cc:1275] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
W20260709 13:17:09.077939 13631 consensus_meta.cc:287] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.529s	user 0.008s	sys 0.001s
I20260709 13:17:08.945354 12996 consensus_queue.cc:260] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:09.081838 13424 consensus_meta.cc:287] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.530s	user 0.003s	sys 0.007s
W20260709 13:17:09.196583 13409 consensus_meta.cc:287] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.930s	user 0.017s	sys 0.005s
W20260709 13:17:09.083319 13655 consensus_meta.cc:287] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.500s	user 0.009s	sys 0.000s
I20260709 13:17:09.196586 12996 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:09.202955 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:09.196825 13424 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:09.204782 10875 raft_consensus.cc:1240] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:09.207417 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:09.207842 13424 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:09.208345 10944 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:09.083458 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:09.214732 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:09.216238 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:09.217502 10941 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891"
dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
mode: GRACEFUL
new_leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
 from {username='slave'} at 127.0.0.1:46052
I20260709 13:17:09.218129 10941 raft_consensus.cc:606] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Received request to transfer leadership to TS 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:09.085857 13638 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:09.000900 13759 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:09.221324 13759 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:09.086549 10793 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 7.
W20260709 13:17:09.091243 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 138124us
I20260709 13:17:09.091730 10801 raft_consensus.cc:1240] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
W20260709 13:17:08.975353 13764 log.cc:927] Time spent T 00000000000000000000000000000000 P ae957c8fc66c4b6ba063d0c74cbf5484: Append to log took a long time: real 0.150s	user 0.000s	sys 0.000s
I20260709 13:17:09.100708 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:09.101004 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 373224us
I20260709 13:17:09.222694 10801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 1971) took 3083 ms. Trace:
I20260709 13:17:09.223063 10801 rpcz_store.cc:276] 0709 13:17:06.139581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.090993 (+2951412us) service_pool.cc:224] Handling call
0709 13:17:09.091451 (+   458us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:09.222399 (+130948us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:09.222640 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:09.224408 10801 raft_consensus.cc:1275] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:17:09.224939 10801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 1979) took 3083 ms. Trace:
I20260709 13:17:09.225276 10801 rpcz_store.cc:276] 0709 13:17:06.141934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.223808 (+3081874us) service_pool.cc:224] Handling call
0709 13:17:09.224092 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:09.224730 (+   638us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:09.224889 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:09.104987 10947 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:09.196967 10799 raft_consensus.cc:1240] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:09.227722 10799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 1978) took 3086 ms. Trace:
I20260709 13:17:09.228209 10799 rpcz_store.cc:276] 0709 13:17:06.141650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.122666 (+2981016us) service_pool.cc:224] Handling call
0709 13:17:09.196790 (+ 74124us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:09.227407 (+ 30617us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:09.227650 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:08.818439 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:17:09.107262 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 520240us
I20260709 13:17:09.114087 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:09.218577 10941 raft_consensus.cc:614] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:09.127681 10945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 1951) took 4329 ms. Trace:
I20260709 13:17:09.231537 10945 rpcz_store.cc:276] 0709 13:17:04.792178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:06.647824 (+1855646us) service_pool.cc:224] Handling call
0709 13:17:06.724984 (+ 77160us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:06.745016 (+ 20032us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:17:07.148252 (+403236us) log.cc:844] Serialized 14 byte log entry
0709 13:17:07.371271 (+223019us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:07.371895 (+   624us) log.cc:844] Serialized 26 byte log entry
0709 13:17:07.372070 (+   175us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:17:07.429758 (+ 57688us) log.cc:844] Serialized 14 byte log entry
0709 13:17:07.430133 (+   375us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:07.430248 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:09.121849 (+1691601us) raft_consensus.cc:1685] finished
0709 13:17:09.121869 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:09.122001 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32955,"prepare.queue_time_us":1012516,"prepare.run_cpu_time_us":105,"prepare.run_wall_time_us":124,"raft.queue_time_us":223791,"raft.run_cpu_time_us":552,"raft.run_wall_time_us":134239,"thread_start_us":488208,"threads_started":2,"wal-append.queue_time_us":741630}
I20260709 13:17:09.128859 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:09.131997 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 193777us
I20260709 13:17:09.239840 10947 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 9.
I20260709 13:17:09.242079 10948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36714 (request call id 379) took 1011 ms. Trace:
I20260709 13:17:09.242647 10948 rpcz_store.cc:276] 0709 13:17:08.230289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.113861 (+883572us) service_pool.cc:224] Handling call
0709 13:17:09.241994 (+128133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:09.243510 10801 raft_consensus.cc:1275] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:17:09.094278 13710 consensus_queue.cc:260] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:09.151080 13760 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:17:09.098549 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:09.114101 10795 raft_consensus.cc:1240] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 13:17:09.153120 10796 pending_rounds.cc:85] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 6
I20260709 13:17:09.247793 10795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 1975) took 3107 ms. Trace:
W20260709 13:17:09.243597 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:09.243592 10916 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:09.130954 13429 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:09.245880 10941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:46052 (request call id 8) took 1798 ms. Trace:
I20260709 13:17:09.257808 10941 rpcz_store.cc:276] 0709 13:17:07.485000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.217366 (+1732366us) service_pool.cc:224] Handling call
0709 13:17:09.234276 (+ 16910us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:09.246174 10942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36714 (request call id 378) took 1027 ms. Trace:
I20260709 13:17:09.258404 10942 rpcz_store.cc:276] 0709 13:17:08.229905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.728935 (+499030us) service_pool.cc:224] Handling call
0709 13:17:09.246092 (+517157us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:09.248237 10796 pending_rounds.cc:107] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 6.7
W20260709 13:17:09.245464 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 313307us
I20260709 13:17:09.259318 10796 raft_consensus.cc:2889] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:09.260833 10841 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:09.248255 10795 rpcz_store.cc:276] 0709 13:17:06.140781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.104861 (+2964080us) service_pool.cc:224] Handling call
0709 13:17:09.113861 (+  9000us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:09.247518 (+133657us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:09.247736 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:09.152469 13678 raft_consensus.cc:697] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:09.112351 13665 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:09.254912 10799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1566) took 2981 ms. Trace:
I20260709 13:17:09.122012 13633 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:09.244342 10801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1564) took 3018 ms. Trace:
I20260709 13:17:09.268700 10916 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:09.193605 13718 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:09.192523 13657 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:09.267700 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:09.271029 10801 rpcz_store.cc:276] 0709 13:17:06.226295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.226349 (+3000054us) service_pool.cc:224] Handling call
0709 13:17:09.226690 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:09.244025 (+ 17335us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:09.244276 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:09.267858 10942 raft_consensus.cc:1240] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:17:09.265162 10842 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:09.272598 10801 raft_consensus.cc:1240] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:09.272909 10842 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:09.273180 10941 raft_consensus.cc:1240] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 13:17:09.273306 10801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1568) took 2991 ms. Trace:
I20260709 13:17:09.273790 10801 rpcz_store.cc:276] 0709 13:17:06.282208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.272049 (+2989841us) service_pool.cc:224] Handling call
0709 13:17:09.272397 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:09.273029 (+   632us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:09.273244 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:09.263881 10795 raft_consensus.cc:1275] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:17:09.268709 10799 rpcz_store.cc:276] 0709 13:17:06.273500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.229165 (+2955665us) service_pool.cc:224] Handling call
0709 13:17:09.254239 (+ 25074us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:09.254466 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:09.254503 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:09.254557 (+    54us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:09.254608 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:09.254676 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:09.254841 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:09.275252 10795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1567) took 2996 ms. Trace:
I20260709 13:17:09.275767 10795 rpcz_store.cc:276] 0709 13:17:06.278713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.261734 (+2983021us) service_pool.cc:224] Handling call
0709 13:17:09.262049 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:09.274947 (+ 12898us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:09.275181 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:09.248996 13655 raft_consensus.cc:2804] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:09.278437 10841 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:09.278812 10942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 1959) took 4375 ms. Trace:
I20260709 13:17:09.275611 10842 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:09.279433 10801 raft_consensus.cc:1240] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 6. Current term is 8. Ops: []
I20260709 13:17:09.279623 10941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 1960) took 4376 ms. Trace:
I20260709 13:17:09.281250 10942 rpcz_store.cc:276] 0709 13:17:04.903017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.267291 (+4364274us) service_pool.cc:224] Handling call
0709 13:17:09.267630 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:09.278427 (+ 10797us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:09.278710 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:09.281716 10941 rpcz_store.cc:276] 0709 13:17:04.903261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.272640 (+4369379us) service_pool.cc:224] Handling call
0709 13:17:09.272970 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:09.279285 (+  6315us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:09.279556 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:09.153344 13274 raft_consensus.cc:740] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:09.153616 13677 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:09.290271 10801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1569) took 3004 ms. Trace:
I20260709 13:17:09.291245 10801 rpcz_store.cc:276] 0709 13:17:06.286256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.274615 (+2988359us) service_pool.cc:224] Handling call
0709 13:17:09.279189 (+  4574us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:09.289807 (+ 10618us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:09.290214 (+   407us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:09.284235 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:09.284432 10942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 1962) took 4380 ms. Trace:
I20260709 13:17:09.293063 10942 rpcz_store.cc:276] 0709 13:17:04.903722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.283500 (+4379778us) service_pool.cc:224] Handling call
0709 13:17:09.283798 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:09.284045 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:09.284083 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:09.284143 (+    60us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:09.284190 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:09.284259 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:09.284373 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:09.274587 13718 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:09.241930 13736 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:09.226410 13331 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:09.295248 10942 raft_consensus.cc:1240] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:09.283299 10941 raft_consensus.cc:1240] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:17:09.287659 10799 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:09.287848 10795 raft_consensus.cc:1275] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:17:09.247089 13759 consensus_queue.cc:1048] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:17:09.308488 10795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1575) took 2572 ms. Trace:
I20260709 13:17:09.309291 10795 rpcz_store.cc:276] 0709 13:17:06.735596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.276863 (+2541267us) service_pool.cc:224] Handling call
0709 13:17:09.287482 (+ 10619us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:09.302074 (+ 14592us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:09.308416 (+  6342us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:09.281217 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 286299us
I20260709 13:17:09.308503 10942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 1964) took 4404 ms. Trace:
I20260709 13:17:09.280229 13633 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:17:09.304867 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:09.307367 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:09.316584 10942 rpcz_store.cc:276] 0709 13:17:04.904243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.294650 (+4390407us) service_pool.cc:224] Handling call
0709 13:17:09.295085 (+   435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:09.295775 (+   690us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:09.308450 (+ 12675us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:09.317605 10945 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 6.
I20260709 13:17:09.319583 10801 raft_consensus.cc:2410] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:17:09.319825 10941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 1961) took 4416 ms. Trace:
I20260709 13:17:09.320237 10941 rpcz_store.cc:276] 0709 13:17:04.903498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.282618 (+4379120us) service_pool.cc:224] Handling call
0709 13:17:09.283024 (+   406us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:09.315397 (+ 32373us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:09.319763 (+  4366us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:09.316744 10767 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:09.288257 13655 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:09.118629 13493 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:09.330147 10945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 371) took 1106 ms. Trace:
I20260709 13:17:09.288483 13772 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:17:09.327333 10942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 1584) took 2247 ms. Trace:
I20260709 13:17:09.255795 13725 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:09.264120 13760 consensus_queue.cc:1048] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 13:17:09.342895 13719 consensus_queue.cc:1059] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:09.343891 13719 raft_consensus.cc:3055] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:09.344255 13719 raft_consensus.cc:740] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:09.344498 10950 raft_consensus.cc:2393] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 8: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:09.331100 10841 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:09.345072 13719 consensus_queue.cc:260] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:17:09.330680 10945 rpcz_store.cc:276] 0709 13:17:08.216871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.320642 (+1103771us) service_pool.cc:224] Handling call
0709 13:17:09.321086 (+   444us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:09.321274 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:09.321296 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:09.321334 (+    38us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:09.321369 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:09.321436 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:09.323081 (+  1645us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:09.346453 13719 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:09.347180 10945 raft_consensus.cc:1240] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:09.347801 10945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 374) took 1130 ms. Trace:
I20260709 13:17:09.348112 10945 rpcz_store.cc:276] 0709 13:17:08.217771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.346664 (+1128893us) service_pool.cc:224] Handling call
0709 13:17:09.347021 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:09.347565 (+   544us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:09.347748 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:09.348716 13493 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:09.349036 10945 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2"
dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
mode: GRACEFUL
new_leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
 from {username='slave'} at 127.0.0.1:46052
I20260709 13:17:09.349361 13493 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:09.349596 10945 raft_consensus.cc:606] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Received request to transfer leadership to TS 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:09.349856 13493 raft_consensus.cc:697] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:09.280745 13773 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:09.337273 13274 consensus_queue.cc:260] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:09.351418 13274 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:09.334254 13776 raft_consensus.cc:2749] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:09.355067 13634 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:09.357025 13725 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:09.349967 10945 raft_consensus.cc:614] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:09.309109 13350 raft_consensus.cc:697] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 7 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:09.360088 10868 raft_consensus.cc:1240] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:09.331118 10942 rpcz_store.cc:276] 0709 13:17:07.079839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.326279 (+2246440us) service_pool.cc:224] Handling call
0709 13:17:09.326656 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:09.326911 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:09.326950 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:09.327007 (+    57us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:09.327059 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:09.327132 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:09.327264 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:09.333474 10767 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:09.365936 10944 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:17:09.366755 10944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41020 (request call id 1973) took 1120 ms. Trace:
I20260709 13:17:09.359166 13588 raft_consensus.cc:2804] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:09.368114 13678 consensus_queue.cc:237] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:09.368979 10945 raft_consensus.cc:1240] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:09.369571 10945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 380) took 1138 ms. Trace:
I20260709 13:17:09.369875 10945 rpcz_store.cc:276] 0709 13:17:08.231141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.368514 (+1137373us) service_pool.cc:224] Handling call
0709 13:17:09.368814 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:09.369338 (+   524us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:09.369521 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:09.371532 10945 raft_consensus.cc:1275] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:17:09.372540 13331 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:09.367195 10944 rpcz_store.cc:276] 0709 13:17:08.246051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.207099 (+961048us) service_pool.cc:224] Handling call
0709 13:17:09.366660 (+159561us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:09.308604 13718 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:09.374982 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:09.376605 10801 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
I20260709 13:17:09.320581 13657 raft_consensus.cc:2749] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:17:09.378177 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 153742us
W20260709 13:17:09.399847 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:09.401897 13778 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:09.402441 13778 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:09.403316 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:09.404074 10871 raft_consensus.cc:2393] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 8: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:09.368152 13350 consensus_queue.cc:237] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:09.404534 13778 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:09.404824 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:09.405844 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:09.377097 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:09.406292 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:09.369802 13633 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:09.331408 13655 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:09.407289 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:09.408051 10916 leader_election.cc:400] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 5. Current term is 6.
I20260709 13:17:09.412070 10916 leader_election.cc:403] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:17:09.412828 10799 raft_consensus.cc:1275] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 13:17:09.413415 10915 leader_election.cc:336] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Timed out: RequestConsensusVote RPC to 127.9.116.132:40555 timed out after 1.903s (SENT)
I20260709 13:17:09.408192 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:09.413931 10799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1570) took 3116 ms. Trace:
I20260709 13:17:09.385686 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:09.277518 13034 consensus_queue.cc:237] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:09.386960 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:09.327068 13635 consensus_queue.cc:237] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:09.413272 10916 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:09.324961 13775 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:09.392019 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:09.308426 13774 consensus_queue.cc:1059] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.102s
I20260709 13:17:09.396225 13665 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:09.407265 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:09.414417 11504 raft_consensus.cc:2393] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 9: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:09.414939 10801 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:09.433844 11510 raft_consensus.cc:1240] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:09.406828 10795 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
I20260709 13:17:09.431030 13759 consensus_queue.cc:1048] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 13:17:09.435667 10916 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:09.436964 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:09.416386 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:09.438730 10867 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:09.406759 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:09.417852 10792 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:09.418728 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 85332.3us
I20260709 13:17:09.441967 10767 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:09.414475 10799 rpcz_store.cc:276] 0709 13:17:06.297420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.275471 (+2978051us) service_pool.cc:224] Handling call
0709 13:17:09.287546 (+ 12075us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:09.413624 (+126078us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:09.413862 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38528}
I20260709 13:17:09.436419 13781 raft_consensus.cc:2804] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:09.466436 11511 raft_consensus.cc:1275] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:17:09.472477 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:09.482081 10767 leader_election.cc:400] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 election: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 5. Current term is 6.
I20260709 13:17:09.411201 13664 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:09.431177 13409 raft_consensus.cc:2749] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 5. Current term is 6.
I20260709 13:17:09.494166 10801 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
I20260709 13:17:09.503566 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:09.505148 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:09.512215 13779 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:09.519474 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:09.473263 10799 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 7.
I20260709 13:17:09.492138 10767 leader_election.cc:403] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
W20260709 13:17:09.544937 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 63054.7us
W20260709 13:17:09.547194 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:09.470438 13781 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:09.470108 13775 raft_consensus.cc:697] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:09.531745 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:09.544086 10795 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 6.
I20260709 13:17:09.549774 10867 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 8.
I20260709 13:17:09.532907 10801 raft_consensus.cc:1240] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:09.535120 13413 raft_consensus.cc:3055] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:09.536638 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:09.537595 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:09.538902 10942 raft_consensus.cc:1240] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:09.383435 13493 consensus_queue.cc:237] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:09.431820 13588 raft_consensus.cc:697] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:09.506901 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:09.553799 10801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 1984) took 1694 ms. Trace:
I20260709 13:17:09.554282 11512 raft_consensus.cc:2376] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 7. Re-sending same reply.
I20260709 13:17:09.554276 10801 rpcz_store.cc:276] 0709 13:17:07.859498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.504430 (+1644932us) service_pool.cc:224] Handling call
0709 13:17:09.532733 (+ 28303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:09.553479 (+ 20746us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:09.553730 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:09.498704 10792 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 6.
I20260709 13:17:09.548039 10767 leader_election.cc:400] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 election: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 8. Current term is 9.
I20260709 13:17:09.557013 10767 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:09.558398 10767 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:09.552641 10916 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:09.564406 10767 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:09.552641 10870 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 6.
I20260709 13:17:09.567797 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:09.568185 10916 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:09.491395 13759 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:09.585160 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:09.586076 10798 raft_consensus.cc:2376] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 5. Re-sending same reply.
I20260709 13:17:09.583213 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:09.583623 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:09.492235 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:09.594997 10916 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:09.601852 10795 raft_consensus.cc:1240] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:09.603058 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:09.604596 10795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 1991) took 1691 ms. Trace:
I20260709 13:17:09.604609 10916 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:09.605046 10795 rpcz_store.cc:276] 0709 13:17:07.913316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.551720 (+1638404us) service_pool.cc:224] Handling call
0709 13:17:09.601673 (+ 49953us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:09.604330 (+  2657us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:09.604540 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:09.606115 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:09.550235 13665 raft_consensus.cc:2749] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:09.608915 10869 raft_consensus.cc:2393] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 7: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:09.563681 13666 raft_consensus.cc:2749] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 5. Current term is 6.
I20260709 13:17:09.565100 13413 raft_consensus.cc:740] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:09.565651 13424 raft_consensus.cc:2804] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:09.580612 13331 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:09.584342 13725 raft_consensus.cc:2804] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:17:09.678228 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 73726us
I20260709 13:17:09.563377 13781 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:09.672565 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:09.611074 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:09.677168 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:09.730350 11512 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:09.609975 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:09.587711 13773 raft_consensus.cc:697] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:09.573477 13664 raft_consensus.cc:2749] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:17:09.708806 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 311281us
I20260709 13:17:09.574554 13777 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:09.610646 11503 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 7.
W20260709 13:17:09.708317 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 362117us
I20260709 13:17:09.751792 10841 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:09.689640 10867 raft_consensus.cc:1240] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:17:09.610002 10801 raft_consensus.cc:1275] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:17:09.753549 10801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1577) took 1149 ms. Trace:
I20260709 13:17:09.753970 10801 rpcz_store.cc:276] 0709 13:17:08.603832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.574537 (+970705us) service_pool.cc:224] Handling call
0709 13:17:09.609569 (+ 35032us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:09.753268 (+143699us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:09.753487 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:09.648061 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:09.756572 11477 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:17:09.688385 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 215878us
I20260709 13:17:09.723062 10916 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:09.726738 10869 raft_consensus.cc:1275] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:17:09.727283 11510 raft_consensus.cc:1240] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 7. Current term is 8. Ops: []
W20260709 13:17:09.732183 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:09.705669 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:09.760594 10799 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:09.762257 10916 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:09.729883 10798 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:09.734935 10941 raft_consensus.cc:1240] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 5 LEADER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:09.613236 13409 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:09.608243 13759 raft_consensus.cc:697] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:09.685838 13760 raft_consensus.cc:2749] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:17:09.574311 13639 raft_consensus.cc:2749] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:17:09.729983 13725 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:09.773847 10798 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 10.
I20260709 13:17:09.734481 13331 raft_consensus.cc:697] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 8 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:09.706269 13429 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:09.740917 13631 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:09.748255 13777 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:09.749071 13588 consensus_queue.cc:237] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:09.789273 10842 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:09.779414 13409 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:09.728184 13774 consensus_queue.cc:1059] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:17:09.785864 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:09.774387 13450 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:17:09.782996 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 50601.2us
I20260709 13:17:09.789901 13633 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:09.815368 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:09.794076 13725 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:09.799515 13678 consensus_queue.cc:1048] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:17:09.799697 13778 consensus_queue.cc:1059] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:09.800052 13723 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:09.799553 13771 consensus_queue.cc:1059] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:09.802814 10841 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:09.799213 10946 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5.
I20260709 13:17:09.782404 10799 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 6.
I20260709 13:17:09.774377 13793 raft_consensus.cc:3055] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:09.794480 13789 raft_consensus.cc:2804] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:09.833374 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:09.796674 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:09.834149 10799 raft_consensus.cc:2376] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 6. Re-sending same reply.
I20260709 13:17:09.802997 13776 consensus_queue.cc:1048] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:09.824595 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:09.837905 11510 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 6: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:09.780952 13350 raft_consensus.cc:2749] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:09.828310 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:09.840970 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:09.831188 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:17:09.814504 13784 log.cc:927] Time spent T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 13:17:09.823310 10794 raft_consensus.cc:2393] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 9: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:09.813757 13409 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:09.859653 11510 raft_consensus.cc:1275] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:17:09.803925 13777 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:09.778847 13413 consensus_queue.cc:260] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:09.803499 13743 consensus_queue.cc:1048] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:17:09.792493 13791 consensus_queue.cc:1059] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 13:17:09.852617 11509 raft_consensus.cc:2393] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 9: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:09.834348 10798 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:09.880002 10915 leader_election.cc:336] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Timed out: RequestConsensusVote RPC to 127.9.116.132:40555 timed out after 2.020s (SENT)
I20260709 13:17:09.880589 10915 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:09.812537 13760 consensus_queue.cc:1048] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
I20260709 13:17:09.816027 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:09.837085 13789 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:09.862316 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:09.859412 13678 raft_consensus.cc:3055] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:09.831645 13769 raft_consensus.cc:2804] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:09.868561 13768 raft_consensus.cc:2804] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:09.866148 13655 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:09.870607 10870 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "12682eb70bc84c76879e04a8628ee586"
 from {username='slave'} at 127.0.0.1:53748
W20260709 13:17:09.871321 13783 log.cc:927] Time spent T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:17:09.847090 13773 consensus_queue.cc:237] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:09.851604 13786 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:17:09.983955 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:09.984892 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 228429us
I20260709 13:17:09.985723 11477 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:09.987603 11477 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:09.841784 12996 consensus_queue.cc:1048] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 13:17:09.877350 13787 raft_consensus.cc:3055] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:09.810878 13631 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:09.839387 13793 raft_consensus.cc:740] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:09.910907 13450 raft_consensus.cc:697] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 6 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:09.883502 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:09.853220 11508 raft_consensus.cc:2393] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 9: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:09.886739 11512 raft_consensus.cc:2410] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:17:09.889740 10798 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 8.
I20260709 13:17:09.778224 13775 consensus_queue.cc:237] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:09.790288 13657 consensus_queue.cc:1048] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:17:10.030869 11507 raft_consensus.cc:2410] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:17:09.840572 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:17:09.915071 10915 leader_election.cc:336] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Timed out: RequestConsensusVote RPC to 127.9.116.132:40555 timed out after 2.058s (SENT)
I20260709 13:17:10.035564 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:17:10.036347 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 121311us
W20260709 13:17:10.037091 10915 leader_election.cc:336] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Timed out: RequestConsensusVote RPC to 127.9.116.132:40555 timed out after 2.098s (SENT)
I20260709 13:17:09.999524 13787 raft_consensus.cc:740] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
W20260709 13:17:09.965166 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:09.894868 13274 consensus_meta.cc:287] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.539s	user 0.007s	sys 0.004s
W20260709 13:17:10.040627 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 241338us
I20260709 13:17:10.044615 10916 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:09.963876 13768 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:09.983297 10800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 1977) took 3842 ms. Trace:
W20260709 13:17:09.997788 10766 leader_election.cc:336] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Timed out: RequestConsensusVote RPC to 127.9.116.132:40555 timed out after 1.568s (SENT)
I20260709 13:17:10.045462 10800 rpcz_store.cc:276] 0709 13:17:06.141370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.115943 (+2974573us) service_pool.cc:224] Handling call
0709 13:17:09.116328 (+   385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:09.116737 (+   409us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:09.116776 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:09.136113 (+ 19337us) log.cc:844] Serialized 26 byte log entry
0709 13:17:09.489389 (+353276us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:09.489428 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:09.489498 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:09.983057 (+493559us) raft_consensus.cc:1685] finished
0709 13:17:09.983077 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:09.983229 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4590,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":13370,"thread_start_us":294183,"threads_started":1,"wal-append.queue_time_us":460214}
I20260709 13:17:10.047083 10766 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:10.047752 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:10.003870 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:17:10.005506 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 202773us
I20260709 13:17:09.890687 13350 raft_consensus.cc:2749] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:09.883358 10799 raft_consensus.cc:1240] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 8. Current term is 9. Ops: [6.5-8.6]
I20260709 13:17:09.896661 13789 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:09.895202 13678 raft_consensus.cc:740] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:10.023729 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:09.974711 13769 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
W20260709 13:17:10.025461 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 236238us
I20260709 13:17:10.030514 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:10.057883 10798 raft_consensus.cc:2368] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for earlier term 5. Current term is 6.
I20260709 13:17:09.997828 13798 raft_consensus.cc:2749] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:09.891927 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:09.841269 13759 consensus_queue.cc:237] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:09.996301 13786 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:10.007393 13794 raft_consensus.cc:2804] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:10.037346 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:10.037531 10915 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:10.065346 10950 raft_consensus.cc:1240] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:09.964341 10870 raft_consensus.cc:606] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Received request to transfer leadership to TS 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:10.066089 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:09.849711 13718 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:10.050854 13768 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:10.050616 10841 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:10.051395 10799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1581) took 1446 ms. Trace:
I20260709 13:17:10.022411 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:10.051684 11509 raft_consensus.cc:1275] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:17:10.055593 13723 raft_consensus.cc:2749] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:10.052294 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:17:10.052816 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 55063.2us
W20260709 13:17:10.057199 11478 connection.cc:256] RPC call timeout handler was delayed by -2.5189028020001842s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2771s for a response
I20260709 13:17:09.928610 13331 consensus_queue.cc:237] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:10.058717 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:10.095460 10798 raft_consensus.cc:2376] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 8. Re-sending same reply.
I20260709 13:17:10.096549 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:10.097213 10798 raft_consensus.cc:2376] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 6. Re-sending same reply.
I20260709 13:17:10.098488 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:10.099483 10798 raft_consensus.cc:2393] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 6: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:10.059290 10842 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:10.066236 10870 raft_consensus.cc:614] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:10.038820 10876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1590) took 1329 ms. Trace:
I20260709 13:17:10.066973 10947 raft_consensus.cc:2393] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 9: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:10.045776 10916 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:10.057646 13760 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:10.026811 13781 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:10.069734 10799 rpcz_store.cc:276] 0709 13:17:08.605083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.859609 (+1254526us) service_pool.cc:224] Handling call
0709 13:17:09.883119 (+ 23510us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:10.051101 (+167982us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:10.051336 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:10.070037 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:10.113005 10944 raft_consensus.cc:2410] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:17:10.083580 11506 raft_consensus.cc:2410] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:17:10.113687 10948 raft_consensus.cc:1275] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:17:10.064851 13786 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:10.116149 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:10.116454 10872 raft_consensus.cc:2393] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 7: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:10.116896 10798 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 6: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:10.074738 10800 raft_consensus.cc:2376] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 10. Re-sending same reply.
I20260709 13:17:10.114167  9682 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:17:10.117856 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:10.058887 13769 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:10.077746 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:10.070032 13614 raft_consensus.cc:2749] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:17:10.156630 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:10.064563 13794 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:10.079751 13678 consensus_queue.cc:260] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:10.104712 10876 rpcz_store.cc:276] 0709 13:17:08.715675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.717953 (+  2278us) service_pool.cc:224] Handling call
0709 13:17:08.736548 (+ 18595us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:09.104108 (+367560us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:09.104140 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:09.130604 (+ 26464us) log.cc:844] Serialized 26 byte log entry
0709 13:17:09.551426 (+420822us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:09.551464 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:09.551539 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:10.038597 (+487058us) raft_consensus.cc:1685] finished
0709 13:17:10.038616 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:10.038739 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":413979,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":41707,"thread_start_us":724545,"threads_started":2,"wal-append.queue_time_us":538987}
I20260709 13:17:10.108654 10841 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:10.109669 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:10.111011 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 65308.7us
W20260709 13:17:10.074679 13677 consensus_meta.cc:287] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.756s	user 0.011s	sys 0.000s
W20260709 13:17:10.112094 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 52901.7us
W20260709 13:17:10.114539 11478 connection.cc:256] RPC call timeout handler was delayed by -1.8898324100000536s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1501s for a response
I20260709 13:17:10.109638 13760 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:17:10.084420 10766 connection.cc:256] RPC call timeout handler was delayed by -1.069985484999961s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1786s for a response
I20260709 13:17:10.173539 10798 raft_consensus.cc:1240] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:10.193490 10842 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:10.196331 10798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1587) took 1585 ms. Trace:
I20260709 13:17:10.093536 13274 consensus_queue.cc:1059] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.099s
I20260709 13:17:10.141229 10800 raft_consensus.cc:1240] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:10.191552 10947 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 5.
I20260709 13:17:10.191679 10916 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:17:10.191891 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 83344.3us
I20260709 13:17:10.201879 10800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1586) took 1592 ms. Trace:
I20260709 13:17:10.204921 10841 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:10.203780 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:10.206101 10841 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:10.120397 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:10.207589 10799 raft_consensus.cc:2393] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 9: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
W20260709 13:17:10.193923 11478 connection.cc:256] RPC call timeout handler was delayed by -1.6657324099999187s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17500000000000002s for a response
I20260709 13:17:10.204916 10800 rpcz_store.cc:276] 0709 13:17:08.609455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:10.134883 (+1525428us) service_pool.cc:224] Handling call
0709 13:17:10.135311 (+   428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:10.201422 (+ 66111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:10.201827 (+   405us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:10.111860 13655 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:10.196683 10798 rpcz_store.cc:276] 0709 13:17:08.611109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:10.173017 (+1561908us) service_pool.cc:224] Handling call
0709 13:17:10.173351 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:10.196071 (+ 22720us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:10.196273 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:10.115916 13633 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:10.111553 13806 log.cc:927] Time spent T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:17:10.110716 13657 consensus_queue.cc:1059] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.089s
I20260709 13:17:10.194585 13776 raft_consensus.cc:2804] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:10.104467 13350 consensus_queue.cc:1048] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.076s
I20260709 13:17:10.197103 10766 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:17:10.244525 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 47492.4us
I20260709 13:17:10.224397 13778 raft_consensus.cc:3055] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:10.231537 13633 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:10.255570 13778 raft_consensus.cc:740] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:10.256723 13778 consensus_queue.cc:260] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:10.257615 13633 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:10.258078 13778 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:10.209668 13773 consensus_queue.cc:1059] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 13:17:10.245576 10766 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:10.204079 13723 consensus_queue.cc:1048] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:17:10.105350 13808 raft_consensus.cc:2749] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:17:10.176254 13629 raft_consensus.cc:3055] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:17:10.208720 11478 connection.cc:256] RPC call timeout handler was delayed by -1.586308956000039s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
W20260709 13:17:10.327170 11478 connection.cc:256] RPC call timeout handler was delayed by -1.5334324099999321s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1897s for a response
W20260709 13:17:10.327539 11478 connection.cc:256] RPC call timeout handler was delayed by -1.4560324100000344s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19830000000000003s for a response
I20260709 13:17:10.090600 13631 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:10.210266 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:10.334894 10799 raft_consensus.cc:2376] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5. Re-sending same reply.
I20260709 13:17:10.211046 13772 raft_consensus.cc:2749] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:10.210970 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:10.338816 10800 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:17:10.214419 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:10.174569 13804 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:10.343156 13804 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:10.196646 13794 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:10.194998 13759 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:10.212957 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:10.198343 13760 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:10.204210 13493 raft_consensus.cc:2804] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:10.199935 13805 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:10.119443 13743 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:10.067586 13793 consensus_queue.cc:260] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:10.284252 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:10.482398 10947 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 7.
I20260709 13:17:10.262946 13639 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:10.286967 13655 raft_consensus.cc:3055] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:10.227048 13678 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:10.207396 13034 raft_consensus.cc:2749] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:10.299548 13620 raft_consensus.cc:2764] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
W20260709 13:17:10.301939 13813 log.cc:927] Time spent T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:17:10.306902 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:10.307749 13718 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:10.300352 13789 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:10.260842 13274 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:10.197360 13677 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:10.236819 13776 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:10.339212 11478 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:17:10.497411 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 158270us
I20260709 13:17:10.498092 11478 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:10.172643 13450 consensus_queue.cc:237] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:10.343427 11477 leader_election.cc:336] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Timed out: RequestConsensusVote RPC to 127.9.116.131:45553 timed out after 2.771s (SENT)
I20260709 13:17:10.404526 10798 raft_consensus.cc:2376] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7. Re-sending same reply.
I20260709 13:17:10.499176 11477 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:10.499876 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:10.405475 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 160016us
I20260709 13:17:10.355482 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:10.500339 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 156957us
I20260709 13:17:10.500808 10799 raft_consensus.cc:2376] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 5. Re-sending same reply.
I20260709 13:17:10.500923 10766 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:10.501096 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:10.406531 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:17:10.501801 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:10.501698 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:10.500391 10798 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:10.502296 10799 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 6.
I20260709 13:17:10.120791 13787 consensus_queue.cc:260] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:10.293094 13723 consensus_queue.cc:1243] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer 12682eb70bc84c76879e04a8628ee586 log is divergent from this leader: its last log entry 7.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:17:10.408324 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:17:10.504526 13634 consensus_meta.cc:287] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 1.149s	user 0.008s	sys 0.005s
I20260709 13:17:10.538571 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:10.349617 10800 raft_consensus.cc:2410] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:17:10.540637 10800 raft_consensus.cc:1240] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:10.541392 10800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1592) took 1804 ms. Trace:
I20260709 13:17:10.541803 10800 rpcz_store.cc:276] 0709 13:17:08.736825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:10.540065 (+1803240us) service_pool.cc:224] Handling call
0709 13:17:10.540419 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:10.541115 (+   696us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:10.541324 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:10.542407 13429 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:10.475632 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 269562us
I20260709 13:17:10.543350 10841 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:10.544869 10794 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 1585) took 1935 ms (client timeout 2193 ms). Trace:
I20260709 13:17:10.545434 13677 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:10.546437 11511 raft_consensus.cc:1240] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:10.548552 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:10.543356 13620 raft_consensus.cc:2749] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:17:10.506146 13633 raft_consensus.cc:2749] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:17:10.551764 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 52102.6us
I20260709 13:17:10.553051 13768 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:10.554476 10870 raft_consensus.cc:1240] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:10.555656 10915 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:10.558033 10870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1549) took 1142 ms. Trace:
W20260709 13:17:10.479048 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 287433us
I20260709 13:17:10.477577 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:10.559072 11512 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 6.
I20260709 13:17:10.559885 13789 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:10.560170 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:10.343600 13804 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:10.485519 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:10.499135 13776 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:10.558063 13774 consensus_queue.cc:1059] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 13:17:10.326930 13629 raft_consensus.cc:740] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:10.564080 13534 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:17:10.564347 13777 consensus_meta.cc:287] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.674s	user 0.009s	sys 0.000s
I20260709 13:17:10.564010 13629 consensus_queue.cc:260] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:10.564796 13534 raft_consensus.cc:697] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 6 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:10.565147 13629 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:10.564850 13777 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:10.483399 13743 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:10.566107 10792 raft_consensus.cc:1240] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:10.565904 13534 consensus_queue.cc:237] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:10.484042 13493 raft_consensus.cc:697] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:10.567539 10792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1578) took 1963 ms. Trace:
I20260709 13:17:10.568043 10792 rpcz_store.cc:276] 0709 13:17:08.604179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.576116 (+971937us) service_pool.cc:224] Handling call
0709 13:17:09.576522 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:10.566498 (+989976us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:10.567265 (+   767us) spinlock_profiling.cc:243] Waited 989 ms on lock 0x7b54002111d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:10.567467 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2275413888}
I20260709 13:17:10.486756 13655 raft_consensus.cc:740] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:10.507757 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:10.512351 13631 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:17:10.591367 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 48082us
I20260709 13:17:10.593041 10841 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:10.596131 10842 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:10.520507 13786 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:10.527037 13781 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:10.481643 13805 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:10.473707 13759 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:10.613612 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:10.535418 13274 raft_consensus.cc:697] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 6 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:10.492398 13639 raft_consensus.cc:697] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:10.493340 13793 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:10.489912 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:10.406126 10874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1586) took 1760 ms. Trace:
I20260709 13:17:10.543462 10942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 376) took 2325 ms. Trace:
I20260709 13:17:10.629240 10942 rpcz_store.cc:276] 0709 13:17:08.218338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.361648 (+1143310us) service_pool.cc:224] Handling call
0709 13:17:09.538599 (+176951us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:09.553342 (+ 14743us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:10.543286 (+989944us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b540018ecd0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:10.543406 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":405958912}
I20260709 13:17:10.629098 10874 rpcz_store.cc:276] 0709 13:17:08.657248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.667270 (+ 10022us) service_pool.cc:224] Handling call
0709 13:17:08.672577 (+  5307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:08.993112 (+320535us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:10.405865 (+1412753us) spinlock_profiling.cc:243] Waited 261 ms on lock 0x7b540006b3d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:10.406061 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":599828352}
I20260709 13:17:10.554932 13663 consensus_queue.cc:1048] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:10.558133 13798 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:10.560987 11512 raft_consensus.cc:2393] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 8: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:10.566422 13743 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:10.503175 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:10.545270 10794 rpcz_store.cc:269] 0709 13:17:08.609173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.719953 (+110780us) service_pool.cc:224] Handling call
0709 13:17:10.544693 (+1824740us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54000bc7d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:10.544815 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2501664640}
I20260709 13:17:10.558609 10870 rpcz_store.cc:276] 0709 13:17:09.416385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:10.104626 (+688241us) service_pool.cc:224] Handling call
0709 13:17:10.104914 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:10.555869 (+450955us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:10.557712 (+  1843us) spinlock_profiling.cc:243] Waited 449 ms on lock 0x7b54000f69d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:10.557947 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1033430784}
I20260709 13:17:10.634608 10799 raft_consensus.cc:2376] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8. Re-sending same reply.
W20260709 13:17:10.574910 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:10.576972 10798 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7.
I20260709 13:17:10.579432 10876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1604) took 1111 ms. Trace:
I20260709 13:17:10.316332 13775 raft_consensus.cc:3055] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:17:10.580891 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:10.588368 11478 leader_election.cc:400] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 5 pre-election: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for earlier term 5. Current term is 6.
W20260709 13:17:10.647104 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 58856.7us
I20260709 13:17:10.613283 13631 raft_consensus.cc:697] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 10 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:10.476014 10875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1587) took 1803 ms. Trace:
I20260709 13:17:10.653357 10875 rpcz_store.cc:276] 0709 13:17:08.672949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.673140 (+   191us) service_pool.cc:224] Handling call
0709 13:17:08.673433 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:09.205154 (+531721us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:10.475769 (+1270615us) spinlock_profiling.cc:243] Waited 531 ms on lock 0x7b540009b5d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:10.475954 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1221569280}
I20260709 13:17:10.655061 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:10.601122 10841 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:10.602385 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:10.597582 10842 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:10.604672 10792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 1994) took 1321 ms. Trace:
I20260709 13:17:10.657827 10792 rpcz_store.cc:276] 0709 13:17:09.283488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:10.571850 (+1288362us) service_pool.cc:224] Handling call
0709 13:17:10.590305 (+ 18455us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:10.604180 (+ 13875us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:10.604229 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:10.604281 (+    52us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:10.604335 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:10.604424 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:10.604595 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11618,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":12955}
I20260709 13:17:10.618861 10795 raft_consensus.cc:1240] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:10.660171 10795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 1995) took 1367 ms. Trace:
I20260709 13:17:10.660636 10795 rpcz_store.cc:276] 0709 13:17:09.292736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:10.584459 (+1291723us) service_pool.cc:224] Handling call
0709 13:17:10.618596 (+ 34137us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:10.659778 (+ 41182us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:10.660090 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:10.619858 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:10.622263 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 121403us
I20260709 13:17:10.662200 10766 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:10.624429 10796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 301) took 5009 ms. Trace:
I20260709 13:17:10.662941 10796 rpcz_store.cc:276] 0709 13:17:05.614926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.040648 (+3425722us) service_pool.cc:224] Handling call
0709 13:17:09.041093 (+   445us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:09.270217 (+229124us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:10.624187 (+1353970us) spinlock_profiling.cc:243] Waited 36.2 ms on lock 0x7b54000d9154. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:10.624370 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":83204992}
I20260709 13:17:10.663451 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:10.664054 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:10.664521 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:10.664836 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:10.665298 10870 raft_consensus.cc:2393] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 9: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:10.665455 10796 raft_consensus.cc:1275] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:17:10.589428 12996 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:17:10.580667 13493 consensus_queue.cc:237] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:10.666276 12996 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:10.602396 13813 log.cc:927] Time spent T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:17:10.666778 13805 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:10.667774 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:10.603123 13778 raft_consensus.cc:2749] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:10.602684 13350 consensus_queue.cc:1048] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:17:10.639885 10876 rpcz_store.cc:276] 0709 13:17:09.467650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:10.192510 (+724860us) service_pool.cc:224] Handling call
0709 13:17:10.206195 (+ 13685us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:10.206522 (+   327us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:17:10.206551 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:10.209327 (+  2776us) log.cc:844] Serialized 26 byte log entry
0709 13:17:10.485476 (+276149us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:10.485520 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:10.485597 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:10.578926 (+ 93329us) raft_consensus.cc:1685] finished
0709 13:17:10.578956 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:10.579335 (+   379us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":105,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":8463,"thread_start_us":275795,"threads_started":1,"wal-append.queue_time_us":318038}
I20260709 13:17:10.613633 13718 raft_consensus.cc:2749] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:10.673403 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:10.676210 10870 raft_consensus.cc:2410] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 13:17:10.676038 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 78516.1us
I20260709 13:17:10.677107 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:10.644387 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 141894us
I20260709 13:17:10.679258 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:17:10.476670 10793 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 1580) took 1871 ms (client timeout 2015 ms). Trace:
I20260709 13:17:10.621340 13789 raft_consensus.cc:2804] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:17:10.650224 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 59429.7us
I20260709 13:17:10.665400 10800 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:10.620558 13725 consensus_meta.cc:287] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.770s	user 0.011s	sys 0.001s
I20260709 13:17:10.633512 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:10.670203 10766 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:10.688023 10766 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:10.688452 10792 raft_consensus.cc:1275] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:17:10.689047 10792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1593) took 1313 ms. Trace:
I20260709 13:17:10.623896 13787 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:10.664562 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:10.674762 10796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 1998) took 1336 ms. Trace:
I20260709 13:17:10.690537 10796 rpcz_store.cc:276] 0709 13:17:09.332579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:10.664911 (+1332332us) service_pool.cc:224] Handling call
0709 13:17:10.665172 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:10.668759 (+  3587us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:10.668954 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:10.649253 11478 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:10.679925 10793 rpcz_store.cc:269] 0709 13:17:08.604815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.699325 (+ 94510us) service_pool.cc:224] Handling call
0709 13:17:10.476512 (+1777187us) spinlock_profiling.cc:243] Waited 386 ms on lock 0x7b5400211bd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:10.476614 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":887910656}
W20260709 13:17:10.682752 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 81820.1us
I20260709 13:17:10.683502 10916 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:10.677160 10842 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:10.693300 10842 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:10.693451 10841 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:10.694137 10842 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:10.627228 13781 consensus_queue.cc:1048] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
W20260709 13:17:10.696413 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 47255.3us
I20260709 13:17:10.698150 11478 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:10.669171 12996 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:10.699077 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:10.699368 13805 consensus_queue.cc:1059] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 13:17:10.699738 11512 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 6.
I20260709 13:17:10.700273 12996 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:10.700644 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:10.700731 13725 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:10.701768 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:10.702509 11512 raft_consensus.cc:2410] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 10 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:17:10.702853 13725 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:10.702922 13633 consensus_queue.cc:1059] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:17:10.695055 13717 raft_consensus.cc:2749] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:10.704423 13772 raft_consensus.cc:2804] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:10.704813 13772 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:10.705155 13772 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:10.706961 13034 raft_consensus.cc:2804] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:10.707288 10766 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:10.707495 13034 raft_consensus.cc:697] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:10.708402 13034 consensus_queue.cc:237] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:10.709450 13429 raft_consensus.cc:2764] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader election decision vote started in defunct term 6: won
W20260709 13:17:10.709807 10766 leader_election.cc:336] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Timed out: RequestConsensusVote RPC to 127.9.116.132:40555 timed out after 1.584s (SENT)
I20260709 13:17:10.710282 10766 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:10.710961 13789 raft_consensus.cc:697] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:10.711922 13789 consensus_queue.cc:237] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:10.698074 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:10.667966 13274 consensus_queue.cc:237] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:10.645776 13775 raft_consensus.cc:740] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:10.714375 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:10.689332 10792 rpcz_store.cc:276] 0709 13:17:09.375733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:10.675144 (+1299411us) service_pool.cc:224] Handling call
0709 13:17:10.688158 (+ 13014us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:10.688825 (+   667us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:10.688995 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:10.718407 10946 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41020 (request call id 1982) took 2142 ms (client timeout 2226 ms). Trace:
W20260709 13:17:10.729574 10766 leader_election.cc:336] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Timed out: RequestConsensusVote RPC to 127.9.116.132:40555 timed out after 1.901s (SENT)
I20260709 13:17:10.730038 10766 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:10.719161 10950 raft_consensus.cc:1240] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 7 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:10.731302 10842 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:10.723242 10868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1576) took 2131 ms. Trace:
W20260709 13:17:10.728524 10946 rpcz_store.cc:269] 0709 13:17:08.575687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.098314 (+522627us) service_pool.cc:224] Handling call
0709 13:17:10.718236 (+1619922us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b54003589d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:10.718352 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":428678016}
I20260709 13:17:10.732527 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:10.732065 10868 rpcz_store.cc:276] 0709 13:17:08.591919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.594914 (+  2995us) service_pool.cc:224] Handling call
0709 13:17:08.623254 (+ 28340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:09.360644 (+737390us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:10.723006 (+1362362us) spinlock_profiling.cc:243] Waited 736 ms on lock 0x7b54003165d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:10.723173 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1693916416}
W20260709 13:17:10.718895 10943 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41020 (request call id 1979) took 2466 ms (client timeout 1919 ms). Trace:
W20260709 13:17:10.733713 10943 rpcz_store.cc:269] 0709 13:17:08.252284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.697319 (+445035us) service_pool.cc:224] Handling call
0709 13:17:10.718678 (+2021359us) spinlock_profiling.cc:243] Waited 764 ms on lock 0x7b54002a58d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:10.718825 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1757071744}
W20260709 13:17:10.732302 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 44418.8us
I20260709 13:17:10.735088 11477 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:10.643796 13798 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:10.728789 13788 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:17:10.632038 13655 consensus_queue.cc:260] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:10.738952 13666 raft_consensus.cc:2749] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:17:10.739568 13655 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:10.716548 13639 consensus_queue.cc:237] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:10.739979 11510 raft_consensus.cc:2410] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:17:10.741629 13666 raft_consensus.cc:2749] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:10.741731 13710 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:10.742707 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 153852us
I20260709 13:17:10.745426 10874 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 9.
I20260709 13:17:10.738119 13631 consensus_queue.cc:237] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:10.746376 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:10.755580 12996 consensus_queue.cc:1059] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:10.678686 13826 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:10.613880 13819 consensus_queue.cc:1048] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.076s
I20260709 13:17:10.756912 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:10.749444 10842 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:10.758404 11477 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:17:10.714111 13409 consensus_meta.cc:287] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.830s	user 0.000s	sys 0.008s
I20260709 13:17:10.715233 13677 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:10.760761 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:10.696197 13771 raft_consensus.cc:2804] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:10.770066 12996 consensus_queue.cc:1243] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer e01ecd008edd4aa2b6b2b47efbc2ea87 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:17:10.742718 13718 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:10.752085 13717 raft_consensus.cc:2749] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:17:10.662309 13769 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:10.714907 13620 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:10.714823 13779 raft_consensus.cc:2749] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:10.719085 13736 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:10.819518 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 78394.7us
I20260709 13:17:10.717162 13768 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:10.763242 13823 log.cc:927] Time spent T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 13:17:10.716441 13777 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:10.729178 13657 consensus_queue.cc:1059] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 13:17:10.715076 13774 consensus_queue.cc:1059] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.086s
I20260709 13:17:10.760375 13666 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
W20260709 13:17:10.804677 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 121227us
I20260709 13:17:10.830925 10766 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:10.768580 13788 raft_consensus.cc:2804] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:10.766978 13781 raft_consensus.cc:3055] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:10.850871 13781 raft_consensus.cc:740] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:10.851790 13781 consensus_queue.cc:260] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:10.853089 13781 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:10.854441 13798 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:10.829442 10800 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 7.
I20260709 13:17:10.715698 13818 consensus_queue.cc:1059] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.104s
I20260709 13:17:10.874931 11478 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:10.761023 13791 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:10.822597 13718 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:10.825433 13633 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:10.756490 13809 consensus_queue.cc:1048] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.075s
I20260709 13:17:10.841601 10869 raft_consensus.cc:1240] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 4. Current term is 6. Ops: []
I20260709 13:17:10.842974 13826 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:17:10.843058 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 93697.4us
I20260709 13:17:10.844307 10766 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:10.847946 13774 raft_consensus.cc:3055] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:10.760680 13725 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:10.854872 13666 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:10.856490 13677 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:10.857221 13657 raft_consensus.cc:3055] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:10.843878 13788 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:10.862046 11510 raft_consensus.cc:1240] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 8. Current term is 9. Ops: [6.6-8.7]
I20260709 13:17:10.842149 13620 raft_consensus.cc:697] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:10.861784 13772 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:10.863436 13786 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:10.863951 10872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 1597) took 1463 ms. Trace:
I20260709 13:17:10.865540 10795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2000) took 1421 ms. Trace:
I20260709 13:17:10.832788 13773 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:10.799068 13409 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:10.866165 13794 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:10.884794 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:10.758392 13778 consensus_queue.cc:1059] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.118s
I20260709 13:17:10.882496 13798 raft_consensus.cc:2804] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:10.931818 13798 raft_consensus.cc:697] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 7 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:10.932241 13409 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:10.932751 13798 consensus_queue.cc:237] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:10.890656 13826 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:10.913414 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:10.936985 10943 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:10.891494 13791 raft_consensus.cc:697] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:17:10.916090 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:10.886222 13633 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:10.885998 13718 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:10.961671 13791 consensus_queue.cc:237] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.2, Last appended by leader: 2, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:10.879694 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:10.835215 13717 consensus_queue.cc:1048] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 13:17:10.881466 13777 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:10.922336 10946 raft_consensus.cc:2376] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 9. Re-sending same reply.
I20260709 13:17:10.908353 13771 raft_consensus.cc:697] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:10.917732 13677 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:10.925917 13794 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:10.927165 13773 raft_consensus.cc:697] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:10.979676 13773 consensus_queue.cc:237] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:10.927677 13429 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:10.983171 13429 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:10.986078 13429 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:10.928313 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:10.987586 10949 raft_consensus.cc:2393] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 8: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:10.931058 13424 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:10.835740 13775 consensus_queue.cc:260] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:10.933295 13760 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:10.988485 13424 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:10.988740 13775 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:10.990171 13760 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:10.908121 13774 raft_consensus.cc:740] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:10.991142 13620 consensus_queue.cc:237] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:10.991437 13774 consensus_queue.cc:260] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:10.992893 13774 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:10.942606 13413 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:10.963299 13759 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:10.995505 13771 consensus_queue.cc:237] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:10.997337 13769 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:10.997491 13777 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:11.004341 11506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56404 (request call id 569) took 1169 ms. Trace:
I20260709 13:17:11.004912 11506 rpcz_store.cc:276] 0709 13:17:09.834971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.835082 (+   111us) service_pool.cc:224] Handling call
0709 13:17:11.004061 (+1168979us) spinlock_profiling.cc:243] Waited 35.8 ms on lock 0x7b5400527954. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:11.004258 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":82279296}
I20260709 13:17:11.019479 13614 raft_consensus.cc:3055] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:11.020085 13614 raft_consensus.cc:740] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:11.021108 13614 consensus_queue.cc:260] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:11.025651 13614 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:11.027320 13350 raft_consensus.cc:3055] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:11.027861 13350 raft_consensus.cc:740] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:11.028921 13350 consensus_queue.cc:260] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:11.032887 13809 consensus_queue.cc:1059] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:17:11.034332 13809 raft_consensus.cc:3055] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:11.034801 13809 raft_consensus.cc:740] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:10.945300 10941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 373) took 2727 ms. Trace:
I20260709 13:17:10.915905 13657 raft_consensus.cc:740] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:11.039559 13678 raft_consensus.cc:3055] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:11.040228 13678 raft_consensus.cc:740] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:11.040449 13657 consensus_queue.cc:260] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:11.042166 13657 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:10.951603 13743 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:10.946687 11507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56404 (request call id 571) took 1111 ms. Trace:
I20260709 13:17:11.042835 11507 rpcz_store.cc:276] 0709 13:17:09.835605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.835742 (+   137us) service_pool.cc:224] Handling call
0709 13:17:10.946421 (+1110679us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b54005014d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:10.946608 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":320254720}
W20260709 13:17:10.957012 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:11.043944 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:11.044149 13743 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:10.917249 13588 raft_consensus.cc:2749] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:10.888553 13779 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:11.045807 13779 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:11.046056 13634 raft_consensus.cc:3055] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:11.046686 13634 raft_consensus.cc:740] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:10.917582 13788 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:10.955932 13826 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:10.961328 13719 raft_consensus.cc:3055] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:10.996040 13772 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:11.041455 13678 consensus_queue.cc:260] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:10.944031 13828 raft_consensus.cc:3055] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:11.060941 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:10.963222 13633 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:11.077901 13779 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:11.072218 13350 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:11.078184 13666 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:11.085369 13719 raft_consensus.cc:740] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:11.090597 13828 raft_consensus.cc:740] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:11.105528 13677 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:11.109851 13678 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:11.035888 13809 consensus_queue.cc:260] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:11.122999 13777 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:11.126693 13788 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:11.131163 13719 consensus_queue.cc:260] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:11.132254 13719 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:11.132323 13788 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:11.136051 13809 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:11.136680 13793 consensus_meta.cc:287] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.505s	user 0.010s	sys 0.001s
I20260709 13:17:11.137331 13828 consensus_queue.cc:260] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:11.140954 13828 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:11.134684 13776 consensus_meta.cc:287] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.573s	user 0.011s	sys 0.004s
I20260709 13:17:11.311348 13772 consensus_queue.cc:1048] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:11.312302 13776 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:10.913666 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:11.323309 13777 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:11.323786 10800 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 5.
I20260709 13:17:11.324324 13777 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:11.107033 13634 consensus_queue.cc:260] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:11.326220 10800 raft_consensus.cc:1275] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:17:11.327594 13777 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:11.327822 13776 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:11.191211  9682 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:17:10.916391 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 71720.8us
I20260709 13:17:11.331053 10766 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:11.196699 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:11.198083 10793 raft_consensus.cc:1240] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:17:10.917727 10795 rpcz_store.cc:276] 0709 13:17:09.437012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:10.857874 (+1420862us) service_pool.cc:224] Handling call
0709 13:17:10.858230 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:10.858424 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:10.858454 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:10.858498 (+    44us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:10.858536 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:10.858592 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:10.858722 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:11.335711 10793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 318) took 1397 ms. Trace:
I20260709 13:17:11.336463 13666 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:11.336866 10795 raft_consensus.cc:1240] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:17:11.337357 13666 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:11.337765 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:10.938841 10801 raft_consensus.cc:1240] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:11.340219 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:11.341337 10874 raft_consensus.cc:2393] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 7: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:11.341243 10801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 313) took 1451 ms. Trace:
I20260709 13:17:11.341840 10801 rpcz_store.cc:276] 0709 13:17:09.890014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:10.915423 (+1025409us) service_pool.cc:224] Handling call
0709 13:17:10.938599 (+ 23176us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:11.340659 (+402060us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:11.341020 (+   361us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:11.199541 10794 raft_consensus.cc:1240] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:17:11.342710 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:11.343030 13639 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:11.343313 10794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 319) took 1403 ms. Trace:
I20260709 13:17:11.343739 10794 rpcz_store.cc:276] 0709 13:17:09.940064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:11.197951 (+1257887us) service_pool.cc:224] Handling call
0709 13:17:11.198541 (+   590us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:11.343026 (+144485us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:11.343254 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:11.343802 13639 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:11.336179 10793 rpcz_store.cc:276] 0709 13:17:09.938428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:11.196621 (+1258193us) service_pool.cc:224] Handling call
0709 13:17:11.197480 (+   859us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:11.335292 (+137812us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:11.335591 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:11.345907 10793 raft_consensus.cc:1240] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:17:11.346362 13639 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:11.207360 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:11.209111 10797 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 1597) took 1822 ms (client timeout 2027 ms). Trace:
I20260709 13:17:11.212067 10798 raft_consensus.cc:1240] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 13:17:10.956218 10870 raft_consensus.cc:2393] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 7: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:10.917604 10872 rpcz_store.cc:276] 0709 13:17:09.400447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.406080 (+  5633us) service_pool.cc:224] Handling call
0709 13:17:10.863721 (+1457641us) spinlock_profiling.cc:243] Waited 708 ms on lock 0x7b540006ccd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:10.863887 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1627870592}
I20260709 13:17:11.376485 13777 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:11.377282 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:11.377156 13777 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:11.377453 13710 consensus_queue.cc:1059] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:11.214373 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:11.378613 10871 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:11.379329 13710 raft_consensus.cc:3055] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:11.379076 13424 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:11.379791 13710 raft_consensus.cc:740] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:11.214931 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1547) took 1812 ms. Trace:
I20260709 13:17:11.380944 10869 rpcz_store.cc:276] 0709 13:17:09.402667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.759127 (+356460us) service_pool.cc:224] Handling call
0709 13:17:09.879825 (+120698us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:10.886931 (+1007106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:11.213617 (+326686us) spinlock_profiling.cc:243] Waited 961 ms on lock 0x7b5400076a54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:11.214410 (+   793us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b5400076a50. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:11.214668 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2488234624}
I20260709 13:17:11.381589 13424 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:11.216572 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:11.223919 10873 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 1546) took 1821 ms (client timeout 1809 ms). Trace:
W20260709 13:17:11.386988 10873 rpcz_store.cc:269] 0709 13:17:09.402445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.490604 (+ 88159us) service_pool.cc:224] Handling call
0709 13:17:11.223318 (+1732714us) spinlock_profiling.cc:243] Waited 641 ms on lock 0x7b540006bdd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:11.223656 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1474342784}
I20260709 13:17:11.061779 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:11.389272 10942 raft_consensus.cc:2393] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 6: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:10.957111 10943 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 7.
I20260709 13:17:11.063725 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:11.403690 10870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 1558) took 1203 ms. Trace:
I20260709 13:17:11.404201 10870 rpcz_store.cc:276] 0709 13:17:10.200404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:10.732279 (+531875us) service_pool.cc:224] Handling call
0709 13:17:11.403338 (+671059us) spinlock_profiling.cc:243] Waited 223 ms on lock 0x7b5400076054. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:11.403518 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":512561024}
W20260709 13:17:11.227917 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:11.039305 10941 rpcz_store.cc:276] 0709 13:17:08.217497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.321135 (+1103638us) service_pool.cc:224] Handling call
0709 13:17:09.321401 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:09.767752 (+446351us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:10.944975 (+1177223us) spinlock_profiling.cc:243] Waited 413 ms on lock 0x7b5400018bd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:10.945224 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":950401408}
I20260709 13:17:11.078022 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:11.077159 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:17:11.412222 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:11.413204 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:11.091285 11509 raft_consensus.cc:2393] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 8: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:11.091256 11512 raft_consensus.cc:3055] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:11.375804 13634 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:11.414489 11512 raft_consensus.cc:740] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:11.415187 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:11.415768 11512 consensus_queue.cc:260] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:11.417158 11512 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:11.232730 10796 raft_consensus.cc:1240] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:17:11.419588 10796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2011) took 1111 ms. Trace:
I20260709 13:17:11.420089 10796 rpcz_store.cc:276] 0709 13:17:10.308433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:11.231527 (+923094us) service_pool.cc:224] Handling call
0709 13:17:11.232147 (+   620us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:11.419119 (+186972us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:11.419494 (+   375us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:11.421458 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:11.422237 10796 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 8.
W20260709 13:17:11.240747 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:11.057300 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:11.427779 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:11.242501 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:11.431177 11506 raft_consensus.cc:2393] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 8: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:11.248833 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:11.435804 11511 raft_consensus.cc:1240] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 10 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:17:11.251484 11510 raft_consensus.cc:1240] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:11.246228 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:11.437712 11508 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 6: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:11.252933 11505 raft_consensus.cc:1240] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 6 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 5. Current term is 6. Ops: []
W20260709 13:17:11.256426 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:11.435835 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:11.447486 10796 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 5.
W20260709 13:17:11.144286 13804 consensus_meta.cc:287] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.577s	user 0.009s	sys 0.005s
I20260709 13:17:11.450577 13804 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:11.452392 10871 raft_consensus.cc:2410] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:17:11.452986 13804 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:11.307806 13493 consensus_queue.cc:1059] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:11.285643 10698 catalog_manager.cc:5697] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 3 to 4, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 4 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
W20260709 13:17:11.204398 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:11.454169 13493 raft_consensus.cc:3055] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:11.347759 10793 raft_consensus.cc:1275] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:17:11.454583 13493 raft_consensus.cc:740] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:11.455753 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:11.455672 13493 consensus_queue.cc:260] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:11.457041 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:11.457279 13493 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:11.458166 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:11.343835 10876 raft_consensus.cc:2393] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 6: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:11.352464 10766 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:11.459300 10867 raft_consensus.cc:2393] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 6: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:17:11.459903 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 107695us
I20260709 13:17:11.460808 13588 raft_consensus.cc:2804] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:11.461524 13588 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:11.461995 13588 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:11.353021 10801 raft_consensus.cc:1275] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:17:11.354614 11478 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:17:11.463352 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:11.463263 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 109114us
I20260709 13:17:11.355206 13779 raft_consensus.cc:2749] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:11.359457 10798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2010) took 1155 ms. Trace:
W20260709 13:17:11.358464 10797 rpcz_store.cc:269] 0709 13:17:09.386720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.391748 (+  5028us) service_pool.cc:224] Handling call
0709 13:17:11.208431 (+1816683us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b54000e9054. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:11.208810 (+   379us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3145532672}
I20260709 13:17:11.375088 10792 raft_consensus.cc:2376] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 8. Re-sending same reply.
I20260709 13:17:11.375756 10767 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:11.259827 13274 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:11.442152 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:17:11.454632 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:11.464787 10798 rpcz_store.cc:276] 0709 13:17:10.203968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:11.210692 (+1006724us) service_pool.cc:224] Handling call
0709 13:17:11.211412 (+   720us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:11.358619 (+147207us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:11.359390 (+   771us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:11.475263 13274 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:11.479619 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:11.480686 10792 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 8.
I20260709 13:17:11.480983 13666 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:11.483335 13793 raft_consensus.cc:2804] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:11.483768 13793 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:11.484221 13793 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:11.484748 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:11.486212 10798 raft_consensus.cc:1275] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:17:11.487666 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 112151us
I20260709 13:17:11.487901 10798 raft_consensus.cc:1275] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:17:11.488659 10767 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:11.489310 10798 raft_consensus.cc:1275] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:17:11.490317 10767 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:11.462280 10766 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:11.483157 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:11.484505 10876 raft_consensus.cc:1275] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:17:11.484432 10915 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:11.495170 11505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56404 (request call id 552) took 2097 ms. Trace:
I20260709 13:17:11.508492 11505 rpcz_store.cc:276] 0709 13:17:09.398015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.398895 (+   880us) service_pool.cc:224] Handling call
0709 13:17:10.947258 (+1548363us) spinlock_profiling.cc:243] Waited 9.25 ms on lock 0x7b5800003c98. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd90a22 00007f9c86ca9c09 00007f9c85e03ee0 00007f9c85ecd0ab 00007f9c85ecd131 00007f9c85e8efc0 00007f9c85e79929 00007f9c85e793d2 00007f9c7f3f0bd4
0709 13:17:11.252412 (+305154us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:11.439016 (+186604us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:11.494920 (+ 55904us) spinlock_profiling.cc:243] Waited 305 ms on lock 0x7b54005e7c50. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:11.495092 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":722065408}
I20260709 13:17:11.509415 11512 raft_consensus.cc:1275] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:17:11.491556 10793 raft_consensus.cc:1275] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:17:11.491359 13421 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:11.497901 11510 raft_consensus.cc:1275] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:17:11.498124 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:11.490752 10798 raft_consensus.cc:1240] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
W20260709 13:17:11.492000 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:11.525095 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:11.524668 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:11.527084 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:11.478576 13274 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:11.529603 10875 raft_consensus.cc:1240] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:11.491000 13639 raft_consensus.cc:2749] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:11.534034 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:17:11.544622 13629 consensus_meta.cc:287] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.979s	user 0.019s	sys 0.003s
I20260709 13:17:11.544853 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:11.545596 10875 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:11.546277 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:11.525811 13777 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:11.555835 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:11.559720 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:11.561260 10873 raft_consensus.cc:2393] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 7: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:11.562985 13421 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:11.567610 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 83466.5us
I20260709 13:17:11.569526 10915 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:11.570642 13781 raft_consensus.cc:2749] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:17:11.570942 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 80762.6us
W20260709 13:17:11.572229 13657 consensus_meta.cc:287] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.527s	user 0.011s	sys 0.004s
I20260709 13:17:11.493579 13826 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:11.545490 13771 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:11.581851 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:11.586308 10870 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:11.575044 13777 raft_consensus.cc:2749] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:11.571033 13629 raft_consensus.cc:2749] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:11.493430 13710 consensus_queue.cc:260] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:11.600842 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:11.601013 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 45043.5us
W20260709 13:17:11.601513 13774 consensus_meta.cc:287] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.545s	user 0.019s	sys 0.013s
I20260709 13:17:11.639691 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:11.594944 13771 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:11.602867 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:11.603173 10841 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.732s	user 0.008s	sys 0.021s
I20260709 13:17:11.604029 10874 raft_consensus.cc:1275] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:17:11.605464 11504 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 8.
I20260709 13:17:11.607095 11477 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:17:11.610268 13614 consensus_meta.cc:287] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.584s	user 0.008s	sys 0.008s
I20260709 13:17:11.612088 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:11.612324 13826 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:11.613555 10697 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42984 (request call id 244) took 19536 ms (client timeout 15000 ms). Trace:
W20260709 13:17:11.656134 11504 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56404 (request call id 568) took 1825 ms (client timeout 1813 ms). Trace:
I20260709 13:17:11.614259 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:11.615278 10799 raft_consensus.cc:1240] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:17:11.615669 10869 raft_consensus.cc:1275] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:17:11.617182 10870 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 8.
I20260709 13:17:11.662071 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:11.662963 10870 raft_consensus.cc:2393] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 8: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:11.664146 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:11.665112 10870 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 6: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:11.666425 10870 raft_consensus.cc:1275] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:17:11.617712 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:11.619110 10698 catalog_manager.cc:5697] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 3 to 7, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 7 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:17:11.620795 10867 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 1554) took 1421 ms (client timeout 1556 ms). Trace:
W20260709 13:17:11.667130 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 190046us
W20260709 13:17:11.660802 11504 rpcz_store.cc:269] 0709 13:17:09.830724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.830902 (+   178us) service_pool.cc:224] Handling call
0709 13:17:11.655812 (+1824910us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b54006202d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:11.656018 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3706127616}
W20260709 13:17:11.622320 13633 consensus_meta.cc:287] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.518s	user 0.012s	sys 0.000s
W20260709 13:17:11.642419 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 59653.2us
I20260709 13:17:11.572932 13421 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:11.648146 13534 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:11.651130 13639 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:11.651943 13725 consensus_meta.cc:287] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.731s	user 0.011s	sys 0.005s
W20260709 13:17:11.652390 10841 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.880s	user 0.009s	sys 0.021s
I20260709 13:17:11.652848 13781 consensus_queue.cc:1059] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:11.654317 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:11.659724 13719 consensus_meta.cc:287] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.527s	user 0.002s	sys 0.011s
I20260709 13:17:11.656281 11503 raft_consensus.cc:2410] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:17:11.656388 13798 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:17:11.656766 13718 consensus_meta.cc:287] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.636s	user 0.016s	sys 0.000s
I20260709 13:17:11.657558 10871 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 9.
I20260709 13:17:11.659077 13771 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:11.660344 10767 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:11.667335 10867 rpcz_store.cc:269] 0709 13:17:10.199422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:10.306585 (+107163us) service_pool.cc:224] Handling call
0709 13:17:11.620526 (+1313941us) spinlock_profiling.cc:243] Waited 966 ms on lock 0x7b54000f78d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:11.620732 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2220692864}
I20260709 13:17:11.672441 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:11.655608 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 48675.5us
I20260709 13:17:11.666671 13710 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:11.679323 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:11.659669 13629 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:11.698485 10874 raft_consensus.cc:1240] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:11.698450 10870 raft_consensus.cc:1275] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:17:11.704020 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:11.680332 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 946015us
I20260709 13:17:11.723781 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:11.697391 13822 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:11.674917 13534 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:11.643086 13793 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:11.732198 10915 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:11.734721 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:11.698570 13725 raft_consensus.cc:2749] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:11.742274 10869 raft_consensus.cc:1275] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:17:11.743737 10799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2009) took 1534 ms. Trace:
I20260709 13:17:11.746330 10793 raft_consensus.cc:1275] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:17:11.698173 13798 raft_consensus.cc:697] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 9 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:11.679522 13633 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:11.749106 10798 raft_consensus.cc:2393] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 7: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:11.744323 10799 rpcz_store.cc:276] 0709 13:17:10.203728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:11.202091 (+998363us) service_pool.cc:224] Handling call
0709 13:17:11.202762 (+   671us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:11.667836 (+465074us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:11.738320 (+ 70484us) spinlock_profiling.cc:243] Waited 412 ms on lock 0x7b540015a4d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:11.738533 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":947718912}
I20260709 13:17:11.701551 13718 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:11.760285 10876 raft_consensus.cc:1275] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 13:17:11.763257 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 301058us
I20260709 13:17:11.764843 10874 raft_consensus.cc:1240] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 5. Current term is 7. Ops: []
I20260709 13:17:11.724104 13639 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:11.767395 10793 raft_consensus.cc:1275] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 13:17:11.769356 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:11.753127 13781 raft_consensus.cc:2749] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:17:11.755189 10698 catalog_manager.cc:5697] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 2 to 7, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 7 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
W20260709 13:17:11.772509 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 185722us
I20260709 13:17:11.777830 10872 raft_consensus.cc:1240] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:11.780179 10766 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:11.782284 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:11.786868 13773 consensus_queue.cc:1048] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.094s
I20260709 13:17:11.788542 10950 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 9.
I20260709 13:17:11.788829 13620 raft_consensus.cc:2749] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:11.789860 13620 consensus_queue.cc:1059] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:11.790686 13620 consensus_queue.cc:1243] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer 12682eb70bc84c76879e04a8628ee586 log is divergent from this leader: its last log entry 7.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:17:11.791275 13620 raft_consensus.cc:2749] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:11.791990 13620 consensus_queue.cc:1059] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:11.792752 13620 consensus_queue.cc:1243] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer 12682eb70bc84c76879e04a8628ee586 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:17:11.795768 10867 raft_consensus.cc:1240] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 LEADER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:17:11.796506 13826 raft_consensus.cc:3055] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:11.796999 13826 raft_consensus.cc:740] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:11.798126 10870 raft_consensus.cc:1240] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:11.798022 13826 consensus_queue.cc:260] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:11.799363 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:11.801180 13798 consensus_queue.cc:237] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:11.802160 13777 raft_consensus.cc:3055] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:11.802328 11477 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:11.754391 13822 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:11.804644 13633 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:11.751304 13629 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:11.806823 13793 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:11.769073 13850 consensus_queue.cc:1048] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.076s
I20260709 13:17:11.810577 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:11.822405 13828 consensus_queue.cc:1059] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:17:11.812999 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:11.660506 10697 rpcz_store.cc:269] 0709 13:16:52.087015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:52.116419 (+ 29404us) service_pool.cc:224] Handling call
0709 13:17:11.613298 (+19496879us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:04.577969 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:04.578629 (+   660us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:04.598683 (+ 20054us) write_op.cc:435] Acquired schema lock
0709 13:17:04.611332 (+ 12649us) tablet.cc:662] Decoding operations
0709 13:17:04.866618 (+255286us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:04.913188 (+ 46570us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:04.942306 (+ 29118us) tablet.cc:685] Acquiring locks for 9 operations
0709 13:17:05.681919 (+739613us) tablet.cc:701] Row locks acquired
0709 13:17:05.723441 (+ 41522us) write_op.cc:260] PREPARE: finished
0709 13:17:05.734366 (+ 10925us) write_op.cc:270] Start()
0709 13:17:05.734575 (+   209us) write_op.cc:276] Timestamp: P: 1783603025732357 usec, L: 0
0709 13:17:05.734600 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:17:05.735523 (+   923us) log.cc:844] Serialized 3622 byte log entry
0709 13:17:06.294481 (+558958us) op_driver.cc:464] REPLICATION: finished
0709 13:17:08.162396 (+1867915us) spinlock_profiling.cc:243] Waited 2.3 ms on lock 0x7b180000e800. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc83f3 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:17:08.162558 (+   162us) write_op.cc:301] APPLY: starting
0709 13:17:08.162645 (+    87us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:08.162895 (+   250us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:08.162925 (+    30us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:08.164068 (+  1143us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:08.164397 (+   329us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=9
0709 13:17:08.164445 (+    48us) write_op.cc:312] APPLY: finished
0709 13:17:08.164799 (+   354us) log.cc:844] Serialized 71 byte log entry
0709 13:17:08.400058 (+235259us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:08.458054 (+ 57996us) write_op.cc:454] Released schema lock
0709 13:17:08.509022 (+ 50968us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:08.570151 (+ 61129us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:08.570184 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":23233,"reported_tablets":43,"child_traces":[["op",{"apply.queue_time_us":875571,"apply.run_cpu_time_us":75631,"apply.run_wall_time_us":407547,"num_ops":9,"prepare.queue_time_us":142643,"prepare.run_cpu_time_us":12997,"prepare.run_wall_time_us":1374539,"raft.queue_time_us":278181,"raft.run_cpu_time_us":12198,"raft.run_wall_time_us":494604,"replication_time_us":531837,"spinlock_wait_cycles":5283072,"thread_start_us":880549,"threads_started":5,"wal-append.queue_time_us":782162,"wal-append.run_cpu_time_us":44860,"wal-append.run_wall_time_us":2709810}]]}
I20260709 13:17:11.804709 13350 raft_consensus.cc:3055] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:11.823138 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:11.813828 13439 consensus_queue.cc:1048] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.010s
W20260709 13:17:11.817677 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:11.820026 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:17:11.847004 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:11.847307 10946 raft_consensus.cc:2368] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 7. Current term is 8.
W20260709 13:17:11.847688 11478 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.116.130:37895 timed out after 1.789s (SENT) [suppressed 57 similar messages]
I20260709 13:17:11.847985 10874 raft_consensus.cc:1240] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 LEADER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:11.836464 13777 raft_consensus.cc:740] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:11.824482 13588 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:11.786912 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:11.787302 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 91682.4us
I20260709 13:17:11.814972 10794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1600) took 1252 ms. Trace:
I20260709 13:17:11.849159 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:11.833180 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:11.817468 10767 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:11.835047 10875 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 7.
I20260709 13:17:11.836341 10871 raft_consensus.cc:1240] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:11.812515 10916 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:11.841572 10799 raft_consensus.cc:1275] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:17:11.820091 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:11.818760 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:11.853322 10873 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 9.
I20260709 13:17:11.850868 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:17:11.857131 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 79828.8us
I20260709 13:17:11.816568 13718 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:11.859179 10870 raft_consensus.cc:1275] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:17:11.859886 10874 raft_consensus.cc:2393] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 9: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:11.816592 13534 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:11.853470 10867 raft_consensus.cc:2393] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 8: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:11.865343 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:11.821611 13639 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:11.852592 10794 rpcz_store.cc:276] 0709 13:17:10.562814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:11.344645 (+781831us) service_pool.cc:224] Handling call
0709 13:17:11.345171 (+   526us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:11.345706 (+   535us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:11.345735 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:11.346325 (+   590us) log.cc:844] Serialized 26 byte log entry
0709 13:17:11.529552 (+183227us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:11.529596 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:11.529661 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:11.814723 (+285062us) raft_consensus.cc:1685] finished
0709 13:17:11.814741 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:11.814908 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1452,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":191,"thread_start_us":182065,"threads_started":1,"wal-append.queue_time_us":422734}
I20260709 13:17:11.819440 13725 consensus_queue.cc:1059] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 13:17:11.863957 10876 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 5.
I20260709 13:17:11.860845 10698 catalog_manager.cc:5697] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 5 to 6, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 6 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:11.849817 13350 raft_consensus.cc:740] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:17:11.891652 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 79286.6us
I20260709 13:17:11.893831 10801 raft_consensus.cc:2393] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 7: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:11.895573 10942 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:11.890121 10868 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 7.
I20260709 13:17:11.856648 13633 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:11.898154 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:11.899746 10949 raft_consensus.cc:2368] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 6. Current term is 7.
I20260709 13:17:11.902263 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:11.902498 13588 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:11.902938 10794 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:11.912992 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 95605.7us
I20260709 13:17:11.895735 13634 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:11.915287 10698 catalog_manager.cc:5697] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 4 to 5, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 5 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:11.917033 10793 raft_consensus.cc:1275] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:17:11.849375 13777 consensus_queue.cc:260] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:11.923513 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:11.924286 10798 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:17:11.926429 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:11.927078 10868 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:11.936599 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:11.938406 10942 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 6.
I20260709 13:17:11.939409 10842 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:11.940027 10798 raft_consensus.cc:1275] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
W20260709 13:17:11.941862 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 159983us
W20260709 13:17:12.013773 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:12.016928 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 72718.1us
I20260709 13:17:11.977257 11507 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 9.
I20260709 13:17:11.925693 13718 consensus_queue.cc:1059] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.094s
W20260709 13:17:12.023350 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 116269us
W20260709 13:17:12.017217 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:12.026213 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:12.029493 13666 raft_consensus.cc:2749] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:12.029960 13861 raft_consensus.cc:2749] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:17:12.030601 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:12.031492 10801 raft_consensus.cc:1275] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:17:11.936149 13634 raft_consensus.cc:697] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 7 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:12.041249 10946 raft_consensus.cc:1275] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 3. (term mismatch)
I20260709 13:17:12.035140 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:11.945837 13781 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:11.946529 13826 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:12.029520 10698 catalog_manager.cc:5697] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 1 to 5, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 5 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:12.022811 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:12.021514 13493 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:12.013059 13588 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:12.042666 10946 pending_rounds.cc:85] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 2
I20260709 13:17:12.017588 13779 raft_consensus.cc:2749] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:11.942899 13725 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:11.941639 13633 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:12.051139 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:12.059934 10698 catalog_manager.cc:5697] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:12.041683 13828 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:12.056648 10916 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:12.053750 10946 pending_rounds.cc:107] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:17:12.064271 10946 raft_consensus.cc:2889] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:12.064648 10946 pending_rounds.cc:107] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:17:12.064956 10946 raft_consensus.cc:2889] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:12.045609 13781 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:12.065369 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:12.064759 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:12.067577 10876 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:12.054287 13493 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:12.068863 13493 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:12.069105 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:12.071027 13777 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:12.074636 13614 raft_consensus.cc:3055] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:12.075129 13614 raft_consensus.cc:740] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:12.076126 13614 consensus_queue.cc:260] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:12.077447 13614 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:12.047242 13439 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:17:12.079842 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:12.084674 13868 log.cc:927] Time spent T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.146s	user 0.000s	sys 0.000s
I20260709 13:17:12.085249 10795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1599) took 1522 ms. Trace:
I20260709 13:17:12.085646 10795 rpcz_store.cc:276] 0709 13:17:10.562454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:11.338233 (+775779us) service_pool.cc:224] Handling call
0709 13:17:11.351664 (+ 13431us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:11.356152 (+  4488us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:11.356187 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:11.356801 (+   614us) log.cc:844] Serialized 26 byte log entry
0709 13:17:11.719944 (+363143us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:11.719996 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:11.720120 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:12.085037 (+364917us) raft_consensus.cc:1685] finished
0709 13:17:12.085056 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:12.085192 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3608,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":130,"thread_start_us":362615,"threads_started":1,"wal-append.queue_time_us":543574}
I20260709 13:17:12.059307 13588 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:12.088124 13439 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:12.089457 13588 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:12.090361 13439 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:12.092643 11507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56404 (request call id 588) took 1031 ms. Trace:
I20260709 13:17:12.093035 11507 rpcz_store.cc:276] 0709 13:17:11.061560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:11.412645 (+351085us) service_pool.cc:224] Handling call
0709 13:17:12.092471 (+679826us) spinlock_profiling.cc:243] Waited 412 ms on lock 0x7b54005fee54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:12.092587 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":946518272}
I20260709 13:17:12.098400 13725 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:17:12.099109 11508 raft_consensus.cc:1275] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:17:12.099215 13828 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:12.100607 11508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56404 (request call id 592) took 1032 ms. Trace:
I20260709 13:17:12.100878 13725 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:12.081460 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:12.101425 13828 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:12.100956 11508 rpcz_store.cc:276] 0709 13:17:11.068385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:11.438352 (+369967us) service_pool.cc:224] Handling call
0709 13:17:12.098727 (+660375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:12.099584 (+   857us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:12.100412 (+   828us) spinlock_profiling.cc:243] Waited 655 ms on lock 0x7b54005fee50. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:12.100554 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1505961216}
I20260709 13:17:12.079286 13781 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:12.115665 10797 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7.
W20260709 13:17:12.140210 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:12.144243 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:12.144655 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:17:12.145102 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 221669us
I20260709 13:17:12.145587 11512 raft_consensus.cc:2368] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 7. Current term is 8.
I20260709 13:17:12.144366 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:12.146487 11509 raft_consensus.cc:2410] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:17:12.145121 11507 raft_consensus.cc:2368] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 5. Current term is 6.
I20260709 13:17:12.114473 13822 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:12.153012 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:12.152909 10698 catalog_manager.cc:5697] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 4 to 7, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 7 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
W20260709 13:17:12.154884 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:12.155311 11504 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:12.162740 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 126701us
I20260709 13:17:12.150295 13725 raft_consensus.cc:2804] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:12.166370 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:12.167484 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:12.173743 11508 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 7.
W20260709 13:17:12.170542 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 88700.8us
I20260709 13:17:12.145376 13629 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:12.178150 10794 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 6.
I20260709 13:17:12.175418 10698 catalog_manager.cc:5697] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 4 to 6, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 6 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:12.143169 13588 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:12.160764 13639 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:12.188482 11510 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:12.189772 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:12.172101 13725 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:12.193097 10766 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:12.186556 10698 catalog_manager.cc:5697] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 4 to 5, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 5 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:12.195955 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:12.171517 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:12.198825 13871 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:12.199697 10946 raft_consensus.cc:1240] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 8. Current term is 9. Ops: []
W20260709 13:17:12.200630 13863 log.cc:927] Time spent T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:17:12.207477 11504 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 7.
I20260709 13:17:12.218379 13874 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:17:12.208068 13421 consensus_queue.cc:1059] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 13:17:12.222560 10949 raft_consensus.cc:1240] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:17:12.223552 13633 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:12.222595 10943 raft_consensus.cc:1275] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 13:17:12.205089 13725 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:12.214632 13861 consensus_queue.cc:1048] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 13:17:12.224519 10943 pending_rounds.cc:85] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 5
I20260709 13:17:12.229254 10943 pending_rounds.cc:107] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:17:12.229651 10943 raft_consensus.cc:2889] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:12.231846 10766 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:12.233331 10766 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:12.233892 13274 consensus_queue.cc:1048] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:17:12.234964 13828 consensus_queue.cc:1048] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:17:12.235672 13274 consensus_queue.cc:1048] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:12.236900 11506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56404 (request call id 589) took 1175 ms. Trace:
I20260709 13:17:12.237349 11506 rpcz_store.cc:276] 0709 13:17:11.061810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:11.432327 (+370517us) service_pool.cc:224] Handling call
0709 13:17:11.433102 (+   775us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:11.533770 (+100668us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:11.535940 (+  2170us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:11.571647 (+ 35707us) log.cc:844] Serialized 26 byte log entry
0709 13:17:11.893098 (+321451us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:11.895284 (+  2186us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:11.900593 (+  5309us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:12.236654 (+336061us) raft_consensus.cc:1685] finished
0709 13:17:12.236684 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:12.236835 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":110527,"raft.queue_time_us":89870,"raft.run_cpu_time_us":232,"raft.run_wall_time_us":11269,"thread_start_us":290375,"threads_started":1,"wal-append.queue_time_us":491981}
I20260709 13:17:12.172224 13634 consensus_queue.cc:237] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:12.239159 10767 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:12.242393 13861 raft_consensus.cc:2749] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:12.243000 11503 raft_consensus.cc:2410] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:17:12.248556 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:12.249804 11508 raft_consensus.cc:1240] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:12.186988 13588 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:12.254266 10868 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 8.
I20260709 13:17:12.243139 13861 raft_consensus.cc:2749] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:17:12.254498 13710 consensus_meta.cc:287] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.543s	user 0.011s	sys 0.000s
I20260709 13:17:12.263434 10766 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:12.265318 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:12.252884 13874 raft_consensus.cc:2749] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:17:12.268963 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:12.256950 13657 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:12.265730 10698 catalog_manager.cc:5697] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 1 to 6, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 6 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:12.277055 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:17:12.279538 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 84760us
I20260709 13:17:12.224159 13639 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:17:12.290381 13829 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:12.290707 11503 raft_consensus.cc:1275] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:17:12.293200 13629 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:12.293694 13828 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:12.294423 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:12.322053 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:12.323292 13862 log.cc:927] Time spent T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.387s	user 0.004s	sys 0.000s
I20260709 13:17:12.295439 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:12.279666 13657 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:12.325142 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:12.295527 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:12.295198 10943 raft_consensus.cc:1275] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:17:12.326107 13657 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:12.326735 13657 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:12.326815 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:12.327119 10943 raft_consensus.cc:1240] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:12.327310 13657 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:12.295219 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:12.299345 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:12.328361 13875 log.cc:927] Time spent T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
I20260709 13:17:12.296087 13439 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:12.328905 10949 raft_consensus.cc:2410] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:17:12.328967 10948 raft_consensus.cc:2410] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 10 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:17:12.329118 13439 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:12.329921 13274 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:12.219458 13871 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:12.335405 13829 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:12.306139 13493 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:12.319566 10796 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7.
I20260709 13:17:12.320189 10946 raft_consensus.cc:2393] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 7: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:12.322279 13850 raft_consensus.cc:3055] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:12.323158 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:12.297183 10944 raft_consensus.cc:1240] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:12.299960 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:12.304768 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 247063us
I20260709 13:17:12.306056 13822 consensus_queue.cc:1048] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:12.330410 13274 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:12.287974 13710 raft_consensus.cc:2749] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:12.317610 11503 pending_rounds.cc:85] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586: Aborting all ops after (but not including) 6
I20260709 13:17:12.343755 10869 raft_consensus.cc:2393] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 10: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:12.346972 10916 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:12.353432 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:12.332734 10698 catalog_manager.cc:5697] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 3 to 4, leader changed from <none> to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131), now has a pending config: VOTER e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129), VOTER 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130), VOTER cb0428cc55984daba1ba63744a5102a2 (127.9.116.131), VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 4 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } } pending_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:17:12.358278 10950 raft_consensus.cc:1275] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:17:12.360989 10946 raft_consensus.cc:1275] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 8. (term mismatch)
W20260709 13:17:12.361039 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 97694.1us
I20260709 13:17:12.361590 10946 pending_rounds.cc:85] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 7
I20260709 13:17:12.372175 10946 pending_rounds.cc:107] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 13:17:12.372583 10946 raft_consensus.cc:2889] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:12.373296 10876 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 6.
W20260709 13:17:12.374686 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 159710us
I20260709 13:17:12.345930 13850 raft_consensus.cc:740] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:12.358348 11503 pending_rounds.cc:107] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:17:12.378566 10766 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:12.358366 13666 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:12.322094 13828 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:12.381016 10767 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:12.348071 13629 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:12.387362 10842 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:12.395982 10842 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:12.391687 10698 catalog_manager.cc:5697] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:12.381806 11503 raft_consensus.cc:2889] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:12.402055 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:12.397176  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:12.360386 13421 consensus_queue.cc:1059] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:17:12.382118 13822 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:12.405673 10945 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 5.
I20260709 13:17:12.383133 13666 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:12.407192 10766 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:12.389881 13439 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:12.411933 13880 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:12.412407 13880 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:12.412325 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:12.412827 13880 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:12.413014 10946 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:12.415553 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:12.411454 10698 catalog_manager.cc:5697] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 6 to 8, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 8 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:12.417258 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:12.401171 13710 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:17:12.397675 10841 connection.cc:256] RPC call timeout handler was delayed by -3.4438840589998563s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22260000000000002s for a response
I20260709 13:17:12.419653 13865 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:12.417528 13882 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:12.390729 13773 consensus_queue.cc:1048] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 13:17:12.422848 10941 raft_consensus.cc:1275] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:17:12.429293 10941 pending_rounds.cc:85] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 6
I20260709 13:17:12.429471 10941 pending_rounds.cc:107] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:17:12.429754 10941 raft_consensus.cc:2889] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:12.406247 13718 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:12.420030 13678 consensus_queue.cc:1048] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 13:17:12.414166 13781 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:12.425263 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:12.403394 13877 consensus_queue.cc:1048] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:17:12.420894 13588 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:12.440178 13829 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:12.431739 13657 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:17:12.443373 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:12.444566 10949 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 8.
I20260709 13:17:12.426312 13865 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:12.449133 10942 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7.
I20260709 13:17:12.435106 13850 consensus_queue.cc:260] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:12.429015 13710 raft_consensus.cc:697] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:12.466952 13421 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:12.467167 13710 consensus_queue.cc:237] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:12.467583 13421 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:12.470120 11478 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:12.450747 10946 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 8.
W20260709 13:17:12.471524 13873 log.cc:927] Time spent T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.281s	user 0.001s	sys 0.000s
I20260709 13:17:12.472139 11478 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:12.473333 13798 raft_consensus.cc:2749] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:17:12.474066 13798 consensus_queue.cc:1048] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:17:12.475503 13798 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:12.475613 13629 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:12.477118 13666 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:12.477995 13865 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:17:12.461021 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 98047.7us
W20260709 13:17:12.462710 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 55590.4us
I20260709 13:17:12.424212 13493 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:12.448925 13828 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:12.409461 13822 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:12.485122 10766 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:12.464695 13808 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:12.486152 10947 raft_consensus.cc:2393] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 9: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:12.487188 13808 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:12.504331 13808 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:12.507176 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:12.491379 10698 catalog_manager.cc:5697] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 2 to 4, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 4 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:12.507584 13822 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:12.508489 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:12.509227 13879 log.cc:927] Time spent T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.219s	user 0.001s	sys 0.000s
I20260709 13:17:12.509382 13666 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:12.508939 10698 catalog_manager.cc:5697] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 6 to 7, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 7 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:12.510030 10948 raft_consensus.cc:1275] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:17:12.511158 11511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56404 (request call id 591) took 1442 ms. Trace:
I20260709 13:17:12.511533 11511 rpcz_store.cc:276] 0709 13:17:11.068218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:11.437220 (+369002us) service_pool.cc:224] Handling call
0709 13:17:11.437764 (+   544us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:11.585528 (+147764us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:11.585560 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:11.673735 (+ 88175us) log.cc:844] Serialized 26 byte log entry
0709 13:17:11.918525 (+244790us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:11.918579 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:11.918679 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:12.505280 (+586601us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b24000d5158. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dc4e9c4 00007f9c7dc4cacb 00007f9c7de07dc9 00007f9c7de07a55 00007f9c7de06fb0 00007f9c80a7d488 00007f9c80a7dbe8 00007f9c80a55398 00007f9c80a5076a 00007f9c80a4fbb6
0709 13:17:12.510868 (+  5588us) raft_consensus.cc:1685] finished
0709 13:17:12.510890 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:12.511096 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":144990,"raft.queue_time_us":147122,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":18058,"spinlock_wait_cycles":31063936,"thread_start_us":220442,"threads_started":1,"wal-append.queue_time_us":499664}
I20260709 13:17:12.483305 13850 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:12.514226 13882 raft_consensus.cc:2749] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:12.515714 10950 raft_consensus.cc:1275] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 13:17:12.516124 10950 pending_rounds.cc:85] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 7
I20260709 13:17:12.516367 10950 pending_rounds.cc:107] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 13:17:12.516602 10950 raft_consensus.cc:2889] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:12.517757 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:12.518483 10944 raft_consensus.cc:2393] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 7: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:12.519707 10946 raft_consensus.cc:1275] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:17:12.520408 13871 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:12.522190 13781 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:12.519897 10916 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:12.522917 13781 consensus_queue.cc:1048] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:12.523442 13629 raft_consensus.cc:2749] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:12.535569 10943 raft_consensus.cc:1240] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:12.536803 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:12.538106 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:12.534078 13350 consensus_queue.cc:260] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:12.541785 13350 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:12.545135 13793 raft_consensus.cc:3055] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:12.545738 13793 raft_consensus.cc:740] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:12.547020 13793 consensus_queue.cc:260] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:12.558348 13793 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:12.550336 13808 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:12.551146 11477 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:12.565512 13634 raft_consensus.cc:2804] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:12.551469 13633 raft_consensus.cc:3055] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:12.566464 13633 raft_consensus.cc:740] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:12.567391 13633 consensus_queue.cc:260] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:12.568723 13633 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:12.569221 13871 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:12.569893 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:12.569763 13871 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:12.570462 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:12.571317 10794 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 8.
I20260709 13:17:12.554124 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:12.571969 13871 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:12.572114 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:12.555037 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:12.555945 10796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2015) took 1172 ms. Trace:
I20260709 13:17:12.566932 13274 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:12.553740 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:12.573130 10793 raft_consensus.cc:1275] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:17:12.556717 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:12.555836 13877 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:12.570533 10792 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 7.
I20260709 13:17:12.574072 10942 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 6.
I20260709 13:17:12.571486 10799 raft_consensus.cc:1275] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:17:12.557512 10766 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:12.572678 10798 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:12.574304 13877 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:12.575309 13871 raft_consensus.cc:2749] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:17:12.575987 13588 consensus_queue.cc:1059] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:12.576267 13871 consensus_queue.cc:1048] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:12.576395 10766 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:12.576889 13588 raft_consensus.cc:3055] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:12.577286 13588 raft_consensus.cc:740] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:12.577459 13779 raft_consensus.cc:2804] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:12.577971 13779 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:12.578359 13779 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:12.580265 13877 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:12.580874 10941 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 6.
I20260709 13:17:12.587745 10766 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:12.573016 10796 rpcz_store.cc:276] 0709 13:17:11.383758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:11.498563 (+114805us) service_pool.cc:224] Handling call
0709 13:17:12.555780 (+1057217us) spinlock_profiling.cc:243] Waited 740 ms on lock 0x7b5400386b54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:12.555888 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1703068928}
I20260709 13:17:12.626849 10798 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 6.
I20260709 13:17:12.487317 13639 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:12.643123 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 242315us
I20260709 13:17:12.643664 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:12.644351 10942 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 6.
I20260709 13:17:12.578202 13588 consensus_queue.cc:260] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:12.665616 10766 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:12.675997 10941 raft_consensus.cc:1275] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:17:12.627076 13620 raft_consensus.cc:2804] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:17:12.664232 13884 log.cc:927] Time spent T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
W20260709 13:17:12.675997 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 110923us
W20260709 13:17:12.681600 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:12.633471 13666 raft_consensus.cc:2749] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:12.677484 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:12.683917 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 211872us
I20260709 13:17:12.687847 10792 raft_consensus.cc:1275] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:17:12.688362 11477 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:12.525950 13865 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:12.669087 13777 consensus_meta.cc:287] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.598s	user 0.015s	sys 0.001s
I20260709 13:17:12.689891 10799 raft_consensus.cc:1275] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:17:12.693957 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:12.700038 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:12.700676 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:12.691826 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:12.680724 13620 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:12.687671 13350 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:12.710273 10793 raft_consensus.cc:2393] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 10: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:12.708912 13877 consensus_queue.cc:1048] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 13:17:12.711521 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:12.712798 13895 consensus_queue.cc:1048] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:17:12.713487 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:12.711203 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:12.710309 13620 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:12.694399 13634 raft_consensus.cc:697] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 6 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:12.707131 13889 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:12.723438 13889 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:12.723758 13634 consensus_queue.cc:237] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:12.725817 13889 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:12.705053 13666 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:12.719964 11478 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:12.726717 13666 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:12.728309 11478 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:17:12.728870 13666 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:12.691368 13639 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:12.739363 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:12.740425 10875 raft_consensus.cc:2393] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 7: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:12.698956 13886 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:12.745230 10876 raft_consensus.cc:1275] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:17:12.740998 11510 consensus_meta.cc:287] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.522s	user 0.012s	sys 0.002s
I20260709 13:17:12.747406 11510 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 7.
I20260709 13:17:12.751395 13886 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:12.753760 13886 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:12.754221 13620 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:12.756491 10868 raft_consensus.cc:2410] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
W20260709 13:17:12.758896 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:12.759024 13588 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:12.763337 13826 consensus_meta.cc:287] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.709s	user 0.010s	sys 0.004s
I20260709 13:17:12.763657 13808 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:12.765020 13865 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:12.765913 13882 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:12.766490 13882 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:12.771334 13882 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:12.737685 13350 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:12.775214 13350 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:12.762513 13798 raft_consensus.cc:2749] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:12.775144 13894 raft_consensus.cc:2804] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:12.776232 13894 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:12.776630 13894 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:12.778550 13798 raft_consensus.cc:2749] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:17:12.779419 13798 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:12.741204 13718 raft_consensus.cc:697] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:12.744345 13888 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:12.704056 13725 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:12.772708 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:12.785445 10948 raft_consensus.cc:1240] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:12.767632 10766 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:12.785946 10941 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 6.
I20260709 13:17:12.780668 13888 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:12.797044 10875 raft_consensus.cc:1240] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:17:12.798301 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:12.782446 13725 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:12.798549 13888 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:12.799810 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:12.714560 13895 consensus_queue.cc:1048] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:17:12.798966 10869 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 7.
W20260709 13:17:12.803573 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 371665us
W20260709 13:17:12.804108 10841 connection.cc:256] RPC call timeout handler was delayed by -2.9684303709998403s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3012s for a response
I20260709 13:17:12.805315 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:12.804363 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 116091us
I20260709 13:17:12.804719 10766 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:17:12.807864 10841 connection.cc:256] RPC call timeout handler was delayed by -2.1057161480000559s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1658s for a response
W20260709 13:17:12.808315 10841 connection.cc:256] RPC call timeout handler was delayed by -1.5279161480000312s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22999999999999998s for a response
I20260709 13:17:12.810200 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:12.811053 10876 raft_consensus.cc:2410] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 13:17:12.807489 10872 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 6.
W20260709 13:17:12.811257 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 83015.3us
W20260709 13:17:12.813601 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:12.813472 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:17:12.814721 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:12.809234 10841 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:12.825744 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:12.819962 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:12.826423 10870 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 6.
I20260709 13:17:12.780103 13798 raft_consensus.cc:697] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 8 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:12.828871 10874 raft_consensus.cc:2410] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 7 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 13:17:12.801659 13886 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:12.819149 13865 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:12.830147 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:12.811369 13666 raft_consensus.cc:2804] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:12.837078 10868 raft_consensus.cc:1275] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:17:12.838258 11478 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:12.840744 10947 raft_consensus.cc:2393] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 8: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:12.825582 13888 consensus_queue.cc:1048] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 13:17:12.812964 13877 consensus_queue.cc:1048] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:17:12.843380 13888 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:12.819514 13828 consensus_queue.cc:1048] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 13:17:12.846128 13880 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:12.837775 13666 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:12.848961 13880 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:12.849264 13666 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:12.850042 11478 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:17:12.850865 13634 raft_consensus.cc:2804] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:12.851320 13634 raft_consensus.cc:697] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 6 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:12.852188 13634 consensus_queue.cc:237] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:12.854276 10766 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:12.856304 13826 raft_consensus.cc:2749] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:12.858307 10766 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:12.858711 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:12.859211 13826 raft_consensus.cc:2804] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:12.859434 10868 raft_consensus.cc:2376] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 9. Re-sending same reply.
I20260709 13:17:12.859676 13826 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:12.860106 13826 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:12.862607 13421 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:12.864673 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:12.865451 10801 raft_consensus.cc:2393] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 7: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:12.813113 13882 consensus_queue.cc:1048] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 13:17:12.866770 10799 raft_consensus.cc:1275] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:17:12.867920 13890 raft_consensus.cc:2749] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:12.868880 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:12.869215 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:12.869544 10792 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 9.
I20260709 13:17:12.870091 10793 raft_consensus.cc:2410] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:17:12.870142 13779 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:12.871192 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:12.871326 10798 raft_consensus.cc:1240] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:12.872499 13779 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:12.868434 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:12.873405 10799 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 8.
I20260709 13:17:12.831779 10842 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:12.877491 10872 raft_consensus.cc:1240] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:12.875734 10766 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:12.884905 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:12.885725 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:12.888134 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:12.889457 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:12.894737 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:12.895735 10872 raft_consensus.cc:1240] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:17:12.895910 10766 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:12.781008 13718 consensus_queue.cc:237] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:12.835129 13865 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:12.832056 13639 raft_consensus.cc:2749] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:12.954548 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:12.899435 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:12.899317 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:12.931152 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:12.902151 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:12.904307 10870 raft_consensus.cc:1240] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:12.905771 10871 raft_consensus.cc:2410] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:17:12.906208 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:12.958456 10869 raft_consensus.cc:2410] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:17:12.955900 10868 raft_consensus.cc:2410] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
W20260709 13:17:12.891851 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 76956.7us
I20260709 13:17:12.908475 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:12.960129 10874 raft_consensus.cc:2393] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 8: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:12.915208 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:12.915436 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:12.916658 13588 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:12.924199 10766 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:12.916586 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:12.923161 10876 raft_consensus.cc:1240] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:12.921414 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:12.902433 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:12.923480 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:12.962581 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:12.969589 13777 raft_consensus.cc:2804] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:12.970409 13777 raft_consensus.cc:697] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:12.971645 13777 consensus_queue.cc:237] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:12.966609 10800 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 6.
I20260709 13:17:12.964738 10949 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 6.
I20260709 13:17:12.975281 10792 raft_consensus.cc:2393] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 6: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:12.979017 10798 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 9.
I20260709 13:17:12.983101 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:12.987215 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:12.988447 13895 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:12.904762 13890 raft_consensus.cc:2804] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:12.990692 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:12.997329 10767 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:12.997771 10870 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 8.
I20260709 13:17:12.992277 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:12.833143 13886 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:13.001785 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:13.002547 10876 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 6: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:13.003496 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:12.989809 10795 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 6.
I20260709 13:17:13.011250 10868 raft_consensus.cc:1275] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:17:13.013708 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:13.014722 11511 raft_consensus.cc:2410] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:17:12.954171 13882 raft_consensus.cc:2749] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:17:13.017234 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 208167us
I20260709 13:17:13.013780 13874 raft_consensus.cc:2749] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:12.974041 13865 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:12.962154 13620 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:13.050519 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:13.051214 10874 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:13.023670 13897 log.cc:927] Time spent T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:17:13.027091 10792 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 9.
I20260709 13:17:13.006376 13772 consensus_queue.cc:1048] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
W20260709 13:17:13.035790 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:13.036402 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:13.059013 10943 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:13.059392 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:13.060185 10800 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 6.
I20260709 13:17:13.023216 13896 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:12.991276 13588 raft_consensus.cc:697] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:12.989022 13895 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:13.054900 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:13.055516 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:13.055740 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:13.062364 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:12.973881 13808 consensus_queue.cc:1048] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.070s
I20260709 13:17:13.065289 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:13.068806 10792 raft_consensus.cc:2410] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:17:13.015240 13889 consensus_queue.cc:1059] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 13:17:13.073252 10876 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 6.
I20260709 13:17:13.042379 13890 raft_consensus.cc:697] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:13.073524 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:13.076761 10873 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 9.
I20260709 13:17:13.077845 10868 raft_consensus.cc:3055] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:13.078372 10868 raft_consensus.cc:740] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:13.083284 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:13.084076 10876 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 6.
I20260709 13:17:13.088462 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:13.089138 10792 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:13.093268 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:13.094188 10795 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:13.095885 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:13.096825 10876 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 9.
I20260709 13:17:13.097796 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:13.098635 10800 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:17:13.088771 13633 consensus_meta.cc:287] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.516s	user 0.009s	sys 0.007s
I20260709 13:17:13.073388 13894 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:13.042685 13798 consensus_queue.cc:237] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:13.088333 13828 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:13.104422 10868 consensus_queue.cc:260] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:13.133298 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:13.068105 13896 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:13.147828 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:13.157305 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:17:13.156665 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 125486us
I20260709 13:17:13.164275 11478 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:13.166924 10868 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:13.187134 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:13.188186 10876 raft_consensus.cc:2410] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:17:13.189317 10943 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 6.
I20260709 13:17:13.189944 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:13.196385 13826 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:13.197642 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:13.198295 10870 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 9.
I20260709 13:17:13.198360 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:13.199554 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:17:13.210381 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:13.213397 13439 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:13.214224 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:13.214813 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:13.214426 13439 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:13.218688 10871 raft_consensus.cc:1240] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 LEADER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 5. Current term is 6. Ops: [5.8-5.8]
I20260709 13:17:13.219695 10792 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
I20260709 13:17:13.220299 10868 raft_consensus.cc:1275] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:17:13.220564 13893 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:13.224512 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:13.225471 10876 raft_consensus.cc:2410] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 9 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:17:13.140724 13828 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:13.135332 13588 consensus_queue.cc:237] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:13.232223 10792 raft_consensus.cc:1240] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 7. Current term is 8. Ops: [5.5-7.6]
W20260709 13:17:13.243142 13900 log.cc:927] Time spent T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.185s	user 0.001s	sys 0.000s
I20260709 13:17:13.248224 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:13.249101 10875 raft_consensus.cc:2393] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 10: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:13.250936 13893 raft_consensus.cc:697] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 8 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:13.251968 13893 consensus_queue.cc:237] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:13.254061 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:13.254952 10867 raft_consensus.cc:2410] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:17:13.219530 13877 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:13.259559 10946 raft_consensus.cc:1275] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:17:13.259505 13877 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:13.260869 10949 raft_consensus.cc:1275] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:17:13.261811 13877 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:13.182947 13890 consensus_queue.cc:237] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:13.269688 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 105500us
I20260709 13:17:13.220472 13826 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:13.289434 10916 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:13.224720 13829 consensus_queue.cc:1059] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 13:17:13.278977 13439 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:13.309937 13421 consensus_queue.cc:1048] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 13:17:13.325389 11478 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:13.294955 13888 raft_consensus.cc:2749] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:13.330770 13899 raft_consensus.cc:2804] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:13.341804 13899 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:13.342404 13899 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:13.330590 13793 consensus_queue.cc:1048] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 13:17:13.334281 13439 raft_consensus.cc:2749] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:13.342310 11510 raft_consensus.cc:1275] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:17:13.345974 10874 raft_consensus.cc:2410] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 13:17:13.348981 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:13.313203 13633 consensus_queue.cc:1048] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:17:13.353082 11511 raft_consensus.cc:1275] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:17:13.354460 11511 pending_rounds.cc:85] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586: Aborting all ops after (but not including) 6
I20260709 13:17:13.354727 11511 pending_rounds.cc:107] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:17:13.354986 11511 raft_consensus.cc:2889] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:13.357997 10795 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 6.
I20260709 13:17:13.349692 13719 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:13.359335 13908 log.cc:927] Time spent T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:17:13.359481 13719 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:13.362524 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 73183.6us
I20260709 13:17:13.364285 10916 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:13.364534 10766 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:13.365334 13350 raft_consensus.cc:2749] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:13.367151 10916 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:13.336632 13798 consensus_queue.cc:1048] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 13:17:13.377400 10949 raft_consensus.cc:1275] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:17:13.378769 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:13.379629 10941 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:13.373878 13588 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:13.380935 13588 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:13.383656 13588 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:13.389690 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:13.391799 10795 raft_consensus.cc:1240] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:17:13.380945 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:13.439162 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:13.379184 13620 raft_consensus.cc:2804] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:13.372524 13896 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:13.371825 13871 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:13.443995 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:13.438663 13666 consensus_meta.cc:287] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.553s	user 0.009s	sys 0.002s
W20260709 13:17:13.451653 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 84578.6us
I20260709 13:17:13.438473 13439 raft_consensus.cc:2749] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:17:13.442380 13588 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:13.460785 10800 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 9.
I20260709 13:17:13.471328 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:13.472350 11503 raft_consensus.cc:2410] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:17:13.454829 13620 raft_consensus.cc:697] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:13.447048 13779 consensus_queue.cc:1048] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 13:17:13.467017 10916 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:13.481365 13895 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:13.483440 13886 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:13.484032 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:13.484714 10946 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:13.455281 13666 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:13.486999 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:13.487869 10793 raft_consensus.cc:2393] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 7: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:13.491937 10946 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 9.
I20260709 13:17:13.456231 13871 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:13.501662 11508 raft_consensus.cc:1240] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:13.494684 13439 raft_consensus.cc:2804] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:13.503713 13439 raft_consensus.cc:697] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 8 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:17:13.493899 13904 log.cc:927] Time spent T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
I20260709 13:17:13.513368 10941 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 7.
I20260709 13:17:13.513715 10797 raft_consensus.cc:1240] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:13.513422 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:13.518510 10766 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:13.519613 10841 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:13.493377 13913 log.cc:927] Time spent T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 13:17:13.505017 13439 consensus_queue.cc:237] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:13.464551 13588 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:13.522532 13639 raft_consensus.cc:2804] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:13.525151 13639 raft_consensus.cc:697] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:13.525096 13620 consensus_queue.cc:237] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:13.526372 13639 consensus_queue.cc:237] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:13.527279 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 63214.2us
I20260709 13:17:13.528119 10916 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:13.530071 13274 raft_consensus.cc:2749] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:13.531077 13886 consensus_queue.cc:1048] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:17:13.532681 13886 consensus_queue.cc:1048] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:13.531174 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:17:13.583244 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 70762.5us
I20260709 13:17:13.590860 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:17:13.644176 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:13.654059 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:13.673818 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:13.579152 13910 raft_consensus.cc:2749] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:17:13.712205 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 124518us
I20260709 13:17:13.583470 13274 consensus_queue.cc:1048] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.136s
W20260709 13:17:13.718802 13717 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.616s	user 0.014s	sys 0.031s
I20260709 13:17:13.583880 13880 consensus_queue.cc:1048] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.147s
W20260709 13:17:13.758421 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 47943us
W20260709 13:17:13.760788 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 46109.2us
I20260709 13:17:13.761803 11478 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
W20260709 13:17:13.751863 13717 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.905s	user 0.015s	sys 0.031s
W20260709 13:17:13.789932 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 260333us
I20260709 13:17:13.807657 10841 leader_election.cc:400] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 election: Vote denied by peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553) with higher term. Message: Invalid argument: T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 7. Current term is 8.
I20260709 13:17:13.808180 10841 leader_election.cc:403] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
W20260709 13:17:13.830075 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 70476.3us
I20260709 13:17:13.791072 13893 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:13.845337 10841 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:13.849148 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 280104us
W20260709 13:17:13.849879 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:13.834087 13910 raft_consensus.cc:2749] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553) with higher term. Message: Invalid argument: T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 7. Current term is 8.
I20260709 13:17:13.864425 10841 leader_election.cc:400] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Vote denied by peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553) with higher term. Message: Invalid argument: T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 6. Current term is 7.
I20260709 13:17:13.864935 10841 leader_election.cc:403] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:17:13.873034 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 56813.9us
W20260709 13:17:13.874462 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 111575us
I20260709 13:17:13.856609 13718 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:13.871160 10795 raft_consensus.cc:2955] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:17:13.887173 10947 raft_consensus.cc:1275] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:17:13.890714 10948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2012) took 1371 ms. Trace:
I20260709 13:17:13.855566 13933 consensus_queue.cc:1059] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:17:13.873375 13871 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:13.891160 10948 rpcz_store.cc:276] 0709 13:17:12.511335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:12.524932 (+ 13597us) service_pool.cc:224] Handling call
0709 13:17:12.626680 (+101748us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:12.792295 (+165615us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:13.882277 (+1089982us) spinlock_profiling.cc:243] Waited 159 ms on lock 0x7b54000a9bd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:13.882535 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":364679040}
I20260709 13:17:13.875577 13588 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:13.881031 13274 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:13.890372 13910 raft_consensus.cc:2749] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553) with higher term. Message: Invalid argument: T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 6. Current term is 7.
W20260709 13:17:13.910812 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 48458.6us
W20260709 13:17:13.924026 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 72385.3us
W20260709 13:17:13.937798 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:13.949621 13894 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:13.950434 13894 consensus_queue.cc:1048] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:17:13.952638 13935 consensus_queue.cc:1048] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:13.954263 13935 consensus_queue.cc:1048] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:13.918089 13274 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:13.937723 13717 consensus_queue.cc:1048] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.025s
W20260709 13:17:13.959786 13932 log.cc:927] Time spent T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 13:17:13.930104 13828 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:13.934307 13850 consensus_queue.cc:1048] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
W20260709 13:17:13.962270 13865 consensus_meta.cc:287] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.904s	user 0.011s	sys 0.000s
W20260709 13:17:13.967280 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 83807.5us
I20260709 13:17:13.915699 13936 consensus_queue.cc:1048] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.066s
W20260709 13:17:13.959960 13899 consensus_meta.cc:287] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.616s	user 0.010s	sys 0.003s
W20260709 13:17:13.959842 13926 log.cc:927] Time spent T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
W20260709 13:17:13.989283 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 62962.3us
W20260709 13:17:13.990818 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 69891.6us
I20260709 13:17:13.932055 13718 consensus_queue.cc:1059] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 13:17:13.993724 13947 consensus_queue.cc:1048] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:13.994087 13828 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:13.996224 13877 raft_consensus.cc:3055] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:13.996866 13666 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:13.999770 13916 log.cc:927] Time spent T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.683s	user 0.002s	sys 0.012s
I20260709 13:17:14.015064 13877 raft_consensus.cc:740] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:17:14.017800 13931 log.cc:927] Time spent T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:17:14.000885 13719 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:14.014642 13828 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:14.019104 13899 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:14.020617 13828 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:14.021176 13899 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:14.021795 13274 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:14.017344 13877 consensus_queue.cc:260] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:14.024577 10916 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:14.027952 13893 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:14.028472 13893 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:17:14.016446 13921 log.cc:927] Time spent T 00000000000000000000000000000000 P ae957c8fc66c4b6ba063d0c74cbf5484: Append to log took a long time: real 0.555s	user 0.006s	sys 0.009s
I20260709 13:17:14.005685 10767 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:14.038801 10916 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:14.044824 10767 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:14.043988 11477 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:17:14.053629 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 173254us
I20260709 13:17:14.067641 11478 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:14.054653 10916 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:14.049620 10767 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:14.039486 13936 raft_consensus.cc:2804] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:14.070580 10916 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:14.072023 10767 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:14.073304 13639 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:14.073756 13639 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:14.074177 13639 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:14.045826 13895 consensus_queue.cc:1048] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
W20260709 13:17:14.079521 13901 log.cc:927] Time spent T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 1.243s	user 0.004s	sys 0.010s
I20260709 13:17:14.068995 13943 consensus_queue.cc:1059] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 13:17:14.072327 13620 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:14.081283 13620 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:14.081708 13620 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:14.021209 13828 consensus_queue.cc:1048] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 13:17:14.084357 10800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2044) took 1022 ms. Trace:
I20260709 13:17:14.084574 10767 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:14.086421 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:14.054512 13871 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:14.045794 13939 consensus_queue.cc:1048] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 13:17:14.051792 13588 consensus_queue.cc:1048] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 13:17:14.091244 13718 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:14.092072 13779 consensus_queue.cc:1048] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:14.045616 13274 consensus_queue.cc:1059] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:17:14.065176 13896 raft_consensus.cc:2804] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:14.031697 13865 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:14.051010 13719 raft_consensus.cc:3055] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:14.041061 13666 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:14.088562 10841 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:14.096776 10942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 1632) took 1328 ms. Trace:
W20260709 13:17:14.098104 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:14.068493 13933 consensus_queue.cc:1059] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 13:17:14.103907 13935 raft_consensus.cc:2749] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:14.104554 13828 raft_consensus.cc:2804] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:14.099711 13588 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:14.084950 10800 rpcz_store.cc:276] 0709 13:17:13.061846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:13.845132 (+783286us) service_pool.cc:224] Handling call
0709 13:17:13.865720 (+ 20588us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:13.866012 (+   292us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:17:14.038038 (+172026us) log.cc:844] Serialized 14 byte log entry
0709 13:17:14.083878 (+ 45840us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:14.083954 (+    76us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:14.084025 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:14.084116 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:14.084282 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19951,"prepare.queue_time_us":72734,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":9626,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":86,"spinlock_wait_cycles":2816,"thread_start_us":133762,"threads_started":2}
I20260709 13:17:14.106596 10942 rpcz_store.cc:276] 0709 13:17:12.768298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:12.768818 (+   520us) service_pool.cc:224] Handling call
0709 13:17:12.772110 (+  3292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:12.799545 (+ 27435us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:17:12.799575 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:12.800087 (+   512us) log.cc:844] Serialized 26 byte log entry
0709 13:17:12.809560 (+  9473us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:12.809638 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:12.809734 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:14.040983 (+1231249us) spinlock_profiling.cc:243] Waited 43.9 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:17:14.083508 (+ 42525us) raft_consensus.cc:1685] finished
0709 13:17:14.083528 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:14.096704 (+ 13176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16964,"raft.queue_time_us":30795,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":9733,"spinlock_wait_cycles":101034240,"thread_start_us":9074,"threads_started":1,"wal-append.queue_time_us":9443}
I20260709 13:17:14.105203 13719 raft_consensus.cc:740] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:14.131707 13719 consensus_queue.cc:260] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:14.133108 13719 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:14.119560 13828 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:14.098572 13895 consensus_queue.cc:1048] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 13:17:14.137252 13871 raft_consensus.cc:2749] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:14.134845 13950 raft_consensus.cc:2749] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:14.130551 13865 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:14.119895 13957 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:14.099532 13718 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:14.135735 13935 raft_consensus.cc:3055] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:14.137094 13828 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:14.135639 13899 raft_consensus.cc:2749] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:14.128793 13949 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:14.138873 13957 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:14.156646 13949 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:14.153723 13935 raft_consensus.cc:740] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:14.159590 13949 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:14.160082 13935 consensus_queue.cc:260] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:14.147035 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 58590.5us
I20260709 13:17:14.128134 13850 raft_consensus.cc:2749] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:17:14.158564 13957 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:14.161460 13935 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:14.181692 13718 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:14.188550 10841 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:14.200866 10841 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:14.205569 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 135069us
W20260709 13:17:14.203732 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 136180us
I20260709 13:17:14.208377 10916 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:14.209201 11478 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:14.209916 10916 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:14.175590 13939 consensus_queue.cc:1048] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 13:17:14.212790 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:14.182507 13893 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:14.191102 13779 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:14.224061 10795 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 6.
I20260709 13:17:14.222715 10916 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:17:14.226117 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:14.239749 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:14.240397 10797 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:14.240514 10874 raft_consensus.cc:1275] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:17:14.242058 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:14.243819 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:14.204610 13865 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:14.245018 10916 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:14.207727 13871 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:14.246444 10871 raft_consensus.cc:1275] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:17:14.248466 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:14.249094 10795 raft_consensus.cc:1275] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:17:14.258196 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:14.206571 13850 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:17:14.196163 13896 raft_consensus.cc:697] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 9 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:14.277398 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:14.230770 13968 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:14.246146 13871 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:14.213967 13588 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:14.279964 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:14.263088 13865 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:14.319258 10876 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 6.
I20260709 13:17:14.319113 13588 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:17:14.276078 11511 raft_consensus.cc:1240] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:14.222925 13666 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:14.208240 13718 consensus_queue.cc:1048] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.070s
I20260709 13:17:14.321290 13588 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:14.321107 13666 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:14.323607 13828 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:14.196197 13936 raft_consensus.cc:697] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 7 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:17:14.306634 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:14.308264 10797 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 9.
I20260709 13:17:14.316056 10798 raft_consensus.cc:1240] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:17:14.290529 13634 consensus_queue.cc:1048] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
W20260709 13:17:14.288838 13909 log.cc:927] Time spent T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.953s	user 0.003s	sys 0.008s
I20260709 13:17:14.283504 13959 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:14.295989 13439 raft_consensus.cc:3055] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:14.317901 10767 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:14.299130 13961 raft_consensus.cc:2804] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:17:14.283474 13917 log.cc:927] Time spent T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.966s	user 0.005s	sys 0.007s
I20260709 13:17:14.282593 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:14.247030 13895 consensus_queue.cc:1048] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.111s
I20260709 13:17:14.255613 13779 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:14.321084 13865 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:14.388832 10916 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:14.448954 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:14.449276 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 60574.6us
I20260709 13:17:14.449618 10874 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 7.
I20260709 13:17:14.450764 10916 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:14.451107 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:14.231338 13639 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:14.325774 13828 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:14.276072 13950 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:14.443441 10801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1620) took 1576 ms. Trace:
W20260709 13:17:14.464185 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 146351us
I20260709 13:17:14.395193 13850 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:14.465330 10767 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:14.465556 13850 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:14.467800 13850 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:14.337546 11504 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 10.
I20260709 13:17:14.444393 13959 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:14.444963 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:14.470455 10876 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:14.453979 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:14.447329 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:14.472247 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:14.473421 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:17:14.387980 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:14.283133 11503 raft_consensus.cc:1240] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:17:14.256582 13943 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:14.222172 13960 raft_consensus.cc:2804] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:14.298444 13966 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:14.445951 13634 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:14.476025 10874 raft_consensus.cc:1275] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:17:14.258850 13620 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:14.460922 13935 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:14.482142 13822 raft_consensus.cc:3055] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:14.451908 10916 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:14.479786 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:14.284720 13893 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:14.462054 10801 rpcz_store.cc:276] 0709 13:17:12.867346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:12.887486 (+ 20140us) service_pool.cc:224] Handling call
0709 13:17:12.962011 (+ 74525us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:13.015514 (+ 53503us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:13.015556 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:13.021858 (+  6302us) log.cc:844] Serialized 52 byte log entry
0709 13:17:13.136602 (+114744us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:13.147318 (+ 10716us) log.cc:844] Serialized 14 byte log entry
0709 13:17:13.147567 (+   249us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:13.147658 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:13.993480 (+845822us) spinlock_profiling.cc:243] Waited 4.61 ms on lock 0x7b18003ccac0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7dd962a6 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc83f3 00007f9c7ddca7c1
0709 13:17:14.284321 (+290841us) spinlock_profiling.cc:243] Waited 4.53 ms on lock 0x7b18003ccac0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc83f3 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:17:14.288020 (+  3699us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:17:14.315045 (+ 27025us) spinlock_profiling.cc:243] Waited 3.3 ms on lock 0x7b18003ccac0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc8522 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:17:14.443187 (+128142us) raft_consensus.cc:1685] finished
0709 13:17:14.443208 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:14.443352 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":13596,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":12613,"raft.run_cpu_time_us":254,"raft.run_wall_time_us":31037,"spinlock_wait_cycles":58717568,"thread_start_us":62420,"threads_started":1,"wal-append.queue_time_us":216873}
W20260709 13:17:14.299321 13958 log.cc:927] Time spent T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 13:17:14.389482 13829 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:14.495201 13966 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:14.493573 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:14.513080 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:14.323876 13871 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:14.494315 10792 raft_consensus.cc:1240] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:17:14.494427 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:14.495213 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:14.519901 10873 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:14.496618 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:14.447108 13439 raft_consensus.cc:740] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:14.460423 13880 raft_consensus.cc:2749] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:14.455200 13961 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:14.498252 10874 pending_rounds.cc:85] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 4
I20260709 13:17:14.526579 10874 pending_rounds.cc:107] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:17:14.526953 10874 raft_consensus.cc:2889] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:14.478015 10876 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 9.
I20260709 13:17:14.328907 13718 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:14.548189 13718 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:14.325500 13666 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:14.476555 13950 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:14.249379 13890 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:14.549877 13950 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:14.550087 13890 raft_consensus.cc:697] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:14.522713 13893 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:14.551364 13893 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:14.507282 10797 raft_consensus.cc:2393] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 7: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:14.553146 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:14.488740 13960 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:14.555016 13960 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:14.557265 13920 log.cc:927] Time spent T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 1.096s	user 0.000s	sys 0.012s
I20260709 13:17:14.474717 13274 consensus_queue.cc:1048] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:17:14.489249 10872 raft_consensus.cc:1275] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:17:14.560413 10872 pending_rounds.cc:85] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 6
I20260709 13:17:14.560734 10872 pending_rounds.cc:107] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:17:14.560848 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:14.561054 10872 raft_consensus.cc:2889] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:14.515166 13966 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:17:14.510507 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 58677.8us
I20260709 13:17:14.563647 10916 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:14.565196 10916 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:14.566262 10916 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:14.520761 10870 raft_consensus.cc:2393] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 8: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:14.521931 10798 raft_consensus.cc:2393] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 8: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:14.569372 10798 raft_consensus.cc:1240] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:17:14.328323 13968 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:14.389760 13896 consensus_queue.cc:237] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:14.571333 13933 raft_consensus.cc:3055] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:14.571766 13933 raft_consensus.cc:740] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:14.572556 13968 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:14.575057 10793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 1626) took 1686 ms. Trace:
I20260709 13:17:14.575436 10793 rpcz_store.cc:276] 0709 13:17:12.889064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:12.889219 (+   155us) service_pool.cc:224] Handling call
0709 13:17:14.574844 (+1685625us) spinlock_profiling.cc:243] Waited 596 ms on lock 0x7b5400212ad4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:14.574994 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1370765184}
I20260709 13:17:14.576918 10793 raft_consensus.cc:1275] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:17:14.505877 13822 raft_consensus.cc:740] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:14.584208 13822 consensus_queue.cc:260] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:14.585290 13822 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:14.528081 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:14.528661 13961 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:14.503739 13634 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:14.607112 10797 raft_consensus.cc:1275] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:17:14.532444 13639 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:14.532742 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:14.611553 10869 raft_consensus.cc:2393] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 5: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:14.532449 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:14.612125 10874 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 8.
I20260709 13:17:14.612500 10842 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:14.612968 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:14.613669 10874 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 6.
W20260709 13:17:14.528708 13898 log.cc:927] Time spent T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 1.515s	user 0.007s	sys 0.010s
I20260709 13:17:14.520258 13947 consensus_queue.cc:1048] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 13:17:14.515278 13829 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:14.539566 13439 consensus_queue.cc:260] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:14.551219 13890 consensus_queue.cc:237] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:14.474282 13826 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:17:14.508858 13970 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:14.587080 10876 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 9.
I20260709 13:17:14.570662 10798 raft_consensus.cc:1240] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 LEADER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:17:14.570662 10792 raft_consensus.cc:1240] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:17:14.588568 10916 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:14.591276 10873 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 7.
I20260709 13:17:14.509997 13943 raft_consensus.cc:2749] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:17:14.531198 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:14.629942 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:14.586272 13779 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:14.614544 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:14.616618 10874 raft_consensus.cc:1240] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 7. Current term is 8. Ops: [5.5-7.6]
I20260709 13:17:14.622083 13871 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:14.635017 10876 raft_consensus.cc:1240] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:17:14.619302 13954 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:14.617656 13880 raft_consensus.cc:2749] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:14.620898 13718 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:14.642970 13871 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:14.640295 13973 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:14.624709 13274 raft_consensus.cc:2749] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:14.682639 10794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2043) took 1643 ms. Trace:
I20260709 13:17:14.685036 13850 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:17:14.689371 10841 connection.cc:256] RPC call timeout handler was delayed by -1.0859544240001924s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1515s for a response
W20260709 13:17:14.690126 10841 connection.cc:256] RPC call timeout handler was delayed by -1.0508544240001356s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1554s for a response
I20260709 13:17:14.629102 13666 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:17:14.692595 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 80163.9us
I20260709 13:17:14.693696 10841 leader_election.cc:400] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 election: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 7. Current term is 8.
W20260709 13:17:14.741278 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:14.683629 13898 log.cc:927] Time spent T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:17:14.661290 13954 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:14.748977 13965 raft_consensus.cc:2804] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:14.681571 13634 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:17:14.765885 13969 log.cc:927] Time spent T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.146s	user 0.000s	sys 0.000s
W20260709 13:17:14.767331 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 66090.9us
I20260709 13:17:14.692131 13439 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:14.572566 13933 consensus_queue.cc:260] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:14.698320 13943 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:14.613904 13936 consensus_queue.cc:237] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:14.762051 13965 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:14.795697 13950 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:14.762691 13954 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:14.866485 13967 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:14.692335 13850 raft_consensus.cc:697] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:14.924068 13865 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:14.952742 10796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2031) took 2284 ms. Trace:
I20260709 13:17:14.953559 10799 raft_consensus.cc:1240] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:17:14.759780 13666 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:14.966100 10796 rpcz_store.cc:276] 0709 13:17:12.667932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:12.668047 (+   115us) service_pool.cc:224] Handling call
0709 13:17:12.685716 (+ 17669us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:12.707457 (+ 21741us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:17:12.710728 (+  3271us) log.cc:844] Serialized 14 byte log entry
0709 13:17:12.829245 (+118517us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:12.834877 (+  5632us) log.cc:844] Serialized 26 byte log entry
0709 13:17:12.835027 (+   150us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:12.835062 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:12.835205 (+   143us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:14.528098 (+1692893us) spinlock_profiling.cc:243] Waited 2.13 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:17:14.952404 (+424306us) raft_consensus.cc:1685] finished
0709 13:17:14.952505 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:14.952655 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1168,"prepare.queue_time_us":1754,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":6589,"raft.run_cpu_time_us":289,"raft.run_wall_time_us":31209,"spinlock_wait_cycles":4889344,"thread_start_us":116768,"threads_started":1,"wal-append.queue_time_us":302858}
W20260709 13:17:14.857509 13777 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.810s	user 0.015s	sys 0.010s
I20260709 13:17:14.851559 13965 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:14.698611 13973 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:14.993232 13777 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.996s	user 0.015s	sys 0.010s
W20260709 13:17:15.051488 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:14.850919 13943 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:15.206508 13977 log.cc:927] Time spent T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.336s	user 0.000s	sys 0.000s
I20260709 13:17:15.193845 13350 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:15.211790 13350 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:15.214888 13979 log.cc:927] Time spent T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.186s	user 0.000s	sys 0.000s
W20260709 13:17:15.214977 13976 log.cc:927] Time spent T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.307s	user 0.001s	sys 0.000s
W20260709 13:17:15.217557 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 524041us
I20260709 13:17:15.218457 10841 leader_election.cc:400] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 election: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 5. Current term is 6.
I20260709 13:17:14.968346 13967 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:14.994391 13865 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:15.221484 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:15.222831 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:15.224198 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 635651us
I20260709 13:17:15.235749 10916 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:15.249804 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:15.250257 13350 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:15.250829 10947 raft_consensus.cc:2410] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:17:15.251428 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:15.251654 13943 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:15.251776 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:15.252197 13943 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:15.252452 10947 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 9.
I20260709 13:17:15.253396 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:15.253720 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:14.684768 10794 rpcz_store.cc:276] 0709 13:17:13.047454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:13.552146 (+504692us) service_pool.cc:224] Handling call
0709 13:17:13.585759 (+ 33613us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:13.586024 (+   265us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:17:14.579241 (+993217us) spinlock_profiling.cc:243] Waited 1.53 ms on lock 0x7b180042dc00. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c80a5d4bc 00007f9c7ddab48b 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc5455 00007f9c848ad3c7 00007f9c80ebf9a7 00007f9c80ecf0bf
0709 13:17:14.579411 (+   170us) log.cc:844] Serialized 14 byte log entry
0709 13:17:14.682240 (+102829us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:14.682317 (+    77us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:14.682394 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:14.682464 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:14.682578 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":155522,"prepare.queue_time_us":317305,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":5251,"spinlock_wait_cycles":3519872,"thread_start_us":274963,"threads_started":2}
I20260709 13:17:15.255425 10943 raft_consensus.cc:1275] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 13:17:15.255816 10943 pending_rounds.cc:85] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 3
I20260709 13:17:15.256033 10943 pending_rounds.cc:107] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 13:17:15.256254 10943 raft_consensus.cc:2889] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:15.256350 10916 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:15.257588 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:15.257913 10916 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:15.258215 10943 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7.
I20260709 13:17:15.258486 13826 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:15.233217 13777 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:15.259037 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:15.259258 13777 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:15.261297 13666 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:15.253913 10947 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:15.263607 10799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1625) took 2382 ms. Trace:
I20260709 13:17:15.264071 10950 raft_consensus.cc:2393] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 10: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:15.265002 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:15.265498 13961 consensus_meta.cc:287] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.667s	user 0.006s	sys 0.000s
I20260709 13:17:15.265933 13961 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:15.266381 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:15.233466 13710 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:15.267319 13710 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:15.269423 13781 raft_consensus.cc:2749] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:15.259119 13826 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:15.261874 13666 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:15.272651 13666 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:15.274274 13966 consensus_meta.cc:287] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.711s	user 0.005s	sys 0.002s
I20260709 13:17:15.249795 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:15.275130 10945 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 6.
I20260709 13:17:15.276068 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:15.276504 13850 consensus_queue.cc:237] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:15.277559 10945 raft_consensus.cc:1275] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:17:15.188656 13893 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:15.261680 13777 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:15.283845 13777 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:15.284351 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:15.284786 13826 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:15.285435 13826 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:15.285918 13826 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:15.269454 13710 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:15.284921 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:15.270263 13781 consensus_queue.cc:1059] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:17:15.290310 13960 consensus_meta.cc:287] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.735s	user 0.004s	sys 0.003s
I20260709 13:17:15.290959 10947 raft_consensus.cc:2410] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:17:15.293889 13973 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:15.295454 13973 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:15.295940 13973 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:15.296288 13967 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:15.297214 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:15.264009 10799 rpcz_store.cc:276] 0709 13:17:12.880875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:12.881051 (+   176us) service_pool.cc:224] Handling call
0709 13:17:14.953293 (+2072242us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:14.967168 (+ 13875us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:15.263413 (+296245us) spinlock_profiling.cc:243] Waited 2.05 s on lock 0x7b54000bd6d0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:15.263545 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4704826112}
I20260709 13:17:15.297982 13973 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:15.298225 10946 raft_consensus.cc:2410] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:17:15.274716 13666 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:15.294296 13829 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:15.298640 13781 raft_consensus.cc:2804] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:15.298913 13829 consensus_queue.cc:1059] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:15.299232 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:15.299886 13829 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:15.300278 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:15.300371 13829 raft_consensus.cc:697] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:15.300413 13886 raft_consensus.cc:3055] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:15.300942 10946 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 9.
I20260709 13:17:15.301090 13886 raft_consensus.cc:740] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:15.301429 13829 consensus_queue.cc:237] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:15.302408 13871 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:15.303114 13781 raft_consensus.cc:697] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 6 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:15.302131 13886 consensus_queue.cc:260] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:15.305070 13886 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:15.305380 13877 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:15.301828 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:15.307425 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:15.307550 11477 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:15.308074 10946 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 4.
I20260709 13:17:15.308948 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:15.310743 13826 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:15.311439 13822 consensus_meta.cc:287] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.726s	user 0.011s	sys 0.001s
I20260709 13:17:15.284350 13777 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:15.313908 13777 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:15.225850 13933 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:15.316654 13781 consensus_queue.cc:237] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:15.319525 11477 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:15.320631 13893 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:15.321156 13893 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:15.321605 13893 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:17:15.323167 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 104854us
I20260709 13:17:15.326056 13950 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:15.327245 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:15.335070 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:15.339305 10944 raft_consensus.cc:2410] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:17:15.343114 10841 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:15.252725 13943 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:15.349898 10873 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.014s	sys 0.024s
W20260709 13:17:15.350396 10873 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.014s	sys 0.024s
I20260709 13:17:15.337401 13960 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:15.368214 10948 raft_consensus.cc:2393] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 7: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:15.369479 13960 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:15.337047 13950 raft_consensus.cc:3055] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:15.292253 13959 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:15.344398 13967 raft_consensus.cc:2804] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:15.370553 13960 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:15.347606 13822 raft_consensus.cc:2749] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:15.382246 13950 raft_consensus.cc:740] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:15.417373 13950 consensus_queue.cc:260] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:15.394402 13967 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:15.420435 13967 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:15.420145 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 83067us
I20260709 13:17:15.424448 10841 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:15.430366 13620 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:15.438833 13960 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:15.439775 10948 raft_consensus.cc:1275] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:17:15.451303 13943 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:15.389266 13954 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:15.398801 10949 raft_consensus.cc:1275] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:17:15.393173 13939 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:15.274698 13966 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:15.408753 13439 consensus_meta.cc:287] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.609s	user 0.009s	sys 0.000s
I20260709 13:17:15.411283 13865 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:15.462029 13719 raft_consensus.cc:2749] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:15.461923 10841 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:15.467993 10949 raft_consensus.cc:1275] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260709 13:17:15.470973 13950 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:17:15.516156 11478 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.060s	user 0.009s	sys 0.054s
W20260709 13:17:15.517307 11478 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.061s	user 0.009s	sys 0.054s
W20260709 13:17:15.518047 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 1.06187e+06us
I20260709 13:17:15.520622 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:15.528959 10941 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 10.
W20260709 13:17:15.469732 13935 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.722s	user 0.028s	sys 0.024s
I20260709 13:17:15.511929 13959 consensus_queue.cc:1048] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:17:15.511598 13968 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:15.534471 13966 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:15.536659 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1646) took 1043 ms. Trace:
W20260709 13:17:15.638700 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:15.466835 13960 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:15.462703 13961 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:15.544996 13985 raft_consensus.cc:2749] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:17:15.492404 13633 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.497s	user 0.006s	sys 0.051s
W20260709 13:17:15.723819 13633 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.728s	user 0.006s	sys 0.052s
W20260709 13:17:15.551276 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 89629.3us
W20260709 13:17:15.724375 10698 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42506 (request call id 153) took 20654 ms (client timeout 15000 ms). Trace:
I20260709 13:17:15.725176 10841 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:15.725502 13965 consensus_meta.cc:287] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.537s	user 0.010s	sys 0.000s
I20260709 13:17:15.725960 13965 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:15.727694 10870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1596) took 1218 ms. Trace:
I20260709 13:17:15.267567  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:17:15.558622 10869 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.040s	sys 0.162s
W20260709 13:17:15.728562 10869 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.041s	sys 0.162s
I20260709 13:17:15.607955 10949 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:15.728942 13890 raft_consensus.cc:2749] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:17:15.729738 13890 consensus_queue.cc:1048] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:17:15.725220 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 397) took 1096 ms. Trace:
I20260709 13:17:15.731925 13890 consensus_queue.cc:1059] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:17:15.715276 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:15.728206 10870 rpcz_store.cc:276] 0709 13:17:14.508903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:14.567587 (+ 58684us) service_pool.cc:224] Handling call
0709 13:17:14.567959 (+   372us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:14.610003 (+ 42044us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:14.610037 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:14.617163 (+  7126us) log.cc:844] Serialized 26 byte log entry
0709 13:17:14.746701 (+129538us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:14.746761 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:14.746879 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:15.727424 (+980545us) raft_consensus.cc:1685] finished
0709 13:17:15.727451 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:15.727615 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29955,"raft.queue_time_us":41958,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":4286,"spinlock_wait_cycles":1556736,"thread_start_us":123566,"threads_started":1,"wal-append.queue_time_us":1059496}
W20260709 13:17:15.715302 13981 log.cc:927] Time spent T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.170s	user 0.000s	sys 0.001s
W20260709 13:17:15.712574 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:15.734935 13985 raft_consensus.cc:3055] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:15.735321 13985 raft_consensus.cc:740] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:15.736137 13985 consensus_queue.cc:260] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:15.737099 13985 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:15.737136 10949 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 7.
W20260709 13:17:15.711045 13274 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.704s	user 0.004s	sys 0.023s
W20260709 13:17:15.738922 13274 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.735s	user 0.004s	sys 0.023s
I20260709 13:17:15.728190 13965 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:15.739684 13954 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:15.720736 11478 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:15.731218 10873 rpcz_store.cc:276] 0709 13:17:14.629251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:14.631882 (+  2631us) service_pool.cc:224] Handling call
0709 13:17:14.632319 (+   437us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:14.656760 (+ 24441us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:14.672368 (+ 15608us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:14.685500 (+ 13132us) log.cc:844] Serialized 26 byte log entry
0709 13:17:15.354471 (+668971us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:15.354530 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:15.354625 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:15.725016 (+370391us) raft_consensus.cc:1685] finished
0709 13:17:15.725036 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:15.725160 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57232,"raft.queue_time_us":10519,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":6816,"thread_start_us":607912,"threads_started":1,"wal-append.queue_time_us":977168}
I20260709 13:17:15.728137 10841 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:17:15.712018 10867 rpcz_store.cc:276] 0709 13:17:14.493591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:14.561953 (+ 68362us) service_pool.cc:224] Handling call
0709 13:17:14.562299 (+   346us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:14.589114 (+ 26815us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:14.589144 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:14.618706 (+ 29562us) log.cc:844] Serialized 26 byte log entry
0709 13:17:14.802872 (+184166us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:14.802918 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:14.802989 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:15.536436 (+733447us) raft_consensus.cc:1685] finished
0709 13:17:15.536456 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:15.536598 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":35507,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":3204,"thread_start_us":122513,"threads_started":1,"wal-append.queue_time_us":895256}
W20260709 13:17:15.711762 13935 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.003s	user 0.030s	sys 0.031s
I20260709 13:17:15.738216 13967 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:15.651257 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:15.753500 13890 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:15.756278 13890 raft_consensus.cc:697] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:15.757308 13890 consensus_queue.cc:237] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:15.761991 11478 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:15.724972 10698 rpcz_store.cc:269] 0709 13:16:55.069644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.103978 (+ 34334us) service_pool.cc:224] Handling call
0709 13:17:15.724254 (+20620276us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:13.154203 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:13.187262 (+ 33059us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:13.187315 (+    53us) write_op.cc:435] Acquired schema lock
0709 13:17:13.187336 (+    21us) tablet.cc:662] Decoding operations
0709 13:17:13.188933 (+  1597us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:13.189043 (+   110us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:13.189078 (+    35us) tablet.cc:685] Acquiring locks for 16 operations
0709 13:17:13.190574 (+  1496us) tablet.cc:701] Row locks acquired
0709 13:17:13.190645 (+    71us) write_op.cc:260] PREPARE: finished
0709 13:17:13.221669 (+ 31024us) write_op.cc:270] Start()
0709 13:17:13.274865 (+ 53196us) write_op.cc:276] Timestamp: P: 1783603033190769 usec, L: 0
0709 13:17:13.284012 (+  9147us) op_driver.cc:348] REPLICATION: starting
0709 13:17:13.361731 (+ 77719us) log.cc:844] Serialized 6588 byte log entry
0709 13:17:13.998219 (+636488us) spinlock_profiling.cc:243] Waited 1.2 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:17:14.395709 (+397490us) op_driver.cc:464] REPLICATION: finished
0709 13:17:14.527973 (+132264us) write_op.cc:301] APPLY: starting
0709 13:17:14.528072 (+    99us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:14.528456 (+   384us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:14.528493 (+    37us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:14.530771 (+  2278us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:14.531375 (+   604us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=16
0709 13:17:14.531463 (+    88us) write_op.cc:312] APPLY: finished
0709 13:17:14.532230 (+   767us) log.cc:844] Serialized 113 byte log entry
0709 13:17:14.533213 (+   983us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:14.533445 (+   232us) write_op.cc:454] Released schema lock
0709 13:17:14.533544 (+    99us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:14.534037 (+   493us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:14.534072 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":125934,"reported_tablets":31,"child_traces":[["op",{"apply.queue_time_us":131916,"apply.run_cpu_time_us":4921,"apply.run_wall_time_us":6060,"num_ops":16,"prepare.queue_time_us":245869,"prepare.run_cpu_time_us":33589,"prepare.run_wall_time_us":327887,"raft.queue_time_us":200317,"raft.run_cpu_time_us":6447,"raft.run_wall_time_us":160031,"replication_time_us":1042599,"spinlock_wait_cycles":2763136,"thread_start_us":408633,"threads_started":4,"wal-append.queue_time_us":97243,"wal-append.run_cpu_time_us":32822,"wal-append.run_wall_time_us":2073983}]]}
I20260709 13:17:15.479278 13939 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:15.767330 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:15.769201 11478 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:15.768460 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:17:15.765946 10842 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.997s	user 0.029s	sys 0.047s
W20260709 13:17:15.772930 10842 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.004s	user 0.030s	sys 0.047s
I20260709 13:17:15.773073 11504 raft_consensus.cc:2393] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 9: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
W20260709 13:17:15.773522 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 1.0051e+06us
I20260709 13:17:15.745124 13961 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:15.777065 10841 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:15.571628 13893 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:15.746049 13954 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:15.800585 10942 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.004s	sys 0.007s
I20260709 13:17:15.741000 13968 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:15.772322 13982 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:15.813715 10942 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.537s	user 0.004s	sys 0.007s
W20260709 13:17:15.830469 10767 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.000s	sys 0.032s
W20260709 13:17:15.833366 10767 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.000s	sys 0.033s
I20260709 13:17:15.799029 13633 raft_consensus.cc:2749] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:17:15.808418 13949 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.525s	user 0.018s	sys 0.045s
I20260709 13:17:15.792196 13980 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:15.815289 10697 catalog_manager.cc:5697] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 4 to 9, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 9 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:17:15.865927 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:15.804423 13961 raft_consensus.cc:697] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 9 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:15.874372 10697 catalog_manager.cc:5697] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:15.928069 10697 catalog_manager.cc:5697] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 2 to 9. New cstate: current_term: 9 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
W20260709 13:17:15.830238 13829 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.001s	sys 0.010s
W20260709 13:17:15.843968 13718 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.016s	sys 0.052s
I20260709 13:17:15.792028 13959 consensus_queue.cc:1048] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.042s
W20260709 13:17:15.869449 13949 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.586s	user 0.019s	sys 0.045s
W20260709 13:17:15.944550 10765 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.013s	sys 0.030s
W20260709 13:17:15.944926 10765 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.013s	sys 0.030s
I20260709 13:17:15.945940 13865 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:15.946624 13865 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:15.949195 13865 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:15.949831 13865 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:15.950889 13966 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:15.951520 13966 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:15.954015 13966 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:17:15.956640 13999 log.cc:927] Time spent T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:17:15.959340 13633 consensus_queue.cc:1059] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:17:15.806146 13954 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:15.870218 13980 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:15.931242 13829 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.002s	sys 0.010s
W20260709 13:17:15.935374 13718 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.016s	sys 0.052s
I20260709 13:17:15.927511 13970 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:15.930819 13719 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:15.964545 13933 consensus_meta.cc:287] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.627s	user 0.013s	sys 0.000s
I20260709 13:17:15.970261 13959 consensus_queue.cc:1243] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer e01ecd008edd4aa2b6b2b47efbc2ea87 log is divergent from this leader: its last log entry 5.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:17:15.843818 13982 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:16.000310 10950 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 6.
W20260709 13:17:16.013988 10944 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.038s	sys 0.047s
W20260709 13:17:16.016685 10944 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.038s	sys 0.047s
I20260709 13:17:16.020017 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1650) took 1490 ms. Trace:
I20260709 13:17:16.020608 10869 rpcz_store.cc:276] 0709 13:17:14.529769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:14.616339 (+ 86570us) service_pool.cc:224] Handling call
0709 13:17:14.627141 (+ 10802us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:14.688516 (+ 61375us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:14.688548 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:14.870964 (+182416us) log.cc:844] Serialized 52 byte log entry
0709 13:17:15.729023 (+858059us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:15.729072 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:15.729146 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:16.019727 (+290581us) raft_consensus.cc:1685] finished
0709 13:17:16.019754 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:16.019937 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45743,"raft.queue_time_us":51465,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":17469,"thread_start_us":675924,"threads_started":1,"wal-append.queue_time_us":1004827}
W20260709 13:17:16.024585 10766 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.745s	user 0.006s	sys 0.024s
W20260709 13:17:16.025111 10766 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.745s	user 0.006s	sys 0.024s
I20260709 13:17:16.025955 10766 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:15.974761 13980 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:16.035383 10945 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.021s	sys 0.042s
W20260709 13:17:16.037708 10945 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.021s	sys 0.042s
I20260709 13:17:15.930814 10697 catalog_manager.cc:5697] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 1 to 7, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 7 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:16.050194 10697 catalog_manager.cc:5697] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
W20260709 13:17:16.057214 10948 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.020s	sys 0.071s
W20260709 13:17:16.057775 10948 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.020s	sys 0.071s
W20260709 13:17:16.060871 10872 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.079s	user 0.013s	sys 0.021s
W20260709 13:17:16.062296 10872 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.080s	user 0.014s	sys 0.021s
W20260709 13:17:16.045882 13850 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.021s	sys 0.043s
I20260709 13:17:15.988314 13970 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:16.079192 10947 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.771s	user 0.003s	sys 0.032s
W20260709 13:17:16.082016 10947 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.774s	user 0.004s	sys 0.032s
I20260709 13:17:16.063794 10697 catalog_manager.cc:5697] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 5 to 7, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 7 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:15.999403 13719 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:15.950354 13865 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:16.051644 10916 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.793s	user 0.008s	sys 0.024s
I20260709 13:17:16.012693 13718 consensus_queue.cc:1059] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 13:17:16.087390 10697 catalog_manager.cc:5697] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:15.975139 13961 consensus_queue.cc:237] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:16.058876 13828 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.764s	user 0.009s	sys 0.024s
W20260709 13:17:16.104660 10916 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.846s	user 0.008s	sys 0.024s
W20260709 13:17:16.027935 13910 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.741s	user 0.014s	sys 0.019s
W20260709 13:17:16.067898 13850 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.022s	sys 0.044s
W20260709 13:17:16.107998 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 850087us
I20260709 13:17:16.067014 13967 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:16.081190 13717 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.015s	sys 0.022s
W20260709 13:17:16.118023 10943 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.031s	sys 0.055s
W20260709 13:17:16.139475 10943 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.031s	sys 0.055s
I20260709 13:17:16.112066 10697 catalog_manager.cc:5697] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 4 to 8, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 8 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:16.114488 10916 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:17:16.109306 13828 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.814s	user 0.011s	sys 0.024s
I20260709 13:17:16.147311 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:16.082938 13939 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:16.115530 13910 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.829s	user 0.015s	sys 0.020s
W20260709 13:17:16.170296 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:16.159040 10697 catalog_manager.cc:5697] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 1 to 6, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 6 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:17:16.131784 13717 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.015s	sys 0.022s
I20260709 13:17:16.186139 10697 catalog_manager.cc:5697] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: config changed from index 2 to 4, term changed from 4 to 6, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to <none>, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129), VOTER 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130), VOTER cb0428cc55984daba1ba63744a5102a2 (127.9.116.131), VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 6 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
W20260709 13:17:16.219317 13877 consensus_meta.cc:287] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.913s	user 0.010s	sys 0.005s
I20260709 13:17:16.220078 10946 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 9.
I20260709 13:17:16.151108 13959 raft_consensus.cc:3055] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:16.220791 13965 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:16.149996 13893 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:16.240115 10950 raft_consensus.cc:1240] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:16.161618 13718 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:17:16.238770 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 468256us
I20260709 13:17:16.241993 10950 raft_consensus.cc:1275] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:17:16.161157 13966 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:16.215094 13939 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:16.211158 14004 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:16.252900 11478 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:16.254020 10948 raft_consensus.cc:1240] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:16.305419 10948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 427) took 1001 ms. Trace:
I20260709 13:17:16.305845 10948 rpcz_store.cc:276] 0709 13:17:15.303739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:16.226565 (+922826us) service_pool.cc:224] Handling call
0709 13:17:16.243478 (+ 16913us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:16.305132 (+ 61654us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:16.305357 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:16.255930 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 478903us
I20260709 13:17:16.219848 10697 catalog_manager.cc:5697] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 1 to 5, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 5 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
W20260709 13:17:16.290081 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 106567us
W20260709 13:17:16.307535 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:16.307471 10916 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:16.229593 14003 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:17:16.309484 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 56698.7us
I20260709 13:17:16.307911 10841 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:16.312101 10697 catalog_manager.cc:5697] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 6 to 8, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 8 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:16.316988 11478 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:16.251665 13910 raft_consensus.cc:3055] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:16.232162 13959 raft_consensus.cc:740] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:16.231642 13954 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:16.231740 14008 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:16.288614 13939 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:16.328382 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:16.314427 10941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 421) took 1037 ms. Trace:
I20260709 13:17:16.333647 10941 rpcz_store.cc:276] 0709 13:17:15.277733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:15.713414 (+435681us) service_pool.cc:224] Handling call
0709 13:17:15.713828 (+   414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:15.714033 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:17:15.744142 (+ 30109us) log.cc:844] Serialized 14 byte log entry
0709 13:17:15.803803 (+ 59661us) spinlock_profiling.cc:243] Waited 1.12 ms on lock 0x7b180029aca0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a5523a 00007f9c80ebfa2a
0709 13:17:16.314067 (+510264us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:16.314120 (+    53us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:16.314173 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:16.314236 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:16.314368 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":96138,"prepare.queue_time_us":28778,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":58,"raft.queue_time_us":75228,"spinlock_wait_cycles":2568960,"thread_start_us":440885,"threads_started":1}
W20260709 13:17:16.321344 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 295434us
I20260709 13:17:16.317720 10697 catalog_manager.cc:5697] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:17:16.324883 10841 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:16.338940 10697 catalog_manager.cc:5697] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 3 to 7, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 7 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:16.317764 13893 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:16.310988 13968 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:16.244843 13965 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:16.340054 10766 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:16.365362 10697 catalog_manager.cc:5697] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 5 to 6, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 6 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:16.324361 13910 raft_consensus.cc:740] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:16.371213 10941 raft_consensus.cc:1240] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:16.372160 10697 catalog_manager.cc:5697] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 6 to 9, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 9 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:16.374707 10697 catalog_manager.cc:5697] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 7 to 9, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 9 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:16.377417 10697 catalog_manager.cc:5697] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 5 to 6, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 6 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:16.335799 13966 raft_consensus.cc:697] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:16.330209 14019 raft_consensus.cc:2749] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:17:16.369889 13850 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:16.327905 14009 consensus_queue.cc:1048] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.061s
I20260709 13:17:16.328106 14003 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:16.360478 14008 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:16.512171 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 510809us
I20260709 13:17:16.328920 14004 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:16.608361 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 283567us
I20260709 13:17:16.371269 13982 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:16.416329 10842 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.017s	sys 0.027s
W20260709 13:17:16.624192 10842 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.848s	user 0.028s	sys 0.032s
W20260709 13:17:16.624742 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 848333us
W20260709 13:17:16.573005 14018 log.cc:927] Time spent T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.229s	user 0.000s	sys 0.000s
I20260709 13:17:16.617867 10841 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:16.374323 13968 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:16.387423 13718 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:16.626295 10842 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:16.403155 10941 raft_consensus.cc:1240] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:17:16.642045 10946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36714 (request call id 429) took 1324 ms. Trace:
W20260709 13:17:16.631304 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 378836us
I20260709 13:17:16.388901 13954 raft_consensus.cc:2749] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:17:16.643697 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:16.642572 10946 rpcz_store.cc:276] 0709 13:17:15.304346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:15.308722 (+  4376us) service_pool.cc:224] Handling call
0709 13:17:16.628796 (+1320074us) spinlock_profiling.cc:243] Waited 910 ms on lock 0x7b5400017cd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:16.628913 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2093886336}
I20260709 13:17:16.608145 13719 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:16.654606 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:16.397033 13717 consensus_queue.cc:1048] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.245s
I20260709 13:17:16.539395 13952 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:17:16.651891 14002 log.cc:927] Time spent T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.414s	user 0.000s	sys 0.001s
I20260709 13:17:16.537468 14019 consensus_queue.cc:1059] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.196s
I20260709 13:17:16.639457 13970 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:16.728475 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 111178us
W20260709 13:17:16.729841 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 422421us
I20260709 13:17:16.730350 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:16.640956 13968 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:16.709209 13880 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:16.735631 10841 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:16.634151 13893 consensus_queue.cc:1048] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.092s
I20260709 13:17:16.736205 10941 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:16.653990 14025 raft_consensus.cc:2749] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:16.622175 13850 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:16.727249 14007 log.cc:927] Time spent T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.476s	user 0.001s	sys 0.000s
I20260709 13:17:16.742718 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:16.414950 13959 consensus_queue.cc:260] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:16.720726 14005 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:16.783572 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:16.742002 14015 log.cc:927] Time spent T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.426s	user 0.000s	sys 0.001s
I20260709 13:17:16.803393 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:17:16.797287 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 418727us
I20260709 13:17:16.805500 10795 raft_consensus.cc:2410] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 13:17:16.780956 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:16.741292 14034 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:16.784943 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:16.779713 14013 log.cc:927] Time spent T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.464s	user 0.001s	sys 0.000s
I20260709 13:17:16.740401 13822 raft_consensus.cc:2749] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:16.740566 13968 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:16.738287 13982 raft_consensus.cc:2804] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:17:16.797258 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 469757us
I20260709 13:17:16.828872 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:16.829450 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:16.830128 10800 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 7.
W20260709 13:17:16.731454 13997 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.771s	user 0.011s	sys 0.032s
I20260709 13:17:16.802255 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:16.651166 13865 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:16.743665 13970 raft_consensus.cc:2804] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:16.625769 13910 consensus_queue.cc:260] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:16.803918 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:16.796706 10867 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 9.
W20260709 13:17:16.794574 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 467869us
W20260709 13:17:16.830443 14011 log.cc:927] Time spent T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.458s	user 0.000s	sys 0.001s
I20260709 13:17:16.744110 14030 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:16.822223 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:16.822876 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:16.866324 10796 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:16.823876 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:16.867497 10799 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:17:16.797031 14023 log.cc:927] Time spent T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.303s	user 0.000s	sys 0.001s
W20260709 13:17:16.837939 13997 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.874s	user 0.013s	sys 0.032s
I20260709 13:17:16.771736 13939 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:16.823431 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:16.835027 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:16.876502 10795 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:16.876717 10792 raft_consensus.cc:2410] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 8 because replica has last-logged OpId of term: 7 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:17:16.655591 13966 consensus_queue.cc:237] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:16.835101 11504 raft_consensus.cc:1275] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 8. (term mismatch)
I20260709 13:17:16.879156 11504 pending_rounds.cc:85] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586: Aborting all ops after (but not including) 7
I20260709 13:17:16.879041 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:16.796263 13719 raft_consensus.cc:2749] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:16.749069 13880 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:16.855908 10767 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.116.130:37895 timed out after 2.292s (SENT) [suppressed 16 similar messages]
W20260709 13:17:16.824666 14022 log.cc:927] Time spent T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.490s	user 0.000s	sys 0.001s
W20260709 13:17:16.830581 14020 log.cc:927] Time spent T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.430s	user 0.000s	sys 0.001s
I20260709 13:17:16.854841 13970 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:16.860873 13910 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:16.836120 14017 log.cc:927] Time spent T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.446s	user 0.001s	sys 0.001s
I20260709 13:17:16.862353 14003 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:16.728958 13952 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:16.863498 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:16.935519 10800 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:16.826339 10941 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 10.
I20260709 13:17:16.937233 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:16.938099 10941 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7.
I20260709 13:17:16.821197 13959 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:16.876049 10801 raft_consensus.cc:2393] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 7: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:16.879882 10794 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 8.
I20260709 13:17:16.834162 13982 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:16.941468 13982 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:16.943660 13997 consensus_queue.cc:1059] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:17:16.944463 13997 consensus_queue.cc:1243] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer cb0428cc55984daba1ba63744a5102a2 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:17:16.944938 13997 consensus_queue.cc:1048] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:16.879532 11504 pending_rounds.cc:107] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586: Aborting uncommitted NO_OP operation due to leader change: 5.8
W20260709 13:17:16.809252 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:16.905784 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 49903.8us
I20260709 13:17:16.946101 11504 raft_consensus.cc:2889] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:16.948858 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:16.863422 14025 consensus_queue.cc:1048] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.146s
I20260709 13:17:16.811340 14005 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:16.830744 14034 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:16.957141 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:17.259691 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:17.007956 10795 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 7.
I20260709 13:17:17.489193 10793 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 7.
I20260709 13:17:16.962921 10796 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 10.
I20260709 13:17:17.038218 10792 raft_consensus.cc:1240] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:17.078073 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:17.491801 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:17:17.260385 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 634138us
W20260709 13:17:17.117386 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:17.377460 10944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 419) took 2128 ms. Trace:
W20260709 13:17:17.071089 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 235094us
I20260709 13:17:16.957648 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:17.492246 10941 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 4 FOLLOWER]: Advancing to term 6
W20260709 13:17:17.258780 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 452970us
W20260709 13:17:17.401645 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:17.143514 10800 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 9.
I20260709 13:17:17.495148 10766 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:17.495779 10797 raft_consensus.cc:1240] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:17.496686 10793 raft_consensus.cc:2410] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
W20260709 13:17:17.326666 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:17.443266 10943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 420) took 2169 ms. Trace:
I20260709 13:17:17.505386 10942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 1642) took 2398 ms. Trace:
I20260709 13:17:17.506822 10942 rpcz_store.cc:276] 0709 13:17:15.107389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:15.233751 (+126362us) service_pool.cc:224] Handling call
0709 13:17:15.275211 (+ 41460us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:15.275520 (+   309us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:15.275548 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:15.276034 (+   486us) log.cc:844] Serialized 26 byte log entry
0709 13:17:15.830949 (+554915us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:15.830990 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:15.831062 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:17.505072 (+1674010us) raft_consensus.cc:1685] finished
0709 13:17:17.505099 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:17.505289 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":24660,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":98,"thread_start_us":554512,"threads_started":1,"wal-append.queue_time_us":835253}
W20260709 13:17:17.258687 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:17.508852 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:17:17.306846 11476 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.940s	user 0.005s	sys 0.006s
W20260709 13:17:17.509582 11476 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.149s	user 0.007s	sys 0.006s
I20260709 13:17:17.509526 10795 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 6.
I20260709 13:17:16.958554 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:17.511179 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:17.512291 10871 raft_consensus.cc:2410] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:17:17.258659 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:17.503695 10943 rpcz_store.cc:276] 0709 13:17:15.274901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:15.468798 (+193897us) service_pool.cc:224] Handling call
0709 13:17:15.506043 (+ 37245us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:15.519123 (+ 13080us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:15.519162 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:15.530624 (+ 11462us) log.cc:844] Serialized 26 byte log entry
0709 13:17:16.140024 (+609400us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:16.140083 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:16.140180 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:17.435815 (+1295635us) spinlock_profiling.cc:243] Waited 81.7 ms on lock 0x7b240025b008. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dc4e9c4 00007f9c7dc4cacb 00007f9c7de07dc9 00007f9c7de07a55 00007f9c7de06fb0 00007f9c80a7d488 00007f9c80a7dbe8 00007f9c80a55398 00007f9c80a5076a 00007f9c80a4fbb6
0709 13:17:17.442853 (+  7038us) raft_consensus.cc:1685] finished
0709 13:17:17.442883 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:17.443179 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":791,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":23225,"spinlock_wait_cycles":188001664,"thread_start_us":608910,"threads_started":1,"wal-append.queue_time_us":830538}
W20260709 13:17:16.936375 14010 log.cc:927] Time spent T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.411s	user 0.000s	sys 0.001s
I20260709 13:17:17.488003 10868 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 9.
W20260709 13:17:17.516402 10798 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.004s	sys 0.008s
I20260709 13:17:16.900465 14043 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:17:17.231341 14001 log.cc:927] Time spent T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.814s	user 0.004s	sys 0.000s
I20260709 13:17:16.879005 13822 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:16.921041 13970 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:17.576002 13959 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:17.576598 13959 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:17.577358 13952 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:17.579746 13982 consensus_meta.cc:287] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.638s	user 0.008s	sys 0.000s
I20260709 13:17:16.955950 14026 raft_consensus.cc:3055] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:17.283569 14008 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:16.899596 13717 consensus_queue.cc:1059] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.646s
W20260709 13:17:17.553365 13980 consensus_meta.cc:287] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 1.500s	user 0.011s	sys 0.002s
W20260709 13:17:17.553632 14029 log.cc:927] Time spent T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.652s	user 0.000s	sys 0.001s
I20260709 13:17:17.493672 10944 rpcz_store.cc:276] 0709 13:17:15.248945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:15.463863 (+214918us) service_pool.cc:224] Handling call
0709 13:17:15.464372 (+   509us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:15.464891 (+   519us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:15.464931 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:15.465643 (+   712us) log.cc:844] Serialized 52 byte log entry
0709 13:17:16.019806 (+554163us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:16.019864 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:16.019964 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:17.377206 (+1357242us) raft_consensus.cc:1685] finished
0709 13:17:17.377230 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:17.377393 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4515,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":24560,"thread_start_us":553649,"threads_started":1,"wal-append.queue_time_us":766141}
W20260709 13:17:17.039475 14021 log.cc:927] Time spent T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.411s	user 0.001s	sys 0.001s
I20260709 13:17:17.531387 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:17.432554 14045 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:17.571725 14037 log.cc:927] Time spent T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.187s	user 0.000s	sys 0.000s
I20260709 13:17:17.080662 14041 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:17.572520 14005 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:17.573210 14004 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:17.575124 13880 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:17:17.575284 13985 consensus_meta.cc:287] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 1.838s	user 0.013s	sys 0.004s
I20260709 13:17:17.111720 14033 raft_consensus.cc:3055] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:17:17.522660 10798 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.004s	sys 0.008s
I20260709 13:17:16.955614 14042 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:17.525008 10874 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:17.110826 14040 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:17.522642 13874 raft_consensus.cc:2804] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:17.641315 10942 raft_consensus.cc:1240] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:17.113624 13967 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:17.553084 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:17.642081 10942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2031) took 1874 ms. Trace:
I20260709 13:17:17.037972 13828 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:17.516470 13850 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:16.880299 14030 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:17.598070 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:17.030331 14003 consensus_queue.cc:1048] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.562s
W20260709 13:17:17.603470 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 97118.8us
I20260709 13:17:17.663475 10943 raft_consensus.cc:2393] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 10: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:17.602721 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:17.663635 11477 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:17.664055 10871 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:17.605261 10941 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 6.
I20260709 13:17:17.007630 14049 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:17.578692 13959 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:17.583577 14026 raft_consensus.cc:740] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:17.592825 13949 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:17.626823 10799 consensus_meta.cc:287] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.746s	user 0.011s	sys 0.000s
I20260709 13:17:17.696171 10799 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 10.
I20260709 13:17:17.627260 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:17.696727 11477 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:17.580195 13982 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:17.642416 10942 rpcz_store.cc:276] 0709 13:17:15.767452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:17.613296 (+1845844us) service_pool.cc:224] Handling call
0709 13:17:17.641129 (+ 27833us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:17.641808 (+   679us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:17.642021 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:17.656802 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:17.600818 11503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56410 (request call id 562) took 1835 ms. Trace:
I20260709 13:17:17.592042 13822 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:17.665611 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 150194us
I20260709 13:17:17.708847 10794 raft_consensus.cc:1240] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:17.709481 10799 raft_consensus.cc:1240] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:17.710692 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:17.711355 10916 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:17.711971 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:17.712563 10916 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:17.712605 10801 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:17:17.666198 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:17.714210 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:17.714854 11507 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 6.
I20260709 13:17:17.657856 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:17.690860 10874 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 6.
I20260709 13:17:17.698523 11509 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 9.
I20260709 13:17:17.665030 10941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36714 (request call id 433) took 1284 ms. Trace:
I20260709 13:17:17.616689 14033 raft_consensus.cc:740] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:17.597872 13980 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:17.717978 10799 raft_consensus.cc:1240] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:17.719412 10801 raft_consensus.cc:1240] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:17.719679 13980 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:17.717806 10941 rpcz_store.cc:276] 0709 13:17:16.380913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:16.996472 (+615559us) service_pool.cc:224] Handling call
0709 13:17:17.664964 (+668492us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:17.656664 13874 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:17.612504 13952 consensus_queue.cc:1059] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.085s
I20260709 13:17:17.726408 10943 raft_consensus.cc:1240] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:17.725183 10942 raft_consensus.cc:1240] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 7. Current term is 8. Ops: []
W20260709 13:17:17.680756 14037 log.cc:927] Time spent T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:17:17.727098 10943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2032) took 1959 ms. Trace:
I20260709 13:17:17.727149 10942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2033) took 1959 ms. Trace:
I20260709 13:17:17.743263 10942 rpcz_store.cc:276] 0709 13:17:15.767905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:17.698131 (+1930226us) service_pool.cc:224] Handling call
0709 13:17:17.724944 (+ 26813us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:17.726963 (+  2019us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:17.727102 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:17.717375 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:17.612540 14058 consensus_queue.cc:1048] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.104s
I20260709 13:17:17.728375 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:17:17.752842 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 43900us
I20260709 13:17:17.740688 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:17.754086 10800 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 6.
I20260709 13:17:17.671854 14004 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:17.745826 10876 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:17.746778 10874 raft_consensus.cc:2393] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 7: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:17.702634 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:17.751394 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:17.603245 14043 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:17.743175 10943 rpcz_store.cc:276] 0709 13:17:15.767669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:17.695406 (+1927737us) service_pool.cc:224] Handling call
0709 13:17:17.726214 (+ 30808us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:17.726866 (+   652us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:17.727045 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:17.608750 14041 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:17.722040 13959 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:17.768870 10942 raft_consensus.cc:1275] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:17:17.769445 10942 pending_rounds.cc:85] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 4
I20260709 13:17:17.769693 10942 pending_rounds.cc:107] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 5.5
I20260709 13:17:17.769970 10942 raft_consensus.cc:2889] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:17.770280 10942 pending_rounds.cc:107] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 13:17:17.770510 10942 raft_consensus.cc:2889] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:17.759140 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:17.761010 10871 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 10.
I20260709 13:17:17.723862 14065 raft_consensus.cc:2804] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:17.762040 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:17.772763 11509 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 5.
I20260709 13:17:17.762512 10949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 422) took 2482 ms. Trace:
I20260709 13:17:17.725764 13874 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:17.722929 13717 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:17.671192 13967 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:17.756587 10800 raft_consensus.cc:1275] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:17:17.767652 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:17.767912 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:17.705376 13893 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:17.671325 13828 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:17.771718 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:17.609639 14045 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:17.824448 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:17.771359 14003 consensus_queue.cc:1048] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:17:17.720865 13949 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:17.807353 10874 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:17.807565 10799 raft_consensus.cc:2393] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 10: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:17.773487 10949 rpcz_store.cc:276] 0709 13:17:15.280967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:15.738041 (+457074us) service_pool.cc:224] Handling call
0709 13:17:15.738399 (+   358us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:15.739938 (+  1539us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:17:15.785125 (+ 45187us) log.cc:844] Serialized 14 byte log entry
0709 13:17:16.244549 (+459424us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:16.288972 (+ 44423us) log.cc:844] Serialized 52 byte log entry
0709 13:17:16.305490 (+ 16518us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:16.305532 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:16.305608 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:17.762284 (+1456676us) raft_consensus.cc:1685] finished
0709 13:17:17.762304 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:17.762453 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48948,"prepare.queue_time_us":58456,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":67,"raft.queue_time_us":23341,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":36340,"thread_start_us":294762,"threads_started":1,"wal-append.queue_time_us":807210}
W20260709 13:17:17.794322 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 97673.4us
I20260709 13:17:17.640491 14042 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:17.802769 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:17.804003 10942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2037) took 1001 ms. Trace:
I20260709 13:17:17.803364 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:17.818812 11510 raft_consensus.cc:1275] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:17:17.821921 10876 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 7.
I20260709 13:17:17.665031 14040 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:17.829955 11509 raft_consensus.cc:1240] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:17:17.830413 10800 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 6.
I20260709 13:17:17.772444 14065 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:17.829151 14051 log.cc:927] Time spent T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.311s	user 0.000s	sys 0.001s
I20260709 13:17:17.767302 14004 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:17.853885 10796 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:17.850337 11506 raft_consensus.cc:2410] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:17:17.854303 10942 rpcz_store.cc:276] 0709 13:17:16.802085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:17.744229 (+942144us) service_pool.cc:224] Handling call
0709 13:17:17.766761 (+ 22532us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:17.803651 (+ 36890us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:17.803935 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:17.855552 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:17.855736 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:17.704017 14049 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:17.856216 10874 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 10.
W20260709 13:17:17.838014 13718 consensus_meta.cc:287] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 1.176s	user 0.013s	sys 0.000s
I20260709 13:17:17.808139 13959 raft_consensus.cc:697] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 10 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
W20260709 13:17:17.856691 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 144202us
I20260709 13:17:17.857801 14079 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:17.825340 13717 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:17.824690 13880 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:17.829383 14054 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:17.900622 11508 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
I20260709 13:17:17.899940 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:17.908022 10801 raft_consensus.cc:1275] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:17:17.908324 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:17.832903 13893 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:17.956444 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:17:17.928748 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:17.966585 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:17.972306 10874 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 8.
W20260709 13:17:17.972702 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 71130.6us
I20260709 13:17:17.973753 10916 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:17.861241 14065 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:17.979080 10871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49290 (request call id 406) took 1049 ms. Trace:
I20260709 13:17:17.899219 14003 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:17.954589 14031 log.cc:927] Time spent T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.442s	user 0.000s	sys 0.001s
W20260709 13:17:17.954274 14070 log.cc:927] Time spent T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.192s	user 0.001s	sys 0.000s
W20260709 13:17:17.916455 14037 log.cc:927] Time spent T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 13:17:17.953883 14077 log.cc:927] Time spent T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.283s	user 0.001s	sys 0.000s
I20260709 13:17:17.979900 10871 rpcz_store.cc:276] 0709 13:17:16.929310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:17.904045 (+974735us) service_pool.cc:224] Handling call
0709 13:17:17.979015 (+ 74970us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:17.909368 14079 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:18.003774 11511 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 9.
I20260709 13:17:17.899508 13718 raft_consensus.cc:2749] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:17.955519 14071 raft_consensus.cc:2749] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:17:18.005657 10914 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.000s	sys 0.018s
W20260709 13:17:18.051097 10914 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.000s	sys 0.019s
I20260709 13:17:18.008076 14072 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:17.954419 14059 consensus_queue.cc:1048] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.243s
I20260709 13:17:17.955034 14034 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:18.005486 14079 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:17.907356 14058 consensus_queue.cc:1048] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.098s
W20260709 13:17:18.001669 14052 log.cc:927] Time spent T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.403s	user 0.001s	sys 0.001s
W20260709 13:17:18.023170 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 49532.4us
I20260709 13:17:17.789074 14026 consensus_queue.cc:260] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:18.031184 11478 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:17:18.106271 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 75161.8us
I20260709 13:17:17.953370 13880 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:18.009904 14051 log.cc:927] Time spent T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:17:17.840240 14033 consensus_queue.cc:260] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:18.040410 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:18.057548 13828 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:18.043335 10796 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 7.
W20260709 13:17:18.053378 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:18.005530 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:17.885941 10949 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4"
dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:41006
I20260709 13:17:18.126827 10949 raft_consensus.cc:606] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 6 LEADER]: Received request to transfer leadership
I20260709 13:17:18.036590 14062 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:18.003063 14003 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:18.031526 13982 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:18.089483 10941 raft_consensus.cc:1240] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:18.045207 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:18.104368 10798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1649) took 1305 ms. Trace:
W20260709 13:17:18.104739 11476 thread.cc:641] server-negotiator [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.017s	sys 0.038s
I20260709 13:17:18.113557 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:18.116652 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:18.121464 10871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 1664) took 1066 ms. Trace:
I20260709 13:17:18.047151 10872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1595) took 3541 ms. Trace:
I20260709 13:17:18.037480 13970 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:18.130872 11509 raft_consensus.cc:2368] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 5. Current term is 6.
W20260709 13:17:18.131143 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:18.135588 14033 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:18.108173 14072 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:18.145874 10871 rpcz_store.cc:276] 0709 13:17:17.054976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:18.008147 (+953171us) service_pool.cc:224] Handling call
0709 13:17:18.121383 (+113236us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:18.135666 10798 rpcz_store.cc:276] 0709 13:17:16.798884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:16.799009 (+   125us) service_pool.cc:224] Handling call
0709 13:17:16.799378 (+   369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:16.799718 (+   340us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:16.799747 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:16.808239 (+  8492us) log.cc:844] Serialized 26 byte log entry
0709 13:17:17.617047 (+808808us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:17.617086 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:17.617156 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:18.103858 (+486702us) raft_consensus.cc:1685] finished
0709 13:17:18.103876 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:18.104307 (+   431us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":537,"raft.queue_time_us":120583,"raft.run_cpu_time_us":302,"raft.run_wall_time_us":119500,"thread_start_us":717653,"threads_started":1,"wal-append.queue_time_us":777771}
I20260709 13:17:17.955942 14054 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:18.147531 11476 thread.cc:608] server-negotiator [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.018s	sys 0.039s
W20260709 13:17:18.152457 14075 log.cc:927] Time spent T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.201s	user 0.001s	sys 0.000s
I20260709 13:17:18.152796 14080 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:17:18.154371 14060 log.cc:927] Time spent T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.362s	user 0.000s	sys 0.001s
I20260709 13:17:18.044878 13959 consensus_queue.cc:237] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:18.156539 10875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 401) took 1243 ms. Trace:
I20260709 13:17:18.156092 14057 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:18.132889 14026 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:18.367626 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:18.182348 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 75521.9us
I20260709 13:17:18.222432 10793 raft_consensus.cc:1240] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 6. Current term is 7. Ops: [4.6-6.7]
I20260709 13:17:18.172034 10950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 432) took 2438 ms. Trace:
I20260709 13:17:18.335772 10944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2029) took 2568 ms. Trace:
I20260709 13:17:18.390384 10944 rpcz_store.cc:276] 0709 13:17:15.766914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:17.598548 (+1831634us) service_pool.cc:224] Handling call
0709 13:17:17.615205 (+ 16657us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:17.643588 (+ 28383us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:17.643632 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:17.669554 (+ 25922us) log.cc:844] Serialized 26 byte log entry
0709 13:17:17.909621 (+240067us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:17.909678 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:17.909973 (+   295us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:18.335321 (+425348us) raft_consensus.cc:1685] finished
0709 13:17:18.335364 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:18.335641 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41531,"raft.queue_time_us":151203,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":18681,"thread_start_us":175854,"threads_started":1,"wal-append.queue_time_us":432608}
I20260709 13:17:18.286188 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:17.701146 11503 rpcz_store.cc:276] 0709 13:17:15.765527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:15.771005 (+  5478us) service_pool.cc:224] Handling call
0709 13:17:15.870327 (+ 99322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:16.336815 (+466488us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:16.336867 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:16.401434 (+ 64567us) log.cc:844] Serialized 26 byte log entry
0709 13:17:16.900621 (+499187us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:17.072403 (+171782us) log.cc:844] Serialized 14 byte log entry
0709 13:17:17.112299 (+ 39896us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:17.112416 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:17.580928 (+468512us) raft_consensus.cc:1685] finished
0709 13:17:17.583544 (+  2616us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:17.597783 (+ 14239us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22556,"prepare.queue_time_us":153587,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":267202,"raft.run_cpu_time_us":372,"raft.run_wall_time_us":21734,"thread_start_us":609208,"threads_started":2,"wal-append.queue_time_us":881985}
W20260709 13:17:18.283030 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 176315us
I20260709 13:17:18.368075 10875 rpcz_store.cc:276] 0709 13:17:16.913051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:16.956703 (+ 43652us) service_pool.cc:224] Handling call
0709 13:17:17.281097 (+324394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:17.515662 (+234565us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:17.515692 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:17.516188 (+   496us) log.cc:844] Serialized 26 byte log entry
0709 13:17:17.729289 (+213101us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:17.729340 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:17.729428 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:18.156320 (+426892us) raft_consensus.cc:1685] finished
0709 13:17:18.156338 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:18.156481 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50959,"raft.queue_time_us":221876,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":6907,"spinlock_wait_cycles":34560,"thread_start_us":133278,"threads_started":1,"wal-append.queue_time_us":490960}
I20260709 13:17:18.224661 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:18.132953 13822 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:18.336288 14056 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:18.383255 10950 rpcz_store.cc:276] 0709 13:17:15.733587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:16.279531 (+545944us) service_pool.cc:224] Handling call
0709 13:17:16.280033 (+   502us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:16.364443 (+ 84410us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:16.364486 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:16.609661 (+245175us) log.cc:844] Serialized 26 byte log entry
0709 13:17:17.008203 (+398542us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:17.008253 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:17.008331 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:18.171734 (+1163403us) raft_consensus.cc:1685] finished
0709 13:17:18.171762 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:18.171955 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":91536,"raft.queue_time_us":68494,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":9016,"thread_start_us":353835,"threads_started":1,"wal-append.queue_time_us":820261}
I20260709 13:17:18.406369 10871 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 7.
I20260709 13:17:18.149014 10872 rpcz_store.cc:276] 0709 13:17:14.497914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:14.562200 (+ 64286us) service_pool.cc:224] Handling call
0709 13:17:14.562516 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:14.610417 (+ 47901us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:17:14.982069 (+371652us) spinlock_profiling.cc:243] Waited 5.18 ms on lock 0x7b18003e9680. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc5455 00007f9c848ad3c7 00007f9c80ebf9a7 00007f9c80ecf0bf 00007f9c80ecf038
0709 13:17:16.064302 (+1082233us) log.cc:844] Serialized 14 byte log entry
0709 13:17:16.606739 (+542437us) log.cc:844] Serialized 14 byte log entry
0709 13:17:16.607848 (+  1109us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:16.608326 (+   478us) log.cc:844] Serialized 26 byte log entry
0709 13:17:16.608470 (+   144us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:16.617472 (+  9002us) log.cc:844] Serialized 14 byte log entry
0709 13:17:16.617651 (+   179us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:16.657677 (+ 40026us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:18.038867 (+1381190us) raft_consensus.cc:1685] finished
0709 13:17:18.038887 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:18.039024 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47723,"prepare.queue_time_us":346749,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":243578,"raft.run_cpu_time_us":340,"raft.run_wall_time_us":16238,"spinlock_wait_cycles":32306176,"thread_start_us":1543431,"threads_started":2,"wal-append.queue_time_us":1023425}
W20260709 13:17:18.407313 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:18.414732 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 360859us
I20260709 13:17:18.403146 10916 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:17:18.290827 14087 log.cc:927] Time spent T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.170s	user 0.000s	sys 0.001s
I20260709 13:17:18.432216 11477 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:17:18.338280 13968 consensus_meta.cc:287] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 1.474s	user 0.010s	sys 0.003s
I20260709 13:17:18.439188 13968 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:17:18.402487 13890 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:18.440392 13890 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:18.434479 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:18.197862 14073 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:18.441782 14073 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:18.442351 13890 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:18.435941 14090 log.cc:927] Time spent T 00000000000000000000000000000000 P ae957c8fc66c4b6ba063d0c74cbf5484: Append to log took a long time: real 0.280s	user 0.002s	sys 0.000s
I20260709 13:17:18.171181 14041 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:17:18.435830 14091 log.cc:927] Time spent T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.213s	user 0.000s	sys 0.000s
I20260709 13:17:18.443060 14056 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:18.443670 10872 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 7.
I20260709 13:17:18.150367 14043 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:18.395470 14040 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:18.136845 14062 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:18.170939 13949 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:17:18.442510 13910 consensus_meta.cc:287] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 1.507s	user 0.010s	sys 0.000s
I20260709 13:17:18.456027 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:18.401850 14057 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:18.526373 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:18.396158 13717 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:18.402155 13893 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:18.484732 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:18.520447 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:18.609951 10875 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:18.590338 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 212058us
I20260709 13:17:18.454221 10842 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:18.514190 14060 log.cc:927] Time spent T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 13:17:18.605666 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:17:18.615911 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 161823us
I20260709 13:17:18.616065 10871 raft_consensus.cc:2376] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 7. Re-sending same reply.
I20260709 13:17:18.616741 10842 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:18.619863 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:18.457099 14004 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:18.439191 14042 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:18.591215 13633 raft_consensus.cc:2804] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:18.594169 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:18.623651 10842 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:18.444249 13822 consensus_queue.cc:1059] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.176s
I20260709 13:17:18.614729 14057 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:18.630610 10869 consensus_meta.cc:287] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.525s	user 0.008s	sys 0.000s
I20260709 13:17:18.477984 14058 consensus_queue.cc:1059] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.149s
I20260709 13:17:18.636281 10869 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 10.
I20260709 13:17:18.634603 13890 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:18.638125 14056 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:18.638859 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:18.615381 14003 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:18.621695 13970 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:18.622136 13910 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:18.619233 13717 raft_consensus.cc:2749] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:18.642181 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:18.512058 14094 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:18.659305 10875 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 8.
I20260709 13:17:18.629164 13633 raft_consensus.cc:697] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 6 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:18.639708 14072 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:17:18.645494 13874 consensus_meta.cc:287] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.828s	user 0.009s	sys 0.002s
I20260709 13:17:18.622713 14043 consensus_queue.cc:1059] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:17:18.645239 14073 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:18.642644 13890 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:18.639868 13995 consensus_queue.cc:1048] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
W20260709 13:17:18.697693 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:18.657387 13910 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:18.710412 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:18.668813 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:18.731777 10943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2039) took 1879 ms. Trace:
I20260709 13:17:18.732896 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:17:18.733413 10915 connection.cc:256] RPC call timeout handler was delayed by -1.4467726650000259s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16870000000000002s for a response
I20260709 13:17:18.733716 10875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 1663) took 1696 ms. Trace:
I20260709 13:17:18.734045 10875 rpcz_store.cc:276] 0709 13:17:17.037498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:18.732629 (+1695131us) service_pool.cc:224] Handling call
0709 13:17:18.733642 (+  1013us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:18.732192 10943 rpcz_store.cc:276] 0709 13:17:16.852375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:17.766346 (+913971us) service_pool.cc:224] Handling call
0709 13:17:17.815461 (+ 49115us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:17.815927 (+   466us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:17:17.855325 (+ 39398us) log.cc:844] Serialized 14 byte log entry
0709 13:17:18.054377 (+199052us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:18.055291 (+   914us) log.cc:844] Serialized 26 byte log entry
0709 13:17:18.055438 (+   147us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:18.055467 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:18.055582 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:18.731536 (+675954us) raft_consensus.cc:1685] finished
0709 13:17:18.731554 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:18.731712 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3750,"prepare.queue_time_us":40194,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":142304,"raft.run_cpu_time_us":345,"raft.run_wall_time_us":51018,"thread_start_us":192512,"threads_started":1,"wal-append.queue_time_us":260382}
I20260709 13:17:18.733349 10944 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 6.
W20260709 13:17:18.745025 14103 log.cc:927] Time spent T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 13:17:18.746543 14062 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:17:18.747269 14101 log.cc:927] Time spent T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:17:18.748607 10942 raft_consensus.cc:1240] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:17:18.709311 13890 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:18.749936 14065 consensus_meta.cc:287] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.746s	user 0.010s	sys 0.000s
I20260709 13:17:18.666862 14094 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:18.750928 14054 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:18.751353 14094 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:18.709311 13874 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:18.755791 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:18.756456 10871 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 7.
I20260709 13:17:18.756659 10948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2026) took 2995 ms. Trace:
I20260709 13:17:18.757699 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:18.758538 10875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 1610) took 1030 ms. Trace:
I20260709 13:17:18.759019 10875 rpcz_store.cc:276] 0709 13:17:17.728181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:18.757461 (+1029280us) service_pool.cc:224] Handling call
0709 13:17:18.758479 (+  1018us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:18.730149 13910 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:18.750355 14065 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:18.772393 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 327545us
I20260709 13:17:18.777848 10944 raft_consensus.cc:1275] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:17:18.757318 10948 rpcz_store.cc:276] 0709 13:17:15.761349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:16.316926 (+555577us) service_pool.cc:224] Handling call
0709 13:17:16.333944 (+ 17018us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:16.401000 (+ 67056us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:17:16.791770 (+390770us) log.cc:844] Serialized 14 byte log entry
0709 13:17:17.057512 (+265742us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:17.283063 (+225551us) log.cc:844] Serialized 26 byte log entry
0709 13:17:17.283235 (+   172us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:17.283474 (+   239us) log.cc:844] Serialized 14 byte log entry
0709 13:17:17.420810 (+137336us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:17.420921 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:18.756472 (+1335551us) raft_consensus.cc:1685] finished
0709 13:17:18.756492 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:18.756608 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":66474,"prepare.queue_time_us":198454,"prepare.run_cpu_time_us":107,"prepare.run_wall_time_us":110,"raft.queue_time_us":413356,"raft.run_cpu_time_us":391,"raft.run_wall_time_us":39847,"thread_start_us":644346,"threads_started":3,"wal-append.queue_time_us":606211}
I20260709 13:17:18.782621 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:18.783285 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:18.791250 10871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 1608) took 1066 ms. Trace:
I20260709 13:17:18.791800 10871 rpcz_store.cc:276] 0709 13:17:17.725066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:18.770376 (+1045310us) service_pool.cc:224] Handling call
0709 13:17:18.791155 (+ 20779us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:18.794518 14113 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:18.795647 14113 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:18.797510 14114 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:18.799070 14113 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:18.796819 14115 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:18.779156 14054 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:18.805454 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:18.807075 10871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 1615) took 1025 ms. Trace:
I20260709 13:17:18.807431 10871 rpcz_store.cc:276] 0709 13:17:17.781506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:18.792653 (+1011147us) service_pool.cc:224] Handling call
0709 13:17:18.807011 (+ 14358us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:18.811878 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:18.812819 11509 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 10.
I20260709 13:17:18.812901 10949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:41006 (request call id 76) took 3536 ms. Trace:
I20260709 13:17:18.815670 10949 rpcz_store.cc:276] 0709 13:17:15.276270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:17.856614 (+2580344us) service_pool.cc:224] Handling call
0709 13:17:18.812427 (+955813us) spinlock_profiling.cc:243] Waited 20.5 ms on lock 0x7b5400313e20. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80e32520 00007f9c80eaf652 00007f9c85e7d60a 00007f9c7f4037d4 00007f9c7f403762 00007f9c7f403691 00007f9c7f40360c 00007f9c7f4028c2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:18.812780 (+   353us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":47260416}
I20260709 13:17:18.813961 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:18.817978 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:18.820007 11509 raft_consensus.cc:1240] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 8. Current term is 10. Ops: []
I20260709 13:17:18.837976 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:18.841183 14102 log.cc:927] Time spent T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
W20260709 13:17:18.842947 14033 consensus_meta.cc:287] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.691s	user 0.009s	sys 0.002s
W20260709 13:17:18.845388 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:18.845966 10796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39442 (request call id 351) took 1540 ms. Trace:
I20260709 13:17:18.847427 11504 raft_consensus.cc:2410] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:17:18.846374 10796 rpcz_store.cc:276] 0709 13:17:17.305594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:17.515630 (+210036us) service_pool.cc:224] Handling call
0709 13:17:18.845784 (+1330154us) spinlock_profiling.cc:243] Waited 211 ms on lock 0x7b540024e5d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:18.845906 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":486269312}
W20260709 13:17:18.853011 14079 consensus_meta.cc:287] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.746s	user 0.009s	sys 0.002s
I20260709 13:17:18.853603 14079 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:18.790493 13633 consensus_queue.cc:237] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:18.820914 14054 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:18.861526 10915 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:18.865397 10915 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:18.859457 13829 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:18.804131 14114 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:18.868547 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:18.865675 13877 raft_consensus.cc:2749] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:18.869874 10948 raft_consensus.cc:1275] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:17:18.876228 10944 raft_consensus.cc:1240] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:18.878373 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:18.879222 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:18.879703 10949 raft_consensus.cc:2393] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 7: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:18.864748 13874 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:18.880797 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:18.862205 13936 consensus_queue.cc:1059] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 13:17:18.883909 13936 raft_consensus.cc:3055] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:18.884230 13936 raft_consensus.cc:740] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:18.834937 14115 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:18.886140 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:18.887444 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:18.888437 11478 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:18.893179 14041 raft_consensus.cc:2749] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:18.893512 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:18.891642 10842 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:18.894178 10801 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 9.
W20260709 13:17:18.896286 14027 log.cc:927] Time spent T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 2.180s	user 0.001s	sys 0.006s
I20260709 13:17:18.897012 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:18.897655 10792 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 10.
I20260709 13:17:18.898294 14109 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:18.898842 14109 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:18.899540 14115 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:18.900974 14109 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:18.904477 11510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56416 (request call id 561) took 1283 ms. Trace:
I20260709 13:17:18.891376 10798 raft_consensus.cc:1275] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 6 index: 2. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260709 13:17:18.904954 11510 rpcz_store.cc:276] 0709 13:17:17.620927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:17.621042 (+   115us) service_pool.cc:224] Handling call
0709 13:17:17.657469 (+ 36427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:17.849344 (+191875us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:18.904237 (+1054893us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b54004e5554. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:18.904410 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":369918208}
I20260709 13:17:18.904161 13890 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:18.876654 14116 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:18.925905 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:18.924412 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:18.924612 10949 raft_consensus.cc:2393] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 9: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:18.930608 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:18.931106 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:18.936398 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:18.936303 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:18.936753 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:18.937238 10796 raft_consensus.cc:2393] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 7: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:18.937872 10949 raft_consensus.cc:2393] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 10: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:18.937075 10948 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 7.
I20260709 13:17:18.940225 10798 raft_consensus.cc:1275] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:17:18.943073 10871 raft_consensus.cc:2368] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 7. Current term is 8.
I20260709 13:17:18.909667 14114 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:18.929388 14117 raft_consensus.cc:2804] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:18.946676 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:18.923452 13995 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:18.876680 13829 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:17:18.947244 10792 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7.
I20260709 13:17:18.948576 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:18.924619 13717 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:18.932541 14119 log.cc:927] Time spent T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:17:18.951215 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:18.952253 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:18.956709 10796 raft_consensus.cc:2393] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 7: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:18.934929 14116 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:18.960462 14126 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:18.963387 14128 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:18.946254 14117 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:18.971266 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:18.885072 13936 consensus_queue.cc:260] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:18.998109 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:19.026657 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:18.983920 11509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56404 (request call id 614) took 1029 ms. Trace:
I20260709 13:17:19.004220 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:19.001973 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:19.042172 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:19.036327 11509 rpcz_store.cc:276] 0709 13:17:17.954106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:18.831644 (+877538us) service_pool.cc:224] Handling call
0709 13:17:18.832519 (+   875us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:18.832803 (+   284us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:17:18.837809 (+  5006us) log.cc:844] Serialized 14 byte log entry
0709 13:17:18.983421 (+145612us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:18.983500 (+    79us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:17:18.983578 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:18.983673 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:18.983840 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2257,"prepare.queue_time_us":6541,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":8684,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":1673,"thread_start_us":145121,"threads_started":1}
I20260709 13:17:19.048434 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:18.974278 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:19.052090 10949 raft_consensus.cc:2393] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 10: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:19.049428 11509 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 7.
I20260709 13:17:19.057555 14128 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:19.060820 14128 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:19.061626 14128 consensus_queue.cc:1048] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:17:19.070628 11505 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56416 (request call id 566) took 1431 ms (client timeout 1647 ms). Trace:
W20260709 13:17:19.005250 14124 log.cc:927] Time spent T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:17:19.077790 14126 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:19.028331 13829 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:19.079062 13967 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:17:19.071368 11505 rpcz_store.cc:269] 0709 13:17:17.639401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:17.639541 (+   140us) service_pool.cc:224] Handling call
0709 13:17:19.003998 (+1364457us) spinlock_profiling.cc:243] Waited 8.2 ms on lock 0x7b5800003c98. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd90a22 00007f9c86ca9c09 00007f9c85e03ee0 00007f9c85ecd0ab 00007f9c85ecd131 00007f9c85e7b0a0 00007f9c85e7a984 00007f9c7f3f56d4 00007f9c7f3f5662
0709 13:17:19.070548 (+ 66550us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18853760}
I20260709 13:17:18.970129 14054 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:18.971287 14116 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:18.970839 14117 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:19.087984 10799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 356) took 1056 ms. Trace:
I20260709 13:17:19.025527 14079 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:19.088491 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 90492.4us
I20260709 13:17:19.088380 10799 rpcz_store.cc:276] 0709 13:17:18.031340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:18.031513 (+   173us) service_pool.cc:224] Handling call
0709 13:17:18.031921 (+   408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:18.032204 (+   283us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:17:18.048653 (+ 16449us) log.cc:844] Serialized 14 byte log entry
0709 13:17:19.006141 (+957488us) spinlock_profiling.cc:243] Waited 1.74 ms on lock 0x7b180010ea00. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a5523a 00007f9c80ebfa2a
0709 13:17:19.087642 (+ 81501us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:19.087700 (+    58us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:19.087754 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:19.087818 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:19.087936 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":16286,"prepare.run_cpu_time_us":3185,"prepare.run_wall_time_us":30205,"raft.queue_time_us":116,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":84,"spinlock_wait_cycles":4065792,"thread_start_us":81218,"threads_started":1,"wal-append.queue_time_us":1038956}
I20260709 13:17:19.092320 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:19.095007 11509 raft_consensus.cc:2410] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 8 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:17:18.965798 13995 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:17:19.095592 14054 consensus_queue.cc:1048] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:19.098174 14030 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:19.099120 14054 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:19.099646 14054 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:19.102029 10842 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:18.964229 13717 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:19.102175 11477 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:19.093088 13936 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:19.109169 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:19.109483 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:19.109417 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:19.110008 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 399) took 2200 ms. Trace:
I20260709 13:17:19.111155 10873 rpcz_store.cc:276] 0709 13:17:16.912288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:16.912455 (+   167us) service_pool.cc:224] Handling call
0709 13:17:17.281128 (+368673us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:17.725641 (+444513us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:17.725686 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:17.741732 (+ 16046us) log.cc:844] Serialized 26 byte log entry
0709 13:17:17.834306 (+ 92574us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:17.834360 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:19.101166 (+1266806us) spinlock_profiling.cc:243] Waited 440 ms on lock 0x7b540019ffd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:19.101215 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:19.101266 (+    51us) raft_consensus.cc:1685] finished
0709 13:17:19.101278 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:19.109955 (+  8677us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3862,"raft.queue_time_us":20761,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":14037,"spinlock_wait_cycles":1012448896,"thread_start_us":46407,"threads_started":1,"wal-append.queue_time_us":94086,"wal-append.run_cpu_time_us":1977,"wal-append.run_wall_time_us":1143316}
I20260709 13:17:19.112872 10949 raft_consensus.cc:3055] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:19.113128 10949 raft_consensus.cc:740] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:19.117432 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:19.118152 10873 raft_consensus.cc:2393] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 10: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:19.113639 10949 consensus_queue.cc:260] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:19.119647 10797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 348) took 1860 ms. Trace:
I20260709 13:17:19.119853 10949 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:19.119944 10841 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:19.120064 10797 rpcz_store.cc:276] 0709 13:17:17.259411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:17.259542 (+   131us) service_pool.cc:224] Handling call
0709 13:17:17.495562 (+236020us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:17.496222 (+   660us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:19.119462 (+1623240us) spinlock_profiling.cc:243] Waited 236 ms on lock 0x7b54002120d0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:19.119592 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":542043520}
I20260709 13:17:19.106561 14028 raft_consensus.cc:2804] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:19.107306 14026 raft_consensus.cc:2749] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:19.122455 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 1618) took 1309 ms. Trace:
I20260709 13:17:19.118898 14054 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:19.107291 14079 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:19.118954 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:19.130893 10942 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 7.
I20260709 13:17:19.125609 10873 rpcz_store.cc:276] 0709 13:17:17.812681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:19.117128 (+1304447us) service_pool.cc:224] Handling call
0709 13:17:19.122296 (+  5168us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:19.094627 14094 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:19.124519 14028 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:19.128089 13910 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:19.115121 13890 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:19.123895 14128 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:17:19.138509 10876 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49290 (request call id 404) took 2213 ms (client timeout 2151 ms). Trace:
I20260709 13:17:19.151404 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:19.155930 10871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 1612) took 1404 ms. Trace:
I20260709 13:17:19.137030 13985 raft_consensus.cc:3055] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:19.158540 10942 raft_consensus.cc:1240] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:19.191171 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 1617) took 1390 ms. Trace:
W20260709 13:17:19.191296 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 72782.7us
I20260709 13:17:19.191864 10871 rpcz_store.cc:276] 0709 13:17:17.751123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:18.820711 (+1069588us) service_pool.cc:224] Handling call
0709 13:17:19.155686 (+334975us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b54000f73d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:19.155868 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":239310720}
I20260709 13:17:19.123667 14057 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:19.190541 13985 raft_consensus.cc:740] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:17:19.191604 10876 rpcz_store.cc:269] 0709 13:17:16.924979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:16.957373 (+ 32394us) service_pool.cc:224] Handling call
0709 13:17:19.138265 (+2180892us) spinlock_profiling.cc:243] Waited 251 ms on lock 0x7b54001e0354. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:19.138404 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":577770880}
I20260709 13:17:19.193234 10873 rpcz_store.cc:276] 0709 13:17:17.800924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:19.150736 (+1349812us) service_pool.cc:224] Handling call
0709 13:17:19.191117 (+ 40381us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:19.153702 14028 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:19.150450 13822 raft_consensus.cc:2749] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:19.198637 10841 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:19.207160 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:19.150578 14040 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:19.215731 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 193033us
W20260709 13:17:19.216368 10915 leader_election.cc:336] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Timed out: RequestConsensusVote RPC to 127.9.116.132:40555 timed out after 1.768s (SENT)
I20260709 13:17:19.216704 10915 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:17:19.204842 13961 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.407s	user 0.014s	sys 0.012s
I20260709 13:17:19.211784 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:19.212248 11477 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:19.218631 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:19.219676 10876 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:19.223006 13890 consensus_queue.cc:1059] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:19.223810 13890 consensus_queue.cc:1243] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer cb0428cc55984daba1ba63744a5102a2 log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:17:19.223886 13995 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:19.217079 14133 raft_consensus.cc:2749] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:19.225795 14057 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:19.190290 14126 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:19.226536 13985 consensus_queue.cc:260] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:19.227269 13717 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:19.227795 13985 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:19.228127 14136 log.cc:927] Time spent T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 13:17:19.227947 14026 consensus_queue.cc:1059] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:17:19.228892 14026 raft_consensus.cc:3055] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:19.229213 14026 raft_consensus.cc:740] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:19.230068 14026 consensus_queue.cc:260] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:19.231489 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:19.232437 10943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2043) took 1107 ms. Trace:
I20260709 13:17:19.139987 14079 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:19.234256 10871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 1619) took 1386 ms. Trace:
I20260709 13:17:19.240046 10871 rpcz_store.cc:276] 0709 13:17:17.847588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:19.194852 (+1347264us) service_pool.cc:224] Handling call
0709 13:17:19.234197 (+ 39345us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:19.239876 10943 rpcz_store.cc:276] 0709 13:17:18.125476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:18.737267 (+611791us) service_pool.cc:224] Handling call
0709 13:17:18.774579 (+ 37312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:18.838284 (+ 63705us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:18.838330 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:18.855334 (+ 17004us) log.cc:844] Serialized 26 byte log entry
0709 13:17:19.024921 (+169587us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:19.024975 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:19.025070 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:19.232221 (+207151us) raft_consensus.cc:1685] finished
0709 13:17:19.232241 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:19.232379 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":22297,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":75,"thread_start_us":218908,"threads_started":2,"wal-append.queue_time_us":261883}
I20260709 13:17:19.218768 14094 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:19.235929 13910 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:19.239969 10867 raft_consensus.cc:2376] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 6. Re-sending same reply.
I20260709 13:17:19.217917 10942 raft_consensus.cc:1275] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:17:19.214833 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 1623) took 1108 ms. Trace:
I20260709 13:17:19.258890 10873 rpcz_store.cc:276] 0709 13:17:18.106460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:19.206823 (+1100363us) service_pool.cc:224] Handling call
0709 13:17:19.214740 (+  7917us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:19.242399 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:19.243058 10876 raft_consensus.cc:2410] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:17:19.261513 10841 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:19.248396 14079 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:19.262562 10867 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49290 (request call id 398) took 2358 ms (client timeout 1913 ms). Trace:
W20260709 13:17:19.252086 11511 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56416 (request call id 573) took 1495 ms (client timeout 1571 ms). Trace:
I20260709 13:17:19.223299 14033 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:19.244338 13982 raft_consensus.cc:3055] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:19.248986 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:19.263557 11511 rpcz_store.cc:269] 0709 13:17:17.756792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:17.768682 (+ 11890us) service_pool.cc:224] Handling call
0709 13:17:19.251847 (+1483165us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b540046d2d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:19.252006 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":355155712}
I20260709 13:17:19.264194 10871 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 6.
I20260709 13:17:19.264839 10871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 1621) took 1268 ms. Trace:
W20260709 13:17:19.263345 10867 rpcz_store.cc:269] 0709 13:17:16.904241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:16.917328 (+ 13087us) service_pool.cc:224] Handling call
0709 13:17:19.231334 (+2314006us) spinlock_profiling.cc:243] Waited 21.5 ms on lock 0x7b5800177098. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd90a22 00007f9c86ca9c09 00007f9c85e03ee0 00007f9c85ecd0ab 00007f9c85ecd131 00007f9c85e7b0a0 00007f9c85e7a984 00007f9c7f3f56d4 00007f9c7f3f5662
0709 13:17:19.262480 (+ 31146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":49343872}
I20260709 13:17:19.265129 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:19.265292 10871 rpcz_store.cc:276] 0709 13:17:17.996617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:19.241198 (+1244581us) service_pool.cc:224] Handling call
0709 13:17:19.264785 (+ 23587us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:19.266462 10876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 1602) took 1544 ms. Trace:
I20260709 13:17:19.266891 10876 rpcz_store.cc:276] 0709 13:17:17.721906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:19.209335 (+1487429us) service_pool.cc:224] Handling call
0709 13:17:19.266405 (+ 57070us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:19.267724 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:19.268308 10876 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 7.
W20260709 13:17:19.218729 13961 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.438s	user 0.015s	sys 0.012s
I20260709 13:17:19.271870 11510 raft_consensus.cc:1240] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 9 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:19.272768 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:19.206526 14128 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:19.273818 10871 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 9.
I20260709 13:17:19.256991 13935 raft_consensus.cc:2749] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:19.279645 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:19.261426 14026 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:19.260499 13822 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:19.219806 14040 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:19.269987 14033 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:19.270203 13982 raft_consensus.cc:740] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:19.304247 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:19.309334 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:19.260361 14024 consensus_queue.cc:1059] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 13:17:19.395169 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 1616) took 1604 ms. Trace:
W20260709 13:17:19.395864 10766 connection.cc:256] RPC call timeout handler was delayed by -1.3943404660001306s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1647s for a response
I20260709 13:17:19.395632 10867 rpcz_store.cc:276] 0709 13:17:17.790375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:19.279817 (+1489442us) service_pool.cc:224] Handling call
0709 13:17:19.395094 (+115277us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:19.263044 14133 consensus_queue.cc:1048] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.119s
I20260709 13:17:19.274863 14135 raft_consensus.cc:2804] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:17:19.396308 10766 connection.cc:256] RPC call timeout handler was delayed by -1.3312125970001034s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17720000000000002s for a response
I20260709 13:17:19.412196 14135 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:19.402462 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:19.412817 14135 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:19.413092 10867 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 10.
I20260709 13:17:19.414090 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:19.415177 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:19.415925 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 1613) took 1650 ms. Trace:
I20260709 13:17:19.416234 10867 rpcz_store.cc:276] 0709 13:17:17.765800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:19.414941 (+1649141us) service_pool.cc:224] Handling call
0709 13:17:19.415868 (+   927us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:19.379484 14033 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:19.274029 14134 consensus_queue.cc:1059] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.142s
I20260709 13:17:19.283329 13910 consensus_queue.cc:1048] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.113s
W20260709 13:17:19.412251 10766 connection.cc:256] RPC call timeout handler was delayed by -1.2826125970002522s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1826s for a response
W20260709 13:17:19.424860 10766 connection.cc:256] RPC call timeout handler was delayed by -1.2034125970001242s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19140000000000001s for a response
W20260709 13:17:19.425251 10766 connection.cc:256] RPC call timeout handler was delayed by -1.1827125970000907s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1937s for a response
I20260709 13:17:19.349314 13961 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:19.407229 14040 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:19.426247 13961 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:19.430593 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:19.431222 10867 raft_consensus.cc:2376] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 6. Re-sending same reply.
W20260709 13:17:19.413108 14016 log.cc:927] Time spent T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 2.857s	user 0.000s	sys 0.012s
I20260709 13:17:19.437920 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:19.439541 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:19.440416 10867 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 7.
I20260709 13:17:19.441507 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:19.442175 10867 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 8.
W20260709 13:17:19.444442 11504 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56404 (request call id 616) took 1489 ms (client timeout 1687 ms). Trace:
W20260709 13:17:19.444906 11504 rpcz_store.cc:269] 0709 13:17:17.954957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:18.116311 (+161354us) service_pool.cc:224] Handling call
0709 13:17:19.444230 (+1327919us) spinlock_profiling.cc:243] Waited 701 ms on lock 0x7b5400467654. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:19.444376 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1612840320}
I20260709 13:17:19.445992 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:19.448201 14049 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:19.451200 10945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2023) took 4227 ms. Trace:
I20260709 13:17:19.451583 10945 rpcz_store.cc:276] 0709 13:17:15.223641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:15.278304 (+ 54663us) service_pool.cc:224] Handling call
0709 13:17:15.278612 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:15.386278 (+107666us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:15.386323 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:15.386960 (+   637us) log.cc:844] Serialized 26 byte log entry
0709 13:17:16.040347 (+653387us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:16.066225 (+ 25878us) log.cc:844] Serialized 14 byte log entry
0709 13:17:16.066640 (+   415us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:18.121065 (+2054425us) spinlock_profiling.cc:243] Waited 4.01 ms on lock 0x7b18001a2300. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7dd962a6 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419
0709 13:17:18.438050 (+316985us) spinlock_profiling.cc:243] Waited 6.42 ms on lock 0x7b18001a2300. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc83f3 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:17:19.245882 (+807832us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b54001e2154. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:19.245929 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:19.395851 (+149922us) spinlock_profiling.cc:243] Waited 16 ms on lock 0x7b240025b1b8. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dc4e9c4 00007f9c7dc4cacb 00007f9c7de07dc9 00007f9c7de07a55 00007f9c7de0709d 00007f9c80a3910e 00007f9c80a5088c 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221
0709 13:17:19.450992 (+ 55141us) raft_consensus.cc:1685] finished
0709 13:17:19.451013 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:19.451127 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":25570,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":43196,"raft.run_cpu_time_us":229,"raft.run_wall_time_us":13187,"spinlock_wait_cycles":307390848,"thread_start_us":630535,"threads_started":1,"wal-append.queue_time_us":967913}
I20260709 13:17:19.453761 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:19.455969 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:19.456791 10875 raft_consensus.cc:2410] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:17:19.457541 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:19.458989 14028 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:19.461139 14028 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:19.378655 13822 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:19.465489 10793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 353) took 2093 ms. Trace:
I20260709 13:17:19.465888 10793 rpcz_store.cc:276] 0709 13:17:17.373133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:17.498216 (+125083us) service_pool.cc:224] Handling call
0709 13:17:17.498672 (+   456us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:18.381167 (+882495us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:19.465214 (+1084047us) spinlock_profiling.cc:243] Waited 700 ms on lock 0x7b5400294d54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:19.465428 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1610221312}
I20260709 13:17:19.466805 13822 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:19.467705 10869 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 1652) took 2725 ms (client timeout 1964 ms). Trace:
W20260709 13:17:19.468102 10869 rpcz_store.cc:269] 0709 13:17:16.741891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:16.742381 (+   490us) service_pool.cc:224] Handling call
0709 13:17:19.467539 (+2725158us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b5400340154. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:19.467638 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2950090496}
I20260709 13:17:19.469456 10869 raft_consensus.cc:1240] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:17:19.470033 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1656) took 2539 ms. Trace:
I20260709 13:17:19.470323 10869 rpcz_store.cc:276] 0709 13:17:16.930219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:19.468879 (+2538660us) service_pool.cc:224] Handling call
0709 13:17:19.469237 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:19.469812 (+   575us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:19.469981 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:19.471334 14128 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:19.471967 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:19.474876 14117 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:19.476109 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:19.499737 11512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56416 (request call id 565) took 1868 ms. Trace:
I20260709 13:17:19.478395 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:19.478681 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:19.479882 11510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56404 (request call id 617) took 1524 ms. Trace:
I20260709 13:17:19.481767 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:19.482352 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1662) took 2506 ms. Trace:
I20260709 13:17:19.482564 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:19.484316 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:19.459040 10867 raft_consensus.cc:1240] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:19.446627 11504 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7.
I20260709 13:17:19.458572 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:19.412834 14129 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:19.497309 10766 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:19.500243 11512 rpcz_store.cc:276] 0709 13:17:17.631867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:17.632541 (+   674us) service_pool.cc:224] Handling call
0709 13:17:19.475901 (+1843360us) spinlock_profiling.cc:243] Waited 8.36 ms on lock 0x7b5800003c98. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd90a22 00007f9c86ca9c09 00007f9c85e03ee0 00007f9c85ecd0ab 00007f9c85ecd131 00007f9c85e7b0a0 00007f9c85e7a984 00007f9c7f3f56d4 00007f9c7f3f5662
0709 13:17:19.499652 (+ 23751us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19220352}
I20260709 13:17:19.500998 11510 rpcz_store.cc:276] 0709 13:17:17.955263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:18.905922 (+950659us) service_pool.cc:224] Handling call
0709 13:17:18.929138 (+ 23216us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:19.272410 (+343272us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:19.479559 (+207149us) spinlock_profiling.cc:243] Waited 343 ms on lock 0x7b54004335d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:19.479809 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":787768576}
I20260709 13:17:19.501399 10799 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
I20260709 13:17:19.514103 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:19.515142 10799 raft_consensus.cc:2393] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 10: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:19.501538 10873 rpcz_store.cc:276] 0709 13:17:16.975892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:19.481187 (+2505295us) service_pool.cc:224] Handling call
0709 13:17:19.481533 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:19.481925 (+   392us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:19.481964 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:19.482020 (+    56us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:19.482074 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:19.482146 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:19.482284 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13136}
I20260709 13:17:19.501914 10792 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:17:19.502825 10800 raft_consensus.cc:1240] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:17:19.503050 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 407) took 2573 ms. Trace:
I20260709 13:17:19.518260 10867 rpcz_store.cc:276] 0709 13:17:16.929626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:19.458415 (+2528789us) service_pool.cc:224] Handling call
0709 13:17:19.458809 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:19.502723 (+ 43914us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:19.502981 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:19.520171 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:19.504326 10875 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10.
I20260709 13:17:19.500375 10869 raft_consensus.cc:1240] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:19.524281 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1600) took 1803 ms. Trace:
I20260709 13:17:19.527559 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:19.528592 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:19.531492 10800 raft_consensus.cc:2393] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 7: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:19.535009 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:19.537134 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:19.537705 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1605) took 1813 ms. Trace:
I20260709 13:17:19.538254 10867 rpcz_store.cc:276] 0709 13:17:17.724320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:19.519321 (+1795001us) service_pool.cc:224] Handling call
0709 13:17:19.525994 (+  6673us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:19.537380 (+ 11386us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:19.537413 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:19.537454 (+    41us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:19.537491 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:19.537551 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:19.537650 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11059}
I20260709 13:17:19.540773 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:19.538318 11512 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
I20260709 13:17:19.484809 13961 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:19.528954 10799 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7.
I20260709 13:17:19.544842 10867 raft_consensus.cc:1240] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:19.549153 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1625) took 1351 ms. Trace:
I20260709 13:17:19.549669 10867 rpcz_store.cc:276] 0709 13:17:18.197793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:19.539548 (+1341755us) service_pool.cc:224] Handling call
0709 13:17:19.544602 (+  5054us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:19.548824 (+  4222us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:19.549082 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:19.552809 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:19.553570 10870 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49290 (request call id 400) took 2640 ms (client timeout 2112 ms). Trace:
W20260709 13:17:19.553905 10870 rpcz_store.cc:269] 0709 13:17:16.912717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:16.912922 (+   205us) service_pool.cc:224] Handling call
0709 13:17:19.552647 (+2639725us) spinlock_profiling.cc:243] Waited 21 ms on lock 0x7b5800177098. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd90a22 00007f9c86ca9c09 00007f9c85e03ee0 00007f9c85ecd0ab 00007f9c85ecd131 00007f9c85e7b0a0 00007f9c85e7a984 00007f9c7f3f56d4 00007f9c7f3f5662
0709 13:17:19.553502 (+   855us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":48257280}
I20260709 13:17:19.525591 14049 consensus_queue.cc:1059] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 13:17:19.474664 14033 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:19.501082 13874 consensus_queue.cc:1048] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 13:17:19.564922 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:19.521339 11504 raft_consensus.cc:2393] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 7: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:19.558120 10867 raft_consensus.cc:1275] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:17:19.566803 10867 pending_rounds.cc:85] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 3
I20260709 13:17:19.567070 10867 pending_rounds.cc:107] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:17:19.562052 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:19.567317 10867 raft_consensus.cc:2889] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:17:19.568437 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:19.527360 10869 rpcz_store.cc:276] 0709 13:17:17.721350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:19.499793 (+1778443us) service_pool.cc:224] Handling call
0709 13:17:19.500150 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:19.523944 (+ 23794us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:19.524199 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:19.570528 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:19.567626 11510 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 6.
I20260709 13:17:19.573134 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:19.573873 10869 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:19.576596 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:19.586853 10870 raft_consensus.cc:1275] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:17:19.587986 10799 raft_consensus.cc:1240] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 8. Current term is 9. Ops: [8.3-8.3]
I20260709 13:17:19.528939 14129 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:19.591102 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:19.591961 11512 raft_consensus.cc:2393] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 7: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:19.592049 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:19.592559 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:19.593096 10800 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:17:19.536327 14116 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:19.596622 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:19.597467 11510 raft_consensus.cc:2410] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:17:19.595000 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:19.599288 10867 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:19.598610 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:19.595217 10792 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
I20260709 13:17:19.607594 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:17:19.613955 14141 log.cc:927] Time spent T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:17:19.617450 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:19.623404 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:19.625087 14129 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:19.618170 10949 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 7.
I20260709 13:17:19.654469 10869 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 7.
I20260709 13:17:19.662246 10948 raft_consensus.cc:2393] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 11: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
W20260709 13:17:19.662285 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 165063us
I20260709 13:17:19.663352 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:19.674273 10800 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
W20260709 13:17:19.687565 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 200294us
I20260709 13:17:19.688845 10767 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:19.703059 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:19.703974 10949 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:19.662068 13936 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:19.763314 13950 raft_consensus.cc:993] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2: : Instructing follower 12682eb70bc84c76879e04a8628ee586 to start an election
I20260709 13:17:19.770493 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:19.770748 13950 raft_consensus.cc:1081] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 6 LEADER]: Signalling peer 12682eb70bc84c76879e04a8628ee586 to start an election
I20260709 13:17:19.771154 10942 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:19.772141 14065 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:19.767416 13880 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:19.771085 13874 raft_consensus.cc:3055] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:19.773849 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:19.774400 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:19.775341 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:19.777087 10867 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10.
I20260709 13:17:19.775748 11512 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 6.
I20260709 13:17:19.775080 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:17:19.774952 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:19.779672 10870 raft_consensus.cc:2393] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 7: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:19.781247 11510 raft_consensus.cc:2393] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 9: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:19.778280 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:19.784036 10946 raft_consensus.cc:2410] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:17:19.783866 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:19.792914 10949 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 7.
W20260709 13:17:19.784451 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 244889us
I20260709 13:17:19.785336 10867 raft_consensus.cc:1275] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:17:19.789731 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:19.798820 10870 raft_consensus.cc:1240] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:19.759085 13936 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:19.979174 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:20.014513 10842 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:19.769232 14117 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:19.777661 13874 raft_consensus.cc:740] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:19.960547 11509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56404 (request call id 624) took 1222 ms. Trace:
I20260709 13:17:19.922617 11510 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e"
dest_uuid: "12682eb70bc84c76879e04a8628ee586"
mode: GRACEFUL
new_leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
 from {username='slave'} at 127.0.0.1:57818
I20260709 13:17:19.894922 11504 raft_consensus.cc:1240] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
W20260709 13:17:19.805953 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:19.999071 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 310349us
I20260709 13:17:19.896298 11512 raft_consensus.cc:1240] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 10 LEADER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:19.913334 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:19.999221 10798 raft_consensus.cc:1240] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:19.888696 10941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2034) took 4120 ms. Trace:
I20260709 13:17:20.001717 10841 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:20.035462 10941 rpcz_store.cc:276] 0709 13:17:15.768141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:17.720822 (+1952681us) service_pool.cc:224] Handling call
0709 13:17:17.721131 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:18.134524 (+413393us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:19.888381 (+1753857us) spinlock_profiling.cc:243] Waited 368 ms on lock 0x7b54003304d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:19.888616 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":846612352}
I20260709 13:17:20.038725 11510 raft_consensus.cc:606] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Received request to transfer leadership to TS e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:20.039306 11510 raft_consensus.cc:614] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:20.038156 10767 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:20.038257 11509 rpcz_store.cc:276] 0709 13:17:18.738420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:19.101946 (+363526us) service_pool.cc:224] Handling call
0709 13:17:19.253061 (+151115us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:19.253324 (+   263us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:19.253362 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:19.253417 (+    55us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:19.253471 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:19.253544 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:19.960287 (+706743us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b540046d2d0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:19.960457 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":346750720}
I20260709 13:17:20.042382 10841 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:20.043015 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:17:19.842015 11508 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56410 (request call id 564) took 3011 ms (client timeout 2135 ms). Trace:
W20260709 13:17:20.001802 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:19.854216 10916 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:20.014991 10949 raft_consensus.cc:2393] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 7: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
W20260709 13:17:20.043912 11508 rpcz_store.cc:269] 0709 13:17:16.830591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:16.837461 (+  6870us) service_pool.cc:224] Handling call
0709 13:17:19.841756 (+3004295us) spinlock_profiling.cc:243] Waited 387 ms on lock 0x7b54004d3fd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:19.841902 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":890661120}
W20260709 13:17:20.055622 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 201457us
I20260709 13:17:20.057401 10842 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:17:19.947369 14012 log.cc:927] Time spent T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 3.320s	user 0.006s	sys 0.005s
I20260709 13:17:19.992439 10801 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:17:19.870523 14044 raft_consensus.cc:2804] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:19.799103 10766 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:20.059072 10842 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:19.912906 14033 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:19.876690 14043 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:19.876925 13719 raft_consensus.cc:3055] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:20.078388 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:20.080083 10842 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:20.083874 10945 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:20.080910 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:20.085798 11508 raft_consensus.cc:2393] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 10: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:20.081110 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:20.081859 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:20.088101 10944 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:20.074033 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:20.089501 10941 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 9.
I20260709 13:17:19.885432 14045 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:19.843063 11505 raft_consensus.cc:1275] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:17:20.059199 10841 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:19.842559 11507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56416 (request call id 567) took 2157 ms. Trace:
I20260709 13:17:20.057716 10916 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:20.063800 10767 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:20.094805 11507 rpcz_store.cc:276] 0709 13:17:17.685642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:17.715589 (+ 29947us) service_pool.cc:224] Handling call
0709 13:17:17.715869 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:17.962624 (+246755us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:17.962653 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:17.971358 (+  8705us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:17.971434 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:19.842308 (+1870874us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b540068d8d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:19.842355 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:19.842476 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":62924,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":14354,"spinlock_wait_cycles":352699648}
I20260709 13:17:19.845142 10942 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10.
I20260709 13:17:20.018587 14116 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:20.070761 10950 raft_consensus.cc:1275] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:17:19.889904 13965 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:19.881927 13968 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:19.902115 14134 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:19.891968 11503 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56404 (request call id 619) took 1877 ms (client timeout 1768 ms). Trace:
I20260709 13:17:20.055147 13910 raft_consensus.cc:2804] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:19.857091 13274 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:20.097426 14145 log.cc:927] Time spent T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 13:17:20.141359 13936 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:20.142234 11505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56404 (request call id 622) took 1407 ms. Trace:
I20260709 13:17:20.124073 14143 raft_consensus.cc:2749] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:20.142628 11505 rpcz_store.cc:276] 0709 13:17:18.735188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:19.083026 (+347838us) service_pool.cc:224] Handling call
0709 13:17:19.842701 (+759675us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:20.094567 (+251866us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:20.141998 (+ 47431us) spinlock_profiling.cc:243] Waited 740 ms on lock 0x7b54004d3fd0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:20.142163 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1701377408}
I20260709 13:17:20.097934 14005 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:20.143833 14005 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:20.144233 14005 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:17:20.094043 14026 consensus_meta.cc:287] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.773s	user 0.010s	sys 0.001s
I20260709 13:17:20.098862 10875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1624) took 1963 ms. Trace:
I20260709 13:17:20.147404 10875 rpcz_store.cc:276] 0709 13:17:18.135941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:19.522758 (+1386817us) service_pool.cc:224] Handling call
0709 13:17:19.523210 (+   452us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:19.539769 (+ 16559us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:19.539797 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:19.540344 (+   547us) log.cc:844] Serialized 26 byte log entry
0709 13:17:19.826195 (+285851us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:19.826233 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:19.826304 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:20.098564 (+272260us) raft_consensus.cc:1685] finished
0709 13:17:20.098589 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:20.098755 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13630,"raft.queue_time_us":20212,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":9565,"thread_start_us":240437,"threads_started":1,"wal-append.queue_time_us":531239}
I20260709 13:17:20.034593 14032 consensus_queue.cc:1048] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 13:17:20.068020 14126 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:19.879060 13982 consensus_queue.cc:260] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:20.100106 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 301063us
I20260709 13:17:20.175994 10766 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:20.100621 14149 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:20.101903 10767 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:20.180775 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 78937.8us
I20260709 13:17:20.102486 14156 raft_consensus.cc:2749] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:20.134420 13910 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:20.092466 14144 consensus_queue.cc:1059] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 6.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 13:17:20.103349 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:20.093091 14043 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:20.103950 14044 raft_consensus.cc:697] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:20.073920 14148 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:20.095695 14133 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:17:20.109575 14135 consensus_meta.cc:287] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.696s	user 0.010s	sys 0.003s
I20260709 13:17:20.062850 13717 raft_consensus.cc:2764] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:17:20.150290 14134 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:20.116707 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:20.118377 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:20.216638 10941 raft_consensus.cc:2393] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 8: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
W20260709 13:17:19.914847 10874 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 1665) took 2804 ms (client timeout 1897 ms). Trace:
W20260709 13:17:19.861805 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 185483us
I20260709 13:17:20.219159 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:17:20.097426 14154 log.cc:927] Time spent T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
I20260709 13:17:20.082916 13719 raft_consensus.cc:740] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:20.059803 14054 consensus_queue.cc:1059] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 13:17:20.068557 13890 raft_consensus.cc:2804] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:20.140178 10947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2024) took 4916 ms. Trace:
I20260709 13:17:20.226004 10947 rpcz_store.cc:276] 0709 13:17:15.223878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:15.306200 (+ 82322us) service_pool.cc:224] Handling call
0709 13:17:15.306541 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:15.306896 (+   355us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:15.306928 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:15.307387 (+   459us) log.cc:844] Serialized 26 byte log entry
0709 13:17:16.083205 (+775818us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:16.109385 (+ 26180us) log.cc:844] Serialized 14 byte log entry
0709 13:17:16.109833 (+   448us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:16.109978 (+   145us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:19.931126 (+3821148us) spinlock_profiling.cc:243] Waited 20.1 ms on lock 0x7b240025b008. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dc4e9c4 00007f9c7dc4cacb 00007f9c7de07dc9 00007f9c7de07a55 00007f9c7de0709d 00007f9c80a3910e 00007f9c80a5088c 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221
0709 13:17:20.139844 (+208718us) raft_consensus.cc:1685] finished
0709 13:17:20.139945 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:20.140085 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":83941,"prepare.queue_time_us":316879,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":121475,"raft.run_cpu_time_us":273,"raft.run_wall_time_us":37127,"spinlock_wait_cycles":46208384,"thread_start_us":775424,"threads_started":1,"wal-append.queue_time_us":1081711}
I20260709 13:17:20.140576 14058 raft_consensus.cc:3055] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:20.141403 13274 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:20.095641 14117 raft_consensus.cc:2804] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:20.155172 10945 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 10.
I20260709 13:17:20.170001 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:20.178126 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:20.178889 10943 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "414992c8073646798991d2b43fd61f1d"
dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
mode: GRACEFUL
new_leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
 from {username='slave'} at 127.0.0.1:46052
I20260709 13:17:20.239499 10943 raft_consensus.cc:606] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Received request to transfer leadership to TS 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:20.239867 10943 raft_consensus.cc:614] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:20.182404 10767 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:20.184864 10766 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:17:20.240808 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 58497.7us
W20260709 13:17:20.241040 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 56312.9us
I20260709 13:17:20.242045 10766 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:20.243577 10766 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:20.067710 13829 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:20.154851 14049 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:20.213174 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:20.179977 14149 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:20.095558 10916 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:20.188812 13968 raft_consensus.cc:2749] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:17:20.217629 10874 rpcz_store.cc:269] 0709 13:17:17.110188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:17.917953 (+807765us) service_pool.cc:224] Handling call
0709 13:17:19.914586 (+1996633us) spinlock_profiling.cc:243] Waited 4.58 ms on lock 0x7b5400076f54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:19.914739 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10543360}
I20260709 13:17:20.221285 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:19.826274 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:20.341779 10794 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:20.199491 13910 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:20.191408 14151 raft_consensus.cc:2749] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:20.223326 10875 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:20.215153 14135 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:20.228928 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:20.258611 14156 raft_consensus.cc:2749] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:20.207257 14148 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:17:20.235186 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:20.360256 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:20.360962 10947 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:20.236579 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:20.237296 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:20.239526 10949 raft_consensus.cc:2393] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 7: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:20.361699 10876 raft_consensus.cc:2376] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 7. Re-sending same reply.
I20260709 13:17:20.253369 11478 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:17:20.215227 13982 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:20.339205 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:20.339082 10946 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 8.
I20260709 13:17:20.234858 14058 raft_consensus.cc:740] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:20.232342 13890 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:20.303785 10950 raft_consensus.cc:2393] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 10: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:20.241986 10767 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:17:20.368505 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 126583us
I20260709 13:17:20.209227 14044 consensus_queue.cc:237] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:20.339748 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:20.374229 11505 raft_consensus.cc:2393] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 10: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:20.339974 10942 raft_consensus.cc:1240] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 10. Current term is 11. Ops: []
I20260709 13:17:20.223820 14133 raft_consensus.cc:697] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:17:20.340507 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 97027.4us
I20260709 13:17:20.339265 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:20.350410 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:17:20.133373 11503 rpcz_store.cc:269] 0709 13:17:18.014990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:18.522092 (+507102us) service_pool.cc:224] Handling call
0709 13:17:19.891677 (+1369585us) spinlock_profiling.cc:243] Waited 466 ms on lock 0x7b54005f96d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:19.891872 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1070691584}
I20260709 13:17:20.256219 13829 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:20.252717 11507 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4"
dest_uuid: "12682eb70bc84c76879e04a8628ee586"
 from {username='slave'} at 127.0.0.1:56404
W20260709 13:17:20.352257 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 256755us
I20260709 13:17:20.233233 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:20.363631 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 110250us
I20260709 13:17:20.282645 14117 raft_consensus.cc:697] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:20.379208 10945 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
I20260709 13:17:20.390321 11478 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:17:20.394033 10766 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:20.395157 11478 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:20.389518 10916 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:20.352329 14045 raft_consensus.cc:2804] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:20.352165 14146 raft_consensus.cc:2749] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:20.392037 10873 raft_consensus.cc:1240] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:20.388140 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1670) took 1293 ms. Trace:
I20260709 13:17:20.352501 14150 raft_consensus.cc:2749] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:17:20.395876 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 311153us
I20260709 13:17:20.395630 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:20.395637 10766 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:20.396715 10916 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:20.359320 13936 raft_consensus.cc:2804] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:20.350410 14149 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:20.388922 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:20.386009 10794 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7.
I20260709 13:17:20.372743 14026 raft_consensus.cc:2804] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:20.420732 10869 rpcz_store.cc:276] 0709 13:17:19.094363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:19.686250 (+591887us) service_pool.cc:224] Handling call
0709 13:17:19.686633 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:19.687227 (+   594us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:17:19.999426 (+312199us) log.cc:844] Serialized 14 byte log entry
0709 13:17:20.387751 (+388325us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:20.387815 (+    64us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:20.387874 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:20.387943 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:20.388077 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":445865,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":771,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":23642,"thread_start_us":653893,"threads_started":2}
I20260709 13:17:20.378904 14158 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:20.424537 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:20.496572 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:17:20.468571 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 73555.6us
I20260709 13:17:20.498903 10950 raft_consensus.cc:2393] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 7: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:20.496274 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:20.500427 10941 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7.
I20260709 13:17:20.487766 11511 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 8.
W20260709 13:17:20.487919 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 92338.4us
I20260709 13:17:20.374734 13890 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:20.490015 10875 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 10.
I20260709 13:17:20.366142 14148 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:20.404982 10947 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
I20260709 13:17:20.393203 13829 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:20.497107 10876 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 9.
W20260709 13:17:20.499492 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 440351us
I20260709 13:17:20.506268 13965 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:20.510215 14150 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:20.511631 14134 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:20.511404 14150 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:20.512643 14150 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:20.513571 11512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56410 (request call id 571) took 1115 ms. Trace:
I20260709 13:17:20.514858 10943 raft_consensus.cc:1240] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:17:20.515848 14065 consensus_queue.cc:1059] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:17:20.467556 10949 raft_consensus.cc:2393] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 10: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:20.514101 11512 rpcz_store.cc:276] 0709 13:17:19.398038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:19.895734 (+497696us) service_pool.cc:224] Handling call
0709 13:17:19.896075 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:20.037612 (+141537us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:20.513298 (+475686us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b5400528120. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80e2cdc0 00007f9c80eb8a02 00007f9c80eb4dd6 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2
0709 13:17:20.513505 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":334813952}
W20260709 13:17:20.525050 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:20.503353 10875 raft_consensus.cc:3055] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:17:20.485661 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 89033.6us
I20260709 13:17:20.527832 10875 raft_consensus.cc:740] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:20.527170 10916 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:20.422129 14151 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:20.536172 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:20.419066 14116 raft_consensus.cc:2804] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:20.540047 14116 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:20.540606 14116 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:20.422619 13975 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:20.547118 13975 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:20.547717 13975 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:20.418895 14156 consensus_queue.cc:1048] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.116s
I20260709 13:17:20.555773 10876 raft_consensus.cc:1275] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:17:20.420032 14045 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:20.539705 14151 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:20.536891 10875 consensus_queue.cc:260] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:20.506343 14019 raft_consensus.cc:2804] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:20.574606 14019 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:20.574901 10793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2085) took 1083 ms. Trace:
I20260709 13:17:20.575067 14019 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:20.575296 10793 rpcz_store.cc:276] 0709 13:17:19.491371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:19.498970 (+  7599us) service_pool.cc:224] Handling call
0709 13:17:19.529574 (+ 30604us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:20.002942 (+473368us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:20.002991 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:20.015119 (+ 12128us) log.cc:844] Serialized 26 byte log entry
0709 13:17:20.420958 (+405839us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:20.421085 (+   127us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:20.421214 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:20.574635 (+153421us) raft_consensus.cc:1685] finished
0709 13:17:20.574656 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:20.574843 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":240919,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":92,"thread_start_us":811007,"threads_started":2,"wal-append.queue_time_us":511909}
I20260709 13:17:20.576121 14117 consensus_queue.cc:237] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:20.501478 14026 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:20.580514 14026 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:20.508283 14158 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:20.585686 10944 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 8.
I20260709 13:17:20.553956 14120 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:20.527146 13936 raft_consensus.cc:697] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 7 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:20.559158 10872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 408) took 1464 ms. Trace:
I20260709 13:17:20.534101 13965 raft_consensus.cc:697] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 10 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:20.591059 14109 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:20.562686 14045 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:20.571558 14151 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:20.517831 14108 consensus_queue.cc:1048] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.071s
I20260709 13:17:20.597432 10944 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "695ff1b5a68e4111917d909909488d39"
dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
mode: GRACEFUL
new_leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
 from {username='slave'} at 127.0.0.1:46052
I20260709 13:17:20.610036 14043 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:20.609915 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:20.628373 10944 raft_consensus.cc:606] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Received request to transfer leadership to TS e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:20.613521 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:20.615154 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:20.623301 11477 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:20.615854 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:20.552390 14144 consensus_queue.cc:1059] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:17:20.538534 14058 consensus_queue.cc:260] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 8.3, Last appended by leader: 3, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:20.632654 14170 consensus_queue.cc:1059] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:20.633257 13274 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:20.633455 14170 consensus_queue.cc:1243] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Peer cb0428cc55984daba1ba63744a5102a2 log is divergent from this leader: its last log entry 9.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:17:20.636137 11478 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:20.616456 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:20.619814 10946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41014 (request call id 1679) took 1186 ms. Trace:
I20260709 13:17:20.639709 10946 rpcz_store.cc:276] 0709 13:17:19.432975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:19.839608 (+406633us) service_pool.cc:224] Handling call
0709 13:17:20.619579 (+779971us) spinlock_profiling.cc:243] Waited 266 ms on lock 0x7b54001e2654. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:20.619729 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":611160960}
I20260709 13:17:20.620900 14005 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:20.398397 13719 consensus_queue.cc:260] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:20.622051 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:20.622669 10841 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:20.643754 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:20.628867 10944 raft_consensus.cc:614] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:20.651818 14135 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:20.589880 14158 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:20.592846 14133 consensus_queue.cc:237] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:20.655462 14161 log.cc:927] Time spent T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:17:20.655984 14049 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:20.627942 14120 raft_consensus.cc:697] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:20.661322 13965 consensus_queue.cc:237] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:20.662312 11478 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:17:20.663221 13874 consensus_queue.cc:260] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:20.666348 14041 raft_consensus.cc:2804] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:20.666246 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:20.654157 13936 consensus_queue.cc:237] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:20.596009 10872 rpcz_store.cc:276] 0709 13:17:19.094651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:19.772643 (+677992us) service_pool.cc:224] Handling call
0709 13:17:19.772948 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:19.773154 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:17:20.167078 (+393924us) log.cc:844] Serialized 14 byte log entry
0709 13:17:20.558825 (+391747us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:20.558880 (+    55us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:20.558932 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:20.558992 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:20.559102 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":365865,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":25,"raft.queue_time_us":1538,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":11581,"thread_start_us":784788,"threads_started":2}
W20260709 13:17:20.673883 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 51296.8us
I20260709 13:17:20.701625 10941 raft_consensus.cc:2393] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 10: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:20.675344 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:20.688350 11478 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:20.702375 10841 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:20.634253 14148 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:20.690258 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:20.703225 10944 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 6.
W20260709 13:17:20.687229 11506 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.017s	sys 0.020s
I20260709 13:17:20.672003 14046 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:20.707864 10841 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:20.686057 14126 raft_consensus.cc:2749] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:17:20.686360 14049 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:20.715934 10841 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:20.614778 14019 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:20.693595 14057 raft_consensus.cc:2804] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:20.628526 14109 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:20.712170 11506 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.018s	sys 0.020s
I20260709 13:17:20.728118 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:20.729420 10941 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:20.708155 13874 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:20.733345 14057 raft_consensus.cc:697] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 10 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:20.734328 14057 consensus_queue.cc:237] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:20.737150 10795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2082) took 1260 ms. Trace:
I20260709 13:17:20.734326 13910 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:20.737607 10795 rpcz_store.cc:276] 0709 13:17:19.476791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:19.480296 (+  3505us) service_pool.cc:224] Handling call
0709 13:17:19.480591 (+   295us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:19.661500 (+180909us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:19.661547 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:19.663788 (+  2241us) log.cc:844] Serialized 26 byte log entry
0709 13:17:20.151405 (+487617us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:20.151446 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:20.151585 (+   139us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:20.736936 (+585351us) raft_consensus.cc:1685] finished
0709 13:17:20.736958 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:20.737091 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17618,"raft.queue_time_us":220288,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":2347,"thread_start_us":475145,"threads_started":2,"wal-append.queue_time_us":691790}
I20260709 13:17:20.713826 14032 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:20.710626 14041 raft_consensus.cc:697] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 7 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:20.743353 14041 consensus_queue.cc:237] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:20.747363 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1601) took 3025 ms. Trace:
I20260709 13:17:20.747807 10873 rpcz_store.cc:276] 0709 13:17:17.721646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:19.516293 (+1794647us) service_pool.cc:224] Handling call
0709 13:17:19.516671 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:20.424954 (+908283us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:20.747095 (+322141us) spinlock_profiling.cc:243] Waited 875 ms on lock 0x7b540009a1d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:20.747297 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2012704000}
I20260709 13:17:20.753386 11512 raft_consensus.cc:2393] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 9: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:20.720798 14143 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:20.742130 14032 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:20.718394 14046 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:20.811765 10941 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7.
I20260709 13:17:20.770771 14032 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:17:20.829427 14172 log.cc:927] Time spent T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:17:20.905190 14046 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:20.906680 14109 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:20.756126 14074 consensus_queue.cc:1048] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.067s
W20260709 13:17:20.920092 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:20.780261 14120 consensus_queue.cc:237] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:20.931849 14148 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:20.936080 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 298002us
I20260709 13:17:20.937137 11477 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:20.938139 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:20.940050 10842 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:20.818429 14129 raft_consensus.cc:3055] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:17:20.953691 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 451419us
W20260709 13:17:20.952310 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:20.956825 10842 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:20.967190 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:20.969720 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:20.970386 10801 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:20.829047 13975 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:20.968170 10795 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 6.
I20260709 13:17:20.977903 11477 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:20.981583 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:20.984596 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:20.985551 10797 raft_consensus.cc:2393] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 8: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:20.922186 14026 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:20.989780 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:20.992751 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:20.993510 10872 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:21.003536 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:21.004726 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:20.974411 14129 raft_consensus.cc:740] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:21.008380 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:21.011188 11510 raft_consensus.cc:2368] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 7. Current term is 8.
I20260709 13:17:21.009012 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:21.013321 11509 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 9.
I20260709 13:17:20.979645 14135 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:17:21.033385 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:21.033430 10767 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.664s	user 0.004s	sys 0.026s
I20260709 13:17:21.002025 11477 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:21.036458 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:20.988200 13982 raft_consensus.cc:2804] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:21.037500 11510 raft_consensus.cc:2410] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:17:20.985226 14126 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:20.988907 13910 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:21.058911 11476 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.008s	sys 0.036s
W20260709 13:17:21.063766 11476 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.008s	sys 0.036s
W20260709 13:17:21.059767 10916 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.005s	sys 0.013s
I20260709 13:17:20.994817 14074 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:21.045015 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:21.046770 10801 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:17:21.070421 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:17:21.056370 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 54400.4us
W20260709 13:17:21.034313 10767 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.004s	sys 0.026s
W20260709 13:17:21.073891 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 704674us
I20260709 13:17:21.101225 11509 raft_consensus.cc:1240] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:17:21.005986 14005 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:17:21.105688 11508 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.000s	sys 0.017s
I20260709 13:17:21.105692 10766 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:21.106459 11508 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.000s	sys 0.017s
W20260709 13:17:21.064446 10916 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.005s	sys 0.013s
W20260709 13:17:21.108599 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 573433us
W20260709 13:17:21.033134 14065 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.010s	sys 0.008s
I20260709 13:17:20.985867 14149 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:21.061637 14043 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:21.118207 10767 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:21.115624 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:21.046515 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:21.059700 13967 consensus_queue.cc:1048] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.084s
I20260709 13:17:21.113209 13633 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:21.142795 10942 raft_consensus.cc:1240] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 7 LEADER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:17:21.101125 14074 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:21.125717 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 409866us
I20260709 13:17:21.116827 14148 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:21.160566 10841 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:21.126225 10766 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:21.046336 14150 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:21.120245 14019 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:21.100529 14126 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:21.166870 14044 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.027s	sys 0.024s
I20260709 13:17:21.124183 14147 consensus_queue.cc:1059] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.066s
I20260709 13:17:21.180214 14074 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:17:21.190488 14175 log.cc:927] Time spent T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 13:17:21.184670 14065 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:21.189643 14128 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:21.188006 14191 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:21.200666 14158 consensus_meta.cc:287] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.508s	user 0.011s	sys 0.000s
W20260709 13:17:21.208204 14116 consensus_meta.cc:287] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.667s	user 0.012s	sys 0.000s
W20260709 13:17:21.243251 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 255848us
I20260709 13:17:21.189126 14148 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:21.217594 14178 log.cc:927] Time spent T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 13:17:21.158699 14129 consensus_queue.cc:260] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:21.251857 13890 consensus_meta.cc:287] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.712s	user 0.010s	sys 0.002s
I20260709 13:17:21.210265 14182 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:21.199438 14044 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.027s	sys 0.025s
W20260709 13:17:21.260811 14133 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.006s	sys 0.022s
I20260709 13:17:21.252463 14065 raft_consensus.cc:697] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 8 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
W20260709 13:17:21.208426 14033 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.015s	sys 0.040s
I20260709 13:17:21.252161 14179 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:21.270828 13975 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:21.225904 14128 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:21.246083 14158 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:21.297958 14033 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.015s	sys 0.041s
I20260709 13:17:21.261960 14116 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:21.318482 10872 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 9.
W20260709 13:17:21.295447 14133 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.006s	sys 0.023s
I20260709 13:17:21.297159 14026 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:17:21.321378 14177 log.cc:927] Time spent T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
I20260709 13:17:21.319967 14195 raft_consensus.cc:2749] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:21.303929 14129 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:21.293339 13890 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:21.321081 14044 raft_consensus.cc:2804] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:21.321702 14179 raft_consensus.cc:2749] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:21.358942 10947 raft_consensus.cc:2393] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 9: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:21.265970 14191 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:21.365206 14149 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:21.295542 14182 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:21.339898 14135 raft_consensus.cc:697] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:17:21.362233 14173 log.cc:927] Time spent T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.211s	user 0.000s	sys 0.001s
W20260709 13:17:21.362071 14174 log.cc:927] Time spent T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.216s	user 0.001s	sys 0.001s
I20260709 13:17:21.383473 14150 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:21.370082 14044 raft_consensus.cc:697] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:21.362550 14202 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:21.390106 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:21.390384 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 208730us
I20260709 13:17:21.394137 11503 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546"
dest_uuid: "12682eb70bc84c76879e04a8628ee586"
mode: GRACEFUL
new_leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
 from {username='slave'} at 127.0.0.1:57818
W20260709 13:17:21.406158 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 288002us
I20260709 13:17:21.359083 14194 consensus_queue.cc:1059] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
W20260709 13:17:21.391453 14151 consensus_meta.cc:287] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.721s	user 0.005s	sys 0.006s
I20260709 13:17:21.383888 14116 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:21.385329 13890 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:21.413708 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 150980us
W20260709 13:17:21.420396 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 294255us
I20260709 13:17:21.421705 10766 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:21.407562 10767 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:21.399461 10841 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:21.396457 14195 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:21.431226 10841 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:21.442830 10767 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:21.408737 14128 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:21.409914 14147 consensus_queue.cc:1243] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer 468a8fd46e824f62b9c2c0d3532a6d26 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:17:21.373044 14065 consensus_queue.cc:237] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:21.413951 11503 raft_consensus.cc:606] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 10 LEADER]: Received request to transfer leadership to TS e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:21.463188 14183 log.cc:927] Time spent T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.200s	user 0.001s	sys 0.000s
I20260709 13:17:21.427663 14194 raft_consensus.cc:3055] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:17:21.467381 14180 log.cc:927] Time spent T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
I20260709 13:17:21.473786 10841 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:21.409516 14202 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:21.447111 14188 log.cc:927] Time spent T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
W20260709 13:17:21.472358 14197 log.cc:927] Time spent T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 13:17:21.472918 14148 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:21.485284 14199 log.cc:927] Time spent T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 13:17:21.481848 14147 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:21.465003 14135 consensus_queue.cc:237] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:21.509505 14198 log.cc:927] Time spent T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
W20260709 13:17:21.511186 14049 consensus_meta.cc:287] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.782s	user 0.012s	sys 0.000s
I20260709 13:17:21.446545 14126 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:21.444665 13717 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:21.489079 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:21.484112 14194 raft_consensus.cc:740] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:21.492022 13633 raft_consensus.cc:2749] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:21.492329 10948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 453) took 1446 ms. Trace:
W20260709 13:17:21.493990 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:21.461736 14150 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:21.493057 14203 raft_consensus.cc:2749] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:17:21.472100 11506 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.696s	user 0.020s	sys 0.043s
I20260709 13:17:21.443729 13982 raft_consensus.cc:697] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:21.461397 13890 raft_consensus.cc:2749] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:21.445892 14211 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:21.425079 14151 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:21.490595 14108 raft_consensus.cc:3055] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:21.503952 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:21.504746 14032 consensus_meta.cc:287] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.678s	user 0.011s	sys 0.000s
I20260709 13:17:21.504900 14202 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:21.506140 14182 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:21.482692 14044 consensus_queue.cc:237] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:21.507895 14196 log.cc:927] Time spent T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
W20260709 13:17:21.504746 13829 consensus_meta.cc:287] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.946s	user 0.008s	sys 0.004s
I20260709 13:17:21.508757 14158 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:21.542263 13829 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:21.512498 14193 log.cc:927] Time spent T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
I20260709 13:17:21.497964 14046 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:21.521940 10948 rpcz_store.cc:276] 0709 13:17:20.045767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:20.045903 (+   136us) service_pool.cc:224] Handling call
0709 13:17:20.046303 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:20.046574 (+   271us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:17:20.362736 (+316162us) log.cc:844] Serialized 14 byte log entry
0709 13:17:20.672538 (+309802us) log.cc:844] Serialized 14 byte log entry
0709 13:17:21.491804 (+819266us) spinlock_profiling.cc:243] Waited 9.24 ms on lock 0x7b18002020c0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7dd42c0a 00007f9c87482219 00007f9c7ddcaa40 00007f9c7ddc54e4 00007f9c7ddc5455 00007f9c80ead060 00007f9c80ebf716 00007f9c80ed0287 00007f9c80ed0208
0709 13:17:21.491965 (+   161us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:21.492015 (+    50us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:21.492069 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:21.492143 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:21.492271 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":802446,"prepare.queue_time_us":528921,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":823339,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":9400,"spinlock_wait_cycles":68352128,"thread_start_us":621990,"threads_started":2,"wal-append.queue_time_us":625700}
W20260709 13:17:21.530303 14192 log.cc:927] Time spent T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.179s	user 0.001s	sys 0.001s
W20260709 13:17:21.528384 11506 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.763s	user 0.020s	sys 0.045s
W20260709 13:17:21.540148 14201 log.cc:927] Time spent T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.182s	user 0.000s	sys 0.001s
I20260709 13:17:21.445652 14195 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:21.536553 14005 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:21.511868 14120 consensus_queue.cc:1059] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:17:21.511605 14049 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:21.538058 14108 raft_consensus.cc:740] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:21.540125 14211 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:21.550729 14043 raft_consensus.cc:2749] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:21.549880 14150 raft_consensus.cc:697] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:17:21.564296 14190 log.cc:927] Time spent T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
I20260709 13:17:21.553750 14046 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:21.567950 14049 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:21.569722 14046 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:21.572710 14045 consensus_meta.cc:287] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.923s	user 0.012s	sys 0.000s
W20260709 13:17:21.645503 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 169375us
I20260709 13:17:21.676036 10766 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:21.678520 14217 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:21.679492 14217 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:21.680083 14217 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:21.680665 13982 consensus_queue.cc:237] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:21.505321 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:21.562193 14005 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:21.685966 10948 raft_consensus.cc:1240] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:17:21.526388 13717 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:21.689417 10942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2054) took 1468 ms. Trace:
I20260709 13:17:21.691473 10766 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:21.691684 13829 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:21.692216 11506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56404 (request call id 627) took 2670 ms. Trace:
I20260709 13:17:21.657850 10871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 431) took 1020 ms. Trace:
I20260709 13:17:21.654333 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:17:21.654309 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 180465us
I20260709 13:17:21.658202 10875 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:21.660125 10876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 429) took 1299 ms. Trace:
W20260709 13:17:21.665638 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:21.666728 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:21.667706 10842 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:21.668560 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:17:21.668887 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 559264us
W20260709 13:17:21.507227 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 84522us
I20260709 13:17:21.642088 10698 catalog_manager.cc:5697] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 4 to 6, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 6 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:21.590191 14058 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:21.588924 14074 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:21.588423 14108 consensus_queue.cc:260] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:21.739364 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:21.740031 14108 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:21.740528 10873 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:21.741509 14074 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:21.586733 14150 consensus_queue.cc:237] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:17:21.584473 14151 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:21.592207 14176 raft_consensus.cc:3055] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:21.744202 14176 raft_consensus.cc:740] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:17:21.591503 14185 log.cc:927] Time spent T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.375s	user 0.001s	sys 0.004s
W20260709 13:17:21.584988 14189 log.cc:927] Time spent T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.177s	user 0.000s	sys 0.001s
I20260709 13:17:21.746606 14176 consensus_queue.cc:260] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:21.748313 14176 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:21.573925 14191 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:21.560043 14194 consensus_queue.cc:260] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 7, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:21.748349 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 57174.9us
I20260709 13:17:21.751513 14194 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:21.751914 10841 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:21.753499 11508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56410 (request call id 583) took 1173 ms. Trace:
I20260709 13:17:21.563248 14211 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:21.754055 11508 rpcz_store.cc:276] 0709 13:17:20.580119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:20.580712 (+   593us) service_pool.cc:224] Handling call
0709 13:17:20.581082 (+   370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:20.581348 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:17:20.581377 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:20.581841 (+   464us) log.cc:844] Serialized 26 byte log entry
0709 13:17:21.106911 (+525070us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:21.106972 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:21.107125 (+   153us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:21.723383 (+616258us) raft_consensus.cc:1685] finished
0709 13:17:21.725605 (+  2222us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:21.753303 (+ 27698us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":524692,"threads_started":1,"wal-append.queue_time_us":717299}
I20260709 13:17:21.589437 14195 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:21.693145 10871 rpcz_store.cc:276] 0709 13:17:20.637627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:20.666958 (+ 29331us) service_pool.cc:224] Handling call
0709 13:17:20.667347 (+   389us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:20.667716 (+   369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:20.667745 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:20.668322 (+   577us) log.cc:844] Serialized 52 byte log entry
0709 13:17:21.123670 (+455348us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:17:21.150403 (+ 26733us) log.cc:844] Serialized 14 byte log entry
0709 13:17:21.515683 (+365280us) log.cc:844] Serialized 14 byte log entry
0709 13:17:21.656812 (+141129us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:21.657024 (+   212us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:21.657149 (+   125us) raft_consensus.cc:1685] finished
0709 13:17:21.657185 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:21.657563 (+   378us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":169686,"prepare.queue_time_us":26535,"prepare.run_cpu_time_us":173,"prepare.run_wall_time_us":2435,"raft.queue_time_us":153057,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":17145,"thread_start_us":818042,"threads_started":2,"wal-append.queue_time_us":607731}
I20260709 13:17:21.702505 14181 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:21.703293 10948 raft_consensus.cc:1240] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:21.756893 14181 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:21.758174 10948 raft_consensus.cc:1240] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:21.690433 13717 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:21.689917 10942 rpcz_store.cc:276] 0709 13:17:20.220746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:20.388321 (+167575us) service_pool.cc:224] Handling call
0709 13:17:20.425657 (+ 37336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:21.158419 (+732762us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:21.688630 (+530211us) spinlock_profiling.cc:243] Waited 717 ms on lock 0x7b54000172d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:21.689116 (+   486us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1648338688}
I20260709 13:17:21.544430 10943 raft_consensus.cc:1240] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:17:21.693780 10876 rpcz_store.cc:276] 0709 13:17:20.360863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:20.562595 (+201732us) service_pool.cc:224] Handling call
0709 13:17:20.594443 (+ 31848us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:20.757300 (+162857us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:20.757331 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:20.757373 (+    42us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:20.757407 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:21.659491 (+902084us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b5400078854. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:21.659587 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:21.659899 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":373999872}
I20260709 13:17:21.760092 14181 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:21.707381 10795 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 7.
I20260709 13:17:21.718572 14005 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:21.719470 10915 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:21.678337 14049 consensus_queue.cc:1059] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 13:17:21.724293 13822 raft_consensus.cc:2749] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:21.762588 14049 consensus_queue.cc:1243] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer 468a8fd46e824f62b9c2c0d3532a6d26 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:17:21.763374 14044 raft_consensus.cc:3055] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:21.763777 14044 raft_consensus.cc:740] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:21.764933 14044 consensus_queue.cc:260] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:21.766113 14044 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:21.766883 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 99544.5us
I20260709 13:17:21.731384 10797 raft_consensus.cc:2393] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 10: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:21.767979 10842 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:21.732656 14134 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:21.742461 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:21.726282 11507 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:21.753401 10766 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:21.655027 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:21.701548 14148 raft_consensus.cc:2749] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:21.778446 10766 leader_election.cc:400] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 election: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 7. Current term is 8.
I20260709 13:17:21.778923 10766 leader_election.cc:403] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:17:21.779109 14148 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:21.779631 14148 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:21.780058 14148 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:17:21.780934 11511 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56404 (request call id 628) took 2537 ms (client timeout 1528 ms). Trace:
W20260709 13:17:21.781430 11511 rpcz_store.cc:269] 0709 13:17:19.243866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:19.264576 (+ 20710us) service_pool.cc:224] Handling call
0709 13:17:21.780489 (+2515913us) spinlock_profiling.cc:243] Waited 870 ms on lock 0x7b540021e654. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:21.780692 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2001196288}
I20260709 13:17:21.787781 14135 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:21.788551 14135 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:21.549674 14032 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:21.791016 14129 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:21.791765 14129 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:21.793113 14032 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:21.698256 11506 rpcz_store.cc:276] 0709 13:17:19.021256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:19.871511 (+850255us) service_pool.cc:224] Handling call
0709 13:17:19.871919 (+   408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:19.872144 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:17:20.174959 (+302815us) log.cc:844] Serialized 14 byte log entry
0709 13:17:21.691175 (+1516216us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:21.691354 (+   179us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:21.691519 (+   165us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:21.691685 (+   166us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:21.691902 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17403,"prepare.queue_time_us":624584,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":177837,"raft.run_cpu_time_us":69,"raft.run_wall_time_us":69,"thread_start_us":1628882,"threads_started":3,"wal-append.queue_time_us":890455}
I20260709 13:17:21.796657 10872 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:21.797959 14144 raft_consensus.cc:2804] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:21.798381 14144 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:21.798741 14144 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:21.799144 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:21.800040 14169 raft_consensus.cc:2804] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:21.800876 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:21.802419 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:21.803084 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:21.803422 10800 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 9.
I20260709 13:17:21.804318 10799 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 7: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:21.804575 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:21.805722 10799 raft_consensus.cc:1240] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:21.802399 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:21.802421 10792 raft_consensus.cc:1240] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:21.807195 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:21.807379 14049 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:21.808244 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:21.806507 10800 raft_consensus.cc:1275] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:17:21.784440 11507 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:21.813695 11512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56410 (request call id 582) took 1287 ms. Trace:
W20260709 13:17:21.814620 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 63008.4us
I20260709 13:17:21.815088 13890 raft_consensus.cc:2749] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:21.815726 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:21.816606 10792 raft_consensus.cc:2393] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 10: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:21.816468 10841 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:21.817299 13890 raft_consensus.cc:2749] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:21.820667 10841 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:21.822220 13719 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:21.822276 13890 raft_consensus.cc:2749] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:21.822366 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:21.830209 10793 raft_consensus.cc:2393] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 9: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:21.814278 11512 rpcz_store.cc:276] 0709 13:17:20.523016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:20.535896 (+ 12880us) service_pool.cc:224] Handling call
0709 13:17:21.807082 (+1271186us) spinlock_profiling.cc:243] Waited 213 ms on lock 0x7b54004d3fd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:21.810557 (+  3475us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":489766784}
I20260709 13:17:21.835542 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:21.839648 10793 raft_consensus.cc:2410] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:17:21.821049 14169 raft_consensus.cc:697] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:21.844414 10943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2061) took 1260 ms. Trace:
I20260709 13:17:21.843942 10698 catalog_manager.cc:5697] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:21.724490 14045 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:21.850345 10875 raft_consensus.cc:1275] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:17:21.830338 14129 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:21.862919 10801 raft_consensus.cc:2393] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 7: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:21.874456 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:21.875669 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:21.876811 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:21.880020 10698 catalog_manager.cc:5697] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 2 to 9, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 9 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:21.856284 10766 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:21.877526 10801 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 7.
I20260709 13:17:21.857343 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:21.862257 13995 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 13:17:21.864008 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:21.857067 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:21.849540 10943 rpcz_store.cc:276] 0709 13:17:20.583871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:20.584086 (+   215us) service_pool.cc:224] Handling call
0709 13:17:20.606568 (+ 22482us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:21.760095 (+1153527us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:21.843925 (+ 83830us) spinlock_profiling.cc:243] Waited 938 ms on lock 0x7b54003d8e54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:21.844233 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2156310912}
I20260709 13:17:21.868256 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:21.867656 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:21.873289 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:21.891113 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:21.891681 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:21.892379 14045 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:21.892870 10876 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 7.
I20260709 13:17:21.893129 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:21.892458 10871 raft_consensus.cc:2376] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 8. Re-sending same reply.
I20260709 13:17:21.871790 10947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41020 (request call id 2062) took 1261 ms. Trace:
I20260709 13:17:21.895100 10947 rpcz_store.cc:276] 0709 13:17:20.610690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:20.616148 (+  5458us) service_pool.cc:224] Handling call
0709 13:17:21.851433 (+1235285us) spinlock_profiling.cc:243] Waited 720 ms on lock 0x7b54003f0054. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:21.871580 (+ 20147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1654920576}
I20260709 13:17:21.895874 10792 raft_consensus.cc:1275] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:17:21.897393 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:21.898234 10792 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 10.
I20260709 13:17:21.898126 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:21.898666 10794 raft_consensus.cc:2376] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 7. Re-sending same reply.
I20260709 13:17:21.900079 10800 raft_consensus.cc:2393] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 11: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:21.900267 10792 raft_consensus.cc:1240] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:21.894001 10870 raft_consensus.cc:2376] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10. Re-sending same reply.
I20260709 13:17:21.892382 10793 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
I20260709 13:17:21.892462 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:21.909420 10875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 425) took 1687 ms. Trace:
I20260709 13:17:21.909984 10875 rpcz_store.cc:276] 0709 13:17:20.221994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:20.502850 (+280856us) service_pool.cc:224] Handling call
0709 13:17:20.503211 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:21.656747 (+1153536us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b58007436d0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80e110aa 00007f9c80e93447 00007f9c80eabfe7 00007f9c80ead7de 00007f9c80eb7b49 00007f9c80eb8416 00007f9c80eb4d8f 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4
0709 13:17:21.851827 (+195080us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:21.909349 (+ 57522us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":233250048}
I20260709 13:17:21.925915 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:17:21.927587 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 79632.7us
I20260709 13:17:21.929431 10800 raft_consensus.cc:1275] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 13:17:21.929994 14229 log.cc:927] Time spent T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:17:21.932485 14217 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:21.937273 13874 consensus_meta.cc:287] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 1.188s	user 0.004s	sys 0.011s
W20260709 13:17:21.940739 11478 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.116.129:39685 timed out after 2.404s (SENT) [suppressed 29 similar messages]
W20260709 13:17:21.943375 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:21.942962 10698 catalog_manager.cc:5697] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 6 to 7, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 7 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:21.947288 14120 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:21.948210 14120 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:21.949990 13995 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:21.951373 14046 raft_consensus.cc:3055] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:21.951881 14046 raft_consensus.cc:740] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:21.952549 10842 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:21.953243 14046 consensus_queue.cc:260] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:21.960686 14046 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:21.961891 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:21.963307 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 245949us
I20260709 13:17:21.964222 14223 raft_consensus.cc:2764] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:17:21.950642 13995 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:21.968716 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:21.969254 13995 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:21.969442 10870 raft_consensus.cc:1275] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:17:21.969866 10950 raft_consensus.cc:2410] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:17:21.970489 13829 consensus_queue.cc:1048] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:17:21.971352 10841 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:21.972419 10841 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:21.974439 13822 raft_consensus.cc:2804] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:21.974915 13822 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:21.975327 13822 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:21.975579 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:21.977726 10950 raft_consensus.cc:1275] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:17:21.955394 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 124415us
I20260709 13:17:21.955639 14143 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:17:21.957718 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 204661us
I20260709 13:17:21.959126 10943 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740"
dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
mode: GRACEFUL
new_leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
 from {username='slave'} at 127.0.0.1:46052
I20260709 13:17:21.954582 13910 consensus_queue.cc:1048] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:21.928232 10801 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7.
I20260709 13:17:21.981786 10943 raft_consensus.cc:606] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Received request to transfer leadership to TS e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:21.992833 10943 raft_consensus.cc:614] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:21.992617 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:21.993793 10875 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 7.
I20260709 13:17:21.994427 14120 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:21.995112 10916 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:21.995658 10796 raft_consensus.cc:1240] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:21.997821 10916 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:22.000234 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:22.000679 14005 raft_consensus.cc:2804] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:22.001164 14005 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:22.001598 14005 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:22.001650 14129 raft_consensus.cc:2804] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:22.002378 14129 raft_consensus.cc:697] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:17:22.003233 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:22.003032 11507 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:22.003427 14129 consensus_queue.cc:237] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:22.007282 14032 consensus_queue.cc:1048] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:17:22.008612  9682 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:17:22.009614 10941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2066) took 1397 ms. Trace:
I20260709 13:17:22.011060 14062 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:22.010129 10941 rpcz_store.cc:276] 0709 13:17:20.611932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:20.813419 (+201487us) service_pool.cc:224] Handling call
0709 13:17:20.824629 (+ 11210us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:21.494447 (+669818us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:21.494476 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:21.496634 (+  2158us) log.cc:844] Serialized 26 byte log entry
0709 13:17:21.884162 (+387528us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:21.884226 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:21.884318 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:22.009179 (+124861us) raft_consensus.cc:1685] finished
0709 13:17:22.009220 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:22.009403 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":672363,"raft.queue_time_us":675640,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":10094,"thread_start_us":351247,"threads_started":1,"wal-append.queue_time_us":493753}
I20260709 13:17:21.980379 14143 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:21.994457 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:21.998023 13829 raft_consensus.cc:2804] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:21.944897 10698 catalog_manager.cc:5697] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:21.908993 14135 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:22.033667 11507 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:22.035480 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:22.022073 14143 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:22.022065 14045 raft_consensus.cc:2749] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:22.027150 13829 raft_consensus.cc:697] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 10 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:22.063800 10871 raft_consensus.cc:1240] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 LEADER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:22.066335 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:17:22.067283 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 60153.3us
I20260709 13:17:22.067988 13865 consensus_queue.cc:1059] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:17:22.069599 13874 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:22.070920 14169 consensus_queue.cc:237] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:22.058622 10946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 459) took 1406 ms. Trace:
I20260709 13:17:22.080461 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:22.087823 10876 raft_consensus.cc:2376] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 7. Re-sending same reply.
I20260709 13:17:22.091359 11507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:56404 (request call id 634) took 1797 ms. Trace:
I20260709 13:17:22.096140 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:22.096261 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: true dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:22.091778 11507 rpcz_store.cc:276] 0709 13:17:20.251216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:20.252549 (+  1333us) service_pool.cc:224] Handling call
0709 13:17:22.006055 (+1753506us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54007e1b54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eaca3d 00007f9c85e7ccd0 00007f9c7f401254 00007f9c7f4011e2 00007f9c7f401111 00007f9c7f40108c 00007f9c7f400342 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:22.048616 (+ 42561us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":272,"spinlock_wait_cycles":3001828608}
I20260709 13:17:22.092633 10872 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 8.
I20260709 13:17:22.080782 10698 catalog_manager.cc:5697] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:17:22.102371 10870 raft_consensus.cc:2410] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 13:17:22.104156 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:22.118963 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:22.120137 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:22.120265 10950 raft_consensus.cc:3055] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:22.120684 10950 raft_consensus.cc:740] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:22.121678 10950 consensus_queue.cc:260] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:22.123612 13274 consensus_queue.cc:1048] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 13:17:22.125607 13874 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:22.128149 13874 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:22.089745 14144 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:22.132890 14144 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:22.088622 10946 rpcz_store.cc:276] 0709 13:17:20.646948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:21.514578 (+867630us) service_pool.cc:224] Handling call
0709 13:17:21.515016 (+   438us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:21.679753 (+164737us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:17:21.680350 (+   597us) log.cc:844] Serialized 14 byte log entry
0709 13:17:21.968184 (+287834us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:21.968236 (+    52us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:17:21.968285 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:22.040503 (+ 72218us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b540018ecd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:22.040566 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:22.053279 (+ 12713us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":235772,"prepare.queue_time_us":12761,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":78,"raft.queue_time_us":236781,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":92,"spinlock_wait_cycles":376723328,"thread_start_us":42283,"threads_started":1,"wal-append.queue_time_us":90779}
I20260709 13:17:22.134809 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:22.135012 14179 raft_consensus.cc:3055] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:22.135583 14179 raft_consensus.cc:740] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:22.136154 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:22.136780 14179 consensus_queue.cc:260] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:22.137743 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:22.138502 10875 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:22.139556 13995 consensus_queue.cc:1059] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:22.141666 13995 raft_consensus.cc:3055] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:22.142024 13995 raft_consensus.cc:740] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:22.144039 10944 raft_consensus.cc:1275] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:17:22.145490 10916 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:22.138453 14179 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:22.150940 10875 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 7.
W20260709 13:17:22.152093 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:22.152801 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 212299us
I20260709 13:17:22.094094 14049 raft_consensus.cc:2749] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 7. Current term is 8.
I20260709 13:17:22.159070 10950 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:22.195276 10698 catalog_manager.cc:5697] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:22.197186 10698 catalog_manager.cc:5697] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:17:22.198942 13274 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:22.199496 13274 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:22.199927 13274 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:22.200135  9682 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:17:22.254168 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:22.205315 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:22.292062 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:22.307014 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:22.307689 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:22.311154 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:22.291039 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:22.291618 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:22.198594 10698 catalog_manager.cc:5697] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:17:22.319998 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:17:22.281991 14148 consensus_meta.cc:287] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.501s	user 0.004s	sys 0.008s
W20260709 13:17:22.343739 14058 consensus_meta.cc:287] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.503s	user 0.013s	sys 0.000s
I20260709 13:17:22.344487 11511 raft_consensus.cc:2410] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:17:22.344370 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:22.345654 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:22.346339 10867 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 8.
I20260709 13:17:22.346737 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:22.347752 11505 raft_consensus.cc:2393] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 8: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
W20260709 13:17:22.348503 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:22.349489 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:22.349613 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:22.350311 11506 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:22.350425 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:22.351018 11478 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:22.351307 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:22.334352 11512 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 7: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:22.343339 14148 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:22.351188 11510 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:22.353780 14148 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:22.354115 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:22.354643 14191 raft_consensus.cc:2804] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:22.206348 14208 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:22.355808 11508 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 7.
I20260709 13:17:22.355971 14208 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:22.204401 11507 raft_consensus.cc:1240] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:17:22.359362 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:22.360819 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:22.361318 11503 raft_consensus.cc:1240] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:22.361724 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:22.362178 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:22.204806 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:22.366343 10697 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42968 (request call id 245) took 9667 ms. Trace:
I20260709 13:17:22.361248 10842 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:22.368108 14144 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:22.368775 14144 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:22.369328 14144 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:22.371079 10874 raft_consensus.cc:1240] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:17:22.366763 10697 rpcz_store.cc:276] 0709 13:17:12.698728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:12.708694 (+  9966us) service_pool.cc:224] Handling call
0709 13:17:12.771916 (+ 63222us) spinlock_profiling.cc:243] Waited 2.4 ms on lock 0x7b54002b0110. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd910b2 00007f9c86d0aa89 00007f9c86ca2da6 00007f9c86c74088 00007f9c86d7237e 00007f9c852332a4 00007f9c85233232 00007f9c85233161 00007f9c852330dc
0709 13:17:12.781978 (+ 10062us) spinlock_profiling.cc:243] Waited 9.07 ms on lock 0x7b54002b0110. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90e9c 00007f9c7dd910f6 00007f9c86d0aa89 00007f9c86ca2da6 00007f9c86c74088 00007f9c86d7237e 00007f9c852332a4 00007f9c85233232 00007f9c85233161 00007f9c852330dc
0709 13:17:22.366108 (+9584130us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:17.038834 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:17.492226 (+453392us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:17.507020 (+ 14794us) write_op.cc:435] Acquired schema lock
0709 13:17:17.517030 (+ 10010us) tablet.cc:662] Decoding operations
0709 13:17:17.573664 (+ 56634us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:17.573730 (+    66us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:17.573757 (+    27us) tablet.cc:685] Acquiring locks for 18 operations
0709 13:17:17.696464 (+122707us) tablet.cc:701] Row locks acquired
0709 13:17:17.702027 (+  5563us) write_op.cc:260] PREPARE: finished
0709 13:17:17.731233 (+ 29206us) write_op.cc:270] Start()
0709 13:17:17.751382 (+ 20149us) write_op.cc:276] Timestamp: P: 1783603037725593 usec, L: 0
0709 13:17:17.758520 (+  7138us) op_driver.cc:348] REPLICATION: starting
0709 13:17:17.857633 (+ 99113us) log.cc:844] Serialized 7137 byte log entry
0709 13:17:18.810605 (+952972us) op_driver.cc:464] REPLICATION: finished
0709 13:17:18.903721 (+ 93116us) write_op.cc:301] APPLY: starting
0709 13:17:18.903809 (+    88us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:18.904164 (+   355us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:18.904302 (+   138us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:19.063896 (+159594us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:19.064374 (+   478us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=18
0709 13:17:19.064450 (+    76us) write_op.cc:312] APPLY: finished
0709 13:17:19.065005 (+   555us) log.cc:844] Serialized 125 byte log entry
0709 13:17:19.065234 (+   229us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:19.065405 (+   171us) write_op.cc:454] Released schema lock
0709 13:17:19.065464 (+    59us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:19.065862 (+   398us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:19.065891 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":132,"reported_tablets":47,"spinlock_wait_cycles":26367488,"child_traces":[["op",{"apply.queue_time_us":74830,"apply.run_cpu_time_us":9429,"apply.run_wall_time_us":162122,"num_ops":18,"prepare.queue_time_us":315345,"prepare.run_cpu_time_us":85225,"prepare.run_wall_time_us":1135243,"raft.queue_time_us":164510,"raft.run_cpu_time_us":13304,"raft.run_wall_time_us":46307,"replication_time_us":1047095,"thread_start_us":602894,"threads_started":4,"wal-append.queue_time_us":298265,"wal-append.run_cpu_time_us":41460,"wal-append.run_wall_time_us":1909859}]]}
I20260709 13:17:22.360821 11510 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:17:22.203428 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:22.312005 14005 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:22.466619 10945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 461) took 1393 ms. Trace:
I20260709 13:17:22.462138 10767 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:22.451604 10875 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 7.
I20260709 13:17:22.417157 10944 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:22.375524 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:22.487193 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:22.421234 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:22.289492 14211 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:22.492421 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:22.337801 14044 consensus_meta.cc:287] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.571s	user 0.009s	sys 0.006s
W20260709 13:17:22.494349 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 70965.9us
W20260709 13:17:22.495668 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:22.374042 13717 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:22.415127 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:22.491482 10698 catalog_manager.cc:5697] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 6 to 9. New cstate: current_term: 9 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:22.302167 13995 consensus_queue.cc:260] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:22.476163 10945 rpcz_store.cc:276] 0709 13:17:21.073553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:21.544928 (+471375us) service_pool.cc:224] Handling call
0709 13:17:21.674705 (+129777us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:21.959037 (+284332us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:21.959071 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:21.959730 (+   659us) log.cc:844] Serialized 26 byte log entry
0709 13:17:22.154215 (+194485us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:17:22.154280 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:22.154439 (+   159us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:22.466303 (+311864us) raft_consensus.cc:1685] finished
0709 13:17:22.466320 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:22.466450 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":283386,"raft.queue_time_us":283975,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":115,"thread_start_us":193915,"threads_started":1,"wal-append.queue_time_us":504621}
I20260709 13:17:22.511884 10947 raft_consensus.cc:1240] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:22.516605 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:22.515870 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:22.374413 14217 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:22.463340 14208 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:22.520244 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:22.521090 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:22.521808 13890 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:22.523051 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:22.553509 10943 raft_consensus.cc:2393] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 10: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:22.524437 14058 consensus_queue.cc:1059] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:22.524982 14005 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:22.527273 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 260992us
I20260709 13:17:22.527483 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:22.408809 13829 consensus_queue.cc:237] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:22.536576 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:17:22.537417 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 101888us
I20260709 13:17:22.538151 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:22.538707 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:22.538827 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:17:22.540335 10873 consensus_meta.cc:287] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.799s	user 0.012s	sys 0.002s
I20260709 13:17:22.539976 10698 catalog_manager.cc:5697] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 3 to 6, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 6 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:22.560245 10798 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 7.
I20260709 13:17:22.542107 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:22.556238 10797 raft_consensus.cc:2410] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:17:22.406607 14194 consensus_queue.cc:1048] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.145s
I20260709 13:17:22.570097 10873 raft_consensus.cc:2410] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:17:22.574393 10870 raft_consensus.cc:1240] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:22.518014 14148 raft_consensus.cc:2804] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:22.581375 14148 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:22.587814 14148 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:22.591363 10948 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7.
I20260709 13:17:22.592597 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:22.593215 10948 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
I20260709 13:17:22.594020 13633 consensus_queue.cc:1048] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:17:22.600945 10874 raft_consensus.cc:1240] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:22.602262 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:22.605441 14062 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:22.611902 10798 raft_consensus.cc:1275] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:17:22.612314 14211 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:22.613740 14208 raft_consensus.cc:3055] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:22.614202 14208 raft_consensus.cc:740] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:22.615115 14208 consensus_queue.cc:260] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:22.616708 14208 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:22.622555 10766 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:22.623477 13865 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:22.624856 14217 raft_consensus.cc:2749] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:22.624608 13865 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:22.627111 13865 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:22.627056 14217 consensus_queue.cc:1059] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:22.627974 13865 raft_consensus.cc:3055] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:22.551730 13717 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:22.628384 13865 raft_consensus.cc:740] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:22.629601 10876 raft_consensus.cc:2410] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 10 because replica has last-logged OpId of term: 8 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 2.
I20260709 13:17:22.631560 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:22.631966 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:22.632633 10868 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:22.632871 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:22.633634 10876 raft_consensus.cc:2393] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 10: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:22.635176 10875 raft_consensus.cc:1275] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 13:17:22.636309 13717 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:22.630688 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:22.637102 13865 consensus_queue.cc:260] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:22.638402 13865 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:22.561777 14109 consensus_queue.cc:1059] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.121s
I20260709 13:17:22.634462 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:22.641623 10876 raft_consensus.cc:2410] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 8 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:17:22.644152 13995 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:22.644726 13274 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:22.646040 10946 raft_consensus.cc:2393] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 11: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:22.586153 13890 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:22.648587 11506 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10.
I20260709 13:17:22.649104 13890 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:22.649441 10871 raft_consensus.cc:1240] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 7. Current term is 8. Ops: [7.3-7.3]
I20260709 13:17:22.598393 10767 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:22.591194 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:22.640718 14143 consensus_meta.cc:287] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.553s	user 0.000s	sys 0.010s
I20260709 13:17:22.660966 10795 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 8.
I20260709 13:17:22.568825 10698 catalog_manager.cc:5697] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: config changed from index 2 to 4, term changed from 3 to 7, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to <none>, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New cstate: current_term: 7 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:17:22.665000 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:22.666527 10950 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 7.
I20260709 13:17:22.667431 10944 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
W20260709 13:17:22.668816 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:22.676446 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:22.680604 10873 raft_consensus.cc:2393] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 7: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
W20260709 13:17:22.689980 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:22.695065 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:22.666414 13822 consensus_meta.cc:287] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.691s	user 0.010s	sys 0.002s
I20260709 13:17:22.606220 14062 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:22.703569 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:22.677646 13274 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:22.708071 10876 raft_consensus.cc:2393] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 10: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:22.709112 10698 catalog_manager.cc:5697] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:22.694731 14211 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:17:22.731493 10915 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.753s	user 0.024s	sys 0.047s
W20260709 13:17:22.732092 10915 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.754s	user 0.025s	sys 0.048s
W20260709 13:17:22.732502 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 754262us
I20260709 13:17:22.677165 14143 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:22.744171 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 184148us
W20260709 13:17:22.744939 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 391629us
I20260709 13:17:22.713007 14191 raft_consensus.cc:697] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 9 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
W20260709 13:17:22.758553 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:22.761379 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:22.774771 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:22.782982 10876 raft_consensus.cc:2410] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:17:22.784325 10698 catalog_manager.cc:5697] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:22.705370 13822 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:22.739283 14211 raft_consensus.cc:697] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:17:22.803846 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 58632us
I20260709 13:17:22.804147 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:22.805269 10944 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 7: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:22.784144 14217 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:22.817101 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:22.819133 10950 raft_consensus.cc:1240] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:22.822111 14109 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:22.841368 11478 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:17:22.822258 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:22.826953 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:22.793823 14144 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:22.837523 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:22.809142 10698 catalog_manager.cc:5697] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 5 to 8, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 8 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:22.846823 10946 raft_consensus.cc:2393] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 11: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:22.821645 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:22.850322 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:22.851675 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:17:22.840111 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 316057us
I20260709 13:17:22.858029 14243 consensus_queue.cc:1048] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 13:17:22.830116 14148 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:22.871344 10767 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:22.872510 10875 raft_consensus.cc:1240] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 7. Current term is 9. Ops: []
I20260709 13:17:22.873708 10942 raft_consensus.cc:2393] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 9: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:22.875070 10868 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
I20260709 13:17:22.875420 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:22.876328 10798 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:22.878167 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:22.881927 10874 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 8.
I20260709 13:17:22.884968 10792 raft_consensus.cc:1240] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:22.885350 10944 raft_consensus.cc:2393] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 8: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:22.886281 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:22.887157 10767 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:22.887425 10800 raft_consensus.cc:1275] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:17:22.888463 10767 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:22.890209 14179 raft_consensus.cc:2749] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:17:22.890575 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:22.890215 10767 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:22.860623 14243 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:22.891832 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:22.893901 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:22.898092 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 324175us
I20260709 13:17:22.900290 10795 raft_consensus.cc:1240] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:17:22.901108 10792 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:17:22.847538 14108 consensus_queue.cc:1048] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 13:17:22.901471 10698 catalog_manager.cc:5697] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:17:22.844319 14217 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:22.902099 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:22.902771 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:22.901930 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:22.906730 10876 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:22.853147 14109 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:22.908033 10796 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
W20260709 13:17:22.911739 14239 log.cc:927] Time spent T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
I20260709 13:17:22.913753 13822 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:22.904690 11478 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:17:22.897574 14243 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:22.918036 14243 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:22.846901 14191 consensus_queue.cc:237] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:22.898798 14049 raft_consensus.cc:2749] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:22.927902 10793 raft_consensus.cc:1275] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:17:22.936009 10873 raft_consensus.cc:1275] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:17:22.940832 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:22.941895 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:17:22.943050 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:22.944492 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:22.945714 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:22.947419 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:22.947429 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:22.905956 13865 raft_consensus.cc:2749] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:22.948611 10842 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:22.952646 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:22.954015 10801 raft_consensus.cc:2393] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 9: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:22.957158 10794 raft_consensus.cc:1275] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:17:22.957350 10870 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10.
W20260709 13:17:22.918764 10916 connection.cc:256] RPC call timeout handler was delayed by -3.6997982439997941s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1897s for a response
I20260709 13:17:22.956328 10842 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:22.960382 10867 raft_consensus.cc:1275] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:17:22.958425 10798 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7.
I20260709 13:17:22.964365 10868 raft_consensus.cc:2410] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:17:22.896207 14211 consensus_queue.cc:237] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:22.968140 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:17:22.963234 10916 connection.cc:256] RPC call timeout handler was delayed by -3.6394982439999239s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19640000000000002s for a response
W20260709 13:17:22.971624 10916 connection.cc:256] RPC call timeout handler was delayed by -3.6331982439999138s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19710000000000003s for a response
I20260709 13:17:22.970496 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:22.928746 14062 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:22.974274 10798 raft_consensus.cc:2410] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 9 because replica has last-logged OpId of term: 7 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 13:17:22.974555 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 84426.3us
I20260709 13:17:22.984786 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:22.960594 13822 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:22.976289 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:17:22.979847 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:22.981850 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:22.955120 14143 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:22.985507 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:22.987180 10871 raft_consensus.cc:1240] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:17:22.971176 14169 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:22.989053 10767 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:22.952868 14134 consensus_queue.cc:1059] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:17:22.970467 14194 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:23.005985 14109 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:23.006819 14246 raft_consensus.cc:2804] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:23.007515 14194 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:23.008941 13865 raft_consensus.cc:2749] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:17:23.009951 10767 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:23.010246 14194 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:23.003291 10867 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 9.
I20260709 13:17:22.984696 13633 consensus_queue.cc:1059] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.010s
I20260709 13:17:22.991473 13822 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:23.024040 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:23.017328 10767 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:23.004695 14169 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:23.000476 13910 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:23.027741 10946 raft_consensus.cc:1275] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:17:23.036791 14169 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:23.037405 14179 raft_consensus.cc:2749] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:23.038146 10792 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:23.030174 13865 raft_consensus.cc:2749] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:23.041391 10767 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:23.042753 10767 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:23.043541 14049 raft_consensus.cc:2749] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:23.044049 10767 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:23.044272 14049 raft_consensus.cc:2749] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:23.045253 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:23.014323 14144 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:23.025240 13822 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:23.057564 10766 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:23.017702 14246 raft_consensus.cc:697] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 8 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:23.055881 10793 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:23.062160 10792 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 9.
I20260709 13:17:23.065444 10944 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 7.
W20260709 13:17:23.066670 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:23.067457 14049 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:23.067759 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:23.062093 14179 raft_consensus.cc:2749] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:23.070631 14243 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:23.074823 14043 consensus_queue.cc:1048] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.123s
I20260709 13:17:23.075565 14043 consensus_queue.cc:1243] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer e01ecd008edd4aa2b6b2b47efbc2ea87 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:17:23.055100 13910 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:23.091785 13910 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:23.093227 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:23.102942 14245 raft_consensus.cc:3055] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:23.103619 14245 raft_consensus.cc:740] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:23.104801 14245 consensus_queue.cc:260] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:23.111200 10874 raft_consensus.cc:2410] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:17:23.118031 10869 raft_consensus.cc:2410] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:17:23.121049 14217 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:23.024148 14248 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:23.122299 14248 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:23.137935 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 93978.4us
W20260709 13:17:23.139582 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:23.140986 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:23.141680 10801 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7.
W20260709 13:17:23.160305 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 226179us
I20260709 13:17:23.160785 10767 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:23.190387 10876 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 9.
I20260709 13:17:23.193380 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:23.157856 14148 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:23.135439 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:23.228601 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:23.205977 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 66716.3us
I20260709 13:17:23.234555 10767 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:23.229171 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:23.125177 14248 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:23.219894 14246 consensus_queue.cc:237] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:23.238766 10946 raft_consensus.cc:1275] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:17:23.220909 10841 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:23.221480 14251 log.cc:927] Time spent T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 13:17:23.222434 10916 connection.cc:256] RPC call timeout handler was delayed by -2.9455982439999389s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2735s for a response
I20260709 13:17:23.223915 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:23.226627 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:23.241461 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:23.106503 14245 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:23.232340 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:23.246363 10792 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 7.
I20260709 13:17:23.242218 10943 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10.
I20260709 13:17:23.231583 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:23.238214 10801 pending_rounds.cc:85] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 3
W20260709 13:17:23.251731 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:23.243932 10766 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:23.253315 10795 raft_consensus.cc:2410] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 11 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:17:23.254124 10796 raft_consensus.cc:2393] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 8: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:23.187183 14044 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:23.257222 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:23.137941 14243 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:23.206037 14062 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:23.245558 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:23.197991 14179 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:23.246922 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:23.268877 10799 raft_consensus.cc:1240] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:23.270112 10798 raft_consensus.cc:1275] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:17:23.280205 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: true dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:23.270893 13995 consensus_queue.cc:1048] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:17:23.275622 10793 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 11.
I20260709 13:17:23.283066 10792 raft_consensus.cc:1240] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:17:23.277527 10796 raft_consensus.cc:1240] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:23.255151 10842 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:23.251775 10801 pending_rounds.cc:107] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:17:23.082597 14169 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:23.280946 10875 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 7: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:23.281631 10869 raft_consensus.cc:1240] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:23.287358 10842 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:23.284847 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:23.285333 10801 raft_consensus.cc:2889] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:23.288064 10874 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 8.
I20260709 13:17:23.198313 14217 consensus_queue.cc:1059] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.083s
I20260709 13:17:23.261458 14148 raft_consensus.cc:2764] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
W20260709 13:17:23.293977 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 59531.3us
I20260709 13:17:23.289765 13910 raft_consensus.cc:2804] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:23.275939 14005 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:23.291433 10941 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 8.
I20260709 13:17:23.296303 13719 raft_consensus.cc:2804] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:23.313935 13719 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:23.314334 13719 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:23.320078 10915 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:23.322321 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:23.323228 13910 raft_consensus.cc:697] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:23.324544 13910 consensus_queue.cc:237] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:23.324990 13719 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:23.329608 14109 raft_consensus.cc:3055] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:23.330228 14109 raft_consensus.cc:740] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:23.331194 11477 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:17:23.332970 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 112247us
I20260709 13:17:23.339566 10841 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:23.285478 14248 raft_consensus.cc:2749] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:23.284444 14062 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:23.360364 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:23.338272 13274 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:23.361100 10793 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:23.342427 14181 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:23.349550 13865 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:23.307224 14179 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:23.371897 10792 raft_consensus.cc:1240] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:17:23.365788 10842 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:23.335836 14005 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:23.355335 14148 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:17:23.388711 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 55031us
I20260709 13:17:23.390228 10841 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:23.352453 14049 raft_consensus.cc:697] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:23.367065 13274 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:23.367372 14181 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:23.362682 14194 raft_consensus.cc:2749] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:23.401266  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:23.392189 13995 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:17:23.399295 13274 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:23.411973 13995 raft_consensus.cc:697] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:23.412973 13995 consensus_queue.cc:237] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:23.399513 14181 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:23.398588 14043 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:23.396435 13865 raft_consensus.cc:697] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:23.373708 14248 consensus_queue.cc:1048] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.033s
W20260709 13:17:23.418188 14241 log.cc:927] Time spent T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.502s	user 0.000s	sys 0.005s
W20260709 13:17:23.417196 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 356058us
I20260709 13:17:23.375234 14109 consensus_queue.cc:260] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:23.445433 11510 raft_consensus.cc:1275] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:17:23.463363 10793 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
I20260709 13:17:23.407860 14148 consensus_queue.cc:1059] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 13:17:23.483054 10841 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:23.484637 14109 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:17:23.485294 14253 log.cc:927] Time spent T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.325s	user 0.001s	sys 0.001s
I20260709 13:17:23.486886 14245 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:23.487656 14245 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:23.490254 14245 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:23.417770 13829 consensus_queue.cc:1059] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:17:23.574966 13829 consensus_queue.cc:1243] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer 12682eb70bc84c76879e04a8628ee586 log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:17:23.575619 13829 raft_consensus.cc:3055] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:23.576068 13829 raft_consensus.cc:740] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:23.577185 13829 consensus_queue.cc:260] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:23.491153 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:23.579458 13719 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:23.580621 14258 log.cc:927] Time spent T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
W20260709 13:17:23.584220 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:23.492653 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:23.490302 10944 raft_consensus.cc:1275] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:17:23.585568 11507 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 11.
W20260709 13:17:23.539609 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:23.539196 13865 consensus_queue.cc:237] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:23.547488 14254 log.cc:927] Time spent T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.215s	user 0.000s	sys 0.000s
I20260709 13:17:23.547806 11504 raft_consensus.cc:1275] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:17:23.549496 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:23.647578 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:23.565285 14062 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:23.583057 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:23.613783 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:23.659693 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 187858us
I20260709 13:17:23.664661 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:23.510083 14169 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:23.608630 14129 raft_consensus.cc:2749] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:17:23.551838 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:23.478590 14005 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:23.574383 13822 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:23.674232 11478 connection.cc:256] RPC call timeout handler was delayed by -1.0365739249998569s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1888s for a response
W20260709 13:17:23.643121 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 52899.8us
I20260709 13:17:23.676224 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:23.683903 10946 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:23.654076 13829 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:23.642644 14044 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:17:23.680083 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:23.685706 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 389839us
I20260709 13:17:23.668322 14144 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:23.691066 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:23.703922 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 460101us
W20260709 13:17:23.742216 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 61925.6us
I20260709 13:17:23.742486 10842 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:23.744257 10841 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:23.681501 11506 raft_consensus.cc:1275] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:17:23.696779 14044 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:23.685446 14109 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:23.775483 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 71172.3us
I20260709 13:17:23.775552 14109 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:23.776449 10767 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:23.777258 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:23.777730 14109 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:23.780506 14043 raft_consensus.cc:2749] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:23.780673 10944 raft_consensus.cc:1275] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:17:23.760594 10916 connection.cc:256] RPC call timeout handler was delayed by -2.3479095529999086s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1525s for a response
I20260709 13:17:23.762590 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:23.781351 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:23.766919 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:23.783181 10950 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 11.
W20260709 13:17:23.767477 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:23.767338 10841 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:23.688325 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:23.766343 14208 raft_consensus.cc:2804] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:23.774979 14248 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:23.806723 10944 raft_consensus.cc:3055] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:23.814613 10944 raft_consensus.cc:740] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:17:23.811734 10916 connection.cc:256] RPC call timeout handler was delayed by -2.0239095529998394s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1885s for a response
I20260709 13:17:23.748055 14194 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:23.755273 14179 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:23.827140 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 71194.9us
I20260709 13:17:23.819046 10841 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:23.768312 14169 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:23.818488 14109 raft_consensus.cc:2804] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:23.755719 14191 consensus_queue.cc:1059] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.074s
I20260709 13:17:23.864233 10766 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:23.756559 14144 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:23.837091 10944 consensus_queue.cc:260] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:23.874210 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 63548.5us
I20260709 13:17:23.876580 10841 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:23.878444 10841 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:23.829116 14208 raft_consensus.cc:697] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:17:23.879323 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 102967us
I20260709 13:17:23.861887 14244 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:23.892087 10944 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:23.884367 10767 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:23.868363 13717 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:23.868530 14194 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:23.868992 14109 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:23.868794 14179 raft_consensus.cc:3055] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:23.906494 13719 raft_consensus.cc:2804] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:17:23.944343 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 122866us
W20260709 13:17:23.918134 14262 log.cc:927] Time spent T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:17:23.884461 14191 consensus_queue.cc:1243] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer e01ecd008edd4aa2b6b2b47efbc2ea87 log is divergent from this leader: its last log entry 5.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:17:23.885816 13910 raft_consensus.cc:2749] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:17:23.958142 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 79780.4us
I20260709 13:17:23.869920 14217 consensus_queue.cc:1059] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.067s
I20260709 13:17:23.906939 14270 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:23.734167 14049 consensus_queue.cc:237] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:23.946521 14109 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:23.978410 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 134886us
I20260709 13:17:23.979152 14274 raft_consensus.cc:2749] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:17:23.957391 13719 raft_consensus.cc:697] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:23.912031 13274 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:23.984427 14278 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:17:23.985066 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 100800us
I20260709 13:17:23.878583 14169 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:23.996218 10766 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:23.947250 14179 raft_consensus.cc:740] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:17:24.002526 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 182295us
W20260709 13:17:24.003311 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:24.031495 10767 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:24.033331 10767 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:23.915789 14208 consensus_queue.cc:237] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:24.063364 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 67211.9us
I20260709 13:17:24.065165 13865 raft_consensus.cc:2749] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:17:24.065940 13865 consensus_queue.cc:1048] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:24.066354 14272 raft_consensus.cc:2749] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:24.039242 14260 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:17:24.105701 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 144576us
I20260709 13:17:24.015708 14217 consensus_queue.cc:1048] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 13:17:24.120237 10841 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:24.124402 10841 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:24.126209 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 52045.4us
I20260709 13:17:24.065181 14275 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:24.151175 10944 raft_consensus.cc:1275] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:17:24.151755 10944 pending_rounds.cc:85] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 7
I20260709 13:17:24.152043 10944 pending_rounds.cc:107] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:17:24.152333 10944 raft_consensus.cc:2889] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:24.113762 14260 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:24.172116 14275 raft_consensus.cc:697] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:24.137497 13717 raft_consensus.cc:2749] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:24.130879 14044 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:24.188791 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:24.179019 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 114914us
I20260709 13:17:24.191478 13719 consensus_queue.cc:237] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:24.182809 14296 raft_consensus.cc:2749] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:24.208323 10767 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:24.176163 14260 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:24.129921 14129 consensus_queue.cc:1048] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.067s
I20260709 13:17:24.130219 14274 consensus_queue.cc:1048] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.085s
W20260709 13:17:24.226269 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 62748.6us
I20260709 13:17:23.934832 14244 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:24.227975 14289 log.cc:927] Time spent T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:17:24.229478 11478 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:24.230944 14191 consensus_queue.cc:1059] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:17:24.229733 14291 log.cc:927] Time spent T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:17:24.129772 14179 consensus_queue.cc:260] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:24.231309 14108 raft_consensus.cc:2749] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:24.204830 14275 consensus_queue.cc:237] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:24.232931 14286 log.cc:927] Time spent T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.188s	user 0.000s	sys 0.000s
W20260709 13:17:24.252475 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 188220us
I20260709 13:17:24.003844 14270 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:24.252127 10801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2146) took 1053 ms. Trace:
I20260709 13:17:24.233235 14044 raft_consensus.cc:2749] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:24.231070 14144 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:24.232033 14191 raft_consensus.cc:3055] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:17:24.248827 14181 consensus_meta.cc:287] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.813s	user 0.013s	sys 0.000s
I20260709 13:17:24.184499 14276 consensus_queue.cc:1048] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.106s
I20260709 13:17:24.255045 10801 rpcz_store.cc:276] 0709 13:17:23.195235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:23.233938 (+ 38703us) service_pool.cc:224] Handling call
0709 13:17:23.234349 (+   411us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:23.690908 (+456559us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:23.690938 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:23.703254 (+ 12316us) log.cc:844] Serialized 52 byte log entry
0709 13:17:23.947635 (+244381us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:23.947683 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:23.947748 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:24.247406 (+299658us) raft_consensus.cc:1685] finished
0709 13:17:24.247439 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:24.248089 (+   650us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41379,"raft.queue_time_us":166910,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":17756,"thread_start_us":526746,"threads_started":2,"wal-append.queue_time_us":260136}
W20260709 13:17:24.264991 10946 consensus_meta.cc:287] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.581s	user 0.010s	sys 0.000s
I20260709 13:17:24.268966 14292 consensus_queue.cc:1048] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 13:17:24.267982 14179 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:24.282986 10946 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:17:24.283524 14244 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:24.267166 14287 log.cc:927] Time spent T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.241s	user 0.001s	sys 0.000s
I20260709 13:17:24.269811 14109 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:24.270644 13274 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:24.271467 14281 log.cc:927] Time spent T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.222s	user 0.000s	sys 0.000s
I20260709 13:17:24.272581 10947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 483) took 1081 ms. Trace:
I20260709 13:17:24.273094 11478 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:24.279160 14169 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:24.280233 14248 raft_consensus.cc:2749] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:24.280758 10794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1692) took 1032 ms. Trace:
W20260709 13:17:24.292225 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:24.306450 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:24.308081 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:24.292126 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 73059.5us
I20260709 13:17:24.299324 10947 rpcz_store.cc:276] 0709 13:17:23.191040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:23.191161 (+   121us) service_pool.cc:224] Handling call
0709 13:17:23.232040 (+ 40879us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:23.247979 (+ 15939us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:17:23.587452 (+339473us) log.cc:844] Serialized 14 byte log entry
0709 13:17:23.862481 (+275029us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:23.891192 (+ 28711us) log.cc:844] Serialized 26 byte log entry
0709 13:17:23.891414 (+   222us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:23.891464 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:23.891564 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:24.272256 (+380692us) raft_consensus.cc:1685] finished
0709 13:17:24.272292 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:24.272495 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9011,"prepare.queue_time_us":306174,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":21832,"raft.run_cpu_time_us":258,"raft.run_wall_time_us":19800,"thread_start_us":509636,"threads_started":2,"wal-append.queue_time_us":303146}
I20260709 13:17:24.266601 14191 raft_consensus.cc:740] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:24.319535 14246 raft_consensus.cc:3055] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:24.319995 14246 raft_consensus.cc:740] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
W20260709 13:17:24.323042 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 50118.7us
I20260709 13:17:24.325519 11478 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:24.313863 10916 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:24.303624 10794 rpcz_store.cc:276] 0709 13:17:23.248635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:23.276880 (+ 28245us) service_pool.cc:224] Handling call
0709 13:17:23.290985 (+ 14105us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:23.704134 (+413149us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:23.704173 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:23.768545 (+ 64372us) log.cc:844] Serialized 52 byte log entry
0709 13:17:23.970772 (+202227us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:23.970825 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:23.970894 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:24.280534 (+309640us) raft_consensus.cc:1685] finished
0709 13:17:24.280555 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:24.280698 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13300,"raft.queue_time_us":133961,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":12865,"thread_start_us":539325,"threads_started":2,"wal-append.queue_time_us":251787}
I20260709 13:17:24.317787 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:24.286851 14181 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:24.332078 14301 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.332608 14301 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:24.334075 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:24.334704 10796 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
I20260709 13:17:24.335173 14181 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:24.339071 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:24.339541 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:24.347993 10944 raft_consensus.cc:1275] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:17:24.351328 13274 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:24.352670 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:24.353322 10949 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:24.354009 14194 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:24.356571 14109 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:24.361336 14288 log.cc:927] Time spent T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.296s	user 0.000s	sys 0.011s
I20260709 13:17:24.347811 10916 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:24.362964 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:24.362766 13274 raft_consensus.cc:2749] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:24.364202 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:24.364786 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:24.365093 10943 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 7.
I20260709 13:17:24.366662 14298 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.340574 11478 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:24.317016 14108 raft_consensus.cc:2749] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:24.391430 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:24.392949 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 45245us
I20260709 13:17:24.394933 10945 raft_consensus.cc:1275] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:17:24.396665 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:24.398634 10946 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
I20260709 13:17:24.302891 14292 consensus_queue.cc:1048] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.066s
I20260709 13:17:24.408110 10916 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:24.377954 14244 raft_consensus.cc:2749] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:17:24.428033 10841 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:24.430044 10841 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:17:24.431550 10841 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:24.402412 14148 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:24.406239 14292 consensus_queue.cc:1243] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer 468a8fd46e824f62b9c2c0d3532a6d26 log is divergent from this leader: its last log entry 8.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:17:24.402696 14245 consensus_queue.cc:1048] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:17:24.431985 14274 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.437428 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:24.437752 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:24.438297 10916 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:24.419168 14043 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.428100 13865 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.420907 14005 raft_consensus.cc:2749] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:24.367239 14298 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:24.446975 10794 raft_consensus.cc:2393] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 6: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:24.441171 10949 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 9.
I20260709 13:17:24.432494 14169 raft_consensus.cc:2749] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:24.428081 14062 raft_consensus.cc:2749] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:24.451217 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:24.462965 10793 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:17:24.452852 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:24.435696 14248 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:24.454241 14134 raft_consensus.cc:2749] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:24.451895 14274 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:24.471184 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:24.472148 14274 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:24.454977 14005 raft_consensus.cc:2749] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:17:24.455112 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 114624us
I20260709 13:17:24.449230 10801 raft_consensus.cc:1275] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
W20260709 13:17:24.472795 14267 log.cc:927] Time spent T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.218s	user 0.000s	sys 0.001s
I20260709 13:17:24.473131 14248 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:24.473344 11478 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:24.473647 14248 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:24.474673 11478 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:24.443034 13995 consensus_queue.cc:1048] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:17:24.477299 14311 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:24.452677 14043 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:24.479638 11503 raft_consensus.cc:1240] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 8 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:17:24.480156 14043 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:24.480446 14284 log.cc:927] Time spent T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.414s	user 0.000s	sys 0.013s
W20260709 13:17:24.485034 13829 consensus_meta.cc:287] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.795s	user 0.011s	sys 0.000s
I20260709 13:17:24.485122 14311 raft_consensus.cc:2749] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:24.485677 13822 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:24.486238 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:24.465764 10943 raft_consensus.cc:3055] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:24.486940 10943 raft_consensus.cc:740] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:24.487769 10943 consensus_queue.cc:260] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:24.488734 10943 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:24.444764 14247 consensus_queue.cc:1048] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:17:24.461261 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:24.476656 14310 raft_consensus.cc:2749] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:24.448717 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: true dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:24.499997 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:24.500541 10800 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 7: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:24.501165 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:24.525939 10867 raft_consensus.cc:1240] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:17:24.527103 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:24.528048 11504 raft_consensus.cc:2393] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 7: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:24.527980 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:24.527168 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:24.528801 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:24.529364 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 104597us
I20260709 13:17:24.529377 10792 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:24.532503 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:24.532892 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:24.533957 10842 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:24.534088 11507 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 7.
I20260709 13:17:24.534716 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:24.535326 10946 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:24.532871 10798 raft_consensus.cc:1275] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:17:24.536873 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:24.537825 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:24.490645 14134 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.538663 11510 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:24.541119 10950 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10.
I20260709 13:17:24.449326 14260 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:24.450261 14148 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:24.542719 10799 raft_consensus.cc:1275] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:17:24.540771 11477 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:24.560699 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:24.561560 10798 raft_consensus.cc:2376] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 8. Re-sending same reply.
I20260709 13:17:24.545087 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:24.563016 10796 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:24.544284 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:24.500348 14311 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.554955 10867 raft_consensus.cc:1275] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:17:24.564759 10869 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:24.570807 10766 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:24.574056 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:24.462674 13865 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:24.576653 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:24.575153 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:24.579799 10867 raft_consensus.cc:2410] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:17:24.577340 10947 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 7.
W20260709 13:17:24.589488 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 53961.4us
I20260709 13:17:24.561604 14194 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:17:24.591774 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 153528us
W20260709 13:17:24.597503 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:24.601042 11478 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:24.597815 10793 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 10.
I20260709 13:17:24.604310 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:24.599210 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:24.562341 14301 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:24.606365 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:24.607021 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:24.605221 10867 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10.
I20260709 13:17:24.607673 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:24.608294 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:24.626434 10873 raft_consensus.cc:1240] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:24.625201 10798 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:17:24.628302 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:24.578801 14306 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:24.626111 10801 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 7.
I20260709 13:17:24.632300 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:24.633162 10874 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:24.585027 14274 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:24.619755 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:24.620283 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:24.577586 14032 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:24.542979 14043 consensus_queue.cc:1048] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:17:24.543529 14245 consensus_queue.cc:1048] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 13:17:24.623581 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:24.610211 10766 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:24.581282 14270 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:24.635893 10916 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:24.610998 11510 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10.
I20260709 13:17:24.646377 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:24.607419 14194 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:24.560058 14134 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:24.650507 10792 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
I20260709 13:17:24.672986 10842 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:24.654130 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:24.656877 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:24.656997 10946 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
I20260709 13:17:24.655730 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:24.662604 10793 raft_consensus.cc:2393] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 10: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
W20260709 13:17:24.662595 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 52512us
I20260709 13:17:24.578437 14272 consensus_queue.cc:1059] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:17:24.634847 14306 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:24.677126 10872 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 7.
I20260709 13:17:24.681708 11504 raft_consensus.cc:2393] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 10: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:24.578094 14311 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:24.649780 11503 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 8 LEARNER]: Advancing to term 10
I20260709 13:17:24.685587 10798 raft_consensus.cc:1275] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:17:24.689405 10801 raft_consensus.cc:1240] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 6. Current term is 7. Ops: [4.6-6.7]
I20260709 13:17:24.673696 14301 raft_consensus.cc:2804] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:24.675283 14194 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:24.679240 14274 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:24.680173 13829 raft_consensus.cc:2764] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election decision vote started in defunct term 8: won
W20260709 13:17:24.767089 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:24.746074 10874 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 8.
I20260709 13:17:24.746272 10916 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
W20260709 13:17:24.674202 14278 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.609s	user 0.007s	sys 0.009s
I20260709 13:17:24.687521 14244 raft_consensus.cc:2749] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:17:24.790076 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 257175us
I20260709 13:17:24.648712 13822 consensus_queue.cc:1048] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 13:17:24.614223 14246 consensus_queue.cc:260] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:24.790501 10915 leader_election.cc:336] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Timed out: RequestConsensusVote RPC to 127.9.116.132:40555 timed out after 1.528s (SENT)
I20260709 13:17:24.792268 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:24.687286 14272 consensus_queue.cc:1243] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer 12682eb70bc84c76879e04a8628ee586 log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:17:24.679378 14179 consensus_queue.cc:1059] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.068s
I20260709 13:17:24.683792 14306 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:24.800513 10767 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:24.684971 14308 consensus_queue.cc:1048] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.116s
W20260709 13:17:24.803717 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 202802us
I20260709 13:17:24.771167 14032 raft_consensus.cc:697] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 9 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:24.808606 10841 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:24.810606 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 64447.8us
I20260709 13:17:24.821892 10874 raft_consensus.cc:1240] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:17:24.815477 10872 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:17:24.802482 14278 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.738s	user 0.017s	sys 0.019s
I20260709 13:17:24.740442 14144 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:24.797844 14049 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:24.798616 14301 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:24.823362 10841 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:24.807721 14309 log.cc:927] Time spent T 00000000000000000000000000000000 P ae957c8fc66c4b6ba063d0c74cbf5484: Append to log took a long time: real 0.329s	user 0.002s	sys 0.001s
I20260709 13:17:24.804009 14292 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:24.829027 10943 raft_consensus.cc:1275] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:17:24.834520 10943 pending_rounds.cc:85] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 3
I20260709 13:17:24.827244 10916 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:24.834823 10943 pending_rounds.cc:107] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:17:24.835216 10943 raft_consensus.cc:2889] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:24.833698 14298 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:24.833817 14314 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:24.836361 14324 raft_consensus.cc:2804] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:24.837014 14324 raft_consensus.cc:697] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:24.837148 14311 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:24.837466 14269 consensus_queue.cc:1059] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 13:17:24.836655 14314 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:24.824617 14308 consensus_queue.cc:1243] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer e01ecd008edd4aa2b6b2b47efbc2ea87 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:17:24.825980 13829 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:24.859782 11503 raft_consensus.cc:1275] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 10 LEARNER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:17:24.862994 14314 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:24.865221 10916 connection.cc:256] RPC call timeout handler was delayed by -1.2681220810000013s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1585s for a response
W20260709 13:17:24.865691 10916 connection.cc:256] RPC call timeout handler was delayed by -1.0719220810001389s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18030000000000002s for a response
W20260709 13:17:24.847910 14316 log.cc:927] Time spent T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
W20260709 13:17:24.848826 14315 log.cc:927] Time spent T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.215s	user 0.001s	sys 0.000s
I20260709 13:17:24.849740 14032 consensus_queue.cc:237] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:24.850953 13274 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:24.834777 14049 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:24.851447 10874 raft_consensus.cc:1275] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:17:24.878652 10796 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 8.
I20260709 13:17:24.852303 14191 consensus_queue.cc:260] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:24.854699 14302 log.cc:927] Time spent T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.320s	user 0.000s	sys 0.001s
I20260709 13:17:24.855330 14243 consensus_queue.cc:1048] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:17:24.835134 14260 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:24.856050 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:24.839002 14292 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:24.871460 13865 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:24.857280 14299 raft_consensus.cc:3055] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:17:24.825692 14278 raft_consensus.cc:697] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:24.894770 14191 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:24.895694 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:24.895761 14278 consensus_queue.cc:237] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:24.859476 13829 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:24.842437 13633 consensus_queue.cc:1059] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:17:24.898950 11506 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 6.
I20260709 13:17:24.852207 14327 raft_consensus.cc:2749] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:17:24.860627 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:24.899710 14299 raft_consensus.cc:740] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:24.860981 10872 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 9.
I20260709 13:17:24.879282 14129 consensus_queue.cc:1048] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 13:17:24.900496 13829 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:24.845396 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:24.900871 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:24.900740 14299 consensus_queue.cc:260] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:24.847512 11504 raft_consensus.cc:1240] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 9 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:17:24.828290 14144 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:24.902140 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:24.902623 14144 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:24.903491 10872 raft_consensus.cc:2376] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 10. Re-sending same reply.
I20260709 13:17:24.904348 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:24.904971 10872 raft_consensus.cc:2393] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 11: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:24.867190 10869 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 8.
I20260709 13:17:24.834334 14208 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:24.908571 11505 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 10.
I20260709 13:17:24.834605 14301 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:24.838086 14324 consensus_queue.cc:237] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:24.852371 14134 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:24.843786 14246 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:24.924167 11504 raft_consensus.cc:1240] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:24.880831 13274 raft_consensus.cc:697] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:24.881760 14274 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:24.882735 14049 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:24.882828 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:24.886426 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:24.889313 10792 raft_consensus.cc:3055] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:24.889882 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:24.894611 10795 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:24.894712 14292 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:24.843398 14248 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:24.894924 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:24.902133 11503 raft_consensus.cc:1240] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:24.906364 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:24.866153 10916 connection.cc:256] RPC call timeout handler was delayed by -1.0647220810001272s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1811s for a response
I20260709 13:17:24.910930 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:24.880162 14169 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:24.864681 14311 consensus_queue.cc:1059] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 13:17:24.902168 14299 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:25.046911 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:25.049366 10869 raft_consensus.cc:2410] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:17:25.007884 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:25.051360 11505 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:25.055088 14323 raft_consensus.cc:3055] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:24.899749 14326 raft_consensus.cc:3055] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:25.055711 14323 raft_consensus.cc:740] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
W20260709 13:17:25.047448 10916 connection.cc:256] RPC call timeout handler was delayed by -1.0530220810001083s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1824s for a response
I20260709 13:17:25.055933 14326 raft_consensus.cc:740] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
W20260709 13:17:25.056398 10916 connection.cc:256] RPC call timeout handler was delayed by -1.0071220810000341s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1875s for a response
I20260709 13:17:25.057144 10916 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:25.056939 14323 consensus_queue.cc:260] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:25.058156 13829 raft_consensus.cc:2749] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:25.057439 14326 consensus_queue.cc:260] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:25.058475 14323 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:25.059516 10916 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:25.060423 13829 raft_consensus.cc:2749] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:25.062407 10916 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:25.063251 13829 raft_consensus.cc:2749] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:25.064546 10916 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:25.065338 13829 raft_consensus.cc:2749] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:17:25.047348 10792 raft_consensus.cc:740] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:17:25.007308 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 206871us
I20260709 13:17:25.067338 10792 consensus_queue.cc:260] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:25.068557 10792 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:25.069473 10916 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:25.070266 13829 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:25.071604 10916 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:25.072408 13829 raft_consensus.cc:2749] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:24.909893 14208 raft_consensus.cc:697] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:25.054144 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:25.076745 10916 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:25.083555 11505 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 8.
I20260709 13:17:25.086019 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:25.086632 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:25.087050 10874 raft_consensus.cc:1240] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:25.073025 14269 consensus_queue.cc:1059] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:17:25.087905 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:25.086165 10796 raft_consensus.cc:1275] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:17:25.046850 14169 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:25.046260 13274 consensus_queue.cc:237] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:25.215519 10868 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10.
I20260709 13:17:25.090651 10792 raft_consensus.cc:1240] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:25.153203 10872 raft_consensus.cc:1240] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
W20260709 13:17:25.098556 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:25.090044 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:25.125566 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 48892.1us
W20260709 13:17:25.167685 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 333814us
I20260709 13:17:25.116149 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:25.222055 10916 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:25.090723 10874 raft_consensus.cc:1240] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:17:25.116336 11478 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:25.223085 10841 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:25.182925 10795 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 10.
I20260709 13:17:25.226294 10841 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:25.224717 11505 raft_consensus.cc:1240] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 9. Current term is 10. Ops: []
W20260709 13:17:25.225907 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 86426.8us
W20260709 13:17:25.161131 14211 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 13:17:25.059525 14326 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:25.229117 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 112861us
I20260709 13:17:25.094640 14208 consensus_queue.cc:237] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:25.150121 14308 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:25.094064 14248 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:25.236579 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:25.240233 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:25.242689 10947 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:25.230698 14181 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:25.126214 13829 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:25.254994 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:25.256788 10872 raft_consensus.cc:1240] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 9. Current term is 10. Ops: [7.3-9.4]
I20260709 13:17:25.228475 14336 raft_consensus.cc:2749] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:25.110466 14306 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:25.260852 14306 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:25.263742 10871 raft_consensus.cc:1240] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:17:25.263939 14169 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:25.112454 14144 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:25.266526 14144 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:25.271737 10916 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:25.272441 14329 log.cc:927] Time spent T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:17:25.273958 11478 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:25.280187 14148 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:25.285553 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:25.300127 10946 raft_consensus.cc:3055] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:25.286412 10941 raft_consensus.cc:1240] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:25.287207 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:25.301019 10948 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:25.288013 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:25.240392 14314 raft_consensus.cc:2749] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:17:25.274596 14335 log.cc:927] Time spent T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:17:25.243286 14308 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:25.105885 14194 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:25.300510 10946 raft_consensus.cc:740] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:25.235251 14330 raft_consensus.cc:2804] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:25.302119 10943 raft_consensus.cc:2393] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 10: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:25.304996 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:17:25.320881 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:25.324620 11505 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7.
I20260709 13:17:25.236908 13822 consensus_queue.cc:1048] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.065s
W20260709 13:17:25.327752 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 56103.5us
I20260709 13:17:25.334649 10841 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:25.338091 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 64194.3us
I20260709 13:17:25.352806 11478 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:25.353509 10841 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:25.358193 10916 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:25.358335 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:25.360427 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:25.360980 10944 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:25.362185 11478 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:25.319607 14181 raft_consensus.cc:2749] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:25.328430 10946 consensus_queue.cc:260] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:25.368191 14194 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:25.370265 14323 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:25.389904 14292 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:25.373124 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:25.376513 10947 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 12.
I20260709 13:17:25.338626 14330 raft_consensus.cc:697] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:25.327471 14269 consensus_queue.cc:1243] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer e01ecd008edd4aa2b6b2b47efbc2ea87 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:17:25.371955 14308 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:25.334688 14217 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:25.353890 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:25.409256 11506 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:25.411094 14294 raft_consensus.cc:2749] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:25.412449 14217 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:25.415016 14269 raft_consensus.cc:2749] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:25.415743 14269 raft_consensus.cc:2749] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:17:25.416402 14292 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:25.416518 13829 raft_consensus.cc:2749] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:25.417376 14217 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:25.396602 10841 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:17:25.402777 10946 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:25.407145 14134 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:25.407929 11508 raft_consensus.cc:2410] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:17:25.408422 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:25.422104 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:25.424741 10944 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
I20260709 13:17:25.429790 10949 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:17:25.432960 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:17:25.433694 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 75529.6us
I20260709 13:17:25.433681 11510 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
I20260709 13:17:25.437793 10916 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:25.438153 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 76106.1us
I20260709 13:17:25.440653 11478 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:25.439920 10948 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
I20260709 13:17:25.441833 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:25.442302 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:25.442636 10944 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:17:25.443282 10948 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 10.
I20260709 13:17:25.445111 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:25.337294 14306 consensus_queue.cc:1059] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.084s
I20260709 13:17:25.446497 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:25.447423 10947 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:25.444435 14323 raft_consensus.cc:697] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 10 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:25.455680 10841 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:17:25.459879 14345 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:17:25.465643 14308 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:25.466068 14308 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:25.466421 14308 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:25.441855 14300 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:25.445145 14244 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:25.439042 14194 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:25.451012 14044 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:25.483577 10944 raft_consensus.cc:1240] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 7. Current term is 8. Ops: [5.5-7.6]
I20260709 13:17:25.441711 14217 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:25.493646 10841 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:25.475203 11478 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:25.516078 10916 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:25.493844 10946 raft_consensus.cc:1275] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:17:25.477715 14300 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:25.521445 10870 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 8.
I20260709 13:17:25.586398 10841 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:17:25.586853 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 111716us
I20260709 13:17:25.587627 10841 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:25.495364 11477 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:25.487509 14211 raft_consensus.cc:3055] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:25.521138 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:25.597025 10801 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 8.
I20260709 13:17:25.589164 11478 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:25.484589 10947 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
W20260709 13:17:25.595681 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 100378us
I20260709 13:17:25.518677 14300 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:25.608434 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:17:25.611514 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:25.621291 10794 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 10.
I20260709 13:17:25.622402 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:17:25.520784 14301 consensus_meta.cc:287] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.596s	user 0.011s	sys 0.000s
I20260709 13:17:25.623365 11505 raft_consensus.cc:2376] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 6. Re-sending same reply.
I20260709 13:17:25.484565 14244 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:25.581806 14269 raft_consensus.cc:2804] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:25.521786 14336 raft_consensus.cc:2749] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:17:25.596045 14247 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:17:25.595912 14129 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:25.596493 14211 raft_consensus.cc:740] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:25.640326 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:25.512338 14343 consensus_queue.cc:1059] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.135s
I20260709 13:17:25.643785 10800 raft_consensus.cc:1240] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:25.518710 14292 consensus_queue.cc:1059] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.106s
I20260709 13:17:25.511559 14194 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:25.517308 14044 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:25.645691 11511 raft_consensus.cc:1240] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:25.603868 14246 raft_consensus.cc:2804] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:25.625947 14257 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:25.583292 14217 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:25.592983 14181 raft_consensus.cc:2749] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:25.596397 14294 consensus_queue.cc:1048] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 13:17:25.682139 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:25.683584 14294 consensus_queue.cc:1243] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer cb0428cc55984daba1ba63744a5102a2 log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:17:25.684033 11511 raft_consensus.cc:2368] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 7. Current term is 8.
I20260709 13:17:25.684684 10876 raft_consensus.cc:1275] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:17:25.595811 14062 consensus_queue.cc:1048] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:17:25.684993 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:25.685590 11511 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 8.
I20260709 13:17:25.685796 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:25.681874 14275 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:25.686388 10876 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:25.686451 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:25.687500 11509 raft_consensus.cc:2376] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10. Re-sending same reply.
I20260709 13:17:25.688151 14062 consensus_queue.cc:1048] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:17:25.689715 14044 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:25.667008 14320 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:25.690446 14320 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:25.638091 14244 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:25.683072 10874 raft_consensus.cc:1275] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:17:25.686740 14275 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:25.697175 14275 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:25.635619 14301 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:25.699082 14301 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:25.702241 14062 consensus_queue.cc:1048] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:17:25.706575 14194 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:25.708379 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:25.709183 10871 raft_consensus.cc:2393] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 7: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:25.650771 14247 raft_consensus.cc:697] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:25.710825 14247 consensus_queue.cc:237] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:25.682852 14257 raft_consensus.cc:697] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 8 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:25.713529 14257 consensus_queue.cc:237] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:25.715770 10797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2158) took 1382 ms. Trace:
I20260709 13:17:25.716166 10797 rpcz_store.cc:276] 0709 13:17:24.333137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:24.336497 (+  3360us) service_pool.cc:224] Handling call
0709 13:17:24.336794 (+   297us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:25.295405 (+958611us) spinlock_profiling.cc:243] Waited 1.57 ms on lock 0x7b18003e8240. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c80a5d4bc 00007f9c7ddab48b 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc5455 00007f9c80ead060 00007f9c80eadbfb 00007f9c80eb864b
0709 13:17:25.295556 (+   151us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:17:25.295582 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:25.296119 (+   537us) log.cc:844] Serialized 26 byte log entry
0709 13:17:25.446261 (+150142us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:25.446324 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:25.446393 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:25.715431 (+269038us) raft_consensus.cc:1685] finished
0709 13:17:25.715451 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:25.715704 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2046,"raft.queue_time_us":55410,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":21827,"spinlock_wait_cycles":3601664,"thread_start_us":233628,"threads_started":2,"wal-append.queue_time_us":300207}
I20260709 13:17:25.644743 14129 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:25.680115 14246 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:25.781863 10876 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 10.
I20260709 13:17:25.782869 10875 raft_consensus.cc:1240] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:25.678453 14296 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:25.671456 14312 consensus_queue.cc:1048] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.104s
I20260709 13:17:25.643013 14323 consensus_queue.cc:237] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:25.765856 14129 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:25.751089 14320 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:25.839035 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 342248us
I20260709 13:17:25.604007 14330 consensus_queue.cc:237] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.5, Last appended by leader: 5, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:25.792964 14246 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:25.796039 14341 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:25.846143 14301 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:25.835042 14296 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:25.772792 14211 consensus_queue.cc:260] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:25.853145 14326 consensus_meta.cc:287] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.619s	user 0.008s	sys 0.003s
W20260709 13:17:25.874876 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 236840us
I20260709 13:17:25.923235 10915 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:25.910379 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:25.849196 10766 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:25.925645 11504 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 7.
W20260709 13:17:25.928381 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 79319.7us
I20260709 13:17:25.929315 14275 raft_consensus.cc:2749] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:25.910513 14308 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:25.932827 14300 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:25.934238 14308 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:25.934484 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:25.897171 14301 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:25.931439 11506 consensus_meta.cc:287] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.522s	user 0.010s	sys 0.001s
I20260709 13:17:25.935869 11506 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 12.
I20260709 13:17:25.937283 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:25.940304 10766 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:25.884318 14346 log.cc:927] Time spent T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 13:17:25.936866 14300 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:25.900633 14217 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:25.897475 14296 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:25.912412 14134 raft_consensus.cc:2749] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:25.990095 10766 leader_election.cc:400] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 election: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 7. Current term is 8.
I20260709 13:17:25.863961 14269 raft_consensus.cc:697] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 10 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:25.908042 14354 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:25.980441 14292 consensus_queue.cc:1243] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer e01ecd008edd4aa2b6b2b47efbc2ea87 log is divergent from this leader: its last log entry 10.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:17:25.995461 10868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 1672) took 1398 ms. Trace:
I20260709 13:17:25.996567 10869 raft_consensus.cc:1275] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:17:26.007355 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:26.040334 11510 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:26.007256 14361 consensus_queue.cc:1059] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:17:25.936802 14301 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:25.957770 14275 consensus_queue.cc:1048] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 13:17:26.020431 11508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56416 (request call id 596) took 1520 ms. Trace:
I20260709 13:17:26.044153 11508 rpcz_store.cc:276] 0709 13:17:24.499842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:24.535945 (+ 36103us) service_pool.cc:224] Handling call
0709 13:17:24.860442 (+324497us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b5800003c98. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd90a22 00007f9c86ca9c09 00007f9c85e03ee0 00007f9c85ecd0ab 00007f9c85ecd131 00007f9c85e7b0a0 00007f9c85e7a984 00007f9c7f3f56d4 00007f9c7f3f5662
0709 13:17:26.020194 (+1159752us) spinlock_profiling.cc:243] Waited 507 ms on lock 0x7b54005014d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:26.020361 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1191012096}
I20260709 13:17:26.024976 11503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56404 (request call id 641) took 1160 ms. Trace:
I20260709 13:17:25.912674 14246 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:26.046074 11503 rpcz_store.cc:276] 0709 13:17:24.864603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:24.864791 (+   188us) service_pool.cc:224] Handling call
0709 13:17:24.901967 (+ 37176us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:25.051091 (+149124us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:26.024683 (+973592us) spinlock_profiling.cc:243] Waited 36.9 ms on lock 0x7b54005802d0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:26.024893 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":84855680}
I20260709 13:17:25.992568 14044 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:25.992859 14148 raft_consensus.cc:2804] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:26.038748 10868 rpcz_store.cc:276] 0709 13:17:24.596807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:24.597298 (+   491us) service_pool.cc:224] Handling call
0709 13:17:25.995150 (+1397852us) spinlock_profiling.cc:243] Waited 567 ms on lock 0x7b5400340154. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:25.995340 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1304323840}
I20260709 13:17:26.000645 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:26.006289 10766 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:17:26.059193 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 53019.1us
I20260709 13:17:26.055020 11503 raft_consensus.cc:1275] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:17:26.026633 14248 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.064356 11511 raft_consensus.cc:2368] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 10. Current term is 11.
I20260709 13:17:25.879447 14341 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:26.052325 13829 raft_consensus.cc:3055] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:26.063347 14301 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:26.065439 11506 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:26.080899 14248 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:26.093966 14248 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:26.099074 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:26.063593 14148 raft_consensus.cc:697] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:26.071218 14292 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:26.079243 14244 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:26.106303 10801 pending_rounds.cc:85] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 4
I20260709 13:17:26.104296 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:26.106769 10801 pending_rounds.cc:107] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:17:26.107028 10798 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:26.107157 10801 raft_consensus.cc:2889] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:26.108323 10948 raft_consensus.cc:1275] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:17:26.108691 10948 pending_rounds.cc:85] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 5
I20260709 13:17:26.108897 10948 pending_rounds.cc:107] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:17:26.109103 10948 raft_consensus.cc:2889] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:26.109611 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:26.112617 14244 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:26.088959 10916 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:26.059603 14354 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:26.116535 10916 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:26.117302 14354 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:26.117909 14354 raft_consensus.cc:2749] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:26.118171 14360 raft_consensus.cc:2764] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election decision vote started in defunct term 10: won
I20260709 13:17:26.071772 14129 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:26.079416 13829 raft_consensus.cc:740] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:26.123854 13829 consensus_queue.cc:260] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:26.125442 13829 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:26.055933 14044 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:26.129415 14044 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:26.131254 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:26.135795 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:26.136401 10868 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:26.137470 11504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56404 (request call id 645) took 1085 ms. Trace:
I20260709 13:17:26.137054 14269 consensus_queue.cc:237] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:26.137848 11504 rpcz_store.cc:276] 0709 13:17:25.052544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:25.052654 (+   110us) service_pool.cc:224] Handling call
0709 13:17:26.137313 (+1084659us) spinlock_profiling.cc:243] Waited 839 ms on lock 0x7b54005f96d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:26.137414 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1929359744}
I20260709 13:17:26.138986 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:26.140846 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:26.141496 10795 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
I20260709 13:17:26.144392 14341 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:26.070329 14304 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:26.145345 10868 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 8.
I20260709 13:17:26.147351 10796 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:26.148015 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:26.153365 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:26.151309 10876 raft_consensus.cc:1240] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:26.154184 10795 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:17:26.149053 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:26.119663 10916 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:26.158717 10916 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:26.159839 11510 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
I20260709 13:17:26.160660 11508 raft_consensus.cc:1240] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:26.164067 10798 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 10.
I20260709 13:17:26.183552 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:26.184029 10841 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:17:26.122478 14129 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:26.184779 11510 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
W20260709 13:17:26.188563 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:26.091565 14301 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:26.197726 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:26.198446 10797 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:26.198606 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:26.119860 14246 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:26.198009 10796 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:17:26.202543 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:17:26.179791 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 256637us
I20260709 13:17:26.203522 10876 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:26.193415 10766 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:26.205341 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:26.206105 10868 raft_consensus.cc:2393] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 9: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:26.207583 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:26.209471 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:26.210472 10868 raft_consensus.cc:2393] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 10: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:26.215538 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:26.188944 14324 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.088s	user 0.005s	sys 0.011s
W20260709 13:17:26.218924 14366 log.cc:927] Time spent T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:17:26.220009 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:26.220656 10871 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 8.
I20260709 13:17:26.221342 14148 consensus_queue.cc:237] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:26.186607 14298 raft_consensus.cc:3055] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:26.231721 10766 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:26.183256 14343 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:17:26.191903 13910 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:26.237329 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 78628.7us
I20260709 13:17:26.216804 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:26.239020 10916 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:26.239178 10868 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:26.237255 10915 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:26.213040 14323 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.245714 14331 raft_consensus.cc:2749] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:26.186539 14244 consensus_queue.cc:1059] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 13:17:26.238052 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:26.341078 10798 raft_consensus.cc:2393] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 11: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:26.233484 14298 raft_consensus.cc:740] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:26.348373 10766 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:26.249305 10874 raft_consensus.cc:1240] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:17:26.230926 10876 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
I20260709 13:17:26.351940 10876 raft_consensus.cc:1275] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:17:26.194674 14354 raft_consensus.cc:2749] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:26.194802 14275 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:26.250484 10868 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
I20260709 13:17:26.199951 14301 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:26.196338 14248 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:17:26.258759 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:26.194455 14330 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:26.226158 14217 raft_consensus.cc:2749] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:26.215978 14129 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:26.329284 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 63475.3us
I20260709 13:17:26.195765 14304 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:26.238653 14324 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.137s	user 0.005s	sys 0.011s
I20260709 13:17:26.247383 14360 consensus_queue.cc:1059] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:17:26.394338 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1698) took 1499 ms. Trace:
I20260709 13:17:26.394862 10869 rpcz_store.cc:276] 0709 13:17:24.898204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:25.076167 (+177963us) service_pool.cc:224] Handling call
0709 13:17:25.996097 (+919930us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:26.039136 (+ 43039us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:26.394118 (+354982us) spinlock_profiling.cc:243] Waited 918 ms on lock 0x7b5400340150. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:26.394262 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2111952128}
W20260709 13:17:26.345075 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 106124us
W20260709 13:17:26.367128 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 130104us
I20260709 13:17:26.267176 13910 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:26.333171 14319 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:26.333894 14362 raft_consensus.cc:2749] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:17:26.407155 14370 log.cc:927] Time spent T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.206s	user 0.000s	sys 0.001s
I20260709 13:17:26.407771 14044 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:26.408344 14044 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:26.410476 14044 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:26.336493 14292 raft_consensus.cc:697] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 8 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:26.306396 14296 consensus_queue.cc:1048] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.064s
W20260709 13:17:26.399648 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:26.433209 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:26.398409 10797 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:17:26.339681 14323 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:26.404125 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:26.367854 13274 raft_consensus.cc:2804] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:26.439320 13274 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:26.439906 13274 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:26.403270 10874 raft_consensus.cc:1275] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:17:26.441661 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:26.365617 14275 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:26.433104 14324 raft_consensus.cc:2749] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:26.378832 14330 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:26.491916 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:26.410631 14217 consensus_queue.cc:1048] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.081s
I20260709 13:17:26.435024 14044 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:26.525452 13274 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:26.415621 14129 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:26.521912 14299 raft_consensus.cc:3055] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:26.520578 14244 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.433730 14298 consensus_queue.cc:260] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:26.558607 13910 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:26.405666 14319 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:17:26.587456 14299 raft_consensus.cc:740] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:26.552063 14044 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:26.802721 10875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 460) took 1127 ms. Trace:
I20260709 13:17:26.803205 10875 rpcz_store.cc:276] 0709 13:17:25.675246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:25.687613 (+ 12367us) service_pool.cc:224] Handling call
0709 13:17:25.782647 (+ 95034us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:25.783298 (+   651us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:26.802503 (+1019205us) spinlock_profiling.cc:243] Waited 94.7 ms on lock 0x7b540006c2d0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:26.802651 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":217776512}
I20260709 13:17:26.807184 13274 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:26.815883 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 438276us
I20260709 13:17:26.511104 14292 consensus_queue.cc:237] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:26.821785 10767 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:26.792214 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 443892us
I20260709 13:17:26.823714 10766 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:17:26.794967 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 391603us
I20260709 13:17:26.855751 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:26.857364 10793 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:26.808261 14374 log.cc:927] Time spent T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.279s	user 0.001s	sys 0.001s
I20260709 13:17:26.859951 10842 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:26.735415 14244 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:26.826726 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:26.908392 10798 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:26.809371 14248 raft_consensus.cc:697] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:26.795492 14324 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:26.828850 14217 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.896062 14247 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:17:26.975881 10840 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.003s	sys 0.014s
W20260709 13:17:26.976506 10840 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.003s	sys 0.014s
I20260709 13:17:26.897357 10842 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:26.981400 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 84136.3us
I20260709 13:17:26.857295 10916 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:26.884410 10945 raft_consensus.cc:3055] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:26.818354 14391 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:17:26.859959 14377 log.cc:927] Time spent T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.352s	user 0.001s	sys 0.001s
I20260709 13:17:26.982769 10945 raft_consensus.cc:740] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:26.989356 10793 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 8.
I20260709 13:17:27.029075 10798 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 11.
W20260709 13:17:27.035377 10839 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.025s	sys 0.046s
W20260709 13:17:27.036043 10839 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.025s	sys 0.046s
I20260709 13:17:26.973795 14247 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:26.937526 14275 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:27.038995 14148 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.814s	user 0.015s	sys 0.015s
I20260709 13:17:27.008275 14304 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:27.014550 10842 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:17:27.046725 13829 consensus_meta.cc:287] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.921s	user 0.011s	sys 0.003s
I20260709 13:17:27.047456 14247 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:27.008955 14324 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:27.047051 14296 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:27.060184 14148 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.832s	user 0.015s	sys 0.015s
I20260709 13:17:26.985980 14217 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:27.053345 14312 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.960629 14299 consensus_queue.cc:260] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:27.052906 14323 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:27.012253 14391 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:27.103283 14382 log.cc:927] Time spent T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.375s	user 0.001s	sys 0.000s
I20260709 13:17:27.101572 14296 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:27.105247 14323 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:27.105544 14296 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:27.105834 14323 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:27.081988 14248 consensus_queue.cc:237] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:27.108664 14323 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:27.109594 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 458) took 1435 ms. Trace:
I20260709 13:17:27.109994 10873 rpcz_store.cc:276] 0709 13:17:25.674513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:25.676379 (+  1866us) service_pool.cc:224] Handling call
0709 13:17:25.707750 (+ 31371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:25.723303 (+ 15553us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:17:25.915092 (+191789us) spinlock_profiling.cc:243] Waited 1.94 ms on lock 0x7b18002ed980. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc5455 00007f9c848ad3c7 00007f9c80ebf9a7 00007f9c80ecf0bf 00007f9c80ecf038
0709 13:17:26.146417 (+231325us) log.cc:844] Serialized 14 byte log entry
0709 13:17:26.492092 (+345675us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:26.492603 (+   511us) log.cc:844] Serialized 26 byte log entry
0709 13:17:26.492772 (+   169us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:17:26.511776 (+ 19004us) log.cc:844] Serialized 14 byte log entry
0709 13:17:26.512938 (+  1162us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:26.513016 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:27.109386 (+596370us) raft_consensus.cc:1685] finished
0709 13:17:27.109407 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:27.109536 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":76252,"prepare.queue_time_us":544191,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":85,"raft.queue_time_us":688660,"raft.run_cpu_time_us":452,"raft.run_wall_time_us":53822,"spinlock_wait_cycles":4464384,"thread_start_us":464485,"threads_started":2,"wal-append.queue_time_us":788841}
I20260709 13:17:27.068091 14275 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:27.113502 14275 raft_consensus.cc:697] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:17:27.113894 14375 log.cc:927] Time spent T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.295s	user 0.000s	sys 0.001s
I20260709 13:17:27.079460 14304 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:27.115283 14299 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:27.114727 14275 consensus_queue.cc:237] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:27.114894 14304 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:27.066491 14389 raft_consensus.cc:2804] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:17:27.111279 14381 log.cc:927] Time spent T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.291s	user 0.001s	sys 0.000s
I20260709 13:17:27.079734 14392 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:17:27.123291 10767 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.116.130:37895 timed out after 1.632s (SENT) [suppressed 15 similar messages]
I20260709 13:17:27.098454 13995 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:27.131422 10798 raft_consensus.cc:1275] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 13:17:27.131829 10798 pending_rounds.cc:85] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 3
I20260709 13:17:27.132059 10798 pending_rounds.cc:107] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 13:17:27.132272 10798 raft_consensus.cc:2889] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:27.133183 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:27.134039 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:27.143674 10793 raft_consensus.cc:1275] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:17:27.123085 14389 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:27.121057 14324 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:27.120522 14330 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:27.118840 14393 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:27.154297 13995 raft_consensus.cc:697] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:27.157127 14393 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:27.157546 13995 consensus_queue.cc:237] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:27.128152 14148 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:27.159893 14380 log.cc:927] Time spent T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.369s	user 0.001s	sys 0.000s
I20260709 13:17:27.160233 14148 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:27.162027 10943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 1718) took 1860 ms. Trace:
I20260709 13:17:27.162422 10943 rpcz_store.cc:276] 0709 13:17:25.301726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:25.595401 (+293675us) service_pool.cc:224] Handling call
0709 13:17:25.622864 (+ 27463us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:25.851352 (+228488us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:25.851383 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:25.851964 (+   581us) log.cc:844] Serialized 52 byte log entry
0709 13:17:26.328002 (+476038us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:26.328040 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:26.328103 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:27.161791 (+833688us) raft_consensus.cc:1685] finished
0709 13:17:27.161814 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:27.161963 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16216,"raft.queue_time_us":367754,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":25396,"thread_start_us":637344,"threads_started":2,"wal-append.queue_time_us":660202}
I20260709 13:17:27.150687 10766 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:27.056869 14319 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:27.167583 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:27.169143 10950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2129) took 1719 ms. Trace:
I20260709 13:17:27.168952 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:27.169032 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:27.170408 10798 raft_consensus.cc:2410] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:17:27.105057 14391 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:27.169639 10950 rpcz_store.cc:276] 0709 13:17:25.450268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:26.253695 (+803427us) service_pool.cc:224] Handling call
0709 13:17:26.254108 (+   413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:26.339925 (+ 85817us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:26.339958 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:26.340452 (+   494us) log.cc:844] Serialized 26 byte log entry
0709 13:17:26.792790 (+452338us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:26.792834 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:26.792907 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:27.168927 (+376020us) raft_consensus.cc:1685] finished
0709 13:17:27.168946 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:27.169087 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":85420,"raft.queue_time_us":86241,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":12305,"thread_start_us":391160,"threads_started":1,"wal-append.queue_time_us":669908}
I20260709 13:17:27.159859 14393 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:27.119836 14312 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:27.188845 10766 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:17:27.165285 14379 log.cc:927] Time spent T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.346s	user 0.000s	sys 0.001s
I20260709 13:17:27.154371 14389 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:17:27.197744 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 74480.4us
I20260709 13:17:27.197888 14412 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:27.177959 14044 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:27.163443 14395 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:27.139750 14392 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:27.215003 10766 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:27.236372 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:27.241554 14400 log.cc:927] Time spent T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:17:27.242374 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:27.195942 14244 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:27.269753 10871 raft_consensus.cc:1240] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:27.230361 14314 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:17:27.270498 11505 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56416 (request call id 601) took 1647 ms (client timeout 1815 ms). Trace:
W20260709 13:17:27.271148 11505 rpcz_store.cc:269] 0709 13:17:25.623027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:27.242080 (+1619053us) service_pool.cc:224] Handling call
0709 13:17:27.270409 (+ 28329us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:27.273977 10950 raft_consensus.cc:1275] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 5. (term mismatch)
I20260709 13:17:27.274118 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:27.275760 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:27.276510 10941 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 9.
I20260709 13:17:27.238389 14405 consensus_queue.cc:1059] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:27.238997 14406 consensus_queue.cc:1048] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:27.274487 10950 pending_rounds.cc:85] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 4
I20260709 13:17:27.278954 10950 pending_rounds.cc:107] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 13:17:27.279268 10950 raft_consensus.cc:2889] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:27.274006 14314 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:27.201581 14407 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:17:27.285652 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:27.286312 10948 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 9
W20260709 13:17:27.287576 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:27.224647 14044 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:17:27.237640 10801 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.669s	user 0.007s	sys 0.006s
I20260709 13:17:27.226610 14395 raft_consensus.cc:697] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:17:27.294960 10801 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.008s	sys 0.006s
I20260709 13:17:27.225812 14148 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:27.306509 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:27.307021 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:27.306613 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:27.307451 10950 raft_consensus.cc:2393] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 9: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:17:27.314769 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 457522us
I20260709 13:17:27.316493 10916 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:27.302834 14407 raft_consensus.cc:697] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:27.215757 14412 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:27.298348 14362 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:27.303006 13829 raft_consensus.cc:2749] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:27.324637 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:27.325619 13274 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:27.306321 14405 consensus_queue.cc:1243] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Peer 468a8fd46e824f62b9c2c0d3532a6d26 log is divergent from this leader: its last log entry 9.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:17:27.309664 14269 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:27.331472 10916 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:27.367229 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:27.312485 14378 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:17:27.460089 10949 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41020 (request call id 2115) took 2207 ms (client timeout 1829 ms). Trace:
I20260709 13:17:27.433633 10801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2177) took 1336 ms. Trace:
W20260709 13:17:27.393321 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:27.497774 10941 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 10.
W20260709 13:17:27.442620 11506 consensus_meta.cc:287] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 1.250s	user 0.008s	sys 0.006s
I20260709 13:17:27.312209 14044 raft_consensus.cc:697] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:27.453671 14327 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:27.372376 14395 consensus_queue.cc:237] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:27.353931 14384 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:27.354724 14409 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:27.421408 14367 consensus_queue.cc:1059] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 13:17:27.520426 14367 consensus_queue.cc:1243] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer 468a8fd46e824f62b9c2c0d3532a6d26 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:17:27.520620 14392 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:27.520954 14367 raft_consensus.cc:3055] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:27.521374 14367 raft_consensus.cc:740] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:27.522710 14333 raft_consensus.cc:3055] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:27.523306 14333 raft_consensus.cc:740] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:27.371186 14217 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:27.312593 14360 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:27.524282 14360 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:27.374181 14345 consensus_queue.cc:1059] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.100s
I20260709 13:17:27.427502 10948 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 9.
I20260709 13:17:27.525651 14345 consensus_queue.cc:1243] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer 468a8fd46e824f62b9c2c0d3532a6d26 log is divergent from this leader: its last log entry 8.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:17:27.312546 14354 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:27.526306 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:27.485685 14301 consensus_meta.cc:287] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 1.097s	user 0.012s	sys 0.002s
I20260709 13:17:27.526983 10948 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 7.
I20260709 13:17:27.526710 14354 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:27.527563 14419 log.cc:927] Time spent T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 13:17:27.498469 10949 rpcz_store.cc:269] 0709 13:17:25.236081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:25.236347 (+   266us) service_pool.cc:224] Handling call
0709 13:17:27.443594 (+2207247us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b54001e2654. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:27.443767 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":430442240}
I20260709 13:17:27.528043 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:27.485518 14312 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:27.528586 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:27.529904 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:27.503286 10766 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:17:27.530486 14364 log.cc:927] Time spent T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 1.521s	user 0.016s	sys 0.005s
I20260709 13:17:27.531756 14412 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:27.531975 10766 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:27.533015 11508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56416 (request call id 602) took 1897 ms. Trace:
I20260709 13:17:27.533466 11508 rpcz_store.cc:276] 0709 13:17:25.635428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:26.055518 (+420090us) service_pool.cc:224] Handling call
0709 13:17:26.160501 (+104983us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:26.181626 (+ 21125us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:27.532833 (+1351207us) spinlock_profiling.cc:243] Waited 94.5 ms on lock 0x7b54005f0350. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:27.532953 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":217452800}
I20260709 13:17:27.523382 13829 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:27.535070 13829 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:27.517747 14327 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:27.537253 13829 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:27.537322 14331 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:27.527197 14301 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:27.537806 14331 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:27.538194 14331 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:27.538568 10794 raft_consensus.cc:2393] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 9: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:27.539664 14301 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:27.540309 14301 consensus_queue.cc:1048] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:17:27.542071 14383 log.cc:927] Time spent T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.788s	user 0.002s	sys 0.000s
I20260709 13:17:27.528681 10948 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 6.
I20260709 13:17:27.543517 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:27.543938 14407 consensus_queue.cc:237] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:27.544513 10942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2114) took 2310 ms. Trace:
I20260709 13:17:27.544757 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:27.545405 10948 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 8.
I20260709 13:17:27.546316 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:27.514204 14243 raft_consensus.cc:2749] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:27.547823 10766 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:27.548942 14409 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:27.549310 14409 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:27.549645 14409 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:27.530510 10949 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 7.
I20260709 13:17:27.551759 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:27.552273 10949 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:17:27.497754 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 166375us
I20260709 13:17:27.555670 10948 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 9.
I20260709 13:17:27.524546 14333 consensus_queue.cc:260] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:27.545125 10942 rpcz_store.cc:276] 0709 13:17:25.234387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:25.234532 (+   145us) service_pool.cc:224] Handling call
0709 13:17:25.243457 (+  8925us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:26.113487 (+870030us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:26.113515 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:26.113552 (+    37us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:26.113585 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:27.544243 (+1430658us) spinlock_profiling.cc:243] Waited 870 ms on lock 0x7b54001e1254. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:27.544292 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:27.544404 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2000471424}
I20260709 13:17:27.499940 10801 rpcz_store.cc:276] 0709 13:17:26.097032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:26.100926 (+  3894us) service_pool.cc:224] Handling call
0709 13:17:26.106054 (+  5128us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:26.498955 (+392901us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:26.499005 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:26.499834 (+   829us) log.cc:844] Serialized 52 byte log entry
0709 13:17:27.304897 (+805063us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:27.305183 (+   286us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:27.305570 (+   387us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:27.433370 (+127800us) raft_consensus.cc:1685] finished
0709 13:17:27.433395 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:27.433565 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44432,"raft.queue_time_us":78475,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":6735,"thread_start_us":1043305,"threads_started":2,"wal-append.queue_time_us":881670}
I20260709 13:17:27.559172 10800 raft_consensus.cc:1275] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 13:17:27.560086 10798 raft_consensus.cc:1240] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:17:27.559306 10795 raft_consensus.cc:1240] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:27.563081 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:27.556329 10873 raft_consensus.cc:1240] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:27.566222 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:27.567257 10873 raft_consensus.cc:2393] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 10: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:27.568382 10766 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:27.573596 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:27.575659 10948 raft_consensus.cc:2393] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 11: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:27.645388 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:27.503819 14292 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:27.534454 14354 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:27.652773 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:27.481961 14247 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:27.653064 13274 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:27.657326 13274 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:27.653690 10942 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 7.
I20260709 13:17:27.518704 14384 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:27.660574 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:27.661655 10874 raft_consensus.cc:2393] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 7: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:27.505136 11506 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:17:27.664680 10766 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:27.652422 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:27.543548 14301 raft_consensus.cc:2804] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:27.674646 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:27.643836 14217 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:27.574975 14345 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:27.681749 10949 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 10.
I20260709 13:17:27.684827 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:27.644830 14312 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:27.500221 14269 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:27.689536 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:27.656219 14327 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:27.689611 13274 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:27.662920 14333 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:27.667003 14292 raft_consensus.cc:697] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 10 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:27.507762 14362 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:27.667001 14354 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:27.694254 14345 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:27.643116 13829 consensus_queue.cc:1048] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 13:17:27.508003 14378 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:27.702353 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:27.709800 10941 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 11.
I20260709 13:17:27.686231 14392 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:17:27.679411 11506 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56410 (request call id 608) took 1744 ms (client timeout 1875 ms). Trace:
I20260709 13:17:27.688257 14217 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:17:27.727142 14358 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.640s	user 0.013s	sys 0.014s
I20260709 13:17:27.727437 14217 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:17:27.728399 14420 log.cc:927] Time spent T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.245s	user 0.000s	sys 0.001s
I20260709 13:17:27.705946 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:27.522298 14367 consensus_queue.cc:260] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:27.733218 14404 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:27.733805 14404 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:27.706939 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 468) took 1304 ms. Trace:
I20260709 13:17:27.702765 13274 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:27.734812 14398 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:27.690192 14301 raft_consensus.cc:697] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:27.694139 14312 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:27.735268 13274 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:27.739778 14367 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:27.740056 14243 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:27.740243 14360 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:27.740628 14243 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:27.741801 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:27.742187 10876 raft_consensus.cc:1275] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:17:27.742523 10868 raft_consensus.cc:2393] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 9: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:17:27.742818 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:27.718273 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:27.743335 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:27.743556 10947 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:27.735050 10869 rpcz_store.cc:276] 0709 13:17:26.402922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:26.403086 (+   164us) service_pool.cc:224] Handling call
0709 13:17:26.403400 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:26.736648 (+333248us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:26.736693 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:26.737367 (+   674us) log.cc:844] Serialized 26 byte log entry
0709 13:17:27.063256 (+325889us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:27.063315 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:27.702348 (+639033us) spinlock_profiling.cc:243] Waited 333 ms on lock 0x7b5400076054. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:27.702395 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:27.702437 (+    42us) raft_consensus.cc:1685] finished
0709 13:17:27.702449 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:27.706866 (+  4417us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13356,"raft.queue_time_us":56051,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":14942,"spinlock_wait_cycles":765245824,"thread_start_us":324194,"threads_started":1,"wal-append.queue_time_us":368717}
I20260709 13:17:27.690263 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:27.726706 14305 raft_consensus.cc:2804] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:27.754323 14305 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:27.754812 14305 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:27.736865 14392 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:27.741035 13274 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:17:27.704008 14299 consensus_meta.cc:287] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.572s	user 0.006s	sys 0.005s
W20260709 13:17:27.727520 14358 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.719s	user 0.013s	sys 0.014s
I20260709 13:17:27.745393 10942 raft_consensus.cc:2410] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:17:27.746464 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:27.716869 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:27.717235 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:27.735409 14398 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:27.751736 14301 consensus_queue.cc:237] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:27.752693 14243 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:27.752951 14269 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:27.779151 10873 raft_consensus.cc:2393] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 10: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:27.779798 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:27.788571 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:27.780963 10944 raft_consensus.cc:1275] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:17:27.789312 10944 pending_rounds.cc:85] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 3
I20260709 13:17:27.789578 10944 pending_rounds.cc:107] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 13:17:27.789816 10944 raft_consensus.cc:2889] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:27.756722 10946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2124) took 2336 ms. Trace:
W20260709 13:17:27.724714 11506 rpcz_store.cc:269] 0709 13:17:25.935465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:25.936777 (+  1312us) service_pool.cc:224] Handling call
0709 13:17:27.677958 (+1741181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:27.779827 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:27.782579 10941 raft_consensus.cc:1240] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:27.738204 14431 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:27.791493 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:27.790707 10946 rpcz_store.cc:276] 0709 13:17:25.420059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:26.082126 (+662067us) service_pool.cc:224] Handling call
0709 13:17:26.104504 (+ 22378us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:26.104944 (+   440us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:26.104971 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:26.105544 (+   573us) log.cc:844] Serialized 52 byte log entry
0709 13:17:26.235042 (+129498us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:26.305909 (+ 70867us) log.cc:844] Serialized 14 byte log entry
0709 13:17:26.354320 (+ 48411us) log.cc:844] Serialized 14 byte log entry
0709 13:17:26.409978 (+ 55658us) spinlock_profiling.cc:243] Waited 16.2 ms on lock 0x7b180042af00. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc83f3 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:17:27.229902 (+819924us) spinlock_profiling.cc:243] Waited 10 ms on lock 0x7b180042af00. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7dd42c0a 00007f9c87482219 00007f9c7ddcaa40 00007f9c7ddc54e4 00007f9c7ddc5455 00007f9c80ead060 00007f9c80ebf716 00007f9c80ed0287 00007f9c80ed0208
0709 13:17:27.230147 (+   245us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:27.230240 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:27.541431 (+311191us) spinlock_profiling.cc:243] Waited 26.5 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:17:27.756491 (+215060us) raft_consensus.cc:1685] finished
0709 13:17:27.756514 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:27.756663 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":933348,"prepare.queue_time_us":117362,"prepare.run_cpu_time_us":128,"prepare.run_wall_time_us":10202,"raft.queue_time_us":931699,"raft.run_cpu_time_us":280,"raft.run_wall_time_us":283,"spinlock_wait_cycles":121123328,"thread_start_us":128261,"threads_started":1,"wal-append.queue_time_us":426227}
I20260709 13:17:27.755638 14345 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:27.701552 14044 consensus_queue.cc:237] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:27.711205 14354 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:27.807195 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:27.824904 10873 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 8.
I20260709 13:17:27.808109 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:27.808166 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:27.807263 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:27.817494 10947 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 8.
I20260709 13:17:27.826177 10876 raft_consensus.cc:2393] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 10: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:17:27.809329 14389 consensus_meta.cc:287] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.572s	user 0.012s	sys 0.000s
W20260709 13:17:27.832259 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 176895us
I20260709 13:17:27.834122 11477 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:27.835178 10941 raft_consensus.cc:1275] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:17:27.837110 10946 pending_rounds.cc:85] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 3
I20260709 13:17:27.837435 10946 pending_rounds.cc:107] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 13:17:27.837661 10946 raft_consensus.cc:2889] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:27.809029 14247 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:27.840111 10947 raft_consensus.cc:1240] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 8. Current term is 9. Ops: []
W20260709 13:17:27.848676 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:27.848752 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:27.850212 10876 raft_consensus.cc:2410] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 9 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:17:27.851548 10947 raft_consensus.cc:1275] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:17:27.737035 14404 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:27.883173 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:27.822973 14431 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:27.899084 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:27.899654 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:27.899968 10876 raft_consensus.cc:2393] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 10: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:27.863687 14312 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:27.908630 10916 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:27.836529 14389 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:27.809333 14292 consensus_queue.cc:237] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:27.912386 13274 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:27.932833 13274 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:27.933285 13274 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:27.911094 14378 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:27.898751 14433 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:27.902222 14299 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.024250 10799 raft_consensus.cc:1240] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:17:28.050721 10916 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:27.958667 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:28.077548 13274 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:27.910092 14269 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:17:27.963593 14427 log.cc:927] Time spent T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.241s	user 0.000s	sys 0.000s
W20260709 13:17:27.957913 14434 log.cc:927] Time spent T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:17:28.084040 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:28.084290 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:28.043710 14211 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:28.009496 14423 log.cc:927] Time spent T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.254s	user 0.000s	sys 0.000s
I20260709 13:17:27.911670 14435 raft_consensus.cc:2804] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:17:27.963593 14430 log.cc:927] Time spent T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.239s	user 0.000s	sys 0.000s
I20260709 13:17:28.101858 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:28.083801 13274 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:28.087003 10800 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 10.
I20260709 13:17:28.023413 14298 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:28.088543 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:28.068588 14345 consensus_queue.cc:1059] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:17:28.091058 14432 consensus_queue.cc:1059] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.024s
W20260709 13:17:28.092551 10870 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 1675) took 3493 ms (client timeout 2245 ms). Trace:
I20260709 13:17:28.022666 14398 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:28.043748 14392 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:28.021876 14389 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:27.963435 14384 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:17:28.090678 14426 log.cc:927] Time spent T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.180s	user 0.000s	sys 0.000s
I20260709 13:17:28.105625 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:28.105934 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:28.129757 10800 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 7.
I20260709 13:17:28.106981 14447 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:28.099797 14435 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:28.074740 14299 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:28.088662 13829 raft_consensus.cc:2749] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:17:28.108917 10870 rpcz_store.cc:269] 0709 13:17:24.600776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:24.655436 (+ 54660us) service_pool.cc:224] Handling call
0709 13:17:28.092304 (+3436868us) spinlock_profiling.cc:243] Waited 843 ms on lock 0x7b540037ad54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:28.092435 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1939586304}
I20260709 13:17:28.099267 14358 raft_consensus.cc:3055] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:28.070880 14360 consensus_queue.cc:1048] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
I20260709 13:17:28.089395 14217 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:28.112936 10875 raft_consensus.cc:1275] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 8. (term mismatch)
I20260709 13:17:28.177953 10875 pending_rounds.cc:85] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 7
I20260709 13:17:28.116003 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:17:28.119323 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:28.107865 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:28.131304 10796 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2181) took 2017 ms (client timeout 1937 ms). Trace:
I20260709 13:17:28.103787 14032 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:28.179867 10796 rpcz_store.cc:269] 0709 13:17:26.098238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:26.109334 (+ 11096us) service_pool.cc:224] Handling call
0709 13:17:28.115042 (+2005708us) spinlock_profiling.cc:243] Waited 37 ms on lock 0x7b54000bbdd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:28.115141 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":85121280}
W20260709 13:17:28.112951 14331 consensus_meta.cc:287] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.574s	user 0.010s	sys 0.001s
W20260709 13:17:28.174921 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 85184.7us
I20260709 13:17:28.183116 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:28.023739 14433 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:28.175944 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:28.189103 10874 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:28.086244 14269 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:28.183802 10793 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
I20260709 13:17:28.184831 10875 pending_rounds.cc:107] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 13:17:28.190975 10875 raft_consensus.cc:2889] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:28.190649 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:28.192304 10793 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 9.
I20260709 13:17:28.193246 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:28.162586 14436 log.cc:927] Time spent T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:17:28.105525 14367 consensus_queue.cc:1059] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 13:17:28.130084 14345 consensus_queue.cc:1243] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer e01ecd008edd4aa2b6b2b47efbc2ea87 log is divergent from this leader: its last log entry 10.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:17:28.185302 10916 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:28.203280 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:28.193769 10793 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:17:28.194851 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:28.167279 14354 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:28.173657 14358 raft_consensus.cc:740] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:28.166742 14435 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:28.211534 10916 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:28.173250 14447 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:28.217691 10795 raft_consensus.cc:2410] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
W20260709 13:17:28.212679 14409 consensus_meta.cc:287] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.660s	user 0.014s	sys 0.000s
I20260709 13:17:28.207396 14345 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:28.187803 14032 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:28.231303 14367 raft_consensus.cc:2804] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:28.231792 14367 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:28.232265 14367 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:28.239717 14444 log.cc:927] Time spent T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 13:17:28.241799 14441 log.cc:927] Time spent T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 13:17:28.248061 10793 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 10.
I20260709 13:17:28.251981 10916 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:28.190471 14331 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:28.256634 10793 raft_consensus.cc:1240] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 9. Current term is 10. Ops: [9.4-9.4]
I20260709 13:17:28.257505 11507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56410 (request call id 601) took 2949 ms. Trace:
I20260709 13:17:28.258656 11507 rpcz_store.cc:276] 0709 13:17:25.304121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:25.322231 (+ 18110us) service_pool.cc:224] Handling call
0709 13:17:25.334292 (+ 12061us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:25.336573 (+  2281us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:25.336617 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:25.355319 (+ 18702us) log.cc:844] Serialized 26 byte log entry
0709 13:17:27.373625 (+2018306us) spinlock_profiling.cc:243] Waited 6.48 ms on lock 0x7b180040bf40. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a550d1 00007f9c80e70a0f
0709 13:17:27.513819 (+140194us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:27.513862 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:27.514169 (+   307us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:28.243411 (+729242us) raft_consensus.cc:1685] finished
0709 13:17:28.243430 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:28.253443 (+ 10013us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2239,"raft.queue_time_us":17450,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":6404,"spinlock_wait_cycles":14910720,"thread_start_us":139975,"threads_started":1,"wal-append.queue_time_us":2334941}
I20260709 13:17:28.229835 14345 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:28.272027 10874 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 9.
I20260709 13:17:28.249270 14358 consensus_queue.cc:260] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:28.265257 14432 consensus_queue.cc:1059] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 13:17:28.280125 14032 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:28.281481 14269 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:28.230578 14409 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:28.284085 14447 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:28.284993 14217 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:28.285377 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:28.257453 13829 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:28.287869 14452 log.cc:927] Time spent T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:17:27.645597  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:28.261152 14354 raft_consensus.cc:2804] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:28.300486 10944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 1724) took 1045 ms. Trace:
I20260709 13:17:28.308197 10871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1683) took 2096 ms. Trace:
W20260709 13:17:28.309078 14305 consensus_meta.cc:287] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.554s	user 0.008s	sys 0.000s
I20260709 13:17:28.266383 14345 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:28.311470 10798 raft_consensus.cc:1275] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:17:28.312395 14331 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:28.408761 10871 rpcz_store.cc:276] 0709 13:17:26.212029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:26.221292 (+  9263us) service_pool.cc:224] Handling call
0709 13:17:26.221552 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:27.270260 (+1048708us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:28.307893 (+1037633us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54000f78d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:28.308111 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2410043776}
I20260709 13:17:28.408690 10944 rpcz_store.cc:276] 0709 13:17:27.255550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:27.255799 (+   249us) service_pool.cc:224] Handling call
0709 13:17:27.262401 (+  6602us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:27.798439 (+536038us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:28.300187 (+501748us) spinlock_profiling.cc:243] Waited 518 ms on lock 0x7b540039a154. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:28.300411 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1191609728}
W20260709 13:17:28.314160 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:28.317056 10800 raft_consensus.cc:1240] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:28.407632 13829 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:28.318140 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:28.324270 14405 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.419564 11508 raft_consensus.cc:1240] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 9. Current term is 10. Ops: []
W20260709 13:17:28.419866 11511 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56416 (request call id 604) took 2783 ms (client timeout 2577 ms). Trace:
I20260709 13:17:28.325289 14439 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:28.325479 10793 raft_consensus.cc:1240] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 10. Current term is 11. Ops: [9.9-10.10]
I20260709 13:17:28.326728 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:28.424007 10796 raft_consensus.cc:2410] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 10 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:17:28.326359 14299 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:28.399897 10915 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:28.418344 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:28.418727 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:28.392442 14358 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:17:28.420333 11511 rpcz_store.cc:269] 0709 13:17:25.635985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:25.686242 (+ 50257us) service_pool.cc:224] Handling call
0709 13:17:28.419669 (+2733427us) spinlock_profiling.cc:243] Waited 377 ms on lock 0x7b54006202d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:28.419804 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":866066176}
I20260709 13:17:28.419415 14405 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:28.410856 10798 pending_rounds.cc:85] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 7
I20260709 13:17:28.431037 10798 pending_rounds.cc:107] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:17:28.431373 10798 raft_consensus.cc:2889] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:28.406188 14378 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.426586 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:28.427851 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:28.417754 14032 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.432552 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:28.432278 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:28.431883 14274 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.437024 14274 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:28.421767 14393 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.712s	user 0.009s	sys 0.010s
I20260709 13:17:28.437871 11508 raft_consensus.cc:1240] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:17:28.420249 14191 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.407979 14354 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:28.432404 14299 raft_consensus.cc:2749] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:17:28.440438 14393 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.733s	user 0.010s	sys 0.010s
I20260709 13:17:28.455631 11512 raft_consensus.cc:1275] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:17:28.427480 14439 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:28.421304 14305 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:28.459537 10801 raft_consensus.cc:1240] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:28.435344 11506 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 10.
I20260709 13:17:28.449229 14354 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:28.463071 14305 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:28.432155 14378 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:28.464023 14274 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:28.451375 14191 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:28.462467 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:28.465384 10949 raft_consensus.cc:1240] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 10 LEADER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 9. Current term is 10. Ops: []
W20260709 13:17:28.459416 14448 log.cc:927] Time spent T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.251s	user 0.001s	sys 0.000s
I20260709 13:17:28.471316 11507 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 7.
W20260709 13:17:28.472777 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 60350us
I20260709 13:17:28.463418 13274 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:28.442020 14032 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:28.474222 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:28.477277 13274 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:28.478751 14450 consensus_queue.cc:1175] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" responder_term: 9 status { last_received { term: 9 index: 4 } last_committed_idx: 4 last_received_current_leader { term: 9 index: 4 } }
I20260709 13:17:28.479681 14129 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:28.480008 14378 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:28.481750 14191 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:28.471475 13829 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:28.485893 13274 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:28.486613 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:28.485627 14435 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:28.491843 14435 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:28.495635 14304 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:28.496445 14431 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:28.497448 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 457) took 2823 ms. Trace:
I20260709 13:17:28.498073 10942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41014 (request call id 1735) took 1010 ms. Trace:
I20260709 13:17:28.498494 10942 rpcz_store.cc:276] 0709 13:17:27.487796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:27.659853 (+172057us) service_pool.cc:224] Handling call
0709 13:17:28.497872 (+838019us) spinlock_profiling.cc:243] Waited 66.9 ms on lock 0x7b54000177d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:28.498016 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":153978240}
I20260709 13:17:28.499943 11478 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:28.500275 11509 raft_consensus.cc:1240] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:28.500861 11478 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:28.500785 10945 consensus_queue.cc:260] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:28.501899 10945 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:28.505247 14257 raft_consensus.cc:2804] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:17:28.497833 10867 rpcz_store.cc:276] 0709 13:17:25.674181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:25.687182 (+ 13001us) service_pool.cc:224] Handling call
0709 13:17:25.687438 (+   256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:25.898509 (+211071us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:17:26.059011 (+160502us) log.cc:844] Serialized 14 byte log entry
0709 13:17:26.332994 (+273983us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:26.333053 (+    59us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:17:26.333104 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:28.496358 (+2163254us) spinlock_profiling.cc:243] Waited 2.38 ms on lock 0x7b1800339400. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7dd962a6 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419
0709 13:17:28.497183 (+   825us) spinlock_profiling.cc:243] Waited 211 ms on lock 0x7b54002f08d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:28.497235 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:28.497366 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":149,"prepare.queue_time_us":356067,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":10466,"raft.queue_time_us":255,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":13808,"spinlock_wait_cycles":490394752,"thread_start_us":433575,"threads_started":2,"wal-append.queue_time_us":680975,"wal-append.run_cpu_time_us":2270,"wal-append.run_wall_time_us":1413483}
I20260709 13:17:28.506098 14439 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:28.505092 14431 consensus_queue.cc:1059] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:17:28.505141 14403 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:28.507556 14403 raft_consensus.cc:697] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 8 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:28.511175 14439 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:28.513441 13274 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:28.514047 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:28.512669 14257 raft_consensus.cc:697] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 11 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:28.515317 14246 consensus_queue.cc:1048] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:17:28.516512 10941 raft_consensus.cc:1240] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:28.516650 14361 raft_consensus.cc:3055] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:28.516414 14257 consensus_queue.cc:237] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:28.517156 14361 raft_consensus.cc:740] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:28.516939 14428 consensus_queue.cc:1048] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:17:28.513940 13274 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:28.512686 14431 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.521296 14244 consensus_queue.cc:1059] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:17:28.521745 14431 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:28.522200 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:28.521500 14314 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:28.522903 10944 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 7.
I20260709 13:17:28.532176 14191 raft_consensus.cc:3055] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:28.533474 14191 raft_consensus.cc:740] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:28.536078 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:28.536300 11477 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:28.536695 10941 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:17:28.518105 14361 consensus_queue.cc:260] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:28.511749 14439 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:28.540336 13274 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:28.538686 14440 raft_consensus.cc:2804] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:28.546317 14440 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:28.546741 14440 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:28.534353 14191 consensus_queue.cc:260] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:28.535948 14314 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.538564 14361 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:28.553121 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:28.558243 10943 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 8.
I20260709 13:17:28.558760 10943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41014 (request call id 1721) took 1315 ms. Trace:
I20260709 13:17:28.547261 14439 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:28.559101 10943 rpcz_store.cc:276] 0709 13:17:27.243401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:27.243522 (+   121us) service_pool.cc:224] Handling call
0709 13:17:28.553005 (+1309483us) spinlock_profiling.cc:243] Waited 3.89 ms on lock 0x7b5800112598. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd90eee 0000000000375939 00007f9c85e041e0 00007f9c85ecd0c4 00007f9c85ecd131 00007f9c85e7b0a0 00007f9c85e7a984 00007f9c7f3f56d4 00007f9c7f3f5662
0709 13:17:28.558694 (+  5689us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8945664}
I20260709 13:17:28.531791 14367 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:28.547511 14409 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:28.508479 14403 consensus_queue.cc:237] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:28.535912 14244 consensus_queue.cc:1059] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:17:28.566576 14191 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:28.577412 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:28.559705 14314 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:28.579275 10944 raft_consensus.cc:1240] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 7. Current term is 8. Ops: []
W20260709 13:17:28.579515 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:28.554301 14354 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:28.578989 14456 log.cc:927] Time spent T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 13:17:28.623126 10945 raft_consensus.cc:1275] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:17:28.623978 10945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 1715) took 3334 ms. Trace:
I20260709 13:17:28.639765 10941 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 8.
I20260709 13:17:28.624408 10945 rpcz_store.cc:276] 0709 13:17:25.289325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:25.492840 (+203515us) service_pool.cc:224] Handling call
0709 13:17:25.590423 (+ 97583us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:26.884283 (+1293860us) spinlock_profiling.cc:243] Waited 443 ms on lock 0x7b540015d9a0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80e2cdc0 00007f9c80eb71c0 00007f9c80eb8231 00007f9c80eb4d8f 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c
0709 13:17:28.500433 (+1616150us) spinlock_profiling.cc:243] Waited 214 ms on lock 0x7b540015d9a0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80e3306f 00007f9c80eabf60 00007f9c80ead7de 00007f9c80eb7b49 00007f9c80eb8416 00007f9c80eb4d8f 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62
0709 13:17:28.623638 (+123205us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:28.623905 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1510382080}
W20260709 13:17:28.642484 14357 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
W20260709 13:17:28.643674 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:28.585085 14246 raft_consensus.cc:3055] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:17:28.645515 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:28.647050 11507 raft_consensus.cc:1275] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:17:28.647962 14314 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:28.645269 14032 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:28.589321 14386 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:28.644201 14432 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.664637 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:28.665565 10943 raft_consensus.cc:2410] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 11 because replica has last-logged OpId of term: 9 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 3.
I20260709 13:17:28.641973 14431 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:28.672302 10942 raft_consensus.cc:1275] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:17:28.673509 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:28.645274 14246 raft_consensus.cc:740] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:28.679437 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:28.682247 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:28.657562 14435 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.699553 10941 raft_consensus.cc:1275] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 13:17:28.691515 14454 log.cc:927] Time spent T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.416s	user 0.004s	sys 0.000s
I20260709 13:17:28.697252 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:28.702528 10792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2176) took 2605 ms. Trace:
I20260709 13:17:28.706322 10792 rpcz_store.cc:276] 0709 13:17:26.096752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:26.100197 (+  3445us) service_pool.cc:224] Handling call
0709 13:17:26.100605 (+   408us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:26.180835 (+ 80230us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:26.180877 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:26.407494 (+226617us) spinlock_profiling.cc:243] Waited 4.24 ms on lock 0x7b48002e0158. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c85ddeaa1 00007f9c85dd5b37 00007f9c85dd59ca 00007f9c85ecd26f 00007f9c85eccf37 00007f9c85ee155c 00007f9c85ee1508 00007f9c85ee1488 00007f9c85ee1443 00007f9c85ee06ec 00007f9c84e0aa96
0709 13:17:27.726857 (+1319363us) spinlock_profiling.cc:243] Waited 2.2 ms on lock 0x7b1800196000. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7dd962a6 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7f7aea11 00007f9c7f7be372 00007f9c80ef0a01 00007f9c80eb543a
0709 13:17:28.565204 (+838347us) spinlock_profiling.cc:243] Waited 9.14 ms on lock 0x7b2c000014c8. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7f7aea11 00007f9c7f7be372 00007f9c80ef0a01 00007f9c80eb543a 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2
0709 13:17:28.565751 (+   547us) log.cc:844] Serialized 52 byte log entry
0709 13:17:28.589099 (+ 23348us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:28.589154 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:28.589268 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:28.697077 (+107809us) raft_consensus.cc:1685] finished
0709 13:17:28.697099 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:28.702442 (+  5343us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":85479,"raft.run_cpu_time_us":1193,"raft.run_wall_time_us":215250,"spinlock_wait_cycles":35822464,"thread_start_us":22905,"threads_started":1,"wal-append.queue_time_us":109499}
I20260709 13:17:28.659816 14386 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:28.686165 14358 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.700488 14431 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.723961 14367 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:28.710047 14435 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:28.713629 14354 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:28.732846 10915 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:28.737054 13274 raft_consensus.cc:2749] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:28.737321 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:28.733809 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:28.740243 13829 consensus_queue.cc:1059] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:17:28.741099 14435 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:28.738965 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:28.742663 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:28.744634 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:28.744977 11512 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:28.745869 11509 raft_consensus.cc:2393] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 9: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:28.743729 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:28.741261 14384 raft_consensus.cc:3055] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:17:28.744442 10947 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
I20260709 13:17:28.665789 14432 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:28.751157 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:28.759445 14386 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:28.762027 14360 raft_consensus.cc:3055] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:28.762529 14360 raft_consensus.cc:740] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:28.763381 14360 consensus_queue.cc:260] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:28.764351 14360 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:28.776115 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:28.777187 11506 raft_consensus.cc:2410] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 13 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:17:28.785068 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:28.786290 10947 raft_consensus.cc:1275] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:17:28.788149 10947 raft_consensus.cc:1275] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:17:28.790360 14362 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:28.735189 14358 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:28.750550 14384 raft_consensus.cc:740] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:28.797777 14432 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:28.797914 14433 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:28.808036 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1700) took 1002 ms. Trace:
I20260709 13:17:28.809824 10872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 463) took 3105 ms. Trace:
I20260709 13:17:28.738910 14431 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:28.738549 14345 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:28.802565 11512 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 9.
I20260709 13:17:28.812709 14405 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:28.809628 14144 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:28.814055 14311 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.815167 10949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 510) took 1506 ms. Trace:
I20260709 13:17:28.808410 10869 rpcz_store.cc:276] 0709 13:17:27.805614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:27.805739 (+   125us) service_pool.cc:224] Handling call
0709 13:17:27.806014 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:27.806215 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:17:27.832123 (+ 25908us) log.cc:844] Serialized 14 byte log entry
0709 13:17:27.901699 (+ 69576us) log.cc:844] Serialized 14 byte log entry
0709 13:17:28.310660 (+408961us) spinlock_profiling.cc:243] Waited 25.9 ms on lock 0x7b1800345160. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7dd962a6 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc83f3 00007f9c7ddca7c1
0709 13:17:28.492209 (+181549us) spinlock_profiling.cc:243] Waited 7.83 ms on lock 0x7b1800345160. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc83f3 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:17:28.807584 (+315375us) spinlock_profiling.cc:243] Waited 20.3 ms on lock 0x7b1800345160. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7dd42c0a 00007f9c87482219 00007f9c7ddcaa40 00007f9c7ddc54e4 00007f9c7ddc5455 00007f9c80ead060 00007f9c80ebf716 00007f9c80ed0287 00007f9c80ed0208
0709 13:17:28.807739 (+   155us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:28.807789 (+    50us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:28.807837 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:28.807901 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:28.807993 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11694,"prepare.queue_time_us":2273,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":61,"raft.queue_time_us":1352,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":6913,"spinlock_wait_cycles":124287616,"thread_start_us":90345,"threads_started":2,"wal-append.queue_time_us":131375}
I20260709 13:17:28.744716 14246 consensus_queue.cc:260] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:28.839964 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:28.840521 14246 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:28.843163 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:17:28.843667 10841 connection.cc:256] RPC call timeout handler was delayed by -1.7808733619999657s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1829s for a response
W20260709 13:17:28.844079 10841 connection.cc:256] RPC call timeout handler was delayed by -1.4990893149997646s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1937s for a response
W20260709 13:17:28.844376 10841 connection.cc:256] RPC call timeout handler was delayed by -1.4493632020000859s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16720000000000002s for a response
W20260709 13:17:28.844771 10841 connection.cc:256] RPC call timeout handler was delayed by -1.3672886189999645s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15510000000000002s for a response
W20260709 13:17:28.845296 10841 connection.cc:256] RPC call timeout handler was delayed by -1.0383734320000713s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1875s for a response
I20260709 13:17:28.845500 14129 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.846064 14129 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:28.847239 14308 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:28.847748 14308 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:28.848042 14129 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:28.848584 14129 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.848984 14129 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:28.850031 14308 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:28.850221 14169 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:28.850827 14169 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:28.851516 14431 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:28.852761 14169 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:28.853452 14398 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.853883 14398 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:28.829413 11507 raft_consensus.cc:1275] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:17:28.854629 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:28.855756 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:28.857631 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:28.850083 14327 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.862638 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:28.828421 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:28.830579 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:28.869771 11509 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 10.
I20260709 13:17:28.810135 10872 rpcz_store.cc:276] 0709 13:17:25.704813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:25.705378 (+   565us) service_pool.cc:224] Handling call
0709 13:17:25.705777 (+   399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:25.771015 (+ 65238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:25.771062 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:25.771695 (+   633us) log.cc:844] Serialized 26 byte log entry
0709 13:17:25.980688 (+208993us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:26.406012 (+425324us) spinlock_profiling.cc:243] Waited 31.3 ms on lock 0x7b48001602d8. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c85ddeaa1 00007f9c85dd5b37 00007f9c85dd59ca 00007f9c85ecd26f 00007f9c85eccf37 00007f9c85ee155c 00007f9c85ee1508 00007f9c85ee1488 00007f9c85ee1443 00007f9c85ee06ec 00007f9c84e0aa96
0709 13:17:27.499620 (+1093608us) spinlock_profiling.cc:243] Waited 4.98 ms on lock 0x7b18002edda0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc5455 00007f9c848ad3c7 00007f9c80ebf9a7 00007f9c80ecf0bf 00007f9c80ecf038
0709 13:17:27.676556 (+176936us) log.cc:844] Serialized 14 byte log entry
0709 13:17:28.809504 (+1132948us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:28.809587 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:28.809635 (+    48us) raft_consensus.cc:1685] finished
0709 13:17:28.809647 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:28.809780 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":962126,"prepare.queue_time_us":350766,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":64915,"raft.run_cpu_time_us":3194,"raft.run_wall_time_us":631223,"spinlock_wait_cycles":85326208,"thread_start_us":568347,"threads_started":3,"wal-append.queue_time_us":698519,"wal-append.run_cpu_time_us":3183,"wal-append.run_wall_time_us":1132116}
I20260709 13:17:28.833581 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:28.870930 14327 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:28.872892 14327 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:28.873425 14144 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:28.873618 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:28.876554 14144 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:28.876736 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:28.877925 10801 raft_consensus.cc:2410] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:17:28.829783 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:28.817009 14341 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:28.878737 10798 raft_consensus.cc:1240] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 9. Current term is 10. Ops: []
W20260709 13:17:28.864308 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 125629us
I20260709 13:17:28.879387 14341 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:28.879823 10915 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:28.878753 14311 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:28.881141 10915 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:28.881436 14341 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:28.864882 14367 consensus_queue.cc:1048] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:17:28.882272 11512 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 8.
I20260709 13:17:28.882951 14367 raft_consensus.cc:2764] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:17:28.883165 14360 raft_consensus.cc:2804] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:28.883415 14311 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:28.886456 10841 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:17:28.899142 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:28.865454 14354 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.888562 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:28.866184 10867 raft_consensus.cc:2410] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:17:28.913004 10871 raft_consensus.cc:1240] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 9. Current term is 10. Ops: [7.3-9.4]
I20260709 13:17:28.914243 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:28.914901 10792 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 7.
I20260709 13:17:28.868132 14432 consensus_queue.cc:1059] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:28.868572 14342 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.870347 10948 raft_consensus.cc:1240] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 10. Current term is 11. Ops: []
I20260709 13:17:28.840583 11510 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 10.
I20260709 13:17:28.838696 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:28.919173 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:28.919313 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:28.919538 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:28.919826 10875 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 10.
I20260709 13:17:28.921308 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:28.921489 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:28.922169 10798 raft_consensus.cc:2393] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 10: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:28.878027 14345 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:28.884042 14360 raft_consensus.cc:697] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 7 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:28.925405 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:28.925063 14384 consensus_queue.cc:260] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:28.884335 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 394368us
I20260709 13:17:28.925829 14360 consensus_queue.cc:237] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:28.926755 10767 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:28.860548 14134 consensus_queue.cc:1048] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:28.927731 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:28.928315 10876 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 11.
I20260709 13:17:28.929109 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:28.930049 10872 raft_consensus.cc:1240] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:28.929306 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:28.885159 14409 consensus_queue.cc:1048] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:28.859690 14296 consensus_queue.cc:1048] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:28.932789 11512 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e"
dest_uuid: "12682eb70bc84c76879e04a8628ee586"
mode: GRACEFUL
new_leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
 from {username='slave'} at 127.0.0.1:57818
I20260709 13:17:28.932884 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:28.933506 11512 raft_consensus.cc:606] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Received request to transfer leadership to TS e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:28.845500 14298 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:28.822595 14358 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:28.887980 10869 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 9.
W20260709 13:17:28.897835 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 51841.5us
I20260709 13:17:28.888294 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:28.904309 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:28.939296 10800 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 8.
I20260709 13:17:28.939430 10841 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:28.905097 10915 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:28.915340 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:28.940555 11509 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 10.
I20260709 13:17:28.883673 14439 consensus_queue.cc:1048] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:17:28.918392 10946 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 6.
I20260709 13:17:28.923846 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:28.942643 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:28.943460 13719 consensus_queue.cc:1048] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:28.945226 10794 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 7.
I20260709 13:17:28.924002 11477 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:28.924842 10792 raft_consensus.cc:1240] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 9. Current term is 10. Ops: [9.4-9.4]
I20260709 13:17:28.931354 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:28.933874 11512 raft_consensus.cc:614] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:28.934576 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:28.947285 10798 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:28.837352 10949 rpcz_store.cc:276] 0709 13:17:27.308481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:27.753478 (+444997us) service_pool.cc:224] Handling call
0709 13:17:27.753799 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:28.476175 (+722376us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:28.814976 (+338801us) spinlock_profiling.cc:243] Waited 711 ms on lock 0x7b540018ecd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:28.815115 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1636068736}
I20260709 13:17:28.936165 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:28.936446 10875 raft_consensus.cc:2410] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 10 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:17:28.912217 14308 raft_consensus.cc:2764] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:17:28.961941 14311 raft_consensus.cc:2804] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:28.962589 14311 raft_consensus.cc:697] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 8 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:28.963474 14311 consensus_queue.cc:237] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:28.949224 10871 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 11.
I20260709 13:17:28.966097 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:28.966849 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:28.967350 10867 raft_consensus.cc:1275] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:17:28.967765 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:28.937160 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:28.968699 10872 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 8.
I20260709 13:17:28.949846 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:28.970909 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:17:28.973300 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:28.973513 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:28.975605 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:28.976192 10801 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:28.976712 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:17:28.980134 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 53474.9us
I20260709 13:17:28.981099 10767 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:28.982219 10767 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:28.954128 10868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1708) took 1009 ms. Trace:
I20260709 13:17:28.955139 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:28.929273 14342 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:28.969069 10916 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:28.982318 10867 raft_consensus.cc:1240] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:28.983233 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:28.982815 10868 rpcz_store.cc:276] 0709 13:17:27.945164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:27.945446 (+   282us) service_pool.cc:224] Handling call
0709 13:17:28.023878 (+ 78432us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:28.813815 (+789937us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:28.813846 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:28.814314 (+   468us) log.cc:844] Serialized 26 byte log entry
0709 13:17:28.932235 (+117921us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:28.932274 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:28.932348 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:28.953900 (+ 21552us) raft_consensus.cc:1685] finished
0709 13:17:28.953924 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:28.954066 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":789547,"raft.queue_time_us":790668,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":82,"thread_start_us":117549,"threads_started":1,"wal-append.queue_time_us":137905}
I20260709 13:17:28.984807 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:28.986202 10916 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:28.938495 14314 raft_consensus.cc:3055] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:17:28.951421 10841 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:28.988822 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:28.988950 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:28.993975 10868 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 10.
I20260709 13:17:28.995064 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:28.989660 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:28.989784 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:28.996227 10792 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:28.989886 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:28.996546 14432 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:28.997316 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:28.998159 10876 raft_consensus.cc:2410] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 11 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:17:28.990728 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:28.991144 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:28.992043 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:28.916683 14354 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:29.000792 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 49447.8us
I20260709 13:17:29.002266 10798 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 10.
I20260709 13:17:28.992363 10916 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:29.003669 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:29.005057 10766 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:29.005218 10767 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:29.002574 10841 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:29.009377 11478 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:29.010886 10868 raft_consensus.cc:2393] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 9: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:29.012951 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:29.013562 10872 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 8.
I20260709 13:17:29.015774 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:29.016860 14354 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:28.965590 14440 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:29.018146 14342 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:29.018756 10867 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 7.
I20260709 13:17:29.019398 14333 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:29.019825 14333 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:29.011751 14274 raft_consensus.cc:2749] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:29.020210 14333 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:29.021518 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:28.985816 14409 raft_consensus.cc:2749] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:29.023391 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:28.985654 14405 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:29.024380 10948 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 9.
I20260709 13:17:29.024814 14405 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:29.026865 14296 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:29.027320 14296 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:29.027693 14296 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:29.030467 14398 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:29.031312 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:29.031947 11510 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 6.
I20260709 13:17:29.035019 11512 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650"
dest_uuid: "12682eb70bc84c76879e04a8628ee586"
mode: GRACEFUL
new_leader_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:57818
I20260709 13:17:28.970649 14358 raft_consensus.cc:2749] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:17:29.037938 10792 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 11.
I20260709 13:17:29.038415 10841 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:29.040959 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:28.983721 14341 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:17:29.039989 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 47694us
I20260709 13:17:28.984691 14148 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:29.048326 10943 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 10.
I20260709 13:17:29.040330 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:29.052074 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 45400.3us
I20260709 13:17:29.062674 10916 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:29.066488 10841 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:29.070075 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:28.992916 14314 raft_consensus.cc:740] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:28.990604 14459 raft_consensus.cc:2749] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:28.997411 14345 raft_consensus.cc:2749] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:29.085783 10916 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:17:29.086288 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 221896us
I20260709 13:17:29.087423 10916 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:17:29.073328 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:29.088760 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 83607.7us
I20260709 13:17:29.088889 10916 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:29.089955 10767 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:28.995893 11477 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:28.948944 14298 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:29.096756 10841 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:29.083360 10766 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:29.097853 10842 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:29.097600 10767 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:29.099867 10916 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:29.035897 14331 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:17:29.101867 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 106014us
I20260709 13:17:29.044296 14407 raft_consensus.cc:2749] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:29.104473 10841 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:29.105404 10842 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:29.106978 10766 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:29.108462 10767 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:29.043838 14248 raft_consensus.cc:2749] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:28.999558 14446 consensus_queue.cc:1059] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.084s
I20260709 13:17:29.110033 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:29.081647 14043 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:29.118851 10942 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 7.
I20260709 13:17:29.113041 10916 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:29.073097 14354 raft_consensus.cc:2749] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:29.121677 14406 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:29.095908 14134 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:29.082523 14341 raft_consensus.cc:697] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:29.096604 14405 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:29.133316 10915 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:29.134553 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:29.117446 14435 raft_consensus.cc:2804] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:29.101457 14362 raft_consensus.cc:2804] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:29.101773 14409 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:29.140236 11512 raft_consensus.cc:606] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Received request to transfer leadership to TS cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:29.111538 14386 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:29.111778 14361 raft_consensus.cc:2749] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:29.147998 10941 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
I20260709 13:17:29.142465 10867 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:29.110036 14358 raft_consensus.cc:2749] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:29.131621 14406 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:29.044579 14217 consensus_queue.cc:1059] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.073s
I20260709 13:17:29.121546 14301 raft_consensus.cc:2749] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:29.115862 14432 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:29.137392 10767 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:17:29.151556 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:29.135278 10916 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:29.114323 14445 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:29.152398 11512 raft_consensus.cc:614] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:29.082990 14305 consensus_queue.cc:1048] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 13:17:29.125808 14327 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:29.072940 14148 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:29.138935 14134 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:29.124730 14403 consensus_queue.cc:1059] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 13:17:29.136672 10766 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:29.131840 13995 raft_consensus.cc:2749] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:29.115072 14169 consensus_queue.cc:1048] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:17:29.180936 11510 raft_consensus.cc:1275] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:17:29.122670 14331 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:29.141379 14304 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:29.179376 14432 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:29.182452 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 77129us
I20260709 13:17:29.139773 14378 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:17:29.183167 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 48028.4us
I20260709 13:17:29.145545 14435 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:29.124191 14360 raft_consensus.cc:2749] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:29.152383 14248 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:29.143174 14129 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:29.148808 14043 raft_consensus.cc:697] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:29.141304 14278 consensus_queue.cc:1048] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.054s
I20260709 13:17:29.152246 14362 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:29.136035 14399 consensus_queue.cc:1048] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 13:17:29.167354 14450 raft_consensus.cc:2804] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:29.157444 14386 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:29.216909 14044 raft_consensus.cc:2749] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:17:29.197652 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 60328.7us
I20260709 13:17:29.254087 10767 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:29.200978 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 96575.4us
I20260709 13:17:29.172799 14406 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:29.205286 14445 raft_consensus.cc:697] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:29.203316 14134 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:17:29.208717 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 72135us
I20260709 13:17:29.265244 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:29.226354 10916 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:29.268909 10916 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:29.270008 10916 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:29.233464 14148 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:29.141250 14341 consensus_queue.cc:237] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:29.235833 14358 consensus_queue.cc:1048] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:17:29.202251 14459 consensus_queue.cc:1048] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:17:29.198408 14345 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:29.290951 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:29.234570 10867 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 8.
I20260709 13:17:29.202649 13717 raft_consensus.cc:2749] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:29.197544 14217 raft_consensus.cc:2749] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:17:29.234582 14474 log.cc:927] Time spent T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:17:29.226380 14432 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:29.228643 14378 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:29.255234 10767 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:29.242420 14403 consensus_queue.cc:1243] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer cb0428cc55984daba1ba63744a5102a2 log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:17:29.289510 14360 raft_consensus.cc:2804] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:29.320561 14360 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:29.320348 14333 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:29.274983 14409 consensus_queue.cc:1048] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:17:29.321584 14296 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:29.176862 14314 consensus_queue.cc:260] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:29.257352 14393 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:29.250571 14362 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:29.361013 14248 raft_consensus.cc:697] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:29.361954 14248 consensus_queue.cc:237] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:29.362927 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 107202us
I20260709 13:17:29.364166 10841 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:29.363785 10696 catalog_manager.cc:5697] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 5 to 7, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 7 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:29.365760 10841 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:29.366186 14314 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:29.367365 10873 raft_consensus.cc:1240] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 10. Current term is 11. Ops: []
I20260709 13:17:29.367753 14479 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:29.368165 14479 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:29.368482 14479 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:29.369915 14169 consensus_queue.cc:1048] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:17:29.372162 14169 consensus_queue.cc:1048] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:29.374652 14296 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:29.287225 10916 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:29.252120 14450 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:29.252015 14386 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:29.251477 14440 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:29.293216 11478 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:17:29.286816 14431 raft_consensus.cc:2804] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:29.287022 14361 raft_consensus.cc:2749] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:29.242867 14435 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:29.305774 14378 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:29.439956 14431 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:29.290657 14345 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:17:29.316262 14472 log.cc:927] Time spent T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.112s	user 0.002s	sys 0.000s
W20260709 13:17:29.318133 10698 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42506 (request call id 154) took 12456 ms (client timeout 15000 ms). Trace:
I20260709 13:17:29.319442 14331 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:29.320958 14360 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:29.323305 14388 raft_consensus.cc:1064] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87: attempting to promote NON_VOTER 12682eb70bc84c76879e04a8628ee586 to VOTER
I20260709 13:17:29.320394 14298 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:29.324158 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 68994.2us
I20260709 13:17:29.300395 14043 consensus_queue.cc:237] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:29.328487 14134 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:29.328964 14391 raft_consensus.cc:3055] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:29.332616 14392 raft_consensus.cc:2749] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:29.497303 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:17:29.459884 10698 rpcz_store.cc:269] 0709 13:17:16.861274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:16.876165 (+ 14891us) service_pool.cc:224] Handling call
0709 13:17:29.318064 (+12441899us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:23.773224 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:24.064806 (+291582us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:24.064868 (+    62us) write_op.cc:435] Acquired schema lock
0709 13:17:24.064892 (+    24us) tablet.cc:662] Decoding operations
0709 13:17:24.359572 (+294680us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:24.359648 (+    76us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:24.359676 (+    28us) tablet.cc:685] Acquiring locks for 16 operations
0709 13:17:24.360996 (+  1320us) tablet.cc:701] Row locks acquired
0709 13:17:24.361047 (+    51us) write_op.cc:260] PREPARE: finished
0709 13:17:24.361209 (+   162us) write_op.cc:270] Start()
0709 13:17:24.361359 (+   150us) write_op.cc:276] Timestamp: P: 1783603044361153 usec, L: 0
0709 13:17:24.361379 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:17:24.362136 (+   757us) log.cc:844] Serialized 6370 byte log entry
0709 13:17:25.240544 (+878408us) op_driver.cc:464] REPLICATION: finished
0709 13:17:25.522073 (+281529us) write_op.cc:301] APPLY: starting
0709 13:17:25.522176 (+   103us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:25.522488 (+   312us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:25.522607 (+   119us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:25.890653 (+368046us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:25.891053 (+   400us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=16
0709 13:17:25.891099 (+    46us) write_op.cc:312] APPLY: finished
0709 13:17:25.891508 (+   409us) log.cc:844] Serialized 113 byte log entry
0709 13:17:25.891729 (+   221us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:25.891893 (+   164us) write_op.cc:454] Released schema lock
0709 13:17:25.891959 (+    66us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:25.892376 (+   417us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:25.892404 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":148645,"reported_tablets":31,"child_traces":[["op",{"apply.queue_time_us":241194,"apply.run_cpu_time_us":6335,"apply.run_wall_time_us":370308,"num_ops":16,"prepare.queue_time_us":577546,"prepare.run_cpu_time_us":23676,"prepare.run_wall_time_us":733235,"raft.queue_time_us":362534,"raft.run_cpu_time_us":9661,"raft.run_wall_time_us":140495,"replication_time_us":859354,"thread_start_us":597109,"threads_started":4,"wal-append.queue_time_us":90384,"wal-append.run_cpu_time_us":19364,"wal-append.run_wall_time_us":2443047}]]}
I20260709 13:17:29.332733 14367 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:29.334981 14032 raft_consensus.cc:3055] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:29.322570 14333 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:29.341327 14323 raft_consensus.cc:3055] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:17:29.341463 14476 log.cc:927] Time spent T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
I20260709 13:17:29.275231 14445 consensus_queue.cc:237] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:29.291260 14384 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:29.347276 11512 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "414992c8073646798991d2b43fd61f1d"
dest_uuid: "12682eb70bc84c76879e04a8628ee586"
mode: GRACEFUL
new_leader_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:57818
W20260709 13:17:29.351759 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 129576us
I20260709 13:17:29.354395 14433 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:29.355227 14406 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:29.266595 10766 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:29.484560 14391 raft_consensus.cc:740] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:29.373796 14169 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:29.392580 10801 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 8.
I20260709 13:17:29.294471 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:29.529902 10871 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 10.
I20260709 13:17:29.420498 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:29.437626 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:29.531662 10944 raft_consensus.cc:2410] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:17:29.419021 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:29.532742 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:29.411141 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:29.534021 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:29.416219 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 129065us
I20260709 13:17:29.409463 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:29.441532 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 148441us
I20260709 13:17:29.535960 10916 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:29.416738 14450 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:29.449134 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:29.301874 14354 consensus_queue.cc:1048] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
W20260709 13:17:29.452195 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 155509us
I20260709 13:17:29.455947 10841 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:29.395081 14304 raft_consensus.cc:2749] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:29.390627 14362 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:29.416844 14439 raft_consensus.cc:2749] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:29.451620 14440 raft_consensus.cc:2804] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:29.457484 14431 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:17:29.479925 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:29.397264 14393 raft_consensus.cc:2749] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:29.483125 10800 raft_consensus.cc:1240] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 10. Current term is 11. Ops: []
I20260709 13:17:29.483173 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:29.568536 10794 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 7.
I20260709 13:17:29.461041 14345 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:29.490670 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:29.496124 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:29.499478 10793 raft_consensus.cc:1275] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:17:29.394762 14479 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:29.512912 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:29.580090 10950 raft_consensus.cc:1240] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 10. Current term is 11. Ops: []
I20260709 13:17:29.580190 10799 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 8.
I20260709 13:17:29.462083 13995 consensus_queue.cc:1048] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:17:29.496862 14181 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:29.521533 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 254986us
I20260709 13:17:29.365492 10696 catalog_manager.cc:5697] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:17:29.586148 10696 catalog_manager.cc:5697] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 6 to 7, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 7 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:17:29.559360 14361 raft_consensus.cc:2804] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:29.524858 14342 consensus_queue.cc:1059] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:29.519786 11512 raft_consensus.cc:606] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 8 LEADER]: Received request to transfer leadership to TS cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:29.528251 10796 pending_rounds.cc:85] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 7
I20260709 13:17:29.540262 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:29.503363 14032 raft_consensus.cc:740] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:29.551946 10916 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:29.512228 14323 raft_consensus.cc:740] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:29.561470 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:29.608894 10945 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:29.561064 10792 raft_consensus.cc:1240] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:17:29.564291 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:29.611872 10801 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
I20260709 13:17:29.565590 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:29.520402 14333 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:29.569787 11508 raft_consensus.cc:1275] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 13:17:29.569782 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:29.616834 11508 pending_rounds.cc:85] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586: Aborting all ops after (but not including) 5
I20260709 13:17:29.617054 10794 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:29.617156 11508 pending_rounds.cc:107] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 13:17:29.617324 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:29.546221 14169 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:29.506968 14367 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:29.530578 14331 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:29.578534 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:29.580763 10798 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 10.
I20260709 13:17:29.581332 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:29.524322 14144 consensus_queue.cc:1059] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:29.587630 10696 catalog_manager.cc:5697] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 4 to 6, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 6 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:29.585055 10766 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:29.592427 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:29.595427 10796 pending_rounds.cc:107] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:17:29.632481 10796 raft_consensus.cc:2889] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:29.546904 14433 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:29.639091 11478 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:29.614391 10873 raft_consensus.cc:1240] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:29.617540 11508 raft_consensus.cc:2889] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:17:29.571985 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 116148us
I20260709 13:17:29.640573 10841 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:29.639619 10696 catalog_manager.cc:5697] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 4 to 8, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 8 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:17:29.579649 14336 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:29.669354 14409 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:29.669663 14336 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:29.670078 14336 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:29.670037 14409 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:29.673509 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 88513us
I20260709 13:17:29.674407 10766 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:29.675393 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:29.676791 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:29.677003 10795 raft_consensus.cc:1240] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:29.577323 14440 raft_consensus.cc:697] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 8 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:29.619402 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:29.634737 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:29.635174 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:29.689467 10800 raft_consensus.cc:2410] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:17:29.634627 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:29.636453 10867 raft_consensus.cc:1275] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:17:29.638375 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:29.690874 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:29.639583 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:29.692376 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:29.587213 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:29.662043 14246 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:29.662760 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:29.663528 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:29.666491 14439 raft_consensus.cc:2749] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:29.637208 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:29.641950 10696 catalog_manager.cc:5697] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 6 to 7, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 7 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:29.664669 14367 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:29.666316 14433 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:29.631362 14333 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:29.668814 14341 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:29.669123 10841 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:29.727743 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:29.594190 14361 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:29.680367 14396 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:29.693481 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:29.701594 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:29.701529 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:29.703279 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:29.710983 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 71985.5us
I20260709 13:17:29.744179 11478 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:29.712546 10696 catalog_manager.cc:5697] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 8 to 9, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 9 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:29.746205 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:29.616676 14393 consensus_queue.cc:1048] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.110s
W20260709 13:17:29.747464 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 78416.9us
I20260709 13:17:29.752689 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:29.717950 10798 raft_consensus.cc:1240] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:29.714581 10945 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 11.
I20260709 13:17:29.632292 14331 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:29.632594 14169 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:29.594959 14181 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:29.731297 10800 raft_consensus.cc:1275] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:17:29.747308 10696 catalog_manager.cc:5697] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:29.745609 11478 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:29.758291 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:29.701159 14298 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:29.717793 10767 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:29.621953 14403 consensus_queue.cc:1059] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.106s
W20260709 13:17:29.764691 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 212785us
I20260709 13:17:29.700946 14390 raft_consensus.cc:2804] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:29.773816 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:29.774576 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:29.843447 10945 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 8.
I20260709 13:17:29.844547 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:17:29.775168 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:29.846048 10916 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:29.777397 10794 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 8.
I20260709 13:17:29.837796 10841 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:29.836092 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 90590.1us
I20260709 13:17:29.849021 11478 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:29.850195 11478 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:29.837268 10696 catalog_manager.cc:5697] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 4 to 8, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 8 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:29.739552 14341 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:29.838490 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 49707.9us
I20260709 13:17:29.726562 14412 raft_consensus.cc:2749] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:29.855968 10766 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:29.748929 14361 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:29.857829 10947 raft_consensus.cc:2393] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 9: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:29.597492 14391 consensus_queue.cc:260] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:29.872361 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:29.872917 10868 raft_consensus.cc:1275] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:17:29.874459 10800 raft_consensus.cc:1275] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:17:29.861369 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:29.863998 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 146302us
I20260709 13:17:29.748355 14433 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:29.762979 14331 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:29.762768 14296 raft_consensus.cc:2749] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:29.859587 10916 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:29.879627 11510 raft_consensus.cc:1275] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:17:29.880180 10841 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:29.721450 14246 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:29.788173 14402 raft_consensus.cc:2749] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:29.835366 14298 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:29.845027 14421 raft_consensus.cc:2749] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:29.893625 10767 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:29.894837 10944 raft_consensus.cc:2393] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 8: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:29.899528 10767 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:29.857565 14390 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:29.860693 14341 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:29.867442 14439 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:29.907444 11507 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546"
dest_uuid: "12682eb70bc84c76879e04a8628ee586"
mode: GRACEFUL
new_leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
 from {username='slave'} at 127.0.0.1:57818
I20260709 13:17:29.881909 14217 raft_consensus.cc:2804] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:29.873754 14305 raft_consensus.cc:2749] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:17:29.786350 14396 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:29.835183 13995 consensus_queue.cc:1048] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.066s
I20260709 13:17:29.927488 10944 raft_consensus.cc:1275] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:17:29.930462 11507 raft_consensus.cc:606] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 10 LEADER]: Received request to transfer leadership to TS e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:29.901726 14391 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:29.902146 14298 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:29.896293 13717 raft_consensus.cc:2749] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:29.901612 14403 raft_consensus.cc:2804] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:29.901579 14392 raft_consensus.cc:2749] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:17:29.902887 14257 raft_consensus.cc:2749] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:29.753240 14323 consensus_queue.cc:260] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:29.919298 14405 raft_consensus.cc:2749] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:17:29.921427 14390 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:30.045368 14478 raft_consensus.cc:3055] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:30.045825 14478 raft_consensus.cc:740] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:30.046669 14478 consensus_queue.cc:260] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:30.047394 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:30.054385 14378 consensus_meta.cc:287] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.605s	user 0.003s	sys 0.003s
I20260709 13:17:30.048146 14425 consensus_queue.cc:1059] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.007s
W20260709 13:17:30.060622 14488 log.cc:927] Time spent T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.314s	user 0.000s	sys 0.002s
W20260709 13:17:30.010083 14483 log.cc:927] Time spent T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.152s	user 0.000s	sys 0.000s
I20260709 13:17:30.061817 14246 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:30.064714 10868 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:17:30.065431 10868 raft_consensus.cc:606] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 LEADER]: Received request to transfer leadership to TS cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:30.067864 14181 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:17:30.011464 14482 log.cc:927] Time spent T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.371s	user 0.001s	sys 0.000s
I20260709 13:17:29.897589 14412 consensus_queue.cc:1048] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 13:17:30.072041 11511 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca"
dest_uuid: "12682eb70bc84c76879e04a8628ee586"
mode: GRACEFUL
new_leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
 from {username='slave'} at 127.0.0.1:57818
I20260709 13:17:30.072901 11511 raft_consensus.cc:606] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 11 LEADER]: Received request to transfer leadership to TS e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:30.012408 14490 log.cc:927] Time spent T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
I20260709 13:17:30.012841 14406 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:30.013192 14311 raft_consensus.cc:3055] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:17:30.075234 14311 raft_consensus.cc:740] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:29.905292 14407 consensus_queue.cc:1048] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.076s
I20260709 13:17:30.076167 14311 consensus_queue.cc:260] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:30.077200 14311 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:30.082666 14323 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:30.086187 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:30.087154 14384 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:30.087457  9682 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:17:30.087850 14384 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:30.088496 14384 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:29.874855 14032 consensus_queue.cc:260] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:30.090548 14032 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:29.933645 14395 raft_consensus.cc:2749] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:30.093292 10875 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:30.095326  9682 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:17:30.095427 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:29.992303 10943 raft_consensus.cc:1275] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 13:17:30.097379 10943 pending_rounds.cc:85] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 9
I20260709 13:17:30.097712 10943 pending_rounds.cc:107] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:17:30.098001 10943 raft_consensus.cc:2889] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:17:30.013715 14485 log.cc:927] Time spent T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
I20260709 13:17:30.023129 14134 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:30.024835 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:30.027274 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:30.114216 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:30.029137 10941 raft_consensus.cc:1275] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 13:17:30.032338 14480 raft_consensus.cc:3055] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:30.038059 11477 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:17:30.127437 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 89466.5us
W20260709 13:17:29.901669 14459 consensus_queue.cc:1175] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" responder_term: 8 status { last_received { term: 8 index: 9 } last_committed_idx: 8 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 7. Current term is 8. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:17:30.043865 11507 raft_consensus.cc:1275] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:17:30.016017 14409 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:29.893013 14433 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:29.957584 14217 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:30.110167 14425 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:29.880678 14440 consensus_queue.cc:237] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:30.121871 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:30.133260 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:30.138298 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 288155us
I20260709 13:17:30.139729 10793 raft_consensus.cc:1275] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:17:29.976516 14362 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:30.141680 10842 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:30.005023 14403 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:30.173491 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:30.183351 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:30.197394 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:30.199795 11478 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:30.188573 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:30.126992 14494 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:30.202445 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 60849.6us
I20260709 13:17:30.126927 14480 raft_consensus.cc:740] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:30.140584 14412 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:17:30.158648 14367 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.158417 14479 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:30.210085 11478 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:30.116051 14378 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:30.209909 14144 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:30.190568 14217 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:30.238590 10942 raft_consensus.cc:2393] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 12: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:30.211952 14478 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:29.960019 14388 consensus_queue.cc:237] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 6, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:30.215339 10875 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 9.
I20260709 13:17:30.198987 14403 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:30.190925 14333 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:30.144270 14181 consensus_queue.cc:1048] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
W20260709 13:17:30.219677 14336 consensus_meta.cc:287] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.549s	user 0.011s	sys 0.000s
I20260709 13:17:30.190088 14406 consensus_queue.cc:1048] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 13:17:30.243659 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:30.243166 14314 consensus_meta.cc:287] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.876s	user 0.012s	sys 0.004s
I20260709 13:17:30.192186 14354 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:30.190394 14246 consensus_queue.cc:1059] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 13:17:30.188417 14421 consensus_queue.cc:1048] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:17:30.248517 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:30.198273 14425 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:30.275066 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:30.255046 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:30.215824 14499 raft_consensus.cc:2749] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:30.263751 10872 raft_consensus.cc:1240] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 10. Current term is 11. Ops: []
I20260709 13:17:30.234052 14144 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:30.216507 14248 consensus_queue.cc:1059] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:17:30.234221 14412 raft_consensus.cc:697] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 11 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
W20260709 13:17:30.251425 14432 consensus_meta.cc:287] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.947s	user 0.011s	sys 0.000s
I20260709 13:17:30.273340 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:30.243064 14402 raft_consensus.cc:2749] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:17:30.273818 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 375020us
I20260709 13:17:30.208750 14494 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:30.216450 14367 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:30.287829 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:30.288010 10766 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:30.293788 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:30.305143 11512 raft_consensus.cc:1275] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:17:30.284061 14144 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:30.307906 10766 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:30.309508 10767 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:30.312017 10767 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:30.312788 14433 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:30.312896 14169 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:30.313553 14304 raft_consensus.cc:2749] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:17:30.293915 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 434413us
I20260709 13:17:30.260421 13995 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:30.295957 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:30.317435 10916 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:30.299656 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:17:30.298831 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:30.319407 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:30.320776 11507 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 9.
W20260709 13:17:30.321641 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:30.322352 10871 raft_consensus.cc:2393] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 11: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:30.325433 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:17:30.334143 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 453989us
I20260709 13:17:30.335585 10841 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:30.343995 10766 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:30.346375 10766 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:30.295611 14404 consensus_queue.cc:1048] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 13:17:30.267529 14336 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:30.352038 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:30.352900 11509 raft_consensus.cc:1240] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 7. Current term is 8. Ops: [6.8-7.9]
I20260709 13:17:30.357853 10872 raft_consensus.cc:1240] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 11. Current term is 12. Ops: []
W20260709 13:17:30.306372 14191 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.011s	sys 0.003s
I20260709 13:17:30.350431 13995 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:30.350477 14499 consensus_queue.cc:1048] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 13:17:30.353967 14246 consensus_queue.cc:1048] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:17:30.354206 14304 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:30.362902 14433 raft_consensus.cc:2804] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:30.364004 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:30.282356 14354 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:30.458705 14354 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:30.310527 14409 raft_consensus.cc:2749] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:30.311053 14407 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.459600 14407 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:30.461745 14407 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:30.390950 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 79004.6us
I20260709 13:17:30.463086 10767 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:30.323570 14405 raft_consensus.cc:2749] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:17:30.409922 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 63629us
I20260709 13:17:30.466362 14505 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.466965 14505 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:30.468580 14390 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:30.469391 14505 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:30.466915 10766 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:30.411302 10841 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:30.354710 14360 consensus_meta.cc:287] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.874s	user 0.011s	sys 0.000s
W20260709 13:17:30.359606 14435 consensus_meta.cc:287] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.909s	user 0.012s	sys 0.000s
I20260709 13:17:30.300098 14432 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:30.408298 14300 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
W20260709 13:17:30.386257 14445 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.023s	sys 0.024s
W20260709 13:17:30.418301 14496 log.cc:927] Time spent T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.172s	user 0.000s	sys 0.000s
W20260709 13:17:30.418850 14492 log.cc:927] Time spent T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.210s	user 0.000s	sys 0.000s
I20260709 13:17:30.292625 14497 consensus_queue.cc:1059] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 13:17:30.448165 10800 raft_consensus.cc:2410] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 10 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:17:30.446657 10872 raft_consensus.cc:1240] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:17:30.297546 14480 consensus_queue.cc:260] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:30.365612 14396 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:30.538990 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:30.513314 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 50381.9us
W20260709 13:17:30.511317 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:30.522429 10950 raft_consensus.cc:1240] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 11. Current term is 12. Ops: []
W20260709 13:17:30.409345 14386 consensus_meta.cc:287] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.955s	user 0.007s	sys 0.006s
W20260709 13:17:30.551750 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 140576us
I20260709 13:17:30.455111 14367 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:30.456455 14494 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:30.456660 14336 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:30.447773 14433 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:30.408222 14479 raft_consensus.cc:2749] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:30.511114 10870 raft_consensus.cc:1275] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 13:17:30.522329 10942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41020 (request call id 2161) took 1094 ms. Trace:
I20260709 13:17:30.511065 10945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2162) took 1082 ms. Trace:
I20260709 13:17:30.566557 11478 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:30.572875 10942 rpcz_store.cc:276] 0709 13:17:29.427899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:29.533801 (+105902us) service_pool.cc:224] Handling call
0709 13:17:30.522117 (+988316us) spinlock_profiling.cc:243] Waited 700 ms on lock 0x7b54000a96d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:30.522270 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1609062784}
I20260709 13:17:30.574761 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:30.470968 10794 raft_consensus.cc:1240] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 7. Current term is 8. Ops: []
W20260709 13:17:30.401207 14191 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.011s	sys 0.004s
W20260709 13:17:30.548573 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:30.581583 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:30.575734 10942 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 9.
I20260709 13:17:30.422757 14246 consensus_queue.cc:1243] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer e01ecd008edd4aa2b6b2b47efbc2ea87 log is divergent from this leader: its last log entry 5.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:17:30.535240 11510 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 11.
I20260709 13:17:30.453584 14439 raft_consensus.cc:2749] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:30.470718 14390 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:30.398260 10916 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:30.551229 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 84526.3us
I20260709 13:17:30.544695 10767 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:30.386799 14362 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:17:30.549744 14345 consensus_meta.cc:287] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.963s	user 0.009s	sys 0.000s
I20260709 13:17:30.616636 14345 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:30.542243 14300 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:30.620350 14432 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:30.620798 14300 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:30.533874 14450 consensus_meta.cc:287] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.971s	user 0.008s	sys 0.002s
I20260709 13:17:30.621584 14450 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:30.623713 14450 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:30.624516 10946 raft_consensus.cc:1240] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:17:30.565925 14509 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:30.627386 14509 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:30.550809 14360 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:30.629323 10949 raft_consensus.cc:2393] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 9: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:30.629980 14509 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:30.631335 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:30.632366 14480 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:30.632629 10796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 412) took 1113 ms. Trace:
W20260709 13:17:30.632901 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 88297.5us
I20260709 13:17:30.633090 10796 rpcz_store.cc:276] 0709 13:17:29.518829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:29.522437 (+  3608us) service_pool.cc:224] Handling call
0709 13:17:29.528036 (+  5599us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:29.675102 (+147066us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:29.675142 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:29.675595 (+   453us) log.cc:844] Serialized 26 byte log entry
0709 13:17:29.756899 (+ 81304us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:29.756934 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:29.756997 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:30.632405 (+875408us) raft_consensus.cc:1685] finished
0709 13:17:30.632424 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:30.632566 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":80937,"threads_started":1,"wal-append.queue_time_us":509382}
I20260709 13:17:30.634730 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:30.634979 14405 raft_consensus.cc:2749] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:17:30.638103 14361 consensus_meta.cc:287] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.760s	user 0.009s	sys 0.000s
I20260709 13:17:30.386601 14378 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:30.525806 14402 raft_consensus.cc:993] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586: : Instructing follower e01ecd008edd4aa2b6b2b47efbc2ea87 to start an election
I20260709 13:17:30.477255 14425 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:30.577589 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:30.694191 11477 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:30.697005 10950 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10.
I20260709 13:17:30.698114 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:30.698175 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:30.496582 14333 raft_consensus.cc:3055] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:17:30.483393 13995 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:30.701287 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:30.701915 10950 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
I20260709 13:17:30.415884 14503 raft_consensus.cc:2749] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:30.421630 14421 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:30.543920 14445 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.781s	user 0.023s	sys 0.031s
I20260709 13:17:30.570070 10945 rpcz_store.cc:276] 0709 13:17:29.428753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:29.845544 (+416791us) service_pool.cc:224] Handling call
0709 13:17:29.845853 (+   309us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:29.862999 (+ 17146us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:29.863033 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:29.876763 (+ 13730us) log.cc:844] Serialized 52 byte log entry
0709 13:17:30.268532 (+391769us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:30.268588 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:30.268678 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:30.510866 (+242188us) raft_consensus.cc:1685] finished
0709 13:17:30.510882 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:30.511005 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12510,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":29778,"thread_start_us":308803,"threads_started":1,"wal-append.queue_time_us":588211}
I20260709 13:17:30.707136 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:30.707257 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:30.707741 10945 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:30.707767 11512 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:30.521704 14169 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:30.470991 14275 raft_consensus.cc:2749] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:17:30.550560 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:30.700408 14333 raft_consensus.cc:740] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:30.425740 14393 raft_consensus.cc:2749] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:30.714036 10947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2163) took 1284 ms. Trace:
I20260709 13:17:30.714493 10947 rpcz_store.cc:276] 0709 13:17:29.429663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:29.877518 (+447855us) service_pool.cc:224] Handling call
0709 13:17:29.877844 (+   326us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:30.130951 (+253107us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:30.130978 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:30.134649 (+  3671us) log.cc:844] Serialized 52 byte log entry
0709 13:17:30.587248 (+452599us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:30.587445 (+   197us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:30.587988 (+   543us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:30.713818 (+125830us) raft_consensus.cc:1685] finished
0709 13:17:30.713840 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:30.713981 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39832,"raft.queue_time_us":125554,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":119,"thread_start_us":633565,"threads_started":2,"wal-append.queue_time_us":517880}
I20260709 13:17:30.524099 14342 raft_consensus.cc:3055] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:17:30.600638 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:30.567163 14311 raft_consensus.cc:3055] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:30.605753 14459 consensus_queue.cc:1048] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:30.566859 14433 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:30.609598 14298 consensus_meta.cc:287] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.515s	user 0.007s	sys 0.000s
I20260709 13:17:30.422338 14412 consensus_queue.cc:237] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:30.611428 14217 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:30.564570 14396 raft_consensus.cc:2749] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:30.614877 14497 raft_consensus.cc:2749] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:30.564849 14386 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:30.615602 14362 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:30.564508 14392 raft_consensus.cc:2749] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:17:30.494119 14331 consensus_meta.cc:287] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.615s	user 0.007s	sys 0.000s
W20260709 13:17:30.642298 14391 consensus_meta.cc:287] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.648s	user 0.009s	sys 0.000s
I20260709 13:17:30.644690 14345 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:30.646574 10767 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:30.796423 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 149948us
I20260709 13:17:30.797209 10767 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:30.630419 14360 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:17:30.648409 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 43980.2us
W20260709 13:17:30.648787 14032 consensus_meta.cc:287] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.558s	user 0.008s	sys 0.002s
I20260709 13:17:30.649201 14390 consensus_queue.cc:1059] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:30.653980 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:30.750218 14367 raft_consensus.cc:2749] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:17:30.654939 14431 consensus_meta.cc:287] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 1.073s	user 0.008s	sys 0.003s
W20260709 13:17:30.656311 14341 consensus_meta.cc:287] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.717s	user 0.007s	sys 0.003s
I20260709 13:17:30.609789 14403 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:30.662410 10944 raft_consensus.cc:1240] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 8. Current term is 9. Ops: [6.4-8.5]
I20260709 13:17:30.663585 10949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41020 (request call id 2160) took 1236 ms. Trace:
I20260709 13:17:30.665889 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:30.666059 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:30.828387 10796 raft_consensus.cc:1275] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:17:30.829903 10796 raft_consensus.cc:1240] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 9. Current term is 10. Ops: []
W20260709 13:17:30.667615 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:30.830868 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:30.669994 10841 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:30.833324 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 163401us
I20260709 13:17:30.834162 10841 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:30.673465 14478 consensus_queue.cc:1059] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:30.665129 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:17:30.682454 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 284242us
I20260709 13:17:30.448088 14304 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:30.686666 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:30.749238 14298 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:30.811995 14409 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:30.542236 14435 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:30.703696 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:30.716578 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:30.853293 10794 raft_consensus.cc:1240] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:30.718837 10870 raft_consensus.cc:1240] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:17:30.666611 14144 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:30.848503 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:30.860832 10796 raft_consensus.cc:2393] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 11: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:30.705405 14446 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:30.723677 14342 raft_consensus.cc:740] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:30.723776 14311 raft_consensus.cc:740] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:30.694905 14361 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:30.748864 10945 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:17:30.761368 10797 raft_consensus.cc:1275] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:17:30.873615 10869 raft_consensus.cc:1240] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:30.850528 14043 raft_consensus.cc:2749] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:30.758710 11512 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 11.
I20260709 13:17:30.784945 10793 raft_consensus.cc:1240] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:30.793825 10792 raft_consensus.cc:1240] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 13:17:30.745425 14181 raft_consensus.cc:1081] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 10 LEADER]: Signalling peer e01ecd008edd4aa2b6b2b47efbc2ea87 to start an election
I20260709 13:17:30.800011 10799 raft_consensus.cc:2410] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 9 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:17:30.801234 14439 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.750010 14459 consensus_queue.cc:1243] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer 12682eb70bc84c76879e04a8628ee586 log is divergent from this leader: its last log entry 7.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:17:30.808923 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:30.654349 14384 consensus_meta.cc:287] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.565s	user 0.009s	sys 0.001s
I20260709 13:17:30.810527 10798 raft_consensus.cc:1240] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:30.810638 10766 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:17:30.814694 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:30.864928 14446 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:30.815799 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 67185us
I20260709 13:17:30.840016 10916 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:30.825666 10949 rpcz_store.cc:276] 0709 13:17:29.427056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:29.532486 (+105430us) service_pool.cc:224] Handling call
0709 13:17:30.661842 (+1129356us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b540039a154. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:30.663321 (+  1479us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2520187264}
I20260709 13:17:30.690168 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:30.723804 14421 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:30.691134 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:30.812077 14305 raft_consensus.cc:2749] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:30.853299 11511 raft_consensus.cc:1240] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:17:30.859148 10950 raft_consensus.cc:2393] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 9: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:30.799177 14331 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:30.863361 10943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 1759) took 1163 ms. Trace:
I20260709 13:17:30.865316 10841 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:30.911877 10943 rpcz_store.cc:276] 0709 13:17:29.699980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:30.252402 (+552422us) service_pool.cc:224] Handling call
0709 13:17:30.252798 (+   396us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:30.253246 (+   448us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:30.253729 (+   483us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:30.254312 (+   583us) log.cc:844] Serialized 26 byte log entry
0709 13:17:30.693752 (+439440us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:30.743789 (+ 50037us) log.cc:844] Serialized 14 byte log entry
0709 13:17:30.745879 (+  2090us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:30.745958 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:30.863168 (+117210us) raft_consensus.cc:1685] finished
0709 13:17:30.863187 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:30.863305 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":52124,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":5367,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":23866,"thread_start_us":394477,"threads_started":1,"wal-append.queue_time_us":542221}
I20260709 13:17:30.848912 14304 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:30.868924 10800 raft_consensus.cc:1240] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 11. Current term is 12. Ops: []
W20260709 13:17:30.867952 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 55142.7us
I20260709 13:17:30.920612 10798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 422) took 1331 ms. Trace:
I20260709 13:17:30.822738 14388 consensus_queue.cc:1048] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.040s
W20260709 13:17:30.921298 14529 log.cc:927] Time spent T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 13:17:30.922200 13719 raft_consensus.cc:3055] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:30.922608 13719 raft_consensus.cc:740] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:30.868391 14129 consensus_queue.cc:1048] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:17:30.825917 14407 consensus_queue.cc:1059] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:17:30.874711 10945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2169) took 1001 ms. Trace:
I20260709 13:17:30.879611 11504 raft_consensus.cc:1240] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:17:30.826283 14431 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:30.836023 14341 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:30.815084 10946 raft_consensus.cc:2393] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 11: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
W20260709 13:17:30.891094 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 80532.1us
I20260709 13:17:30.897344 10949 raft_consensus.cc:1240] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 7. Current term is 8. Ops: []
W20260709 13:17:30.897048 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 57160.1us
I20260709 13:17:30.854183 14480 consensus_queue.cc:1048] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:17:30.914194 10943 raft_consensus.cc:1240] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 11. Current term is 12. Ops: []
I20260709 13:17:30.912781 10950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2171) took 1017 ms. Trace:
I20260709 13:17:30.924111 11508 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 10.
I20260709 13:17:30.871444 14405 consensus_queue.cc:1059] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:17:30.849272 14396 raft_consensus.cc:2749] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:30.894414 14508 consensus_queue.cc:1048] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 13:17:30.890753 14439 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:30.890399 14404 consensus_queue.cc:1059] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 13:17:30.949105 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:30.945350 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:31.071851 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:31.201524 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:17:30.985260 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:31.102807 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:31.006304 10947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2166) took 1170 ms. Trace:
I20260709 13:17:30.906773 14391 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:31.073767 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:30.939865 14421 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:31.060972 14243 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:31.050652 14181 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.933552 10801 raft_consensus.cc:1240] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:31.121135 10941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2168) took 1264 ms. Trace:
I20260709 13:17:31.202471 10916 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:31.206378 10943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 1763) took 1299 ms. Trace:
I20260709 13:17:30.921021 10798 rpcz_store.cc:276] 0709 13:17:29.589155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:29.785473 (+196318us) service_pool.cc:224] Handling call
0709 13:17:29.856169 (+ 70696us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:30.889546 (+1033377us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:30.920390 (+ 30844us) spinlock_profiling.cc:243] Waited 954 ms on lock 0x7b54003ecbd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:30.920557 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2194432896}
I20260709 13:17:31.211179 10943 rpcz_store.cc:276] 0709 13:17:29.906547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:30.913721 (+1007174us) service_pool.cc:224] Handling call
0709 13:17:30.913998 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:31.206013 (+292015us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:31.206325 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:31.210812 10941 rpcz_store.cc:276] 0709 13:17:29.856973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:30.860311 (+1003338us) service_pool.cc:224] Handling call
0709 13:17:30.860608 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:30.860809 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:17:30.861094 (+   285us) log.cc:844] Serialized 14 byte log entry
0709 13:17:31.048835 (+187741us) log.cc:844] Serialized 14 byte log entry
0709 13:17:31.120652 (+ 71817us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:31.120729 (+    77us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:31.120794 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:31.120889 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:31.121058 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":71334,"prepare.queue_time_us":18454,"prepare.run_cpu_time_us":98,"prepare.run_wall_time_us":1792,"raft.queue_time_us":71587,"raft.run_cpu_time_us":236,"raft.run_wall_time_us":9541,"thread_start_us":64081,"threads_started":1}
I20260709 13:17:30.943511 11478 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:31.212661 10916 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:31.204577 10950 rpcz_store.cc:276] 0709 13:17:29.895397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:30.912020 (+1016623us) service_pool.cc:224] Handling call
0709 13:17:30.912299 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:30.912470 (+   171us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:30.912494 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:30.912534 (+    40us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:30.912570 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:30.912621 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:30.912729 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:31.217687 10916 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:30.906198 14446 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:31.202181 10949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2170) took 1319 ms. Trace:
I20260709 13:17:31.246759 10949 rpcz_store.cc:276] 0709 13:17:29.882413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:30.896882 (+1014469us) service_pool.cc:224] Handling call
0709 13:17:30.897154 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:31.201952 (+304798us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:31.202129 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:31.248214 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:31.195189 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:31.049865 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:31.203783 10947 rpcz_store.cc:276] 0709 13:17:29.835799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:30.715922 (+880123us) service_pool.cc:224] Handling call
0709 13:17:30.716218 (+   296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:30.716534 (+   316us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:30.716563 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:30.724938 (+  8375us) log.cc:844] Serialized 26 byte log entry
0709 13:17:30.905561 (+180623us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:30.905601 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:30.905675 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:31.005923 (+100248us) raft_consensus.cc:1685] finished
0709 13:17:31.005957 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:31.006208 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":95,"raft.run_cpu_time_us":4773,"raft.run_wall_time_us":24945,"thread_start_us":102934,"threads_started":1,"wal-append.queue_time_us":174721}
W20260709 13:17:30.925724 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 60483.1us
I20260709 13:17:31.213009 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:31.213663 10870 raft_consensus.cc:1275] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:17:31.199021 10945 rpcz_store.cc:276] 0709 13:17:29.873690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:30.873908 (+1000218us) service_pool.cc:224] Handling call
0709 13:17:30.874227 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:30.874406 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:30.874434 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:30.874476 (+    42us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:30.874507 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:30.874557 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:30.874658 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:31.064073 14532 log.cc:927] Time spent T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.171s	user 0.000s	sys 0.000s
I20260709 13:17:30.937114 14479 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:17:31.238210 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 294804us
I20260709 13:17:30.918020 14345 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:17:31.252070 10943 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 10.
I20260709 13:17:31.330080 10943 raft_consensus.cc:1240] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:31.330937 10943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 541) took 1297 ms. Trace:
I20260709 13:17:31.331342 10943 rpcz_store.cc:276] 0709 13:17:30.033046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:31.329425 (+1296379us) service_pool.cc:224] Handling call
0709 13:17:31.329836 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:31.330605 (+   769us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:31.330867 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:31.222297 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:31.253722 10871 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 9: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:31.139483 14540 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:31.194841 14506 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:30.930578 14144 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:31.258077 10948 raft_consensus.cc:1240] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:31.258281 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:31.113685 14331 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:31.258251 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:31.259718 10841 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:30.938498 14361 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:31.253331 14425 raft_consensus.cc:2749] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:31.176942 14409 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:31.060253 14217 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:30.937292 14403 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:31.336566 10943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 542) took 1303 ms. Trace:
I20260709 13:17:31.208148 14391 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:31.050403 14360 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:30.938120 14311 consensus_queue.cc:260] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:30.935711 14305 consensus_queue.cc:1048] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.341s
I20260709 13:17:31.354596 14431 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:30.931823 14386 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:31.365451 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 60737.8us
I20260709 13:17:31.549898 11478 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:31.206203 14314 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:31.510349 14391 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:31.550942 14314 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:30.894403 14384 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:31.406414 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:31.220410 14378 raft_consensus.cc:2804] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:31.497807 14409 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:31.555239 14378 raft_consensus.cc:697] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 10 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:31.556165 14378 consensus_queue.cc:237] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:31.422883 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:31.559290 14542 log.cc:927] Time spent T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.342s	user 0.000s	sys 0.001s
W20260709 13:17:31.443009 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:30.935415 14406 consensus_queue.cc:1059] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.364s
I20260709 13:17:31.456004 10947 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 9: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:31.578125 10947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 1765) took 1521 ms. Trace:
I20260709 13:17:31.254868 14509 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:31.236624 14367 raft_consensus.cc:2804] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:31.068238 13719 consensus_queue.cc:260] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:31.059713 14362 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:31.598215 14367 raft_consensus.cc:697] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 8 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:31.599597 14367 consensus_queue.cc:237] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:31.215958 14181 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:31.608798 14181 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:31.612967 14304 consensus_meta.cc:287] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.687s	user 0.009s	sys 0.000s
I20260709 13:17:31.485281 10949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 543) took 1451 ms. Trace:
I20260709 13:17:31.498160 10943 rpcz_store.cc:276] 0709 13:17:30.033308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:31.335576 (+1302268us) service_pool.cc:224] Handling call
0709 13:17:31.335935 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:31.336156 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:31.336192 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:31.336248 (+    56us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:31.336303 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:31.336371 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:31.336500 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:31.650355 10949 rpcz_store.cc:276] 0709 13:17:30.033562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:31.484170 (+1450608us) service_pool.cc:224] Handling call
0709 13:17:31.484585 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:31.484835 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:31.484873 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:31.484930 (+    57us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:31.484981 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:31.485051 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:31.485204 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:31.437697 14275 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.925190 14342 consensus_queue.cc:260] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:31.254638 14505 consensus_queue.cc:1059] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.214s
I20260709 13:17:31.688669 14505 raft_consensus.cc:3055] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:31.689079 14505 raft_consensus.cc:740] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:31.689895 14505 consensus_queue.cc:260] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:31.691262 10800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2228) took 1494 ms. Trace:
I20260709 13:17:31.691689 10800 rpcz_store.cc:276] 0709 13:17:30.197262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:30.712979 (+515717us) service_pool.cc:224] Handling call
0709 13:17:30.713267 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:30.919642 (+206375us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:31.690988 (+771346us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b54002111d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:31.691199 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":357575040}
I20260709 13:17:31.208985 14243 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:31.693614 14406 raft_consensus.cc:3055] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:17:31.694337 14406 raft_consensus.cc:740] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:31.516086 10942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2165) took 1730 ms. Trace:
I20260709 13:17:31.524928 14311 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:31.706046 10795 raft_consensus.cc:1275] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 9. (term mismatch)
I20260709 13:17:31.695086 10942 rpcz_store.cc:276] 0709 13:17:29.786198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:30.587283 (+801085us) service_pool.cc:224] Handling call
0709 13:17:30.587606 (+   323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:30.588321 (+   715us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:30.588355 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:30.655436 (+ 67081us) log.cc:844] Serialized 26 byte log entry
0709 13:17:30.793700 (+138264us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:30.793771 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:30.793866 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:31.515801 (+721935us) raft_consensus.cc:1685] finished
0709 13:17:31.515827 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:31.516010 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":94,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":82,"thread_start_us":119491,"threads_started":1,"wal-append.queue_time_us":172577}
I20260709 13:17:31.706580 10795 pending_rounds.cc:85] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 8
I20260709 13:17:31.706943 10795 pending_rounds.cc:107] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 7.9
I20260709 13:17:31.707248 10795 raft_consensus.cc:2889] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:31.540877 14305 consensus_queue.cc:1243] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer 468a8fd46e824f62b9c2c0d3532a6d26 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:17:31.548796 14431 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:31.431012 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:31.578665 10947 rpcz_store.cc:276] 0709 13:17:30.057088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:31.577122 (+1520034us) service_pool.cc:224] Handling call
0709 13:17:31.577505 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:31.577736 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:31.577776 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:31.577829 (+    53us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:31.577876 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:31.577936 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:31.578052 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:31.525352 14440 consensus_queue.cc:1175] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" responder_term: 9 status { last_received { term: 6 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 8. Current term is 9. Ops: [8.7-8.7]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:17:31.525938 14360 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:31.463229 14506 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:17:31.434746 14540 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:31.516557 14144 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:17:31.587397 14509 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:31.560884 14513 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.023s	sys 0.079s
I20260709 13:17:31.434291 14333 consensus_queue.cc:260] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:31.275929 14298 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:31.677448 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 127636us
I20260709 13:17:31.651950 10943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 545) took 1511 ms. Trace:
I20260709 13:17:31.685604 10946 raft_consensus.cc:1275] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:17:31.687358 14384 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:31.693617 14342 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:31.665280 14304 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:31.804700 14360 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:31.460055 14479 raft_consensus.cc:697] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:31.805418 10792 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 9.
W20260709 13:17:31.697396 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:31.807245 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:31.762387 14513 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.820s	user 0.024s	sys 0.079s
I20260709 13:17:31.697921 11504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56416 (request call id 625) took 1422 ms. Trace:
I20260709 13:17:31.709822 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:31.808497 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:31.809257 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:31.809656 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:31.809909 10800 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 8.
I20260709 13:17:31.810415 10792 raft_consensus.cc:2393] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 8: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:31.810580 14506 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:31.790509 14144 raft_consensus.cc:697] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:31.711548 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:31.812001 14144 consensus_queue.cc:237] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.7, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:31.812851 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:31.812923 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:31.814093 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:31.814203 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:31.669189 14433 consensus_meta.cc:287] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.920s	user 0.008s	sys 0.004s
I20260709 13:17:31.815347 14439 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:31.706195 14504 raft_consensus.cc:2749] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:31.815539 14433 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:31.715983 10799 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39442 (request call id 420) took 2122 ms (client timeout 1801 ms). Trace:
I20260709 13:17:31.771003 10943 rpcz_store.cc:276] 0709 13:17:30.141054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:31.650811 (+1509757us) service_pool.cc:224] Handling call
0709 13:17:31.651238 (+   427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:31.651486 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:31.651524 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:31.651584 (+    60us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:17:31.651638 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:31.651712 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:31.651875 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:31.777196 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:31.772404 10946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 544) took 1738 ms. Trace:
I20260709 13:17:31.817209 10946 rpcz_store.cc:276] 0709 13:17:30.033852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:31.553398 (+1519546us) service_pool.cc:224] Handling call
0709 13:17:31.669688 (+116290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:31.772043 (+102355us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:31.772334 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:31.817597 10943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 1769) took 1211 ms. Trace:
I20260709 13:17:31.781477 11510 raft_consensus.cc:1275] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 10 LEARNER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:17:31.782835 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:31.817975 10943 rpcz_store.cc:276] 0709 13:17:30.605842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:31.816825 (+1210983us) service_pool.cc:224] Handling call
0709 13:17:31.817115 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:31.817299 (+   184us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:31.817325 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:31.817365 (+    40us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:31.817394 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:31.817444 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:31.817541 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:31.783799 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:17:31.779402 10873 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.685s	user 0.006s	sys 0.010s
I20260709 13:17:31.791406 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:31.464745 14435 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:31.795293 14384 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:31.796999 14503 raft_consensus.cc:3055] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:31.799110 14505 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:31.802103 14403 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:31.696959 14275 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:31.762318 14509 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:17:31.807780 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:31.784639 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:31.819288 11505 raft_consensus.cc:2393] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 11: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:31.819379 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:31.808410 11504 rpcz_store.cc:276] 0709 13:17:30.275336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:30.275458 (+   122us) service_pool.cc:224] Handling call
0709 13:17:30.295794 (+ 20336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:31.203204 (+907410us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:31.697672 (+494468us) spinlock_profiling.cc:243] Waited 579 ms on lock 0x7b54005802d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:31.697860 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1331826432}
I20260709 13:17:31.786028 14549 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:31.818945 10872 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 10.
W20260709 13:17:31.816547 10799 rpcz_store.cc:269] 0709 13:17:29.574401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:29.588755 (+ 14354us) service_pool.cc:224] Handling call
0709 13:17:31.696323 (+2107568us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b5400387a54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:31.696828 (+   505us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2717881216}
W20260709 13:17:31.824193 10873 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.730s	user 0.007s	sys 0.010s
I20260709 13:17:31.822568 14409 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:31.733134 14431 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:31.806427 14479 consensus_queue.cc:237] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:32.032419 10841 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.015s	sys 0.050s
W20260709 13:17:32.064734 10841 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.021s	sys 0.050s
I20260709 13:17:31.899552 10943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2175) took 1273 ms. Trace:
W20260709 13:17:31.996832 10794 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.928s	user 0.011s	sys 0.012s
W20260709 13:17:32.065634 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 805930us
W20260709 13:17:31.896952 14412 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.024s	sys 0.029s
W20260709 13:17:32.059109 10915 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.022s	sys 0.043s
W20260709 13:17:32.067356 14412 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.811s	user 0.026s	sys 0.035s
W20260709 13:17:32.067451 10915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.023s	sys 0.043s
I20260709 13:17:32.060312 10793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2239) took 1604 ms. Trace:
W20260709 13:17:31.996589 10914 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.010s	sys 0.013s
W20260709 13:17:32.059998 10875 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.017s	sys 0.047s
W20260709 13:17:32.069164 10875 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.018s	sys 0.047s
W20260709 13:17:31.981694 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:32.069561 14384 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:32.070166 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:32.070652 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:31.938452 14298 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:32.071859 14431 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:31.825843 10946 raft_consensus.cc:1275] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:17:31.826225 14341 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:32.069002 10914 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.704s	user 0.010s	sys 0.015s
I20260709 13:17:32.063596 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:31.953227 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:32.076998 10870 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:32.073874 10841 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:32.073366 10946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2174) took 1448 ms. Trace:
I20260709 13:17:32.065585 10943 rpcz_store.cc:276] 0709 13:17:30.625643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:31.818890 (+1193247us) service_pool.cc:224] Handling call
0709 13:17:31.898846 (+ 79956us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:31.899090 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:31.899126 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:31.899184 (+    58us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:31.899255 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:31.899325 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:31.899481 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:32.078944 10946 rpcz_store.cc:276] 0709 13:17:30.625338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:31.817969 (+1192631us) service_pool.cc:224] Handling call
0709 13:17:31.822763 (+  4794us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:32.073086 (+250323us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.073298 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:31.761929 14243 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:32.082559 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:32.072042 14384 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:32.078552 14345 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.011s	sys 0.015s
I20260709 13:17:32.063977 14503 raft_consensus.cc:740] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:31.897960 14547 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:31.897688 14305 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:31.898403 14314 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:32.083721 14341 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:31.852515 14388 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:32.098284 11506 raft_consensus.cc:2410] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
W20260709 13:17:32.092859 14345 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.011s	sys 0.016s
I20260709 13:17:31.854148 14504 consensus_queue.cc:1048] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.238s
I20260709 13:17:32.072098 14298 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:32.087446 14551 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:31.773737 14406 consensus_queue.cc:260] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:32.084724 14403 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:32.100234 14433 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:32.101087 14412 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:32.103240 14388 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:32.118849 14390 raft_consensus.cc:3055] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:17:32.119304 14390 raft_consensus.cc:740] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:32.097900 14305 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:32.122143 14305 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:32.124102 14540 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:32.128861 14403 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:32.129563 14406 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:32.119216 14392 raft_consensus.cc:3055] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:17:32.134619 14391 consensus_meta.cc:287] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.583s	user 0.006s	sys 0.002s
I20260709 13:17:32.135002 14392 raft_consensus.cc:740] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:17:32.136600 14538 log.cc:927] Time spent T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 1.036s	user 0.000s	sys 0.012s
I20260709 13:17:32.137478 14388 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:32.139242 14392 consensus_queue.cc:260] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:32.140108 14511 raft_consensus.cc:3055] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:32.139994 14412 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:32.140656 14533 log.cc:927] Time spent T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 1.243s	user 0.011s	sys 0.000s
I20260709 13:17:32.145274 14511 raft_consensus.cc:740] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:32.145545 14391 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:32.145367 14392 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:32.148941 14412 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:32.144824 14446 consensus_meta.cc:287] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.828s	user 0.010s	sys 0.000s
I20260709 13:17:32.149658 14509 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:32.151897 14509 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:32.180737  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:31.852065 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:17:32.065930 10794 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.013s	user 0.013s	sys 0.013s
I20260709 13:17:32.191296 10792 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 9.
I20260709 13:17:32.193341 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:32.193545 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:32.194447 10798 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:32.194697 10792 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 10.
I20260709 13:17:32.208316 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:32.204443 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:32.068470 10793 rpcz_store.cc:276] 0709 13:17:30.456117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:30.887228 (+431111us) service_pool.cc:224] Handling call
0709 13:17:30.892276 (+  5048us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:30.892745 (+   469us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:30.892788 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:30.913594 (+ 20806us) log.cc:844] Serialized 26 byte log entry
0709 13:17:31.496361 (+582767us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:31.496420 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:31.496521 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:32.060052 (+563531us) raft_consensus.cc:1685] finished
0709 13:17:32.060073 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:32.060233 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":113,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":13453,"thread_start_us":428639,"threads_started":1,"wal-append.queue_time_us":863071}
I20260709 13:17:32.074720 10800 raft_consensus.cc:2393] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 10: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
W20260709 13:17:31.839570 10766 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.018s	sys 0.037s
W20260709 13:17:32.210062 10766 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.006s	user 0.024s	sys 0.042s
I20260709 13:17:32.210597 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:32.211076 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:32.212177 10800 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 11.
I20260709 13:17:32.212534 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:32.213441 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:32.213553 10793 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 8.
I20260709 13:17:32.206745 10801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2211) took 2678 ms. Trace:
I20260709 13:17:32.214911 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:32.215026 10801 rpcz_store.cc:276] 0709 13:17:29.527802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:29.677628 (+149826us) service_pool.cc:224] Handling call
0709 13:17:29.677919 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:31.231416 (+1553497us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.205993 (+974577us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b5400386154. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:32.206473 (+   480us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2887321856}
I20260709 13:17:32.216044 10792 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 10.
I20260709 13:17:32.217039 10801 raft_consensus.cc:1240] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:32.217902 10801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2243) took 1666 ms. Trace:
I20260709 13:17:32.218191 10792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2245) took 1385 ms. Trace:
I20260709 13:17:32.219470 10800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2242) took 1674 ms. Trace:
I20260709 13:17:32.221319 10795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2214) took 2653 ms. Trace:
I20260709 13:17:32.197211 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:32.226325 10795 rpcz_store.cc:276] 0709 13:17:29.541501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:29.677829 (+136328us) service_pool.cc:224] Handling call
0709 13:17:31.696442 (+2018613us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:31.707700 (+ 11258us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.194401 (+486701us) spinlock_profiling.cc:243] Waited 2.02 s on lock 0x7b5400387a50. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:32.194689 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4641805056}
I20260709 13:17:32.227814 10796 raft_consensus.cc:1240] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:32.228333 10795 raft_consensus.cc:1275] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 10 index: 4. (index mismatch)
I20260709 13:17:32.228542 10796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2246) took 1380 ms. Trace:
I20260709 13:17:32.228987 10795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2247) took 1379 ms. Trace:
I20260709 13:17:32.228940 10796 rpcz_store.cc:276] 0709 13:17:30.847850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.226990 (+1379140us) service_pool.cc:224] Handling call
0709 13:17:32.227495 (+   505us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:32.228305 (+   810us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.228492 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:32.229493 10795 rpcz_store.cc:276] 0709 13:17:30.849315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.227623 (+1378308us) service_pool.cc:224] Handling call
0709 13:17:32.227948 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:32.228769 (+   821us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.228941 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:32.231034 10796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2248) took 1342 ms. Trace:
I20260709 13:17:32.231521 10796 rpcz_store.cc:276] 0709 13:17:30.888456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.229881 (+1341425us) service_pool.cc:224] Handling call
0709 13:17:32.230294 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:32.230638 (+   344us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:32.230663 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:32.230730 (+    67us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:32.230775 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.230858 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:32.230969 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:32.201156 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:32.241132 10797 raft_consensus.cc:2410] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 9 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:17:32.241866 10797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39442 (request call id 431) took 1307 ms. Trace:
I20260709 13:17:32.242379 10797 rpcz_store.cc:276] 0709 13:17:30.974754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:31.707563 (+732809us) service_pool.cc:224] Handling call
0709 13:17:32.200732 (+493169us) spinlock_profiling.cc:243] Waited 2.08 ms on lock 0x7b5800002798. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd90a22 00007f9c86ca9c09 00007f9c85e03ee0 00007f9c85ecd0ab 00007f9c85ecd131 00007f9c85e7b0a0 00007f9c85e7a984 00007f9c7f3f56d4 00007f9c7f3f5662
0709 13:17:32.241757 (+ 41025us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4789120}
I20260709 13:17:32.244921 10797 pending_rounds.cc:85] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 5
I20260709 13:17:32.245412 10797 pending_rounds.cc:107] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 13:17:32.245859 10797 raft_consensus.cc:2889] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:32.232148 10800 rpcz_store.cc:276] 0709 13:17:30.544557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.214662 (+1670105us) service_pool.cc:224] Handling call
0709 13:17:32.218627 (+  3965us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:32.219076 (+   449us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:32.219105 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:32.219167 (+    62us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:32.219209 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.219288 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:32.219392 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:32.235952 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:32.218251 10801 rpcz_store.cc:276] 0709 13:17:30.551067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.216209 (+1665142us) service_pool.cc:224] Handling call
0709 13:17:32.216742 (+   533us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:32.217570 (+   828us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.217822 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:32.247505 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:32.248533 10801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 430) took 1314 ms. Trace:
I20260709 13:17:32.248775 10800 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546"
dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
 from {username='slave'} at 127.0.0.1:39442
I20260709 13:17:32.249032 10801 rpcz_store.cc:276] 0709 13:17:30.934465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.247391 (+1312926us) service_pool.cc:224] Handling call
0709 13:17:32.247795 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:32.248146 (+   351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:32.248168 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:32.248252 (+    84us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:32.248296 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.248346 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:32.248465 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:32.198132 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:32.250921 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:32.251782 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:32.252630 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:32.253132 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 185847us
I20260709 13:17:32.255354 10841 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:32.259171 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:32.259313 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:32.260501 10868 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 10.
I20260709 13:17:32.261119 10766 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:32.261514 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:32.261837 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:32.268865 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:32.249523 10800 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:32.265290 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:32.271436 10875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1740) took 1320 ms. Trace:
I20260709 13:17:32.266294 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 490) took 2026 ms. Trace:
I20260709 13:17:32.272266 10869 rpcz_store.cc:276] 0709 13:17:30.240119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:30.240366 (+   247us) service_pool.cc:224] Handling call
0709 13:17:30.240722 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:30.874058 (+633336us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.265647 (+1391589us) spinlock_profiling.cc:243] Waited 633 ms on lock 0x7b540006b8d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:32.266057 (+   410us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1455088000}
I20260709 13:17:32.271853 10875 rpcz_store.cc:276] 0709 13:17:30.951418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:30.952432 (+  1014us) service_pool.cc:224] Handling call
0709 13:17:30.952870 (+   438us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:31.111132 (+158262us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:17:31.357584 (+246452us) log.cc:844] Serialized 14 byte log entry
0709 13:17:32.070533 (+712949us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:32.071306 (+   773us) log.cc:844] Serialized 26 byte log entry
0709 13:17:32.071440 (+   134us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:32.071474 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.071541 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:32.270995 (+199454us) raft_consensus.cc:1685] finished
0709 13:17:32.271031 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:32.271265 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":695722,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":216045,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":21725,"thread_start_us":788966,"threads_started":2,"wal-append.queue_time_us":672123}
I20260709 13:17:32.276628 10875 raft_consensus.cc:1275] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:17:32.270569 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1746) took 1053 ms. Trace:
I20260709 13:17:32.278189 10867 rpcz_store.cc:276] 0709 13:17:31.217186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.269100 (+1051914us) service_pool.cc:224] Handling call
0709 13:17:32.269594 (+   494us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:32.270016 (+   422us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:32.270054 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:32.270130 (+    76us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:32.270173 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.270239 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:32.270360 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:32.263257 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:32.283221 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:32.270998 10868 raft_consensus.cc:1240] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 10. Current term is 11. Ops: [10.11-10.11]
I20260709 13:17:32.271045 10800 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:32.287467 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:32.289072 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:32.292531 10943 raft_consensus.cc:1240] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:32.293525 10943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2176) took 1667 ms. Trace:
I20260709 13:17:32.293522 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:32.294214 10943 rpcz_store.cc:276] 0709 13:17:30.625964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.079919 (+1453955us) service_pool.cc:224] Handling call
0709 13:17:32.291923 (+212004us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:32.293137 (+  1214us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.293407 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:32.295680 10942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 1766) took 1771 ms. Trace:
I20260709 13:17:32.290454 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:32.295863 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:32.296100 10944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 1757) took 2596 ms. Trace:
I20260709 13:17:32.296885 10943 raft_consensus.cc:2393] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 7: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:32.297139 10868 pending_rounds.cc:85] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 2
I20260709 13:17:32.297503 10868 pending_rounds.cc:107] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 13:17:32.090111 10946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2177) took 1463 ms. Trace:
I20260709 13:17:32.298609 10868 raft_consensus.cc:2889] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:32.298245 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:32.219024 10792 rpcz_store.cc:276] 0709 13:17:30.832357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.216844 (+1384487us) service_pool.cc:224] Handling call
0709 13:17:32.217359 (+   515us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:32.217803 (+   444us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:32.217831 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:32.217904 (+    73us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:32.217951 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.218013 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:32.218121 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:32.299191 10946 rpcz_store.cc:276] 0709 13:17:30.626275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.080398 (+1454123us) service_pool.cc:224] Handling call
0709 13:17:32.089586 (+  9188us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:32.089779 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:32.089809 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:32.089846 (+    37us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:32.089882 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.089937 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:32.090049 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:32.296624 10944 rpcz_store.cc:276] 0709 13:17:29.699387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:30.248550 (+549163us) service_pool.cc:224] Handling call
0709 13:17:30.662063 (+413513us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:30.825673 (+163610us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.295688 (+1470015us) spinlock_profiling.cc:243] Waited 412 ms on lock 0x7b540039a150. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:32.295923 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":948640640}
I20260709 13:17:32.358465 14480 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:32.359530 14480 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:32.361105 14300 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:32.361274 10798 raft_consensus.cc:2410] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:17:32.361618 14300 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:32.362025 14300 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:32.362082 10798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39442 (request call id 433) took 1288 ms. Trace:
I20260709 13:17:32.362360 14480 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:32.362524 10798 rpcz_store.cc:276] 0709 13:17:31.074166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.192357 (+1118191us) service_pool.cc:224] Handling call
0709 13:17:32.362004 (+169647us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:32.362987 14217 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:32.363821 14342 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:32.296161 10942 rpcz_store.cc:276] 0709 13:17:30.524657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:31.707275 (+1182618us) service_pool.cc:224] Handling call
0709 13:17:31.707718 (+   443us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:31.708174 (+   456us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:31.708215 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:31.708783 (+   568us) log.cc:844] Serialized 26 byte log entry
0709 13:17:32.035750 (+326967us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:32.035811 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.035908 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:32.295087 (+259179us) raft_consensus.cc:1685] finished
0709 13:17:32.295128 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:32.295466 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1030,"raft.queue_time_us":3247,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":12897,"thread_start_us":326568,"threads_started":1,"wal-append.queue_time_us":418140}
I20260709 13:17:32.101578 14345 raft_consensus.cc:697] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 11 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:32.367264 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:32.371721 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:32.366506 14506 consensus_queue.cc:1048] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:32.120477 14390 consensus_queue.cc:260] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:32.149852 14446 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:32.401364 10800 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:32.142872 14503 consensus_queue.cc:260] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:32.146195 14511 consensus_queue.cc:260] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:32.096761 14547 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:32.425073 13719 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:32.425522 14547 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:32.114379 14551 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:32.082413 14549 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:32.379221 14412 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:32.380388 14407 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:32.380779 14405 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:32.381305 14409 consensus_queue.cc:1059] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:32.381784 10870 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 9.
I20260709 13:17:32.382341 14440 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:32.382344 14362 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:32.380400 10947 raft_consensus.cc:1240] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 10 LEADER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:32.382381 14386 raft_consensus.cc:2749] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:17:32.384613 14298 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:32.384820 14393 consensus_queue.cc:1048] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:17:32.385378 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 124568us
I20260709 13:17:32.385983 10949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 547) took 1882 ms. Trace:
I20260709 13:17:32.391271 14435 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:32.388453 14391 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:32.389290 11478 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.116.129:39685 timed out after 1.571s (SENT) [suppressed 9 similar messages]
I20260709 13:17:32.402320 14342 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:32.402567 14217 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:32.403841 14345 consensus_queue.cc:237] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:32.404389 14497 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:32.405575 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:32.405853 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:32.406697 14043 raft_consensus.cc:2955] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Committing config change with OpId 3.4: config changed from index 2 to 4, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
W20260709 13:17:32.377120 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 77235.2us
I20260709 13:17:32.421801 11511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56410 (request call id 632) took 1740 ms. Trace:
W20260709 13:17:32.460889 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:32.444464 10800 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:32.387611 14275 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:32.456581 10766 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:32.456377 14406 consensus_queue.cc:1059] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 13:17:32.458053 10874 raft_consensus.cc:1240] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:32.458529 14333 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:32.460788 14446 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:32.460886 10948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 539) took 2900 ms. Trace:
I20260709 13:17:32.462378 11506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56410 (request call id 633) took 1780 ms. Trace:
I20260709 13:17:32.462581 14440 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:32.463358 14549 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:32.463624 10793 raft_consensus.cc:2393] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 10: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:32.464605 11504 raft_consensus.cc:1240] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 8 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:32.466056 14551 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:32.467041 14407 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:32.468186 14304 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:32.468863 14342 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:32.471145 11511 rpcz_store.cc:276] 0709 13:17:30.681156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:30.684501 (+  3345us) service_pool.cc:224] Handling call
0709 13:17:30.684921 (+   420us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:30.911353 (+226432us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.421176 (+1509823us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b54008b79d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:32.421552 (+   376us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":386680064}
I20260709 13:17:32.484402 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:32.483945 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 202893us
I20260709 13:17:32.486006 10800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:39442 (request call id 432) took 1497 ms. Trace:
I20260709 13:17:32.478587 11506 rpcz_store.cc:276] 0709 13:17:30.681555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:30.690876 (+  9321us) service_pool.cc:224] Handling call
0709 13:17:32.461879 (+1771003us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54007e1b54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:32.462130 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2696568704}
I20260709 13:17:32.490891 10800 rpcz_store.cc:276] 0709 13:17:31.073720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.248532 (+1174812us) service_pool.cc:224] Handling call
0709 13:17:32.472714 (+224182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:32.478088 10948 rpcz_store.cc:276] 0709 13:17:29.559978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:29.905536 (+345558us) service_pool.cc:224] Handling call
0709 13:17:29.920420 (+ 14884us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:31.432628 (+1512208us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.460335 (+1027707us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b540039ab54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:32.460664 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3074609920}
I20260709 13:17:32.451753 10949 rpcz_store.cc:276] 0709 13:17:30.503251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:31.669139 (+1165888us) service_pool.cc:224] Handling call
0709 13:17:31.669571 (+   432us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:31.670191 (+   620us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:31.670233 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:31.671084 (+   851us) log.cc:844] Serialized 26 byte log entry
0709 13:17:32.033722 (+362638us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:17:32.033785 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.033888 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:32.354459 (+320571us) raft_consensus.cc:1685] finished
0709 13:17:32.370866 (+ 16407us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:32.385805 (+ 14939us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":18333,"raft.run_cpu_time_us":54,"raft.run_wall_time_us":55,"thread_start_us":362208,"threads_started":1,"wal-append.queue_time_us":460317}
I20260709 13:17:32.502750 10943 raft_consensus.cc:2410] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 11 because replica has last-logged OpId of term: 9 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 3.
I20260709 13:17:32.503269 10949 raft_consensus.cc:1275] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:17:32.503705 14217 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:32.503932 10949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2180) took 1876 ms. Trace:
I20260709 13:17:32.504712 10943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2181) took 1877 ms. Trace:
I20260709 13:17:32.460806 14412 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:32.508589 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:32.509694 14032 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:32.510612 14550 log.cc:927] Time spent T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:17:32.510432 14032 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:32.510937 14547 raft_consensus.cc:3055] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:17:32.511484 14547 raft_consensus.cc:740] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:32.512486 14497 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:32.513226 14032 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:32.508600 14391 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:17:32.516142 10948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2178) took 1889 ms. Trace:
I20260709 13:17:32.516726 10948 rpcz_store.cc:276] 0709 13:17:30.626587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.514698 (+1888111us) service_pool.cc:224] Handling call
0709 13:17:32.515280 (+   582us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:32.515720 (+   440us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:32.515749 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:32.515815 (+    66us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:32.515861 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.515935 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:32.516064 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:32.518222 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:17:32.516168 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:32.519469 14547 consensus_queue.cc:260] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:32.520776 10916 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:32.525036 14412 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:32.504417 10949 rpcz_store.cc:276] 0709 13:17:30.627219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.502192 (+1874973us) service_pool.cc:224] Handling call
0709 13:17:32.502758 (+   566us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:32.503660 (+   902us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.503851 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:32.531314 10942 raft_consensus.cc:2393] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 9: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:32.548113 14341 raft_consensus.cc:2749] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:32.505118 10943 rpcz_store.cc:276] 0709 13:17:30.627526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.503567 (+1876041us) service_pool.cc:224] Handling call
0709 13:17:32.504001 (+   434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:32.504399 (+   398us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:32.504422 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:32.504482 (+    60us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:32.504522 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.504572 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:32.504661 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:32.537506 10946 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 8.
I20260709 13:17:32.537637 14497 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:32.539889 14440 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:32.541083 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1741) took 1586 ms. Trace:
I20260709 13:17:32.543792 14407 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:32.457443 14405 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:32.544263 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:32.493086 14435 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:32.545033 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:32.548415 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:32.451453 14362 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:32.594274 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 78458.4us
W20260709 13:17:32.598160 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:32.602883 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:32.606580 14435 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:32.607757 11477 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:32.610977 14499 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:32.611847 14499 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:32.613766 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 93140us
I20260709 13:17:32.617285 10948 raft_consensus.cc:1240] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:17:32.618151 10948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 549) took 1926 ms. Trace:
I20260709 13:17:32.618522 10948 rpcz_store.cc:276] 0709 13:17:30.691575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.616257 (+1924682us) service_pool.cc:224] Handling call
0709 13:17:32.616825 (+   568us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:32.617837 (+  1012us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.618079 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:32.620544 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:32.622865 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 197444us
I20260709 13:17:32.625412 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:32.617352 14568 log.cc:927] Time spent T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.111s	user 0.002s	sys 0.000s
I20260709 13:17:32.580888 10873 rpcz_store.cc:276] 0709 13:17:30.954334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:31.049904 (+ 95570us) service_pool.cc:224] Handling call
0709 13:17:31.071877 (+ 21973us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:31.072136 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:17:32.072869 (+1000733us) log.cc:844] Serialized 14 byte log entry
0709 13:17:32.540044 (+467175us) log.cc:844] Serialized 14 byte log entry
0709 13:17:32.540270 (+   226us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:32.540341 (+    71us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:32.540561 (+   220us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.540723 (+   162us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:32.540914 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":679966,"prepare.run_cpu_time_us":133,"prepare.run_wall_time_us":13847,"raft.queue_time_us":2150,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":173,"spinlock_wait_cycles":58252416,"thread_start_us":1444768,"threads_started":2,"wal-append.queue_time_us":433350}
I20260709 13:17:32.584120 14551 raft_consensus.cc:2749] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:17:32.622658 14547 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:32.624559 10841 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:32.606376 14300 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:32.618567 14392 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:32.641878 10944 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:32.619467 14362 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:32.645567 10949 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 9.
I20260709 13:17:32.645622 10916 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:32.655763 10794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 425) took 2330 ms. Trace:
W20260709 13:17:32.657114 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:32.656227 10794 rpcz_store.cc:276] 0709 13:17:30.308879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:30.875645 (+566766us) service_pool.cc:224] Handling call
0709 13:17:30.896390 (+ 20745us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:30.896884 (+   494us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:30.896924 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:30.897628 (+   704us) log.cc:844] Serialized 52 byte log entry
0709 13:17:32.191379 (+1293751us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:32.490192 (+298813us) log.cc:844] Serialized 14 byte log entry
0709 13:17:32.563112 (+ 72920us) log.cc:844] Serialized 14 byte log entry
0709 13:17:32.597111 (+ 33999us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.627695 (+ 30584us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:32.636388 (+  8693us) raft_consensus.cc:1685] finished
0709 13:17:32.636410 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:32.638846 (+  2436us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":97184,"prepare.queue_time_us":485862,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":65,"raft.queue_time_us":106076,"raft.run_cpu_time_us":471,"raft.run_wall_time_us":45071,"thread_start_us":1433415,"threads_started":2,"wal-append.queue_time_us":1189316}
I20260709 13:17:32.658169 14390 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:32.663030 10943 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 9.
I20260709 13:17:32.664682 10943 raft_consensus.cc:1240] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:17:32.670486 10941 raft_consensus.cc:1275] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:17:32.675061 14402 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:17:32.692342 10950 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 10.
W20260709 13:17:32.699723 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 79530.2us
W20260709 13:17:32.702652 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:32.709586 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 62149us
W20260709 13:17:32.704888 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 315785us
I20260709 13:17:32.714123 10942 raft_consensus.cc:1275] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 10 index: 4. (term mismatch)
I20260709 13:17:32.714601 10942 pending_rounds.cc:85] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 3
I20260709 13:17:32.714860 10942 pending_rounds.cc:107] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 9.4
I20260709 13:17:32.715197 10942 raft_consensus.cc:2889] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:17:32.716964 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:32.756672 10948 raft_consensus.cc:1240] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 10. Current term is 11. Ops: []
W20260709 13:17:32.762230 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 69975.4us
I20260709 13:17:32.762904 11477 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:17:32.765432 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:32.766817 10948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 550) took 1842 ms. Trace:
I20260709 13:17:32.767889 10948 rpcz_store.cc:276] 0709 13:17:30.924792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.636641 (+1711849us) service_pool.cc:224] Handling call
0709 13:17:32.692209 (+ 55568us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:32.761696 (+ 69487us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.766732 (+  5036us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:32.769976 14549 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:32.770629 14549 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:32.771188 14549 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:17:32.772368 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 223162us
I20260709 13:17:32.774020 10841 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:32.779359 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:32.779990 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:32.736658 14341 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:32.706893 14506 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:32.717618 14333 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:32.705019 14392 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:32.812880 10942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2185) took 2153 ms. Trace:
I20260709 13:17:32.809408 10949 raft_consensus.cc:3055] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:32.814910 10949 raft_consensus.cc:740] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:17:32.808339 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 45722.1us
I20260709 13:17:32.713940 14217 consensus_queue.cc:1059] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.094s
I20260709 13:17:32.823252 10942 rpcz_store.cc:276] 0709 13:17:30.659856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.591638 (+1931782us) service_pool.cc:224] Handling call
0709 13:17:32.645195 (+ 53557us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:32.807395 (+162200us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.811576 (+  4181us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:32.833055 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:32.832610 14570 consensus_queue.cc:1059] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 13:17:32.834945 10876 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 8.
I20260709 13:17:32.833421 10945 raft_consensus.cc:1240] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 8 LEADER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 7. Current term is 8. Ops: []
W20260709 13:17:32.840845 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 133548us
I20260709 13:17:32.824080 14300 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:32.847275 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:32.833781 14435 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:32.832723 14573 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:32.837996 14386 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:17:32.876408 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 102590us
I20260709 13:17:32.840108 14479 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:17:32.867182 14331 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:32.841665 14407 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:32.882346 14405 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:32.883183 14459 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:32.883935 14459 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:32.887148 14572 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:32.868623 14435 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:32.888078 14392 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:32.887976 14572 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:32.904289 10876 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 1738) took 1955 ms (client timeout 1936 ms). Trace:
W20260709 13:17:32.917644 10876 rpcz_store.cc:269] 0709 13:17:30.948533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:30.948794 (+   261us) service_pool.cc:224] Handling call
0709 13:17:32.904049 (+1955255us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b5400098dd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:32.904209 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3079877888}
I20260709 13:17:32.890152 14407 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:32.914599 14435 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:32.920769 14407 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:32.891628 10916 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:32.868173 14275 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:32.892343 10949 consensus_queue.cc:260] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:32.893282 14331 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:32.895056 14296 consensus_queue.cc:1059] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:32.922673 10916 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:32.895774 10841 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:32.878836 14569 log.cc:927] Time spent T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:17:32.842693 14409 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:32.901081 10947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2172) took 2315 ms. Trace:
I20260709 13:17:32.929293 10947 rpcz_store.cc:276] 0709 13:17:30.586214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:31.715297 (+1129083us) service_pool.cc:224] Handling call
0709 13:17:31.715596 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:32.456615 (+741019us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.900809 (+444194us) spinlock_profiling.cc:243] Waited 631 ms on lock 0x7b54001e1254. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:32.901026 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1451416448}
I20260709 13:17:32.841991 14506 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:32.904999 14144 consensus_queue.cc:1059] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:32.906109 14396 consensus_queue.cc:1048] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:17:32.906957 14555 consensus_queue.cc:1048] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:17:32.907876 14563 log.cc:927] Time spent T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.392s	user 0.000s	sys 0.001s
I20260709 13:17:32.907982 14300 consensus_queue.cc:1059] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:32.903512 14386 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:32.869204 14298 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:32.869504 14341 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:32.911351 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 213917us
I20260709 13:17:32.836907 14333 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:32.917272 14572 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:32.886545 14459 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:32.925199 14367 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:32.926447 10795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 428) took 1981 ms. Trace:
I20260709 13:17:32.927313 14296 raft_consensus.cc:3055] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:17:32.922576 14275 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:32.940214 10949 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:32.938277 14551 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:32.932927 13995 raft_consensus.cc:2749] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:32.932286 14503 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:32.949564 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 96077us
I20260709 13:17:32.944382 10795 rpcz_store.cc:276] 0709 13:17:30.933874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.230394 (+1296520us) service_pool.cc:224] Handling call
0709 13:17:32.230908 (+   514us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:32.231626 (+   718us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:17:32.575548 (+343922us) log.cc:844] Serialized 14 byte log entry
0709 13:17:32.768898 (+193350us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:32.826750 (+ 57852us) log.cc:844] Serialized 52 byte log entry
0709 13:17:32.826996 (+   246us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:17:32.838177 (+ 11181us) log.cc:844] Serialized 14 byte log entry
0709 13:17:32.866543 (+ 28366us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.866664 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:32.914052 (+ 47388us) raft_consensus.cc:1685] finished
0709 13:17:32.914075 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:32.915159 (+  1084us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49912,"prepare.queue_time_us":384472,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":65,"raft.queue_time_us":185863,"raft.run_cpu_time_us":411,"raft.run_wall_time_us":409,"spinlock_wait_cycles":7252608,"thread_start_us":482799,"threads_started":2,"wal-append.queue_time_us":269259}
I20260709 13:17:32.944890 14296 raft_consensus.cc:740] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:32.959837 14296 consensus_queue.cc:260] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:32.960505 14341 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:32.961284 14296 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:32.961561 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:32.962236 10943 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:32.964424 14333 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:32.966049 14298 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:32.967738 10797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 429) took 2033 ms. Trace:
I20260709 13:17:32.968755 14298 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:32.971045 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: true dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:32.972390 10944 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
I20260709 13:17:32.970937 14333 consensus_queue.cc:1059] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:17:32.944823 14367 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:32.976439 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:32.977149 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:32.979148 10943 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
I20260709 13:17:32.980439 14497 consensus_queue.cc:1048] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:17:32.981709 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:32.982514 10950 raft_consensus.cc:2393] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 9: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:32.984627 10948 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:32.984891 14275 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:32.986657 10801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 435) took 1195 ms. Trace:
I20260709 13:17:32.944800 14144 consensus_queue.cc:1243] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer e01ecd008edd4aa2b6b2b47efbc2ea87 log is divergent from this leader: its last log entry 8.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:17:32.987102 10801 rpcz_store.cc:276] 0709 13:17:31.791759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.250348 (+458589us) service_pool.cc:224] Handling call
0709 13:17:32.250875 (+   527us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:32.403580 (+152705us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:17:32.578679 (+175099us) log.cc:844] Serialized 14 byte log entry
0709 13:17:32.715665 (+136986us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:32.715735 (+    70us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:32.715794 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.986438 (+270644us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b54001aa4d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:32.986482 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:32.986604 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":209932,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":68377,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":118,"spinlock_wait_cycles":235760512,"thread_start_us":230220,"threads_started":2,"wal-append.queue_time_us":317097}
I20260709 13:17:32.992372 14499 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:32.949326 14331 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:32.968111 10797 rpcz_store.cc:276] 0709 13:17:30.934175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.243652 (+1309477us) service_pool.cc:224] Handling call
0709 13:17:32.244370 (+   718us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:17:32.271256 (+ 26886us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:32.271286 (+    30us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:17:32.272131 (+   845us) log.cc:844] Serialized 78 byte log entry
0709 13:17:32.472350 (+200219us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:32.697800 (+225450us) log.cc:844] Serialized 14 byte log entry
0709 13:17:32.698907 (+  1107us) log.cc:844] Serialized 14 byte log entry
0709 13:17:32.776416 (+ 77509us) spinlock_profiling.cc:243] Waited 5.91 ms on lock 0x7b1800471100. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc8522 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:17:32.967185 (+190769us) spinlock_profiling.cc:243] Waited 1.72 ms on lock 0x7b1800471100. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7dd42c0a 00007f9c87482219 00007f9c80a6bd70 00007f9c80a5f293 00007f9c80a54cea 00007f9c80a5523a 00007f9c80ebfa2a 00007f9c80ecf0bf 00007f9c80ecf038
0709 13:17:32.967450 (+   265us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.967518 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:32.967555 (+    37us) raft_consensus.cc:1685] finished
0709 13:17:32.967565 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:32.967686 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55750,"prepare.queue_time_us":142883,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":81,"raft.queue_time_us":135929,"raft.run_cpu_time_us":366,"raft.run_wall_time_us":4155,"spinlock_wait_cycles":17551488,"thread_start_us":406695,"threads_started":2,"wal-append.queue_time_us":330815}
W20260709 13:17:32.999819 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 104286us
I20260709 13:17:33.001998 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:33.002506 10841 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:33.003417 13995 raft_consensus.cc:2749] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:17:33.000293 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 77780.3us
I20260709 13:17:33.005568 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:33.006877 14345 consensus_queue.cc:1059] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:17:33.007184 10916 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:33.007997 14345 raft_consensus.cc:3055] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:17:33.008407 14345 raft_consensus.cc:740] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:33.009346 13719 consensus_queue.cc:1059] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:33.009425 14345 consensus_queue.cc:260] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:33.010531 14362 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:33.013039 11477 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:33.013648 14440 consensus_queue.cc:1059] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:17:33.014195 14572 consensus_queue.cc:1048] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:33.019970 14362 raft_consensus.cc:697] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:33.023216 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:32.979590 14367 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:33.021405 10949 raft_consensus.cc:1275] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:17:33.031540 14573 raft_consensus.cc:2749] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:33.031874 11477 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:33.033282 14497 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:33.033747 14497 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:33.034482 14497 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:33.034816 14555 consensus_queue.cc:1048] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:33.039179 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:33.030251 14478 consensus_queue.cc:1059] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:17:33.039666 14499 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:17:33.039238 11508 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56404 (request call id 675) took 2440 ms (client timeout 2225 ms). Trace:
I20260709 13:17:33.040710 14478 raft_consensus.cc:3055] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:17:33.040949 11508 rpcz_store.cc:269] 0709 13:17:30.598672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:30.599318 (+   646us) service_pool.cc:224] Handling call
0709 13:17:33.018639 (+2419321us) spinlock_profiling.cc:243] Waited 292 ms on lock 0x7b54001d1354. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:33.039005 (+ 20366us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":671505280}
I20260709 13:17:33.041286 14478 raft_consensus.cc:740] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:33.043224 10947 raft_consensus.cc:1240] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 11. Current term is 12. Ops: []
I20260709 13:17:33.043743 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:33.044432 10874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 503) took 1257 ms. Trace:
I20260709 13:17:33.042668 14478 consensus_queue.cc:260] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:33.028467 10949 pending_rounds.cc:85] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 7
I20260709 13:17:33.044821 10874 rpcz_store.cc:276] 0709 13:17:31.787052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.272818 (+485766us) service_pool.cc:224] Handling call
0709 13:17:32.445445 (+172627us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:32.476900 (+ 31455us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:33.044227 (+567327us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b54001090d0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:33.044379 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":390835456}
W20260709 13:17:33.044716 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:33.047102 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:33.055552 10949 pending_rounds.cc:107] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 7.8
W20260709 13:17:33.056763 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:33.033737 14573 consensus_queue.cc:1048] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:33.052934 14511 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:33.044466 10950 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 11.
I20260709 13:17:33.064512 10945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 540) took 2995 ms. Trace:
I20260709 13:17:33.067026 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:33.067946 10943 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 10.
I20260709 13:17:33.068065 10949 raft_consensus.cc:2889] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:33.069082 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:33.066772 10945 rpcz_store.cc:276] 0709 13:17:30.032718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:31.258758 (+1226040us) service_pool.cc:224] Handling call
0709 13:17:31.260949 (+  2191us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:32.864125 (+1603176us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:33.018035 (+153910us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b54000168d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:33.027957 (+  9922us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3312575360}
I20260709 13:17:32.957661 14409 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:33.051687 14478 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:33.079221 10868 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 9.
I20260709 13:17:33.049134 14386 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:33.083774 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:33.079674 10944 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7.
I20260709 13:17:33.087553 10949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 553) took 1876 ms. Trace:
I20260709 13:17:33.087975 10949 rpcz_store.cc:276] 0709 13:17:31.205420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.685166 (+1479746us) service_pool.cc:224] Handling call
0709 13:17:32.777810 (+ 92644us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:33.081682 (+303872us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:33.081882 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:33.088953 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:33.040495 14499 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:33.090183 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:33.096194 14549 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:33.077904 14590 log.cc:927] Time spent T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
W20260709 13:17:33.099890 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 75847.7us
I20260709 13:17:33.100147 14388 raft_consensus.cc:3055] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:17:33.144789 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 112976us
I20260709 13:17:33.059711 14362 consensus_queue.cc:237] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:33.105615 14577 log.cc:927] Time spent T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.388s	user 0.004s	sys 0.000s
I20260709 13:17:33.110950 14345 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:33.119827 10950 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7.
I20260709 13:17:33.102341 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:33.153532 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:33.125229 14499 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:33.149101 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:33.144961 10841 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:33.155303 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:33.071570 14497 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:33.144397 14388 raft_consensus.cc:740] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:33.089599 14435 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:33.166358 13719 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:17:33.167680 10942 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 8.
I20260709 13:17:33.169790 14506 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:33.170074 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:33.171115 14342 raft_consensus.cc:3055] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:33.171567 14342 raft_consensus.cc:740] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:33.172518 14342 consensus_queue.cc:260] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:33.174165 14342 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:33.177685 10944 raft_consensus.cc:2393] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 11: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:33.179463 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:33.182489 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:33.183876 10943 raft_consensus.cc:2393] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 12: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:33.185708 10949 raft_consensus.cc:2393] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 11: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:33.188956 14435 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:33.189386 14549 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:33.191741 14388 consensus_queue.cc:260] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:33.193497 14388 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:33.198927 14551 raft_consensus.cc:2749] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:33.166381 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:33.200014 10943 raft_consensus.cc:1275] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:17:33.200212 11508 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 12.
I20260709 13:17:33.205392 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:33.206216 10941 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 7.
W20260709 13:17:33.217900 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:33.218544 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:33.231395 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 224365us
I20260709 13:17:33.227613 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:17:33.251683 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:33.252223 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:33.253888 10944 raft_consensus.cc:1240] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:33.223439 14390 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:33.255626 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:33.259151 10799 raft_consensus.cc:2410] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:17:33.259958 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:33.256986 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:33.261806 10800 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:33.235616 14409 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:33.261019 10945 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10.
I20260709 13:17:33.224639 14435 consensus_queue.cc:1059] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.035s
W20260709 13:17:33.266263 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 138387us
I20260709 13:17:33.270633 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:33.271483 10795 raft_consensus.cc:2393] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 9: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:33.276993 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:33.277691 10798 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 6.
I20260709 13:17:33.279439 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:33.284808 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:33.285511 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:33.285501 10945 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 11.
I20260709 13:17:33.287374 10797 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 9.
I20260709 13:17:33.300616 14503 consensus_queue.cc:1048] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:17:33.303205 14592 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:33.303907 14592 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:33.305558 10767 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:33.306185 14592 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:33.306851 14409 raft_consensus.cc:2749] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:33.311164 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:33.312085 10944 raft_consensus.cc:2393] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 8: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:33.309687 10801 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 9.
I20260709 13:17:33.315675 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:33.322777 10950 raft_consensus.cc:2410] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:17:33.324540 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:33.326475 14390 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:33.324534 14591 log.cc:927] Time spent T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:17:33.328500 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:33.328853 14390 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:33.329097 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:33.329329 10874 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10.
I20260709 13:17:33.329798 10869 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 8.
I20260709 13:17:33.330351 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:33.334843 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:33.333112 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 111538us
I20260709 13:17:33.343863 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:33.346954 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:33.348048 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:33.349900 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:33.348906 10868 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10.
I20260709 13:17:33.358358 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:33.361531 10766 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:33.365851 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:33.367859 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:33.367933 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:33.369105 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:33.370501 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:33.368616 10867 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 9.
I20260709 13:17:33.374892 10874 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10.
I20260709 13:17:33.377542 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:33.377810 10841 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:33.378851 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:33.380822 14599 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:33.381826 10945 raft_consensus.cc:1275] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:17:33.381590 14599 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:33.383844 10799 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39442 (request call id 434) took 1612 ms (client timeout 1589 ms). Trace:
I20260709 13:17:33.384057 14599 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:33.384214 14367 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:33.384799 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:33.385587 14596 log.cc:927] Time spent T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:17:33.385661 14479 raft_consensus.cc:3055] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:33.386019 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:33.380904 10793 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 1740) took 2138 ms (client timeout 2321 ms). Trace:
W20260709 13:17:33.388407 10793 rpcz_store.cc:269] 0709 13:17:31.242045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.214481 (+972436us) service_pool.cc:224] Handling call
0709 13:17:33.380684 (+1166203us) spinlock_profiling.cc:243] Waited 211 ms on lock 0x7b54000d8754. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:33.380833 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":485048704}
I20260709 13:17:33.389447 10942 raft_consensus.cc:1240] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
W20260709 13:17:33.384279 10799 rpcz_store.cc:269] 0709 13:17:31.771479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.203896 (+432417us) service_pool.cc:224] Handling call
0709 13:17:33.383620 (+1179724us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54001cacd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:33.383757 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2413814144}
I20260709 13:17:33.382449 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:33.315982 14573 consensus_queue.cc:1059] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.063s
I20260709 13:17:33.386138 14479 raft_consensus.cc:740] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:33.393002 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:33.369663 14331 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:33.386562 14367 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:33.410223 10867 raft_consensus.cc:1240] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:33.414824 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:33.356437 14390 consensus_queue.cc:1048] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:17:33.424513 10948 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
I20260709 13:17:33.424558 10800 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 10.
I20260709 13:17:33.416471 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:33.417594 10767 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:33.427093 10870 raft_consensus.cc:2376] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 8. Re-sending same reply.
I20260709 13:17:33.433848 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:33.436332 10873 raft_consensus.cc:2393] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 9: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:33.403405 14600 raft_consensus.cc:3055] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:33.456717 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:33.453014 13719 raft_consensus.cc:2804] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:33.434890 14391 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:17:33.461453 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 43915.2us
I20260709 13:17:33.468961 10767 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:33.412866 14333 consensus_queue.cc:1048] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
W20260709 13:17:33.467437 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 105995us
I20260709 13:17:33.476017 10766 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:33.470798 10767 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:33.473874 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:33.454212 14600 raft_consensus.cc:740] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:33.482388 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:33.473389 14296 consensus_meta.cc:287] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.509s	user 0.008s	sys 0.006s
I20260709 13:17:33.486289 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:33.462042 13719 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:33.467444 14392 raft_consensus.cc:2804] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:17:33.495512 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 130280us
I20260709 13:17:33.477246 14275 raft_consensus.cc:2804] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:33.505414 10872 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 12.
I20260709 13:17:33.458380 14331 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:33.483366 14409 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:33.518750 14540 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:17:33.518656 14600 consensus_queue.cc:260] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:33.519562 14540 raft_consensus.cc:697] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:33.520336 14600 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:33.520538 14540 consensus_queue.cc:237] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:33.524221 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:33.501308 13719 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:33.535341 10841 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:33.505414 14392 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:33.535259 10766 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:33.508761 14275 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:33.536772 11512 raft_consensus.cc:1275] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:17:33.542985 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:33.547482 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:33.513394 14409 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:33.550746 10874 raft_consensus.cc:2393] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 8: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:33.551052 11509 raft_consensus.cc:1240] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 10. Current term is 11. Ops: [6.6-10.8]
I20260709 13:17:33.548214 14392 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:33.552995 14574 raft_consensus.cc:3055] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:33.553088 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:33.553833 11503 raft_consensus.cc:1240] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 10. Current term is 11. Ops: []
W20260709 13:17:33.556357 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 54168.1us
I20260709 13:17:33.569526 10841 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:33.548049 11511 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 8.
W20260709 13:17:33.536504 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 65483.6us
I20260709 13:17:33.571575 10767 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:33.574887 11510 raft_consensus.cc:2393] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 9: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:33.551808 14275 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:33.582362 10841 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:33.583420 10767 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:33.553030 14336 raft_consensus.cc:3055] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:33.549475 10766 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:33.501190 14479 consensus_queue.cc:260] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:33.568861 14574 raft_consensus.cc:740] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:33.573031 14409 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:33.608289 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:33.613989 10841 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:33.581916 14572 raft_consensus.cc:2804] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:33.631289 11512 raft_consensus.cc:1275] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 13:17:33.633148 14601 log.cc:927] Time spent T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
I20260709 13:17:33.633910 14134 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:33.634377 14134 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:33.634824 14134 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:33.640825 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:33.641655 10873 raft_consensus.cc:2393] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 12: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:33.647426 14304 raft_consensus.cc:2804] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:33.647881 14304 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:33.648242 14304 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:33.650593 14390 raft_consensus.cc:2749] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:33.651350 14390 raft_consensus.cc:2749] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:33.652029 14390 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:33.652405 14390 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:33.652747 14390 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:33.653542 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: true dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:33.654212 11510 raft_consensus.cc:3055] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:17:33.654480 11510 raft_consensus.cc:740] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:33.655289 11510 consensus_queue.cc:260] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:33.656302 11510 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:33.657606 14331 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:33.602036 14336 raft_consensus.cc:740] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:33.658157 11478 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:33.688429 14607 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:33.689004 14607 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:33.689420 14607 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:33.605477 14506 raft_consensus.cc:2804] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:33.691411 14506 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:33.691852 14506 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:33.629989 14572 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:33.695464 14572 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:33.695608 14549 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:33.696171 14549 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:33.698370 14549 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:33.659022 14336 consensus_queue.cc:260] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:33.659608 14574 consensus_queue.cc:260] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:33.660311 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:33.717118 10874 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:33.664222 11510 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
I20260709 13:17:33.666136 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:33.675283 13995 raft_consensus.cc:2749] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:17:33.675812 14402 raft_consensus.cc:3055] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:17:33.677139 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 63261.7us
I20260709 13:17:33.734957 14402 raft_consensus.cc:740] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:33.736011 14402 consensus_queue.cc:260] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:33.737075 14402 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:33.739362 14478 raft_consensus.cc:3055] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:33.739794 14478 raft_consensus.cc:740] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:33.740736 14478 consensus_queue.cc:260] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:33.616976 10916 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:33.742189 14478 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:33.743104 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:33.743336 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:33.744199 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:33.742895 14555 consensus_queue.cc:1059] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:17:33.745499 14296 raft_consensus.cc:3055] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:17:33.745857 14296 raft_consensus.cc:740] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:33.746691 14296 consensus_queue.cc:260] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:33.748219 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:33.748893 11509 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 9.
I20260709 13:17:33.749408 14296 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:33.749742 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:33.757671 14144 consensus_queue.cc:1048] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:17:33.701603 11477 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:33.707226 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:33.691670 11478 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:33.719338 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:17:33.720108 14511 consensus_meta.cc:287] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.644s	user 0.011s	sys 0.000s
I20260709 13:17:33.720551 14336 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:33.721083 14388 consensus_meta.cc:287] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.527s	user 0.003s	sys 0.008s
I20260709 13:17:33.724464 14574 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:33.815409 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:33.671236 14499 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:33.787456 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:33.773294 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:33.775028 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:17:33.807210 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:33.824643 11509 raft_consensus.cc:2410] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:17:33.807649 10874 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 11.
I20260709 13:17:33.728796 14497 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:33.772805 14342 consensus_meta.cc:287] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.598s	user 0.012s	sys 0.002s
I20260709 13:17:33.822448 11512 raft_consensus.cc:2368] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 9. Current term is 10.
W20260709 13:17:33.772359 14345 consensus_meta.cc:287] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.618s	user 0.009s	sys 0.000s
W20260709 13:17:33.826603 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 125100us
I20260709 13:17:33.827389 10941 raft_consensus.cc:2393] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 10: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:33.826519 10871 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 10.
W20260709 13:17:33.860770 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:33.722728 14573 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:33.882576 10869 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 8.
I20260709 13:17:33.885160 14386 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:33.885496 10946 raft_consensus.cc:2410] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 11 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
W20260709 13:17:33.894552 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:33.895344 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:33.822999 14391 raft_consensus.cc:697] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:33.901648 10870 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 10.
I20260709 13:17:33.831522 14043 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:33.830994 14435 raft_consensus.cc:2804] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:33.758705 13719 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:33.850973 14600 raft_consensus.cc:2804] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:33.923177 10943 raft_consensus.cc:1240] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:33.831161 13995 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:33.878859 14503 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:33.878611 14511 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:33.934165 11509 raft_consensus.cc:1240] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
W20260709 13:17:33.944147 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 252581us
I20260709 13:17:33.900661 14614 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:33.912243 14367 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:33.951907 10867 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 11.
I20260709 13:17:33.885917 14386 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:33.926960 14435 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:33.961537 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:33.927251 14600 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:33.902249 14333 consensus_queue.cc:1048] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 13:17:33.918277 14595 raft_consensus.cc:2749] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:17:33.899325 14573 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:33.922163 14341 consensus_queue.cc:1059] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.082s
I20260709 13:17:33.770054 14607 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:33.922263 14043 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:33.939076 14296 consensus_queue.cc:1048] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.083s
W20260709 13:17:34.045785 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 462571us
I20260709 13:17:34.046595 10767 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:34.021055 14600 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:33.947392 14511 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:34.012133 14435 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:33.936445 14409 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:34.094197 14511 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:34.094372 14409 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:34.096035 14511 raft_consensus.cc:3055] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:17:34.096439 14511 raft_consensus.cc:740] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:34.056061 11478 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:33.942979 14506 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:34.097345 14511 consensus_queue.cc:260] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:34.098763 14506 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:33.941402 13995 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:34.070871 10767 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:34.101382 13995 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:34.104622 14435 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:34.108409 14435 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:34.119223 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:34.121980 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:34.045176 11506 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.014s	sys 0.018s
I20260709 13:17:34.075404 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:34.076567 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:34.154289 11505 raft_consensus.cc:1240] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 10. Current term is 11. Ops: []
I20260709 13:17:34.054529 14296 consensus_queue.cc:1243] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer 12682eb70bc84c76879e04a8628ee586 log is divergent from this leader: its last log entry 7.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:17:34.023569 14540 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:34.098917 14331 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:17:34.064400 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 514935us
I20260709 13:17:34.160876 10946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41020 (request call id 2189) took 1086 ms. Trace:
I20260709 13:17:34.162041 10946 rpcz_store.cc:276] 0709 13:17:33.061526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:33.066321 (+  4795us) service_pool.cc:224] Handling call
0709 13:17:34.126069 (+1059748us) spinlock_profiling.cc:243] Waited 798 ms on lock 0x7b540018ecd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:34.147627 (+ 21558us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1835961728}
I20260709 13:17:34.098590 14511 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:34.130587 10798 raft_consensus.cc:2393] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 11: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:34.148064 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:34.004729 14367 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:34.031289 14617 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:34.151185 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:34.012578 14503 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:34.149222 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:34.180733 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:17:34.161082 14304 consensus_meta.cc:287] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.512s	user 0.008s	sys 0.007s
I20260709 13:17:34.045123 14614 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:34.157126 14610 log.cc:927] Time spent T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:17:34.161252 14331 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:34.176360 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:17:34.156260 11506 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.018s	sys 0.018s
I20260709 13:17:34.186703 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:34.212034 10799 raft_consensus.cc:2393] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 9: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
W20260709 13:17:34.187930 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 117150us
I20260709 13:17:34.213256 10767 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:34.189371 14607 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:34.178150 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:34.179189 14388 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:34.200695 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:34.216109 10946 raft_consensus.cc:2393] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 11: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:34.169565 14435 raft_consensus.cc:2749] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:34.170708 14409 raft_consensus.cc:2804] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:17:34.224830 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 202792us
W20260709 13:17:34.215217 14572 consensus_meta.cc:287] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.519s	user 0.003s	sys 0.007s
W20260709 13:17:34.216437 14392 consensus_meta.cc:287] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.634s	user 0.005s	sys 0.005s
W20260709 13:17:34.215408 14134 consensus_meta.cc:287] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.580s	user 0.008s	sys 0.003s
I20260709 13:17:34.195412 14499 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:34.163172 14540 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:34.218190 14331 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:34.245797 10766 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:34.246834 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:34.247820 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 304260us
W20260709 13:17:34.278220 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 661556us
W20260709 13:17:34.291886 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 78729.7us
W20260709 13:17:34.299198 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:34.299716 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:34.300096 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:34.300027 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 385111us
W20260709 13:17:34.301617 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:34.303791 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:34.179190 14305 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:34.278942 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:34.280277 13719 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:34.285842 10798 raft_consensus.cc:1240] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:34.303893 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:34.313637 10874 raft_consensus.cc:1240] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:34.235404 14388 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:34.293460 14402 consensus_meta.cc:287] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.556s	user 0.011s	sys 0.001s
I20260709 13:17:34.206601 14304 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:34.317479 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:34.244999 14572 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:34.293033 14618 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:34.291086 14622 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:34.321123 10868 raft_consensus.cc:2393] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 11: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:34.244779 14392 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:34.332988 10873 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 9.
I20260709 13:17:34.318849 10875 raft_consensus.cc:1240] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:34.180343 14391 consensus_queue.cc:237] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:34.337895 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:34.305023 14627 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:34.340102 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:34.340880 10796 raft_consensus.cc:1240] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:17:34.341526 10916 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:34.331240 14623 raft_consensus.cc:2749] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:17:34.343348 14626 log.cc:927] Time spent T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:17:34.382022 14635 raft_consensus.cc:2749] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:17:34.344563 14614 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:34.345607 14503 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:34.391170 14631 log.cc:927] Time spent T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
W20260709 13:17:34.346208 14619 log.cc:927] Time spent T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.176s	user 0.000s	sys 0.000s
W20260709 13:17:34.331017 14615 log.cc:927] Time spent T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.309s	user 0.000s	sys 0.001s
I20260709 13:17:34.391710 14614 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:34.348272 14574 consensus_queue.cc:1048] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:17:34.345638 14628 log.cc:927] Time spent T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:17:34.351567 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:34.394552 10874 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10.
I20260709 13:17:34.395552 11506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56410 (request call id 640) took 1142 ms. Trace:
I20260709 13:17:34.395954 11506 rpcz_store.cc:276] 0709 13:17:33.252814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:33.252975 (+   161us) service_pool.cc:224] Handling call
0709 13:17:33.264220 (+ 11245us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:33.414011 (+149791us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:33.414038 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:33.457546 (+ 43508us) log.cc:844] Serialized 52 byte log entry
0709 13:17:34.211770 (+754224us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:34.211809 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:34.211875 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:34.395332 (+183457us) raft_consensus.cc:1685] finished
0709 13:17:34.395351 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:34.395496 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14750,"raft.queue_time_us":70941,"raft.run_cpu_time_us":252,"raft.run_wall_time_us":32621,"thread_start_us":796412,"threads_started":2,"wal-append.queue_time_us":755875}
I20260709 13:17:34.397662 14392 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:34.355234 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 66335.1us
W20260709 13:17:34.353641 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:34.357522 14390 consensus_meta.cc:287] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.652s	user 0.007s	sys 0.000s
I20260709 13:17:34.399003 10794 raft_consensus.cc:1240] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:17:34.399155 14390 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:34.400271 14625 log.cc:927] Time spent T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 13:17:34.401480 14390 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:34.402105 11478 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:34.359638 10793 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 11.
W20260709 13:17:34.371897 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:34.406587 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:34.339057 14605 log.cc:927] Time spent T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.772s	user 0.001s	sys 0.004s
I20260709 13:17:34.234239 14617 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:34.343503 14540 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:34.410928 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:34.382014 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 51241.1us
I20260709 13:17:34.385980 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:34.278148 14134 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:17:34.392421 14614 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:34.391729 14503 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:17:34.448997 14503 raft_consensus.cc:697] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:34.449967 14503 consensus_queue.cc:237] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:34.404735 10797 raft_consensus.cc:1240] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:34.359030 14388 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:34.403631 10799 raft_consensus.cc:2368] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 10. Current term is 11.
I20260709 13:17:34.391191 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:34.357347 14572 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:34.334722 14386 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:34.360414 14618 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:34.455214 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:34.456041 10799 raft_consensus.cc:2393] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 11: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:34.456334 14618 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:34.382987 14622 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:34.456892 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:34.458033 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:34.458424 14622 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:34.334980 14573 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:34.415645 10916 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:34.459417 14573 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:34.459810 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 47924.2us
I20260709 13:17:34.415196 11478 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:17:34.460789 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 45715.3us
I20260709 13:17:34.461771 11478 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:34.416097 14624 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:34.413789 14574 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:34.319715 14305 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:34.420967 14627 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:34.421510 14435 raft_consensus.cc:2804] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:34.421586 10876 raft_consensus.cc:1240] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:17:34.421710 14636 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:34.422745 10841 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:34.477697 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:17:34.477984 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 55313.4us
I20260709 13:17:34.427171 14540 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:17:34.416764 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 53924.9us
I20260709 13:17:34.419003 14549 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:34.429785 14331 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:34.432263 14600 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:34.440210 14367 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:34.444478 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:34.467886 14499 raft_consensus.cc:2804] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:34.448312 14304 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:34.489933 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:34.414224 14620 log.cc:927] Time spent T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.360s	user 0.001s	sys 0.000s
I20260709 13:17:34.460121 14573 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:34.479686 10841 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:34.460421 14446 raft_consensus.cc:3055] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:17:34.481925 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:34.483198 10793 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 12.
I20260709 13:17:34.446759 14617 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:34.481902 10916 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:34.485000 10795 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 12.
W20260709 13:17:34.479038 14336 consensus_meta.cc:287] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.657s	user 0.003s	sys 0.007s
I20260709 13:17:34.475133 14624 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:34.470139 14635 raft_consensus.cc:2804] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:34.469233 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:34.475253 14435 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:34.482761 14341 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:34.508829 10800 raft_consensus.cc:2393] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 9: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
W20260709 13:17:34.506333 14275 consensus_meta.cc:287] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.886s	user 0.007s	sys 0.004s
I20260709 13:17:34.477399 14574 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:34.485152 14499 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:34.555794 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:34.555785 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 310093us
I20260709 13:17:34.494205 14134 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:34.559703 14479 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:34.507817 11478 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:34.580626 11506 raft_consensus.cc:2368] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 11. Current term is 12.
I20260709 13:17:34.585275 10766 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:34.585852 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 104025us
I20260709 13:17:34.495180 14630 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:17:34.581898 10871 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 1744) took 3223 ms (client timeout 2321 ms). Trace:
I20260709 13:17:34.586999 14499 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:17:34.587693 10871 rpcz_store.cc:269] 0709 13:17:31.338184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.264644 (+926460us) service_pool.cc:224] Handling call
0709 13:17:34.561922 (+2297278us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b5400077454. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:34.562031 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3575392640}
W20260709 13:17:34.590328 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 110737us
I20260709 13:17:34.505939 14446 raft_consensus.cc:740] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:34.590742 10916 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:34.591249 10797 raft_consensus.cc:1275] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 8. (term mismatch)
I20260709 13:17:34.596304 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:34.591603 10797 pending_rounds.cc:85] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 7
I20260709 13:17:34.597581 10797 pending_rounds.cc:107] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 10.8
W20260709 13:17:34.597697 10696 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42956 (request call id 259) took 12536 ms (client timeout 15000 ms). Trace:
I20260709 13:17:34.597878 10797 raft_consensus.cc:2889] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:34.485471 14549 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:34.598928 10767 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:34.553421 14409 raft_consensus.cc:697] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:34.580418 14362 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:34.580250 14435 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:34.598148 10796 raft_consensus.cc:1240] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:34.561528 14635 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:34.586398 14555 raft_consensus.cc:2749] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:17:34.602227 14622 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:34.587610 14630 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:34.626330 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 118640us
I20260709 13:17:34.632458 10767 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:34.554176 14540 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:34.653589 11478 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:34.484129 14636 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:34.602887 14595 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:34.581827 14275 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:34.651568 10941 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:17:34.662653 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:17:34.663527 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 72861us
I20260709 13:17:34.607825 14633 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:34.669901 10767 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:34.675693 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:34.618896 14134 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:34.628006 14592 raft_consensus.cc:2804] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:34.702872 10941 raft_consensus.cc:2393] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 11: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:34.705165 10916 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:34.619397 14635 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:34.624466 14402 raft_consensus.cc:2749] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:34.709245 10841 leader_election.cc:400] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 11. Current term is 12.
I20260709 13:17:34.669157 13719 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:17:34.714499 14620 log.cc:927] Time spent T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:17:34.706916 14342 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:34.750366 10798 raft_consensus.cc:2393] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 9: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:34.676487 14392 consensus_queue.cc:1059] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:17:34.715144 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:34.719545 14478 consensus_meta.cc:287] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.977s	user 0.014s	sys 0.000s
I20260709 13:17:34.720890 14305 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:34.722046 14331 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:34.732057 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 62278.5us
I20260709 13:17:34.736281 10841 leader_election.cc:403] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:17:34.761634 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 93737.9us
I20260709 13:17:34.746984 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:34.761682 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:34.763031 10873 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 10.
I20260709 13:17:34.763723 10875 raft_consensus.cc:2410] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:17:34.652726 14388 raft_consensus.cc:2804] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:34.746376 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:34.771224 10941 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 9
W20260709 13:17:34.709168 14640 log.cc:927] Time spent T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:17:34.650874 14630 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:34.669252 14618 raft_consensus.cc:2749] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:34.707345 14607 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:34.797138 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:34.733037 14592 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:17:34.807531 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 47782.1us
W20260709 13:17:34.776839 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 178110us
W20260709 13:17:34.779955 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 138707us
I20260709 13:17:34.785207 10949 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 11.
I20260709 13:17:34.749771 14304 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:34.750968 14367 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:34.821672 10766 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:34.821972 14367 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:34.822525 14367 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:34.658360 14622 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:34.794646 14478 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:17:34.798056 11504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56410 (request call id 642) took 1514 ms. Trace:
I20260709 13:17:34.749286 14134 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:34.770982 14388 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:17:34.838178 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:34.816097 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:34.877566 10795 raft_consensus.cc:2393] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 9: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:34.685417 14595 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:34.818734 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:34.878559 10794 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:34.819440 14275 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:34.821472 14629 log.cc:927] Time spent T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.578s	user 0.004s	sys 0.000s
I20260709 13:17:34.744335 14645 raft_consensus.cc:2749] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:17:34.756184 13719 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:34.816064 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:34.839108 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:34.743752 14409 consensus_queue.cc:237] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:34.786216 14342 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:34.881177 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:34.810410 14592 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:34.884773 10766 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:34.923995 10766 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:34.926110 10766 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:34.848774 10916 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:34.806478 14617 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:34.813033 14607 raft_consensus.cc:697] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 10 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:34.899883 10943 raft_consensus.cc:1240] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 10. Current term is 11. Ops: []
I20260709 13:17:34.677778 14446 consensus_queue.cc:260] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:34.900431 14555 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:34.819444 14304 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:34.904945 10944 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:34.832265 11504 rpcz_store.cc:276] 0709 13:17:33.283843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:33.294454 (+ 10611us) service_pool.cc:224] Handling call
0709 13:17:33.315700 (+ 21246us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:33.469557 (+153857us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:33.469625 (+    68us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:33.533966 (+ 64341us) log.cc:844] Serialized 52 byte log entry
0709 13:17:33.756637 (+222671us) spinlock_profiling.cc:243] Waited 37.7 ms on lock 0x7b48002409d8. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c85ddeaa1 00007f9c85dd5b37 00007f9c85dd59ca 00007f9c85ecd26f 00007f9c85eccf37 00007f9c85ee155c 00007f9c85ee1508 00007f9c85ee1488 00007f9c85ee1443 00007f9c85ee06ec 00007f9c84e0aa96
0709 13:17:33.777643 (+ 21006us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:33.827886 (+ 50243us) log.cc:844] Serialized 14 byte log entry
0709 13:17:33.828425 (+   539us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:33.828497 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:34.786946 (+958449us) raft_consensus.cc:1685] finished
0709 13:17:34.797805 (+ 10859us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:34.797976 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18164,"prepare.queue_time_us":50032,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":71439,"raft.run_cpu_time_us":3243,"raft.run_wall_time_us":408786,"spinlock_wait_cycles":86801152,"thread_start_us":326633,"threads_started":2,"wal-append.queue_time_us":384607}
I20260709 13:17:34.907635 11478 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:34.912269 11512 raft_consensus.cc:1275] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:17:34.912606 11507 raft_consensus.cc:1240] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:17:34.914983 14614 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:34.917292 13719 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:34.904253 14402 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:34.882370 14388 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:34.928727 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:17:34.598134 10696 rpcz_store.cc:269] 0709 13:17:21.956822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:21.958524 (+  1702us) service_pool.cc:224] Handling call
0709 13:17:34.485291 (+12526767us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:30.691115 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:30.691716 (+   601us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:30.691752 (+    36us) write_op.cc:435] Acquired schema lock
0709 13:17:30.691771 (+    19us) tablet.cc:662] Decoding operations
0709 13:17:30.826439 (+134668us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:30.849489 (+ 23050us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:30.851308 (+  1819us) tablet.cc:685] Acquiring locks for 9 operations
0709 13:17:30.941183 (+ 89875us) tablet.cc:701] Row locks acquired
0709 13:17:30.941222 (+    39us) write_op.cc:260] PREPARE: finished
0709 13:17:30.941405 (+   183us) write_op.cc:270] Start()
0709 13:17:30.941544 (+   139us) write_op.cc:276] Timestamp: P: 1783603050941349 usec, L: 0
0709 13:17:30.941573 (+    29us) op_driver.cc:348] REPLICATION: starting
0709 13:17:30.942325 (+   752us) log.cc:844] Serialized 3632 byte log entry
0709 13:17:32.458841 (+1516516us) op_driver.cc:464] REPLICATION: finished
0709 13:17:32.809902 (+351061us) write_op.cc:301] APPLY: starting
0709 13:17:32.823859 (+ 13957us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:32.832785 (+  8926us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:32.832960 (+   175us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:32.882870 (+ 49910us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:32.883288 (+   418us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=9
0709 13:17:32.883336 (+    48us) write_op.cc:312] APPLY: finished
0709 13:17:32.883808 (+   472us) log.cc:844] Serialized 71 byte log entry
0709 13:17:32.884063 (+   255us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:32.884369 (+   306us) write_op.cc:454] Released schema lock
0709 13:17:32.884429 (+    60us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:32.884936 (+   507us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:32.884974 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":39,"child_traces":[["op",{"apply.queue_time_us":323082,"apply.run_cpu_time_us":4996,"apply.run_wall_time_us":97760,"num_ops":9,"prepare.queue_time_us":548974,"prepare.run_cpu_time_us":108436,"prepare.run_wall_time_us":1117323,"raft.queue_time_us":649519,"raft.run_cpu_time_us":11768,"raft.run_wall_time_us":180735,"replication_time_us":1517017,"thread_start_us":1734592,"threads_started":4,"wal-append.queue_time_us":864268,"wal-append.run_cpu_time_us":20962,"wal-append.run_wall_time_us":2080098}]]}
I20260709 13:17:34.880792 14386 raft_consensus.cc:2749] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:34.766606 14633 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:34.934738 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:34.875954 14636 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:34.943498 11508 raft_consensus.cc:1240] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:34.929559 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:34.983592 10949 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 9.
I20260709 13:17:34.985155 10795 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:34.896231 14627 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:34.815419 14392 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:34.903445 14600 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:34.966724 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 59214.4us
I20260709 13:17:34.927845 14300 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:34.937640 14645 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:17:34.975359 10945 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.003s	sys 0.006s
I20260709 13:17:34.968675 14440 raft_consensus.cc:2749] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:17:34.980412 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:34.941264 14574 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:34.981557 11506 raft_consensus.cc:2393] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 11: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:34.987718 11503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56416 (request call id 643) took 1333 ms. Trace:
I20260709 13:17:34.959856 14540 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:34.969834 14304 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:34.849592 14499 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:35.038810 14633 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:35.040140 14499 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:35.013581 11478 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:35.015781 10945 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.004s	sys 0.007s
I20260709 13:17:35.020848 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:34.942103 14617 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:34.979818 14446 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:34.970038 14555 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:35.051990 10944 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
W20260709 13:17:35.044555 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 149229us
I20260709 13:17:35.044767 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:35.011899 14390 raft_consensus.cc:2749] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:35.012668 14645 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:17:35.059249 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:35.030673 11503 rpcz_store.cc:276] 0709 13:17:33.653965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:33.654097 (+   132us) service_pool.cc:224] Handling call
0709 13:17:33.654688 (+   591us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:33.655213 (+   525us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:33.655237 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:33.656273 (+  1036us) log.cc:844] Serialized 564 byte log entry
0709 13:17:33.937091 (+280818us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:33.937141 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:33.937224 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:34.821133 (+883909us) spinlock_profiling.cc:243] Waited 36.7 ms on lock 0x7b24000d5308. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dc4e9c4 00007f9c7dc4cacb 00007f9c7de07dc9 00007f9c7de07a55 00007f9c7de0709d 00007f9c80a3910e 00007f9c80a5088c 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221
0709 13:17:34.987407 (+166274us) raft_consensus.cc:1685] finished
0709 13:17:34.987428 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:34.987657 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":84492672,"thread_start_us":280436,"threads_started":1,"wal-append.queue_time_us":510697}
I20260709 13:17:35.014818 14331 raft_consensus.cc:2749] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:17:35.040179 14649 log.cc:927] Time spent T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
W20260709 13:17:35.061658 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 135452us
W20260709 13:17:35.064998 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:35.066617 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:35.067276 10873 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
I20260709 13:17:35.069187 14627 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:35.069471 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:35.065006 10766 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:35.071388 10942 raft_consensus.cc:1240] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:35.072136 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:35.037590 14636 raft_consensus.cc:2749] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:17:35.076702 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:35.077366 10867 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:35.051709 14499 raft_consensus.cc:2749] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:35.012720 14402 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:35.026808 14300 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:35.082613 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:35.029058 14392 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:35.083912 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:35.084739 10873 raft_consensus.cc:2410] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:17:35.085575 10842 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:35.087374 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:35.090091 10842 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:35.070412 14645 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:35.093537 10842 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:35.095597 11478 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:35.081514 14386 raft_consensus.cc:2749] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:35.102182 11478 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:35.069972 14627 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:35.060678 14643 consensus_queue.cc:1059] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.284s
I20260709 13:17:35.106351 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:35.103718 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:35.086817 14362 raft_consensus.cc:2749] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:35.109454 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:35.112155 10873 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
I20260709 13:17:35.117084 14300 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:35.120294 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 301999us
I20260709 13:17:35.121201 10916 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:35.097666 14336 raft_consensus.cc:2749] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:35.097725 14574 raft_consensus.cc:2749] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:17:35.100528 14600 raft_consensus.cc:2749] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:17:35.072991 14607 consensus_queue.cc:237] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:35.132793 10794 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 9.
I20260709 13:17:35.090502 14624 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:35.132979 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 46576.2us
I20260709 13:17:35.131457 10916 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:35.137686 13995 consensus_queue.cc:1059] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:35.109501 13719 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:35.109412 14606 raft_consensus.cc:2749] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:35.199544 10941 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:17:35.123013 14342 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:35.138624 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:35.207294 10873 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
I20260709 13:17:35.209353 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:17:35.213430 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 108747us
I20260709 13:17:35.129886 14300 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
W20260709 13:17:35.191380 14652 log.cc:927] Time spent T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 13:17:35.218721 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:35.132989 14134 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:35.225167 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:35.225893 11510 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
I20260709 13:17:35.132630 14622 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:35.197501 13719 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:35.228474 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:35.236025 10797 raft_consensus.cc:1240] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 13:17:35.232301 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:35.229566 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:35.232941 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:35.239516 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:35.191289 14665 raft_consensus.cc:2749] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:17:35.261883 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:35.221925 14043 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:35.243589 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:35.262887 10766 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:35.263882 11508 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 7.
I20260709 13:17:35.248178 14386 raft_consensus.cc:3055] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:35.251235 14555 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:35.202996 14662 consensus_queue.cc:1059] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
W20260709 13:17:35.252765 14663 log.cc:927] Time spent T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:17:35.253537 14573 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:35.283210 10766 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:17:35.253693 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 122363us
I20260709 13:17:35.257400 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:35.260200 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:35.213133 14595 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:35.234211 11503 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 9.
I20260709 13:17:35.265201 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:35.285858 11511 raft_consensus.cc:2393] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 12: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:35.286548 10942 raft_consensus.cc:2410] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:17:35.265340 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:35.264896 10943 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:17:35.267901 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 154763us
I20260709 13:17:35.268193 10867 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
I20260709 13:17:35.269312 10841 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:35.241734 13719 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:35.285075 11512 raft_consensus.cc:2393] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 9: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:35.289825 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:35.291360 10766 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:35.294258 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:35.294262 10841 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:35.266523 14386 raft_consensus.cc:740] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:17:35.286082 14479 consensus_meta.cc:287] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.682s	user 0.009s	sys 0.002s
I20260709 13:17:35.273166 14618 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:35.234733 14300 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:35.298997 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:35.297775 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:35.304400 11504 raft_consensus.cc:2393] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 9: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:35.305351 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:35.298748 14667 raft_consensus.cc:2749] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:35.309252 10841 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:35.309970 14478 raft_consensus.cc:2749] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 11. Current term is 12.
I20260709 13:17:35.299548 14555 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:35.310947 14555 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:35.311340 14555 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:35.312829 10697 catalog_manager.cc:5697] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 7 to 8, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 8 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:35.300211 14622 raft_consensus.cc:2749] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:35.299386 14043 consensus_queue.cc:1048] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:17:35.316926 10873 raft_consensus.cc:2410] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:17:35.319514 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:35.325657 10942 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:17:35.327565 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:35.285753 14572 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:35.304410 10766 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:35.295814 14503 raft_consensus.cc:2804] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:35.348248 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:35.349861 10795 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 12.
I20260709 13:17:35.346103 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:35.353446 11503 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:17:35.357872 10697 catalog_manager.cc:5697] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 6 to 9, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 9 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
W20260709 13:17:35.367146 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 64605.8us
I20260709 13:17:35.302160 14617 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:35.317651 14618 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:35.370455 10943 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 12.
I20260709 13:17:35.370362 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:35.371850 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:35.372511 10943 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
I20260709 13:17:35.347469 14479 raft_consensus.cc:2804] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:17:35.345834 14662 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:35.340507 14342 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:35.352962 14503 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:35.386183 11503 pending_rounds.cc:85] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586: Aborting all ops after (but not including) 5
I20260709 13:17:35.409150 11503 pending_rounds.cc:107] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 13:17:35.409519 11503 raft_consensus.cc:2889] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:35.391223 10870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1772) took 1089 ms. Trace:
I20260709 13:17:35.388409 14386 consensus_queue.cc:260] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:35.405958 14662 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:35.411369 14662 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:35.411746 14386 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:17:35.413678 14435 consensus_meta.cc:287] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.778s	user 0.007s	sys 0.002s
I20260709 13:17:35.414086 14435 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:35.415867 10869 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 9.
I20260709 13:17:35.415390 10697 catalog_manager.cc:5697] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 6 to 8, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 8 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:35.393786 14627 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:35.420917 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 52520.5us
I20260709 13:17:35.423358 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:35.424136 10795 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
W20260709 13:17:35.405004 14635 consensus_meta.cc:287] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.674s	user 0.007s	sys 0.004s
I20260709 13:17:35.428496 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:35.433252 14300 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:35.430056 10697 catalog_manager.cc:5697] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: leader changed from <none> to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 6 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:35.436872 10697 catalog_manager.cc:5697] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 2 to 6, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 6 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:35.439108 14573 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:35.438656 10697 catalog_manager.cc:5697] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 3 to 6, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 6 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:35.441064 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:35.443053 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:35.443217 10796 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 12.
I20260709 13:17:35.443760 10795 raft_consensus.cc:2393] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 11: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:35.444469 14573 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:35.447497 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:35.446696 14573 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:35.448139 10950 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 11.
I20260709 13:17:35.448988 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:35.449524 10950 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 11.
I20260709 13:17:35.450935 14595 consensus_queue.cc:1059] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:17:35.451215 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:35.451550 10915 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:17:35.387566 14341 consensus_meta.cc:287] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.805s	user 0.009s	sys 0.002s
I20260709 13:17:35.451831 10793 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 10.
I20260709 13:17:35.452250 14341 raft_consensus.cc:2749] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:35.452122 14665 consensus_queue.cc:1059] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:17:35.453251 14362 raft_consensus.cc:3055] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:35.453603 14362 raft_consensus.cc:740] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:35.341475 14134 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:17:35.454394 14362 consensus_queue.cc:260] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:35.455722 14362 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:35.456660 10797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2271) took 1098 ms. Trace:
I20260709 13:17:35.457031 10797 rpcz_store.cc:276] 0709 13:17:34.358575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:34.627325 (+268750us) service_pool.cc:224] Handling call
0709 13:17:34.659165 (+ 31840us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:35.236553 (+577388us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:35.456399 (+219846us) spinlock_profiling.cc:243] Waited 576 ms on lock 0x7b5400212ad4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:35.456597 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1325930240}
I20260709 13:17:35.366973 14392 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:35.372516 14618 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:35.344666 14572 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:35.458959 14336 raft_consensus.cc:3055] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:35.450559 14667 consensus_queue.cc:1048] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.009s
I20260709 13:17:35.461678 14572 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:35.462353 10945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2211) took 1702 ms. Trace:
I20260709 13:17:35.396251 10867 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 12.
I20260709 13:17:35.399447 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:35.397992 10916 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:17:35.392009 14630 consensus_meta.cc:287] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.579s	user 0.014s	sys 0.000s
I20260709 13:17:35.451720 14595 consensus_queue.cc:1243] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer 12682eb70bc84c76879e04a8628ee586 log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:17:35.459407 14336 raft_consensus.cc:740] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:17:35.401584 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 91235.2us
W20260709 13:17:35.481246 14650 log.cc:927] Time spent T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.604s	user 0.003s	sys 0.007s
I20260709 13:17:35.470685 10697 catalog_manager.cc:5697] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 5 to 7, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 7 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:35.482995 14336 consensus_queue.cc:260] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:35.483955 14630 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:35.485121 10841 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:35.485718 10942 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:17:35.488633 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:17:35.488799 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 98988.1us
I20260709 13:17:35.532634 10794 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:17:35.440387 14635 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:35.417236 14503 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:35.467320 14367 consensus_meta.cc:287] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.644s	user 0.004s	sys 0.006s
I20260709 13:17:35.491688 14669 raft_consensus.cc:3055] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:17:35.553627 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:35.495085 14618 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:35.495759 14362 raft_consensus.cc:2804] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:35.496258 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:35.556072 10941 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:35.497416 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:35.558561 10872 raft_consensus.cc:1275] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:17:35.500923 14627 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:35.501613 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:35.558986 10872 pending_rounds.cc:85] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 5
I20260709 13:17:35.559257 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:35.503342 11509 raft_consensus.cc:1240] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:35.504949 14435 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:35.506137 14549 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:35.507277 14645 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:35.508869  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:35.547030 14680 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:35.508008 11477 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:35.392647 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:35.572804 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:35.573349 10943 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 7.
I20260709 13:17:35.574101 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:35.527034 10697 catalog_manager.cc:5697] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 8 to 9, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 9 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:35.530758 14336 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:35.576375 10943 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
I20260709 13:17:35.576462 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:35.491701 14595 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:35.495124 14388 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:35.462747 10945 rpcz_store.cc:276] 0709 13:17:33.760342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:33.776631 (+ 16289us) service_pool.cc:224] Handling call
0709 13:17:33.776952 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:34.255929 (+478977us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:34.255961 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:34.285193 (+ 29232us) log.cc:844] Serialized 26 byte log entry
0709 13:17:35.461788 (+1176595us) spinlock_profiling.cc:243] Waited 1.58 ms on lock 0x7b180040fc60. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c80a5d4bc 00007f9c7ddab48b 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a550d1
0709 13:17:35.461933 (+   145us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:17:35.461969 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:35.462040 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:35.462078 (+    38us) raft_consensus.cc:1685] finished
0709 13:17:35.462088 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:35.462296 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":134083,"raft.queue_time_us":186616,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":7084,"spinlock_wait_cycles":3631360,"thread_start_us":1026267,"threads_started":2,"wal-append.queue_time_us":676310}
W20260709 13:17:35.498889 10800 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 1749) took 1456 ms (client timeout 1529 ms). Trace:
I20260709 13:17:35.578420 10871 raft_consensus.cc:2393] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 9: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:35.560977 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:35.578892 10767 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:35.579208 10944 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
I20260709 13:17:35.580014 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:35.580585 10944 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 9.
I20260709 13:17:35.580972 10767 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:35.581344 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:35.559266 10872 pending_rounds.cc:107] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:17:35.581959 10872 raft_consensus.cc:2889] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:35.462496 14555 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:35.426404 14342 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:35.585070 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:35.586047 10794 raft_consensus.cc:2410] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:17:35.526604 14479 raft_consensus.cc:697] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:35.528957 14638 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:35.564497 10946 raft_consensus.cc:1240] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:35.555667 14669 raft_consensus.cc:740] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:35.599134 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:35.410568 10870 rpcz_store.cc:276] 0709 13:17:34.301739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:34.301847 (+   108us) service_pool.cc:224] Handling call
0709 13:17:34.314724 (+ 12877us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:34.314965 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:17:34.633026 (+318061us) log.cc:844] Serialized 14 byte log entry
0709 13:17:34.937207 (+304181us) log.cc:844] Serialized 14 byte log entry
0709 13:17:35.308284 (+371077us) spinlock_profiling.cc:243] Waited 8.22 ms on lock 0x7b1800463e40. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7dd962a6 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9
0709 13:17:35.309732 (+  1448us) spinlock_profiling.cc:243] Waited 25.5 ms on lock 0x7b1800463e40. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc5455 00007f9c80ead060 00007f9c80ebf716 00007f9c80ed0287 00007f9c80ed0208
0709 13:17:35.390866 (+ 81134us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:35.390924 (+    58us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:35.390978 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:35.391048 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:35.391167 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17767,"prepare.queue_time_us":338057,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":101,"raft.queue_time_us":42669,"raft.run_cpu_time_us":272,"raft.run_wall_time_us":14529,"spinlock_wait_cycles":77485056,"thread_start_us":635145,"threads_started":3,"wal-append.queue_time_us":402190}
I20260709 13:17:35.554411 14506 raft_consensus.cc:2804] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:35.571127 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:35.534368 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 49337.2us
I20260709 13:17:35.534937 10916 leader_election.cc:400] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 election: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 10. Current term is 11.
I20260709 13:17:35.602142 10916 leader_election.cc:403] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:17:35.570911 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:35.555851 14362 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:35.578476 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:35.604657 10945 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
W20260709 13:17:35.578912 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 70967.6us
W20260709 13:17:35.578301 10800 rpcz_store.cc:269] 0709 13:17:34.042079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:34.043152 (+  1073us) service_pool.cc:224] Handling call
0709 13:17:35.498674 (+1455522us) spinlock_profiling.cc:243] Waited 354 ms on lock 0x7b54000bd1d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:35.498834 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":814975360}
W20260709 13:17:35.605122 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 72062us
I20260709 13:17:35.584045 10767 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:35.458510 13719 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:35.606395 10916 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:17:35.579434 14592 consensus_meta.cc:287] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.619s	user 0.010s	sys 0.001s
I20260709 13:17:35.585847 10697 catalog_manager.cc:5697] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 2 to 7, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 7 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:17:35.606046 11477 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:35.585634 14440 raft_consensus.cc:2804] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:17:35.619266 10766 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:35.465973 14304 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:35.599466 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:35.601101 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:35.561606 14367 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:35.560353 14627 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:35.607801 10950 raft_consensus.cc:2393] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 9: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:35.644511 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:35.585601 10842 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:35.585127 14392 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:35.609942 10767 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:35.610888 10869 raft_consensus.cc:2410] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:17:35.588104 14391 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:35.585084 14386 consensus_queue.cc:1059] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:17:35.651754 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:35.652740 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:35.653505 10870 raft_consensus.cc:2393] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 11: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:35.654028 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:35.654263 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:35.654842 10868 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:35.575975 14662 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:35.656931 10697 catalog_manager.cc:5697] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 4 to 5, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 5 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
W20260709 13:17:35.613981 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:35.614288 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:35.614008 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:35.608381 14549 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:35.644090 14624 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:35.624953 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:35.626964 11478 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:35.665808 11478 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:35.651841 14331 raft_consensus.cc:2749] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:17:35.616950 14506 raft_consensus.cc:697] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:35.671468 10872 raft_consensus.cc:1275] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:17:35.615657 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:35.646487 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 60961.7us
I20260709 13:17:35.639729 10943 raft_consensus.cc:2393] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 11: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:35.649266 10767 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:35.649183 10766 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:35.609344 14362 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:35.654058 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:35.612540 14617 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:35.648244 14392 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:35.655026 14623 raft_consensus.cc:2749] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:35.672448 10871 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 9.
I20260709 13:17:35.667460 10949 raft_consensus.cc:1240] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 10. Current term is 11. Ops: []
I20260709 13:17:35.675791 10767 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:35.676085 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:35.602939 14638 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:35.637398 10941 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 9.
I20260709 13:17:35.693657 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:35.694253 10941 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 9.
I20260709 13:17:35.655602 14572 raft_consensus.cc:2804] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:35.638741 14676 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:35.587114 14680 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:35.682228 11478 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:35.683658 10948 raft_consensus.cc:2393] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 9: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:17:35.816803 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 134703us
I20260709 13:17:35.816964 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:35.817869 10948 raft_consensus.cc:2410] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:17:35.651993 14391 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:35.644217 14668 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:35.652822 14440 raft_consensus.cc:697] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 12 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:35.673882 11506 raft_consensus.cc:1275] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:17:35.611850 14667 consensus_queue.cc:1059] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 13:17:35.692106 10943 raft_consensus.cc:1275] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:17:35.701395 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:35.702049 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:35.680249 10916 connection.cc:256] RPC call timeout handler was delayed by -2.3009397339999396s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23210000000000003s for a response
I20260709 13:17:35.834334 10870 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 11.
I20260709 13:17:35.703568 10950 raft_consensus.cc:1240] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:35.639160 14592 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:35.675092 14606 raft_consensus.cc:2749] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:17:35.704598 10766 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:35.680774 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:35.683153 14275 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:17:35.684427 14617 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:35.855785 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:35.704712 14572 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:35.707813 14386 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:35.691452 14634 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:35.701086 14331 raft_consensus.cc:2749] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:35.687726 14392 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:35.738083 14506 consensus_queue.cc:237] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:35.842041 14305 raft_consensus.cc:3055] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:35.888307 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:35.679527 14624 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:35.775879 13719 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:35.760746 14388 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:35.789664 10868 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 11.
I20260709 13:17:35.890419 14555 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:35.736770 14479 consensus_queue.cc:237] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:35.737771 14630 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:35.705874 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:35.798919 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:35.818676 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:35.948511 10948 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:35.834895 10943 raft_consensus.cc:1240] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:35.790939 14635 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:35.759974 14508 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:35.868588 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:35.960719 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:35.960445 14508 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:35.961494 10797 raft_consensus.cc:2393] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 10: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:35.962347 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:35.834015 14391 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:35.963200 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:35.964035 10798 raft_consensus.cc:2393] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 11: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:35.964870 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:35.965919 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:35.966470 10798 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:17:35.879140 10947 raft_consensus.cc:1240] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:35.687516 14671 raft_consensus.cc:2804] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:35.967723 10875 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 9.
I20260709 13:17:35.677618 14669 consensus_queue.cc:260] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:35.858431 14617 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:35.899286 14662 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:35.904134 14367 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:35.908761 14622 raft_consensus.cc:3055] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:17:36.003495 14622 raft_consensus.cc:740] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:36.003894 14508 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:36.004376 14622 consensus_queue.cc:260] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:36.005976 14622 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:35.909538 10874 raft_consensus.cc:2393] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 9: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:35.910408 14627 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:35.911682 11509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56410 (request call id 655) took 1499 ms. Trace:
I20260709 13:17:36.007434 11509 rpcz_store.cc:276] 0709 13:17:34.412624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:34.418235 (+  5611us) service_pool.cc:224] Handling call
0709 13:17:34.475774 (+ 57539us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:35.561977 (+1086203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:35.911383 (+349406us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b5400691c54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:35.911606 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2362687744}
I20260709 13:17:35.911847 14592 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:36.009063 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:35.863833 14572 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:36.009919 10798 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:17:35.917495 10942 raft_consensus.cc:1275] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:17:35.875486 14623 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:17:35.922474 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 217938us
W20260709 13:17:36.013674 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:35.835292 14668 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:36.014986 14668 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:35.864387 14275 raft_consensus.cc:697] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:36.016166 14680 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:36.016346 14275 consensus_queue.cc:237] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:36.017282 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:36.018340 10799 raft_consensus.cc:2393] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 10: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:35.875316 14618 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:35.931198 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:35.931785 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:36.019807 10799 raft_consensus.cc:1240] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:36.020160 11507 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 10: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:35.930867 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:35.708201 14645 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:35.883098 14607 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:36.021505 14607 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:36.022230 14671 raft_consensus.cc:697] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 9 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:35.933609 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:35.890908 14305 raft_consensus.cc:740] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:36.022770 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:36.023452 10943 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 7.
I20260709 13:17:35.892191 14606 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:36.023645 14305 consensus_queue.cc:260] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:36.023514 14671 consensus_queue.cc:237] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:36.024353 14606 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:36.024875 14305 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:35.890053 14634 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:35.921146 11478 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:36.027700 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 106665us
I20260709 13:17:35.975287 10797 raft_consensus.cc:2393] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 11: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:36.031750 14607 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:36.031968 14701 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:36.035429 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:36.036226 10942 raft_consensus.cc:2393] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 8: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:35.953747 14676 raft_consensus.cc:697] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 9 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:36.037333 14362 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:36.038235 14676 consensus_queue.cc:237] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:36.039543 14362 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:36.040572 10944 raft_consensus.cc:2393] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 12: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:35.982450 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:36.042030 10792 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 9.
I20260709 13:17:35.982643 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:17:35.983724 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:35.993718 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:36.043092 10794 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 9.
I20260709 13:17:35.994680 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:35.934973 14638 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:36.001353 10793 raft_consensus.cc:1240] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:17:36.002851 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:35.863982 14435 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:17:36.044351 10798 raft_consensus.cc:1240] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:17:36.044433 10766 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:36.042941 10796 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 9.
I20260709 13:17:36.055615 10948 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 10.
I20260709 13:17:36.076632 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:36.026538 14606 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:36.080929 10842 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:36.081866 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:36.084455 10766 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:36.077548 10943 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 11.
I20260709 13:17:36.093874 10842 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:36.098801 10916 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:36.099404 11477 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:36.097708 10766 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:36.107229 10842 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:36.110162 10766 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:36.111902 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:36.122679 13719 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:36.123294 13719 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:36.123694 13719 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:36.098961 14435 raft_consensus.cc:697] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 12 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:36.133385 14435 consensus_queue.cc:237] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:36.138195 14688 log.cc:927] Time spent T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.353s	user 0.002s	sys 0.004s
I20260709 13:17:36.088091 14549 raft_consensus.cc:2749] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 10. Current term is 11.
I20260709 13:17:35.935021 14440 consensus_queue.cc:237] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:36.124759 10842 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:36.069123 14692 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:36.100587 14592 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:36.101120 14305 raft_consensus.cc:2749] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:36.090562 14362 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:36.121461 14680 raft_consensus.cc:2749] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:17:36.147423 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 66927.4us
I20260709 13:17:36.120870 13995 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:36.091336 14643 raft_consensus.cc:2749] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:17:36.151558 10842 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:36.140244 14386 raft_consensus.cc:697] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:17:36.166194 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 553816us
I20260709 13:17:36.125511 14627 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:36.120569 14669 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:36.143724 10766 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:36.149937 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 120855us
I20260709 13:17:36.149396 14662 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:36.178294 10766 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:36.139381 14607 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:36.150557 14367 raft_consensus.cc:2749] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:17:36.141993 14503 consensus_meta.cc:287] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.581s	user 0.011s	sys 0.000s
I20260709 13:17:36.172269 10916 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:36.173202 10867 raft_consensus.cc:1240] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:36.167577 14627 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:36.185642 14627 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:36.167275 10841 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:17:36.163846 13995 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:36.157918 14508 raft_consensus.cc:2804] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:36.184161 10949 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:36.168740 14675 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:36.171424 13719 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:36.185130 14643 raft_consensus.cc:2749] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:17:36.144322 14674 log.cc:927] Time spent T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:17:36.177742 14511 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:36.195734 10766 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:36.219789 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:36.220439 11512 raft_consensus.cc:1240] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:36.197182 10916 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:36.161841 14592 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:36.222306 13719 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:36.201496 10841 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:36.209139 14624 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:17:36.213624 14595 consensus_meta.cc:287] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.625s	user 0.004s	sys 0.003s
I20260709 13:17:36.186872 14503 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:36.178980 14607 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:36.195735 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:36.225703 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:36.222193 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:36.164356 14362 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:36.191341 13995 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:36.227505 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:36.420436 10948 raft_consensus.cc:2393] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 12: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:36.224030 14511 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:36.197234 14336 consensus_queue.cc:1059] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:17:36.217263 14680 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:36.241250 14342 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:36.218032 14662 raft_consensus.cc:2749] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:36.160085 14692 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:36.268105 14549 raft_consensus.cc:2749] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:36.386147 14643 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:36.190289 14043 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:36.266343 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:36.292878 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:36.255435 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:36.453491 10950 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
I20260709 13:17:36.415925 14595 raft_consensus.cc:2749] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:36.342247 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:36.275563 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:36.257952 10946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2212) took 1663 ms. Trace:
I20260709 13:17:36.256702 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:36.457080 10795 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:17:36.457629 11505 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:17:36.458137 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:36.246475 14617 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:36.458760 10795 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
I20260709 13:17:36.314164 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:17:36.244269 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 47159.5us
I20260709 13:17:36.459862 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:36.459923 10916 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:36.242689 10949 raft_consensus.cc:1275] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:17:36.460419 14617 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:36.241276 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 45605.8us
I20260709 13:17:36.243537 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:36.301683 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:36.461627 11512 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 10: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:36.190433 14702 raft_consensus.cc:2749] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:36.462147 10916 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:36.334223 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:36.406564 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:36.420228 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:36.408267 11507 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:17:36.405405 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:36.464457 10798 raft_consensus.cc:1275] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:17:36.385552 10945 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
I20260709 13:17:36.427706 10875 raft_consensus.cc:1275] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:17:36.427830 10792 raft_consensus.cc:1240] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:36.428591 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:36.434711 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:36.435703 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:36.295969 14684 log.cc:927] Time spent T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.511s	user 0.011s	sys 0.000s
I20260709 13:17:36.437316 10947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2219) took 1670 ms. Trace:
W20260709 13:17:36.286965 14683 log.cc:927] Time spent T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.511s	user 0.009s	sys 0.000s
I20260709 13:17:36.266662 14708 raft_consensus.cc:2804] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:36.441941 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:36.443051 14362 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:36.443797 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:36.475117 10947 rpcz_store.cc:276] 0709 13:17:34.766956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:34.767097 (+   141us) service_pool.cc:224] Handling call
0709 13:17:34.767376 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:35.967276 (+1199900us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:36.437035 (+469759us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54003f0054. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:36.437253 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2549851392}
I20260709 13:17:36.475826 10948 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 10: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:36.444020 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:36.434544 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:36.451109 14667 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.011s	sys 0.004s
I20260709 13:17:36.266930 14638 raft_consensus.cc:2804] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:36.190366 14508 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:36.272467 14592 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:36.301944 14634 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:36.421100 14607 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:36.457249 10946 rpcz_store.cc:276] 0709 13:17:34.594690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:34.603017 (+  8327us) service_pool.cc:224] Handling call
0709 13:17:34.634856 (+ 31839us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:35.601607 (+966751us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:36.257709 (+656102us) spinlock_profiling.cc:243] Waited 929 ms on lock 0x7b5400174654. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:36.257891 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2137517440}
I20260709 13:17:36.461526 10766 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:36.462386 14702 raft_consensus.cc:2749] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:36.229741 10841 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:36.434454 14511 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:36.390328 10767 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:36.457116 11504 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 11.
I20260709 13:17:36.468675 10795 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:17:36.472995 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:36.466349 10793 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:17:36.528792 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 138578us
W20260709 13:17:36.251895 14698 log.cc:927] Time spent T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.254s	user 0.001s	sys 0.000s
I20260709 13:17:36.476421 10797 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 10.
I20260709 13:17:36.256347 14665 raft_consensus.cc:3055] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:17:36.295395 14574 consensus_queue.cc:1059] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.158s
I20260709 13:17:36.475769 10870 raft_consensus.cc:2393] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 11: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:36.479140 10944 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 11.
I20260709 13:17:36.479744 14362 raft_consensus.cc:2764] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:17:36.243727 14392 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:36.198335 14675 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:36.540755 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:36.295063 14391 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:36.261919 14572 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:36.452347 14630 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:36.457244 14043 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:36.473366 14342 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:36.550534 14043 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:36.550565 14630 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:36.550745 14342 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:36.553325 14342 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:36.491755 10916 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:36.533192 14671 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:36.484016 14708 raft_consensus.cc:697] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:36.560427 14675 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:36.562093 10800 raft_consensus.cc:2368] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 12. Current term is 13.
I20260709 13:17:36.534842 14617 consensus_queue.cc:1048] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
W20260709 13:17:36.563449 14713 log.cc:927] Time spent T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 13:17:36.565563 14671 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:36.566211 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:36.538877 14402 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:36.452800 14680 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:36.535989 14665 raft_consensus.cc:740] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:36.570685 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:36.571378 10950 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 11.
W20260709 13:17:36.494808 14667 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.011s	sys 0.004s
W20260709 13:17:36.526962 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 297307us
W20260709 13:17:36.531248 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 69794.9us
I20260709 13:17:36.532277 10871 raft_consensus.cc:1240] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:36.533828 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:36.591477 10797 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 9.
I20260709 13:17:36.591779 10766 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:36.534857 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:36.535344 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:36.537235 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:36.538695 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:36.533740 14627 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:36.607028 10793 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 10.
W20260709 13:17:36.617509 14712 log.cc:927] Time spent T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.174s	user 0.002s	sys 0.000s
I20260709 13:17:36.637428 10876 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 8.
I20260709 13:17:36.545753 10795 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 11.
I20260709 13:17:36.501981 14638 raft_consensus.cc:697] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:36.548188 11478 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:36.536373 14668 consensus_meta.cc:287] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.521s	user 0.005s	sys 0.011s
W20260709 13:17:36.554493 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 62895.7us
I20260709 13:17:36.255041 14386 consensus_queue.cc:237] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:36.535101 14662 raft_consensus.cc:2804] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:36.533957 14506 raft_consensus.cc:2804] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:36.534971 14409 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:36.557399 11508 raft_consensus.cc:1240] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 12 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 11. Current term is 12. Ops: [7.8-11.10]
I20260709 13:17:36.470682 14643 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:36.573742 14680 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:36.573899 14692 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:36.556107 14634 raft_consensus.cc:3055] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:17:36.558398 14708 consensus_queue.cc:237] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:36.574581 14607 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:36.575733 14503 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:36.579849 14671 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:36.583704 14595 raft_consensus.cc:2749] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:36.583739 14617 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:36.584786 14665 consensus_queue.cc:260] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:36.587085 11507 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 9.
I20260709 13:17:36.498226 14508 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:36.684324 10794 raft_consensus.cc:1240] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:36.658859 10916 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:36.933935 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 385812us
W20260709 13:17:36.840368 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 250723us
I20260709 13:17:36.939539 11478 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:36.939747 10841 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:36.941874 10841 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:36.658429 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:36.937036 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 345315us
I20260709 13:17:36.942975 10766 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:36.943967 11478 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:36.643217 14336 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:36.657029 14622 consensus_meta.cc:287] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.651s	user 0.011s	sys 0.002s
W20260709 13:17:36.871528 14714 log.cc:927] Time spent T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.231s	user 0.000s	sys 0.000s
I20260709 13:17:36.946278 10841 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:36.855432 14722 log.cc:927] Time spent T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.269s	user 0.001s	sys 0.000s
W20260709 13:17:36.948809 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 290062us
I20260709 13:17:36.948644 14511 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:36.949271 14435 consensus_queue.cc:1048] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:36.949990 14721 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:17:36.950280 14676 raft_consensus.cc:2749] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:36.875706 14572 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:36.934103 14506 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:36.974223 14506 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:36.974279 14721 raft_consensus.cc:697] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 11 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:36.975323 14721 consensus_queue.cc:237] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:36.977202 14043 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:36.978662 14630 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:36.667829 14446 raft_consensus.cc:2749] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:17:36.938624 14662 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:36.934577 14634 raft_consensus.cc:740] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:36.884171 14627 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:36.871827 14391 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:36.870828 14600 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:36.946612 14574 raft_consensus.cc:2749] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:36.972604 10841 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:37.023182 14635 raft_consensus.cc:3055] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:17:37.023588 14635 raft_consensus.cc:740] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:17:37.023620 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 51051.8us
I20260709 13:17:37.024520 14635 consensus_queue.cc:260] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:36.968240 14134 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:36.973207 14676 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:37.027271 14676 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:37.029358 14676 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:36.981549 10916 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:36.946813 14671 raft_consensus.cc:2749] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:36.951606 14503 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:37.030431 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 48956.4us
I20260709 13:17:37.030524 14503 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:37.030978 14595 raft_consensus.cc:2804] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:37.031234 10916 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:37.031459 14595 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:36.846594 13995 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:36.955675 14665 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:36.962777 14409 raft_consensus.cc:2749] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:17:36.988678 10873 raft_consensus.cc:2368] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 9. Current term is 10.
I20260709 13:17:36.950350 14342 consensus_queue.cc:1059] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 10.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 13:17:36.681005 14668 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:37.047206 14304 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:37.047289 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:37.048022 11511 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:37.048707 14668 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:37.021327 14511 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:37.050927 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:37.042156 14503 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:36.984126 14662 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:37.032958 10916 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:36.945930 14680 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:37.063225 10916 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:37.064198 10916 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:37.059516 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:37.067800 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:17:37.070930 10916 connection.cc:256] RPC call timeout handler was delayed by -1.4189611480001076s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:17:37.036294 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:36.884565 14638 consensus_queue.cc:237] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:37.043442 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:37.044727 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:36.957842 14336 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:36.983072 14043 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:37.042598 14667 consensus_queue.cc:1048] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 13:17:37.125213 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:37.125771 14336 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:37.126502 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:17:37.051095 14716 log.cc:927] Time spent T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.304s	user 0.001s	sys 0.001s
I20260709 13:17:36.944121 14617 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:36.987790 14669 raft_consensus.cc:2749] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:37.031901 14595 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:37.141515 14617 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:37.142292 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:37.052628 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:37.143738 10944 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
I20260709 13:17:37.021834 13719 raft_consensus.cc:3055] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:37.144573 13719 raft_consensus.cc:740] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:17:37.055887 10874 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 1772) took 1432 ms (client timeout 1757 ms). Trace:
W20260709 13:17:37.145387 10874 rpcz_store.cc:269] 0709 13:17:35.623790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:35.625610 (+  1820us) service_pool.cc:224] Handling call
0709 13:17:37.055664 (+1430054us) spinlock_profiling.cc:243] Waited 254 ms on lock 0x7b540009a6d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:37.055813 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":585171840}
W20260709 13:17:37.057137 10801 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 1754) took 3007 ms (client timeout 2138 ms). Trace:
W20260709 13:17:37.146433 10801 rpcz_store.cc:269] 0709 13:17:34.044763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:34.068690 (+ 23927us) service_pool.cc:224] Handling call
0709 13:17:37.051870 (+2983180us) spinlock_profiling.cc:243] Waited 2.33 s on lock 0x7b54003bd3d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:37.052001 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5363838720}
I20260709 13:17:37.058732 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:37.060952 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:37.042733 14692 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:37.061677 14392 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:37.072630 10942 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:17:37.073329 14362 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:37.073568 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:37.073859 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:37.074158 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:37.074394 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:37.043819 14574 raft_consensus.cc:2749] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:37.123611 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:37.045410 14635 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:37.185402 10872 raft_consensus.cc:2393] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 11: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:17:37.136382 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:37.187153 10945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41014 (request call id 1806) took 1697 ms. Trace:
I20260709 13:17:37.136445 10873 raft_consensus.cc:2368] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 12. Current term is 13.
I20260709 13:17:37.147861 10950 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
I20260709 13:17:37.187538 10945 rpcz_store.cc:276] 0709 13:17:35.489543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:35.613686 (+124143us) service_pool.cc:224] Handling call
0709 13:17:37.187007 (+1573321us) spinlock_profiling.cc:243] Waited 488 ms on lock 0x7b54001e2b54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:37.187099 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1121405184}
I20260709 13:17:37.143127 14634 consensus_queue.cc:260] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:37.172302 13719 consensus_queue.cc:260] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:37.181258 11477 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:37.139179 14702 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:17:37.182327 14676 consensus_queue.cc:1048] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:37.127372 14549 raft_consensus.cc:2749] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:37.062218 14680 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:37.041335 14600 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:37.208691 14680 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:37.210139 14600 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:37.193704 14508 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:37.212565 14508 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:37.195294 14506 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:37.196372 14643 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:37.173234 14692 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:37.215320 14692 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:37.216167 14607 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:37.214578 14506 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:37.174942 14362 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:37.223886 14362 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:37.224881 14692 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:37.198426 14702 raft_consensus.cc:697] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 11 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:37.203230 13719 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:37.203051 11504 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8"
dest_uuid: "12682eb70bc84c76879e04a8628ee586"
mode: GRACEFUL
new_leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
 from {username='slave'} at 127.0.0.1:57818
I20260709 13:17:37.266160 11504 raft_consensus.cc:606] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Received request to transfer leadership to TS e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:37.266538 11504 raft_consensus.cc:614] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:37.227507 11511 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 10.
I20260709 13:17:37.233243 14634 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:17:37.234776 14720 log.cc:927] Time spent T 00000000000000000000000000000000 P ae957c8fc66c4b6ba063d0c74cbf5484: Append to log took a long time: real 0.671s	user 0.000s	sys 0.015s
I20260709 13:17:37.235448 10800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2296) took 1254 ms. Trace:
I20260709 13:17:37.262562 10841 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:37.269690 10800 rpcz_store.cc:276] 0709 13:17:35.981252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:36.002504 (+ 21252us) service_pool.cc:224] Handling call
0709 13:17:37.235224 (+1232720us) spinlock_profiling.cc:243] Waited 516 ms on lock 0x7b54000d9654. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:37.235379 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1186694912}
W20260709 13:17:37.240403 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 59265.5us
I20260709 13:17:37.240553 14624 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:37.197095 14134 raft_consensus.cc:2749] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:17:37.258975 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 187322us
W20260709 13:17:37.260138 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:37.226480 13995 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:37.271044 10841 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:37.273702 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:37.281270 10945 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 10.
I20260709 13:17:37.274541 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:37.275637 10942 raft_consensus.cc:1275] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:17:37.276132 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:37.255342 14595 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:37.282838 10943 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 10.
I20260709 13:17:37.285465 10947 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:17:37.287743 14600 consensus_queue.cc:1048] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 13:17:37.289814 14727 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:37.301025 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:37.302119 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:37.304016 10841 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:17:37.304549 14336 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:37.326901 14336 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:37.327342 14511 raft_consensus.cc:3055] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:17:37.327816 14511 raft_consensus.cc:740] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:37.328688 14511 consensus_queue.cc:260] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.4, Last appended by leader: 4, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:37.329123 14336 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:37.305178 14731 raft_consensus.cc:2804] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:37.329957 14511 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:37.330338 14731 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:37.330740 14731 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:37.320262 14667 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:37.261159 14506 consensus_queue.cc:1059] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 13:17:37.306036 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:37.306321 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:37.340520 10795 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:37.340520 10801 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:37.307027 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:37.307310 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:37.341741 10793 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 8.
I20260709 13:17:37.308280 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:37.342531 10797 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 11.
I20260709 13:17:37.308559 10871 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:17:37.343364 10871 raft_consensus.cc:606] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Received request to transfer leadership to TS cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:37.343696 10871 raft_consensus.cc:614] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:37.309252 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:37.309573 14662 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:37.312906 14680 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:37.315577 14734 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:37.276499 14624 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:37.317338 10916 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:37.318811 10941 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:17:37.318693 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 374773us
I20260709 13:17:37.294726 11477 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:37.348093 11478 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:37.283370 10950 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:37.285317 14595 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:17:37.334537 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 391624us
I20260709 13:17:37.305513 14702 consensus_queue.cc:237] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:37.354341 14734 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:37.356609 14734 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:37.334893 10841 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:17:37.358942 14699 raft_consensus.cc:2804] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:37.359404 14699 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:37.359730 14699 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:37.361045 10841 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:37.361334 10941 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 11.
I20260709 13:17:37.362534 14572 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:37.363050 14572 raft_consensus.cc:697] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:37.363933 14572 consensus_queue.cc:237] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:37.364363 14735 consensus_queue.cc:1059] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:17:37.333659 14667 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:37.365473 14735 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:37.366539 14667 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:37.366963 14479 consensus_queue.cc:1059] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:37.367436 11504 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "741efe79e97646b396f0091dc10c745e"
dest_uuid: "12682eb70bc84c76879e04a8628ee586"
mode: GRACEFUL
new_leader_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:57818
I20260709 13:17:37.368045 11504 raft_consensus.cc:606] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 9 LEADER]: Received request to transfer leadership to TS cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:37.369051 14479 raft_consensus.cc:3055] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:37.369468 14479 raft_consensus.cc:740] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:37.277215 14638 raft_consensus.cc:2749] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:17:37.371173 14634 raft_consensus.cc:3055] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:37.345196 10799 raft_consensus.cc:2368] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 12. Current term is 13.
I20260709 13:17:37.371563 14634 raft_consensus.cc:740] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:37.371832 14692 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:17:37.348243 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 53611.6us
I20260709 13:17:37.372473 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:37.372423 14634 consensus_queue.cc:260] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:37.373212 10947 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:17:37.349900 14600 raft_consensus.cc:2764] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:17:37.373700 14634 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:37.374135 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:37.375309 14506 raft_consensus.cc:3055] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:37.375461 10943 raft_consensus.cc:1275] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:17:37.375758 14506 raft_consensus.cc:740] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:37.350919 14680 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:37.376554 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:37.377166 10943 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
I20260709 13:17:37.379086 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:37.379638 10942 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:37.383195 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:37.383862 10945 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:17:37.292021 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:37.350572 14728 consensus_queue.cc:1048] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:17:37.352623 14624 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:37.339020 14043 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:37.400337 14043 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:37.400740 14043 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:37.364256 14662 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:37.331753 14635 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:37.403308 14635 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:37.405411 14635 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:37.349293 11478 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:37.406316 14600 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:17:37.406528 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 57319.6us
I20260709 13:17:37.410302 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:37.410811 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:37.411381 10869 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:37.411470 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:37.412084 10876 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 10.
I20260709 13:17:37.413337 10876 raft_consensus.cc:1275] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:17:37.415311 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:37.415736 13719 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:37.415901 10871 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:37.416198 13719 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:37.370296 14479 consensus_queue.cc:260] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:37.351416 10916 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:37.418022 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 66668.9us
I20260709 13:17:37.418758 10916 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:37.376688 14506 consensus_queue.cc:260] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:37.387550 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:37.420967 10949 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 11.
I20260709 13:17:37.388233 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:37.421689 10948 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:37.388336 10801 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 10.
I20260709 13:17:37.421798 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:37.424214 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 61661us
I20260709 13:17:37.424717 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:37.425040 10841 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:37.393501 14726 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:37.393932 14386 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:37.351737 14275 consensus_queue.cc:1048] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:17:37.439292 10799 raft_consensus.cc:1240] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:37.440398 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:37.441097 10799 raft_consensus.cc:2393] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 10: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:37.440781 14624 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:37.441907 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:37.442631 10799 raft_consensus.cc:2393] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 13: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:37.338280 14362 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:37.395556 11504 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb"
dest_uuid: "12682eb70bc84c76879e04a8628ee586"
mode: GRACEFUL
new_leader_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:57818
I20260709 13:17:37.458432 11504 raft_consensus.cc:606] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 8 LEADER]: Received request to transfer leadership to TS cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:37.374706 10947 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:37.399454 10950 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 12.
I20260709 13:17:37.425294 10801 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:37.425912 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:37.389667 10795 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 12.
I20260709 13:17:37.427376 10872 raft_consensus.cc:2393] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 11: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:37.419982 10916 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:37.432435 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:37.433445 10868 raft_consensus.cc:1240] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:17:37.468954 10869 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 13.
I20260709 13:17:37.433897 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:37.470155 10869 raft_consensus.cc:1240] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:37.435993 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:37.471001 10874 raft_consensus.cc:2393] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 10: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:37.436686 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:37.472002 11477 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:37.436832 14699 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:37.437278 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:37.437748 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:37.436348 14446 consensus_queue.cc:1059] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:17:37.487905 11478 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:37.489500 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 69611.4us
I20260709 13:17:37.438359 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:37.438292 14386 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:37.333160 14727 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:37.450933 14697 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:37.406862 14600 raft_consensus.cc:697] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 10 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:37.453974 14506 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:37.454936 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:37.455201 14479 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:37.455561 14726 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:37.434747 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:37.472342 10870 raft_consensus.cc:2410] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 10 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:17:37.473250 10797 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:17:37.437893 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:37.478274 10945 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:17:37.484637 10801 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 10.
I20260709 13:17:37.532831 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:37.457497 14692 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:37.465353 10841 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:37.492394 10793 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
I20260709 13:17:37.351727 10766 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:37.493605 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:37.490465 10916 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:37.507885 13719 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:37.508096 10950 raft_consensus.cc:1275] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:17:37.509238 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:37.490943 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 62731.3us
I20260709 13:17:37.510113 10942 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 10.
I20260709 13:17:37.512116 10868 raft_consensus.cc:1275] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:17:37.513558 10873 raft_consensus.cc:2393] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 13: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:37.516819 10941 raft_consensus.cc:1275] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:17:37.518332 10949 raft_consensus.cc:1275] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:17:37.512925 14275 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:37.485839 10798 raft_consensus.cc:1240] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 11. Current term is 12. Ops: []
I20260709 13:17:37.517485 14667 raft_consensus.cc:2764] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
W20260709 13:17:37.534986 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 47214.8us
I20260709 13:17:37.512557 14634 raft_consensus.cc:2749] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:17:37.547606 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 82342.9us
I20260709 13:17:37.548113 10941 pending_rounds.cc:85] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 9
I20260709 13:17:37.549324 11504 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 10.
I20260709 13:17:37.578478 10941 pending_rounds.cc:107] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:17:37.549646 10949 pending_rounds.cc:85] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 8
I20260709 13:17:37.552414 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:37.578850 10941 raft_consensus.cc:2889] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:37.579036 10949 pending_rounds.cc:107] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:17:37.579442 10949 raft_consensus.cc:2889] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:37.578071 11508 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1"
dest_uuid: "12682eb70bc84c76879e04a8628ee586"
mode: GRACEFUL
new_leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
 from {username='slave'} at 127.0.0.1:57818
I20260709 13:17:37.580121 11508 raft_consensus.cc:606] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Received request to transfer leadership to TS e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:37.521919 10948 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
I20260709 13:17:37.580492 11508 raft_consensus.cc:614] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:37.578171 13719 consensus_queue.cc:1059] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
W20260709 13:17:37.534317 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 62437us
I20260709 13:17:37.556587 10842 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:37.518179 14721 raft_consensus.cc:2749] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:17:37.553089 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 201417us
I20260709 13:17:37.585018 10766 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:37.554587 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 64197us
I20260709 13:17:37.587765 10916 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:37.590025 10916 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:37.581606 13719 raft_consensus.cc:3055] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:37.559715 10796 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:37.582012 10842 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:37.590999 13719 raft_consensus.cc:740] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:37.592145 13719 consensus_queue.cc:260] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:37.593907 13719 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:37.615161 10842 leader_election.cc:400] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 election: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 12. Current term is 13.
I20260709 13:17:37.615644 10842 leader_election.cc:403] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:17:37.615037 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:37.616532 10792 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:17:37.532073 14435 raft_consensus.cc:3055] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:37.548172 14446 raft_consensus.cc:2749] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:37.547487 14607 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:17:37.599998 10766 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:37.618304 14336 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:37.621471 14336 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:37.621613 14643 raft_consensus.cc:2749] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:17:37.623631 14362 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:37.619051 14342 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:17:37.624846 14699 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:37.619192 14634 consensus_queue.cc:1059] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 13:17:37.619105 14665 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:37.625883 14695 consensus_queue.cc:1048] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:17:37.586195 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 205745us
I20260709 13:17:37.650380 10767 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:37.653493 14643 raft_consensus.cc:2749] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:17:37.654139 14643 consensus_queue.cc:1048] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:17:37.627506 14549 raft_consensus.cc:2804] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:37.615473 14728 raft_consensus.cc:2749] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:37.619791 14676 consensus_queue.cc:1048] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:17:37.672756 14676 consensus_queue.cc:1243] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer cb0428cc55984daba1ba63744a5102a2 log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 13:17:37.631438 14739 log.cc:927] Time spent T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.109s	user 0.002s	sys 0.000s
I20260709 13:17:37.634907 10766 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:37.556134 14572 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:37.673709 14572 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:37.674125 14572 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:37.649977 14342 raft_consensus.cc:697] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:37.675236 14342 consensus_queue.cc:237] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:37.649739 14665 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:37.676522 14665 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:37.557269 14702 raft_consensus.cc:2749] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:37.679090 10767 leader_election.cc:400] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 election: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 9. Current term is 10.
I20260709 13:17:37.680053 14728 raft_consensus.cc:2764] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election decision vote started in defunct term 9: won
I20260709 13:17:37.680500 10767 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:37.671643 14386 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:37.683589 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:37.684401 10946 raft_consensus.cc:2393] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 13: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:37.684933 10796 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:17:37.686424 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:37.687161 10948 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:37.689864 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:37.690547 10949 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:37.661250 10842 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:37.600147 10841 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:37.573194 14708 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:37.619930 14435 raft_consensus.cc:740] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:37.635216 14336 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:37.531199 14697 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:37.583367 14367 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:37.691103 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:37.719617 10941 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:37.641417 14730 consensus_queue.cc:1059] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 13:17:37.600871 10916 leader_election.cc:400] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 12. Current term is 13.
I20260709 13:17:37.727907 10916 leader_election.cc:403] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:17:37.692456 10942 raft_consensus.cc:1275] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:17:37.692935 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:37.696842 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:37.699141 10948 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 8.
I20260709 13:17:37.728503 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:17:37.737789 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 137063us
I20260709 13:17:37.738677 10916 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:37.735724 10945 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10.
I20260709 13:17:37.703377 10871 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 13.
I20260709 13:17:37.641124 14511 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:37.705192 10792 raft_consensus.cc:1275] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:17:37.707047 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 106974us
W20260709 13:17:37.707249 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 46097.2us
I20260709 13:17:37.617992 14701 consensus_queue.cc:1048] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:17:37.632066 14726 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:37.647188 14607 raft_consensus.cc:697] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 12 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:37.716310 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:37.710554 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:37.718433 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:37.683596 14386 raft_consensus.cc:2804] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:37.753068 10798 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 11.
I20260709 13:17:37.669823 14549 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:37.691726 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:37.753021  9682 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:17:37.762509 10950 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:37.723666 10794 raft_consensus.cc:1275] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:17:37.704720 14600 consensus_queue.cc:237] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:37.744928 14362 raft_consensus.cc:2804] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:37.634410 14729 consensus_queue.cc:1048] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.096s
I20260709 13:17:37.740000 10916 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:37.730532 10949 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 13.
I20260709 13:17:37.704818 14708 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:37.742551 10841 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:17:37.785955 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 43600.4us
I20260709 13:17:37.786170 10950 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10.
I20260709 13:17:37.714185 14634 raft_consensus.cc:2749] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:37.747225 10801 raft_consensus.cc:1240] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:37.748106 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:37.789413 10766 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:37.726514 14435 consensus_queue.cc:260] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:37.793215 10942 raft_consensus.cc:2393] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 10: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:37.714557 14734 raft_consensus.cc:2804] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:37.771371 14549 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:37.726962 14367 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:37.743438 14511 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:37.814999 10942 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4"
dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:41006
I20260709 13:17:37.815634 10942 raft_consensus.cc:606] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Received request to transfer leadership
I20260709 13:17:37.815965 10942 raft_consensus.cc:614] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:37.815999 14511 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:37.713817 14643 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:37.816848 14643 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:37.817478 14742 log.cc:927] Time spent T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:17:37.753234 10944 raft_consensus.cc:2410] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 10 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:17:37.721498 14676 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:37.819152 14643 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:37.819628 14676 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:37.821971 14676 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:37.578351 14275 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:37.741400 14680 raft_consensus.cc:2749] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 12. Current term is 13.
I20260709 13:17:37.741292 14702 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 12. Current term is 13.
I20260709 13:17:37.787108 13719 raft_consensus.cc:2804] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:37.824358 13719 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:37.824437 14275 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:37.824749 13719 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:37.785495 14726 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:37.789767 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:37.786674 10841 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:37.796583 14362 raft_consensus.cc:697] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 10 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:37.750066 10941 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 13.
I20260709 13:17:37.836709 13719 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:37.808772 14662 raft_consensus.cc:2804] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:37.839810 13719 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:37.799662 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:37.833273 14607 consensus_queue.cc:237] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:37.803437 10946 raft_consensus.cc:1240] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:17:37.841064 14362 consensus_queue.cc:237] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:37.774801 14386 raft_consensus.cc:697] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:37.845057 14386 consensus_queue.cc:237] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:37.785432 14708 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:37.847160 14043 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:37.847851 14367 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:37.849323 14043 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:37.779925 14624 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:37.794337 14635 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:37.827229 10792 raft_consensus.cc:1275] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:17:37.829571 14435 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:37.791128 14727 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:37.777400 14740 log.cc:927] Time spent T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:17:37.795620 14734 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:37.831997 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 45410.8us
I20260709 13:17:37.796478 14667 raft_consensus.cc:2804] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:17:37.861166 10841 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:17:37.799151 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 216637us
I20260709 13:17:37.845643 14697 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:17:37.845098 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 102647us
W20260709 13:17:37.751540 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 167693us
W20260709 13:17:37.787987 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 48124.2us
W20260709 13:17:37.836246 14731 consensus_meta.cc:287] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.503s	user 0.008s	sys 0.002s
I20260709 13:17:37.801533 14728 consensus_queue.cc:1048] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:17:37.809653 14572 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:37.889917 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:37.832330 14702 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:37.835822 14511 raft_consensus.cc:2804] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:37.903693 10867 raft_consensus.cc:1275] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:17:37.876164 10871 raft_consensus.cc:1240] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:17:37.876565 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:37.880442 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:37.885494 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:37.916663 10872 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:37.919246 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:37.914670 14508 raft_consensus.cc:3055] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:37.921132 14508 raft_consensus.cc:740] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:37.888221 10841 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:37.891101 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:37.893940 14675 consensus_queue.cc:1048] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:37.894605 14680 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:37.855959 14662 raft_consensus.cc:697] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:37.903371 10915 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:37.904017 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:37.933413 10873 raft_consensus.cc:2368] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 9. Current term is 10.
I20260709 13:17:37.904968 10801 raft_consensus.cc:2393] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 12: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:37.905208 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:37.834604 14726 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:37.935546 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 47428.9us
I20260709 13:17:37.936362 10841 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:37.939445 11478 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:37.924153 10869 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:37.944371 10947 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 12.
I20260709 13:17:37.945071 10945 raft_consensus.cc:1240] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:17:37.945601 11478 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:37.946465 13719 raft_consensus.cc:2804] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:37.946976 13719 raft_consensus.cc:697] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:37.947013 10841 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:37.948343 10841 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:37.925132 10842 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:37.859587 14667 raft_consensus.cc:697] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 12 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:37.858757 14734 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:37.939594 10949 pending_rounds.cc:85] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 7
I20260709 13:17:37.955586 10949 pending_rounds.cc:107] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:17:37.955987 10949 raft_consensus.cc:2889] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:37.956408 10949 pending_rounds.cc:107] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:17:37.956514 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:37.956712 10949 raft_consensus.cc:2889] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:37.935479  9682 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:37.957850 10868 raft_consensus.cc:1240] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:37.950589 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:37.958595 10946 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:37.953388 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:37.959720 11504 raft_consensus.cc:2393] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 10: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:37.907981 14671 raft_consensus.cc:3055] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:37.912406 10916 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:37.960368 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:37.905025 14572 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:37.913453 14511 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:37.964927 11509 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:37.983368 10869 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 11.
I20260709 13:17:37.979297 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:37.979725 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 186326us
I20260709 13:17:37.976392 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:37.980079 13719 consensus_queue.cc:237] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:37.989533 11478 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:17:37.932264 14680 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:37.984474 10766 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:37.995780 10766 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:37.999691 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:38.002177 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:38.003057 11508 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
I20260709 13:17:38.004240 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:38.005025 11511 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:38.006078 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:37.984071 10867 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 12.
I20260709 13:17:37.903870 14699 consensus_queue.cc:1048] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 13:17:37.943550 14634 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:37.922257 14508 consensus_queue.cc:260] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:37.996934 10766 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:38.021464 10766 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:38.021900 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:37.950206 14635 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:37.902694 14336 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:37.950378 14701 raft_consensus.cc:2749] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:37.954923 14134 raft_consensus.cc:2804] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:37.962450 14671 raft_consensus.cc:740] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:38.015406 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:37.905934 14665 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:37.907747 14702 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:38.023643 10766 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:37.954191 14675 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:37.979218 14695 consensus_queue.cc:1059] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 13:17:38.031361 11478 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:37.943850 14479 consensus_queue.cc:1059] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.078s
I20260709 13:17:37.902908 14731 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:37.950019 14595 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:37.998692 14572 raft_consensus.cc:2804] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:37.995162 14275 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:17:38.020876 14735 consensus_meta.cc:287] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.655s	user 0.010s	sys 0.000s
I20260709 13:17:37.994642 13995 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:38.054651 14508 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:38.054318 14134 raft_consensus.cc:697] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:38.013877 14729 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:38.007807 14727 raft_consensus.cc:2749] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:17:38.045572 10947 raft_consensus.cc:2368] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for earlier term 11. Current term is 12.
I20260709 13:17:38.040616 10696 catalog_manager.cc:5697] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:38.112883 10696 catalog_manager.cc:5697] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:38.091976 14701 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:37.979074 14728 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:38.102315 14735 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:38.096692 14755 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:38.052670 11477 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:38.017254 14342 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:37.994776 14511 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:38.027642 14600 raft_consensus.cc:2804] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:17:37.994174 14676 consensus_queue.cc:1048] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:17:38.027850 14635 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:37.995918 14748 consensus_queue.cc:1048] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.096s
I20260709 13:17:38.096050 11511 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 8.
I20260709 13:17:38.097216 14572 raft_consensus.cc:697] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:38.093437 14595 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:38.097852 14479 raft_consensus.cc:3055] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:17:38.098649 14746 log.cc:927] Time spent T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.243s	user 0.001s	sys 0.000s
W20260709 13:17:38.098760 14754 log.cc:927] Time spent T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 13:17:38.054821 14675 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:37.961450 14749 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:17:38.183039 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 257997us
I20260709 13:17:38.146595 14730 raft_consensus.cc:2749] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:38.056578 14362 raft_consensus.cc:2749] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:17:38.028784 14634 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:17:38.116539 10696 catalog_manager.cc:5697] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 2 to 9, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 9 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:38.118803 14726 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:38.000730 14680 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:17:38.126394 10697 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42984 (request call id 245) took 15241 ms (client timeout 15000 ms). Trace:
I20260709 13:17:37.953989 14624 consensus_queue.cc:1048] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.074s
I20260709 13:17:38.164780 14600 raft_consensus.cc:697] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 11 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:38.101902 14751 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:38.112581 14729 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:38.155537 10872 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 12.
I20260709 13:17:38.108976 14753 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:38.202306 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:38.124619 14701 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:38.203343 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:38.203608 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:38.204558 10796 raft_consensus.cc:2368] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 10. Current term is 11.
I20260709 13:17:38.205243 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:38.205513 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:38.205837 10797 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 9.
I20260709 13:17:38.206085 10800 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 9.
W20260709 13:17:38.167464 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 219211us
I20260709 13:17:38.207589 10841 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:38.171005 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 258363us
I20260709 13:17:38.209571 10916 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:38.104873 14275 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:38.132602 14727 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:38.056090 14662 consensus_queue.cc:237] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:38.104506 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 169355us
I20260709 13:17:38.104832 14667 consensus_queue.cc:237] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:38.189044 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 165482us
I20260709 13:17:38.130477 14728 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:38.253644 10948 raft_consensus.cc:1275] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:17:38.190696 10696 catalog_manager.cc:5697] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 9 to 11, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 11 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:17:38.255900 10696 catalog_manager.cc:5697] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 7 to 8, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 8 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
W20260709 13:17:38.144441 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 91899us
I20260709 13:17:38.215401 14729 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:38.165930 14595 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:38.185604 14675 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:38.151495 14342 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:38.199399 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:38.210104 10797 raft_consensus.cc:1275] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:17:38.211431 10796 raft_consensus.cc:1275] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:17:38.211985 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:38.184693 14749 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:38.146211 14755 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:38.226257 10916 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:38.218595 11509 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 12.
I20260709 13:17:38.227192 13719 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:17:38.232219 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:38.202358 14758 log.cc:927] Time spent T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 13:17:38.233637 10943 raft_consensus.cc:1240] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:38.288028 10798 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:38.288655 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:38.289321 10943 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:38.236593 14762 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:17:38.299302 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:38.300109 10801 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:38.190599 14134 consensus_queue.cc:237] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:38.297492 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:38.234835 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:38.302592 10794 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:38.196799 14726 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:38.165480 14479 raft_consensus.cc:740] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:38.260283 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:38.261124 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:38.220578 14701 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:38.264231 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:38.306680 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:38.306947 10945 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:38.307310 11508 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:38.266988 13995 raft_consensus.cc:697] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:38.270217 11477 leader_election.cc:400] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 election: Vote denied by peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553) with higher term. Message: Invalid argument: T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for earlier term 11. Current term is 12.
I20260709 13:17:38.325407 11477 leader_election.cc:403] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:17:38.275590 10796 raft_consensus.cc:1275] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:17:38.326864 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:38.327739 10796 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:17:38.192054 10697 rpcz_store.cc:269] 0709 13:17:22.884758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:22.886941 (+  2183us) service_pool.cc:224] Handling call
0709 13:17:38.126293 (+15239352us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:36.159819 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:36.207208 (+ 47389us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:36.207263 (+    55us) write_op.cc:435] Acquired schema lock
0709 13:17:36.207282 (+    19us) tablet.cc:662] Decoding operations
0709 13:17:36.208278 (+   996us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:36.208368 (+    90us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:36.208398 (+    30us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:17:36.263727 (+ 55329us) tablet.cc:701] Row locks acquired
0709 13:17:36.263789 (+    62us) write_op.cc:260] PREPARE: finished
0709 13:17:36.264000 (+   211us) write_op.cc:270] Start()
0709 13:17:36.264154 (+   154us) write_op.cc:276] Timestamp: P: 1783603056263924 usec, L: 0
0709 13:17:36.264177 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:17:36.264964 (+   787us) log.cc:844] Serialized 4077 byte log entry
0709 13:17:36.563956 (+298992us) spinlock_profiling.cc:243] Waited 5.21 ms on lock 0x7b1800133060. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc83f3 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:17:37.233521 (+669565us) spinlock_profiling.cc:243] Waited 3.44 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:17:37.352395 (+118874us) op_driver.cc:464] REPLICATION: finished
0709 13:17:37.520126 (+167731us) write_op.cc:301] APPLY: starting
0709 13:17:37.520209 (+    83us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:37.520427 (+   218us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:37.520449 (+    22us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:37.521572 (+  1123us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:37.521868 (+   296us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=10
0709 13:17:37.521919 (+    51us) write_op.cc:312] APPLY: finished
0709 13:17:37.522207 (+   288us) log.cc:844] Serialized 77 byte log entry
0709 13:17:37.533926 (+ 11719us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:37.553077 (+ 19151us) write_op.cc:454] Released schema lock
0709 13:17:37.577807 (+ 24730us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:37.634437 (+ 56630us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:37.637019 (+  2582us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1611,"reported_tablets":50,"spinlock_wait_cycles":1714048,"child_traces":[["op",{"apply.queue_time_us":167472,"apply.run_cpu_time_us":4138,"apply.run_wall_time_us":100574,"num_ops":10,"prepare.queue_time_us":217183,"prepare.run_cpu_time_us":13885,"prepare.run_wall_time_us":304316,"raft.queue_time_us":41748,"raft.run_cpu_time_us":20917,"raft.run_wall_time_us":184892,"replication_time_us":1088037,"spinlock_wait_cycles":19927680,"thread_start_us":370245,"threads_started":4,"wal-append.queue_time_us":200924}]]}
I20260709 13:17:38.276115 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:38.234268 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:38.290138 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 63959.9us
I20260709 13:17:38.333504 10916 leader_election.cc:400] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 12. Current term is 13.
I20260709 13:17:38.290575 10915 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:38.290824 10946 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 11.
I20260709 13:17:38.294816 13719 raft_consensus.cc:697] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:38.335611 13719 consensus_queue.cc:237] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:38.295958 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:38.296123 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:38.296684 10876 raft_consensus.cc:1240] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:38.301695 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:38.337118 10871 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
I20260709 13:17:38.337311 10873 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
I20260709 13:17:38.233639 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:38.238452 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:38.308979 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 113720us
I20260709 13:17:38.257503 10696 catalog_manager.cc:5697] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:17:38.339357 10945 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 10.
I20260709 13:17:38.339941 10767 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:38.339955 10696 catalog_manager.cc:5697] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 4 to 12, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 12 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:38.342048 10696 catalog_manager.cc:5697] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 7 to 10, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 10 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:17:38.344385 10696 catalog_manager.cc:5697] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 7 to 9, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 9 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:38.346481 10696 catalog_manager.cc:5697] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 7 to 8, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 8 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:17:38.336552 10796 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 10.
I20260709 13:17:38.354923 10948 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
W20260709 13:17:38.357296 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 87290us
I20260709 13:17:38.270843 14769 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:38.348377 10696 catalog_manager.cc:5697] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:17:38.382962 10801 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 11.
I20260709 13:17:38.384413 10794 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:17:38.387826 10795 raft_consensus.cc:1240] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 11. Current term is 12. Ops: [10.6-11.7]
I20260709 13:17:38.244328 14727 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:38.397202 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 144009us
I20260709 13:17:38.396996 10696 catalog_manager.cc:5697] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 7 to 9, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 9 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:38.288089 14665 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:38.409895 10696 catalog_manager.cc:5697] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 7 to 11, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 11 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:38.277290 14749 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:38.417949 10696 catalog_manager.cc:5697] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 6 to 9, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 9 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:38.420631 10696 catalog_manager.cc:5697] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 9 to 10, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 10 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:17:38.294466 14760 raft_consensus.cc:2749] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:38.236064 14572 consensus_queue.cc:237] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:38.306259 14702 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:38.422417 10696 catalog_manager.cc:5697] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 1 to 8. New cstate: current_term: 8 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:38.431329 10696 catalog_manager.cc:5697] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 6 to 7, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 7 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:38.433202 10798 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 10.
I20260709 13:17:38.434019 10696 catalog_manager.cc:5697] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 4 to 8, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 8 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:38.436275 10696 catalog_manager.cc:5697] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 8 to 10, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 10 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:17:38.438894 10696 catalog_manager.cc:5697] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 5 to 7, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 7 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:17:38.389755 14676 raft_consensus.cc:2749] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553) with higher term. Message: Invalid argument: T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for earlier term 11. Current term is 12.
W20260709 13:17:38.426831 14756 log.cc:927] Time spent T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.311s	user 0.000s	sys 0.001s
W20260709 13:17:38.463128 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 128807us
W20260709 13:17:38.440627 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 142042us
I20260709 13:17:38.433259 14362 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:17:38.446892 14479 consensus_queue.cc:260] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:38.451998 14767 consensus_queue.cc:1059] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:38.452952 14727 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:38.456059 14342 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:38.440343 10696 catalog_manager.cc:5697] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 2 to 10, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 10 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:17:38.407887 14753 raft_consensus.cc:697] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:38.395828 14735 consensus_queue.cc:1048] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 13:17:38.478718 10916 leader_election.cc:400] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 9. Current term is 10.
I20260709 13:17:38.506129 10916 leader_election.cc:403] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:17:38.409441 14762 consensus_queue.cc:1048] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.063s
I20260709 13:17:38.474634 10842 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:17:38.488634 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 193096us
I20260709 13:17:38.500290 11508 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 12.
I20260709 13:17:38.501524 10943 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
W20260709 13:17:38.480902 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 190410us
I20260709 13:17:38.308128 14600 consensus_queue.cc:237] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:38.512316 14479 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:17:38.492311 14757 log.cc:927] Time spent T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.327s	user 0.000s	sys 0.001s
W20260709 13:17:38.518537 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 44013.8us
I20260709 13:17:38.409652 14769 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:38.524296 10796 raft_consensus.cc:1275] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:17:38.638099 10842 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:38.515972 10841 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:17:38.505074 14435 consensus_meta.cc:287] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.632s	user 0.010s	sys 0.001s
I20260709 13:17:38.503899 13995 consensus_queue.cc:237] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:38.529325 14275 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:38.447975 14386 consensus_queue.cc:1048] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 13:17:38.510962 14762 consensus_queue.cc:1243] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer 468a8fd46e824f62b9c2c0d3532a6d26 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 13:17:38.536418 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 127568us
I20260709 13:17:38.541489 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:38.544706 10792 raft_consensus.cc:1275] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:17:38.513789 14634 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:38.513521 14362 raft_consensus.cc:697] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 12 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:38.517460 14765 consensus_queue.cc:1059] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.118s
I20260709 13:17:38.607080 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:38.680559 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:38.604959 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 265090us
I20260709 13:17:38.681517 10943 raft_consensus.cc:2393] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 11: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:38.682057 10767 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:38.472553 14785 consensus_queue.cc:1048] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 13:17:38.547751 14342 consensus_queue.cc:1048] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.100s
I20260709 13:17:38.691226 10942 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:38.550046 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:38.520392 14767 raft_consensus.cc:2749] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:17:38.519721 14760 raft_consensus.cc:2749] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 9. Current term is 10.
W20260709 13:17:38.701582 14783 log.cc:927] Time spent T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:17:38.622061 10873 raft_consensus.cc:1275] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 3. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:17:38.702740 14755 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:38.472601 14665 consensus_queue.cc:1059] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.077s
I20260709 13:17:38.631407 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:38.525743 14768 consensus_queue.cc:1048] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.095s
W20260709 13:17:38.647827 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 132045us
I20260709 13:17:38.706240 10841 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:38.653786 10794 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 9.
I20260709 13:17:38.655284 14787 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:38.649319 14780 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:17:38.661023 10916 leader_election.cc:400] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 election: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 10. Current term is 11.
I20260709 13:17:38.717399 10916 leader_election.cc:403] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:17:38.673476 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:38.674448 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:38.608361 10795 raft_consensus.cc:1275] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:17:38.674989 10948 raft_consensus.cc:1275] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:17:38.676029 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:38.676947 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:17:38.641621 14763 log.cc:927] Time spent T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.371s	user 0.000s	sys 0.001s
W20260709 13:17:38.547189 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 144272us
I20260709 13:17:38.687649 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:38.696189 11507 raft_consensus.cc:1240] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 12 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:17:38.697822 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:17:38.681102 14692 consensus_meta.cc:287] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 1.127s	user 0.011s	sys 0.003s
I20260709 13:17:38.657490 14773 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:38.728524 10943 raft_consensus.cc:2393] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 9: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:38.728986 10942 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 12.
W20260709 13:17:38.714340 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 76304.2us
I20260709 13:17:38.732929 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:38.733561 10949 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:17:38.648389 14728 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:38.548813 14727 consensus_queue.cc:1059] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.143s
I20260709 13:17:38.721336 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:38.685395 14634 raft_consensus.cc:2804] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:38.741441 10943 raft_consensus.cc:1275] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:17:38.743952 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 83064.6us
I20260709 13:17:38.757421 10916 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:38.765659 10916 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:38.769464 10949 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 10.
I20260709 13:17:38.742660 14134 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:38.739297 14692 raft_consensus.cc:2749] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:38.786715 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:38.755734 14760 raft_consensus.cc:2749] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 10. Current term is 11.
I20260709 13:17:38.739377 14780 raft_consensus.cc:697] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:38.814302 10841 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:38.815189 14765 raft_consensus.cc:3055] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:17:38.680305 14753 consensus_queue.cc:237] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:38.816772 14767 raft_consensus.cc:2749] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:17:38.817080 14752 log.cc:927] Time spent T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.146s	user 0.000s	sys 0.000s
I20260709 13:17:38.817407 14786 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:17:38.820757 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 55171.2us
I20260709 13:17:38.856057 10916 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:38.820922 10766 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:38.816262 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 134243us
I20260709 13:17:38.860409 10767 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:38.756691 14702 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:38.773866 14665 raft_consensus.cc:2804] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:38.745819 14773 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:38.766495 14634 raft_consensus.cc:697] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:38.830729 10841 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
W20260709 13:17:38.894321 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 63727.4us
I20260709 13:17:38.822021 10946 raft_consensus.cc:1275] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:17:38.890888 10949 raft_consensus.cc:1275] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:17:38.896597 10767 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:38.896859 10841 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:38.897482 10946 raft_consensus.cc:1275] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 13:17:38.898063 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 138956us
I20260709 13:17:38.824052 14765 raft_consensus.cc:740] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:38.821868 14789 raft_consensus.cc:2804] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:38.773088 14675 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:38.905061 10767 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:38.855304 14786 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:38.855029 14785 raft_consensus.cc:2749] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:17:38.891739 14506 consensus_meta.cc:287] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 1.360s	user 0.011s	sys 0.003s
W20260709 13:17:38.901647 14508 consensus_meta.cc:287] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.794s	user 0.013s	sys 0.000s
I20260709 13:17:38.942227 10875 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10.
I20260709 13:17:38.778352 14362 consensus_queue.cc:237] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:38.892813 14386 raft_consensus.cc:2749] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:38.893245 14643 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:38.945900 10767 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:38.944281 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 88344us
I20260709 13:17:38.890434 14665 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:38.812649 14134 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:38.899202 14727 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:17:38.902491 14768 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:17:38.898630 14762 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:39.001680 14762 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:38.898288 14766 log.cc:927] Time spent T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.341s	user 0.000s	sys 0.001s
I20260709 13:17:38.915380 14755 raft_consensus.cc:2749] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:38.906533 14789 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:39.004746 14762 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:39.004819 14789 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:39.007675 14134 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:39.009317 14727 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:39.010102 14727 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:38.987834 14665 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:17:39.017050 14759 log.cc:927] Time spent T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.919s	user 0.000s	sys 0.009s
I20260709 13:17:39.019166 14797 raft_consensus.cc:2749] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:39.023778 14634 consensus_queue.cc:237] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:39.036242 14805 log.cc:927] Time spent T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 13:17:39.039667 14798 log.cc:927] Time spent T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
W20260709 13:17:39.041565 14803 log.cc:927] Time spent T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
I20260709 13:17:39.043726 14773 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:17:39.045110 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 145988us
I20260709 13:17:38.933701 14692 raft_consensus.cc:2749] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:17:38.979660 10767 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:39.047580 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 68039.7us
I20260709 13:17:39.046757 14768 raft_consensus.cc:697] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:39.050249 14768 consensus_queue.cc:237] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:38.977337 10916 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:38.968122 14794 consensus_queue.cc:1059] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.105s
I20260709 13:17:38.893479 14702 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:39.054121 14794 consensus_queue.cc:1243] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer 468a8fd46e824f62b9c2c0d3532a6d26 log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 13:17:39.054185 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 76979.9us
I20260709 13:17:39.054777 14794 raft_consensus.cc:3055] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:39.055179 14794 raft_consensus.cc:740] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:39.055135 14760 raft_consensus.cc:2749] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:38.939687 14786 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:38.903062 14767 consensus_queue.cc:1048] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 13:17:39.055116 10916 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:39.056159 14794 consensus_queue.cc:260] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:39.057370 14760 raft_consensus.cc:2749] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:17:39.057906 14769 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:38.985973 14643 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:38.994884 14708 consensus_meta.cc:287] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 1.148s	user 0.013s	sys 0.005s
I20260709 13:17:38.998955 14781 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:38.999502  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:39.075966 14781 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:39.092064 14781 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:17:39.095786 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:38.976810 14807 raft_consensus.cc:2804] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:39.067975 14728 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:38.970971 14780 consensus_queue.cc:237] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:38.993566 14765 consensus_queue.cc:260] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:39.150506 14511 consensus_meta.cc:287] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.957s	user 0.009s	sys 0.004s
I20260709 13:17:39.108552 10941 raft_consensus.cc:1240] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:39.104120 11508 raft_consensus.cc:1275] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:17:39.147938 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:39.104588 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:39.323650 10947 raft_consensus.cc:1240] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 12 LEADER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 11. Current term is 12. Ops: []
W20260709 13:17:39.458904 10767 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.116.130:37895 timed out after 2.140s (SENT) [suppressed 11 similar messages]
I20260709 13:17:39.505955 11511 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7.
W20260709 13:17:39.323441 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 268384us
I20260709 13:17:39.416899 10766 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:39.412742 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:39.507715 10871 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 10.
I20260709 13:17:39.485098 14667 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:17:39.279287 14751 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.012s	user 0.007s	sys 0.023s
W20260709 13:17:39.365626 14549 consensus_meta.cc:287] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 1.548s	user 0.017s	sys 0.003s
I20260709 13:17:39.530651 10871 raft_consensus.cc:1275] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:17:39.088201 14635 consensus_meta.cc:287] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.911s	user 0.009s	sys 0.005s
W20260709 13:17:39.257678 14804 log.cc:927] Time spent T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.264s	user 0.000s	sys 0.000s
I20260709 13:17:39.258603 14794 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:17:39.452739 14701 consensus_meta.cc:287] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 1.055s	user 0.007s	sys 0.006s
I20260709 13:17:39.595498 10867 pending_rounds.cc:85] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 4
I20260709 13:17:39.561950 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:39.414593 14787 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.650s	user 0.016s	sys 0.031s
W20260709 13:17:39.746119 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 287293us
I20260709 13:17:39.615386 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:39.754019 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:39.105963 13995 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:39.770292 11478 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:17:39.749969 10867 pending_rounds.cc:107] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:17:39.771355 10867 raft_consensus.cc:2889] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:39.567960 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:39.569135 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 152311us
I20260709 13:17:39.571470 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:39.774171 10868 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 7.
I20260709 13:17:39.756095 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:39.524104 10916 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:17:39.616328 14823 log.cc:927] Time spent T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
W20260709 13:17:39.613510 14814 log.cc:927] Time spent T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.322s	user 0.000s	sys 0.001s
I20260709 13:17:39.256333 14795 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:39.256156 14770 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:17:39.355188 14729 consensus_meta.cc:287] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 1.079s	user 0.009s	sys 0.005s
I20260709 13:17:39.763059 14367 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:39.752540 14832 raft_consensus.cc:2749] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:39.763844 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:39.795086 14367 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:39.343874 14643 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:39.493592 14728 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:39.796461 14728 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:39.795141 14729 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:39.797564 14367 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:39.798193 14367 raft_consensus.cc:2749] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:39.408118 14697 raft_consensus.cc:993] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586: : Instructing follower cb0428cc55984daba1ba63744a5102a2 to start an election
I20260709 13:17:39.801868 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:39.802701 11512 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 12.
I20260709 13:17:39.804433 10915 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:39.525660 14692 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:17:39.759466 14787 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.993s	user 0.029s	sys 0.052s
W20260709 13:17:39.655478 14680 consensus_meta.cc:287] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 1.456s	user 0.013s	sys 0.000s
I20260709 13:17:39.445519 14675 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:39.273891 14662 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:39.105762 14479 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:39.525496 14769 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:39.560405 14362 consensus_queue.cc:1048] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.286s
W20260709 13:17:39.602545 14821 log.cc:927] Time spent T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 13:17:39.792905 10869 raft_consensus.cc:1240] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:17:39.601279 14806 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:39.793182 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:39.820663 14680 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:39.768594 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:17:39.886041 14823 log.cc:927] Time spent T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:17:39.795769 10870 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 11.
I20260709 13:17:39.656862 14790 raft_consensus.cc:3055] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:39.808633 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:39.937119 11511 raft_consensus.cc:2393] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 7: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:39.340534 14785 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:39.415881 14767 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:39.559885 14549 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:39.792057 14595 consensus_meta.cc:287] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 1.515s	user 0.012s	sys 0.001s
W20260709 13:17:39.807348 14734 consensus_meta.cc:287] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 1.811s	user 0.014s	sys 0.000s
I20260709 13:17:39.095628 14708 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:39.791924 10868 raft_consensus.cc:1275] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:17:39.504654 14511 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:39.792743 13995 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:39.895063 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:39.752457 14701 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:39.949003 14824 log.cc:927] Time spent T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.202s	user 0.000s	sys 0.001s
I20260709 13:17:39.949743 10874 raft_consensus.cc:1240] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:39.950006 14701 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:39.560891 14667 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:39.952378 14667 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:39.809598 14787 raft_consensus.cc:697] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:39.952904 13995 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:39.953876 14787 consensus_queue.cc:237] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:39.955621 14511 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:39.834527 14836 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:39.752635 14635 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:39.831688 14675 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:39.907687 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:39.910322 10801 raft_consensus.cc:2393] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 10: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
W20260709 13:17:39.752118 14751 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.488s	user 0.011s	sys 0.058s
I20260709 13:17:39.918912 11504 raft_consensus.cc:2393] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 12: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:39.583699 14815 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:39.828687 14692 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:39.524480 14822 consensus_queue.cc:1175] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "12682eb70bc84c76879e04a8628ee586" responder_term: 12 status { last_received { term: 12 index: 9 } last_committed_idx: 8 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 11. Current term is 12. Ops: [11.9-11.9]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:17:39.834022 14479 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:39.868984 14671 consensus_queue.cc:260] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:39.931177 10876 raft_consensus.cc:2368] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 9. Current term is 10.
I20260709 13:17:39.930984 10688 maintenance_manager.cc:419] P ae957c8fc66c4b6ba063d0c74cbf5484: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033345
I20260709 13:17:39.931449 14731 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:39.931875 14362 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:39.931578 14806 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:39.813179 14795 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:39.933429 14820 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:39.933709 14680 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:39.933900 14828 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:39.933985 14729 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:39.886343 14600 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:39.794024 14770 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:39.827417 14832 consensus_queue.cc:1048] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.070s
W20260709 13:17:40.229362 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 705334us
I20260709 13:17:39.815135 14367 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:40.234952 14367 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:40.235054 14729 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:39.837527 14769 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:39.946094 14734 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:39.834916 14807 raft_consensus.cc:697] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:39.973179 10869 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
I20260709 13:17:39.936589 14790 raft_consensus.cc:740] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:17:40.033854 14665 consensus_meta.cc:287] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 1.018s	user 0.011s	sys 0.003s
I20260709 13:17:39.943166 14767 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:40.177950 10800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2324) took 1970 ms. Trace:
I20260709 13:17:39.943348 14785 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:40.142590 14667 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:40.102280 14781 consensus_meta.cc:287] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 1.010s	user 0.010s	sys 0.001s
W20260709 13:17:40.085664 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 276576us
I20260709 13:17:40.033438 10944 raft_consensus.cc:1240] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:40.019037 10767 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:40.013844 10797 raft_consensus.cc:1240] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:39.975890 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:40.155809 14840 log.cc:927] Time spent T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 13:17:40.139599 14336 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:40.033421 14823 log.cc:927] Time spent T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 13:17:39.975958 14751 raft_consensus.cc:697] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:40.091070 14829 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:40.007472 14701 raft_consensus.cc:1081] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 8 LEADER]: Signalling peer cb0428cc55984daba1ba63744a5102a2 to start an election
I20260709 13:17:39.963061 14753 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:40.013160 14635 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:40.011631 14549 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:40.144428 14830 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:40.225462 10871 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 9.
I20260709 13:17:40.184562 14675 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:39.831910 14662 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:40.235430 14367 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:40.240343 14692 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:40.209772 14362 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:40.293176 14769 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:39.956728 14595 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:40.243496 14789 consensus_meta.cc:287] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 1.238s	user 0.015s	sys 0.004s
I20260709 13:17:40.246025 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:40.243968 14729 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:40.247776 14795 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:17:40.248413 10747 auto_rebalancer.cc:893] Removing replace marker failed: Replica cb0428cc55984daba1ba63744a5102a2 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 10 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:17:40.250375 14702 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:40.252987 10800 rpcz_store.cc:276] 0709 13:17:38.207298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:38.208559 (+  1261us) service_pool.cc:224] Handling call
0709 13:17:38.208902 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:38.273630 (+ 64728us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:38.273657 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:38.273697 (+    40us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:38.273731 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:40.177564 (+1903833us) spinlock_profiling.cc:243] Waited 64.5 ms on lock 0x7b54000bd6d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:40.177645 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:40.177861 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":148442496}
I20260709 13:17:40.307968 14782 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:40.309118 10950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 615) took 2330 ms. Trace:
I20260709 13:17:40.228024 14600 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:40.256300 10870 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10.
I20260709 13:17:40.260381 14790 consensus_queue.cc:260] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:40.227672 14828 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:40.014710 14836 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:40.261411 14767 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:40.260711 14751 consensus_queue.cc:237] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:40.221551 14820 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:40.262075 14781 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:40.263163 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 244260us
I20260709 13:17:40.330353 10767 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:40.331696 10767 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:40.263681 14785 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:40.263993 14667 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:40.264415 14770 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:40.264700 14806 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:40.265473 14727 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:40.274665 10868 raft_consensus.cc:1240] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 8. Current term is 9. Ops: []
W20260709 13:17:40.275483 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 239364us
I20260709 13:17:40.314198 14832 raft_consensus.cc:2749] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:40.195888 14815 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:40.265799 14479 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:40.279633 14753 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:40.289516 14675 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:40.275586 14749 consensus_meta.cc:287] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 1.836s	user 0.009s	sys 0.006s
I20260709 13:17:40.291477 14708 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:40.291757 14662 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:40.301578 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:17:40.304886 10747 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
I20260709 13:17:40.258998 14665 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:40.306749 14786 consensus_meta.cc:287] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 1.245s	user 0.014s	sys 0.007s
I20260709 13:17:40.307070 14729 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:40.302908 14701 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:40.309553 10950 rpcz_store.cc:276] 0709 13:17:37.978519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:37.982495 (+  3976us) service_pool.cc:224] Handling call
0709 13:17:37.982830 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:38.102595 (+119765us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:38.102642 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:38.103277 (+   635us) log.cc:844] Serialized 26 byte log entry
0709 13:17:38.277824 (+174547us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:17:38.277862 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:40.308666 (+2030804us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b5400017cd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:40.308729 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:40.308780 (+    51us) raft_consensus.cc:1685] finished
0709 13:17:40.308793 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:40.309057 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9869,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":7113,"spinlock_wait_cycles":253568256,"thread_start_us":135251,"threads_started":1,"wal-append.queue_time_us":543742,"wal-append.run_cpu_time_us":1760,"wal-append.run_wall_time_us":1026017}
I20260709 13:17:40.259673 14728 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:40.319214 10948 raft_consensus.cc:2410] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:17:40.276564 14829 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:40.294641 14362 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:40.281795 14844 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:40.302296 14734 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:40.341672 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 110481us
I20260709 13:17:40.285271 14549 consensus_queue.cc:1048] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:17:40.314611 14600 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:40.354704 10870 raft_consensus.cc:1240] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:17:40.294993 14635 consensus_queue.cc:1048] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:17:40.357216 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:40.337714 14790 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:40.308571 14782 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:40.373380 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:40.378116 11512 raft_consensus.cc:1275] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:17:40.342064 14767 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:40.340427 14795 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:40.382687 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:40.309579 14789 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:40.385743 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:40.265769 14807 consensus_queue.cc:237] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:40.378320 10747 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:17:40.393919 14728 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:40.394025 14665 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:40.331558 14828 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:40.370100 14727 raft_consensus.cc:2749] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:40.377252 14702 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:40.399763 10916 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:40.391763 14662 raft_consensus.cc:2804] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:17:40.404083 14849 log.cc:927] Time spent T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
I20260709 13:17:40.386458 14749 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:40.387858 14786 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:40.415226 14860 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:17:40.419215 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:40.448211 10916 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:17:40.420951 14851 log.cc:927] Time spent T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:17:40.398953 14769 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:40.403192 14549 consensus_queue.cc:1243] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer 468a8fd46e824f62b9c2c0d3532a6d26 log is divergent from this leader: its last log entry 8.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:17:41.303036 14769 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:40.417877 14789 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:41.303644 14882 raft_consensus.cc:2804] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:41.304214 14882 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:41.304549 14882 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:41.305101 10949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2270) took 2589 ms. Trace:
I20260709 13:17:41.305335 14769 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:40.281863 14830 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:41.305446 10949 rpcz_store.cc:276] 0709 13:17:38.715710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:38.771506 (+ 55796us) service_pool.cc:224] Handling call
0709 13:17:38.771781 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:38.895718 (+123937us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:41.304912 (+2409194us) spinlock_profiling.cc:243] Waited 92 ms on lock 0x7b5400376754. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:41.305053 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":211486464}
I20260709 13:17:40.397089 14701 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:40.449988 14749 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:40.419816 10869 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 12.
I20260709 13:17:40.403401 14844 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:41.309396 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49290 (request call id 549) took 1110 ms. Trace:
I20260709 13:17:41.309759 10869 rpcz_store.cc:276] 0709 13:17:40.199411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:40.301241 (+101830us) service_pool.cc:224] Handling call
0709 13:17:41.309335 (+1008094us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:41.310971 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 862799us
I20260709 13:17:41.311161 14844 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:40.421685 14795 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:40.423094 14767 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:40.442058 14662 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:17:41.087113 14850 log.cc:927] Time spent T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.250s	user 0.000s	sys 0.001s
I20260709 13:17:41.313479 14767 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:41.313490 14662 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:41.016059 14367 consensus_meta.cc:287] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.716s	user 0.003s	sys 0.007s
I20260709 13:17:40.446035 14665 raft_consensus.cc:2749] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:41.001291 14856 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:40.442402 14702 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:41.314425 14367 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:41.314733 14856 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:41.316320 14702 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:41.316498 14367 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:41.316880 14856 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:41.015545 14836 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:41.314409 14769 consensus_queue.cc:1048] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 13:17:41.022379 14595 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:41.102210 14848 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.002s	sys 0.007s
W20260709 13:17:41.318069 14848 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.923s	user 0.002s	sys 0.008s
I20260709 13:17:41.313264 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:41.313293 14795 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:17:41.108476 14854 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.730s	user 0.004s	sys 0.009s
W20260709 13:17:41.320451 14854 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.967s	user 0.005s	sys 0.009s
I20260709 13:17:41.102990 14781 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:41.321236 10916 connection.cc:256] RPC call timeout handler was delayed by -1.0678223380000418s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1915s for a response
I20260709 13:17:40.996806 14667 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:41.097420 14753 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:41.321768 14367 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.321972 14667 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.105490 14782 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:41.110549 14790 consensus_meta.cc:287] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.703s	user 0.007s	sys 0.002s
I20260709 13:17:40.791541 14708 consensus_queue.cc:1059] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.323s
I20260709 13:17:41.108358 14829 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:41.322428 14667 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:40.491559 14860 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:41.324681 14860 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:17:41.324783 10916 connection.cc:256] RPC call timeout handler was delayed by -1.0678223380000418s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1915s for a response
W20260709 13:17:41.113003 14794 consensus_meta.cc:287] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 1.358s	user 0.014s	sys 0.000s
W20260709 13:17:41.119262 14857 log.cc:927] Time spent T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 13:17:41.325511 14702 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:41.326089 14702 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:41.326498 14790 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:41.326947 14782 raft_consensus.cc:3055] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:17:41.327030 14790 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:41.327518 14782 raft_consensus.cc:740] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:41.328517 14782 consensus_queue.cc:260] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:41.329278 14790 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:41.125923 10868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1828) took 1370 ms. Trace:
I20260709 13:17:41.329744 10868 rpcz_store.cc:276] 0709 13:17:39.755608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:39.954960 (+199352us) service_pool.cc:224] Handling call
0709 13:17:39.955289 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:40.356753 (+401464us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:41.125519 (+768766us) spinlock_profiling.cc:243] Waited 313 ms on lock 0x7b54000f69d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:41.125833 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":720407424}
I20260709 13:17:41.330780 14595 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:17:41.167520 14726 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.760s	user 0.013s	sys 0.011s
I20260709 13:17:41.237716 14764 raft_consensus.cc:3055] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:17:41.331852 10947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2257) took 3103 ms. Trace:
W20260709 13:17:41.185575 14813 log.cc:927] Time spent T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 2.186s	user 0.004s	sys 0.000s
I20260709 13:17:41.332147 14764 raft_consensus.cc:740] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:41.332307 10947 rpcz_store.cc:276] 0709 13:17:38.228866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:38.229426 (+   560us) service_pool.cc:224] Handling call
0709 13:17:38.253807 (+ 24381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:39.506258 (+1252451us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:41.331625 (+1825367us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b5400399c54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:41.331795 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2459919744}
I20260709 13:17:41.287535 14635 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:41.289351 14861 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.333120 14764 consensus_queue.cc:260] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:41.333572 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:41.333573 14635 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:41.334858 10947 raft_consensus.cc:2393] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 10: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:41.324692 14667 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:41.332074 14726 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.936s	user 0.014s	sys 0.011s
I20260709 13:17:41.335711 10947 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb"
dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:36714
I20260709 13:17:41.336197 14635 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:41.293502 14876 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.293437 14875 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.296116 14877 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.336925 14876 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:41.298310 14880 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:41.337491 14877 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:41.331859 10868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1834) took 1250 ms. Trace:
I20260709 13:17:41.338167 14880 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:41.338263 10868 rpcz_store.cc:276] 0709 13:17:40.081090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.330919 (+1249829us) service_pool.cc:224] Handling call
0709 13:17:41.331245 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:41.331541 (+   296us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:41.331569 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:41.331610 (+    41us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:41.331650 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:41.331710 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:41.331806 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3158,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":89}
I20260709 13:17:41.340138 14877 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:41.340442 14880 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:41.297448 14879 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.300282 14786 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:41.299485 14881 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.307611 14830 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:41.307936 14701 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:41.311429 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:40.419711 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:41.342872 10949 raft_consensus.cc:1240] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 12. Current term is 13. Ops: []
I20260709 13:17:41.310926 14882 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:41.103084 14815 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:41.124821 14600 consensus_meta.cc:287] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.710s	user 0.009s	sys 0.001s
I20260709 13:17:41.345072 14600 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:41.344278 10949 raft_consensus.cc:1275] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:17:41.347183 14600 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:41.347458 10949 raft_consensus.cc:1275] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:17:41.345973 14662 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:41.349625 14662 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:41.349889 14876 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:41.330189 14702 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:41.331339 14595 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:41.334146 14764 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:41.353199 14595 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:41.271602 14765 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:41.322330 14367 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:41.359202 14833 log.cc:927] Time spent T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 1.405s	user 0.000s	sys 0.004s
I20260709 13:17:41.359401 14367 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:41.336143 14726 raft_consensus.cc:697] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:41.360821 14726 consensus_queue.cc:237] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:41.333863 14861 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:41.363461 14861 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:17:41.292295 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:41.336472 10947 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:41.366750 10947 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:41.336999 14875 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:41.296757 14878 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.371158 14878 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:41.372447 14875 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:41.373437 14878 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:41.321173 14769 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:41.341148 14879 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:41.341681 14881 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:41.351174 14882 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:41.379442 14879 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:41.379652 14881 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:41.380105 14769 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:41.381495 10796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 477) took 1624 ms. Trace:
I20260709 13:17:41.381878 10796 rpcz_store.cc:276] 0709 13:17:39.757490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:39.757641 (+   151us) service_pool.cc:224] Handling call
0709 13:17:39.764129 (+  6488us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:40.252037 (+487908us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:40.252084 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:40.252159 (+    75us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:17:40.252210 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:41.381246 (+1129036us) spinlock_profiling.cc:243] Waited 488 ms on lock 0x7b54000c1f54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:41.381309 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:41.381439 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1121517824}
I20260709 13:17:41.386021 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1832) took 1564 ms. Trace:
I20260709 13:17:41.340790 10868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 548) took 1144 ms. Trace:
I20260709 13:17:41.377074 10947 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:41.387039 10868 rpcz_store.cc:276] 0709 13:17:40.196899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.339783 (+1142884us) service_pool.cc:224] Handling call
0709 13:17:41.340120 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:41.340307 (+   187us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:41.340332 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:41.340375 (+    43us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:17:41.340407 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:41.340615 (+   208us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:41.340730 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:41.386456 10869 rpcz_store.cc:276] 0709 13:17:39.821616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.319212 (+1497596us) service_pool.cc:224] Handling call
0709 13:17:41.319558 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:41.319784 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:17:41.370103 (+ 50319us) log.cc:844] Serialized 14 byte log entry
0709 13:17:41.385625 (+ 15522us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:41.385700 (+    75us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:41.385761 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:41.385832 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:41.385958 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":37725,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"thread_start_us":65071,"threads_started":2,"wal-append.queue_time_us":12862}
I20260709 13:17:41.387873 14769 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:41.391856 10947 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:41.400943 10944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 620) took 2654 ms. Trace:
I20260709 13:17:41.401427 10944 rpcz_store.cc:276] 0709 13:17:38.746536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:38.828861 (+ 82325us) service_pool.cc:224] Handling call
0709 13:17:38.893716 (+ 64855us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:40.255845 (+1362129us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:41.400643 (+1144798us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b5400015ed4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:41.400870 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2577479424}
W20260709 13:17:41.406564 10871 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 1825) took 1839 ms (client timeout 1824 ms). Trace:
W20260709 13:17:41.407109 10871 rpcz_store.cc:269] 0709 13:17:39.567528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:39.567674 (+   146us) service_pool.cc:224] Handling call
0709 13:17:41.406353 (+1838679us) spinlock_profiling.cc:243] Waited 452 ms on lock 0x7b540009b5d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:41.406486 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1040518912}
W20260709 13:17:41.472815 14827 log.cc:927] Time spent T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 1.604s	user 0.003s	sys 0.000s
I20260709 13:17:41.478175 14820 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:41.483505 14600 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.484220 14793 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.485329 14793 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:41.484153 14600 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:41.490067 14600 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:41.495803 10767 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.162s	user 0.007s	sys 0.005s
W20260709 13:17:41.496196 10767 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.163s	user 0.007s	sys 0.005s
W20260709 13:17:41.496481 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 1.16327e+06us
I20260709 13:17:41.516991 14882 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.517632 14882 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:41.518074 10799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2328) took 3305 ms. Trace:
I20260709 13:17:41.519513 14879 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:41.520041 14879 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:41.522464 14879 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:41.525628 14882 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:41.545151 14879 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.545264 14882 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.545735 14879 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:41.514537 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1798) took 1989 ms. Trace:
I20260709 13:17:41.545869 14882 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:41.546583 10867 rpcz_store.cc:276] 0709 13:17:39.525245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:39.525377 (+   132us) service_pool.cc:224] Handling call
0709 13:17:39.595201 (+ 69824us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:39.835304 (+240103us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:39.835342 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:39.836309 (+   967us) log.cc:844] Serialized 52 byte log entry
0709 13:17:40.186137 (+349828us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:40.199224 (+ 13087us) log.cc:844] Serialized 14 byte log entry
0709 13:17:40.199447 (+   223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:40.199516 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:41.298048 (+1098532us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b180046ba00. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7dd42c0a 00007f9c87482219 00007f9c84d8958f 00007f9c7ddc8571 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd
0709 13:17:41.514040 (+215992us) raft_consensus.cc:1685] finished
0709 13:17:41.514186 (+   146us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:41.514441 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8041,"prepare.queue_time_us":13431,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":2736,"raft.queue_time_us":161565,"raft.run_cpu_time_us":410,"raft.run_wall_time_us":17129,"spinlock_wait_cycles":26618624,"thread_start_us":354110,"threads_started":2,"wal-append.queue_time_us":364370,"wal-append.run_cpu_time_us":3736,"wal-append.run_wall_time_us":1209822}
I20260709 13:17:41.548382 14882 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:17:41.530169 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:41.546150 10876 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 1796) took 2985 ms (client timeout 1915 ms). Trace:
W20260709 13:17:41.549433 10876 rpcz_store.cc:269] 0709 13:17:38.575021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:38.606649 (+ 31628us) service_pool.cc:224] Handling call
0709 13:17:41.545917 (+2939268us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b540009a6d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:41.546078 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2874313088}
I20260709 13:17:41.543799 14881 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.518486 10799 rpcz_store.cc:276] 0709 13:17:38.212548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:38.212675 (+   127us) service_pool.cc:224] Handling call
0709 13:17:38.232161 (+ 19486us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:38.445259 (+213098us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:38.445307 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:38.445368 (+    61us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:17:38.445416 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:41.517819 (+3072403us) spinlock_profiling.cc:243] Waited 213 ms on lock 0x7b5400387a54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:41.517869 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:41.518015 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":489472256}
I20260709 13:17:41.550614 14881 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:41.552735 14881 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:41.554637 14879 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:41.555227 14826 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.555873 14826 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:41.566897 10794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 473) took 2495 ms. Trace:
I20260709 13:17:41.566989 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:41.568135 10798 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 9.
I20260709 13:17:41.565857 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:41.569586 10799 raft_consensus.cc:2393] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 12: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:41.570936 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:41.571411 14826 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:41.571724 10799 raft_consensus.cc:2393] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 10: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:41.572186 14826 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.567332 10794 rpcz_store.cc:276] 0709 13:17:39.071715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:39.072933 (+  1218us) service_pool.cc:224] Handling call
0709 13:17:39.283202 (+210269us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:39.283583 (+   381us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:17:39.335424 (+ 51841us) log.cc:844] Serialized 14 byte log entry
0709 13:17:41.460035 (+2124611us) spinlock_profiling.cc:243] Waited 6.71 ms on lock 0x7b18003fc880. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a5523a 00007f9c80ebfa2a
0709 13:17:41.538160 (+ 78125us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:41.538802 (+   642us) log.cc:844] Serialized 52 byte log entry
0709 13:17:41.538970 (+   168us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:41.556037 (+ 17067us) log.cc:844] Serialized 14 byte log entry
0709 13:17:41.556302 (+   265us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:41.556373 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:41.566652 (+ 10279us) raft_consensus.cc:1685] finished
0709 13:17:41.566672 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:41.566841 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":85961,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":72,"raft.queue_time_us":89182,"raft.run_cpu_time_us":440,"raft.run_wall_time_us":75661,"spinlock_wait_cycles":15442816,"thread_start_us":94168,"threads_started":2,"wal-append.queue_time_us":2166084}
I20260709 13:17:41.572819 14826 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:41.576980 14826 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:41.577215 10872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1815) took 3301 ms. Trace:
I20260709 13:17:41.577605 10872 rpcz_store.cc:276] 0709 13:17:38.275480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:38.275681 (+   201us) service_pool.cc:224] Handling call
0709 13:17:38.275995 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:38.325152 (+ 49157us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:38.325202 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:38.325259 (+    57us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:38.325313 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:41.576951 (+3251638us) spinlock_profiling.cc:243] Waited 48.9 ms on lock 0x7b5400078854. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:41.576999 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:41.577131 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":112431360}
I20260709 13:17:41.579437 10767 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:17:41.587158 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 83409.3us
I20260709 13:17:41.587486 10767 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:41.564491 14708 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.589414 14708 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:41.591261 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:41.588680 14826 raft_consensus.cc:2804] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:41.591771 14826 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:41.591941 14765 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:41.592777 14826 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:41.592319 14708 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:41.592615 10793 raft_consensus.cc:1275] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 13:17:41.594697 14708 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.594890 10793 pending_rounds.cc:85] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 8
I20260709 13:17:41.595186 10793 pending_rounds.cc:107] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 13:17:41.595448 10793 raft_consensus.cc:2889] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:41.596455 14864 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:41.597194 14275 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.597057 10793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 475) took 2519 ms. Trace:
I20260709 13:17:41.598562 10793 rpcz_store.cc:276] 0709 13:17:39.077732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:39.080230 (+  2498us) service_pool.cc:224] Handling call
0709 13:17:39.092061 (+ 11831us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:41.595892 (+2503831us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:41.596801 (+   909us) spinlock_profiling.cc:243] Waited 2.5 s on lock 0x7b54000e8b54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:41.596997 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5749885312}
I20260709 13:17:41.597039 14864 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:41.606146 10767 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:41.598335 14275 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:41.600446 14708 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:41.601236 14794 consensus_queue.cc:1059] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 13:17:41.615975 14863 raft_consensus.cc:2804] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:41.616449 14863 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:41.616852 14863 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:41.621819 14869 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.622416 14869 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:41.624137 14822 raft_consensus.cc:3055] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:41.624603 14822 raft_consensus.cc:740] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:41.627759 14822 consensus_queue.cc:260] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:41.629568 14822 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:41.631275 14708 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:41.635766 14765 raft_consensus.cc:2749] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:41.639767 14275 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:41.661976 14864 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:41.663594 14793 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:41.667833 14863 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:41.670835 14863 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:41.679355 14826 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:41.681955 14826 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:41.682667 14879 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.683292 14879 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:41.685520 14879 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:41.694605 14782 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:17:41.706077 10942 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41020 (request call id 2263) took 3437 ms (client timeout 2036 ms). Trace:
W20260709 13:17:41.706552 10942 rpcz_store.cc:269] 0709 13:17:38.268146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:38.268306 (+   160us) service_pool.cc:224] Handling call
0709 13:17:41.705804 (+3437498us) spinlock_profiling.cc:243] Waited 354 ms on lock 0x7b5400331154. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:41.705969 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":814805376}
I20260709 13:17:41.718758 14881 consensus_queue.cc:1048] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:41.720459 14881 consensus_queue.cc:1048] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:17:41.723115 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:41.724496 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:41.725301 10794 raft_consensus.cc:2393] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 12: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:41.726675 10793 raft_consensus.cc:1240] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:17:41.728422 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:41.729009 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:41.729246 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:41.729619 10798 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 8.
I20260709 13:17:41.729789 10800 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
I20260709 13:17:41.730406 10794 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:41.737484 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:41.738878 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:41.739192 10796 raft_consensus.cc:1275] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:17:41.740487 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:41.741330 10793 raft_consensus.cc:2410] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:17:41.736879 10794 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 10.
I20260709 13:17:41.739521 10799 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 8.
I20260709 13:17:41.728104 14782 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.744448 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:41.744997 10799 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:41.745918 14782 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:41.751343 14782 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:41.754024 10946 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.431s	user 0.007s	sys 0.011s
I20260709 13:17:41.755896 10948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41020 (request call id 2269) took 3040 ms. Trace:
I20260709 13:17:41.762315 10948 rpcz_store.cc:276] 0709 13:17:38.715414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:38.721004 (+  5590us) service_pool.cc:224] Handling call
0709 13:17:41.755657 (+3034653us) spinlock_profiling.cc:243] Waited 1.58 s on lock 0x7b54001e1754. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:41.755830 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3627913600}
I20260709 13:17:41.760097 10874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1821) took 3463 ms. Trace:
I20260709 13:17:41.763370 10874 rpcz_store.cc:276] 0709 13:17:38.296775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:38.296889 (+   114us) service_pool.cc:224] Handling call
0709 13:17:38.352410 (+ 55521us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:39.950213 (+1597803us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:41.759834 (+1809621us) spinlock_profiling.cc:243] Waited 1.6 s on lock 0x7b540006c7d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:41.760031 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3673062272}
I20260709 13:17:41.761304 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:41.766424 10946 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.444s	user 0.008s	sys 0.012s
I20260709 13:17:41.767611 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:41.768508 10793 raft_consensus.cc:2410] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 11 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 13:17:41.773029 11478 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.375s	user 0.007s	sys 0.006s
W20260709 13:17:41.773592 11478 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.375s	user 0.007s	sys 0.006s
I20260709 13:17:41.743532 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:41.779340 10875 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 1810) took 4368 ms (client timeout 2140 ms). Trace:
I20260709 13:17:41.779438 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:41.780186 10948 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10.
W20260709 13:17:41.779951 10875 rpcz_store.cc:269] 0709 13:17:37.411159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:37.412199 (+  1040us) service_pool.cc:224] Handling call
0709 13:17:41.779135 (+4366936us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b5400097ed4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:41.779243 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3378472704}
I20260709 13:17:41.781684 10799 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 10.
W20260709 13:17:41.785220 14802 log.cc:927] Time spent T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 2.874s	user 0.006s	sys 0.000s
I20260709 13:17:41.785481 10793 raft_consensus.cc:1275] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:17:41.786697 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:41.786839 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:41.811544 10792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 474) took 2735 ms. Trace:
I20260709 13:17:41.789630 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:41.812040 10800 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 7.
I20260709 13:17:41.812597 10794 raft_consensus.cc:2410] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:17:41.813455 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:41.814055 10794 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 12.
I20260709 13:17:41.790671 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:41.796053 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:41.815076 10798 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 12.
I20260709 13:17:41.815104 10795 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 10.
I20260709 13:17:41.815728 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:41.816010 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:41.817512 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:41.809283 10796 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 11.
I20260709 13:17:41.818125 10947 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:17:41.817687 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:41.819059 10794 raft_consensus.cc:2393] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 10: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:41.819545 10944 raft_consensus.cc:1240] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:17:41.820209 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:41.820773 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:41.823024 10943 raft_consensus.cc:2393] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 12: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:41.822351 10949 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
I20260709 13:17:41.825948 10950 raft_consensus.cc:2393] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 11: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:41.811949 10792 rpcz_store.cc:276] 0709 13:17:39.077343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:39.077497 (+   154us) service_pool.cc:224] Handling call
0709 13:17:39.103735 (+ 26238us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:17:39.104235 (+   500us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:39.104278 (+    43us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:17:39.289870 (+185592us) log.cc:844] Serialized 78 byte log entry
0709 13:17:39.509734 (+219864us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:39.509795 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:39.509891 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:41.472136 (+1962245us) spinlock_profiling.cc:243] Waited 16.8 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:17:41.793079 (+320943us) raft_consensus.cc:1685] finished
0709 13:17:41.793101 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:41.811482 (+ 18381us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2783,"raft.queue_time_us":1461,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":102409,"spinlock_wait_cycles":38732032,"thread_start_us":166481,"threads_started":1,"wal-append.queue_time_us":449107}
I20260709 13:17:41.826690 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:41.827778 10796 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 8.
I20260709 13:17:41.831820 11478 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:41.838227 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:41.839167 10944 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:41.823190 14842 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.839226 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:41.845777 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:41.846397 10943 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10.
I20260709 13:17:41.846524 10793 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 10.
I20260709 13:17:41.841639 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:41.848090 10949 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
I20260709 13:17:41.850924 14701 consensus_queue.cc:1048] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:17:41.829031 14795 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.861099 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:41.849849 10947 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
I20260709 13:17:41.867375 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:41.871835 10943 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10.
W20260709 13:17:41.872959 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:41.871868 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:17:41.868052 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:41.869479 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:41.841187 14435 raft_consensus.cc:2804] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:41.864868 11478 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:41.877760 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:41.869031 14793 consensus_queue.cc:1048] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 13:17:41.891280 10793 raft_consensus.cc:2393] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 12: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:41.893390 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:41.894877 14869 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:41.895071 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:41.895988 10793 raft_consensus.cc:2368] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 11. Current term is 12.
I20260709 13:17:41.897593 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:41.898679 10793 raft_consensus.cc:2393] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 11: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:41.899873 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:41.901273 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:41.901819 10793 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 11.
I20260709 13:17:41.902558 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:41.903203 10942 raft_consensus.cc:1240] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:41.903496 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:41.904101 10793 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 9.
I20260709 13:17:41.904260 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:41.888084 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:41.904745 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:41.911011 10942 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
I20260709 13:17:41.881202 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:41.911983 10943 raft_consensus.cc:2410] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 11 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:17:41.905167 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:41.913467 10947 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 9.
I20260709 13:17:41.914865 10944 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 8.
I20260709 13:17:41.916812 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:41.923485 10943 raft_consensus.cc:2393] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 10: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:41.923811 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:41.925729 10944 raft_consensus.cc:2393] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 9: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:41.869391 14795 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:41.917672 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:41.863467 14842 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:41.922196 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 57470.1us
I20260709 13:17:41.922365 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:41.908375 14435 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:41.905684 14816 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.929510 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:41.930299 10949 raft_consensus.cc:2410] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 12 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:17:41.932951 10942 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 8.
I20260709 13:17:41.934383 11478 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:41.935309 11478 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:41.937516 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:41.938022 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:41.938751 11505 raft_consensus.cc:2393] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 11: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:41.940938 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:41.941746 11509 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:41.945933 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:41.946803 10872 raft_consensus.cc:2410] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:17:41.946817 10867 raft_consensus.cc:1240] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:41.947460 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:41.948078 10876 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10.
I20260709 13:17:41.948411 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:41.950709 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:41.951081 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:41.950603 10944 raft_consensus.cc:1275] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:17:41.931679 14793 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:41.956034 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:41.956154 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:41.957537 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:41.934310 14435 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:41.961643 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:41.965597 10766 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:41.977849 10766 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:41.975405 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:41.951704 10943 raft_consensus.cc:2393] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 12: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:41.947224 14818 raft_consensus.cc:2804] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:41.948019 14852 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:41.998313 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:41.999425 10946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 1852) took 3137 ms. Trace:
I20260709 13:17:41.999640 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:42.000751 11508 raft_consensus.cc:2393] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 11: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:42.003396 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:42.004022 11506 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 11.
I20260709 13:17:41.999853 10946 rpcz_store.cc:276] 0709 13:17:38.861728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:38.898160 (+ 36432us) service_pool.cc:224] Handling call
0709 13:17:38.900325 (+  2165us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:38.900512 (+   187us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:17:38.932147 (+ 31635us) log.cc:844] Serialized 14 byte log entry
0709 13:17:38.932468 (+   321us) log.cc:844] Serialized 14 byte log entry
0709 13:17:40.322357 (+1389889us) spinlock_profiling.cc:243] Waited 5.4 ms on lock 0x7b180042a360. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7dd42c0a 00007f9c87482219 00007f9c80a6bd70 00007f9c80a5f293 00007f9c80a54cea 00007f9c80a5523a 00007f9c80ebfa2a 00007f9c80ecf0bf 00007f9c80ecf038
0709 13:17:41.766895 (+1444538us) log.cc:844] Serialized 14 byte log entry
0709 13:17:41.957406 (+190511us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:41.957457 (+    51us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:41.957500 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:41.957579 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:41.999330 (+ 41751us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4951,"prepare.queue_time_us":871923,"prepare.run_cpu_time_us":578,"prepare.run_wall_time_us":7398,"raft.queue_time_us":7991,"raft.run_cpu_time_us":282,"raft.run_wall_time_us":36432,"spinlock_wait_cycles":12424448,"thread_start_us":1634139,"threads_started":2}
I20260709 13:17:42.005452 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:42.006211 10874 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:42.007875 10766 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:42.010145 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:42.010049 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:41.959527 14874 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:41.966529 14793 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:42.014032 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:42.016316 10766 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:42.017875 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:41.947423 14816 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:42.020287 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:42.021317 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:42.002255 14782 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:42.036242 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:42.036912 11503 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 10.
I20260709 13:17:42.038451 11509 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
I20260709 13:17:42.039304 10871 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
W20260709 13:17:42.033695 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:42.040355 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:42.041005 11509 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 9.
I20260709 13:17:42.032974 10766 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:42.042050 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:42.031178 11506 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 8.
I20260709 13:17:42.047137 10872 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10.
I20260709 13:17:42.042060 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: true dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:42.048297 11507 raft_consensus.cc:3055] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:42.048610 11507 raft_consensus.cc:740] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:42.020782 14643 raft_consensus.cc:2804] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:42.002008 14818 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:42.002123 14852 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:42.051642 10766 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:42.014819 14770 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:42.053085 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:42.023128 14793 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:42.061671 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:42.060552 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:42.060340 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:42.070251 10766 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:42.071805 10766 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:42.073004 14826 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:42.073508 14826 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:42.073870 14826 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:42.080539 10941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 1853) took 3218 ms. Trace:
I20260709 13:17:42.081563 14764 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:42.082154 14764 raft_consensus.cc:697] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:42.082576 10766 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:42.074620 10945 raft_consensus.cc:1275] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:17:42.083071 14764 consensus_queue.cc:237] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:42.081115 10941 rpcz_store.cc:276] 0709 13:17:38.862120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:39.038044 (+175924us) service_pool.cc:224] Handling call
0709 13:17:39.038487 (+   443us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:39.493338 (+454851us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:42.080181 (+2586843us) spinlock_profiling.cc:243] Waited 69.8 ms on lock 0x7b54000a9bd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:42.080453 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":160549760}
I20260709 13:17:42.091260 10797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2333) took 3793 ms. Trace:
I20260709 13:17:42.091670 10797 rpcz_store.cc:276] 0709 13:17:38.297992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:38.298111 (+   119us) service_pool.cc:224] Handling call
0709 13:17:38.298400 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:40.256148 (+1957748us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:42.091014 (+1834866us) spinlock_profiling.cc:243] Waited 1.72 s on lock 0x7b5400211bd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:42.091191 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3944715520}
I20260709 13:17:42.092808 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:42.092823 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:42.093555 10797 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 10.
I20260709 13:17:42.094440 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:42.097677 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:42.097894 14772 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:42.098104 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:42.098430 14772 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:42.099200 14797 raft_consensus.cc:2804] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:42.099493 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:42.100250 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:42.100395 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:42.036273 14879 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:42.042374 14782 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:42.100855 10869 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 9.
I20260709 13:17:42.101100 14782 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:42.101073 14879 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:42.101748 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:42.055222 14643 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:42.063911 14726 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:42.096208 14795 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:42.041697 14869 raft_consensus.cc:2804] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:42.096423 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:42.055574 14818 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:42.096644 14841 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:42.046923 14878 consensus_queue.cc:1048] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 13:17:42.096966 14820 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:42.099591 14797 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:42.095707 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:42.053086 11507 consensus_queue.cc:260] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:42.013722 14874 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:42.103404 14879 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:42.157910 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:42.282248 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:42.100672 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:42.279271 10872 raft_consensus.cc:2393] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 11: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:42.279691 10871 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 10.
I20260709 13:17:42.283218 10869 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 12.
I20260709 13:17:42.285562 11507 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:42.287508 10870 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 10.
I20260709 13:17:42.289927 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:42.290154 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:42.290423 10868 raft_consensus.cc:2393] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 10: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:42.291437 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:42.291689 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:42.292064 11509 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 9.
I20260709 13:17:42.292495 11504 raft_consensus.cc:2393] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 7: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:42.291298 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:42.293475 10868 raft_consensus.cc:2393] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 10: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:42.053576 10874 raft_consensus.cc:2410] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:17:42.297596 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:42.100613 14772 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:42.054579 14836 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:17:42.061182 14770 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:42.299762 10876 raft_consensus.cc:1240] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 LEADER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:42.300067 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:42.302373 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:42.302664 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:42.307320 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:42.307541 10875 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 7.
I20260709 13:17:42.302891 14878 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:42.303007 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:42.126241 14662 consensus_queue.cc:1048] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.177s
I20260709 13:17:42.279742 14726 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:42.303599 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:42.204782 14842 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:42.308431 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:42.319675 10875 raft_consensus.cc:2393] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 9: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:42.321008 14816 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:42.318176 14726 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:42.321969 14793 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:42.317201 14836 raft_consensus.cc:697] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:42.323637 14836 consensus_queue.cc:237] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:42.202153 14826 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:42.326850 14826 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:42.327127 10797 raft_consensus.cc:2368] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 9. Current term is 10.
I20260709 13:17:42.327041 14816 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:42.327701 14816 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:42.329540 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1829) took 2538 ms. Trace:
I20260709 13:17:42.329839 14816 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:42.298069 14838 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:42.321009 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:42.331077 14838 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:42.331581 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:42.331847 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:42.332942 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:42.333516 10870 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 10.
I20260709 13:17:42.307565 11512 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:42.335920 11511 raft_consensus.cc:2393] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 11: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:42.336826 14793 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:42.334286 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:42.340018 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:42.340950 10870 raft_consensus.cc:2393] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 12: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:42.319640 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:42.320607 10950 raft_consensus.cc:1275] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:17:42.307559 11507 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 9.
I20260709 13:17:42.320649 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:42.310001 10915 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:42.301107 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:42.307920 11508 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10.
I20260709 13:17:42.313272 11503 raft_consensus.cc:2410] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:17:42.314576 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:42.344780 10871 raft_consensus.cc:2393] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 11: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:42.055670 14852 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:42.288084 14643 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:42.345783 11512 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
I20260709 13:17:42.289556 14797 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:42.329962 10873 rpcz_store.cc:276] 0709 13:17:39.792439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:40.081014 (+288575us) service_pool.cc:224] Handling call
0709 13:17:40.137505 (+ 56491us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:40.406248 (+268743us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:40.406292 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:40.406352 (+    60us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:40.406402 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:42.329327 (+1922925us) spinlock_profiling.cc:243] Waited 268 ms on lock 0x7b540037ad54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:42.329374 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:42.329483 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":617377152}
I20260709 13:17:42.288992 14869 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:42.347976 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:42.309319 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:42.348840 10945 raft_consensus.cc:1275] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:17:42.349283 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:42.350044 10942 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:42.350131 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:42.315656 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:42.351351 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:42.352043 10875 raft_consensus.cc:2393] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 10: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:42.348778 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:42.352540 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:42.352835 10946 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:17:42.353240 10945 raft_consensus.cc:2410] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:17:42.354012 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:42.349310 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:42.278220 14820 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:42.355860 11512 raft_consensus.cc:1275] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:17:42.388208 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:42.389029 10874 raft_consensus.cc:2393] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 10: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:42.310698 14770 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:42.354568 10941 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:42.390642 10766 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:42.368844 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:42.391182 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:42.316589 14878 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:42.392345 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:42.364876 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:42.355297 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:42.393723 10942 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 11.
W20260709 13:17:42.401716 10841 connection.cc:256] RPC call timeout handler was delayed by -2.26399160699998s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2036s for a response
W20260709 13:17:42.402257 10841 connection.cc:256] RPC call timeout handler was delayed by -1.5325810070000898s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19810000000000003s for a response
I20260709 13:17:42.402613 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:42.354768 10872 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:42.338737 14726 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:42.395330 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:42.407538 10875 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
I20260709 13:17:42.408393 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:42.408704 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: true dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:42.408937 10875 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 11.
I20260709 13:17:42.399150 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:42.409902 10870 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 11.
I20260709 13:17:42.410555 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:42.411325 10944 raft_consensus.cc:2393] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 9: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:42.351217 14869 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:42.394403 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:42.403733 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:42.418144 10946 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 11.
I20260709 13:17:42.419094 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:42.419435 10946 raft_consensus.cc:1275] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 13:17:42.419847 10871 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 9.
I20260709 13:17:42.297261 14841 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:42.393906 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:42.406956 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:42.402127 10945 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:17:42.415115 10941 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 10.
I20260709 13:17:42.423089 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:42.416602 10947 raft_consensus.cc:1240] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:17:42.416744 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:42.424412 10870 raft_consensus.cc:1240] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:17:42.417254 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:42.425575 10869 raft_consensus.cc:2410] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 14 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:17:42.425807 10870 raft_consensus.cc:1275] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:17:42.397573 14572 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:42.445356 10872 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
I20260709 13:17:42.427639 14726 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:42.428292 14841 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:42.429121 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:42.431030 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:42.432190 14772 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:42.403720 14863 raft_consensus.cc:2804] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:42.435133 14816 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:42.436558 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:42.438961 14894 log.cc:927] Time spent T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.270s	user 0.001s	sys 0.000s
I20260709 13:17:42.439101 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:42.470451 10796 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:17:42.470949 14772 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:42.471550 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:42.472151 10796 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 12.
I20260709 13:17:42.473054 14772 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:42.473663 14772 consensus_queue.cc:1048] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:42.474695 14841 consensus_queue.cc:1059] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:17:42.474895 14816 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:42.475723 14841 consensus_queue.cc:1048] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:42.441715 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:42.477849 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:42.478375 10795 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 10.
I20260709 13:17:42.413671 10766 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:42.479727 10942 raft_consensus.cc:1240] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:42.480506 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:42.480963 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:42.481384 10947 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 8.
I20260709 13:17:42.482508 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:42.483341 10946 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 10.
I20260709 13:17:42.483446 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:42.444489 10870 pending_rounds.cc:85] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 6
I20260709 13:17:42.484604 10947 raft_consensus.cc:2410] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 13 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:17:42.484790 10870 pending_rounds.cc:107] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:17:42.485193 10870 raft_consensus.cc:2889] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:17:42.488018 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 74425.4us
I20260709 13:17:42.448707 10950 raft_consensus.cc:1275] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:17:42.464720 10792 raft_consensus.cc:2410] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:17:42.463966 14863 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:42.404377 14435 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:42.440449 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:42.488137 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:42.507625 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:42.491240 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:42.472996 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:42.495427 10874 raft_consensus.cc:1275] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:17:42.446954 10916 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:42.490561 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:42.496642 10767 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:42.498339 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:42.512732 10767 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:42.514170 10767 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:42.519891 10767 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:42.536835 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:42.537439 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:42.537679 10870 raft_consensus.cc:1275] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:17:42.490007 10869 pending_rounds.cc:85] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 2
I20260709 13:17:42.539919 10869 pending_rounds.cc:107] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 8.3
I20260709 13:17:42.540300 10869 raft_consensus.cc:2889] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:42.538132 10792 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 10.
I20260709 13:17:42.537578 10949 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 9.
I20260709 13:17:42.542303 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:42.543035 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:42.545840 10792 raft_consensus.cc:2393] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 12: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:42.470566 14878 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:42.547109 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:42.546770 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:42.549643 10792 raft_consensus.cc:2410] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:17:42.548516 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:42.551759 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:42.552330 10795 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:42.555644 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:42.556491 10796 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 7.
W20260709 13:17:42.520092 11533 debug-util.cc:398] Leaking SignalData structure 0x7b08002da180 after lost signal to thread 9685
I20260709 13:17:42.561251 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:42.561846 11506 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:42.459731 14572 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:42.498939 14863 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:42.565327 11508 raft_consensus.cc:1240] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 12. Current term is 13. Ops: []
W20260709 13:17:42.571120 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 261112us
I20260709 13:17:42.497174 14764 consensus_queue.cc:1059] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 13:17:42.492513 14662 consensus_queue.cc:1059] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 13:17:42.576490 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:42.579898 11504 raft_consensus.cc:2410] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:17:42.581277 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:42.581418 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:42.563277 14878 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:42.584574 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:42.585820 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:42.586256 10875 raft_consensus.cc:1275] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:17:42.591711 11508 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 10.
I20260709 13:17:42.592337 10870 raft_consensus.cc:2368] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 9. Current term is 10.
W20260709 13:17:42.591259 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 186324us
I20260709 13:17:42.606518 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:42.607733 11508 raft_consensus.cc:2393] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 11: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
W20260709 13:17:42.609447 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 166086us
I20260709 13:17:42.610915 11478 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:42.578826 14726 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:42.613116 10915 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:17:42.596486 14900 log.cc:927] Time spent T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
W20260709 13:17:42.616571 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 169731us
I20260709 13:17:42.617835 14879 consensus_queue.cc:1059] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:42.584697 14816 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:42.595687 14901 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:42.618518 10915 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:42.590728 14878 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:42.624385 14878 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:42.629124 14818 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:42.629786 14753 raft_consensus.cc:2804] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:42.630115 14753 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:42.630338 14753 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:42.630470 11503 raft_consensus.cc:1240] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:42.615032 14662 raft_consensus.cc:2749] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:42.615947 14764 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:17:42.621533 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:42.623353 14879 raft_consensus.cc:2804] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:42.626268 14906 raft_consensus.cc:3055] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:17:42.626602 14838 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:42.624620 14820 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:42.635982 14882 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:42.635624 14878 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:42.638013 14753 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:42.635995 14435 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:42.621296 11477 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:42.620049 14726 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:42.611377 14730 consensus_queue.cc:1048] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.032s
W20260709 13:17:42.643652 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:42.622319 14795 consensus_queue.cc:1048] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:17:42.648922 14879 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:42.650754 14906 raft_consensus.cc:740] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:42.674118 14753 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:42.642696 14816 consensus_queue.cc:1048] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 13:17:42.654246 14662 raft_consensus.cc:2749] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:42.655726 14882 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:42.653650 14764 raft_consensus.cc:697] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:42.626902 14852 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:42.660044 14726 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:42.641305 14901 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:42.691443 14901 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:42.691855 14749 consensus_queue.cc:1048] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:42.692499 14906 consensus_queue.cc:260] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:42.693509 14906 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:42.685870 14882 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:42.697533 14770 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:42.699934 14891 log.cc:927] Time spent T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 1.048s	user 0.003s	sys 0.000s
I20260709 13:17:42.700037 14869 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:42.703694 14878 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:42.686030 14911 raft_consensus.cc:3055] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:17:42.704456 14911 raft_consensus.cc:740] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:42.704723 14869 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:42.705322 14911 consensus_queue.cc:260] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:42.706238 14911 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:17:42.707814 14782 consensus_meta.cc:287] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.606s	user 0.005s	sys 0.005s
I20260709 13:17:42.708308 14782 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:42.676926 14879 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:42.677178 14912 raft_consensus.cc:3055] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:42.711126 14874 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:42.711544 14912 raft_consensus.cc:740] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:42.682312 14826 raft_consensus.cc:3055] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:42.712313 14770 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:42.695415 14797 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:42.712679 14826 raft_consensus.cc:740] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:42.713558 14826 consensus_queue.cc:260] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:42.714358 14797 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:42.712364 14852 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:42.717314 14863 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:42.720425 14863 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:42.712468 14912 consensus_queue.cc:260] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:42.723906 14818 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:42.724145 14912 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:42.714521 14826 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:42.683610 14902 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:42.735733 14902 raft_consensus.cc:697] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 13 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:42.737139 14902 consensus_queue.cc:237] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:42.739725 14882 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:42.733873 14879 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:42.742065 14882 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:42.744712 14879 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:17:42.785346 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 265349us
I20260709 13:17:42.787931 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:42.793589 10841 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:42.795794 10801 raft_consensus.cc:2393] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 12: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:42.800083 10841 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:42.801108 10796 raft_consensus.cc:2393] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 12: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:42.805719 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:42.807029 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:42.809079 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:42.811038 10915 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:42.619700 10916 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:17:42.825899 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 206099us
W20260709 13:17:42.803516 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 314218us
I20260709 13:17:42.660022 14624 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:17:42.829682 10870 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 10.
I20260709 13:17:42.830183 14761 consensus_queue.cc:1048] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:42.831240 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:42.832264 10870 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 12.
I20260709 13:17:42.832839 14761 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:42.833421 14761 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:42.833949 10870 raft_consensus.cc:1240] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:17:42.835421 11507 raft_consensus.cc:1240] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:42.685127 14643 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:42.837656 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:42.702190 14764 consensus_queue.cc:237] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:42.834264 10795 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 11.
I20260709 13:17:42.863632 10801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 1801) took 1064 ms. Trace:
I20260709 13:17:42.870276 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:42.833983 14761 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:42.864849 10801 rpcz_store.cc:276] 0709 13:17:41.792801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:42.331326 (+538525us) service_pool.cc:224] Handling call
0709 13:17:42.848429 (+517103us) spinlock_profiling.cc:243] Waited 463 ms on lock 0x7b54000c1f54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:42.857292 (+  8863us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1064135552}
I20260709 13:17:42.675601 14435 consensus_queue.cc:1059] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.009s
I20260709 13:17:42.881915 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:42.883934 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:42.884538 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:42.884897 10801 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 7.
I20260709 13:17:42.885607 10800 raft_consensus.cc:2410] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:17:42.887234 14879 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:17:42.888075 14879 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:42.888711 14879 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:42.891146 10767 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:42.891875 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 81281.4us
I20260709 13:17:42.892133 14780 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:42.892768 14780 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:42.893049 10915 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:42.893401 14780 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:42.895388 10796 raft_consensus.cc:2410] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:17:42.895146 14786 consensus_queue.cc:1048] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:42.897231 10796 raft_consensus.cc:1240] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:17:42.891891 14572 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:42.899540 14874 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:42.900043 14874 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:42.900454 14874 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:42.903353 14911 raft_consensus.cc:2804] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:42.903559 10915 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:42.904013 14911 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:42.904481 14911 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:42.906504 14882 raft_consensus.cc:2804] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:17:42.923435 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 66643.8us
I20260709 13:17:42.928188 10796 raft_consensus.cc:1240] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 10. Current term is 11. Ops: [8.7-10.8]
I20260709 13:17:42.930034 10796 raft_consensus.cc:1240] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:42.931032 14826 raft_consensus.cc:2749] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:42.931348 10800 raft_consensus.cc:1240] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:17:42.935469 10946 raft_consensus.cc:1240] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 9. Current term is 10. Ops: []
W20260709 13:17:42.937460 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 46659us
I20260709 13:17:42.938736 10767 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:42.941167 10794 raft_consensus.cc:1240] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 11. Current term is 12. Ops: []
I20260709 13:17:42.944734 10793 raft_consensus.cc:1240] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 10. Current term is 11. Ops: []
I20260709 13:17:42.946765 10841 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:17:42.948628 14572 raft_consensus.cc:2804] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:42.949177 14572 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:42.949649 14572 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:42.891212 10916 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:42.953451 10800 raft_consensus.cc:1240] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:17:42.954360 10792 raft_consensus.cc:1275] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:17:42.962713 10795 raft_consensus.cc:2410] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 12 because replica has last-logged OpId of term: 9 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 3.
I20260709 13:17:42.956959 10798 raft_consensus.cc:1275] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 13:17:42.950722 10841 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:42.966699 10792 pending_rounds.cc:85] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 4
I20260709 13:17:42.968053 10798 pending_rounds.cc:85] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 5
I20260709 13:17:42.907285 14882 raft_consensus.cc:697] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:42.951678 14881 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:42.978711 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:42.978399 14786 raft_consensus.cc:2749] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:42.960865 10767 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:42.975261 10792 pending_rounds.cc:107] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:17:42.975813 10798 pending_rounds.cc:107] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 13:17:42.984259 10798 raft_consensus.cc:2889] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:42.983512 10792 raft_consensus.cc:2889] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:42.986456 10796 raft_consensus.cc:1240] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:17:42.973731 14826 raft_consensus.cc:2749] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:17:43.006064 14769 raft_consensus.cc:2749] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:42.979436 14881 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:43.017282 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 66723.2us
I20260709 13:17:43.017879 14881 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:43.018953 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:43.019914 14572 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:43.020709 14882 consensus_queue.cc:237] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:43.022890 10795 raft_consensus.cc:1275] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 13:17:43.023279 14836 raft_consensus.cc:2804] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:43.024047 14836 raft_consensus.cc:697] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:43.024343 10795 raft_consensus.cc:1275] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:17:43.025287 14836 consensus_queue.cc:237] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:43.027225 14841 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:43.028029 14841 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:43.033819 10766 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:17:43.035306 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 124251us
I20260709 13:17:43.039392 14782 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:43.044019 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:43.049431 10873 consensus_meta.cc:287] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.648s	user 0.016s	sys 0.004s
I20260709 13:17:43.050009 10873 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 9.
I20260709 13:17:43.051651 14671 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:43.019929 10944 raft_consensus.cc:2393] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 10: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:43.060577 10949 raft_consensus.cc:1275] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:17:43.061911 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:43.063338 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:43.063696 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:43.065130 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:43.066021 10942 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:43.063413 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:17:43.067209 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:43.067332 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:17:43.068084 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 125994us
I20260709 13:17:43.069226 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:17:43.054557 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 173766us
I20260709 13:17:43.062534 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:43.022544 14572 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:43.084299 14797 raft_consensus.cc:2749] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:43.084990 14770 consensus_queue.cc:1048] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:17:43.085835 14818 raft_consensus.cc:3055] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:17:43.103327 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 84811.2us
I20260709 13:17:43.105743 10948 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:17:43.109954 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:43.110260 14912 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:43.117317 14912 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:43.110544 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:43.121763 14822 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:43.122509 14822 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:43.114059 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 45107.4us
I20260709 13:17:43.123936 10767 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:43.115099 14797 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:43.114907 14781 consensus_queue.cc:1048] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:43.096812 14818 raft_consensus.cc:740] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:43.124555 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:43.102962 14782 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:43.130640 10796 raft_consensus.cc:1275] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:17:43.123108 11533 debug-util.cc:398] Leaking SignalData structure 0x7b08002d97a0 after lost signal to thread 10733
I20260709 13:17:43.082484 14911 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:43.134240 14755 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:43.134498 10842 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:43.134243 14675 consensus_queue.cc:1059] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.026s
W20260709 13:17:43.131590 11506 consensus_meta.cc:287] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.569s	user 0.010s	sys 0.010s
I20260709 13:17:43.137725 10842 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:43.137859 14572 raft_consensus.cc:2804] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:43.137965 14675 raft_consensus.cc:3055] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:43.138477 14572 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:43.139008 14572 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:43.138953 10842 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:43.141026 14879 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:43.143955 14726 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:43.146968 14797 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:43.084357  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:43.149796 14797 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:43.136590 14755 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:43.151336 14755 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:43.153420 14912 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:43.155923 14726 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:43.157452 10916 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:43.156850 14761 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:43.158639 14902 consensus_queue.cc:1048] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:43.159986 14761 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:43.160528 14902 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:43.161121 14902 raft_consensus.cc:697] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 10 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:17:43.164937 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:43.169570 14853 raft_consensus.cc:2804] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:43.169735 11506 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 10.
I20260709 13:17:43.170157 14853 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:43.170601 14853 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:43.170883 14793 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:17:43.171597 14897 log.cc:927] Time spent T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 13:17:43.149816 14643 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:43.169654 14508 raft_consensus.cc:2804] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:43.172663 14508 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:43.173053 14508 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:43.175261 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:43.176091 10947 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
I20260709 13:17:43.133085 14772 consensus_queue.cc:1048] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.202s
I20260709 13:17:43.138507 14675 raft_consensus.cc:740] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:43.129941 14841 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:43.162297 14902 consensus_queue.cc:237] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:43.177172 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:43.203703 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:43.204629 10947 raft_consensus.cc:2393] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 10: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:43.205703 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:43.205973 14818 consensus_queue.cc:260] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:43.175310 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 51572.8us
I20260709 13:17:43.207366 14818 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:43.207724 10767 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:43.208182 10797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 1818) took 1189 ms. Trace:
I20260709 13:17:43.208616 10797 rpcz_store.cc:276] 0709 13:17:42.019131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:42.097894 (+ 78763us) service_pool.cc:224] Handling call
0709 13:17:43.207771 (+1109877us) spinlock_profiling.cc:243] Waited 228 ms on lock 0x7b54003bddd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:43.207989 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":525042432}
I20260709 13:17:43.209578 10767 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:43.134320 14782 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:43.183626 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 59049.3us
I20260709 13:17:43.211071 14826 raft_consensus.cc:2749] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:43.212284 10876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 563) took 1065 ms. Trace:
I20260709 13:17:43.212678 10876 rpcz_store.cc:276] 0709 13:17:42.145348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:42.145632 (+   284us) service_pool.cc:224] Handling call
0709 13:17:42.278351 (+132719us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:42.313794 (+ 35443us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:43.210344 (+896550us) spinlock_profiling.cc:243] Waited 21.2 ms on lock 0x7b5400076054. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:43.210671 (+   327us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":48801024}
I20260709 13:17:43.213188 14782 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:43.213388 11478 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:43.183907 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 562) took 1057 ms. Trace:
I20260709 13:17:43.202064 11477 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:43.218264 14600 raft_consensus.cc:2804] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:17:43.186661 11533 debug-util.cc:398] Leaking SignalData structure 0x7b08002cd180 after lost signal to thread 10825
I20260709 13:17:43.191174 10944 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:43.191723 10946 raft_consensus.cc:1275] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:17:43.176962 11508 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 10.
I20260709 13:17:43.192236 10945 raft_consensus.cc:1240] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 8. Current term is 9. Ops: [6.8-8.9]
I20260709 13:17:43.218719 14600 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:43.225167 14600 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:43.206599 10947 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10.
I20260709 13:17:43.214394 10867 rpcz_store.cc:276] 0709 13:17:42.126752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:42.127233 (+   481us) service_pool.cc:224] Handling call
0709 13:17:42.165453 (+ 38220us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:42.412552 (+247099us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:42.412582 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:42.412622 (+    40us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:42.412678 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:43.183433 (+770755us) spinlock_profiling.cc:243] Waited 247 ms on lock 0x7b54001095d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:43.183518 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:43.183710 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":567814528}
I20260709 13:17:43.225476 10874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 564) took 1058 ms. Trace:
I20260709 13:17:43.231837 11477 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:43.231674 10948 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
I20260709 13:17:43.229462 14643 raft_consensus.cc:2804] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:17:43.239603 14643 raft_consensus.cc:697] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:43.239820 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:43.240126 11477 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:43.241425 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:43.240983 14643 consensus_queue.cc:237] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:43.241828 11477 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:43.242317 10947 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:43.237478 10842 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:43.232064 10874 rpcz_store.cc:276] 0709 13:17:42.166694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:42.413144 (+246450us) service_pool.cc:224] Handling call
0709 13:17:42.448343 (+ 35199us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:42.510551 (+ 62208us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:43.225007 (+714456us) spinlock_profiling.cc:243] Waited 28.4 ms on lock 0x7b54002d4454. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:43.225294 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":65324288}
I20260709 13:17:43.200155 14863 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:43.193262 14911 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:43.245203 14446 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:43.251065 14847 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:43.251394 11478 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:43.232522 14852 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:43.257637 14446 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:43.263706 14446 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:43.261972 14852 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:43.247843 14793 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:43.284078 11478 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:43.193396 14874 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:43.288372 14793 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:43.290704 14793 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:43.288599 14852 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:43.285351 14624 consensus_queue.cc:1059] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 13:17:43.292938 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:43.279327 10942 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
I20260709 13:17:43.185130 14675 consensus_queue.cc:260] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:43.271229 14858 raft_consensus.cc:2749] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:43.303321 10950 raft_consensus.cc:1240] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 11. Current term is 12. Ops: []
I20260709 13:17:43.303356 10916 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:43.266175 14782 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:43.258131 14863 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:43.308164 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 79113.8us
I20260709 13:17:43.311120 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:43.317739 14799 raft_consensus.cc:3055] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:17:43.320312 11533 debug-util.cc:398] Leaking SignalData structure 0x7b08003d2f60 after lost signal to thread 10900
I20260709 13:17:43.321560 10948 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 12.
I20260709 13:17:43.263952 14847 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:43.271754 14780 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:43.323516 11533 debug-util.cc:398] Leaking SignalData structure 0x7b08003d2ee0 after lost signal to thread 10901
I20260709 13:17:43.334666 10766 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:17:43.338238 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:43.336571 14607 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:43.341692 10797 raft_consensus.cc:1240] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:43.341925 14911 raft_consensus.cc:2804] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:43.342558 14911 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:43.343004 14911 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:43.347383 10799 raft_consensus.cc:1240] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 11. Current term is 12. Ops: []
I20260709 13:17:43.347591 14869 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:43.348152 14869 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:43.340476 14726 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:43.348889 14869 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:43.349066 14726 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:43.349949 11533 debug-util.cc:398] Leaking SignalData structure 0x7b08003b5d60 after lost signal to thread 10974
I20260709 13:17:43.350961 14882 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:17:43.351454 11533 debug-util.cc:398] Leaking SignalData structure 0x7b08002f5f20 after lost signal to thread 11536
I20260709 13:17:43.348120 14912 consensus_queue.cc:1048] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:17:43.351955 11533 debug-util.cc:398] Leaking SignalData structure 0x7b08003c8bc0 after lost signal to thread 11537
I20260709 13:17:43.291230 14755 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:43.310101 14770 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:43.347589 10766 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:43.260617 14826 consensus_queue.cc:1048] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 13:17:43.355297 10842 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:43.318356 14782 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:43.360919 10945 raft_consensus.cc:1275] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:17:43.319046 14671 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:43.331562 14761 raft_consensus.cc:2804] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:43.311189 14879 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:43.351639 14882 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:43.335374 14822 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:43.351536 14726 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:43.318423 14799 raft_consensus.cc:740] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:43.366941 10944 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
W20260709 13:17:43.367581 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 83727.4us
I20260709 13:17:43.365564 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:43.381260 10766 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:43.363924 14761 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:43.389875 10946 raft_consensus.cc:1240] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:43.387286 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:43.391613 10797 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:43.364845 14607 raft_consensus.cc:697] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 10 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
W20260709 13:17:43.395740 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:43.393735 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:43.398435 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 140075us
I20260709 13:17:43.383160 14781 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:17:43.404574 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 63703.6us
I20260709 13:17:43.406415 10799 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:43.409494 10949 raft_consensus.cc:1240] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 10 LEADER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:43.390573 14841 raft_consensus.cc:2749] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:17:43.408789 14726 raft_consensus.cc:2804] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:17:43.411676 14930 log.cc:927] Time spent T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.194s	user 0.001s	sys 0.000s
I20260709 13:17:43.412067 10793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2341) took 1222 ms. Trace:
I20260709 13:17:43.411679 14607 consensus_queue.cc:237] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:43.389932 14761 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:43.421386 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:43.427042 10944 raft_consensus.cc:1240] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:43.428280 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:43.423435 14822 consensus_queue.cc:1059] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 13:17:43.430174 14755 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:43.430239 14822 raft_consensus.cc:3055] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:43.417493 10793 rpcz_store.cc:276] 0709 13:17:42.189937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:42.945810 (+755873us) service_pool.cc:224] Handling call
0709 13:17:42.946304 (+   494us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:42.966161 (+ 19857us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:42.966237 (+    76us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:42.983965 (+ 17728us) log.cc:844] Serialized 26 byte log entry
0709 13:17:43.125723 (+141758us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:43.125788 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:43.125886 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:43.411808 (+285922us) raft_consensus.cc:1685] finished
0709 13:17:43.411839 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:43.411992 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4135,"raft.queue_time_us":19168,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":2548,"thread_start_us":41657,"threads_started":1,"wal-append.queue_time_us":240861}
I20260709 13:17:43.431272 10842 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:43.420447 14675 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:43.439450 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:43.422188 14781 raft_consensus.cc:697] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:43.443586 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:43.440891 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:43.440955 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:43.447738 11507 raft_consensus.cc:1275] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:17:43.442423 11508 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 12.
I20260709 13:17:43.448410 14446 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:43.449087 10916 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:43.424057 14726 raft_consensus.cc:697] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:43.406128 14508 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:43.397127 14770 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:43.454972 14772 raft_consensus.cc:2749] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:17:43.412719 14799 consensus_queue.cc:260] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:43.455184 11503 raft_consensus.cc:2393] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 9: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:43.456584 14799 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:17:43.459496 14925 log.cc:927] Time spent T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.290s	user 0.000s	sys 0.001s
I20260709 13:17:43.461684 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:43.462469 10941 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
I20260709 13:17:43.463608 14726 consensus_queue.cc:237] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:43.461691 11512 raft_consensus.cc:1275] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:17:43.467342 14863 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:43.468730 14882 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:43.432767 14822 raft_consensus.cc:740] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:43.475351 14879 raft_consensus.cc:2804] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:43.475571 14770 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:43.480662 10799 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 11.
W20260709 13:17:43.483994 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 60133.2us
W20260709 13:17:43.489444 11533 debug-util.cc:398] Leaking SignalData structure 0x7b0800375280 after lost signal to thread 14804
I20260709 13:17:43.490867 10916 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:43.493337 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:43.496134 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:43.501047 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:43.482856 14782 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:43.514395 10797 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 9.
I20260709 13:17:43.466536 14793 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:43.486181 14879 raft_consensus.cc:697] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 10 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:17:43.501096 14934 log.cc:927] Time spent T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:17:43.519164 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:43.526511 10867 raft_consensus.cc:2393] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 8: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:43.522729 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:43.514546 14780 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:43.529127 10875 raft_consensus.cc:1275] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:17:43.526512 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:43.530515 11508 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:43.531174 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 49200.5us
I20260709 13:17:43.532500 10916 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:43.533331 14882 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:43.533746 14882 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:43.534135 14882 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:17:43.534934 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:43.535346 10916 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
W20260709 13:17:43.535527 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:43.536363 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 53986.4us
W20260709 13:17:43.535455 11533 debug-util.cc:398] Leaking SignalData structure 0x7b08002a11e0 after lost signal to thread 14846
I20260709 13:17:43.541018 11508 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
I20260709 13:17:43.479858 14781 consensus_queue.cc:237] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:43.539317 10948 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 9.
I20260709 13:17:43.519271 14902 raft_consensus.cc:2749] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:17:43.561914 10944 raft_consensus.cc:1275] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:17:43.532517 14931 log.cc:927] Time spent T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.244s	user 0.003s	sys 0.000s
I20260709 13:17:43.568411 10947 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
I20260709 13:17:43.570130 10947 raft_consensus.cc:1240] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 10. Current term is 11. Ops: [6.4-10.6]
W20260709 13:17:43.564899 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:43.572458 10841 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:43.582803 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:17:43.587824 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:43.590548 14446 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:43.599365 14508 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:43.602895 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:17:43.601549 14933 log.cc:927] Time spent T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
W20260709 13:17:43.618340 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 45972.9us
I20260709 13:17:43.621002 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:43.572474 14600 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:43.627960 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 189898us
I20260709 13:17:43.630076 10875 raft_consensus.cc:2376] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 12. Re-sending same reply.
I20260709 13:17:43.632074 10943 raft_consensus.cc:1240] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:17:43.632607 14600 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:43.633420 10841 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:43.633659 10943 raft_consensus.cc:1240] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 13. Current term is 14. Ops: [13.6-13.6]
I20260709 13:17:43.634887 14772 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:43.635507 14772 raft_consensus.cc:697] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:43.607163 14869 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:43.636488 14772 consensus_queue.cc:237] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:43.637653 14869 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:43.637487 14572 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:43.638612 10842 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:43.639191 14634 raft_consensus.cc:2749] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:17:43.640118 10842 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:43.640344 14841 consensus_queue.cc:1048] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:43.641882 14508 raft_consensus.cc:2749] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:43.642362 14134 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:43.642589 14874 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:43.642762 14508 consensus_queue.cc:1059] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:43.644038 14780 raft_consensus.cc:3055] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:17:43.640617 14671 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:43.644443 14780 raft_consensus.cc:740] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:43.644670 14671 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:43.645550 14780 consensus_queue.cc:260] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:43.646627 10949 raft_consensus.cc:1275] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 13:17:43.647595 14928 log.cc:927] Time spent T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.462s	user 0.002s	sys 0.000s
I20260709 13:17:43.649055 10871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1809) took 1447 ms. Trace:
I20260709 13:17:43.649104 14793 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:43.649507 10871 rpcz_store.cc:276] 0709 13:17:42.207458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:42.420610 (+213152us) service_pool.cc:224] Handling call
0709 13:17:42.421013 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:42.906474 (+485461us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b180041ee40. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c80a5d4bc 00007f9c7ddab48b 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc5455 00007f9c80ead060 00007f9c80eadbfb 00007f9c80eb864b
0709 13:17:42.907583 (+  1109us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:42.907621 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:42.994397 (+ 86776us) log.cc:844] Serialized 26 byte log entry
0709 13:17:43.097638 (+103241us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:43.097712 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:43.097799 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:43.646706 (+548907us) spinlock_profiling.cc:243] Waited 3.23 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:17:43.648691 (+  1985us) raft_consensus.cc:1685] finished
0709 13:17:43.648716 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:43.648890 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":152148,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":9682,"spinlock_wait_cycles":33314944,"thread_start_us":255734,"threads_started":2,"wal-append.queue_time_us":122334}
I20260709 13:17:43.645960 10944 raft_consensus.cc:1275] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:17:43.651314 10841 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:43.653245 10841 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:43.653923 14847 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:43.656102 10943 raft_consensus.cc:1240] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 10. Current term is 11. Ops: []
I20260709 13:17:43.615433 14881 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:43.660688 14881 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:43.661211 14572 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:43.558923 14822 consensus_queue.cc:260] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:43.664775 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:17:43.665866 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:43.668252 14572 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:17:43.668736 14572 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:43.668839 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 133660us
I20260709 13:17:43.669214 14572 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:43.670560 10916 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:43.580269 14879 consensus_queue.cc:237] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:43.675650 14943 raft_consensus.cc:2749] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:43.677527 14671 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:43.688596 14841 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:43.689316 14841 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:43.689571 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:43.689808 14841 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:43.690301 10950 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:17:43.696128 14929 log.cc:927] Time spent T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.471s	user 0.002s	sys 0.000s
I20260709 13:17:43.688354 14671 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:43.699652 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:43.700953 11503 raft_consensus.cc:1240] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 8. Current term is 9. Ops: [6.8-8.9]
I20260709 13:17:43.701431 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:43.702765 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:43.703296 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:43.704584 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:43.704829 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:43.703378 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:43.701027 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:43.706548 11506 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:43.708469 10797 raft_consensus.cc:1240] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:43.705461 11510 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10.
I20260709 13:17:43.690029 14852 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:43.661157 14881 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:43.640161 14634 consensus_queue.cc:1059] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.084s
I20260709 13:17:43.724154 10916 leader_election.cc:400] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 11. Current term is 12.
I20260709 13:17:43.737111 10916 leader_election.cc:403] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:17:43.727283 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:43.738148 10942 raft_consensus.cc:2368] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for earlier term 10. Current term is 11.
I20260709 13:17:43.667109 14780 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:43.728976 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:43.737964 10915 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:43.742453 14911 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:43.745049 14911 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:43.760607 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:43.659489 14853 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:43.736914 14671 consensus_queue.cc:1059] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 13:17:43.746757 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:43.768193 10941 raft_consensus.cc:2393] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 9: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:43.747767 14874 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:43.747866 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:43.747012 14882 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:43.747056 10867 raft_consensus.cc:1240] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 11. Current term is 12. Ops: []
I20260709 13:17:43.750349 14852 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:43.750978 11503 raft_consensus.cc:1240] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:43.751605 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:43.754182 11506 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
I20260709 13:17:43.757658 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:43.758728 10943 raft_consensus.cc:1240] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:43.759760 10950 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 11.
I20260709 13:17:43.775530 10871 raft_consensus.cc:2393] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 11: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:17:43.755903 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:43.755115 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:43.760382 10801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1819) took 1731 ms. Trace:
I20260709 13:17:43.776433 10799 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 9.
I20260709 13:17:43.764655 10876 raft_consensus.cc:2393] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 10: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:43.771859 11477 leader_election.cc:400] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 election: Vote denied by peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553) with higher term. Message: Invalid argument: T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for earlier term 10. Current term is 11.
I20260709 13:17:43.772882 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:43.777554 11477 leader_election.cc:403] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
W20260709 13:17:43.781642 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:43.787626 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 145817us
I20260709 13:17:43.776340 10801 rpcz_store.cc:276] 0709 13:17:42.023645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:42.885682 (+862037us) service_pool.cc:224] Handling call
0709 13:17:42.972154 (+ 86472us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:43.007465 (+ 35311us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:43.007517 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:43.008406 (+   889us) log.cc:844] Serialized 26 byte log entry
0709 13:17:43.107127 (+ 98721us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:43.107176 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:43.107260 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:43.695365 (+588105us) spinlock_profiling.cc:243] Waited 2.29 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:17:43.697225 (+  1860us) raft_consensus.cc:1685] finished
0709 13:17:43.697248 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:43.754446 (+ 57198us) spinlock_profiling.cc:243] Waited 84.8 ms on lock 0x7b54002116d0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:43.754600 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16201,"raft.queue_time_us":39106,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":78,"spinlock_wait_cycles":200259200,"thread_start_us":98209,"threads_started":1,"wal-append.queue_time_us":216553}
I20260709 13:17:43.778469 14902 raft_consensus.cc:2764] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election decision vote started in defunct term 10: won
I20260709 13:17:43.787614 14879 raft_consensus.cc:2749] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 11. Current term is 12.
W20260709 13:17:43.796782 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 143609us
I20260709 13:17:43.786114 14818 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:43.775317 14874 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:43.824627 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 151293us
W20260709 13:17:43.826339 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:43.827064 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:43.826053 14772 consensus_queue.cc:1059] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 13:17:43.843132 14761 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:43.845593 14761 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:43.849187 14881 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:43.852623 14874 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:43.853781 14572 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:43.855113 10874 raft_consensus.cc:2393] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 11: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:43.855194 10842 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:43.856004 14572 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:43.856060 14675 raft_consensus.cc:2749] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:43.857046 10944 raft_consensus.cc:1240] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 12. Current term is 13. Ops: []
I20260709 13:17:43.863631 11477 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:43.864869 14572 consensus_queue.cc:1059] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:43.865121 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:43.866495 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:43.866689 14675 consensus_queue.cc:1048] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:43.867190 10941 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 9.
I20260709 13:17:43.868774 14572 consensus_queue.cc:1048] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:43.870551 14600 raft_consensus.cc:3055] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:17:43.871050 14600 raft_consensus.cc:740] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:43.871393 14572 consensus_queue.cc:1048] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:43.873037 14799 consensus_queue.cc:1059] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:17:43.874018 14675 consensus_queue.cc:1048] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:43.876345 14675 consensus_queue.cc:1048] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:43.868804 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:43.878504 10942 raft_consensus.cc:2393] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 14: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:43.880070 10942 raft_consensus.cc:1275] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 10 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:17:43.874444 14799 raft_consensus.cc:3055] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:43.868166 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:43.871963 14600 consensus_queue.cc:260] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:43.890287 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:43.891077 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:43.891178 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:43.884258 14799 raft_consensus.cc:740] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:43.893204 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:43.893147 14799 consensus_queue.cc:260] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:43.893954 10801 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 12.
I20260709 13:17:43.894282 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:43.895215 14882 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:43.889508 14947 raft_consensus.cc:3055] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:43.895955 14947 raft_consensus.cc:740] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:43.896904 14947 consensus_queue.cc:260] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:43.828243 14634 consensus_queue.cc:1059] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:17:43.888734 14948 consensus_queue.cc:1059] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:17:43.898381 10796 raft_consensus.cc:2410] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:17:43.893695 10948 raft_consensus.cc:1275] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:17:43.891000 10799 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 12.
I20260709 13:17:43.890901 14600 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:43.913368 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:43.914417 10795 raft_consensus.cc:2393] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 11: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:43.917937 10798 raft_consensus.cc:1275] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:17:43.836908 14841 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:43.921869 10794 raft_consensus.cc:1240] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:43.916464 11478 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:43.886229 14671 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:17:43.923259 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:43.924688 10766 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:43.906311 14799 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:43.927325 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:43.923236 10800 raft_consensus.cc:1240] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 10. Current term is 11. Ops: []
I20260709 13:17:43.923357 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:43.928824 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:43.928226 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:43.931401 10794 raft_consensus.cc:2393] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 11: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:43.938704 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:43.943491 11478 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:43.939424 10800 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:43.907450 14780 consensus_queue.cc:1059] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:17:43.928963 14947 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:43.958186 14881 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:43.958370 14446 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:17:43.912864 14912 consensus_queue.cc:1048] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:17:43.959399 14938 consensus_queue.cc:1048] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 13:17:43.965996 14959 raft_consensus.cc:2764] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:17:43.966740 14446 raft_consensus.cc:697] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 11 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:43.968201 10800 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 10.
I20260709 13:17:43.943166 14761 raft_consensus.cc:3055] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:17:43.992928 14761 raft_consensus.cc:740] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:43.994155 14761 consensus_queue.cc:260] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:43.995486 14761 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:43.939030 14853 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:43.944697 14948 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:43.999013 14853 raft_consensus.cc:3055] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:43.998986 14948 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:43.999662 14853 raft_consensus.cc:740] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:44.000597 14853 consensus_queue.cc:260] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:44.001277 14948 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:44.001881 14853 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:44.003280 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:44.003010 14852 consensus_queue.cc:1059] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:17:44.003888 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:43.991158 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:43.953660 14797 raft_consensus.cc:2749] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:17:43.973671 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 47733.7us
I20260709 13:17:44.006094 10841 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:43.952890 14671 raft_consensus.cc:697] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 11 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:44.011127 14841 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:44.011623 10841 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:44.012383 14948 raft_consensus.cc:2749] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:44.016813 10916 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:17:44.025732 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 55811.7us
I20260709 13:17:44.027366 11478 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:44.008733 14793 raft_consensus.cc:3055] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:17:44.029078 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 104473us
I20260709 13:17:44.004436 14852 consensus_queue.cc:1059] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:17:44.041754 10841 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:17:44.049376 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:44.049504 11478 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:44.052942 10766 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:44.056474 10766 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:44.063944 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:44.039683 14793 raft_consensus.cc:740] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:44.048120 14943 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:44.060551 10766 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:44.066694 14701 raft_consensus.cc:3055] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:17:44.067623 14671 consensus_queue.cc:237] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:44.079069 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:44.074779 14701 raft_consensus.cc:740] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:44.081624 10766 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:44.057054 14841 raft_consensus.cc:2749] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:44.041359 14948 consensus_queue.cc:1048] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:17:44.088279 10867 raft_consensus.cc:1240] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:17:44.088120 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:44.088786 10874 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:17:44.089103 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:44.070683 14858 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:44.071655 14912 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:44.072914 14755 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:44.074046 14943 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:44.085350 14964 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:44.097262 14961 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:44.097818 14961 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:44.098224 14961 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:44.098694 14726 consensus_queue.cc:1048] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:44.097478 14964 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:44.102963 14964 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:44.081772 14797 raft_consensus.cc:2804] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:17:44.107223 10916 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:17:44.107932 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:44.087090 14847 raft_consensus.cc:2804] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:17:44.088899 14959 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:17:44.118500 14954 log.cc:927] Time spent T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
W20260709 13:17:44.120002 14949 log.cc:927] Time spent T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.383s	user 0.005s	sys 0.000s
W20260709 13:17:44.125840 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:44.093648 14869 raft_consensus.cc:2804] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:17:44.128266 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 78847us
W20260709 13:17:44.128906 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 47597.5us
I20260709 13:17:44.096436 14912 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:44.124571 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:44.108497 14943 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:44.108711 14755 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:44.133252 14755 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:44.133363 10943 raft_consensus.cc:1275] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:17:44.133514 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:44.133857 14793 consensus_queue.cc:260] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:44.134687 10875 raft_consensus.cc:2410] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:17:44.135171 14793 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:44.081632 14643 consensus_queue.cc:1048] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:17:44.137349 11478 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:44.132658 14869 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:44.160620 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 130983us
I20260709 13:17:44.111327 14938 consensus_queue.cc:1048] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
I20260709 13:17:44.081346 14446 consensus_queue.cc:237] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:44.137032 14912 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:44.132016 14882 raft_consensus.cc:2804] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:44.171123 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:44.114235 14858 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:44.163520 14869 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:44.186105 10841 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:17:44.183892 10916 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:44.189949 10875 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:44.194319 10871 raft_consensus.cc:1275] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:17:44.196470 10874 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
W20260709 13:17:44.204169 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:44.183610 14882 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:44.097312 14701 consensus_queue.cc:260] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:44.200999 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:44.214339 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:44.216735 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:44.233052 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:44.234658 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:44.237129 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:44.234120 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:44.239742 10949 raft_consensus.cc:1240] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 11. Current term is 12. Ops: []
I20260709 13:17:44.240049 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:44.239180 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:44.241017 10942 raft_consensus.cc:2393] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 11: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:44.243533 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:44.194612 14797 raft_consensus.cc:697] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:44.204628 14799 raft_consensus.cc:2804] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:44.207536 14841 raft_consensus.cc:2804] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:44.203394 14874 raft_consensus.cc:2804] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:44.232389 14882 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:44.250890 11512 raft_consensus.cc:1240] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 10. Current term is 11. Ops: [8.6-10.7]
I20260709 13:17:44.260178 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:44.259729 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:44.206243 14781 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:44.251466 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:44.283321 14797 consensus_queue.cc:237] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:44.285315 10950 raft_consensus.cc:1240] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 10 LEADER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:17:44.237826 14822 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:44.286363 14781 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:44.254367 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:44.289974 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:44.290563 10841 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:44.240633 14847 raft_consensus.cc:697] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 11 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:44.293864 14847 consensus_queue.cc:237] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:44.295091 14964 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:44.253639 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:44.251375 11504 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
I20260709 13:17:44.297873 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:44.190289 14772 consensus_queue.cc:1048] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 13:17:44.206329 14959 raft_consensus.cc:697] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 9 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:44.258639 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:44.268738 14858 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:44.270473 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:44.271360 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:44.302531 10946 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:44.272485 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:44.277580 14701 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:44.255260 14799 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:44.280315 14869 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:44.280493 14961 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:44.289136 10875 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
I20260709 13:17:44.214197 14761 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:44.271577 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:44.307242 10767 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:44.308362 10842 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:44.330433 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 193199us
I20260709 13:17:44.330575 10876 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 9.
I20260709 13:17:44.332020 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:44.348529 10874 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
I20260709 13:17:44.339588 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:44.349779 10873 raft_consensus.cc:2368] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 9. Current term is 10.
I20260709 13:17:44.349551 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:44.258363 14874 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:17:44.350778 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:44.298681 14841 raft_consensus.cc:697] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:44.350191 11504 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 12.
I20260709 13:17:44.355532 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:44.331619 14134 raft_consensus.cc:3055] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:44.365370 14134 raft_consensus.cc:740] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:44.365762 11511 raft_consensus.cc:1275] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:17:44.365774 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:44.365381 14948 consensus_queue.cc:1048] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:17:44.366269 14134 consensus_queue.cc:260] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:44.367030 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:44.365152 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:44.367349 14134 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:44.367843 11512 raft_consensus.cc:2393] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 9: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:44.340651 14799 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:17:44.369463 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 71700.5us
I20260709 13:17:44.370609 10916 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:44.371351 10871 raft_consensus.cc:1240] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:17:44.333078 14853 consensus_queue.cc:1059] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:17:44.373095 14782 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:17:44.374033 14818 consensus_meta.cc:287] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.561s	user 0.008s	sys 0.003s
I20260709 13:17:44.374507 14818 raft_consensus.cc:2749] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553) with higher term. Message: Invalid argument: T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for earlier term 10. Current term is 11.
W20260709 13:17:44.376770 14969 log.cc:927] Time spent T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:17:44.372534 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:44.388520 10792 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:44.390620 11507 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 9.
I20260709 13:17:44.395318 10800 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:44.395536 10793 raft_consensus.cc:1240] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:17:44.391021 14965 consensus_queue.cc:1059] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:17:44.401167 14960 consensus_queue.cc:1059] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:17:44.401625 14958 consensus_queue.cc:1059] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:17:44.405089 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 96838.6us
W20260709 13:17:44.405344 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:44.358002 14874 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:44.355976 14970 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:44.425750 10870 raft_consensus.cc:1275] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:17:44.425931 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:17:44.427295 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:44.430089 10793 raft_consensus.cc:1275] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:17:44.355764 14508 consensus_queue.cc:1048] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 13:17:44.395536 14911 raft_consensus.cc:2749] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:17:44.432868 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:44.435721 11507 raft_consensus.cc:1275] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:17:44.436462 10842 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:44.362112 14634 consensus_queue.cc:1048] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 13:17:44.444926 11511 raft_consensus.cc:1240] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:17:44.445257 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:44.393569 14761 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:44.451146 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 144019us
I20260709 13:17:44.454118 10767 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:44.455804 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:44.456741 10800 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
I20260709 13:17:44.473183 10869 raft_consensus.cc:1275] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 10 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:17:44.483971 10792 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
I20260709 13:17:44.485114 14772 raft_consensus.cc:2749] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:17:44.495540 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 136380us
I20260709 13:17:44.500152 10841 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:44.502962 10841 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:44.447528 14882 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:44.490727 14675 raft_consensus.cc:3055] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:17:44.432142 14970 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:44.472244 14948 raft_consensus.cc:2749] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:44.471513 14911 consensus_queue.cc:1059] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:17:44.494025 14938 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:44.561506 14634 raft_consensus.cc:2749] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:17:44.559330 10941 raft_consensus.cc:2393] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 10: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
W20260709 13:17:44.577109 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 133074us
I20260709 13:17:44.563126 11505 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 11.
W20260709 13:17:44.558560 14966 log.cc:927] Time spent T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.352s	user 0.001s	sys 0.000s
I20260709 13:17:44.518702 14675 raft_consensus.cc:740] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:44.555454 14961 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:44.550212 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 51206.9us
I20260709 13:17:44.502882 14797 raft_consensus.cc:2749] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:17:44.465765 14959 consensus_queue.cc:237] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:44.640632 10798 raft_consensus.cc:1240] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 10. Current term is 11. Ops: []
I20260709 13:17:44.471581 14841 consensus_queue.cc:237] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:44.605379 10916 leader_election.cc:400] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 9. Current term is 10.
I20260709 13:17:44.646502 10916 leader_election.cc:403] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:17:44.608484 14781 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:44.608124 14799 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:44.616230 14981 log.cc:927] Time spent T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
W20260709 13:17:44.617273 14982 log.cc:927] Time spent T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 13:17:44.636025 14853 consensus_queue.cc:1059] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:44.637988 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:44.638301 14797 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:44.642921 10842 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:44.576545 14869 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:44.664685 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:44.665705 14643 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:44.665874 10766 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.116.132:40555 timed out after 1.690s (SENT) [suppressed 9 similar messages]
I20260709 13:17:44.666565 14643 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:44.669539 14643 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:44.597328 14983 consensus_queue.cc:1048] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.071s
I20260709 13:17:44.675231 11507 raft_consensus.cc:1240] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
W20260709 13:17:44.671365 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 66148.8us
I20260709 13:17:44.685079 11512 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
I20260709 13:17:44.630590 14912 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:44.635612 14938 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:44.705856 14978 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:17:44.706080 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 63277.9us
W20260709 13:17:44.707473 14937 log.cc:927] Time spent T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.548s	user 0.004s	sys 0.000s
I20260709 13:17:44.708772 14961 raft_consensus.cc:2749] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:17:44.709236 14998 consensus_queue.cc:1048] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:44.710014 14998 consensus_queue.cc:1243] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer e01ecd008edd4aa2b6b2b47efbc2ea87 log is divergent from this leader: its last log entry 8.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:17:44.710306 14572 raft_consensus.cc:3055] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:44.710855 14572 raft_consensus.cc:740] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:44.711939 14572 consensus_queue.cc:260] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:44.712971 14988 raft_consensus.cc:2749] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 9. Current term is 10.
I20260709 13:17:44.713173 10842 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:44.713416 14572 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:44.714120 14958 raft_consensus.cc:2749] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:44.715931 10842 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:44.715981 14965 raft_consensus.cc:3055] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:17:44.716958 14965 raft_consensus.cc:740] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:44.717921 14998 raft_consensus.cc:2804] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:44.719028 14985 raft_consensus.cc:2749] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:17:44.718215 14965 consensus_queue.cc:260] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:44.720386 14882 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:44.720592 14965 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:44.721302 10767 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:44.721671 14984 log.cc:927] Time spent T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.169s	user 0.000s	sys 0.001s
I20260709 13:17:44.722561 14643 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:44.716967 14961 consensus_queue.cc:1059] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:44.723368 10698 catalog_manager.cc:5697] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:17:44.726224 14977 log.cc:927] Time spent T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:17:44.718516 14998 raft_consensus.cc:697] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:44.727679 14998 consensus_queue.cc:237] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:44.731884 14945 raft_consensus.cc:3055] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:17:44.732333 14945 raft_consensus.cc:740] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:44.733108 14945 consensus_queue.cc:260] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:44.734277 14945 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:44.736105 10767 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:44.737823 11511 raft_consensus.cc:1275] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:17:44.734967 10698 catalog_manager.cc:5697] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:44.738974 14770 raft_consensus.cc:2749] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:17:44.738720 14959 consensus_queue.cc:1048] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:17:44.739737 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:44.739782 11509 raft_consensus.cc:1275] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:17:44.741426 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:44.750016 11511 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10.
I20260709 13:17:44.760205 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:44.764539 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:44.766127 10767 leader_election.cc:400] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 election: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 9. Current term is 10.
I20260709 13:17:44.768461 10767 leader_election.cc:403] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
W20260709 13:17:44.768913 14986 log.cc:927] Time spent T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
W20260709 13:17:44.769274 14947 consensus_meta.cc:287] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.808s	user 0.011s	sys 0.001s
I20260709 13:17:44.770357 14643 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:44.773744 11478 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:44.663973 14781 consensus_queue.cc:1048] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.087s
W20260709 13:17:44.777333 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 111481us
I20260709 13:17:44.779450 10946 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
W20260709 13:17:44.761798 14982 log.cc:927] Time spent T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:17:44.782483 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:44.784587 11508 raft_consensus.cc:1240] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:44.786777 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:44.787458 10867 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:44.762542 14761 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:44.672950 14797 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:44.787210 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:44.790906 10797 raft_consensus.cc:2410] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:17:44.791031 10766 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:44.779254 14847 raft_consensus.cc:2749] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:17:44.801255 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 48476.7us
I20260709 13:17:44.801357 10870 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10.
I20260709 13:17:44.706563 14978 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:44.810184 14993 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 9. Current term is 10.
W20260709 13:17:44.812750 14755 consensus_meta.cc:287] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.653s	user 0.012s	sys 0.002s
I20260709 13:17:44.813235 14755 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:44.815071 10799 raft_consensus.cc:1240] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:17:44.820866 10767 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:44.828773 10766 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:44.827543 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 297429us
I20260709 13:17:44.844712 10766 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:44.844928 10841 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:17:44.845889 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:44.844893 10946 raft_consensus.cc:1275] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:17:44.846601 10841 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:44.847741 14761 raft_consensus.cc:2749] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:17:44.799053 10875 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:17:44.848753 10875 raft_consensus.cc:606] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Received request to transfer leadership to TS e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:44.848191 10767 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:44.851338 10875 raft_consensus.cc:614] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:44.855193 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:44.777153 14675 consensus_queue.cc:260] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:44.837965 14964 consensus_meta.cc:287] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.542s	user 0.012s	sys 0.002s
I20260709 13:17:44.821961 14993 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:17:44.860345 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:44.862197 10841 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:44.865803 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:44.867435 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:44.890355 11512 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:17:44.851593 14701 consensus_meta.cc:287] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.511s	user 0.009s	sys 0.005s
I20260709 13:17:44.890689 10873 raft_consensus.cc:2368] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 10. Current term is 11.
I20260709 13:17:44.871987 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:44.876821 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 77208.5us
I20260709 13:17:44.851634 14822 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:17:44.849086 14799 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:44.889055 10797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39442 (request call id 507) took 1135 ms. Trace:
I20260709 13:17:44.897348 10841 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:44.897500 11478 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:44.854748 14780 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:17:44.851652 14643 raft_consensus.cc:2804] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:44.871443 10868 raft_consensus.cc:1240] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 7. Current term is 8. Ops: []
W20260709 13:17:44.856386 10916 connection.cc:256] RPC call timeout handler was delayed by -2.1277241889999914s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1503s for a response
I20260709 13:17:44.897758 10797 rpcz_store.cc:276] 0709 13:17:43.753944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:43.754061 (+   117us) service_pool.cc:224] Handling call
0709 13:17:44.888872 (+1134811us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54001cacd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:44.889000 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2337155456}
I20260709 13:17:44.896224 14841 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:17:44.895787 14968 consensus_queue.cc:1048] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 13:17:44.900127 14947 consensus_queue.cc:1059] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:17:44.900385 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:44.900859 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:44.901295 14989 raft_consensus.cc:2749] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:44.888677 14770 consensus_queue.cc:1059] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 13:17:44.901964 14993 consensus_queue.cc:1048] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:44.903622 10841 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:44.903805 10916 connection.cc:256] RPC call timeout handler was delayed by -2.0826636249998955s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1574s for a response
I20260709 13:17:44.868279 14782 raft_consensus.cc:2749] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:44.907555 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:44.909842 10767 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:44.871519 14983 raft_consensus.cc:2749] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:44.871579 14964 raft_consensus.cc:2749] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:44.906072 14643 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:17:44.948395 14943 consensus_meta.cc:287] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.789s	user 0.011s	sys 0.000s
W20260709 13:17:44.947947 14997 log.cc:927] Time spent T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
I20260709 13:17:44.955101 14675 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:44.900368 14972 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:44.941437 10875 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 8.
I20260709 13:17:44.952028 14978 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:44.953068 14970 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:44.947141 14755 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:44.947961 14912 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:44.954257 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 47852.8us
I20260709 13:17:44.905880 14780 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:44.954396 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 56963.1us
W20260709 13:17:44.952157 14999 log.cc:927] Time spent T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:17:44.949420 14947 raft_consensus.cc:3055] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:17:44.953421 14938 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:44.961906 14991 log.cc:927] Time spent T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
W20260709 13:17:44.977537 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 60093us
I20260709 13:17:45.060106 14968 raft_consensus.cc:3055] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:17:45.094525 14947 raft_consensus.cc:740] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:45.100389 14947 consensus_queue.cc:260] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:45.102160 14947 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:45.010385 11512 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
I20260709 13:17:45.012339 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:45.107988 10873 raft_consensus.cc:2393] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 12: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:45.025887 10870 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:17:45.051474 10916 connection.cc:256] RPC call timeout handler was delayed by -2.0646636249998664s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15940000000000001s for a response
I20260709 13:17:45.108248 11478 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:45.108593 10916 connection.cc:256] RPC call timeout handler was delayed by -2.0089130639998984s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16140000000000002s for a response
W20260709 13:17:45.108915 10916 connection.cc:256] RPC call timeout handler was delayed by -1.9654797690000123s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16160000000000002s for a response
W20260709 13:17:45.109305 10916 connection.cc:256] RPC call timeout handler was delayed by -1.9273683599999458s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1714s for a response
W20260709 13:17:45.109679 10916 connection.cc:256] RPC call timeout handler was delayed by -1.8769683599998643s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17700000000000002s for a response
W20260709 13:17:45.110059 10916 connection.cc:256] RPC call timeout handler was delayed by -1.8702636250000069s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18100000000000002s for a response
W20260709 13:17:45.025993 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 116272us
W20260709 13:17:45.052486 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 148938us
I20260709 13:17:45.117380 10841 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:45.056316 14797 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:45.086089 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:45.086989 10766 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:45.087574 10872 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 9.
I20260709 13:17:45.051962 14781 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:17:45.093084 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:45.058734 14989 raft_consensus.cc:2749] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:17:45.056512 14643 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:45.091751 14780 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:45.073074 10874 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 9.
I20260709 13:17:45.000715 14799 consensus_queue.cc:1048] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.111s
I20260709 13:17:45.067320 14943 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:45.131105 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:45.110548 10916 connection.cc:256] RPC call timeout handler was delayed by -1.7460636249998061s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1948s for a response
W20260709 13:17:45.134752 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:45.134720 10916 connection.cc:256] RPC call timeout handler was delayed by -1.7254241889997957s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.195s for a response
W20260709 13:17:45.135492 10916 connection.cc:256] RPC call timeout handler was delayed by -1.7049086079998688s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1579s for a response
W20260709 13:17:45.135967 10916 connection.cc:256] RPC call timeout handler was delayed by -1.6714622729998609s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1724s for a response
W20260709 13:17:45.136423 10916 connection.cc:256] RPC call timeout handler was delayed by -1.6504334530000051s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19890000000000002s for a response
W20260709 13:17:45.136884 10916 connection.cc:256] RPC call timeout handler was delayed by -1.5849636250000003s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2127s for a response
W20260709 13:17:45.137307 10916 connection.cc:256] RPC call timeout handler was delayed by -1.5490622729998904s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18600000000000003s for a response
W20260709 13:17:45.137742 10916 connection.cc:256] RPC call timeout handler was delayed by -1.5451334529998348s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2106s for a response
W20260709 13:17:45.138159 10916 connection.cc:256] RPC call timeout handler was delayed by -1.5352797689999989s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2094s for a response
I20260709 13:17:45.103809 14968 raft_consensus.cc:740] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:45.107698 14701 raft_consensus.cc:2764] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader election decision vote started in defunct term 12: won
W20260709 13:17:45.140321 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 53451.3us
I20260709 13:17:45.146463 11478 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:45.150151 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:45.123202 14781 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:45.121989 14822 raft_consensus.cc:697] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:17:45.138577 10916 connection.cc:256] RPC call timeout handler was delayed by -1.5264636249999057s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21920000000000003s for a response
W20260709 13:17:45.160983 10916 connection.cc:256] RPC call timeout handler was delayed by -1.5057636249998723s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2215s for a response
W20260709 13:17:45.161625 10916 connection.cc:256] RPC call timeout handler was delayed by -1.4569064529998741s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1915s for a response
I20260709 13:17:45.178375 10947 raft_consensus.cc:1240] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 11. Current term is 12. Ops: [11.5-11.5]
I20260709 13:17:45.183609 14755 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:45.148356 14793 consensus_queue.cc:1059] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
W20260709 13:17:45.183593 10916 connection.cc:256] RPC call timeout handler was delayed by -1.4371797690000676s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2203s for a response
I20260709 13:17:45.096285 14972 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:45.190733 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:45.142067 14869 raft_consensus.cc:2804] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:17:45.122884 14990 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:45.148386 14600 raft_consensus.cc:2749] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:17:45.210741 10916 connection.cc:256] RPC call timeout handler was delayed by -1.3842636249999032s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23500000000000001s for a response
I20260709 13:17:45.180011 15012 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:45.214187 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:45.220510 10945 raft_consensus.cc:1275] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:17:45.222559 10916 connection.cc:256] RPC call timeout handler was delayed by -1.2979622729999392s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21389999999999998s for a response
I20260709 13:17:45.211606 10873 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 8.
I20260709 13:17:45.210166 14772 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:45.223448 10943 raft_consensus.cc:1240] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:17:45.180754 14858 raft_consensus.cc:2749] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:17:45.215811 14991 log.cc:927] Time spent T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:17:45.176838 14781 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:17:45.261550 10916 connection.cc:256] RPC call timeout handler was delayed by -1.2907622729999275s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2147s for a response
W20260709 13:17:45.274521 10916 connection.cc:256] RPC call timeout handler was delayed by -1.2796334529998603s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24009999999999998s for a response
I20260709 13:17:45.273931 14972 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:45.271106 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:45.252208 14869 raft_consensus.cc:697] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:45.183686 14912 consensus_queue.cc:1059] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.069s
I20260709 13:17:45.269266 14761 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:45.272156 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:45.277745 10915 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:17:45.283072 10916 connection.cc:256] RPC call timeout handler was delayed by -1.0834797690001778s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2596s for a response
W20260709 13:17:45.283548 10916 connection.cc:256] RPC call timeout handler was delayed by -1.0282683599998563s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.27130000000000004s for a response
W20260709 13:17:45.280367 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 134015us
I20260709 13:17:45.284873 14987 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:45.259706 14990 raft_consensus.cc:697] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:45.215533 14755 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:45.301900 14134 consensus_queue.cc:1048] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:17:45.303511 15011 log.cc:927] Time spent T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 13:17:45.285446 14987 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:45.307547 15006 log.cc:927] Time spent T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 13:17:45.307742 14869 consensus_queue.cc:237] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.6, Last appended by leader: 6, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:45.288986 14761 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:45.310297 15013 raft_consensus.cc:3055] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:45.311731 10916 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:45.312875 15007 log.cc:927] Time spent T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
W20260709 13:17:45.313741 14874 consensus_meta.cc:287] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.886s	user 0.009s	sys 0.001s
I20260709 13:17:45.317353 10766 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:17:45.352401 10867 consensus_meta.cc:287] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.556s	user 0.012s	sys 0.000s
W20260709 13:17:45.318308 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:45.318879 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:45.352887 10867 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10.
I20260709 13:17:45.323401 10943 raft_consensus.cc:1275] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:17:45.356398 10870 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 11.
I20260709 13:17:45.359764 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:45.360129 10875 raft_consensus.cc:2393] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 10: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:45.361335 10870 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:45.281756 14772 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:45.326589 14995 consensus_queue.cc:1059] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:17:45.363993 11508 raft_consensus.cc:1275] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:17:45.364632 11510 raft_consensus.cc:1240] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 10. Current term is 11. Ops: [8.6-10.7]
I20260709 13:17:45.281419 15012 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:45.368456 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:45.320379 14912 consensus_queue.cc:1048] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:17:45.370678 10867 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
I20260709 13:17:45.361567 14964 consensus_queue.cc:1048] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 13:17:45.213380 14968 consensus_queue.cc:260] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:45.387468 10873 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10.
I20260709 13:17:45.352501 15013 raft_consensus.cc:740] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:17:45.366034 14965 consensus_meta.cc:287] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.645s	user 0.014s	sys 0.009s
I20260709 13:17:45.384768 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:45.313752 14781 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:45.386744 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:45.388935 10795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2381) took 1034 ms. Trace:
I20260709 13:17:45.367471 14970 raft_consensus.cc:2749] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:45.395229 14990 consensus_queue.cc:237] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:45.396198 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:45.397356 14446 raft_consensus.cc:3055] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:17:45.397994 14508 raft_consensus.cc:3055] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:17:45.422564 14508 raft_consensus.cc:740] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:45.398188 11505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56404 (request call id 734) took 1043 ms. Trace:
I20260709 13:17:45.423355 11505 rpcz_store.cc:276] 0709 13:17:44.354468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:44.354576 (+   108us) service_pool.cc:224] Handling call
0709 13:17:45.398023 (+1043447us) spinlock_profiling.cc:243] Waited 7.36 ms on lock 0x7b540070f3d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:45.398130 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16919296}
I20260709 13:17:45.398244 14987 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:45.400278 10799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2382) took 1045 ms. Trace:
I20260709 13:17:45.359027 14874 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:45.407886 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 124427us
I20260709 13:17:45.437799 10915 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:45.434093 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:45.408989 14755 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:45.410532 14781 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:45.453001 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:45.453790 10793 raft_consensus.cc:2368] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 10. Current term is 11.
I20260709 13:17:45.454672 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:45.455127 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:45.456007 10801 raft_consensus.cc:2410] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 14 because replica has last-logged OpId of term: 13 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 5.
I20260709 13:17:45.410833 14772 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:45.328593 14822 consensus_queue.cc:237] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:45.416662 11506 raft_consensus.cc:1240] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 11 LEADER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:17:45.419164 10875 raft_consensus.cc:2410] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 14 because replica has last-logged OpId of term: 13 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 5.
I20260709 13:17:45.456813 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:45.457546 10801 raft_consensus.cc:2393] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 13: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:45.458314 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:45.419608 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 102333us
W20260709 13:17:45.400811 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:45.460493 10766 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:45.461647 10766 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:45.455855 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:45.462956 10793 raft_consensus.cc:2368] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 12. Current term is 13.
I20260709 13:17:45.420365 14968 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:45.426312 10799 rpcz_store.cc:276] 0709 13:17:44.354914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:44.355028 (+   114us) service_pool.cc:224] Handling call
0709 13:17:44.391707 (+ 36679us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:44.815544 (+423837us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:45.400022 (+584478us) spinlock_profiling.cc:243] Waited 423 ms on lock 0x7b54003be2d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:45.400198 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":973022464}
I20260709 13:17:45.434129 10800 raft_consensus.cc:1240] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 12. Current term is 13. Ops: [10.6-12.7]
I20260709 13:17:45.427670 10870 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 13.
I20260709 13:17:45.436442 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:45.408206 15013 consensus_queue.cc:260] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:45.440276 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:45.440434 14964 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:45.438568 14978 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:45.440829 10797 raft_consensus.cc:1240] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 11. Current term is 13. Ops: [11.6-11.6]
I20260709 13:17:45.440460 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:45.489991 10942 raft_consensus.cc:1240] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 10. Current term is 11. Ops: []
I20260709 13:17:45.443392 10876 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:17:45.490657 10876 raft_consensus.cc:606] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Received request to transfer leadership to TS e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:45.444701 14987 raft_consensus.cc:2749] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:17:45.444459 15019 consensus_queue.cc:1048] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:17:45.440286 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 128625us
I20260709 13:17:45.424642 14446 raft_consensus.cc:740] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:45.468638 14781 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:17:45.450079 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 53188.3us
I20260709 13:17:45.450276 10947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 1914) took 1325 ms. Trace:
I20260709 13:17:45.467350 10801 raft_consensus.cc:2393] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 9: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:45.468400 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:45.407929 14761 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:45.392804 10795 rpcz_store.cc:276] 0709 13:17:44.354618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:44.354736 (+   118us) service_pool.cc:224] Handling call
0709 13:17:44.368672 (+ 13936us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:44.398264 (+ 29592us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:17:44.441368 (+ 43104us) log.cc:844] Serialized 14 byte log entry
0709 13:17:44.634382 (+193014us) log.cc:844] Serialized 14 byte log entry
0709 13:17:44.640136 (+  5754us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:44.640602 (+   466us) log.cc:844] Serialized 26 byte log entry
0709 13:17:44.640736 (+   134us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:44.640768 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:44.640829 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:45.388704 (+747875us) raft_consensus.cc:1685] finished
0709 13:17:45.388725 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:45.388870 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39073,"prepare.queue_time_us":46185,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":72,"raft.queue_time_us":39558,"raft.run_cpu_time_us":277,"raft.run_wall_time_us":6380,"thread_start_us":133566,"threads_started":1,"wal-append.queue_time_us":332402}
I20260709 13:17:45.471760 10796 raft_consensus.cc:2393] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 10: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:45.471731 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:45.505422 10947 rpcz_store.cc:276] 0709 13:17:44.124905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:44.125061 (+   156us) service_pool.cc:224] Handling call
0709 13:17:44.125534 (+   473us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:45.211892 (+1086358us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:45.450020 (+238128us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54001e1c54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:45.450198 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2414518656}
I20260709 13:17:45.472376 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:45.510931 10867 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 11.
I20260709 13:17:45.511763 10794 raft_consensus.cc:1275] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:17:45.512223 10867 raft_consensus.cc:1275] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:17:45.512559 10867 pending_rounds.cc:85] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 5
I20260709 13:17:45.512789 10867 pending_rounds.cc:107] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:17:45.483211 10842 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:45.491816 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:45.423629 14508 consensus_queue.cc:260] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:45.498272 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:45.449047 14943 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:45.502269 10767 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:45.509179 10798 raft_consensus.cc:2410] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:17:45.482527 14970 raft_consensus.cc:2749] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:17:45.526448 10799 raft_consensus.cc:2393] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 10: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:45.513099 10794 raft_consensus.cc:1275] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:17:45.513576 10793 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:45.513023 10867 raft_consensus.cc:2889] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:45.484365 14948 consensus_queue.cc:1048] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:17:45.515805 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:17:45.520032 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:45.469069 14755 consensus_queue.cc:1048] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:17:45.525290 10767 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:45.489112 14912 consensus_queue.cc:1059] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 13:17:45.505105 14781 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:45.530153 10801 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 11.
I20260709 13:17:45.549340 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:45.581085 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 135503us
I20260709 13:17:45.582676 10795 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:45.553117 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:45.553792 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:45.493314 14964 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:45.583101 10915 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:45.598351 10943 raft_consensus.cc:1275] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:17:45.531294 14782 raft_consensus.cc:2804] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:45.556365 10798 raft_consensus.cc:1240] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 LEADER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:17:45.530704 14572 raft_consensus.cc:2804] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:45.559376 15020 consensus_queue.cc:1048] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:17:45.560299 14853 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:45.562577 10873 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:17:45.568418 14781 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:45.564478 14446 consensus_queue.cc:260] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:45.566536 14874 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:45.493938 14978 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:45.635779 14675 consensus_meta.cc:287] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.567s	user 0.010s	sys 0.001s
I20260709 13:17:45.576279 15012 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:45.577842 10841 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
W20260709 13:17:45.640967 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 63241.2us
I20260709 13:17:45.600443 10867 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:45.555572 10767 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:17:45.643494 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 88018.2us
I20260709 13:17:45.572861 14943 raft_consensus.cc:2804] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:45.522730 15013 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:45.652753 10794 raft_consensus.cc:1275] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:17:45.573030 14983 raft_consensus.cc:2804] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:45.612033 10798 raft_consensus.cc:1240] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:17:45.612339 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:45.656080 10943 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 11.
I20260709 13:17:45.613726 10799 raft_consensus.cc:1217] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Deduplicated request from leader. Original: 6.5->[7.6-11.8]   Dedup: 7.6->[9.7-11.8]
I20260709 13:17:45.615955 10873 raft_consensus.cc:606] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 LEADER]: Received request to transfer leadership to TS e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:45.657073 10799 pending_rounds.cc:85] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 6
I20260709 13:17:45.657397 10799 pending_rounds.cc:107] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 13:17:45.657622 10799 raft_consensus.cc:2889] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:45.599149 14508 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:17:45.668325 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:45.596076 14988 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:45.668808 15003 consensus_queue.cc:1059] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:45.669196 14988 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:45.670607 14943 raft_consensus.cc:697] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:45.671006 15003 raft_consensus.cc:3055] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:45.671432 15003 raft_consensus.cc:740] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:45.671640 14988 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:45.619031 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 87641.3us
I20260709 13:17:45.672384 15003 consensus_queue.cc:260] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:45.673810 15003 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:45.674029 10916 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:45.675457 15020 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:45.675872 15020 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:45.676258 15020 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:45.622625 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:45.678056 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:45.624701 10941 raft_consensus.cc:1275] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:17:45.678735 10941 pending_rounds.cc:85] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 6
I20260709 13:17:45.679033 10941 pending_rounds.cc:107] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:17:45.679253 10941 raft_consensus.cc:2889] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:45.650331 14853 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:45.685480 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:45.686091 10941 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:45.686347 14853 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:45.622370 10797 raft_consensus.cc:1240] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 10. Current term is 11. Ops: []
I20260709 13:17:45.613516 15021 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:45.687196 15021 raft_consensus.cc:697] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 13 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:45.687454 11478 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:45.688102 15021 consensus_queue.cc:237] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:45.688635 11478 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:45.688943 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:45.689667 10792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 511) took 1051 ms. Trace:
I20260709 13:17:45.567482 14780 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:45.612736 14847 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:45.701784 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:45.702669 10943 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
I20260709 13:17:45.703820 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:45.704572 10943 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 11.
I20260709 13:17:45.705437 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:45.690026 10792 rpcz_store.cc:276] 0709 13:17:44.638130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:44.649183 (+ 11053us) service_pool.cc:224] Handling call
0709 13:17:44.649618 (+   435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:44.844453 (+194835us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:44.844496 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:44.844550 (+    54us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:17:44.844583 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:45.689457 (+844874us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b54002120d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:45.689501 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:45.689613 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":433932672}
I20260709 13:17:45.636644 14978 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:45.609741 14782 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:45.639436 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:45.712731 10942 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:45.639542 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:45.650141 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:45.645452 10767 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:45.650576 10793 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 10.
I20260709 13:17:45.656513 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:45.564646 11506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56410 (request call id 704) took 1365 ms. Trace:
I20260709 13:17:45.652935 14989 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:45.662765 14772 consensus_queue.cc:1059] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:17:45.663726 14643 consensus_meta.cc:287] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.524s	user 0.009s	sys 0.001s
I20260709 13:17:45.664228 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:45.665534 14964 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:45.667024 14983 raft_consensus.cc:697] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:45.667541 14761 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:45.668213 10842 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:45.587868 14987 consensus_queue.cc:1059] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 13:17:45.653532 15012 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:45.671646 14943 consensus_queue.cc:237] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:45.628273 15008 log.cc:927] Time spent T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.570s	user 0.003s	sys 0.000s
I20260709 13:17:45.693076 10795 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
I20260709 13:17:45.693243 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:45.690092 10797 raft_consensus.cc:2410] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:17:45.694449 10869 raft_consensus.cc:1275] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 13:17:45.759305 10869 pending_rounds.cc:85] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 5
I20260709 13:17:45.759714 10869 pending_rounds.cc:107] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 13:17:45.760042 10869 raft_consensus.cc:2889] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:45.696424 10950 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 9.
I20260709 13:17:45.676891 10916 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:45.618567 14572 raft_consensus.cc:697] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:45.716041 10801 raft_consensus.cc:1240] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
W20260709 13:17:45.721196 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 75821.6us
I20260709 13:17:45.715927 11506 rpcz_store.cc:276] 0709 13:17:44.199513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:44.199632 (+   119us) service_pool.cc:224] Handling call
0709 13:17:44.199946 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:45.416399 (+1216453us) spinlock_profiling.cc:243] Waited 16.3 ms on lock 0x7b540017c120. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80e2cdc0 00007f9c80eb71c0 00007f9c80eb8231 00007f9c80eb4d8f 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c
0709 13:17:45.456840 (+ 40441us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:45.564319 (+107479us) spinlock_profiling.cc:243] Waited 836 ms on lock 0x7b54001f90d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:45.564551 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1960106112}
I20260709 13:17:45.722473 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:45.764034 10943 raft_consensus.cc:2410] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:17:45.723723 10867 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
I20260709 13:17:45.725967 10793 raft_consensus.cc:3055] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:17:45.727437 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:45.735549 10876 raft_consensus.cc:1240] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:17:45.764581 10793 raft_consensus.cc:740] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:45.722314 14793 consensus_queue.cc:1059] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
W20260709 13:17:45.747265 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 79128.2us
I20260709 13:17:45.769233 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:45.769842 10867 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10.
I20260709 13:17:45.720834 14782 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:45.751523 15012 raft_consensus.cc:697] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:45.722078 14974 raft_consensus.cc:2749] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:17:45.722774 14858 raft_consensus.cc:2749] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:17:45.761879 10942 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10.
I20260709 13:17:45.657157 14134 consensus_queue.cc:1048] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.085s
I20260709 13:17:45.745512 10792 raft_consensus.cc:1275] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:17:45.766727 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:45.765411 14978 raft_consensus.cc:2804] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:45.741914 14989 raft_consensus.cc:697] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:45.780807 10793 consensus_queue.cc:260] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:45.792285 10793 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:45.777065 10767 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:45.793444 10767 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:45.777696 10867 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:17:45.796851 10867 raft_consensus.cc:606] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Received request to transfer leadership to TS e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:45.775112 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:45.783766 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:45.726810 14847 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:45.735643 14643 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:45.771309 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 79371.3us
W20260709 13:17:45.785400 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 108625us
I20260709 13:17:45.783625 10943 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 9.
I20260709 13:17:45.811589 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:45.812418 10943 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:17:45.765857 14987 raft_consensus.cc:2749] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:45.752072 14853 raft_consensus.cc:2749] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:45.771797 14822 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:45.804689 10942 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 12.
I20260709 13:17:45.810066 11478 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:45.804519 10950 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 9.
I20260709 13:17:45.805104 10867 raft_consensus.cc:614] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:45.748638 14761 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:45.819252 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:45.851657 10873 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 9.
I20260709 13:17:45.846410 10916 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:45.849783 10867 raft_consensus.cc:1275] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:17:45.846221 10942 raft_consensus.cc:1275] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 13:17:45.841624 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:45.821929 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:45.826610 10841 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:45.861299 10867 raft_consensus.cc:1275] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:17:45.862648 10867 raft_consensus.cc:1240] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 10. Current term is 11. Ops: []
I20260709 13:17:45.867187 10841 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:45.806932 14970 raft_consensus.cc:2804] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:45.803620 15013 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:17:45.846026 15030 log.cc:927] Time spent T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:17:45.869383 10793 raft_consensus.cc:1275] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:17:45.904053 10793 pending_rounds.cc:85] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 7
I20260709 13:17:45.904407 10793 pending_rounds.cc:107] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:17:45.904718 10793 raft_consensus.cc:2889] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:45.820497 14964 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:45.895870 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:45.847842 14675 raft_consensus.cc:3055] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:45.909312 10942 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 12.
W20260709 13:17:45.911124 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 43729.9us
I20260709 13:17:45.912014 10841 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:45.804518 14972 consensus_queue.cc:1048] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 13:17:45.867873 14853 raft_consensus.cc:2804] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:45.922320 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:45.894668 14970 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:45.848100 14822 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:45.860563 14948 consensus_queue.cc:1059] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.057s
I20260709 13:17:45.803467 14983 consensus_queue.cc:237] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:45.893911 14945 raft_consensus.cc:2749] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:45.928035 10841 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
W20260709 13:17:45.961721 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 115423us
I20260709 13:17:45.915477 14965 raft_consensus.cc:2749] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:45.910174 14990 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
W20260709 13:17:45.929914 15034 log.cc:927] Time spent T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:17:45.991735 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:45.918401 14853 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:45.995121 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 185143us
I20260709 13:17:45.917303 14947 raft_consensus.cc:2804] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:45.910552 14675 raft_consensus.cc:740] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:45.927304 14970 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:45.999589 10868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1878) took 1727 ms. Trace:
W20260709 13:17:46.003424 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 75464.8us
I20260709 13:17:46.009845 10916 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:45.916592 14964 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:45.906956 15012 consensus_queue.cc:237] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:45.988919 14780 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:45.997143 14974 raft_consensus.cc:2804] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:46.015991 10868 rpcz_store.cc:276] 0709 13:17:44.271754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:44.272980 (+  1226us) service_pool.cc:224] Handling call
0709 13:17:44.273269 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:44.907060 (+633791us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:45.999276 (+1092216us) spinlock_profiling.cc:243] Waited 598 ms on lock 0x7b540009bad4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:45.999518 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1375207168}
I20260709 13:17:45.918310 15037 consensus_queue.cc:1059] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.096s
W20260709 13:17:46.021239 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 155019us
I20260709 13:17:46.001422 14761 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:46.013808 14841 raft_consensus.cc:2804] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:45.912088 14572 consensus_queue.cc:237] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:46.028614 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:46.029862 10916 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:46.037424 10875 raft_consensus.cc:1240] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:17:46.059569 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:46.069792 11505 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
I20260709 13:17:46.023272 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:46.013892 14853 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:46.067546 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:46.068192 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:46.066288 15013 raft_consensus.cc:697] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:46.122020 11507 raft_consensus.cc:2393] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 8: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:46.022720 14965 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:46.123823 11508 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 9.
I20260709 13:17:46.013494 14947 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:46.019955 14882 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:46.011348 14990 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:46.158969 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:46.159518 10941 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 13.
I20260709 13:17:46.062822 14945 raft_consensus.cc:2749] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:45.983060 14989 consensus_queue.cc:237] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:46.171571 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:17:46.166019 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 136213us
I20260709 13:17:46.172271 10793 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 10.
I20260709 13:17:46.172710 10916 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:46.062592 14988 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:46.173537 14988 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:46.176129 14988 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:46.189340 11510 raft_consensus.cc:2393] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 13: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:46.068485 14841 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:46.027689 14675 consensus_queue.cc:260] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:46.200910 10874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1896) took 1086 ms. Trace:
I20260709 13:17:46.204222 10874 rpcz_store.cc:276] 0709 13:17:45.114030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:45.636399 (+522369us) service_pool.cc:224] Handling call
0709 13:17:45.636714 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:45.637107 (+   393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:45.637132 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:45.637627 (+   495us) log.cc:844] Serialized 26 byte log entry
0709 13:17:45.725219 (+ 87592us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:45.725393 (+   174us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:45.725520 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:46.200663 (+475143us) raft_consensus.cc:1685] finished
0709 13:17:46.200717 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:46.200851 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":875,"raft.run_cpu_time_us":72,"raft.run_wall_time_us":73,"thread_start_us":86656,"threads_started":1,"wal-append.queue_time_us":127287}
I20260709 13:17:46.205799 10870 raft_consensus.cc:1275] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:17:46.210350 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:46.211575 11505 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:46.184449 15037 consensus_queue.cc:1243] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer e01ecd008edd4aa2b6b2b47efbc2ea87 log is divergent from this leader: its last log entry 11.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:17:46.156440 14974 raft_consensus.cc:697] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 10 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:46.165627 14947 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:17:46.217654 10872 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49290 (request call id 580) took 1772 ms (client timeout 1576 ms). Trace:
I20260709 13:17:46.218470 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:17:46.199138 15042 log.cc:927] Time spent T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.185s	user 0.001s	sys 0.000s
I20260709 13:17:46.167687 15021 raft_consensus.cc:2749] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:46.185899 14780 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:17:46.230413 10872 rpcz_store.cc:269] 0709 13:17:44.432640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:44.433183 (+   543us) service_pool.cc:224] Handling call
0709 13:17:46.205226 (+1772043us) spinlock_profiling.cc:243] Waited 631 ms on lock 0x7b54001095d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:46.205336 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1451323648}
I20260709 13:17:46.196503 15038 raft_consensus.cc:3055] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:17:46.194067 14761 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:46.228619 15044 log.cc:927] Time spent T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 13:17:46.206060 15033 log.cc:927] Time spent T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.300s	user 0.001s	sys 0.000s
I20260709 13:17:46.184703 14965 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:46.195158 14643 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:46.236327 10941 raft_consensus.cc:1275] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:17:46.206382 14841 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:46.201534 14988 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:17:46.238562 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:46.301283 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:46.199597 14989 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:46.205572 14847 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:46.302078 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 105820us
I20260709 13:17:46.320254 11477 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:46.320550 11505 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 14.
I20260709 13:17:46.240455 15038 raft_consensus.cc:740] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:46.275808 14675 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:46.322666 10841 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:46.329048 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 156449us
W20260709 13:17:46.334564 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:46.241287 14780 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:46.245553 14761 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:46.327626 14793 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:17:46.348450 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:46.335508 15054 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:17:46.341497 14988 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:46.406621 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:46.323135 14989 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:46.412956 10766 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:46.414189 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:46.419574 10941 raft_consensus.cc:2393] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 10: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
W20260709 13:17:46.374547 10943 consensus_meta.cc:287] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.519s	user 0.007s	sys 0.000s
W20260709 13:17:46.414105 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 47902.1us
I20260709 13:17:46.368980 14793 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:46.441630 10766 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:17:46.412022 15036 log.cc:927] Time spent T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.399s	user 0.001s	sys 0.001s
W20260709 13:17:46.425271 15052 log.cc:927] Time spent T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 13:17:46.433233 10949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2324) took 1690 ms. Trace:
I20260709 13:17:46.436147 14965 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:46.415853 15045 log.cc:927] Time spent T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.221s	user 0.000s	sys 0.001s
I20260709 13:17:46.412227 15038 consensus_queue.cc:260] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:46.438470 14780 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:46.341230 14974 consensus_queue.cc:237] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:46.327111 15013 consensus_queue.cc:237] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:46.375890 14995 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:46.445581 15012 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:46.363451 14822 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:46.411594 15057 log.cc:927] Time spent T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.083s	user 0.001s	sys 0.001s
W20260709 13:17:46.414609 14781 consensus_meta.cc:287] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.759s	user 0.009s	sys 0.002s
I20260709 13:17:46.420773 10943 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
I20260709 13:17:46.450551 10947 raft_consensus.cc:1275] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:17:46.414486 14643 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:46.448565 10949 rpcz_store.cc:276] 0709 13:17:44.745295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:44.748257 (+  2962us) service_pool.cc:224] Handling call
0709 13:17:44.748719 (+   462us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:44.749206 (+   487us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:17:44.749248 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:44.760236 (+ 10988us) log.cc:844] Serialized 26 byte log entry
0709 13:17:45.848859 (+1088623us) spinlock_profiling.cc:243] Waited 3.23 ms on lock 0x7b180043a5c0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a550d1 00007f9c80e70a0f
0709 13:17:46.017760 (+168901us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:46.017804 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:46.017883 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:46.432820 (+414937us) raft_consensus.cc:1685] finished
0709 13:17:46.432848 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:46.433136 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3605,"raft.queue_time_us":3369,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":11739,"spinlock_wait_cycles":7424256,"thread_start_us":168723,"threads_started":1,"wal-append.queue_time_us":1562614}
I20260709 13:17:46.416069 15054 raft_consensus.cc:697] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:46.460567 10916 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:46.444213 14793 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:17:46.409463 15047 log.cc:927] Time spent T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 13:17:46.450963 15055 raft_consensus.cc:2804] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:46.418236 14988 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:46.466066 14990 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:46.466665 15012 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:46.455063 14968 consensus_queue.cc:1048] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:17:46.452513 14797 consensus_queue.cc:1059] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 10.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.023s
W20260709 13:17:46.512454 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 63999.9us
W20260709 13:17:46.512317 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 57603.4us
I20260709 13:17:46.516884 15012 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:46.486387 14858 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:46.517071 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:46.517856 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:46.533425 11505 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 11.
I20260709 13:17:46.483338 15038 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:46.486420 15055 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:46.478245 15037 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:46.523828 11507 raft_consensus.cc:1240] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:17:46.485646 14945 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:46.518522 15072 consensus_queue.cc:1048] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:17:46.529062 15062 log.cc:927] Time spent T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:17:46.487293 14643 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:46.544384 14755 raft_consensus.cc:3055] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:17:46.468295 14995 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:46.571768 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 62034.9us
I20260709 13:17:46.479501 14781 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:46.569319 10766 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:46.581631 15063 log.cc:927] Time spent T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 13:17:46.591249 14869 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 13:17:46.592864 11512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56410 (request call id 715) took 1443 ms. Trace:
I20260709 13:17:46.544083 15055 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:46.597051 14761 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:46.597358 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:46.601233 10766 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:46.599968 11512 rpcz_store.cc:276] 0709 13:17:45.149136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:45.149300 (+   164us) service_pool.cc:224] Handling call
0709 13:17:45.149646 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:45.600143 (+450497us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:17:45.763383 (+163240us) log.cc:844] Serialized 14 byte log entry
0709 13:17:45.808766 (+ 45383us) log.cc:844] Serialized 14 byte log entry
0709 13:17:45.811384 (+  2618us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:45.811455 (+    71us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:17:45.811528 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:46.592604 (+781076us) spinlock_profiling.cc:243] Waited 450 ms on lock 0x7b54001d1354. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:46.592649 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:46.592801 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":245540,"raft.queue_time_us":145728,"raft.run_cpu_time_us":339,"raft.run_wall_time_us":48215,"spinlock_wait_cycles":1045121536,"thread_start_us":192836,"threads_started":2}
I20260709 13:17:46.597690 14755 raft_consensus.cc:740] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:46.626241 10797 raft_consensus.cc:1240] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 10. Current term is 11. Ops: []
I20260709 13:17:46.627382 10800 raft_consensus.cc:1240] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:17:46.619825 14869 raft_consensus.cc:2749] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:17:46.604010 14853 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:46.629854 15073 consensus_queue.cc:1048] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.019s
I20260709 13:17:46.628752 10766 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:46.544248 14858 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:46.631381 10796 raft_consensus.cc:1240] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:17:46.605739 14983 raft_consensus.cc:2749] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:46.600134 14847 consensus_queue.cc:1048] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 13:17:46.645701 14945 raft_consensus.cc:2749] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:46.667531 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 584) took 1218 ms. Trace:
I20260709 13:17:46.669376 14755 consensus_queue.cc:260] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:46.670274 14912 raft_consensus.cc:2749] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:17:46.668376 15012 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:46.672042 10870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1850) took 1813 ms. Trace:
I20260709 13:17:46.642903 10868 raft_consensus.cc:1240] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 11. Current term is 12. Ops: []
I20260709 13:17:46.570840 14822 consensus_queue.cc:1048] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 13:17:46.678100 10867 rpcz_store.cc:276] 0709 13:17:45.448605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:45.878336 (+429731us) service_pool.cc:224] Handling call
0709 13:17:45.878847 (+   511us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:46.017961 (+139114us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:46.017992 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:46.033447 (+ 15455us) log.cc:844] Serialized 26 byte log entry
0709 13:17:46.330972 (+297525us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:17:46.340284 (+  9312us) log.cc:844] Serialized 14 byte log entry
0709 13:17:46.340523 (+   239us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:46.340596 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:46.667290 (+326694us) raft_consensus.cc:1685] finished
0709 13:17:46.667310 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:46.667467 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21163,"prepare.queue_time_us":9082,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":97943,"raft.run_cpu_time_us":247,"raft.run_wall_time_us":14633,"thread_start_us":236976,"threads_started":2,"wal-append.queue_time_us":369283}
W20260709 13:17:46.697068 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:46.687487 10870 rpcz_store.cc:276] 0709 13:17:44.858260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:45.510711 (+652451us) service_pool.cc:224] Handling call
0709 13:17:46.205386 (+694675us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:46.206291 (+   905us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:46.671831 (+465540us) spinlock_profiling.cc:243] Waited 694 ms on lock 0x7b54001095d0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:46.671982 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1596781824}
W20260709 13:17:46.700089 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 129533us
W20260709 13:17:46.631925 14782 consensus_meta.cc:287] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.843s	user 0.011s	sys 0.002s
W20260709 13:17:46.654448 14508 consensus_meta.cc:287] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.964s	user 0.008s	sys 0.004s
I20260709 13:17:46.640938 14853 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:46.594652 14841 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:46.712752 14755 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:46.643069 14989 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:46.630370 15072 consensus_queue.cc:1059] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.084s
I20260709 13:17:46.570050 15037 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:46.622749 15021 consensus_queue.cc:1059] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.083s
I20260709 13:17:46.623924 15054 consensus_queue.cc:237] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:46.705662 14782 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:46.727245 10916 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:46.730134 10874 raft_consensus.cc:1240] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 12. Current term is 13. Ops: []
I20260709 13:17:46.731621 10949 raft_consensus.cc:1275] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:17:46.706224 14959 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:46.665567 14947 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:46.727414 15071 log.cc:927] Time spent T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.156s	user 0.001s	sys 0.001s
W20260709 13:17:46.738117 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 127439us
I20260709 13:17:46.720839 15075 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:46.740878 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:46.743050 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:46.737370 15072 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:46.744906 15021 raft_consensus.cc:2749] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:17:46.745774 15075 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:46.740386 14945 raft_consensus.cc:3055] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:17:46.768453 14945 raft_consensus.cc:740] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:46.749065 11512 raft_consensus.cc:1240] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:17:46.755825 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:46.745519 11478 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:46.772437 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:46.773133 10947 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 10.
W20260709 13:17:46.773665 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 46557.7us
I20260709 13:17:46.739961 14959 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:46.781656 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:46.779742 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:46.766621 14970 consensus_meta.cc:287] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.737s	user 0.006s	sys 0.006s
I20260709 13:17:46.748165 14974 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:46.763284 14947 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:46.772863 14988 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:46.815724 14841 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:46.762820 14858 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:46.807888 14970 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:46.824411 15020 consensus_meta.cc:287] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 1.148s	user 0.014s	sys 0.000s
W20260709 13:17:46.827327 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 45826.6us
I20260709 13:17:46.834867 10916 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:46.774994 15072 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:46.842724 11503 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10.
I20260709 13:17:46.825701 15038 consensus_queue.cc:1059] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
W20260709 13:17:46.851475 15070 log.cc:927] Time spent T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 13:17:46.852811 10948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2334) took 1203 ms. Trace:
I20260709 13:17:46.853163 10948 rpcz_store.cc:276] 0709 13:17:45.650756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:45.933953 (+283197us) service_pool.cc:224] Handling call
0709 13:17:45.994751 (+ 60798us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:45.995987 (+  1236us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:45.996026 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:45.996577 (+   551us) log.cc:844] Serialized 26 byte log entry
0709 13:17:46.531927 (+535350us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:46.531978 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:46.532047 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:46.852588 (+320541us) raft_consensus.cc:1685] finished
0709 13:17:46.852607 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:46.852743 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":790,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":11081,"thread_start_us":469173,"threads_started":1,"wal-append.queue_time_us":653683}
I20260709 13:17:46.860333 14881 raft_consensus.cc:3055] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:17:46.860872 14881 raft_consensus.cc:740] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:46.861824 14881 consensus_queue.cc:260] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:46.864634 14970 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:46.859992 14793 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:46.847429 14988 raft_consensus.cc:2804] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:17:46.877928 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 92346.7us
I20260709 13:17:46.811906 14974 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:46.970889 14841 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:46.971589 14841 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:46.972765 14974 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:46.973335 14974 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:46.973547 14841 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:46.974014 14793 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:46.973788 14974 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:46.975811 10766 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:46.976068 14974 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:46.957937 15077 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:46.963626 15072 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:46.964274 15075 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:46.847165 14847 raft_consensus.cc:2804] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:46.948508 14912 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:46.879586 15038 consensus_queue.cc:1243] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer e01ecd008edd4aa2b6b2b47efbc2ea87 log is divergent from this leader: its last log entry 11.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:17:46.971052 14988 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:46.825776 14995 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:46.995615 14988 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:46.995918 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:46.996651 10799 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 13.
I20260709 13:17:46.998361 10872 raft_consensus.cc:1275] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 13:17:46.998772 10872 pending_rounds.cc:85] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 8
I20260709 13:17:46.999027 10872 pending_rounds.cc:107] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:17:46.999383 10872 raft_consensus.cc:2889] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:17:46.999727 15078 log.cc:927] Time spent T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.195s	user 0.001s	sys 0.000s
I20260709 13:17:47.000787 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:46.961760 15081 log.cc:927] Time spent T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 13:17:46.985803 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:46.991907 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:46.994324 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:46.994340 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:47.005225 10796 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 11.
I20260709 13:17:47.005620 10949 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:46.976790 14881 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:46.999061 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:47.005117 10798 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 13.
I20260709 13:17:47.001425 10872 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 9.
W20260709 13:17:47.009547 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:46.987128 14970 raft_consensus.cc:2804] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:47.011526 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:47.011559 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:47.012447 11503 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:46.990311 14974 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:47.015302 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:47.017212 15072 consensus_queue.cc:1048] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:17:46.993779 15077 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:47.019289 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:47.030488 11508 raft_consensus.cc:2393] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 8: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:47.020663 11478 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:47.020810 10948 raft_consensus.cc:2393] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 9: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:47.001214 15021 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.025264 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:47.023092 10872 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 11.
I20260709 13:17:46.844154 15020 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:46.993841 14912 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:47.037675 10916 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:47.044873 10916 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:47.045892 14874 raft_consensus.cc:2804] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:47.046280 14874 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:47.046638 14874 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:47.051026 11503 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
I20260709 13:17:47.052491 10766 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:47.052881 10949 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 12.
I20260709 13:17:47.027402 14970 raft_consensus.cc:697] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:47.057894 10794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 515) took 1553 ms. Trace:
I20260709 13:17:47.059170 10947 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 11.
I20260709 13:17:47.059407 10794 rpcz_store.cc:276] 0709 13:17:45.494173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:45.557668 (+ 63495us) service_pool.cc:224] Handling call
0709 13:17:45.558046 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:45.653263 (+ 95217us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:47.043491 (+1390228us) spinlock_profiling.cc:243] Waited 89.9 ms on lock 0x7b5400387a54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:47.047610 (+  4119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":206793472}
I20260709 13:17:47.063750 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:47.056480 15085 raft_consensus.cc:2804] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:47.065963 15085 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:47.066360 15085 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:47.073041 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1885) took 2299 ms. Trace:
I20260709 13:17:47.055796 15013 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.077491 15020 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:47.232234 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:47.233146 10950 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 13.
I20260709 13:17:47.079546 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:47.234576 10941 raft_consensus.cc:2393] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 12: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:47.060171 14993 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:17:47.073503 10869 rpcz_store.cc:276] 0709 13:17:44.773372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:44.775123 (+  1751us) service_pool.cc:224] Handling call
0709 13:17:44.786840 (+ 11717us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:45.765259 (+978419us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:47.071251 (+1305992us) spinlock_profiling.cc:243] Waited 907 ms on lock 0x7b5400077454. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:47.072948 (+  1697us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2086316160}
I20260709 13:17:47.236308 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:47.081486 14912 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:47.082518 10916 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:47.042089 15021 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:47.030318 14974 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:47.252753 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:47.259819 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:47.260505 10872 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:47.110142 14959 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:47.241356 15054 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.250257 14446 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:47.232301 15072 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:47.073832 15055 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:47.243397 14882 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:47.262238 15012 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:47.239502 15013 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:47.307706 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 130633us
I20260709 13:17:47.284497 15038 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.288785 14882 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:47.317133 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:47.317655 11510 raft_consensus.cc:3055] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:17:47.317934 11503 raft_consensus.cc:2393] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 12: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
W20260709 13:17:47.317183 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 234725us
I20260709 13:17:47.297820 15077 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:47.321326 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:47.318176 11510 raft_consensus.cc:740] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:47.323570 11508 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10.
I20260709 13:17:47.279109 15054 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:47.284082 15072 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:47.330892 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:47.275771 14970 consensus_queue.cc:237] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:47.331683 10766 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:47.338938 15074 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.339542 15074 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:47.340003 15096 raft_consensus.cc:2749] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:47.341957 15074 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:47.344110 15054 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:47.346047 15072 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:47.346562 10869 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 8.
I20260709 13:17:47.348294 10916 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:47.349542 10916 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:47.350907 15095 consensus_queue.cc:1048] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:47.352427 15095 raft_consensus.cc:2749] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:47.353080 15095 raft_consensus.cc:2749] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:47.315479 15038 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:47.354857 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:47.355849 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:47.356503 15038 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:47.356822 15099 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.357338 15099 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:47.359315 10868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1901) took 1838 ms. Trace:
I20260709 13:17:47.359555 15099 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:47.359743 10868 rpcz_store.cc:276] 0709 13:17:45.521362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:46.035874 (+514512us) service_pool.cc:224] Handling call
0709 13:17:46.036237 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:46.698361 (+662124us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:47.359081 (+660720us) spinlock_profiling.cc:243] Waited 606 ms on lock 0x7b54000983d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:47.359254 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1394697600}
I20260709 13:17:47.329901 11510 consensus_queue.cc:260] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:47.360975 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:47.361546 11510 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:47.360174 15099 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:47.368790 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:47.363750 11478 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:17:47.350704 10916 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:47.376209 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:47.328156 15077 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.377296 10948 raft_consensus.cc:2376] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 13. Re-sending same reply.
I20260709 13:17:47.382740 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:47.350663 15083 log.cc:927] Time spent T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.335s	user 0.000s	sys 0.001s
I20260709 13:17:47.383903 10872 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 12.
I20260709 13:17:47.369648 10868 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 12.
I20260709 13:17:47.390437 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:47.391047 10868 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 12.
I20260709 13:17:47.387365 11478 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:47.334522 14983 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:47.401027 11478 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:17:47.370144 15099 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:47.327548 14993 raft_consensus.cc:697] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:47.405570 11478 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:17:47.411772 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:47.385942 15038 raft_consensus.cc:2804] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:17:47.421645 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 71012.9us
I20260709 13:17:47.406773 14959 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:17:47.399758 15097 raft_consensus.cc:2749] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:47.416570 14912 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:47.445801 10916 leader_election.cc:400] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 9. Current term is 10.
I20260709 13:17:47.413267 15099 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:47.420712 15038 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:47.398509 15077 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:47.470674 10876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1849) took 2612 ms. Trace:
W20260709 13:17:47.451213 15088 log.cc:927] Time spent T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.211s	user 0.000s	sys 0.001s
I20260709 13:17:47.457183 14959 raft_consensus.cc:697] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 12 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:47.449654 14974 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:47.447438 14970 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.463353 14993 consensus_queue.cc:237] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:47.463992 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:47.486971 10943 raft_consensus.cc:2393] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 12: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:47.487941 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:47.487483 14959 consensus_queue.cc:237] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:47.488528 10943 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:17:47.491062 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:47.468014 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:47.455313 14912 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:47.471256 10876 rpcz_store.cc:276] 0709 13:17:44.857988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:45.495770 (+637782us) service_pool.cc:224] Handling call
0709 13:17:45.496123 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:45.764789 (+268666us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:47.469222 (+1704433us) spinlock_profiling.cc:243] Waited 239 ms on lock 0x7b5400109ad4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:47.470595 (+  1373us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":550189312}
I20260709 13:17:47.492595 14643 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:47.493543 10876 raft_consensus.cc:1275] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:17:47.493909 10876 pending_rounds.cc:85] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 8
I20260709 13:17:47.494124 10876 pending_rounds.cc:107] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:17:47.494356 10876 raft_consensus.cc:2889] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:47.472664 11503 raft_consensus.cc:1275] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:17:47.498273 15013 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:47.499653 15013 consensus_queue.cc:1048] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:47.460419 15038 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:47.412649 15075 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:17:47.448055 15021 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:47.409541 14983 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:47.420327 15072 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:47.555658 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:47.466259 10916 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:47.460500 10872 raft_consensus.cc:1240] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:17:47.506354 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:47.508431 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:47.557698 10796 raft_consensus.cc:2393] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 12: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:47.557658 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:47.499570 15091 log.cc:927] Time spent T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 13:17:47.533785 10943 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10.
I20260709 13:17:47.554762 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:47.554991 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:47.559728 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:17:47.569411 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 103230us
I20260709 13:17:47.570842 10916 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:47.574076 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:47.580220 10916 leader_election.cc:400] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 election: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 10. Current term is 11.
I20260709 13:17:47.581401 10916 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:47.583413 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:47.510408 15075 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:47.510274 15054 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:47.577003 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:47.587304 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:47.590052 10797 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:17:47.558738 15020 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:47.582437 15096 raft_consensus.cc:2749] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:17:47.511438 14643 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:47.511559 15055 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:47.567437 15053 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.502476 14970 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:47.599073 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:47.568845 15021 raft_consensus.cc:2804] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:47.563730 15072 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:47.628185 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:47.607792 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:47.594046 15013 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:47.635932 10873 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 1907) took 1979 ms (client timeout 2385 ms). Trace:
I20260709 13:17:47.636210 10870 raft_consensus.cc:1275] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:17:47.584460 15097 raft_consensus.cc:2749] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:47.585752 14978 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:47.625239 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:47.595139 15075 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:47.631582 10799 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 8.
I20260709 13:17:47.591243 10916 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:47.638500 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:47.655413 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:47.655869 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:17:47.636490 10873 rpcz_store.cc:269] 0709 13:17:45.656043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:45.663887 (+  7844us) service_pool.cc:224] Handling call
0709 13:17:47.635734 (+1971847us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b54000f78d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:47.635871 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":290068224}
W20260709 13:17:47.671006 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 79883us
I20260709 13:17:47.672430 10916 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:47.632818 15072 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:47.625085 15021 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:47.593660 14882 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:47.697136 10916 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:47.618009 15020 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:47.698179 10944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2338) took 1944 ms. Trace:
I20260709 13:17:47.710139 15095 raft_consensus.cc:2804] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:47.709340 10944 rpcz_store.cc:276] 0709 13:17:45.753601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:46.158827 (+405226us) service_pool.cc:224] Handling call
0709 13:17:46.194068 (+ 35241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:46.194290 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:17:46.210291 (+ 16001us) log.cc:844] Serialized 14 byte log entry
0709 13:17:47.697411 (+1487120us) spinlock_profiling.cc:243] Waited 2.53 ms on lock 0x7b1800302aa0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c80a5d4bc 00007f9c7ddab48b 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a5523a
0709 13:17:47.697672 (+   261us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:47.697755 (+    83us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:47.697832 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:47.697932 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:47.698100 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":36572,"prepare.run_cpu_time_us":9094,"prepare.run_wall_time_us":33694,"raft.queue_time_us":4855,"spinlock_wait_cycles":5820160,"thread_start_us":123666,"threads_started":1,"wal-append.queue_time_us":247907,"wal-append.run_cpu_time_us":1532,"wal-append.run_wall_time_us":1092465}
I20260709 13:17:47.620518 14643 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:47.597240 15054 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:47.719112 15101 consensus_queue.cc:1048] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:47.719533 15054 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:47.685974 15096 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:17:47.719909 14643 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:47.720362 15096 raft_consensus.cc:697] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 11 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:47.683873 14987 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:47.720978 14987 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:17:47.721310 14881 consensus_meta.cc:287] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.708s	user 0.010s	sys 0.001s
I20260709 13:17:47.721369 15096 consensus_queue.cc:237] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:47.722550 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:17:47.723387 15107 log.cc:927] Time spent T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:17:47.724206 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:47.725664 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:47.706862 14912 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:47.628738 15053 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:47.706131 15097 consensus_queue.cc:1048] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:17:47.716475 14882 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:47.753093 14782 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:47.759289 15086 log.cc:927] Time spent T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.712s	user 0.001s	sys 0.001s
I20260709 13:17:47.760524 15111 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:47.761194 15111 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:47.762147 14970 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:47.763193 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:47.764066 10943 raft_consensus.cc:2376] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11. Re-sending same reply.
I20260709 13:17:47.765200 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:47.765643 15049 log.cc:927] Time spent T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 1.521s	user 0.000s	sys 0.021s
I20260709 13:17:47.727988 10950 raft_consensus.cc:2393] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 13: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:47.769044 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:47.728338 10941 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 13.
I20260709 13:17:47.676102 14978 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:17:47.729679 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 227640us
W20260709 13:17:47.778955 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:47.779363 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:47.731631 10801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2392) took 2159 ms. Trace:
I20260709 13:17:47.633466 15084 consensus_queue.cc:1048] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.071s
I20260709 13:17:47.662123 15013 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:47.753513 11510 raft_consensus.cc:1275] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 13:17:47.770399 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:47.803844 11510 pending_rounds.cc:85] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586: Aborting all ops after (but not including) 8
I20260709 13:17:47.766647 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:47.705137 15021 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:17:47.728915 15104 log.cc:927] Time spent T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:17:47.702241 14943 raft_consensus.cc:2749] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:47.779956 10801 rpcz_store.cc:276] 0709 13:17:45.553526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:45.595658 (+ 42132us) service_pool.cc:224] Handling call
0709 13:17:45.715816 (+120158us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:45.763051 (+ 47235us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:47.712130 (+1949079us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b54001aa4d0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:47.712350 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":241856384}
I20260709 13:17:47.721403 14987 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:47.804208 11510 pending_rounds.cc:107] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:17:47.802970 14793 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.875239 14793 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:47.875782 15013 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:47.877501 14793 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:47.879001 11510 raft_consensus.cc:2889] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:47.839531 10873 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dde624b07ed446e5b7347791196396e2"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:17:47.880304 10873 raft_consensus.cc:606] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Received request to transfer leadership to TS e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:47.804699 10943 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 14.
I20260709 13:17:47.880797 10873 raft_consensus.cc:614] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:47.852156 10947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36714 (request call id 670) took 1080 ms. Trace:
I20260709 13:17:47.813951 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:47.881256 10947 rpcz_store.cc:276] 0709 13:17:46.771761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:46.779383 (+  7622us) service_pool.cc:224] Handling call
0709 13:17:47.851963 (+1072580us) spinlock_profiling.cc:243] Waited 273 ms on lock 0x7b5400173754. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:47.852065 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":626839808}
I20260709 13:17:47.881613 11503 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:47.852922 10800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1831) took 2400 ms. Trace:
I20260709 13:17:47.882624 10800 rpcz_store.cc:276] 0709 13:17:45.452890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:45.488960 (+ 36070us) service_pool.cc:224] Handling call
0709 13:17:45.503041 (+ 14081us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:46.627791 (+1124750us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:47.852693 (+1224902us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54003cd054. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:47.852865 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2585350912}
W20260709 13:17:47.854101 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:47.751375 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:47.778992 15085 consensus_meta.cc:287] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.712s	user 0.005s	sys 0.007s
W20260709 13:17:47.960141 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 75005.1us
I20260709 13:17:47.962139 11478 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:47.955123 10944 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
I20260709 13:17:47.980173 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:47.867031 15115 log.cc:927] Time spent T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.200s	user 0.000s	sys 0.001s
I20260709 13:17:47.970566  9682 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:17:47.859894 15003 raft_consensus.cc:3055] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:17:47.873472 14446 consensus_meta.cc:287] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.594s	user 0.008s	sys 0.002s
W20260709 13:17:47.856582 14988 consensus_meta.cc:287] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.860s	user 0.011s	sys 0.000s
I20260709 13:17:47.995216 10798 raft_consensus.cc:2393] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 12: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:47.969172 11478 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:47.869201 14134 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.999465 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:47.845285 14847 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:47.810354 14882 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:47.847299 15077 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:47.860216 14964 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:48.002163 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:47.815982 15017 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.866060 15020 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:47.867712 14983 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:48.007436 11478 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:48.008360 11512 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 12.
I20260709 13:17:48.009001 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:48.018944 11503 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10.
I20260709 13:17:47.868940 15053 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:48.042302 10795 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 8.
I20260709 13:17:48.040172 10875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1904) took 2517 ms. Trace:
I20260709 13:17:48.058156 10798 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 9.
W20260709 13:17:48.008399 14874 consensus_meta.cc:287] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.959s	user 0.007s	sys 0.006s
I20260709 13:17:47.998767 14858 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:17:48.000685 15003 raft_consensus.cc:740] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:48.061355 10793 raft_consensus.cc:1240] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
W20260709 13:17:48.063352 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 55999us
W20260709 13:17:48.111639 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 401885us
I20260709 13:17:48.059532 10874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 587) took 1889 ms. Trace:
W20260709 13:17:48.112934 10945 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.020s	sys 0.015s
W20260709 13:17:48.113305 10945 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.020s	sys 0.015s
I20260709 13:17:47.998509 14912 consensus_queue.cc:1059] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 13:17:48.067901 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:48.096973 10942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2340) took 2303 ms. Trace:
I20260709 13:17:48.040643 14793 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:48.046396 14847 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:17:48.109901 15112 log.cc:927] Time spent T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:17:48.059604 10875 rpcz_store.cc:276] 0709 13:17:45.522280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:46.190977 (+668697us) service_pool.cc:224] Handling call
0709 13:17:46.204407 (+ 13430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:46.387938 (+183531us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:46.387972 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:46.388014 (+    42us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:46.388045 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:48.039796 (+1651751us) spinlock_profiling.cc:243] Waited 183 ms on lock 0x7b54003165d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:48.039870 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:48.040085 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":421623680}
I20260709 13:17:48.068859 10801 raft_consensus.cc:1275] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:17:47.998287 14993 consensus_queue.cc:1048] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 13:17:48.058816 14964 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:48.060580 15074 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:48.112617 10874 rpcz_store.cc:276] 0709 13:17:46.169722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:46.205799 (+ 36077us) service_pool.cc:224] Handling call
0709 13:17:46.206087 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:46.731953 (+525866us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:48.059297 (+1327344us) spinlock_profiling.cc:243] Waited 524 ms on lock 0x7b54000997d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:48.059473 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1204778752}
I20260709 13:17:48.116015 11478 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:48.097332 15084 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:17:48.117293 10798 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 8.
I20260709 13:17:48.053864 14882 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:48.117733 10942 rpcz_store.cc:276] 0709 13:17:45.793907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:46.159258 (+365351us) service_pool.cc:224] Handling call
0709 13:17:46.194834 (+ 35576us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:46.302983 (+108149us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:46.303274 (+   291us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:46.303353 (+    79us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:46.303426 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:48.096657 (+1793231us) spinlock_profiling.cc:243] Waited 86 ms on lock 0x7b54001e2654. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:48.096724 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:48.096896 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":197831424}
I20260709 13:17:48.041842 14134 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:48.019680 15054 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:48.097941 15020 raft_consensus.cc:476] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Not starting pre-election -- already a leader
W20260709 13:17:48.125211 10841 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.021s	sys 0.016s
W20260709 13:17:48.183238 10841 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.021s	sys 0.016s
I20260709 13:17:48.105446 15053 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:48.107136 15116 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:48.109699 14858 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:48.130203 11478 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:48.132146 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:48.198182 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:17:48.198868 10916 connection.cc:256] RPC call timeout handler was delayed by -1.8953451849999965s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1567s for a response
W20260709 13:17:48.199054 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 68950.1us
I20260709 13:17:48.098115 14874 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:48.125622 14912 consensus_queue.cc:1243] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer 468a8fd46e824f62b9c2c0d3532a6d26 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:17:48.121143 14972 consensus_queue.cc:1048] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 13:17:48.182242 15072 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:48.121214 15095 raft_consensus.cc:697] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:48.013216 14988 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:48.190960 14858 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:48.137486 15084 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:48.136888 14970 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:48.199285 10916 connection.cc:256] RPC call timeout handler was delayed by -1.854845184999931s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1612s for a response
W20260709 13:17:48.267700 10916 connection.cc:256] RPC call timeout handler was delayed by -1.7621451850000085s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1715s for a response
W20260709 13:17:48.268174 10916 connection.cc:256] RPC call timeout handler was delayed by -1.7540451849999954s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1724s for a response
W20260709 13:17:48.268527 10916 connection.cc:256] RPC call timeout handler was delayed by -1.546145185000114s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1955s for a response
W20260709 13:17:48.268901 10916 connection.cc:256] RPC call timeout handler was delayed by -1.4399451849999423s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2073s for a response
W20260709 13:17:48.269255 10916 connection.cc:256] RPC call timeout handler was delayed by -1.4084451849998914s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21080000000000002s for a response
W20260709 13:17:48.269621 10916 connection.cc:256] RPC call timeout handler was delayed by -1.3913451850000911s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2127s for a response
W20260709 13:17:48.269989 10916 connection.cc:256] RPC call timeout handler was delayed by -1.3796451850000722s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21400000000000002s for a response
W20260709 13:17:48.270341 10916 connection.cc:256] RPC call timeout handler was delayed by -1.25004518500009s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2284s for a response
I20260709 13:17:48.200472 11478 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:48.200664 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:48.202734 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:48.271533 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 71134.4us
I20260709 13:17:48.214329 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:48.272496 10874 raft_consensus.cc:2410] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:17:48.273461 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:47.972853 15085 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:48.058523 15017 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:48.138334 15120 raft_consensus.cc:2749] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:17:48.185655 15046 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 13:17:48.219592 10868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 597) took 1154 ms. Trace:
I20260709 13:17:48.289611 10949 raft_consensus.cc:1240] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 11. Current term is 12. Ops: []
I20260709 13:17:48.221570 10876 raft_consensus.cc:1240] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:17:48.290525 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:48.291816 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:48.292424 14881 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:48.293001 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:48.292973 14881 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:48.293764 10876 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 11.
I20260709 13:17:48.293949 10794 raft_consensus.cc:1275] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:17:48.294492 10871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1898) took 3179 ms. Trace:
I20260709 13:17:48.294986 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:48.295213 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:48.295658 10794 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
I20260709 13:17:48.283964 14701 raft_consensus.cc:2804] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:48.296270 14701 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:48.296654 14701 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:48.296602 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:48.297293 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:48.187855 15053 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:48.122902 15097 consensus_queue.cc:1059] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.062s
I20260709 13:17:48.235085 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:17:48.299144 10687 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.670s	user 0.010s	sys 0.007s
I20260709 13:17:48.299909 15053 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:48.289844 10868 rpcz_store.cc:276] 0709 13:17:47.065062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:47.402703 (+337641us) service_pool.cc:224] Handling call
0709 13:17:47.403057 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:47.403408 (+   351us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:17:47.439276 (+ 35868us) log.cc:844] Serialized 14 byte log entry
0709 13:17:48.007198 (+567922us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:48.007686 (+   488us) log.cc:844] Serialized 26 byte log entry
0709 13:17:48.007901 (+   215us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:48.007936 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:48.008001 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:48.219322 (+211321us) raft_consensus.cc:1685] finished
0709 13:17:48.219367 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:48.219526 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":36103,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":2936,"raft.queue_time_us":121936,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":5234,"spinlock_wait_cycles":1076096,"thread_start_us":527060,"threads_started":2,"wal-append.queue_time_us":339448}
I20260709 13:17:48.301134 15053 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:48.283949 14970 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:48.237769 14882 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:48.240422 15072 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:48.303936 14970 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:48.275457 15084 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:48.294902 10871 rpcz_store.cc:276] 0709 13:17:45.114748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:45.784698 (+669950us) service_pool.cc:224] Handling call
0709 13:17:45.802705 (+ 18007us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:45.852613 (+ 49908us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:45.852644 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:47.449025 (+1596381us) spinlock_profiling.cc:243] Waited 4.8 ms on lock 0x7b2c00029f28. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7f7aea11 00007f9c7f7be372 00007f9c80ef0a01 00007f9c80eb543a 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2
0709 13:17:47.449506 (+   481us) log.cc:844] Serialized 26 byte log entry
0709 13:17:47.621095 (+171589us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:47.621155 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:48.294191 (+673036us) spinlock_profiling.cc:243] Waited 47.2 ms on lock 0x7b54003174d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:48.294243 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:48.294289 (+    46us) raft_consensus.cc:1685] finished
0709 13:17:48.294303 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:48.294434 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2944,"raft.queue_time_us":116,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":10587,"spinlock_wait_cycles":119565440,"thread_start_us":134855,"threads_started":1,"wal-append.queue_time_us":346592}
I20260709 13:17:48.304970 14945 consensus_queue.cc:260] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:48.306274 14945 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:48.307713 14781 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:48.310192 15111 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:48.243099 14134 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:48.310395 14701 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:48.311107 14134 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:48.311594 14134 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:48.312661 14701 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:48.245205 14874 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:48.247073 10948 raft_consensus.cc:1240] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:48.246500 15095 consensus_queue.cc:237] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:48.138175 15074 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:48.248339 11477 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:48.314579 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:48.330214 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 81962.4us
I20260709 13:17:48.189661 15116 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:48.249147 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:48.281128 14943 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:48.271620 10867 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 11.
I20260709 13:17:48.335970 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:48.120054 14793 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:48.264357 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:48.219553 14446 raft_consensus.cc:2804] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:17:48.270700 10916 connection.cc:256] RPC call timeout handler was delayed by -1.0484451849999914s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2508s for a response
I20260709 13:17:48.186304 15054 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:48.284931 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:48.285859 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:48.286562 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:48.288551 10800 raft_consensus.cc:1240] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:17:48.222290 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:48.410300 10872 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 11: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:48.298058 10797 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 11: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:48.226579 10870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 588) took 1257 ms. Trace:
I20260709 13:17:48.410985 10870 rpcz_store.cc:276] 0709 13:17:46.969094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:46.977137 (+  8043us) service_pool.cc:224] Handling call
0709 13:17:47.635888 (+658751us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:47.636640 (+   752us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:48.226359 (+589719us) spinlock_profiling.cc:243] Waited 642 ms on lock 0x7b54000f78d0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:48.226518 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1476940160}
I20260709 13:17:48.299082 10875 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:48.236290 15013 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:48.315023 10793 raft_consensus.cc:2410] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 12 because replica has last-logged OpId of term: 11 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 4.
I20260709 13:17:48.263598 15020 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:48.186934 15003 consensus_queue.cc:260] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:48.249456 14912 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:48.180264 14964 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:48.279165 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:48.421253 10801 raft_consensus.cc:1275] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:17:48.421383 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:48.332296 14701 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:48.350193 10949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 676) took 1047 ms. Trace:
I20260709 13:17:48.352380 10916 leader_election.cc:400] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 10. Current term is 11.
I20260709 13:17:48.403910 14134 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:48.359203 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:48.372603 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:48.431378 10948 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
I20260709 13:17:48.404058 10795 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:48.301626 15053 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:48.404624 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:48.407593 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:48.436024 10943 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 9.
I20260709 13:17:48.281070 14987 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:48.285475 15040 consensus_queue.cc:1048] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 13:17:48.402376 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:48.448554 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:48.449316 10947 raft_consensus.cc:2393] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 12: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:48.351111 14793 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:48.351456 14446 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:48.404116 14970 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:48.415740 15118 log.cc:927] Time spent T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.203s	user 0.000s	sys 0.001s
I20260709 13:17:48.428001 10916 leader_election.cc:403] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:17:48.427835 10949 rpcz_store.cc:276] 0709 13:17:47.303051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:47.303208 (+   157us) service_pool.cc:224] Handling call
0709 13:17:47.312912 (+  9704us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:48.290075 (+977163us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:48.349839 (+ 59764us) spinlock_profiling.cc:243] Waited 976 ms on lock 0x7b540039ab54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:48.350105 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2245614976}
I20260709 13:17:48.355887 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:48.414266 14881 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:48.404920 15129 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:48.527340 15129 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:48.527992 10947 raft_consensus.cc:1275] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:17:48.529510 14701 raft_consensus.cc:2749] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:48.530318 15129 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:48.445276 10950 raft_consensus.cc:2393] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 11: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:48.534207 15053 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:48.406499 14998 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:48.416913 15013 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:48.535470 15037 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:48.535909 14983 raft_consensus.cc:3055] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:17:48.536357 14983 raft_consensus.cc:740] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:48.536801 15013 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:48.536991 14847 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:48.537197 14983 consensus_queue.cc:260] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:48.538314 15054 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:48.449244 10948 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
I20260709 13:17:48.463764 10795 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 11.
W20260709 13:17:48.428296 15075 consensus_meta.cc:287] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.727s	user 0.005s	sys 0.005s
I20260709 13:17:48.504094 14822 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:48.504364 10766 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:48.505456 15124 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:48.347852 14943 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:48.522639 15099 consensus_meta.cc:287] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 1.063s	user 0.011s	sys 0.000s
I20260709 13:17:48.424762 14912 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:48.439101 10875 raft_consensus.cc:2410] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 12 because replica has last-logged OpId of term: 11 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 4.
W20260709 13:17:48.530764 15121 log.cc:927] Time spent T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:17:48.420071 15020 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:48.565675 14912 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:48.566483 10946 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 12.
I20260709 13:17:48.566820 15020 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:48.533293 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 181024us
I20260709 13:17:48.572252 10916 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:48.533308 14874 raft_consensus.cc:2749] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 10. Current term is 11.
I20260709 13:17:48.448530 15003 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:48.535157 14998 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:48.416044 15119 consensus_queue.cc:1048] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:17:48.492384 15123 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:48.539372 10794 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 10.
I20260709 13:17:48.543248 10698 catalog_manager.cc:5697] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:48.476712 15108 consensus_queue.cc:1048] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:17:48.548569 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:48.548983 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:48.549127 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:48.549583 10869 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 12.
W20260709 13:17:48.550454 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 46174.8us
I20260709 13:17:48.538388 14983 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:48.542943  9682 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:48.554947 14781 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:17:48.581522 14881 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:17:48.487833 14446 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:48.612689 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:48.620260 10949 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:48.539142 15017 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:48.519943 14869 consensus_queue.cc:1048] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 13:17:48.574025 10916 leader_election.cc:400] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 12. Current term is 13.
I20260709 13:17:48.604882 10948 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 12.
I20260709 13:17:48.532590 14987 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:48.539366 14970 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:48.613083 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:48.567970 14822 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:48.610359 15013 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:48.589933 14701 consensus_queue.cc:1059] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 13:17:48.600652 14847 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:48.625705 10916 leader_election.cc:403] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:17:48.626350 11477 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:48.605679 10766 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:48.556267 15075 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:48.571069 15099 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:48.610229 14781 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:48.623772 15085 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:48.622462 15130 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:48.700351 11505 raft_consensus.cc:2368] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 13. Current term is 14.
I20260709 13:17:48.646031 10766 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:48.622637 15054 consensus_queue.cc:1048] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 13:17:48.659277 15073 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:17:48.651604 11477 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:48.657930 10949 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 11.
I20260709 13:17:48.583436 15084 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:48.568689 15124 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:48.687911 14755 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:48.660090 10799 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 11.
I20260709 13:17:48.634716 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:48.630983 15116 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:48.707103 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 61168us
I20260709 13:17:48.684252 14781 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:48.659017 14987 raft_consensus.cc:2749] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 12. Current term is 13.
W20260709 13:17:48.709318 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 57795.1us
I20260709 13:17:48.687757 15037 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:48.728921 15037 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:48.729434 15037 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:48.731437 15074 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:48.733817 14943 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:48.701217 15021 consensus_meta.cc:287] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.834s	user 0.012s	sys 0.005s
I20260709 13:17:48.735215 15021 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:48.647209 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 73312.9us
I20260709 13:17:48.608857 15123 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:48.656558 10841 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:48.738695 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:17:48.739071 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 82575.2us
I20260709 13:17:48.739522 11508 raft_consensus.cc:2376] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 11. Re-sending same reply.
I20260709 13:17:48.740808 10944 raft_consensus.cc:1240] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 12. Current term is 13. Ops: []
I20260709 13:17:48.720446 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:48.722085 10766 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:48.651242 15013 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:48.727731 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:48.743319 11507 raft_consensus.cc:2393] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 11: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:48.712411 15108 raft_consensus.cc:2804] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:48.717622 14701 raft_consensus.cc:2804] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:48.749118 10766 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:48.752424 11504 raft_consensus.cc:1240] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:17:48.716327 14964 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:48.760661 10916 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:48.761430 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 499183us
I20260709 13:17:48.762534 10766 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:48.764797 10950 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 10.
I20260709 13:17:48.769977 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:48.737363 15021 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:48.748081 15054 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:48.776654 11507 raft_consensus.cc:1240] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:48.782297 10916 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:48.767911 10841 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:48.790285 11504 raft_consensus.cc:1240] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:17:48.754204 14869 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:17:48.796978 10841 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:48.804744 10841 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:48.759857 14701 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:48.809927 11477 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:48.725831 14755 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:48.812269 11512 raft_consensus.cc:1240] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 12. Current term is 14. Ops: []
I20260709 13:17:48.762143 15108 raft_consensus.cc:697] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:48.748274 14943 consensus_queue.cc:1059] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 13:17:48.824499 11505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56410 (request call id 727) took 1157 ms. Trace:
I20260709 13:17:48.770507 14993 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:17:48.828333 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 46130.4us
I20260709 13:17:48.775789 14987 raft_consensus.cc:2749] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:48.775631 14782 raft_consensus.cc:2804] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:48.789129 15054 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:48.829936 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:48.837560 10944 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 11.
I20260709 13:17:48.833484 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:48.828158 11505 rpcz_store.cc:276] 0709 13:17:47.654620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:47.655602 (+   982us) service_pool.cc:224] Handling call
0709 13:17:48.811957 (+1156355us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54004335d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:48.812150 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2367253760}
W20260709 13:17:48.821105 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:48.840909 15137 consensus_queue.cc:1059] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:17:48.842474 15132 log.cc:927] Time spent T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.153s	user 0.000s	sys 0.000s
I20260709 13:17:48.842885 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:48.843807 11509 raft_consensus.cc:2393] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 10: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:48.792008 15055 consensus_queue.cc:1048] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:17:48.849273 15124 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:48.850965 15123 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:48.851233 15085 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:48.851922 15085 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:48.854291 15085 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:48.801345 15020 consensus_queue.cc:1048] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:17:48.822084 14701 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:48.700739 15130 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:48.762636 15073 consensus_queue.cc:1059] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 13:17:48.916383 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:48.802120 15053 raft_consensus.cc:2749] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:17:48.800408 15101 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:48.825568 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:48.870524 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:48.858213 11478 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:48.907191 11508 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
I20260709 13:17:48.874723 11477 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:48.829174 10916 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:49.473850 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 644697us
I20260709 13:17:49.474700 10916 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:48.914968 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:49.493713 10946 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 11.
I20260709 13:17:48.912225 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:48.875248 10841 leader_election.cc:400] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 13. Current term is 14.
W20260709 13:17:49.508764 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 633609us
W20260709 13:17:49.510694 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:48.876998 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:49.512084 10942 raft_consensus.cc:2393] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 9: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:49.512917 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:49.513947 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:49.514494 10942 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 10.
I20260709 13:17:48.881719 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:49.515691 10943 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 14.
I20260709 13:17:49.516585 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:48.887676 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:49.517988 10950 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 11.
I20260709 13:17:49.518924 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:49.519479 10950 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:17:49.068630 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:49.181851 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:49.524123 10947 raft_consensus.cc:2393] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 10: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:49.187100 10948 raft_consensus.cc:3055] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:17:49.194346 15151 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:49.525801 10948 raft_consensus.cc:740] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:49.526000 15151 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:49.526728 10948 consensus_queue.cc:260] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:49.527688 10948 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:17:49.232311 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:49.279000 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:49.286535 10766 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.029s	sys 0.032s
W20260709 13:17:49.531065 10766 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.767s	user 0.030s	sys 0.032s
W20260709 13:17:49.531389 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 767610us
I20260709 13:17:49.532289 10766 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:49.532898 14938 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.533423 14938 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:49.328610 10687 maintenance_manager.cc:643] P ae957c8fc66c4b6ba063d0c74cbf5484: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 9.093s	user 0.208s	sys 0.080s Metrics: {"bytes_written":30469,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":350545,"dirs.run_cpu_time_us":570,"dirs.run_wall_time_us":57750,"drs_written":1,"lbm_read_time_us":407,"lbm_reads_lt_1ms":4,"lbm_write_time_us":157773,"lbm_writes_10-100_ms":4,"lbm_writes_lt_1ms":30,"peak_mem_usage":0,"reinsert_count":0,"rows_written":63,"thread_start_us":564999,"threads_started":2,"update_count":21,"wal-append.queue_time_us":503551,"wal-append.run_cpu_time_us":773,"wal-append.run_wall_time_us":1111559}
I20260709 13:17:49.096405 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:49.542467 11505 raft_consensus.cc:2410] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:17:49.542613 10688 maintenance_manager.cc:419] P ae957c8fc66c4b6ba063d0c74cbf5484: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9682 bytes on disk
I20260709 13:17:49.082063 14781 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:49.320398 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 445740us
W20260709 13:17:49.319379 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 461208us
I20260709 13:17:48.868304 14755 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:48.833400 14782 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:49.546746 14781 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:49.578629 14782 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:49.098198 14446 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:48.854700 14858 consensus_meta.cc:287] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.571s	user 0.008s	sys 0.000s
I20260709 13:17:48.813499 14869 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:49.195371 14912 raft_consensus.cc:3055] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:17:48.870224 14945 consensus_meta.cc:287] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.563s	user 0.011s	sys 0.002s
I20260709 13:17:49.608078 14945 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.608332 10941 raft_consensus.cc:1275] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 12. (term mismatch)
I20260709 13:17:49.608743 10941 pending_rounds.cc:85] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 11
I20260709 13:17:49.608981 10941 pending_rounds.cc:107] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 13:17:49.608600 14945 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:49.609216 10941 raft_consensus.cc:2889] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:49.292513 14988 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:48.832356 14993 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:49.614109 14993 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:49.207423 15013 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:49.129546 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:17:48.875401 14978 consensus_meta.cc:287] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 1.066s	user 0.008s	sys 0.003s
I20260709 13:17:49.625365 14978 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:49.593053 14858 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:49.627720 14978 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:48.866643 14998 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:49.628836 14858 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:49.629529 11511 raft_consensus.cc:1240] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 13. Current term is 14. Ops: []
I20260709 13:17:49.172266 15020 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:49.631042 14858 consensus_queue.cc:1048] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:17:49.631757 15020 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:49.634013 15020 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:48.857678 15037 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:49.637573 10947 raft_consensus.cc:1240] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 12. Current term is 13. Ops: []
I20260709 13:17:49.638001 14782 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:49.638885 15037 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:48.858038 15046 consensus_queue.cc:1059] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.186s
I20260709 13:17:48.839821 15054 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:49.096859 14701 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:49.647980 14993 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:49.649748 14701 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:48.865654 14793 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:49.653021 14793 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:48.863554 14970 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:48.863315 15017 raft_consensus.cc:3055] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:17:48.885300 15038 consensus_meta.cc:287] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 1.330s	user 0.014s	sys 0.000s
I20260709 13:17:48.869997 15075 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:49.320889 15085 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:49.674855 15085 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:48.864261 15108 consensus_queue.cc:237] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:48.834254 15111 raft_consensus.cc:2804] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:17:49.235105 15116 raft_consensus.cc:2749] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:48.864135 15128 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.684806 15128 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:49.323019 14799 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:48.981389 15130 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:48.825394 15134 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:49.689239 15134 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:49.689493 15128 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:49.689673 15134 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:48.855901 15137 raft_consensus.cc:3055] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:17:49.691598 15137 raft_consensus.cc:740] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:49.692553 15137 consensus_queue.cc:260] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:49.694067 15137 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:17:49.358160 10876 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.721s	user 0.029s	sys 0.084s
W20260709 13:17:49.695092 10876 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.063s	user 0.030s	sys 0.085s
I20260709 13:17:49.676602 14993 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:49.695914 10876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1863) took 1489 ms. Trace:
I20260709 13:17:49.696251 10876 rpcz_store.cc:276] 0709 13:17:48.206524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:48.294428 (+ 87904us) service_pool.cc:224] Handling call
0709 13:17:48.294673 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:48.304601 (+  9928us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:17:48.632388 (+327787us) log.cc:844] Serialized 14 byte log entry
0709 13:17:49.695509 (+1063121us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:49.695576 (+    67us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:49.695627 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:49.695691 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:49.695842 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":502119,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"thread_start_us":1390056,"threads_started":2,"wal-append.queue_time_us":447059}
I20260709 13:17:49.372558 10873 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 11.
W20260709 13:17:49.699142 10873 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 1857) took 1502 ms (client timeout 1722 ms). Trace:
I20260709 13:17:49.582597 11477 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:17:49.702039 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 119542us
I20260709 13:17:49.443269 15099 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:49.447435 15084 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:49.703336 15084 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:49.703692 15084 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:49.704046 15084 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:49.457820 10698 catalog_manager.cc:5697] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:49.477106 15123 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:49.704999 15123 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:49.705904 10698 catalog_manager.cc:5697] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:49.707087 15123 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:49.477586 15095 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:17:49.478041 14881 raft_consensus.cc:2749] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:17:49.478559 14987 consensus_queue.cc:1059] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:49.708664 14987 raft_consensus.cc:3055] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:17:49.709004 14987 raft_consensus.cc:740] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:49.708416 10698 catalog_manager.cc:5697] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 8 to 9, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 9 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:17:49.709862 14987 consensus_queue.cc:260] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:49.479806 15124 consensus_queue.cc:1048] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:49.483059 14822 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:49.484341 10916 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:17:49.712383 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 231779us
I20260709 13:17:49.485952 15101 raft_consensus.cc:2749] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:49.714433 15101 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:17:49.714840 15101 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:49.715229 15101 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:49.503988 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:49.719266 10916 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:49.719707 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:49.720505 10946 raft_consensus.cc:2410] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:17:49.720384 10916 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:49.721235 10916 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:49.721439 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:49.722921 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:49.723519 10799 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 12.
I20260709 13:17:48.887511 10944 raft_consensus.cc:1240] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 10. Current term is 11. Ops: []
I20260709 13:17:49.528440 15151 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:49.536018 14938 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:49.538981 14968 consensus_queue.cc:1048] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:17:49.551537 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:49.553403 11507 raft_consensus.cc:1240] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 12. Current term is 13. Ops: []
I20260709 13:17:49.565128 10948 raft_consensus.cc:1275] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:17:49.566380 10950 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 14.
I20260709 13:17:49.763250 10942 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 12.
I20260709 13:17:49.538540 15055 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:49.566800 10841 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:49.767306 15055 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:17:49.767560 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 200870us
I20260709 13:17:49.767817 15055 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:49.768302 10841 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:49.769421 10841 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:49.770409 14998 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:17:49.568293 11505 raft_consensus.cc:1240] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:49.770825 14998 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:49.771194 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:49.771564 15053 raft_consensus.cc:2804] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:49.771704 15130 raft_consensus.cc:2804] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:49.772074 15053 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:49.772572 15053 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:49.570377 10687 maintenance_manager.cc:643] P ae957c8fc66c4b6ba063d0c74cbf5484: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.003s	user 0.000s	sys 0.001s Metrics: {"cfile_init":1,"lbm_read_time_us":423,"lbm_reads_lt_1ms":4}
I20260709 13:17:49.585071 15158 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:49.775609 15158 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:49.776158 15116 consensus_queue.cc:1059] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:49.777726 15119 raft_consensus.cc:3055] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:17:49.777927 15158 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:49.778142 15119 raft_consensus.cc:740] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:17:49.590994 10696 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42968 (request call id 246) took 12199 ms (client timeout 15000 ms). Trace:
I20260709 13:17:49.606711 15073 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:49.611183 14945 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:49.779047 15119 consensus_queue.cc:260] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:49.591765 11503 raft_consensus.cc:1240] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:49.611852 10941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2354) took 2038 ms. Trace:
I20260709 13:17:49.596107 14912 raft_consensus.cc:740] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:49.593772 14869 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:49.591198 15157 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.803045 15119 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:49.803215 15157 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:49.805670 15157 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:49.807328 15128 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:49.807901 15128 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:49.810232 10915 leader_election.cc:336] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Timed out: RequestConsensusVote RPC to 127.9.116.132:40555 timed out after 1.518s (SENT)
I20260709 13:17:49.810400 15128 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:49.754633 14988 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.640372 14782 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:49.641652 11504 raft_consensus.cc:2393] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 13: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:49.654990 15017 raft_consensus.cc:740] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:49.655565 11510 raft_consensus.cc:2368] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 11. Current term is 12.
I20260709 13:17:49.655726 15038 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:49.656226 11509 raft_consensus.cc:2393] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 13: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:49.826165 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:49.758846 15020 raft_consensus.cc:2804] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:17:49.658043 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 67233.4us
I20260709 13:17:49.828438 11478 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:49.673782 15054 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:49.172809 15096 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:49.681691 15111 raft_consensus.cc:697] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 11 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:49.927595 15096 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:49.928040 15111 consensus_queue.cc:237] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:49.676025 15085 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:49.929394 15053 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:49.929908 15096 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:49.687742 14799 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:49.930984 10943 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
W20260709 13:17:48.836906 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 67448.3us
I20260709 13:17:49.931432 14799 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:17:49.699617 10873 rpcz_store.cc:269] 0709 13:17:48.196416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:48.197901 (+  1485us) service_pool.cc:224] Handling call
0709 13:17:49.698948 (+1501047us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b540009abd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:49.699079 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2529770752}
I20260709 13:17:49.932893 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:49.933491 10873 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:49.724731 10945 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 10.
I20260709 13:17:49.722440 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:49.726614 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:49.935627 10867 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:49.728312 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:49.936491 10871 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:49.728935 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:49.937974 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:49.710048 10698 catalog_manager.cc:5697] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 6 to 9, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 9 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:17:49.742645 10767 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.116.130:37895 timed out after 1.855s (SENT) [suppressed 10 similar messages]
W20260709 13:17:49.939152 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 196529us
I20260709 13:17:49.940583 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:49.749397 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:49.750027 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:49.941888 10872 raft_consensus.cc:2393] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 13: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:49.753237 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:49.942739 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:49.943537 10870 raft_consensus.cc:2393] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 13: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:49.753646 10875 raft_consensus.cc:1275] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:17:49.734267 14881 consensus_queue.cc:1059] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 13:17:49.748677 10916 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:49.763250 10948 pending_rounds.cc:85] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 6
I20260709 13:17:49.944360 10948 pending_rounds.cc:107] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:17:49.944640 10948 raft_consensus.cc:2889] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:49.945331 10948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2361) took 1032 ms. Trace:
I20260709 13:17:49.945642 10948 rpcz_store.cc:276] 0709 13:17:48.913295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:48.913469 (+   174us) service_pool.cc:224] Handling call
0709 13:17:48.913864 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:49.187002 (+273138us) spinlock_profiling.cc:243] Waited 272 ms on lock 0x7b5400283d20. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80e2cdc0 00007f9c80eb71c0 00007f9c80eb8231 00007f9c80eb4d8f 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c
0709 13:17:49.945084 (+758082us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:49.945278 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":625145088}
W20260709 13:17:49.946424 14943 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 13 current term: 13 is higher.
I20260709 13:17:49.946970 10868 raft_consensus.cc:2393] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 12: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:49.946874 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:49.947525 10948 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:49.947947 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:49.948664 10868 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 11.
I20260709 13:17:49.949636 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:49.950357 10868 raft_consensus.cc:2393] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 14: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:49.772142 15130 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:49.592861 14446 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:49.781606 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:49.954562 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:49.954912 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:49.955536 10796 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:49.783301 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:49.784771 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:49.786895 11477 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:49.792791 11503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56404 (request call id 747) took 1069 ms. Trace:
W20260709 13:17:49.957816 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 171045us
I20260709 13:17:49.781066 10941 rpcz_store.cc:276] 0709 13:17:47.572977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:47.774218 (+201241us) service_pool.cc:224] Handling call
0709 13:17:47.781074 (+  6856us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:49.609665 (+1828591us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:49.611613 (+  1948us) spinlock_profiling.cc:243] Waited 1.83 s on lock 0x7b54001707d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:49.611790 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4201364224}
I20260709 13:17:49.793766 10798 raft_consensus.cc:1240] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:17:49.959569 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:49.798600 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:49.955247 10868 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 10.
I20260709 13:17:49.757771 14987 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:49.811867 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:49.957991 11503 rpcz_store.cc:276] 0709 13:17:48.723813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:48.724001 (+   188us) service_pool.cc:224] Handling call
0709 13:17:48.724301 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:49.780748 (+1056447us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:49.792501 (+ 11753us) spinlock_profiling.cc:243] Waited 867 ms on lock 0x7b54005014d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:49.792714 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1994225408}
I20260709 13:17:49.964624 10794 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 12.
I20260709 13:17:49.965541 10867 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 10.
I20260709 13:17:49.965637 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:49.751384 15124 raft_consensus.cc:2804] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:49.814901 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:49.635691 10949 raft_consensus.cc:2410] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:17:49.813589 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:49.968082 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:49.969533 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:49.825224 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:49.970508 10944 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:49.771221 14998 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:49.833709 10801 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:49.834957 11510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56404 (request call id 748) took 1107 ms. Trace:
I20260709 13:17:49.835933 10915 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:49.838200 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:49.840096 11504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56404 (request call id 753) took 1034 ms. Trace:
I20260709 13:17:49.841423 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:49.842422 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:17:49.842978 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 47315.5us
W20260709 13:17:49.820492 15165 log.cc:927] Time spent T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:17:49.844115 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:49.841091 14858 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:17:49.845157 14912 consensus_queue.cc:260] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:49.837345 15128 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:49.827499 14988 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:49.853421 11506 raft_consensus.cc:2393] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 9: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:49.853917 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:49.855533 11508 raft_consensus.cc:1240] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 12. Current term is 13. Ops: []
I20260709 13:17:49.852766 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:49.851747 14793 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:49.857959 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:49.864079 11512 raft_consensus.cc:1240] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 12. Current term is 13. Ops: []
I20260709 13:17:49.844197 15095 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:49.779063 15073 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:49.870831 11509 raft_consensus.cc:1275] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:17:49.844026 15020 raft_consensus.cc:697] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 11 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:49.884064 15054 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:49.887982 15038 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:17:49.909756 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:49.889019 15101 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:49.844553 15134 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:49.778812 10696 rpcz_store.cc:269] 0709 13:17:37.368350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:37.368555 (+   205us) service_pool.cc:224] Handling call
0709 13:17:37.538176 (+169621us) spinlock_profiling.cc:243] Waited 150 ms on lock 0x7b54002fce54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1f090 00007f9c80ebdf3a 00007f9c86c741cc 00007f9c86d7237e 00007f9c852332a4 00007f9c85233232 00007f9c85233161 00007f9c852330dc 00007f9c85232392 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:37.661443 (+123267us) spinlock_profiling.cc:243] Waited 39.7 ms on lock 0x7b54002b0110. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd91640 00007f9c86c68b76 00007f9c86d72c3e 00007f9c852332a4 00007f9c85233232 00007f9c85233161 00007f9c852330dc 00007f9c85232392 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:49.564819 (+11903376us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:39.035069 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:39.036018 (+   949us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:39.036073 (+    55us) write_op.cc:435] Acquired schema lock
0709 13:17:39.036101 (+    28us) tablet.cc:662] Decoding operations
0709 13:17:39.038419 (+  2318us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:39.038501 (+    82us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:39.038542 (+    41us) tablet.cc:685] Acquiring locks for 21 operations
0709 13:17:39.656250 (+617708us) tablet.cc:701] Row locks acquired
0709 13:17:39.656276 (+    26us) write_op.cc:260] PREPARE: finished
0709 13:17:39.656450 (+   174us) write_op.cc:270] Start()
0709 13:17:39.656584 (+   134us) write_op.cc:276] Timestamp: P: 1783603059656393 usec, L: 0
0709 13:17:39.656610 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:17:39.742215 (+ 85605us) log.cc:844] Serialized 8301 byte log entry
0709 13:17:41.125756 (+1383541us) op_driver.cc:464] REPLICATION: finished
0709 13:17:41.135108 (+  9352us) write_op.cc:301] APPLY: starting
0709 13:17:41.135212 (+   104us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:41.137007 (+  1795us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:41.137153 (+   146us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:41.139657 (+  2504us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:41.141649 (+  1992us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=42
0709 13:17:41.141719 (+    70us) write_op.cc:312] APPLY: finished
0709 13:17:41.144724 (+  3005us) log.cc:844] Serialized 145 byte log entry
0709 13:17:41.144942 (+   218us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:41.145189 (+   247us) write_op.cc:454] Released schema lock
0709 13:17:41.145258 (+    69us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:41.145792 (+   534us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:41.145827 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":48913,"reported_tablets":49,"spinlock_wait_cycles":435368576,"child_traces":[["op",{"apply.queue_time_us":9029,"apply.run_cpu_time_us":5991,"apply.run_wall_time_us":10623,"num_ops":21,"prepare.queue_time_us":440401,"prepare.run_cpu_time_us":43841,"prepare.run_wall_time_us":1168608,"raft.queue_time_us":789917,"raft.run_cpu_time_us":4285,"raft.run_wall_time_us":9722,"replication_time_us":1468931,"thread_start_us":1711565,"threads_started":4,"wal-append.queue_time_us":530411,"wal-append.run_cpu_time_us":20517,"wal-append.run_wall_time_us":2039434}]]}
I20260709 13:17:49.748013 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:49.943842 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:49.950565 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 201969us
I20260709 13:17:49.941262 10946 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
I20260709 13:17:49.962550 10948 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 13.
I20260709 13:17:49.967456 15137 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:17:50.015569 15137 raft_consensus.cc:697] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 14 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:49.963560 10792 raft_consensus.cc:2393] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 11: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:50.015827 14943 raft_consensus.cc:2749] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:49.963698 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:50.017361 10916 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:50.017534 11510 rpcz_store.cc:276] 0709 13:17:48.727194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:48.727408 (+   214us) service_pool.cc:224] Handling call
0709 13:17:49.834727 (+1107319us) spinlock_profiling.cc:243] Waited 910 ms on lock 0x7b54007e1b54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:49.834901 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2093045504}
I20260709 13:17:50.018923 10797 raft_consensus.cc:2368] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 14. Current term is 15.
W20260709 13:17:50.019542 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:50.015635 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 179838us
I20260709 13:17:50.020529 11477 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:49.966771 10942 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:50.017395 11511 raft_consensus.cc:2393] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 9: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:50.001304 10698 catalog_manager.cc:5697] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 7 to 9, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 9 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:17:49.974680 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:49.975633 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:50.026162 10869 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:50.003563 10945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41020 (request call id 2363) took 1087 ms. Trace:
I20260709 13:17:50.027032 10945 rpcz_store.cc:276] 0709 13:17:48.915934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:48.916066 (+   132us) service_pool.cc:224] Handling call
0709 13:17:50.003323 (+1087257us) spinlock_profiling.cc:243] Waited 415 ms on lock 0x7b54001f2f54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:50.003486 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":953618688}
I20260709 13:17:50.015749 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:49.972344 10867 raft_consensus.cc:1275] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:17:50.028524 10948 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:49.977188 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:50.031387 10941 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:49.950026 15096 raft_consensus.cc:3055] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:17:50.020388 11504 rpcz_store.cc:276] 0709 13:17:48.806121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:48.825258 (+ 19137us) service_pool.cc:224] Handling call
0709 13:17:49.839922 (+1014664us) spinlock_profiling.cc:243] Waited 323 ms on lock 0x7b540046d2d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:49.840042 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":741948160}
I20260709 13:17:50.036298 11507 raft_consensus.cc:2393] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 10: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:50.040395 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:50.025349 10870 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 12.
I20260709 13:17:50.021559 10766 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:49.953799 15130 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:50.043404 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:50.044723 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:50.045037 10946 raft_consensus.cc:1275] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:17:50.045725 11508 raft_consensus.cc:2393] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 14: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:50.058979 10916 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:50.072335 10949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41020 (request call id 2360) took 1185 ms. Trace:
I20260709 13:17:50.073048 10915 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:50.072971 10949 rpcz_store.cc:276] 0709 13:17:48.886705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:48.911852 (+ 25147us) service_pool.cc:224] Handling call
0709 13:17:50.072138 (+1160286us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b54000ab4d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:50.072267 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":312512256}
I20260709 13:17:50.072292 10916 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:50.081192 10868 raft_consensus.cc:1275] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 13:17:50.084208 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:50.086455 10873 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 12.
I20260709 13:17:50.071004 10948 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 13.
I20260709 13:17:50.071760 14912 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:50.090411 10949 raft_consensus.cc:1240] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 10. Current term is 11. Ops: []
I20260709 13:17:49.967664 15124 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:50.090819 10947 raft_consensus.cc:1240] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 11. Current term is 12. Ops: []
I20260709 13:17:49.954078 14881 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:50.096630 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:50.098948 10872 raft_consensus.cc:2393] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 13: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:50.003875 15159 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:50.100452 10698 catalog_manager.cc:5697] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:50.108918 10945 raft_consensus.cc:1275] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 13:17:50.110884 11506 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56410 (request call id 728) took 2436 ms (client timeout 1694 ms). Trace:
W20260709 13:17:50.124361 11506 rpcz_store.cc:269] 0709 13:17:47.655014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:47.655126 (+   112us) service_pool.cc:224] Handling call
0709 13:17:50.091601 (+2436475us) spinlock_profiling.cc:243] Waited 2.18 s on lock 0x7b54005f8a54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:50.091781 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5018995072}
I20260709 13:17:50.102041 10942 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 13.
I20260709 13:17:50.044068 15096 raft_consensus.cc:740] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:50.134516 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:50.072055 15108 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:50.097954 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:50.143718 11511 raft_consensus.cc:2393] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 14: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:50.094563 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:49.965147 15119 consensus_queue.cc:1048] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.128s
I20260709 13:17:50.043095 15123 raft_consensus.cc:2749] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:50.058233 15037 raft_consensus.cc:2804] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:50.131306 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:50.130246 11477 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:50.117528 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:50.107545 10946 raft_consensus.cc:1240] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:50.109082 10941 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
I20260709 13:17:50.116598 10944 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 12.
I20260709 13:17:50.089365 10948 raft_consensus.cc:1240] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 12. Current term is 13. Ops: []
I20260709 13:17:50.056613 14858 raft_consensus.cc:697] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 12 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:50.163036 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:50.141290 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:50.142618 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:50.145227 10698 catalog_manager.cc:5697] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:17:50.142874 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 197322us
I20260709 13:17:49.997717 15163 consensus_queue.cc:1048] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.120s
I20260709 13:17:50.130095 14943 raft_consensus.cc:2749] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:17:50.156881 10766 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:50.047210 15128 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:50.003628 14799 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:50.104833 14945 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:17:50.161859 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 141336us
I20260709 13:17:50.170676 10948 raft_consensus.cc:1240] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 12. Current term is 13. Ops: []
I20260709 13:17:50.187443 14968 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:50.218627 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 49489.4us
I20260709 13:17:50.056814 14793 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:50.094995 15167 consensus_queue.cc:1059] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.152s
I20260709 13:17:50.104624 15124 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:50.212692 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:50.232714 10798 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:50.214841 11506 raft_consensus.cc:1240] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:17:50.143100 15108 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:50.224565 10842 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:50.208009 15084 consensus_meta.cc:287] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.504s	user 0.008s	sys 0.004s
I20260709 13:17:50.156039 15157 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:17:50.121301 14782 consensus_queue.cc:1059] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.087s
I20260709 13:17:50.203926 10947 raft_consensus.cc:1240] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:50.240312  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:50.153505 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:50.273212 10868 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:17:50.255116 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 104259us
I20260709 13:17:50.255867 10869 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 14.
I20260709 13:17:50.209846 15053 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:50.211180 15038 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:50.138374 15159 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:50.157173 15055 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:50.278419 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:50.259794 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:50.290019 10797 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:50.208498 14938 consensus_queue.cc:1059] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.094s
I20260709 13:17:50.207404 15151 consensus_queue.cc:1059] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 13:17:50.244946 15108 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:50.210965 14970 consensus_queue.cc:1048] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
W20260709 13:17:50.310308 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 90565us
I20260709 13:17:50.347904 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:50.347940 11512 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:17:50.354868 10971 debug-util.cc:398] Leaking SignalData structure 0x7b0800219c60 after lost signal to thread 9685
W20260709 13:17:50.382220 10971 debug-util.cc:398] Leaking SignalData structure 0x7b0800016180 after lost signal to thread 10733
W20260709 13:17:50.383519 10971 debug-util.cc:398] Leaking SignalData structure 0x7b08003f2300 after lost signal to thread 10825
W20260709 13:17:50.383821 10971 debug-util.cc:398] Leaking SignalData structure 0x7b08003a58c0 after lost signal to thread 10826
W20260709 13:17:50.384873 10971 debug-util.cc:398] Leaking SignalData structure 0x7b08003f1400 after lost signal to thread 10900
W20260709 13:17:50.385143 10971 debug-util.cc:398] Leaking SignalData structure 0x7b08003b18c0 after lost signal to thread 10901
W20260709 13:17:50.386157 10971 debug-util.cc:398] Leaking SignalData structure 0x7b080030d100 after lost signal to thread 10974
W20260709 13:17:50.386428 10971 debug-util.cc:398] Leaking SignalData structure 0x7b080035bb20 after lost signal to thread 10975
I20260709 13:17:50.226217 15160 consensus_queue.cc:1048] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.067s
I20260709 13:17:50.163678 15017 consensus_queue.cc:260] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:50.268971 14446 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:50.149210 15137 consensus_queue.cc:237] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:50.389097 10971 debug-util.cc:398] Leaking SignalData structure 0x7b08003a3e80 after lost signal to thread 11536
W20260709 13:17:50.405333 10971 debug-util.cc:398] Leaking SignalData structure 0x7b080028e8e0 after lost signal to thread 11537
I20260709 13:17:50.399366 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:50.236799 14968 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:50.485317 11478 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:17:50.264963 15084 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:50.499799 14945 raft_consensus.cc:697] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 13 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:50.500160 11478 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:50.500799 14945 consensus_queue.cc:237] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:50.501260 15084 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:50.256943 15173 consensus_queue.cc:1059] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.126s
I20260709 13:17:50.502656 11505 raft_consensus.cc:2393] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 12: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:50.486742 10868 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 9.
I20260709 13:17:50.509796 11512 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
W20260709 13:17:50.489732 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 127996us
I20260709 13:17:50.490126 10797 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 14.
I20260709 13:17:50.230178 15020 consensus_queue.cc:237] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:50.492796 11509 raft_consensus.cc:1240] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 11 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 10. Current term is 11. Ops: []
I20260709 13:17:50.297735 15095 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:17:50.497807 14869 consensus_meta.cc:287] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.672s	user 0.011s	sys 0.012s
I20260709 13:17:50.526124 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:50.506336 15168 log.cc:927] Time spent T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:17:50.278875 15096 consensus_queue.cc:260] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:50.557451 10799 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:50.527149 10800 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:17:50.541303 10871 consensus_meta.cc:287] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.604s	user 0.012s	sys 0.000s
I20260709 13:17:50.565208 10871 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 9.
I20260709 13:17:50.524901 15119 raft_consensus.cc:2749] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:50.526204 15188 raft_consensus.cc:3055] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:50.536787 15134 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:50.575929 14782 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:50.600546 15195 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:50.601109 15189 raft_consensus.cc:2749] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:17:50.553543 15101 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:50.553388 15073 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:50.572933 15193 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:50.387166 14858 consensus_queue.cc:237] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:50.597714 10796 consensus_meta.cc:287] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.639s	user 0.006s	sys 0.001s
I20260709 13:17:50.509085 15185 consensus_queue.cc:1059] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.116s
I20260709 13:17:50.489542 14446 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:50.525617 14869 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:50.442049 14701 consensus_queue.cc:1175] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "cb0428cc55984daba1ba63744a5102a2" responder_term: 12 status { last_received { term: 11 index: 9 } last_committed_idx: 9 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 11. Current term is 12. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:17:50.513083 15095 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:17:50.763769 10766 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.008s	sys 0.000s
W20260709 13:17:50.736627 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 512340us
I20260709 13:17:50.746721 10798 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 10.
I20260709 13:17:50.650486 10799 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 12.
W20260709 13:17:50.886838 10949 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.005s	sys 0.005s
W20260709 13:17:50.650534 10801 consensus_meta.cc:287] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.629s	user 0.009s	sys 0.000s
W20260709 13:17:50.696134 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:50.887712 10941 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.006s	sys 0.002s
I20260709 13:17:50.887850 10801 raft_consensus.cc:1275] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 13:17:50.761387 10876 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.745s	user 0.001s	sys 0.008s
W20260709 13:17:50.879056 10766 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.011s	sys 0.007s
W20260709 13:17:50.889235 10916 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.793s	user 0.005s	sys 0.004s
W20260709 13:17:50.887818 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:50.886380 10796 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 9.
I20260709 13:17:50.890626 10842 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:50.887184 10949 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.667s	user 0.005s	sys 0.005s
W20260709 13:17:50.891392 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 734536us
I20260709 13:17:50.890306 10800 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 9.
W20260709 13:17:50.888509 10875 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.798s	user 0.008s	sys 0.004s
W20260709 13:17:50.888886 10876 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.873s	user 0.002s	sys 0.018s
I20260709 13:17:50.888854 10801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1857) took 1076 ms. Trace:
W20260709 13:17:50.889575 10916 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.793s	user 0.005s	sys 0.004s
W20260709 13:17:50.656879 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 156781us
W20260709 13:17:50.708943 10795 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.004s	sys 0.012s
W20260709 13:17:50.890754 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 135770us
W20260709 13:17:50.884509 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:50.765599 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:50.896622 10875 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.806s	user 0.008s	sys 0.004s
W20260709 13:17:50.897467 10795 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.848s	user 0.004s	sys 0.026s
I20260709 13:17:50.898602 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:50.898698 10801 rpcz_store.cc:276] 0709 13:17:49.812933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:49.813140 (+   207us) service_pool.cc:224] Handling call
0709 13:17:49.833590 (+ 20450us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:50.888556 (+1054966us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:50.888793 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:50.899425 10942 raft_consensus.cc:2368] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 13. Current term is 14.
W20260709 13:17:50.899582 10767 leader_election.cc:336] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Timed out: RequestConsensusVote RPC to 127.9.116.130:37895 timed out after 2.385s (SENT)
I20260709 13:17:50.900041 10767 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:50.900429 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:50.902191 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:50.903016 10942 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 13
W20260709 13:17:50.905146 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:50.908126 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:50.908906 10948 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:17:50.912289 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 816312us
W20260709 13:17:50.888120 10941 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.006s	sys 0.002s
I20260709 13:17:50.559881 15167 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:50.932227 10916 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:50.932384 15167 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:50.933480 10916 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:50.916368 10943 raft_consensus.cc:1240] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 13. Current term is 14. Ops: []
I20260709 13:17:50.934579 15167 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:17:50.830637 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 342305us
I20260709 13:17:50.926036 10842 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:50.699709 14987 consensus_meta.cc:287] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.732s	user 0.009s	sys 0.000s
I20260709 13:17:50.578151 14988 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:50.860086 14912 consensus_meta.cc:287] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.734s	user 0.006s	sys 0.005s
W20260709 13:17:50.799108 15176 log.cc:927] Time spent T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
W20260709 13:17:50.798413 15178 log.cc:927] Time spent T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.302s	user 0.000s	sys 0.000s
W20260709 13:17:50.764262 15187 log.cc:927] Time spent T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.366s	user 0.001s	sys 0.000s
I20260709 13:17:50.907382 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:51.040992 10947 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
I20260709 13:17:51.042411 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:50.598115 15096 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:17:50.609103 15170 log.cc:927] Time spent T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.469s	user 0.000s	sys 0.001s
I20260709 13:17:51.048310 10793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1860) took 1220 ms. Trace:
W20260709 13:17:50.782186 15183 log.cc:927] Time spent T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.213s	user 0.001s	sys 0.000s
I20260709 13:17:50.569058 15038 raft_consensus.cc:697] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 13 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:50.937242 10948 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
I20260709 13:17:51.054837 10948 raft_consensus.cc:1275] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:17:50.942405 11478 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:50.961400 10916 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:50.971441 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:51.057699 10943 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:17:50.975842 15134 raft_consensus.cc:2749] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:17:51.059096 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 119668us
I20260709 13:17:51.060904 11478 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
W20260709 13:17:51.065228 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 103914us
I20260709 13:17:51.066004 10916 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:51.067449 10916 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:17:50.972465 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:50.923642 15151 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.005s	sys 0.019s
I20260709 13:17:50.914651 14793 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:50.914656 15158 consensus_queue.cc:1059] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.090s
I20260709 13:17:50.974390 10841 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:17:50.977502 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 51545.5us
W20260709 13:17:51.091636 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 117331us
I20260709 13:17:51.091850 10842 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:51.092829 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:51.093614 10868 raft_consensus.cc:2393] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 13: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:51.092829 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:17:50.884747 15054 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.002s	sys 0.007s
I20260709 13:17:50.694518 14799 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:50.842265 15159 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:50.694356 15128 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:51.110646 15124 consensus_meta.cc:287] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.845s	user 0.008s	sys 0.000s
I20260709 13:17:50.985656 10946 raft_consensus.cc:1275] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 13:17:50.878638 15188 raft_consensus.cc:740] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:50.773450 15182 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:51.205514 15182 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:51.205753 15188 consensus_queue.cc:260] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:51.207263 15188 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:51.208093 15182 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:51.208937 15219 consensus_queue.cc:1059] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:51.209959 15219 raft_consensus.cc:3055] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:51.210310 15219 raft_consensus.cc:740] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:51.211326 15219 consensus_queue.cc:260] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:50.992517 10764 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.700s	user 0.009s	sys 0.010s
W20260709 13:17:51.213143 10764 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.921s	user 0.009s	sys 0.010s
W20260709 13:17:51.118712 15130 consensus_meta.cc:287] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 1.024s	user 0.003s	sys 0.006s
I20260709 13:17:50.786993 15055 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:50.648635 15053 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:50.661633 14970 raft_consensus.cc:3055] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:17:50.976815 14938 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.007s	sys 0.031s
I20260709 13:17:51.048683 10793 rpcz_store.cc:276] 0709 13:17:49.828217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:49.837499 (+  9282us) service_pool.cc:224] Handling call
0709 13:17:49.837963 (+   464us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:49.844144 (+  6181us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:49.844180 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:49.845183 (+  1003us) log.cc:844] Serialized 52 byte log entry
0709 13:17:49.926190 (+ 81007us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:49.926243 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:49.926319 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:51.048053 (+1121734us) raft_consensus.cc:1685] finished
0709 13:17:51.048078 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:51.048241 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1604,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":83,"thread_start_us":80568,"threads_started":1,"wal-append.queue_time_us":192380}
I20260709 13:17:50.695358 15194 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:50.971768 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:50.987123 15119 raft_consensus.cc:2749] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:50.984452 15101 raft_consensus.cc:2749] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:17:51.076099 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:51.078477 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:51.078509 10876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 618) took 1217 ms. Trace:
I20260709 13:17:51.232563 10871 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 9.
I20260709 13:17:51.081774 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:51.233773 11505 raft_consensus.cc:2393] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 10: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:51.232591 10876 rpcz_store.cc:276] 0709 13:17:49.861133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:50.015562 (+154429us) service_pool.cc:224] Handling call
0709 13:17:50.015812 (+   250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:50.016013 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:17:50.900767 (+884754us) log.cc:844] Serialized 14 byte log entry
0709 13:17:51.078184 (+177417us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:51.078235 (+    51us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:17:51.078289 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:51.078350 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:51.078456 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":561155,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":9570,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":76207,"thread_start_us":1061252,"threads_started":2}
I20260709 13:17:51.082937 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:51.084479 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:51.086527 10942 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
I20260709 13:17:51.090004 11509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56416 (request call id 731) took 1231 ms. Trace:
I20260709 13:17:50.937000 15167 raft_consensus.cc:2749] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:17:51.093962 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 194380us
W20260709 13:17:51.087459 15151 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.725s	user 0.006s	sys 0.019s
W20260709 13:17:51.097117 15177 log.cc:927] Time spent T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.326s	user 0.001s	sys 0.000s
I20260709 13:17:51.098065 15201 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:51.098951 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:51.099054 15134 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:51.239097 15201 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:51.239905 15134 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:51.240345 15134 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:51.241474 15201 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:51.241822 10767 leader_election.cc:336] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Timed out: RequestConsensusVote RPC to 127.9.116.130:37895 timed out after 1.524s (SENT)
W20260709 13:17:51.246037 15180 log.cc:927] Time spent T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.330s	user 0.000s	sys 0.001s
I20260709 13:17:51.099476 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:51.100075 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:51.247570 11503 raft_consensus.cc:2368] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 11. Current term is 12.
I20260709 13:17:51.100787 15205 consensus_queue.cc:1048] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:51.101035 15222 consensus_queue.cc:1048] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:17:51.103520 10794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 540) took 1147 ms. Trace:
I20260709 13:17:51.105034 10875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 620) took 1198 ms. Trace:
I20260709 13:17:51.105556 14869 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:51.110107 11478 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:50.976183 15184 consensus_queue.cc:1048] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.090s
I20260709 13:17:51.111275 14793 consensus_queue.cc:1048] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:17:51.039877 14988 consensus_queue.cc:1048] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 13:17:51.115893 14446 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:51.119684 10842 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:51.220261 14970 raft_consensus.cc:740] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:51.123039 15221 raft_consensus.cc:2804] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:17:51.123826 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 56462us
I20260709 13:17:51.294814 10916 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:51.124414 15210 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:51.297937 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:51.298820 10876 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 12.
I20260709 13:17:51.124773 15158 consensus_queue.cc:1059] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:17:51.125041 15038 consensus_queue.cc:237] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:51.125775 15108 consensus_meta.cc:287] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.764s	user 0.007s	sys 0.001s
I20260709 13:17:51.125893 15218 raft_consensus.cc:3055] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:51.129254 15190 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:17:51.133595 14782 raft_consensus.cc:2749] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:17:51.134320 15085 consensus_meta.cc:287] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 1.205s	user 0.006s	sys 0.002s
I20260709 13:17:51.135142 11504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56416 (request call id 737) took 1170 ms. Trace:
I20260709 13:17:51.282907 15201 raft_consensus.cc:2749] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:17:51.138674 10944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 1974) took 1477 ms. Trace:
W20260709 13:17:51.124439 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 47290.1us
I20260709 13:17:51.314908 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:17:51.315754 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:51.314569 10944 rpcz_store.cc:276] 0709 13:17:49.660790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:50.155179 (+494389us) service_pool.cc:224] Handling call
0709 13:17:50.155555 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:50.563259 (+407704us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:50.563289 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:50.573664 (+ 10375us) log.cc:844] Serialized 26 byte log entry
0709 13:17:50.771185 (+197521us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:50.771300 (+   115us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:50.771513 (+   213us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:51.138460 (+366947us) raft_consensus.cc:1685] finished
0709 13:17:51.138478 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:51.138617 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29473,"raft.queue_time_us":59108,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":7603,"thread_start_us":539950,"threads_started":2,"wal-append.queue_time_us":502036}
I20260709 13:17:51.144976 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:50.984119 10766 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:17:51.139267 15054 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.871s	user 0.002s	sys 0.014s
I20260709 13:17:51.091286 14987 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:17:51.186358 15181 log.cc:927] Time spent T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.694s	user 0.011s	sys 0.000s
W20260709 13:17:51.313233 15173 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.810s	user 0.015s	sys 0.023s
W20260709 13:17:51.198282 15037 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.961s	user 0.001s	sys 0.009s
W20260709 13:17:51.326401 15037 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.096s	user 0.002s	sys 0.009s
I20260709 13:17:51.198594 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 621) took 1046 ms. Trace:
W20260709 13:17:51.200587 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 107816us
I20260709 13:17:51.203159 10946 pending_rounds.cc:85] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 9
I20260709 13:17:50.962685 15193 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:51.203590 15202 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:50.890545 15195 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:51.138435 14912 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.331048 15195 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:51.331280 14912 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:51.332626 15193 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:51.216859 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:51.166478 15209 raft_consensus.cc:2749] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:51.232182 10869 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 11.
I20260709 13:17:51.235800 11510 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 14.
I20260709 13:17:51.213608 15128 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:51.237154 10941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 1973) took 1576 ms. Trace:
I20260709 13:17:51.337127 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:51.212724 15219 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:51.251055 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:51.341795 10793 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:51.234563 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:51.235823 11509 rpcz_store.cc:276] 0709 13:17:49.858265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:50.056601 (+198336us) service_pool.cc:224] Handling call
0709 13:17:50.056978 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:50.514865 (+457887us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:51.089767 (+574902us) spinlock_profiling.cc:243] Waited 436 ms on lock 0x7b54004290d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:51.089944 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1001682176}
I20260709 13:17:51.269276 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:51.325500 15182 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:51.281177 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 171131us
I20260709 13:17:51.345984 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:51.346928 11478 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:51.347412 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:51.280192 10875 rpcz_store.cc:276] 0709 13:17:49.907125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:50.018208 (+111083us) service_pool.cc:224] Handling call
0709 13:17:50.075524 (+ 57316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:50.075931 (+   407us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:50.075970 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:50.090113 (+ 14143us) log.cc:844] Serialized 26 byte log entry
0709 13:17:50.897152 (+807039us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:17:50.897213 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:50.897313 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:51.104786 (+207473us) raft_consensus.cc:1685] finished
0709 13:17:51.104812 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:51.104969 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":164691,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":30051,"thread_start_us":806542,"threads_started":1,"wal-append.queue_time_us":1013033}
I20260709 13:17:51.218967 15130 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:51.348538 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:51.348897 14793 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:17:51.349071 10797 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:51.349432 14793 raft_consensus.cc:697] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 14 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:51.350061 15130 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:51.279619 10794 rpcz_store.cc:276] 0709 13:17:49.956482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:49.995337 (+ 38855us) service_pool.cc:224] Handling call
0709 13:17:49.995679 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:49.996045 (+   366us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:49.996074 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:49.996564 (+   490us) log.cc:844] Serialized 26 byte log entry
0709 13:17:50.168172 (+171608us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:17:50.168210 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:50.168274 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:51.103238 (+934964us) raft_consensus.cc:1685] finished
0709 13:17:51.103260 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:51.103435 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":218,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":115,"thread_start_us":171249,"threads_started":1,"wal-append.queue_time_us":527242}
W20260709 13:17:51.230409 14938 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.871s	user 0.008s	sys 0.031s
I20260709 13:17:51.350325 14793 consensus_queue.cc:237] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:51.351384 10794 raft_consensus.cc:1240] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:17:51.283399 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:51.352530 15134 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:51.353302 10793 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 12.
I20260709 13:17:51.348048 11478 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:51.354758 15134 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:51.336864 15202 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:51.356511 11478 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:51.357551 15134 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.357928 15202 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:51.290572 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 170973us
I20260709 13:17:51.304306 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:51.137832 15213 raft_consensus.cc:2804] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:51.305820 15096 consensus_queue.cc:1048] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 13:17:51.309898 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:51.310870 15101 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:51.310387 15085 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:51.386045 15223 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.387493 15223 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:51.388767 15085 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:51.311311 15095 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:51.389693 15223 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:51.390007 14998 consensus_meta.cc:287] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 1.288s	user 0.007s	sys 0.003s
I20260709 13:17:51.390410 14998 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:51.391376 11508 raft_consensus.cc:1240] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:17:51.307866 11504 rpcz_store.cc:276] 0709 13:17:49.964423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:50.036033 (+ 71610us) service_pool.cc:224] Handling call
0709 13:17:50.036401 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:50.057074 (+ 20673us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:50.057110 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:50.057584 (+   474us) log.cc:844] Serialized 26 byte log entry
0709 13:17:50.222098 (+164514us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:50.222161 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:50.222259 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:51.134904 (+912645us) raft_consensus.cc:1685] finished
0709 13:17:51.134926 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:51.135079 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24413,"raft.queue_time_us":27198,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":14338,"thread_start_us":164176,"threads_started":1,"wal-append.queue_time_us":243623}
I20260709 13:17:51.251621 15194 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:51.392467 14998 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:51.394280 15194 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:51.394351 15101 raft_consensus.cc:697] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:51.313416 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:51.143095 10943 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 12.
I20260709 13:17:51.317062 10948 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 14.
I20260709 13:17:51.395480 15101 consensus_queue.cc:237] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:51.318989 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:51.391014 15095 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:51.396533 10867 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
I20260709 13:17:51.318217 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:51.397518 10872 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 11.
I20260709 13:17:51.318295 10916 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:51.295199 15119 raft_consensus.cc:2749] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:17:51.189845 14968 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:51.303599 15210 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:51.230746 15053 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
W20260709 13:17:51.327700 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:51.327536 10841 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:17:51.327997 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 343949us
I20260709 13:17:51.303783 15218 raft_consensus.cc:740] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:51.403172 10766 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:17:51.403308 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 75812.1us
I20260709 13:17:51.326946 10873 rpcz_store.cc:276] 0709 13:17:50.152406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:50.154455 (+  2049us) service_pool.cc:224] Handling call
0709 13:17:50.154857 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:50.284447 (+129590us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:17:50.559092 (+274645us) log.cc:844] Serialized 14 byte log entry
0709 13:17:50.689154 (+130062us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:50.689235 (+    81us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:17:50.689325 (+    90us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:51.193123 (+503798us) spinlock_profiling.cc:243] Waited 9.32 ms on lock 0x7b180040cae0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7dd962a6 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc8557 00007f9c7ddca7c1
0709 13:17:51.193985 (+   862us) spinlock_profiling.cc:243] Waited 9.55 ms on lock 0x7b180040cae0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc8557 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:17:51.198329 (+  4344us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b5400076054. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:51.198376 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:51.198516 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":373258,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":54,"raft.queue_time_us":62488,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":16163,"spinlock_wait_cycles":339603968,"thread_start_us":201586,"threads_started":2,"wal-append.queue_time_us":147823}
I20260709 13:17:51.330593 10946 pending_rounds.cc:107] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:17:51.337448 10941 rpcz_store.cc:276] 0709 13:17:49.660566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:50.152943 (+492377us) service_pool.cc:224] Handling call
0709 13:17:50.153246 (+   303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:50.213479 (+ 60233us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:50.213510 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:50.214042 (+   532us) log.cc:844] Serialized 26 byte log entry
0709 13:17:51.055633 (+841591us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:51.055694 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:51.218465 (+162771us) spinlock_profiling.cc:243] Waited 54 ms on lock 0x7b5400399754. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:51.218513 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:51.218558 (+    45us) raft_consensus.cc:1685] finished
0709 13:17:51.218572 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:51.237091 (+ 18519us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63028,"raft.queue_time_us":80289,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":7972,"spinlock_wait_cycles":126123264,"thread_start_us":773446,"threads_started":1,"wal-append.queue_time_us":872480}
I20260709 13:17:51.338894 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:51.404031 10946 raft_consensus.cc:2889] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:51.340205 10798 raft_consensus.cc:1240] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:51.321000 14987 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:51.320683 15221 raft_consensus.cc:697] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 10 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:51.319973 15190 raft_consensus.cc:697] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 12 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
W20260709 13:17:51.329161 15173 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.823s	user 0.016s	sys 0.023s
I20260709 13:17:51.343477 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:51.269631 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:51.306028 15108 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:51.344251 10792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 542) took 1376 ms. Trace:
I20260709 13:17:51.333532 14912 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:51.358134 15134 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:51.403316 15210 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:51.423378 15221 consensus_queue.cc:237] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:51.424728 15134 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:51.358412 15231 raft_consensus.cc:2804] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:51.425493 15231 raft_consensus.cc:697] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 9 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:51.344601 15193 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.355674 15201 raft_consensus.cc:2804] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:51.362170 11478 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:51.367710 14988 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.120647 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:51.326700 15037 raft_consensus.cc:697] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 13 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
W20260709 13:17:51.374677 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:51.429005 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 66906.2us
I20260709 13:17:51.381103 10876 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 10.
I20260709 13:17:51.381100 10842 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:51.334076 14869 raft_consensus.cc:2804] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:17:51.432126 14869 raft_consensus.cc:697] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:51.422523 15218 consensus_queue.cc:260] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:51.433108 15037 consensus_queue.cc:237] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:51.434183 15218 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:51.385628 10871 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
I20260709 13:17:51.435217 15167 raft_consensus.cc:3055] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:17:51.435721 15167 raft_consensus.cc:740] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:51.435618 14988 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:51.429492 11477 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:17:51.436993 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 55990.2us
I20260709 13:17:51.437718 10842 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
W20260709 13:17:51.399986 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 81784us
I20260709 13:17:51.403513 10949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 1982) took 1740 ms. Trace:
W20260709 13:17:51.405757 10767 leader_election.cc:336] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Timed out: RequestConsensusVote RPC to 127.9.116.130:37895 timed out after 1.656s (SENT)
I20260709 13:17:51.439770 10916 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:51.375254 15182 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:51.407418 15202 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:51.475744 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:51.416244 10795 raft_consensus.cc:2393] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 11: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:51.416646 10792 rpcz_store.cc:276] 0709 13:17:49.968270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:50.016551 (+ 48281us) service_pool.cc:224] Handling call
0709 13:17:50.016864 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:50.017182 (+   318us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:50.017210 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:50.057480 (+ 40270us) log.cc:844] Serialized 26 byte log entry
0709 13:17:50.160739 (+103259us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:50.160797 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:50.160902 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:51.185547 (+1024645us) spinlock_profiling.cc:243] Waited 16.2 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:17:51.343939 (+158392us) raft_consensus.cc:1685] finished
0709 13:17:51.343964 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:51.344192 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":494,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":22477,"spinlock_wait_cycles":37252864,"thread_start_us":10802,"threads_started":1,"wal-append.queue_time_us":138549}
I20260709 13:17:51.418181 15212 consensus_queue.cc:1059] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 13:17:51.415210 14968 consensus_queue.cc:1048] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 13:17:51.329824 14970 consensus_queue.cc:260] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:51.400987 15213 raft_consensus.cc:697] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 9 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:51.420513 15108 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:51.421458 15192 raft_consensus.cc:3055] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:51.422274 15196 raft_consensus.cc:3055] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:17:51.431377 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:51.402066 15188 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:51.433108 14869 consensus_queue.cc:237] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:51.215328 15124 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:51.405627 10841 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:51.438901 10842 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:51.439095 10949 rpcz_store.cc:276] 0709 13:17:49.663144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:50.219296 (+556152us) service_pool.cc:224] Handling call
0709 13:17:50.219639 (+   343us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:50.891401 (+671762us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:50.891445 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:50.928298 (+ 36853us) log.cc:844] Serialized 52 byte log entry
0709 13:17:51.068898 (+140600us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:51.068976 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:51.069119 (+   143us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:51.403271 (+334152us) raft_consensus.cc:1685] finished
0709 13:17:51.403290 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:51.403453 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2043,"raft.queue_time_us":167705,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":49126,"thread_start_us":811301,"threads_started":2,"wal-append.queue_time_us":302543}
I20260709 13:17:51.404354 15085 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.384122 10797 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 14.
W20260709 13:17:51.540038 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 101240us
I20260709 13:17:51.408643 14987 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:51.428365 10766 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:51.441421 11477 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:17:51.474485 10767 leader_election.cc:336] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Timed out: RequestConsensusVote RPC to 127.9.116.130:37895 timed out after 1.731s (SENT)
I20260709 13:17:51.475011 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:51.435351 15222 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:17:51.541862 15119 raft_consensus.cc:2749] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:17:51.478468 11478 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:51.481105 10916 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:51.494009 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:51.442190 14793 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:51.433099 15053 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:51.478376 15173 raft_consensus.cc:2749] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:51.478139 15134 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:51.478293 15157 raft_consensus.cc:2749] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:51.481000 15084 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:17:51.564247 15084 raft_consensus.cc:697] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 12 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:51.564584 15124 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:51.428661 14782 consensus_queue.cc:1059] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.099s
I20260709 13:17:51.565277 15084 consensus_queue.cc:237] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:51.547838 15085 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:51.567250 15179 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:51.567026 15195 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:51.567773 15179 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:51.568192 15179 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:51.568980 15085 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:51.539156 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 133581us
I20260709 13:17:51.570611 10841 leader_election.cc:400] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 election: Vote denied by peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553) with higher term. Message: Invalid argument: T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 13. Current term is 14.
I20260709 13:17:51.570947 10841 leader_election.cc:403] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:17:51.571594 15124 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.572188 15124 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:51.572956 10841 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:51.436447 15193 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:51.574517 15119 raft_consensus.cc:2749] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553) with higher term. Message: Invalid argument: T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 13. Current term is 14.
I20260709 13:17:51.575062 14998 consensus_queue.cc:1059] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:51.575505 15119 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:51.575930 15054 raft_consensus.cc:3055] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:17:51.542995 10873 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:51.576366 15054 raft_consensus.cc:740] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:51.576560 15193 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:51.426465 15231 consensus_queue.cc:237] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:51.532830 15196 raft_consensus.cc:740] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:51.577426 15054 consensus_queue.cc:260] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:51.577795 15196 consensus_queue.cc:260] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:51.547286 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 105945us
I20260709 13:17:51.578662 15054 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:51.578853 15190 consensus_queue.cc:237] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:51.533264 15192 raft_consensus.cc:740] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:17:51.548285 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 67254.2us
I20260709 13:17:51.581077 10916 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:51.582242 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:17:51.485460 15230 log.cc:927] Time spent T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
W20260709 13:17:51.550029 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 71709.9us
I20260709 13:17:51.560745 10950 raft_consensus.cc:2393] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 12: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:51.528031 15201 raft_consensus.cc:697] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 10 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:51.563146 15134 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:51.555720 14912 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:51.569061 15195 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:51.559713 15184 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:17:51.540813 10842 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:51.574445 15124 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:51.576306 15213 consensus_queue.cc:237] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:51.583595 10916 leader_election.cc:400] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 14. Current term is 15.
I20260709 13:17:51.587200 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:51.609645 10916 leader_election.cc:403] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:17:51.547813 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 119509us
I20260709 13:17:51.611025 10766 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:51.613296 10873 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 12.
I20260709 13:17:51.533442 15202 raft_consensus.cc:697] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:17:51.543076 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 68633.7us
I20260709 13:17:51.534754 15188 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:51.591800 11478 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:17:51.617681 15188 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:51.619529 15037 raft_consensus.cc:2764] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:17:51.621858 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:51.576139 15119 raft_consensus.cc:697] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:51.561450 15075 raft_consensus.cc:2749] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:17:51.625203 11508 raft_consensus.cc:2393] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 13: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:51.561226 15095 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:51.625837 11478 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:17:51.627843 15037 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:17:51.628880 15037 raft_consensus.cc:697] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 12 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:51.527215 15167 consensus_queue.cc:260] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:51.631564 15167 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:51.591063 15130 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.589506 14968 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.589586 15128 raft_consensus.cc:2749] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:51.609126 10915 leader_election.cc:400] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 11. Current term is 12.
I20260709 13:17:51.637785 10915 leader_election.cc:403] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:17:51.542362 15017 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:51.616250 15202 consensus_queue.cc:237] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:51.603340 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 62607us
I20260709 13:17:51.631846 11504 raft_consensus.cc:1240] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 9. Current term is 10. Ops: []
W20260709 13:17:51.629056 10767 leader_election.cc:336] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Timed out: RequestConsensusVote RPC to 127.9.116.130:37895 timed out after 2.148s (SENT)
I20260709 13:17:51.639505 10767 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:51.634205 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:17:51.635888 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 52433.8us
I20260709 13:17:51.641222 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:51.642109 10872 raft_consensus.cc:2393] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 9: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:51.642740 10876 raft_consensus.cc:1275] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:17:51.642620 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:51.636680 15215 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:17:51.639722 11510 raft_consensus.cc:1240] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 14 LEADER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:17:51.595882 15193 raft_consensus.cc:2804] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:51.643760 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:51.644143 10915 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:51.602054 15134 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:51.646360 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:51.647097 10948 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 12: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:51.648295 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:51.648363 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:51.649495 10942 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
I20260709 13:17:51.648481 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:51.648629 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:51.640715 10916 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:51.650522 10868 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 9.
I20260709 13:17:51.603011 15196 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:51.652491 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:51.652098 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:51.653172 10869 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 14.
I20260709 13:17:51.599709 14912 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:51.606621 15184 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:51.605736 15173 raft_consensus.cc:2764] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:17:51.660379 10949 raft_consensus.cc:1275] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:17:51.657068 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:51.661245 10874 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
I20260709 13:17:51.593163 15157 consensus_queue.cc:1059] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:17:51.658910 11478 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:51.695955 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:51.633208 14782 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:17:51.661859 10767 leader_election.cc:336] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Timed out: RequestConsensusVote RPC to 127.9.116.130:37895 timed out after 2.215s (SENT)
I20260709 13:17:51.740139 10868 raft_consensus.cc:1275] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:17:51.735038 10876 raft_consensus.cc:1275] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 13:17:51.750102 10943 raft_consensus.cc:2410] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 14 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:17:51.896427 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:51.727303 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:51.717381 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:51.658749 14945 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:17:51.751698 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:51.611318 15124 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.753574 10941 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:51.752831 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:51.652216 14968 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:51.797163 15235 log.cc:927] Time spent T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.266s	user 0.000s	sys 0.001s
I20260709 13:17:51.703248 15184 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:51.756111 15101 consensus_queue.cc:1048] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:17:51.652074 15212 raft_consensus.cc:2749] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:17:51.726464 15037 consensus_queue.cc:237] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:51.810910 10767 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:51.652797 15193 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:17:51.814465 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:51.666245 14912 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:51.658912 15194 raft_consensus.cc:2749] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 11. Current term is 12.
I20260709 13:17:51.871531 10867 raft_consensus.cc:2368] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 8. Current term is 9.
I20260709 13:17:51.819397 10876 pending_rounds.cc:85] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 6
I20260709 13:17:51.960995 10876 pending_rounds.cc:107] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:17:51.961292 10876 raft_consensus.cc:2889] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:51.902244 10873 raft_consensus.cc:1240] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 10. Current term is 11. Ops: []
I20260709 13:17:51.663427 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:51.896596 10766 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:51.962122 10872 raft_consensus.cc:2393] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 15: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:51.902465 14782 raft_consensus.cc:697] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:51.962775 10873 raft_consensus.cc:1240] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 12. Current term is 13. Ops: []
I20260709 13:17:51.914459 15221 raft_consensus.cc:3055] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:17:51.922209 10875 raft_consensus.cc:2393] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 11: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
W20260709 13:17:51.895120 15219 consensus_meta.cc:287] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.528s	user 0.007s	sys 0.011s
I20260709 13:17:51.922292 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:51.696991 15053 consensus_queue.cc:1059] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 5, Time since last communication: 0.202s
I20260709 13:17:51.931560 10868 raft_consensus.cc:1240] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:51.921802 14945 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:51.658007 15130 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:51.906284 15160 raft_consensus.cc:2749] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:17:51.959677 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 297842us
I20260709 13:17:51.982555 14945 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:17:51.988777 15241 log.cc:927] Time spent T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:17:51.993865 14858 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.995793 14968 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:51.992689 15130 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:51.958622 10941 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
I20260709 13:17:51.755400 15158 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:52.011157 15158 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:52.011502 15073 raft_consensus.cc:3055] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:17:52.012172 15073 raft_consensus.cc:740] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:52.013321 15073 consensus_queue.cc:260] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:51.929528 14943 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:52.015414 14943 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:52.016945 10946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 1972) took 2356 ms. Trace:
I20260709 13:17:51.971459 15225 raft_consensus.cc:2749] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:51.950174 15119 consensus_queue.cc:237] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:52.026685 14943 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:52.013360 15158 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:52.027330 15073 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:51.818027 15201 consensus_queue.cc:237] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:51.976459 10868 raft_consensus.cc:1275] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:17:52.017347 10946 rpcz_store.cc:276] 0709 13:17:49.660328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:50.152021 (+491693us) service_pool.cc:224] Handling call
0709 13:17:50.152343 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:51.404617 (+1252274us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:52.016693 (+612076us) spinlock_profiling.cc:243] Waited 833 ms on lock 0x7b5400015ed4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:52.016884 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1915403136}
W20260709 13:17:51.953298 15210 consensus_meta.cc:287] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.529s	user 0.017s	sys 0.000s
I20260709 13:17:51.981469 15219 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:17:51.981878 15162 raft_consensus.cc:2749] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:51.982036 15053 consensus_queue.cc:1243] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer 12682eb70bc84c76879e04a8628ee586 log is divergent from this leader: its last log entry 14.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:17:51.952415 15193 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:17:52.007472 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:51.954416 14988 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:51.960333 15182 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:51.994541 14858 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:51.637336 15192 consensus_queue.cc:260] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:51.972196 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:51.968178 15221 raft_consensus.cc:740] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:51.936188 15124 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:52.080193 10841 leader_election.cc:336] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Timed out: RequestConsensusVote RPC to 127.9.116.132:40555 timed out after 1.694s (SENT)
I20260709 13:17:52.108695 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:52.109637 10876 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
W20260709 13:17:52.114625 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:52.118074 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:52.057798 15095 raft_consensus.cc:2749] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:52.119110 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:52.119781 10950 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41014 (request call id 1976) took 2448 ms (client timeout 2195 ms). Trace:
I20260709 13:17:52.120064 10945 raft_consensus.cc:2410] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 13 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
W20260709 13:17:52.120287 10950 rpcz_store.cc:269] 0709 13:17:49.661678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:49.969237 (+307559us) service_pool.cc:224] Handling call
0709 13:17:52.109561 (+2140324us) spinlock_profiling.cc:243] Waited 1.59 s on lock 0x7b54003f0054. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:52.109724 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3657159040}
I20260709 13:17:52.123719 10870 raft_consensus.cc:1240] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:17:52.059901 15219 raft_consensus.cc:697] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:52.129930 10948 raft_consensus.cc:1240] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 10. Current term is 11. Ops: []
I20260709 13:17:52.130931 10872 raft_consensus.cc:1240] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 12. Current term is 13. Ops: []
I20260709 13:17:52.133719 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:52.134240 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:52.139724 10944 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:52.137357 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:52.142251 10948 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 13.
I20260709 13:17:52.057462 15188 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:52.151266 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:52.144411 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:52.114358 15101 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:52.050915 15210 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:52.162160 10876 raft_consensus.cc:1240] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:52.156912 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:52.166859 10943 raft_consensus.cc:2393] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 15: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:52.121241 15020 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:52.214603 15240 log.cc:927] Time spent T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.157s	user 0.000s	sys 0.000s
I20260709 13:17:52.222916 15116 raft_consensus.cc:3055] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:17:52.224826 15237 log.cc:927] Time spent T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.424s	user 0.000s	sys 0.004s
I20260709 13:17:52.225868 14912 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:52.169269 14998 consensus_queue.cc:1059] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:17:52.235558 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:52.236078 10867 raft_consensus.cc:1240] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 11. Current term is 12. Ops: [9.9-11.10]
I20260709 13:17:52.238212 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:52.257145 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:52.133822 15192 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:52.246209 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:52.247454 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:52.248957 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:52.236963 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:52.253260 10947 raft_consensus.cc:2393] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 13: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:52.258502 10948 raft_consensus.cc:1240] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 12. Current term is 13. Ops: []
I20260709 13:17:52.298267 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:52.296869 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:52.259629 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:52.300384 10794 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
I20260709 13:17:52.301460 10793 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 10.
I20260709 13:17:52.301769 10798 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:52.302158 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:52.302625 10795 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:52.303134 10876 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
I20260709 13:17:52.254567 10944 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
I20260709 13:17:52.257179 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:52.306109 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:52.306624 10867 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:17:52.308593 10916 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:52.309700 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:52.310609 10799 raft_consensus.cc:2393] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 10: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:52.310770 10800 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 12.
I20260709 13:17:52.312436 10792 raft_consensus.cc:1240] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 12. Current term is 13. Ops: []
I20260709 13:17:52.316327 11506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56416 (request call id 740) took 1232 ms. Trace:
W20260709 13:17:52.319095 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 663912us
I20260709 13:17:52.316974 11506 rpcz_store.cc:276] 0709 13:17:51.083607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:51.083720 (+   113us) service_pool.cc:224] Handling call
0709 13:17:51.100193 (+ 16473us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:51.247887 (+147694us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:51.247922 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:51.248441 (+   519us) log.cc:844] Serialized 26 byte log entry
0709 13:17:51.324673 (+ 76232us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:51.324715 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:52.298850 (+974135us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b54005a37d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:52.298929 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:52.299000 (+    71us) raft_consensus.cc:1685] finished
0709 13:17:52.299022 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:52.316227 (+ 17205us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1193,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":31825,"spinlock_wait_cycles":338667776,"thread_start_us":75868,"threads_started":1,"wal-append.queue_time_us":152965}
I20260709 13:17:52.321036 10841 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:52.257952 10874 raft_consensus.cc:2393] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 12: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:17:52.309875 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 53894.5us
I20260709 13:17:52.325064 10916 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:52.327783 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:52.328713 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:52.335390 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 438844us
I20260709 13:17:52.240690 14970 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:17:52.297260 15243 log.cc:927] Time spent T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.186s	user 0.000s	sys 0.001s
I20260709 13:17:52.119946 14782 consensus_queue.cc:237] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:52.411827 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:52.405799 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:52.405457 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:52.402606 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:52.410029 10867 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
I20260709 13:17:52.416482 10870 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 12.
I20260709 13:17:52.416806 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:52.410576 10795 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 12.
I20260709 13:17:52.418458 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:52.420048 10867 raft_consensus.cc:2393] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 9: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:52.249574 15116 raft_consensus.cc:740] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:52.402096 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:52.427369 10874 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 14.
W20260709 13:17:52.437175 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 112200us
I20260709 13:17:52.443454 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:52.444438 10867 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 11.
I20260709 13:17:52.233176 15101 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:52.446861 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:52.451264  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:52.452257 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:52.445883 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:52.453802 10867 raft_consensus.cc:2410] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
W20260709 13:17:52.453886 10942 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.001s	sys 0.010s
W20260709 13:17:52.454278 10942 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.001s	sys 0.010s
W20260709 13:17:52.412819 15239 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.018s	sys 0.024s
I20260709 13:17:52.438843 10916 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:17:52.457528 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:52.338366 15173 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:52.419369 15182 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:52.459456 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:52.460296 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:52.410058 10766 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:52.256166 14945 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:52.449664 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:52.486115 10874 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 14.
W20260709 13:17:52.463353 15239 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.659s	user 0.019s	sys 0.024s
I20260709 13:17:52.477398 10870 raft_consensus.cc:2368] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 8. Current term is 9.
I20260709 13:17:52.251485 15020 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:52.482599 10767 leader_election.cc:336] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Timed out: RequestConsensusVote RPC to 127.9.116.130:37895 timed out after 1.855s (SENT)
I20260709 13:17:52.450840 15157 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:17:52.487537 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:52.512759 10874 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:52.464610 10916 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:17:52.514217 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:52.514586 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 50111.7us
I20260709 13:17:52.515520 10916 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:52.408272 15095 consensus_queue.cc:1059] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.078s
I20260709 13:17:52.517071 10797 raft_consensus.cc:1275] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:17:52.505903 10767 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:52.314170 15221 consensus_queue.cc:260] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:52.473060 15158 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:52.526616 10949 raft_consensus.cc:1240] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:17:52.527035 15158 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:17:52.498030 15167 consensus_meta.cc:287] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.846s	user 0.007s	sys 0.005s
W20260709 13:17:52.512882 15184 consensus_meta.cc:287] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.536s	user 0.012s	sys 0.000s
I20260709 13:17:52.476171 15182 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:17:52.529682 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 47140us
W20260709 13:17:52.545791 10767 leader_election.cc:336] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Timed out: RequestConsensusVote RPC to 127.9.116.130:37895 timed out after 1.969s (SENT)
I20260709 13:17:52.524966 15096 raft_consensus.cc:2764] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:17:52.484701 15173 raft_consensus.cc:697] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:52.527592 15158 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:17:52.533644 11477 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.638s	user 0.034s	sys 0.016s
W20260709 13:17:52.561863 11477 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.034s	sys 0.016s
I20260709 13:17:52.395815 15219 consensus_queue.cc:237] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:52.481189 10798 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 11.
I20260709 13:17:52.564881 10798 raft_consensus.cc:1275] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:17:52.542991 15095 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:17:52.540596 10871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 629) took 1098 ms. Trace:
I20260709 13:17:52.542903 10874 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 12.
W20260709 13:17:52.571892 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:52.546062 15182 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:52.531121 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:52.497129 15251 raft_consensus.cc:2749] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:52.554458 15221 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:52.535519 14938 raft_consensus.cc:2749] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:52.567589 11477 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:52.571000 10944 raft_consensus.cc:1275] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:17:52.572430 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:52.519856 15157 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:52.571638 10871 rpcz_store.cc:276] 0709 13:17:51.442515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:51.474127 (+ 31612us) service_pool.cc:224] Handling call
0709 13:17:51.482601 (+  8474us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:51.528525 (+ 45924us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:51.528555 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:51.541469 (+ 12914us) log.cc:844] Serialized 590 byte log entry
0709 13:17:51.896161 (+354692us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:51.896206 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:51.896282 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:52.540362 (+644080us) raft_consensus.cc:1685] finished
0709 13:17:52.540383 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:52.540536 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":88,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":3066,"thread_start_us":300623,"threads_started":1,"wal-append.queue_time_us":444687}
W20260709 13:17:52.489251 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 80928.8us
I20260709 13:17:52.549866 15222 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:52.559115 15124 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:52.596428 15157 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:17:52.606928 15196 consensus_meta.cc:287] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.943s	user 0.004s	sys 0.007s
W20260709 13:17:52.607692 15215 consensus_meta.cc:287] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.911s	user 0.009s	sys 0.005s
I20260709 13:17:52.608186 15215 raft_consensus.cc:2749] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 14. Current term is 15.
I20260709 13:17:52.609618 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:52.610319 10870 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:52.612186 10946 raft_consensus.cc:1275] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:17:52.612200 15185 raft_consensus.cc:3055] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:17:52.613930 15185 raft_consensus.cc:740] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:17:52.614157 15218 consensus_meta.cc:287] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 1.179s	user 0.006s	sys 0.007s
I20260709 13:17:52.614774 15185 consensus_queue.cc:260] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:52.615862 15185 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:17:52.618891 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:52.619058 15210 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:52.619396 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:52.521584 15116 consensus_queue.cc:260] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:52.571457 15095 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:52.621587 15095 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:52.621783 15116 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:17:52.621111 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:52.610337 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:52.636905 11506 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
I20260709 13:17:52.634421 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:52.639264 10871 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 13.
I20260709 13:17:52.635413 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:52.546289 15184 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:52.642338 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:52.631208 15195 raft_consensus.cc:2804] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:17:52.629035 15210 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:17:52.633983 15134 consensus_meta.cc:287] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.938s	user 0.014s	sys 0.003s
I20260709 13:17:52.644564 10871 raft_consensus.cc:1275] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 13:17:52.614202 14987 consensus_meta.cc:287] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 1.055s	user 0.007s	sys 0.002s
I20260709 13:17:52.644392 15084 consensus_queue.cc:1048] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:17:52.645581 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:52.628866 15218 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:52.631752 15250 consensus_queue.cc:1048] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.015s
W20260709 13:17:52.646463 15193 consensus_meta.cc:287] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.543s	user 0.007s	sys 0.001s
I20260709 13:17:52.600150 15222 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:17:52.649277 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:52.601517 15073 consensus_meta.cc:287] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.570s	user 0.007s	sys 0.006s
I20260709 13:17:52.576406 15119 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:52.668596 10871 raft_consensus.cc:1275] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:17:52.671424 15020 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:52.592293 15188 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:52.626756 14945 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:52.674484 10943 raft_consensus.cc:2410] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:17:52.636766 14912 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:52.653721 15195 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:52.674465 15134 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:52.684912 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:52.673508 15210 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:52.692068 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:52.713670 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:52.714403 10871 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:52.694453 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:52.715437 10941 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:52.684587 10870 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
I20260709 13:17:52.674360 14987 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:52.727644 14987 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:52.695763 15119 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:52.730003 15119 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:52.716977 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:52.680408 15218 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:52.721311 15054 consensus_meta.cc:287] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 1.142s	user 0.009s	sys 0.001s
I20260709 13:17:52.681948 15173 consensus_queue.cc:237] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:52.723878 10874 raft_consensus.cc:2410] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
W20260709 13:17:52.725675 15017 consensus_meta.cc:287] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 1.070s	user 0.010s	sys 0.000s
I20260709 13:17:52.731757 10870 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:17:52.763595 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 248163us
I20260709 13:17:52.764830 10873 raft_consensus.cc:2393] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 10: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:52.683624 15193 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:52.739368 10941 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 13.
I20260709 13:17:52.748099 10867 raft_consensus.cc:1275] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:17:52.701725 15195 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:52.754626 10766 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:52.765293 10916 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:52.773967 10871 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 13.
I20260709 13:17:52.768090 14446 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:52.779858 14446 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:52.780673 15193 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:52.781028 10916 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:52.781301 15193 consensus_queue.cc:1059] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:52.781977 15188 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:52.782434 15188 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:52.782807 15188 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:52.782889 10941 raft_consensus.cc:1240] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 12. Current term is 13. Ops: []
W20260709 13:17:52.785852 15268 log.cc:927] Time spent T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:17:52.788172 15134 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:52.789899 15251 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:52.790490 15251 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:52.791245 15188 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:52.791744 15210 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:52.792919 15251 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:52.795351 15073 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:17:52.795945 15073 raft_consensus.cc:697] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:52.796512 10916 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:52.796918 15073 consensus_queue.cc:237] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:52.798449 15188 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:52.801338 15020 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:17:52.801744 15020 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:52.802126 15020 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:52.807126 15162 consensus_queue.cc:1048] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:17:52.807667 15258 log.cc:927] Time spent T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 13:17:52.809770 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:52.810747 10796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 1862) took 1474 ms. Trace:
I20260709 13:17:52.811162 10796 rpcz_store.cc:276] 0709 13:17:51.336598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:51.336869 (+   271us) service_pool.cc:224] Handling call
0709 13:17:52.810567 (+1473698us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54000d8254. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:52.810687 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2669676288}
I20260709 13:17:52.813019 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:52.816545 15095 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:52.817708 10944 raft_consensus.cc:1240] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 13. Current term is 14. Ops: []
I20260709 13:17:52.814517 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:52.820149 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:52.820824 10872 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
I20260709 13:17:52.821815 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:52.824419 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:52.825928 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:52.826879 10871 raft_consensus.cc:2393] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 11: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:52.828176 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:52.829674 10944 raft_consensus.cc:1240] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 12. Current term is 13. Ops: []
I20260709 13:17:52.823074 10867 raft_consensus.cc:2376] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 13. Re-sending same reply.
I20260709 13:17:52.834692 10872 raft_consensus.cc:2376] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11. Re-sending same reply.
I20260709 13:17:52.855165 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:52.856405 10872 raft_consensus.cc:2393] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 15: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:17:52.860795 10767 leader_election.cc:336] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Timed out: RequestConsensusVote RPC to 127.9.116.130:37895 timed out after 1.620s (SENT)
I20260709 13:17:52.864195 11509 raft_consensus.cc:1240] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 10 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
W20260709 13:17:52.868718 15248 log.cc:927] Time spent T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.448s	user 0.001s	sys 0.003s
W20260709 13:17:52.869006 10915 leader_election.cc:336] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Timed out: RequestConsensusVote RPC to 127.9.116.132:40555 timed out after 1.995s (SENT)
W20260709 13:17:52.869294 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 48792.3us
I20260709 13:17:52.868312 15020 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:52.881728 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:52.749619 14869 consensus_queue.cc:1048] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.133s
I20260709 13:17:52.882674 15020 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:52.883276 15195 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:52.884179 15184 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:52.768167 15252 consensus_queue.cc:1048] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.117s
I20260709 13:17:52.885552 15195 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:52.870682 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:52.874104 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:52.882640 10872 raft_consensus.cc:2393] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 15: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:52.886889 10944 raft_consensus.cc:2393] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 12: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:52.875960 10948 raft_consensus.cc:1240] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 10. Current term is 11. Ops: []
W20260709 13:17:52.891002 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 85766.3us
I20260709 13:17:52.891085 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:52.891968 10867 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
I20260709 13:17:52.893069 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:52.893632 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:52.899562 14858 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:52.868768 15179 consensus_meta.cc:287] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 1.300s	user 0.013s	sys 0.000s
I20260709 13:17:52.901484 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:52.902911 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:52.906294 11505 raft_consensus.cc:1240] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 13 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 12. Current term is 13. Ops: []
I20260709 13:17:52.920475 10792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 559) took 1301 ms. Trace:
I20260709 13:17:52.920874 10792 rpcz_store.cc:276] 0709 13:17:51.620521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:51.620679 (+   158us) service_pool.cc:224] Handling call
0709 13:17:51.621025 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:52.312866 (+691841us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:52.920236 (+607370us) spinlock_profiling.cc:243] Waited 691 ms on lock 0x7b54002120d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:52.920416 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1589659392}
W20260709 13:17:52.922827 11511 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56404 (request call id 763) took 1837 ms (client timeout 1995 ms). Trace:
W20260709 13:17:52.924643 10947 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41014 (request call id 1996) took 1275 ms (client timeout 1667 ms). Trace:
I20260709 13:17:52.919342 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:17:52.925304 10947 rpcz_store.cc:269] 0709 13:17:51.649440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:51.649817 (+   377us) service_pool.cc:224] Handling call
0709 13:17:52.924392 (+1274575us) spinlock_profiling.cc:243] Waited 352 ms on lock 0x7b54001e2654. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:52.924568 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":808995200}
I20260709 13:17:52.926671 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:52.927691 10872 raft_consensus.cc:2393] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 11: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:52.892889 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:52.907052 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:52.829291 15095 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:52.911629 10870 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
W20260709 13:17:52.916056 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:52.931260 11509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56404 (request call id 767) took 1297 ms. Trace:
I20260709 13:17:52.931638 11509 rpcz_store.cc:276] 0709 13:17:51.633770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:51.633991 (+   221us) service_pool.cc:224] Handling call
0709 13:17:51.634381 (+   390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:52.864648 (+1230267us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:52.912663 (+ 48015us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b54005014d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:52.931185 (+ 18522us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2827482368}
I20260709 13:17:52.932596 10875 raft_consensus.cc:1240] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 9. Current term is 10. Ops: []
W20260709 13:17:52.933269 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:52.919431 11512 raft_consensus.cc:1240] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 11. Current term is 12. Ops: []
W20260709 13:17:52.928198 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:52.929373 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:17:52.923337 11511 rpcz_store.cc:269] 0709 13:17:51.085400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:51.085554 (+   154us) service_pool.cc:224] Handling call
0709 13:17:52.922570 (+1837016us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54007e1b54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:52.922729 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2700216064}
I20260709 13:17:52.900637 15231 consensus_queue.cc:1059] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:17:52.930543 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:52.928380 10945 raft_consensus.cc:1240] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 12. Current term is 13. Ops: []
I20260709 13:17:52.936926 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:52.939882 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:52.940878 10799 raft_consensus.cc:2393] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 13: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:52.931511 10841 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:52.942521 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:52.928742 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:52.942457 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:52.895118 15038 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:52.895956 15209 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:52.890640 15215 consensus_queue.cc:1048] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.073s
I20260709 13:17:52.954756 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:17:53.034679 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:52.943892 10947 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:52.972780 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:52.951694 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:53.388465 10870 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 12.
I20260709 13:17:52.907224 15134 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:52.895407 14945 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:53.364029 10950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 712) took 1239 ms. Trace:
I20260709 13:17:52.944162 10948 raft_consensus.cc:2393] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 15: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:53.395347 10950 rpcz_store.cc:276] 0709 13:17:52.124898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:52.139208 (+ 14310us) service_pool.cc:224] Handling call
0709 13:17:52.139544 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:52.140045 (+   501us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:17:52.140108 (+    63us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:52.614471 (+474363us) spinlock_profiling.cc:243] Waited 6.86 ms on lock 0x7b2c0015d218. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7f7aea11 00007f9c7f7be372 00007f9c80ef0a01 00007f9c80eb543a 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2
0709 13:17:52.614889 (+   418us) log.cc:844] Serialized 26 byte log entry
0709 13:17:52.796953 (+182064us) spinlock_profiling.cc:243] Waited 6.29 ms on lock 0x7b48002415d8. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c85ddeaa1 00007f9c85dd5b37 00007f9c85dd59ca 00007f9c85ecd26f 00007f9c85eccf37 00007f9c85ee155c 00007f9c85ee1508 00007f9c85ee1488 00007f9c85ee1443 00007f9c85ee06ec 00007f9c84e0aa96
0709 13:17:52.891097 (+ 94144us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:52.891146 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:52.891214 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:53.363787 (+472573us) raft_consensus.cc:1685] finished
0709 13:17:53.363808 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:53.363961 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":238,"raft.run_cpu_time_us":5598,"raft.run_wall_time_us":652874,"spinlock_wait_cycles":30255360,"thread_start_us":275807,"threads_started":1,"wal-append.queue_time_us":747005}
I20260709 13:17:53.373350 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:52.963132 11506 raft_consensus.cc:1275] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 13:17:53.387849 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:53.358914 15196 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.004s	sys 0.037s
I20260709 13:17:53.377135 10873 raft_consensus.cc:2393] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 12: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:53.409279 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:53.391621 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:52.930845 15054 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:53.023204 15095 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:53.362022 15157 consensus_meta.cc:287] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.758s	user 0.003s	sys 0.008s
I20260709 13:17:53.372902 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:53.415208 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:53.333002 15188 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:53.402817 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:17:53.408592 15196 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.685s	user 0.004s	sys 0.039s
W20260709 13:17:53.373637 15219 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.715s	user 0.017s	sys 0.044s
W20260709 13:17:53.331833 15190 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.010s	sys 0.056s
I20260709 13:17:52.951258 15116 consensus_queue.cc:1059] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.470s
I20260709 13:17:53.433655 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:53.434417 10872 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10.
W20260709 13:17:53.414557 15213 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.006s	sys 0.013s
I20260709 13:17:53.414547 15210 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:17:53.266871 15254 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.025s	sys 0.031s
I20260709 13:17:52.943048 15124 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:53.399367 15134 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:52.955670 15231 raft_consensus.cc:3055] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:17:52.957554 15185 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:17:53.401453 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:17:53.438403 15213 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.006s	sys 0.014s
I20260709 13:17:53.403136 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:17:53.448992 15073 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.010s	sys 0.005s
W20260709 13:17:53.444259 15254 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.774s	user 0.030s	sys 0.031s
W20260709 13:17:53.464449 15073 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.010s	sys 0.006s
I20260709 13:17:53.393246 15020 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:53.438803 14943 raft_consensus.cc:3055] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:17:53.432520 15219 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.816s	user 0.018s	sys 0.044s
I20260709 13:17:53.390389 15038 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:53.433405 15190 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.787s	user 0.011s	sys 0.057s
I20260709 13:17:53.423623 15267 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:53.424046 11504 raft_consensus.cc:2393] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 15: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:53.435590 15218 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:53.431137 15188 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:53.446131 15202 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.665s	user 0.008s	sys 0.039s
I20260709 13:17:53.441401 14858 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:53.420801 15054 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:53.423189 15095 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:53.426667 15157 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:53.447782 15185 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:53.447965 15231 raft_consensus.cc:740] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:53.444965 15210 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:52.973428 15055 raft_consensus.cc:2955] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Committing config change with OpId 3.4: config changed from index 2 to 4, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:17:53.465372 15020 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:53.488029 15202 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.008s	sys 0.039s
I20260709 13:17:53.448096 15124 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:53.493678 15185 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:17:53.500166 15280 log.cc:927] Time spent T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:17:53.500914 15231 consensus_queue.cc:260] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:53.501430 15101 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:53.503500 15271 log.cc:927] Time spent T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:17:53.517863 15231 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:53.523602 15274 raft_consensus.cc:3055] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:17:53.523878 15266 log.cc:927] Time spent T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.639s	user 0.000s	sys 0.022s
I20260709 13:17:53.524112 15274 raft_consensus.cc:740] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:17:53.558437 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:53.120121 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:53.560144 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:53.562168 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:53.562767 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:53.563143 10766 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:53.071264 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:53.235195 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:52.971637 10767 leader_election.cc:336] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Timed out: RequestConsensusVote RPC to 127.9.116.130:37895 timed out after 1.715s (SENT)
I20260709 13:17:53.568081 10767 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:53.375957 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:53.024277 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:53.410347 10875 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:17:53.569819 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:53.375859 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:52.975368 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:53.571799 10799 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 12.
I20260709 13:17:53.410063 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:53.573992 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:53.577256 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:53.577548 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:53.233248 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:53.578820 10872 raft_consensus.cc:2393] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 12: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:53.579674 10800 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:53.579921 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:53.580801 10872 raft_consensus.cc:2376] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 13. Re-sending same reply.
I20260709 13:17:53.581796 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:53.582496 10872 raft_consensus.cc:2376] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15. Re-sending same reply.
I20260709 13:17:53.583729 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:53.585067 10870 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:53.585196 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:53.585995 10875 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
I20260709 13:17:53.586561 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:53.587323 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:53.587482 10872 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 9.
I20260709 13:17:53.588152 10875 raft_consensus.cc:2376] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11. Re-sending same reply.
I20260709 13:17:53.588621 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:53.589217 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:53.589404 10872 raft_consensus.cc:2376] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11. Re-sending same reply.
I20260709 13:17:53.398319 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:53.429171 10916 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:17:53.413069 10942 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.014s	sys 0.005s
I20260709 13:17:53.444783 10947 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 14.
I20260709 13:17:53.426035 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:53.391772 15209 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:53.681861 15157 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:52.943513 15179 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:53.468386 14943 raft_consensus.cc:740] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:53.690012 15179 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:53.689479 14943 consensus_queue.cc:260] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:53.690495 15134 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:53.691416 15219 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:53.692293 15219 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:53.693650 15020 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:53.695078 10946 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.009s	sys 0.021s
I20260709 13:17:53.598393 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:53.597631 10867 raft_consensus.cc:1275] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:17:53.691464 10868 raft_consensus.cc:2393] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 11: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:53.591234 10871 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:53.608048 10943 raft_consensus.cc:1240] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 13. Current term is 14. Ops: []
I20260709 13:17:53.685497 10870 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
W20260709 13:17:53.697203 10946 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.009s	sys 0.021s
I20260709 13:17:53.633708 15188 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:53.590682 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:53.627771 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:53.633011 15210 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:53.590621 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:53.517762 15192 consensus_meta.cc:287] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 1.211s	user 0.016s	sys 0.006s
I20260709 13:17:53.683854 14998 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:53.716532 14943 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:53.726018 14998 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:53.720256 10800 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
W20260709 13:17:53.721120 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:53.729476 14998 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:53.732545 15222 consensus_meta.cc:287] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 1.040s	user 0.009s	sys 0.005s
I20260709 13:17:53.485181 15267 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:53.733302 15222 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:53.697499 14782 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:17:53.735865 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 764225us
I20260709 13:17:53.736958 15038 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:53.525023 15274 consensus_queue.cc:260] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:53.738598 10945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2386) took 1614 ms. Trace:
I20260709 13:17:53.635253 14858 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:53.705575 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 142841us
I20260709 13:17:53.695359 15252 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:53.705821 15179 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:53.751847 15252 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:53.753547 10946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 1999) took 1115 ms. Trace:
I20260709 13:17:53.811161 10946 rpcz_store.cc:276] 0709 13:17:52.638531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:52.639064 (+   533us) service_pool.cc:224] Handling call
0709 13:17:52.644507 (+  5443us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:52.644713 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:17:53.041213 (+396500us) log.cc:844] Serialized 14 byte log entry
0709 13:17:53.752886 (+711673us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:53.753009 (+   123us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:53.753168 (+   159us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:53.753283 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:53.753489 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11128,"prepare.queue_time_us":791957,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"spinlock_wait_cycles":482493696,"thread_start_us":1059728,"threads_started":2,"wal-append.queue_time_us":480833}
I20260709 13:17:53.813555 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:53.748764 14998 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:53.816226 14998 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:53.818570 10767 leader_election.cc:336] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Timed out: RequestConsensusVote RPC to 127.9.116.130:37895 timed out after 1.732s (SENT)
I20260709 13:17:53.819054 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:17:53.820459 15283 log.cc:927] Time spent T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.298s	user 0.001s	sys 0.001s
I20260709 13:17:53.747768 15219 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:53.822533 15095 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:53.760643 10873 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:53.769590 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:53.822472 10869 raft_consensus.cc:2410] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:17:53.819757 10767 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:53.717139 15108 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:53.826321 15134 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:53.827051 15192 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:53.827756 10947 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:53.829424 15295 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:53.829949 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 400711us
I20260709 13:17:53.830552 10872 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 12: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:53.830466 15157 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:53.831359 15020 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:17:53.835372 10946 raft_consensus.cc:2393] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 11: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:53.746066 10945 rpcz_store.cc:276] 0709 13:17:52.124292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:52.128405 (+  4113us) service_pool.cc:224] Handling call
0709 13:17:52.928187 (+799782us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:52.946441 (+ 18254us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:53.738078 (+791637us) spinlock_profiling.cc:243] Waited 799 ms on lock 0x7b54001e2650. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:53.738350 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1838619904}
I20260709 13:17:53.819898 15209 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:53.847397 15157 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:53.848065 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:53.848939 15267 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:53.849578 15273 log.cc:927] Time spent T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.346s	user 0.001s	sys 0.001s
I20260709 13:17:53.850054 15157 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:53.842595 15179 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:53.851636 15134 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:53.852824 11508 raft_consensus.cc:1240] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 12 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:17:53.853350 15179 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:53.856031 10872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 1956) took 1045 ms. Trace:
I20260709 13:17:53.837253 14998 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:53.858078 10872 rpcz_store.cc:276] 0709 13:17:52.810350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:53.590179 (+779829us) service_pool.cc:224] Handling call
0709 13:17:53.855956 (+265777us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38272}
I20260709 13:17:53.902271 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:53.968331 10794 raft_consensus.cc:1240] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 13. Current term is 14. Ops: []
W20260709 13:17:53.893038 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 75566.4us
W20260709 13:17:53.918231 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:53.913841 10795 raft_consensus.cc:1240] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 13. Current term is 14. Ops: []
I20260709 13:17:53.892151 10793 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 12.
W20260709 13:17:53.975369 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 131084us
I20260709 13:17:53.912307 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:53.814497 15185 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:53.983292 15185 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:53.892298 15020 raft_consensus.cc:697] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 14 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:53.984750 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:53.914489 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:53.986807 10875 raft_consensus.cc:2393] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 14: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:53.988319 15209 raft_consensus.cc:2749] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:53.922034 10871 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 11.
W20260709 13:17:53.881356 10942 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.831s	user 0.021s	sys 0.020s
I20260709 13:17:53.892822 10874 raft_consensus.cc:1275] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
W20260709 13:17:53.891892 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:53.853981 15134 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:53.898311 14970 consensus_meta.cc:287] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 1.456s	user 0.005s	sys 0.009s
I20260709 13:17:53.911864 15037 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:53.896554 14446 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:53.918583 15017 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:53.899475 15124 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:53.892099 15295 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:53.909044 15054 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:54.017993 10867 raft_consensus.cc:1275] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 13:17:54.018435 10867 pending_rounds.cc:85] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 8
I20260709 13:17:54.016620 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:54.024816 10873 raft_consensus.cc:2410] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:17:54.025257 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 1965) took 1137 ms. Trace:
I20260709 13:17:54.025491 10873 rpcz_store.cc:276] 0709 13:17:52.887905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:53.578768 (+690863us) service_pool.cc:224] Handling call
0709 13:17:54.025208 (+446440us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:54.032231 10800 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:17:53.919390 15192 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:54.038237 10872 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:17:54.021991 15281 log.cc:927] Time spent T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.309s	user 0.002s	sys 0.003s
I20260709 13:17:54.032336 10867 pending_rounds.cc:107] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 13:17:54.043071 10867 raft_consensus.cc:2889] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:54.046518 10873 raft_consensus.cc:1275] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 13:17:54.030365 15182 consensus_meta.cc:287] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 1.242s	user 0.013s	sys 0.002s
I20260709 13:17:54.048332 15162 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:53.970954 15213 raft_consensus.cc:2955] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 12 LEADER]: Committing config change with OpId 10.8: config changed from index 4 to 8, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
W20260709 13:17:54.049312 15158 consensus_meta.cc:287] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 1.472s	user 0.018s	sys 0.006s
W20260709 13:17:54.049779 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:54.059341 10766 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:54.051231 10801 raft_consensus.cc:1275] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 13:17:54.053537 15295 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:54.036147 15037 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:54.056090 10793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 1885) took 1032 ms. Trace:
I20260709 13:17:54.057402 15252 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:54.059285 11478 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:17:54.059600 14970 consensus_queue.cc:1059] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:17:54.062283 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:54.070457 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:54.064977 10801 pending_rounds.cc:85] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 9
I20260709 13:17:54.067797 10793 rpcz_store.cc:276] 0709 13:17:53.023742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:53.023927 (+   185us) service_pool.cc:224] Handling call
0709 13:17:54.055845 (+1031918us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b54000e9054. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:54.055999 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":263061888}
I20260709 13:17:54.071092 10801 pending_rounds.cc:107] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:17:54.071398 10801 raft_consensus.cc:2889] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:54.075927 11477 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:17:54.081336 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 262914us
W20260709 13:17:54.082192 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:54.083145 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:54.066469 15182 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:54.068800 15222 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:54.080345 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:54.092031 10945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 720) took 1187 ms. Trace:
I20260709 13:17:54.077116 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:54.095752 11537 heartbeater.cc:646] Failed to heartbeat to 127.9.116.190:36007 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.9.116.190:36007 timed out after 15.000s (SENT)
I20260709 13:17:54.096732 10950 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 10.
I20260709 13:17:54.093125 10945 rpcz_store.cc:276] 0709 13:17:52.904188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:53.891764 (+987576us) service_pool.cc:224] Handling call
0709 13:17:53.892234 (+   470us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:53.892775 (+   541us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:17:53.892814 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:53.893621 (+   807us) log.cc:844] Serialized 26 byte log entry
0709 13:17:54.046120 (+152499us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:54.046159 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:54.046212 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:54.091810 (+ 45598us) raft_consensus.cc:1685] finished
0709 13:17:54.091830 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:54.091968 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6241,"spinlock_wait_cycles":49447680,"thread_start_us":151948,"threads_started":1,"wal-append.queue_time_us":195550}
W20260709 13:17:54.099125 10869 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 1924) took 2188 ms (client timeout 1855 ms). Trace:
W20260709 13:17:54.099660 10869 rpcz_store.cc:269] 0709 13:17:51.911199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:51.921911 (+ 10712us) service_pool.cc:224] Handling call
0709 13:17:54.098592 (+2176681us) spinlock_profiling.cc:243] Waited 1.85 s on lock 0x7b540009bad4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:54.098771 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4255951744}
I20260709 13:17:54.100346 15162 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:54.103554 15276 log.cc:927] Time spent T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.604s	user 0.000s	sys 0.004s
W20260709 13:17:54.108903 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 50041.4us
I20260709 13:17:54.109125 10949 raft_consensus.cc:1240] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 12. Current term is 13. Ops: []
I20260709 13:17:54.111127 10800 raft_consensus.cc:2393] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 12: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:54.112600 15182 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:54.113348 10948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 719) took 1209 ms. Trace:
I20260709 13:17:54.113811 10948 rpcz_store.cc:276] 0709 13:17:52.903946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:53.899593 (+995647us) service_pool.cc:224] Handling call
0709 13:17:53.900147 (+   554us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:53.900857 (+   710us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:17:53.901216 (+   359us) log.cc:844] Serialized 14 byte log entry
0709 13:17:54.060870 (+159654us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:54.074399 (+ 13529us) log.cc:844] Serialized 26 byte log entry
0709 13:17:54.074508 (+   109us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:17:54.074527 (+    19us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:54.074572 (+    45us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:54.113120 (+ 38548us) raft_consensus.cc:1685] finished
0709 13:17:54.113142 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:54.113278 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":174,"prepare.run_cpu_time_us":193,"prepare.run_wall_time_us":197,"raft.queue_time_us":893,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":5953,"spinlock_wait_cycles":32720256,"thread_start_us":81947,"threads_started":1,"wal-append.queue_time_us":181922}
I20260709 13:17:54.117218 15252 raft_consensus.cc:2749] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:54.118269 15252 consensus_queue.cc:1048] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:17:54.125649 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:54.161072 15231 consensus_meta.cc:287] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.585s	user 0.010s	sys 0.004s
W20260709 13:17:54.163442 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 257200us
W20260709 13:17:54.166296 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:54.160835 14869 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:54.049363 15020 consensus_queue.cc:237] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:54.073975 15017 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:54.068115 15158 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:54.167156 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:17:54.243217 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:54.167254 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:54.180230 11503 raft_consensus.cc:2410] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 13 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
W20260709 13:17:54.179623 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:54.177974 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 100632us
I20260709 13:17:54.248687 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:17:54.255025 11507 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56410 (request call id 736) took 3161 ms (client timeout 1694 ms). Trace:
I20260709 13:17:54.172362 11511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56416 (request call id 749) took 1236 ms. Trace:
I20260709 13:17:54.252444 10872 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 14.
I20260709 13:17:54.180764 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:17:54.256453 11507 rpcz_store.cc:269] 0709 13:17:51.092954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:51.120341 (+ 27387us) service_pool.cc:224] Handling call
0709 13:17:54.254542 (+3134201us) spinlock_profiling.cc:243] Waited 2.04 s on lock 0x7b54005f8a54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:54.254815 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4692983680}
I20260709 13:17:54.258868 11508 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:54.256596 11511 rpcz_store.cc:276] 0709 13:17:52.936083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:53.392668 (+456585us) service_pool.cc:224] Handling call
0709 13:17:53.407624 (+ 14956us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:53.407867 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:17:53.746062 (+338195us) log.cc:844] Serialized 14 byte log entry
0709 13:17:54.063207 (+317145us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:54.069270 (+  6063us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:17:54.072983 (+  3713us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:54.077638 (+  4655us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:54.172116 (+ 94478us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":89126,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":58,"raft.queue_time_us":8287,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":144,"thread_start_us":474254,"threads_started":2,"wal-append.queue_time_us":241483}
I20260709 13:17:54.125728 14446 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:17:54.260080 10943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 2002) took 1620 ms. Trace:
I20260709 13:17:54.262754 10943 rpcz_store.cc:276] 0709 13:17:52.639707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:52.712626 (+ 72919us) service_pool.cc:224] Handling call
0709 13:17:52.764042 (+ 51416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:53.743414 (+979372us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:54.253357 (+509943us) spinlock_profiling.cc:243] Waited 843 ms on lock 0x7b54001e2b54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:54.260016 (+  6659us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1939648256}
I20260709 13:17:54.264240 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:54.264421 15017 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:54.265168 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 238799us
I20260709 13:17:54.265470 10871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 1906) took 1375 ms. Trace:
I20260709 13:17:54.265637 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:54.265964 10871 rpcz_store.cc:276] 0709 13:17:52.884754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:52.906825 (+ 22071us) service_pool.cc:224] Handling call
0709 13:17:54.253003 (+1346178us) spinlock_profiling.cc:243] Waited 662 ms on lock 0x7b54003165d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:54.259867 (+  6864us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1522645888}
I20260709 13:17:54.267128 10916 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:54.267462 10941 raft_consensus.cc:1240] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 11. Current term is 12. Ops: []
I20260709 13:17:54.268323 15157 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:54.207289 10869 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:17:54.275825 10869 raft_consensus.cc:606] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Received request to transfer leadership to TS e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:54.276459 10869 raft_consensus.cc:614] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:17:54.181669 15221 consensus_meta.cc:287] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 1.584s	user 0.016s	sys 0.000s
I20260709 13:17:54.278811 15101 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:54.279255 14446 raft_consensus.cc:697] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 12 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:54.279527 15101 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:54.281008 14446 consensus_queue.cc:237] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:54.257925 15162 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:54.285095 15158 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:54.285913 15252 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:54.286659 15252 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:54.287052 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:54.287987 10943 raft_consensus.cc:2393] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 13: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:54.288937 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:54.289485 15252 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:54.171854 15184 raft_consensus.cc:2804] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:17:54.290282 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:54.290967 15184 raft_consensus.cc:697] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 14 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:54.289250 15262 consensus_queue.cc:1059] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:17:54.291283 15252 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:54.292115 15252 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:54.292974 15184 consensus_queue.cc:237] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:54.294219 15231 raft_consensus.cc:3055] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:54.294896 15231 raft_consensus.cc:740] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:54.295956 10949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 2007) took 1422 ms. Trace:
I20260709 13:17:54.296409 10949 rpcz_store.cc:276] 0709 13:17:52.874010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:52.874850 (+   840us) service_pool.cc:224] Handling call
0709 13:17:52.875215 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:54.109961 (+1234746us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:54.295616 (+185655us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b5400376754. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:54.295890 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2836385664}
I20260709 13:17:54.297427 15295 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:54.297703 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:54.298087 15295 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:54.300777 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:54.300947 15295 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:54.301967 10869 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:17:54.302740 10869 raft_consensus.cc:606] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Received request to transfer leadership to TS e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:54.303323 10870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 1902) took 1419 ms. Trace:
I20260709 13:17:54.303741 10870 rpcz_store.cc:276] 0709 13:17:52.883721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:53.389321 (+505600us) service_pool.cc:224] Handling call
0709 13:17:54.303093 (+913772us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b54002f08d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:54.303234 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":373783424}
I20260709 13:17:54.304476 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:54.304715 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:54.304941 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:54.305279 10794 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 10.
I20260709 13:17:54.305809 10793 raft_consensus.cc:2393] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 15: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:54.306355 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:54.307145 10795 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 10.
I20260709 13:17:54.307363 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:54.308575 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:54.309167 10949 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 12.
I20260709 13:17:54.309216 10801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 557) took 3025 ms. Trace:
I20260709 13:17:54.309540 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:54.309739 10801 rpcz_store.cc:276] 0709 13:17:51.283437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:51.283629 (+   192us) service_pool.cc:224] Handling call
0709 13:17:51.293622 (+  9993us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:54.112332 (+2818710us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:54.308899 (+196567us) spinlock_profiling.cc:243] Waited 2.72 s on lock 0x7b5400386654. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:54.309131 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6261840640}
I20260709 13:17:54.310360 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:54.179543 15192 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:54.313836 10949 raft_consensus.cc:2410] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:17:54.310811 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:54.314911 10794 raft_consensus.cc:2410] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:17:54.311517 10868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 1896) took 2013 ms. Trace:
I20260709 13:17:54.315892 10868 rpcz_store.cc:276] 0709 13:17:52.298201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:52.308136 (+  9935us) service_pool.cc:224] Handling call
0709 13:17:54.311307 (+2003171us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b540006ccd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:54.311458 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2936718592}
I20260709 13:17:54.303231 10869 raft_consensus.cc:614] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:17:54.205689 15287 log.cc:927] Time spent T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.681s	user 0.000s	sys 0.003s
I20260709 13:17:54.320119 10766 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
W20260709 13:17:54.311955 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 51113us
I20260709 13:17:54.296010 15231 consensus_queue.cc:260] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:54.167624 14869 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:54.311533 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:54.313772 10943 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
I20260709 13:17:54.323827 11477 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:54.321739 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:54.302099 10841 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:54.294276 15252 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:54.326655 10947 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 11.
W20260709 13:17:54.186122 15298 log.cc:927] Time spent T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
I20260709 13:17:54.185129 15037 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:54.318923 15073 raft_consensus.cc:3055] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:17:54.325798 11508 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 14.
I20260709 13:17:54.328423 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:54.331362 10949 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 13.
W20260709 13:17:54.327435 15302 log.cc:927] Time spent T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.052s	user 0.003s	sys 0.000s
I20260709 13:17:54.331117 15073 raft_consensus.cc:740] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:17:54.334439 15284 log.cc:927] Time spent T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.815s	user 0.006s	sys 0.000s
W20260709 13:17:54.345183 15270 log.cc:927] Time spent T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.828s	user 0.004s	sys 0.004s
I20260709 13:17:54.351980 15017 raft_consensus.cc:2804] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:54.352578 15017 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:54.352944 15017 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:54.354598 15192 raft_consensus.cc:2749] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:54.327230 15274 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:54.333575 15219 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:54.356518 15219 raft_consensus.cc:697] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:54.357633 15219 consensus_queue.cc:237] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:54.364051 15188 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:54.354734 10947 raft_consensus.cc:1240] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 14. Current term is 15. Ops: []
I20260709 13:17:54.374256 10947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 723) took 1460 ms. Trace:
I20260709 13:17:54.375216 10947 rpcz_store.cc:276] 0709 13:17:52.904879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:54.330033 (+1425154us) service_pool.cc:224] Handling call
0709 13:17:54.354429 (+ 24396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:54.365405 (+ 10976us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:54.365666 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:54.376705 10869 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8949b8b2d08942939f64373bccb71936"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:17:54.378892 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:54.384061 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:54.381165 10869 raft_consensus.cc:606] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Received request to transfer leadership to TS e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:54.364717 15188 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:54.386041 15290 log.cc:927] Time spent T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.859s	user 0.005s	sys 0.002s
I20260709 13:17:54.407078 10945 raft_consensus.cc:1275] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:17:54.386686 15073 consensus_queue.cc:260] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:54.389408 15292 log.cc:927] Time spent T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.863s	user 0.008s	sys 0.000s
I20260709 13:17:54.389601 15209 consensus_queue.cc:1059] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:17:54.390203 15108 consensus_queue.cc:1059] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:17:54.391038 15251 consensus_queue.cc:1059] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:17:54.391135 15055 consensus_queue.cc:1059] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:17:54.392320 15196 consensus_queue.cc:1059] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:17:54.390311 10766 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:54.394537 10869 raft_consensus.cc:614] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:54.392910 15190 consensus_queue.cc:1048] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 13:17:54.396044 10948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 725) took 1478 ms. Trace:
I20260709 13:17:54.393018 14970 consensus_queue.cc:1059] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 13:17:54.423095 11477 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:17:54.417394 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 134167us
W20260709 13:17:54.419284 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 117293us
I20260709 13:17:54.421361 10945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2391) took 1027 ms. Trace:
I20260709 13:17:54.425186 10945 rpcz_store.cc:276] 0709 13:17:53.387887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:54.390370 (+1002483us) service_pool.cc:224] Handling call
0709 13:17:54.394033 (+  3663us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:54.407665 (+ 13632us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:54.415300 (+  7635us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:54.422530 10948 rpcz_store.cc:276] 0709 13:17:52.917188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:54.346988 (+1429800us) service_pool.cc:224] Handling call
0709 13:17:54.374621 (+ 27633us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:54.395690 (+ 21069us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:54.395722 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:54.395762 (+    40us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:17:54.395802 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:54.395871 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:54.395984 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:54.400271 15157 raft_consensus.cc:2749] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:17:54.427270 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 160325us
I20260709 13:17:54.407572 15231 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:54.430936 15101 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:54.433339 10841 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:54.434458 15222 consensus_queue.cc:1059] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:17:54.435660 15289 log.cc:927] Time spent T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.908s	user 0.003s	sys 0.001s
I20260709 13:17:54.435899 15222 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:17:54.436507 15222 raft_consensus.cc:697] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:54.442365 10945 raft_consensus.cc:1240] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 12. Current term is 13. Ops: []
I20260709 13:17:54.448204 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:54.453011 11477 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:54.467034 10944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 2011) took 1568 ms. Trace:
I20260709 13:17:54.451184 15295 raft_consensus.cc:3055] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:17:54.462005 10916 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:54.467304 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:54.483462 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:54.462539 10948 raft_consensus.cc:1275] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:17:54.517235 10944 rpcz_store.cc:276] 0709 13:17:52.898194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:52.905195 (+  7001us) service_pool.cc:224] Handling call
0709 13:17:52.905519 (+   324us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:17:52.934099 (+ 28580us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:52.934147 (+    48us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:17:53.386584 (+452437us) log.cc:844] Serialized 78 byte log entry
0709 13:17:53.608983 (+222399us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:53.609113 (+   130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:53.609328 (+   215us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:54.421580 (+812252us) spinlock_profiling.cc:243] Waited 208 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:17:54.446854 (+ 25274us) raft_consensus.cc:1685] finished
0709 13:17:54.446873 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:54.466858 (+ 19985us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28147,"raft.queue_time_us":37338,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":4588,"spinlock_wait_cycles":477552000,"thread_start_us":184430,"threads_started":1,"wal-append.queue_time_us":117522}
I20260709 13:17:54.465162 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:54.464811 10949 raft_consensus.cc:1240] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 10. Current term is 11. Ops: []
I20260709 13:17:54.460999 15184 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:17:54.469441 10801 raft_consensus.cc:1240] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
W20260709 13:17:54.524827 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 63994.2us
I20260709 13:17:54.527684 15055 raft_consensus.cc:2804] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:54.528481 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:54.523290 11503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56404 (request call id 771) took 1115 ms. Trace:
I20260709 13:17:54.527704 10875 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d00373866c144f4981da73bd7ae39019"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:17:54.530448 10800 raft_consensus.cc:2393] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 12: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
W20260709 13:17:54.522903 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 132119us
I20260709 13:17:54.532027 10949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2388) took 1521 ms. Trace:
I20260709 13:17:54.539065 10949 rpcz_store.cc:276] 0709 13:17:53.011099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:54.376512 (+1365413us) service_pool.cc:224] Handling call
0709 13:17:54.429634 (+ 53122us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:54.531692 (+102058us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:54.531950 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:54.533159 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 71251.3us
W20260709 13:17:54.536711 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:54.536409 10875 raft_consensus.cc:606] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Received request to transfer leadership to TS e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:54.540585 10875 raft_consensus.cc:614] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:54.535315 10950 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
I20260709 13:17:54.535826 11503 rpcz_store.cc:276] 0709 13:17:53.402094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:53.847532 (+445438us) service_pool.cc:224] Handling call
0709 13:17:54.472763 (+625231us) spinlock_profiling.cc:243] Waited 331 ms on lock 0x7b5400527954. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:54.517351 (+ 44588us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":761142016}
I20260709 13:17:54.464540 14701 raft_consensus.cc:2804] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:17:54.550254 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:54.552461 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:54.552858 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:54.467448 15196 raft_consensus.cc:3055] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:17:54.522770 15295 raft_consensus.cc:740] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:54.561601 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:54.535864 15184 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:54.566581 10793 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 11.
I20260709 13:17:54.534499 15124 raft_consensus.cc:2764] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:17:54.534482 15055 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:54.569252 15116 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:54.570973 10945 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 12.
I20260709 13:17:54.571383 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:54.579126 10916 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:54.556177 15251 raft_consensus.cc:3055] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:17:54.555763 15073 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:54.587412 10949 raft_consensus.cc:1240] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 13:17:54.558266 14701 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:54.463475 15222 consensus_queue.cc:237] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:54.589764 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:54.592064 10941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 2009) took 1717 ms. Trace:
I20260709 13:17:54.592907 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:54.595499 15223 raft_consensus.cc:3055] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:17:54.596892 14858 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:54.598362 15252 raft_consensus.cc:3055] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:17:54.599085 10876 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 1899) took 2209 ms (client timeout 1727 ms). Trace:
I20260709 13:17:54.599637 15188 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:54.573740 15184 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:54.601200 10916 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:54.559734 15196 raft_consensus.cc:740] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:54.625988 15196 consensus_queue.cc:260] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:54.592576 10941 rpcz_store.cc:276] 0709 13:17:52.874577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:52.876391 (+  1814us) service_pool.cc:224] Handling call
0709 13:17:52.905363 (+ 28972us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:54.268005 (+1362642us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:54.587494 (+319489us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b54000aafd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:54.591998 (+  4504us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3120166784}
I20260709 13:17:54.628135 10941 raft_consensus.cc:1275] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:17:54.629622 10941 raft_consensus.cc:1275] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 10 index: 4. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
W20260709 13:17:54.605232 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:54.544875 15017 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:54.578948 15055 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:54.604158 10944 raft_consensus.cc:1275] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:17:54.641575 10874 raft_consensus.cc:2410] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:17:54.556623 15192 consensus_queue.cc:1048] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 13:17:54.634512 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:17:54.623986 10876 rpcz_store.cc:269] 0709 13:17:52.341994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:52.414640 (+ 72646us) service_pool.cc:224] Handling call
0709 13:17:54.529527 (+2114887us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b5400340654. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:54.550774 (+ 21247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3527975680}
W20260709 13:17:54.641631 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 53765.5us
I20260709 13:17:54.602545 14701 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:54.634357 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:54.588053 15251 raft_consensus.cc:740] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:54.641796 10944 pending_rounds.cc:85] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 9
I20260709 13:17:54.648231 10793 raft_consensus.cc:1240] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 10. Current term is 11. Ops: []
I20260709 13:17:54.645965 10801 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 10.
I20260709 13:17:54.647843 10841 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:54.652668 15274 raft_consensus.cc:2804] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:17:54.653192 15274 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:54.653585 15274 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:54.650205 10944 pending_rounds.cc:107] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:17:54.658015 10944 raft_consensus.cc:2889] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:54.621493 15223 raft_consensus.cc:740] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:54.658968 10946 raft_consensus.cc:2393] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 14: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:54.660627 10800 raft_consensus.cc:1240] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 LEADER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 11. Current term is 12. Ops: []
I20260709 13:17:54.661087 10795 raft_consensus.cc:2410] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:17:54.623903 15252 raft_consensus.cc:740] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:54.658201 14858 consensus_queue.cc:1059] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.017s
I20260709 13:17:54.668044 15251 consensus_queue.cc:260] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:54.669855 15251 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:17:54.671322 15314 log.cc:927] Time spent T 00000000000000000000000000000000 P ae957c8fc66c4b6ba063d0c74cbf5484: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 13:17:54.675400 10945 raft_consensus.cc:1275] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 13:17:54.675223 15295 consensus_queue.cc:260] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:54.676844 15295 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:54.677960 15325 raft_consensus.cc:3055] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:17:54.678450 15325 raft_consensus.cc:740] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:54.680739 15320 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:54.681540 15320 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:54.684892 15325 consensus_queue.cc:260] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:54.635394 15116 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:54.687911 15320 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:54.688865 10949 raft_consensus.cc:1275] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:17:54.690073 14987 raft_consensus.cc:2804] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:17:54.690773 14987 raft_consensus.cc:697] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 12 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:17:54.691092 15308 log.cc:927] Time spent T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.162s	user 0.003s	sys 0.000s
I20260709 13:17:54.691892 14987 consensus_queue.cc:237] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:54.699446 15017 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:54.707261 10796 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 11.
I20260709 13:17:54.726997 10950 raft_consensus.cc:1275] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:17:54.681464 15209 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:54.727742 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 126595us
I20260709 13:17:54.716883 15017 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:54.688884 15116 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:54.741878 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 410065us
I20260709 13:17:54.647133 10941 raft_consensus.cc:2955] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Committing config change with OpId 10.8: config changed from index 4 to 8, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:17:54.726940 15196 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:54.728433 15108 consensus_queue.cc:1048] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 13:17:54.725898 15251 consensus_queue.cc:1048] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
W20260709 13:17:54.755028 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 231997us
I20260709 13:17:54.745910 15101 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:54.727442 15320 consensus_queue.cc:1059] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:17:54.775990 15157 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:54.793087 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:54.799402 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:54.748070 15323 consensus_queue.cc:1048] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:17:54.750254 15116 consensus_queue.cc:1048] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:17:54.802291 10947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2390) took 1783 ms. Trace:
I20260709 13:17:54.810741 15213 consensus_queue.cc:1059] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:17:54.811100 15201 consensus_queue.cc:1048] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:17:54.812115 15213 consensus_queue.cc:1048] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:17:54.812654 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:54.812971 15201 raft_consensus.cc:3055] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:17:54.813400 15201 raft_consensus.cc:740] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:54.813957 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:54.815527 15328 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:54.816067 15328 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:54.816576 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:54.817925 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:54.818295 15328 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:54.819831 10948 raft_consensus.cc:1275] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:17:54.821228 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:54.821815 10948 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:54.803488 15252 consensus_queue.cc:260] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:54.825881 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:54.826416 15327 log.cc:927] Time spent T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 13:17:54.827395 10942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 1998) took 2189 ms. Trace:
I20260709 13:17:54.803066 15223 consensus_queue.cc:260] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:54.814554 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:17:54.815546 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:54.788169 15209 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:54.801668 15326 consensus_queue.cc:1048] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:17:54.799032 15267 raft_consensus.cc:2804] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:17:54.822530 15329 consensus_queue.cc:1048] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 13:17:54.835781 14970 raft_consensus.cc:3055] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:17:54.823190 15222 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:54.798439 15157 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:54.806880 15101 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:54.844005 11478 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:54.848045 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:54.848747 15262 consensus_queue.cc:1048] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:17:54.852140 15325 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:54.855113 10943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 2012) took 1947 ms. Trace:
I20260709 13:17:54.855819 15055 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:54.856775 15317 raft_consensus.cc:3055] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:17:54.859953 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:54.860440 15158 consensus_queue.cc:1048] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:54.860672 10916 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:54.866878 15219 raft_consensus.cc:3055] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:17:54.861889 15251 consensus_queue.cc:1059] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:54.862504 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:54.863273 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:54.778988 15017 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:54.865911 10948 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 11.
I20260709 13:17:54.838047 15184 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:54.868310 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:54.870579 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:54.878140 10841 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:54.883643 15252 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:54.884253 15037 raft_consensus.cc:2749] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:17:54.887676 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:54.861336 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 50880.5us
I20260709 13:17:54.891615 10949 raft_consensus.cc:2393] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 15: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:54.915632 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:54.916232 10949 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 13.
I20260709 13:17:54.868963 10942 rpcz_store.cc:276] 0709 13:17:52.638148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:52.638270 (+   122us) service_pool.cc:224] Handling call
0709 13:17:52.666989 (+ 28719us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:52.766986 (+ 99997us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:17:54.033394 (+1266408us) log.cc:844] Serialized 14 byte log entry
0709 13:17:54.367604 (+334210us) log.cc:844] Serialized 14 byte log entry
0709 13:17:54.375980 (+  8376us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:54.380044 (+  4064us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:54.393912 (+ 13868us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:54.827075 (+433163us) spinlock_profiling.cc:243] Waited 99.7 ms on lock 0x7b5400016dd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:54.827125 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:54.827300 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":723843,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":3523,"raft.queue_time_us":8506,"raft.run_cpu_time_us":404,"raft.run_wall_time_us":8077,"spinlock_wait_cycles":229335296,"thread_start_us":1296513,"threads_started":2,"wal-append.queue_time_us":199578}
I20260709 13:17:54.918254 15101 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:54.919265 10942 raft_consensus.cc:1240] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 12. Current term is 13. Ops: []
I20260709 13:17:54.920454 10942 raft_consensus.cc:1240] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 12. Current term is 13. Ops: []
I20260709 13:17:54.921455 10942 raft_consensus.cc:1240] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 12. Current term is 13. Ops: []
I20260709 13:17:54.922482 10950 raft_consensus.cc:2393] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 13: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:54.923442 15335 consensus_queue.cc:1048] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:17:54.924615 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 46599.7us
I20260709 13:17:54.925334 10841 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:54.926088 15192 raft_consensus.cc:2749] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:54.926841 15192 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:17:54.881343 15222 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:54.926967 15038 consensus_queue.cc:1048] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:54.928772 14869 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:54.909593 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:54.931159 15267 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:54.931720 15267 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:17:54.935009 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:54.896083 15223 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:54.899072 10874 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d32f5e230e464e61b23a41962708d120"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:17:54.939805 10874 raft_consensus.cc:606] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 LEADER]: Received request to transfer leadership to TS e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:54.941574  9682 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:17:54.888387 15317 raft_consensus.cc:740] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:54.907194 15157 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:54.885474 10943 rpcz_store.cc:276] 0709 13:17:52.908122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:54.335296 (+1427174us) service_pool.cc:224] Handling call
0709 13:17:54.335807 (+   511us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:54.336397 (+   590us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:17:54.421937 (+ 85540us) log.cc:844] Serialized 14 byte log entry
0709 13:17:54.753058 (+331121us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:54.788574 (+ 35516us) log.cc:844] Serialized 26 byte log entry
0709 13:17:54.788773 (+   199us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:17:54.791343 (+  2570us) log.cc:844] Serialized 14 byte log entry
0709 13:17:54.791605 (+   262us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:54.791708 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:54.832489 (+ 40781us) spinlock_profiling.cc:243] Waited 1.01 ms on lock 0x7b18000f0640. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc83f3 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:17:54.839791 (+  7302us) spinlock_profiling.cc:243] Waited 2.26 ms on lock 0x7b48002415d8. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c85ddeaa1 00007f9c85dd5b37 00007f9c85dd59ca 00007f9c85ecd26f 00007f9c85eccf37 00007f9c85ee155c 00007f9c85ee1508 00007f9c85ee1488 00007f9c85ee1443 00007f9c85ee06ec 00007f9c84e0aa96
0709 13:17:54.854887 (+ 15096us) raft_consensus.cc:1685] finished
0709 13:17:54.854910 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:54.855052 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10067,"prepare.queue_time_us":49592,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":88,"raft.queue_time_us":814,"raft.run_cpu_time_us":3900,"raft.run_wall_time_us":514867,"spinlock_wait_cycles":10730880,"thread_start_us":339548,"threads_started":2,"wal-append.queue_time_us":321736}
W20260709 13:17:54.911298 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 50750.7us
I20260709 13:17:54.874035 15201 consensus_queue.cc:260] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:54.877133 14970 raft_consensus.cc:740] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:54.889425 15213 consensus_queue.cc:1048] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:17:54.896574 15219 raft_consensus.cc:740] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:54.912319 15209 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:54.972151 15251 consensus_queue.cc:1048] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:17:54.972853 15222 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:54.973400 15192 raft_consensus.cc:697] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:54.974687 15251 consensus_queue.cc:1059] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:17:54.977895 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:54.975705 15055 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:54.979471 14970 consensus_queue.cc:260] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:54.980120 15192 consensus_queue.cc:237] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:54.981058 14970 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:54.981374 15201 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:17:54.981846 15328 consensus_queue.cc:1175] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" responder_term: 13 status { last_received { term: 12 index: 10 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 12. Current term is 13. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:17:54.982661 10915 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.116.132:40555 timed out after 2.212s (SENT) [suppressed 33 similar messages]
I20260709 13:17:54.983847 10874 raft_consensus.cc:1275] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:17:54.974520 15017 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:55.028584 10792 raft_consensus.cc:1240] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 12. Current term is 13. Ops: []
I20260709 13:17:55.032752 15274 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:55.033282 10767 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:55.037388 15209 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:55.028566 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:55.043840 10767 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:55.044375 10796 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:55.043143 15055 raft_consensus.cc:2749] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:54.984226 15326 raft_consensus.cc:3055] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:17:54.976130 14858 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:55.045773 15326 raft_consensus.cc:740] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:55.046085 14858 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:55.046736 15326 consensus_queue.cc:260] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:54.985723 15184 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:55.048378 14858 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:54.979589 15219 consensus_queue.cc:260] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:55.042866 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 60226.9us
I20260709 13:17:55.049687 15219 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:54.903147 14987 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:17:54.983417 15317 consensus_queue.cc:260] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:55.028106 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:55.052650 10875 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 11.
I20260709 13:17:55.045439 15055 raft_consensus.cc:2764] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:17:55.052995 15317 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:54.988646 14701 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:55.043963 15209 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:55.056725 14701 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:55.001303 10916 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:55.048444 15326 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:17:55.060406 10842 connection.cc:256] RPC call timeout handler was delayed by -2.7200493819998428s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:17:55.065915 10796 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 10.
I20260709 13:17:55.056736 15209 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:55.070461 10767 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:54.910085 10945 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 14.
I20260709 13:17:55.100365 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:55.101114 10796 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:55.101004 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:55.102030 10795 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
W20260709 13:17:55.104318 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:55.110255 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:55.111145 10945 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:17:55.115893 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 137217us
I20260709 13:17:55.117066 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:55.117206 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:55.117450 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:55.118103 10792 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 11.
I20260709 13:17:55.119043 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:55.118120 10797 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 14.
I20260709 13:17:55.119838 10799 raft_consensus.cc:2393] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 13: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:55.125083 10766 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:17:55.127003 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 137223us
I20260709 13:17:55.118425 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:55.129904 11477 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:55.136267 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:55.136936 10801 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:17:55.138159 10875 raft_consensus.cc:1275] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:17:55.129820 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:55.124054 15222 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:55.140803 11477 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:55.144716 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:55.146328 10800 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 14.
I20260709 13:17:55.152621 10793 raft_consensus.cc:1275] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:17:55.126717 14987 raft_consensus.cc:697] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 11 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:55.157452 10876 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 9.
I20260709 13:17:55.154663 10916 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:55.140415 14970 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:17:55.142922 15252 raft_consensus.cc:2749] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:55.167528 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:55.148566 14858 raft_consensus.cc:2804] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:17:55.156656 15184 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:17:55.147887 15158 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:55.152156 10874 raft_consensus.cc:2955] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Committing config change with OpId 10.8: config changed from index 4 to 8, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
W20260709 13:17:55.175133 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 104779us
I20260709 13:17:55.178658 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:55.180850 10943 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:55.182905 10796 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 11.
I20260709 13:17:55.189739 10943 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 13.
I20260709 13:17:55.191672 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:55.194630 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:55.167985 14970 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:55.205838 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:55.206718 10798 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
I20260709 13:17:55.205976 10767 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:55.208587 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:55.155275 15222 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:55.176090 15184 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:55.209142 14970 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:55.239176 14858 raft_consensus.cc:697] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 11 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:55.209851 15038 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:55.256518 11477 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:55.221926 15184 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:55.219563 15335 raft_consensus.cc:2749] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:55.261452 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:55.271931 10767 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:55.247536 15295 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:55.284126 10945 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 13.
I20260709 13:17:55.296929 10766 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:55.301885 11512 raft_consensus.cc:1275] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:17:55.301573 14858 consensus_queue.cc:237] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:55.303078 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:55.304075 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:55.306593 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:55.307512 10871 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 13.
W20260709 13:17:55.309383 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:55.270368 15017 raft_consensus.cc:2804] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:55.315413 11508 raft_consensus.cc:2410] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 15 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 12.
I20260709 13:17:55.316340 10943 raft_consensus.cc:2410] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:17:55.321892 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:55.322574 10870 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:17:55.294257 15101 raft_consensus.cc:2764] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election decision vote started in defunct term 12: won
I20260709 13:17:55.324023 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:55.327194 15158 raft_consensus.cc:697] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 13 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:55.313805 15252 raft_consensus.cc:2804] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:55.335317 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:55.280007 15267 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:55.338976 11503 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10.
I20260709 13:17:55.317641 15219 raft_consensus.cc:2749] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:55.341470 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:55.269948 15038 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:55.347582 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 81220.6us
I20260709 13:17:55.350028 10943 raft_consensus.cc:1275] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:17:55.328501 15017 raft_consensus.cc:697] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 13 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
W20260709 13:17:55.354992 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:55.345943 11507 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 14.
I20260709 13:17:55.357149 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:55.312276 15295 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:55.267694 14987 consensus_queue.cc:237] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:55.371440 10767 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:55.371866 15344 log.cc:927] Time spent T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
W20260709 13:17:55.372480 15343 log.cc:927] Time spent T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:17:55.372675 15158 consensus_queue.cc:237] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:55.374130 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:55.375057 15339 log.cc:927] Time spent T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:17:55.376050 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:55.345777 15252 raft_consensus.cc:697] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:55.349023 15192 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:55.356890 15351 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:55.404846 10870 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 11.
I20260709 13:17:55.406427 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:55.390420 15346 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:55.405985 15219 raft_consensus.cc:2804] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:17:55.423589 15341 log.cc:927] Time spent T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 13:17:55.440563 15346 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:55.444257 15295 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:55.447088 15342 log.cc:927] Time spent T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.193s	user 0.000s	sys 0.001s
I20260709 13:17:55.447263 15252 consensus_queue.cc:237] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:55.448084 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:55.448717 10795 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 11.
I20260709 13:17:55.458881 15351 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:55.461371 15351 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:55.461654 10792 raft_consensus.cc:1275] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:17:55.464287 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:55.465117 10801 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 12.
I20260709 13:17:54.804539 10947 rpcz_store.cc:276] 0709 13:17:53.011757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:54.408579 (+1396822us) service_pool.cc:224] Handling call
0709 13:17:54.451697 (+ 43118us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:54.524084 (+ 72387us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:17:54.559680 (+ 35596us) log.cc:844] Serialized 14 byte log entry
0709 13:17:54.787971 (+228291us) log.cc:844] Serialized 14 byte log entry
0709 13:17:54.788183 (+   212us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:54.788235 (+    52us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:54.794497 (+  6262us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:54.794705 (+   208us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:54.794867 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1699,"prepare.queue_time_us":39613,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":2247,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":9860,"spinlock_wait_cycles":233216,"thread_start_us":156139,"threads_started":1}
W20260709 13:17:55.477028 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 115252us
I20260709 13:17:55.497970 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:55.430076 15222 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:55.436954 15219 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:55.405941 15192 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:55.415863 15196 consensus_queue.cc:1048] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.076s
I20260709 13:17:55.510416 10767 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:55.512814 10801 raft_consensus.cc:1275] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 13:17:55.512887 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:55.514567 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:55.521080 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:55.529250 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:55.530164 10870 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:55.536998 11507 raft_consensus.cc:1240] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:17:55.507155 15219 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:55.545578 10869 raft_consensus.cc:1240] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:17:55.551352 10876 raft_consensus.cc:1240] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 11. Current term is 12. Ops: [9.9-11.10]
I20260709 13:17:55.553480 11512 raft_consensus.cc:1275] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:17:55.554862 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:55.532855 15346 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:55.493238 15017 consensus_queue.cc:237] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:55.555794 10872 raft_consensus.cc:2410] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 14 because replica has last-logged OpId of term: 11 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 4.
W20260709 13:17:55.556291 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:55.557212 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:55.557179 10941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 733) took 1254 ms. Trace:
I20260709 13:17:55.539754 14869 raft_consensus.cc:2749] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:55.562353 10941 rpcz_store.cc:276] 0709 13:17:54.302271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:54.630264 (+327993us) service_pool.cc:224] Handling call
0709 13:17:54.630526 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:54.630738 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:17:54.867956 (+237218us) log.cc:844] Serialized 14 byte log entry
0709 13:17:55.556579 (+688623us) log.cc:844] Serialized 14 byte log entry
0709 13:17:55.556892 (+   313us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:55.556936 (+    44us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:17:55.556980 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:55.557037 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:55.557127 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37205,"raft.queue_time_us":138115,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":14879,"thread_start_us":628855,"threads_started":3,"wal-append.queue_time_us":375625}
I20260709 13:17:55.563522 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:55.572494 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:17:55.573141 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 78725.4us
I20260709 13:17:55.546506 15347 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:55.581660 10870 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 15.
I20260709 13:17:55.558427 15267 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:55.561945 15222 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:55.610893 10767 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:55.616043 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:55.616959 10795 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 10.
I20260709 13:17:55.622486 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:55.623409 10801 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 11.
I20260709 13:17:55.627342 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:55.631852 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:55.633035 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:55.636660 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:55.637326 10870 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:55.637977 11503 raft_consensus.cc:2393] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 14: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:55.637954 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:55.639236 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:55.639974 10796 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
I20260709 13:17:55.643896 14869 raft_consensus.cc:2749] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:17:55.645021 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:17:55.649571 15340 log.cc:927] Time spent T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.486s	user 0.001s	sys 0.001s
I20260709 13:17:55.586874 15347 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:55.653931 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:55.654700 10871 raft_consensus.cc:2393] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 11: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:55.656661 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:17:55.657119 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 64580.3us
I20260709 13:17:55.658069 10797 raft_consensus.cc:1275] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:17:55.664130 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:55.690968 10916 connection.cc:256] RPC call timeout handler was delayed by -1.9960599689998162s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17270000000000002s for a response
I20260709 13:17:55.692269 10916 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:55.702579 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:55.707665 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:55.708307 10873 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:55.710125 10842 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:55.605424 15222 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:55.713255 10798 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 14.
W20260709 13:17:55.699011 15343 log.cc:927] Time spent T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:17:55.641325 15038 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:55.722350 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:55.726539 10870 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 14.
I20260709 13:17:55.727640 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:55.728700 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:55.729082 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:55.729257 10871 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 11.
I20260709 13:17:55.729753 10868 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:55.706423 14970 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:55.733661 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 143392us
I20260709 13:17:55.738169 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:55.784731 10870 raft_consensus.cc:2393] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 14: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:55.744964 10798 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 11.
I20260709 13:17:55.747498 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:17:55.751930 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 94132.1us
I20260709 13:17:55.785813 11504 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 14.
I20260709 13:17:55.761387 15274 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:55.773700 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:55.789721 10949 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:55.774757 10948 raft_consensus.cc:1240] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:55.790400 10942 raft_consensus.cc:1275] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:17:55.775311 15347 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:55.775594 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:55.776803 10947 raft_consensus.cc:1240] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 11. Current term is 12. Ops: []
I20260709 13:17:55.731822 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:17:55.728437 15317 consensus_meta.cc:287] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.673s	user 0.005s	sys 0.008s
I20260709 13:17:55.799772 10873 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
I20260709 13:17:55.800465 10950 raft_consensus.cc:2393] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 14: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:55.788074 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:55.790442 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:55.802704 10943 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 11.
I20260709 13:17:55.733124 15192 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:55.792383 10867 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 11.
I20260709 13:17:55.797783 10801 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
I20260709 13:17:55.777788  9682 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:55.806193 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:55.806818 10795 raft_consensus.cc:2393] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 15: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:55.806272 10945 raft_consensus.cc:2393] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 12: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
W20260709 13:17:55.810386 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 113996us
I20260709 13:17:55.766249 15184 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:55.811733 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:55.812543 10801 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
I20260709 13:17:55.817653 10767 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:55.820261 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:55.820456 10870 raft_consensus.cc:1275] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 13:17:55.825944 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:55.827205 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 135040us
I20260709 13:17:55.830286 10841 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:55.835204 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:55.835893 10948 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
I20260709 13:17:55.816798 15355 raft_consensus.cc:2749] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:17:55.840413 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:55.847620 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:55.848939 10870 raft_consensus.cc:2410] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 16 because replica has last-logged OpId of term: 13 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 5.
I20260709 13:17:55.850697 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:55.851769 10916 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:55.855583 10841 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:55.834887 15361 raft_consensus.cc:2804] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:17:55.861944 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 75788.1us
I20260709 13:17:55.828689 15274 consensus_queue.cc:1059] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:17:55.874616 10874 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 15.
W20260709 13:17:55.878574 10915 leader_election.cc:336] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Timed out: RequestConsensusVote RPC to 127.9.116.132:40555 timed out after 1.555s (SENT)
I20260709 13:17:55.879264 10915 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:55.880955 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:55.834416 15347 consensus_queue.cc:1059] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:17:55.882890 10842 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:55.833447 15192 consensus_queue.cc:1048] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:17:55.890996 10767 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:55.927529 15055 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:17:55.928292 15055 raft_consensus.cc:697] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:17:55.929632 15371 log.cc:927] Time spent T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:17:55.929366 15055 consensus_queue.cc:237] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:55.930629 11508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56410 (request call id 748) took 1116 ms. Trace:
I20260709 13:17:55.931067 11508 rpcz_store.cc:276] 0709 13:17:54.814191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:54.814322 (+   131us) service_pool.cc:224] Handling call
0709 13:17:55.930397 (+1116075us) spinlock_profiling.cc:243] Waited 445 ms on lock 0x7b54004335d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:55.930544 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1024201472}
I20260709 13:17:55.931872 10767 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:55.891860 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:55.899230 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:55.932834 10799 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:55.903386 15274 raft_consensus.cc:3055] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:55.934178 15274 raft_consensus.cc:740] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:55.868129 15295 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:17:55.935139 15274 consensus_queue.cc:260] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:55.936430 15274 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:17:55.937307 15330 log.cc:927] Time spent T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 1.058s	user 0.000s	sys 0.005s
I20260709 13:17:55.923549 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:17:55.937819 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:55.938231 11504 raft_consensus.cc:2410] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:17:55.919445 14970 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:55.924103 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:55.939365 11503 raft_consensus.cc:2393] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 15: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:55.935184 15295 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:55.940450 15295 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:55.937212 10868 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 12.
I20260709 13:17:55.948577 10799 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 14.
I20260709 13:17:55.846835 15378 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:55.967615 10797 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 12.
W20260709 13:17:55.949349 15344 log.cc:927] Time spent T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
W20260709 13:17:55.969323 15338 log.cc:927] Time spent T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:17:55.869172 15361 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:55.977782 10949 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 14.
I20260709 13:17:55.958307 10794 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 14.
W20260709 13:17:55.987614 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:55.967958 15252 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:56.010869 15252 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:56.011402 15252 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:56.011670 15116 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:17:56.011864 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 147870us
I20260709 13:17:56.013507 10842 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:56.014415 15116 raft_consensus.cc:2749] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:56.000340 15361 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:56.020812 10842 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:56.025447 15116 raft_consensus.cc:2764] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:17:56.026430 10767 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:56.028702 15184 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:56.030550 10842 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:56.031452 15116 raft_consensus.cc:2749] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:56.031759 10767 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:56.031976 10842 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:56.046450 15116 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:56.047279 15347 raft_consensus.cc:2749] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:56.032044 15196 raft_consensus.cc:2749] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:56.032878 14970 raft_consensus.cc:2749] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:56.033232 10767 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:56.049419 14970 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:56.052048 15252 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:56.055136 15252 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:56.058422 10767 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:56.064579 10800 raft_consensus.cc:1275] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:17:56.065620 10797 raft_consensus.cc:1275] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:17:56.036448 15219 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:56.039729 15222 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:56.040037 15295 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:56.047097 15116 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:56.076301 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:56.080462 10841 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:56.092883 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 60969.3us
I20260709 13:17:56.094655 15348 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:56.095829 15295 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:56.096196 15219 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:56.080632 15252 raft_consensus.cc:2749] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:17:56.104715 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 261038us
W20260709 13:17:56.105301 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 226767us
I20260709 13:17:56.095908 15124 raft_consensus.cc:2749] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:56.107240 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:56.109002 10797 raft_consensus.cc:1275] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:17:56.095845 15116 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:56.113490 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 114840us
W20260709 13:17:56.115008 10915 leader_election.cc:336] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Timed out: RequestConsensusVote RPC to 127.9.116.132:40555 timed out after 1.790s (SENT)
I20260709 13:17:56.116142 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:56.097572 15222 raft_consensus.cc:2749] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:56.103410 15295 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:56.120095 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:56.105929 15326 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:56.126935 10794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2449) took 1024 ms. Trace:
I20260709 13:17:56.127388 10794 rpcz_store.cc:276] 0709 13:17:55.102180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:55.102375 (+   195us) service_pool.cc:224] Handling call
0709 13:17:56.126699 (+1024324us) spinlock_profiling.cc:243] Waited 686 ms on lock 0x7b54000bc7d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:56.126866 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1576811776}
I20260709 13:17:56.131817 10916 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:56.133169 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:56.131268 11504 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
I20260709 13:17:56.138542 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:56.155159 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:56.130771 15295 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:56.133889 15326 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:56.165081 15382 log.cc:927] Time spent T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.134s	user 0.005s	sys 0.000s
W20260709 13:17:56.168826 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:56.169492 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:56.171913 11478 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:56.172427 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:56.173202 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:56.173566 15184 raft_consensus.cc:2749] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:17:56.173943 11508 raft_consensus.cc:2368] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 13. Current term is 14.
I20260709 13:17:56.176393 15326 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:56.176759 15351 raft_consensus.cc:3055] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:17:56.179200 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:56.173578 10799 raft_consensus.cc:1275] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:17:56.182909 11511 raft_consensus.cc:1240] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 13. Current term is 14. Ops: []
I20260709 13:17:56.184751 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:56.185104 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:56.185832 11507 raft_consensus.cc:2393] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 14: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:17:56.184784 10797 raft_consensus.cc:1275] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:17:56.191663 10795 raft_consensus.cc:2393] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 15: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:56.192395 10842 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:56.194741 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:56.195662 11510 raft_consensus.cc:2410] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 15 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 12.
I20260709 13:17:56.196753 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:56.197983 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:56.177191 15351 raft_consensus.cc:740] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:56.204835 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:56.179613 15222 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:56.210916 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:56.215816 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:56.213363 10873 raft_consensus.cc:1275] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:17:56.216745 10943 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:56.212713 10794 raft_consensus.cc:1240] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 13:17:56.218143 10868 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 11.
I20260709 13:17:56.220749 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:56.220873 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:56.219305 15116 consensus_queue.cc:1059] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 13:17:56.222453 10876 raft_consensus.cc:1275] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:17:56.222708 15359 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:56.226071 15378 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:56.226842 10871 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:56.226708 15378 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:56.227990 10842 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:56.229169 15378 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:56.229656 10842 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:56.231065 15192 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:17:56.231863 15192 raft_consensus.cc:2804] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:56.232266 15192 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:56.232677 15192 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:56.193168 15213 consensus_queue.cc:1048] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.050s
W20260709 13:17:56.233883 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 102166us
I20260709 13:17:56.226418 15359 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:56.234669 10916 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:56.223255 15347 raft_consensus.cc:2749] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:56.223558 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:56.236725 10870 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 11.
I20260709 13:17:56.237131 15359 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:56.238297 15101 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:56.238770 15101 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:56.239187 15101 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:56.238996 15222 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:56.241297 15222 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:56.222990 15356 raft_consensus.cc:2749] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:17:56.243207 10916 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:56.204406 15184 consensus_queue.cc:1048] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:17:56.202898 15351 consensus_queue.cc:260] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:56.234512 10871 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 14.
I20260709 13:17:56.246695 15351 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:17:56.248292 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:56.249605 10942 raft_consensus.cc:1275] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:17:56.223846 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:56.250394 10874 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 13
W20260709 13:17:56.257512 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 44451.1us
I20260709 13:17:56.258756 10766 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:56.250135 10942 pending_rounds.cc:85] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 8
I20260709 13:17:56.267787 10950 raft_consensus.cc:1275] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:17:56.267925 10942 pending_rounds.cc:107] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:17:56.269001 10942 raft_consensus.cc:2889] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:56.258759 10841 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:56.269927 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:17:56.267071 15383 log.cc:927] Time spent T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:17:56.275959 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:56.280315 10943 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
I20260709 13:17:56.282079 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:56.284870 10766 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:56.286892 15326 raft_consensus.cc:2749] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:56.287557 15347 raft_consensus.cc:2804] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:17:56.282418 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:56.288138 15347 raft_consensus.cc:697] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:56.285972 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:56.290580 10766 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:56.232239 15116 raft_consensus.cc:3055] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:17:56.300976 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:56.301369 10874 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 13.
W20260709 13:17:56.305261 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 75673.1us
I20260709 13:17:56.306382 10800 raft_consensus.cc:1275] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:17:56.316396 10766 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:56.317468 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:56.318397 10942 raft_consensus.cc:1275] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:17:56.319303 10944 raft_consensus.cc:1275] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 13:17:56.323137 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 79998.3us
I20260709 13:17:56.323179 15188 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:56.287573 15196 raft_consensus.cc:2764] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:17:56.295892 15349 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:17:56.327872 10916 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:56.334424 10915 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:56.269585 15355 consensus_queue.cc:1048] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 13:17:56.310611 15017 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:56.304768 15116 raft_consensus.cc:740] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:17:56.402076 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 85384.5us
I20260709 13:17:56.289161 15347 consensus_queue.cc:237] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:56.338557 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:17:56.406998 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 72949.7us
I20260709 13:17:56.273026 15055 consensus_queue.cc:1048] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 13:17:56.327296 15378 raft_consensus.cc:2749] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:56.413173 10842 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:56.330531 15326 raft_consensus.cc:2749] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:56.412809 10766 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:56.333550 15346 raft_consensus.cc:2764] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
W20260709 13:17:56.406234 15383 log.cc:927] Time spent T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:17:56.427641 10950 raft_consensus.cc:1240] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 13. Current term is 14. Ops: []
W20260709 13:17:56.428653 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:56.406036 15341 log.cc:927] Time spent T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:17:56.431324 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:56.321867 15328 consensus_queue.cc:1048] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.090s
I20260709 13:17:56.439806 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:56.445497 10950 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:56.412627 15020 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:17:56.444404 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 106878us
I20260709 13:17:56.441574 15116 consensus_queue.cc:260] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:56.448323 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:56.459699 10943 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
I20260709 13:17:56.335719 15201 consensus_queue.cc:1048] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.094s
W20260709 13:17:56.449218 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 125163us
I20260709 13:17:56.463822 11478 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:56.327860 15188 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:56.412477 15359 raft_consensus.cc:2764] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:17:56.338415 15325 consensus_queue.cc:1048] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.102s
I20260709 13:17:56.413825 15349 raft_consensus.cc:697] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 11 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:56.466959 15190 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:56.467954 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 133648us
W20260709 13:17:56.471242 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:56.471920 10916 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:56.467188 15320 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:17:56.414211 15017 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:56.412786 15213 consensus_queue.cc:1048] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 13:17:56.482673 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:56.484437 10915 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:56.445658 15358 raft_consensus.cc:2749] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:17:56.485258 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 72536.6us
I20260709 13:17:56.401916 14987 consensus_queue.cc:1059] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.068s
I20260709 13:17:56.490559 10842 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:56.480337 15295 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:56.493139 15116 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:17:56.528788 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 58383.1us
I20260709 13:17:56.490599 15320 raft_consensus.cc:697] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:56.493729 15398 raft_consensus.cc:2749] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:17:56.543056 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 71244.6us
I20260709 13:17:56.507169 15223 consensus_queue.cc:1048] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:17:56.507916 15251 consensus_queue.cc:1048] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:17:56.489539 15361 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:56.490360 15190 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:56.538026 15222 raft_consensus.cc:2804] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:17:56.550145 15394 log.cc:927] Time spent T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
W20260709 13:17:56.564864 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 87953us
W20260709 13:17:56.560185 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 75262.7us
I20260709 13:17:56.556862 15402 raft_consensus.cc:2749] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:56.561585 15222 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:56.580883 15190 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:56.581103 15348 raft_consensus.cc:2749] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:17:56.583076 15397 log.cc:927] Time spent T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
W20260709 13:17:56.584555 15390 log.cc:927] Time spent T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
W20260709 13:17:56.585407 15396 log.cc:927] Time spent T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 13:17:56.595023 10842 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:56.572228 15388 log.cc:927] Time spent T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 13:17:56.600932 11478 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:56.598243 10916 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:56.597257 15222 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:56.571039 15020 raft_consensus.cc:697] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 13 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:17:56.755663 15101 consensus_meta.cc:287] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.516s	user 0.011s	sys 0.000s
I20260709 13:17:56.599752 15262 consensus_queue.cc:1048] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.142s
W20260709 13:17:56.794939 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 310540us
I20260709 13:17:56.603434 15213 consensus_queue.cc:1048] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.148s
I20260709 13:17:56.607260 15358 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:56.802737 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1917) took 1215 ms. Trace:
I20260709 13:17:56.741724 15251 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:56.798306 10915 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:56.600241 15408 consensus_queue.cc:1048] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.199s
W20260709 13:17:56.835666 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 237531us
I20260709 13:17:56.806685 10869 rpcz_store.cc:276] 0709 13:17:55.574469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:55.575605 (+  1136us) service_pool.cc:224] Handling call
0709 13:17:55.591263 (+ 15658us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:55.630167 (+ 38904us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:17:55.630201 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:55.660073 (+ 29872us) log.cc:844] Serialized 26 byte log entry
0709 13:17:56.065767 (+405694us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b180005d340. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a550d1 00007f9c80e70a0f
0709 13:17:56.331970 (+266203us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:56.332030 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:56.332146 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:56.756671 (+424525us) raft_consensus.cc:1685] finished
0709 13:17:56.756693 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:56.790243 (+ 33550us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26361,"raft.queue_time_us":15277,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":19430,"spinlock_wait_cycles":24437120,"thread_start_us":265988,"threads_started":1,"wal-append.queue_time_us":785459}
I20260709 13:17:56.802464 15020 consensus_queue.cc:237] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:56.849550 15188 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:56.795344 15101 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:56.576298 15349 consensus_queue.cc:237] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:56.852080 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 257094us
I20260709 13:17:56.789729 15325 consensus_queue.cc:1048] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 13:17:56.864327 10842 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:56.785845 15405 consensus_queue.cc:1059] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.258s
I20260709 13:17:56.872716 10942 raft_consensus.cc:1240] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:17:56.609807 15320 consensus_queue.cc:237] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:56.867709 10916 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:56.872087 15407 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:56.881458 10916 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:56.878566 15409 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:17:56.879043 15395 log.cc:927] Time spent T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.314s	user 0.001s	sys 0.000s
I20260709 13:17:56.869820 15358 raft_consensus.cc:2804] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:17:56.871348 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 129942us
I20260709 13:17:56.864061 15158 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:56.835301 15251 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:56.889072 10842 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:56.889899 10916 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:56.866367 15017 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:56.912062 10950 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 16.
I20260709 13:17:56.916320 11478 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:56.880196 15188 raft_consensus.cc:2764] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
W20260709 13:17:56.910945 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 112743us
I20260709 13:17:56.919742 15348 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:17:56.934310 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:56.937659 11478 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:56.877102 15405 consensus_queue.cc:1243] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer 468a8fd46e824f62b9c2c0d3532a6d26 log is divergent from this leader: its last log entry 14.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:17:56.891815 15358 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:56.912061 15361 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:56.945415 10915 leader_election.cc:400] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 election: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 13. Current term is 14.
W20260709 13:17:56.943410 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 53543us
I20260709 13:17:56.932816 15355 raft_consensus.cc:2749] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:17:56.949291 10916 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:56.940554 15409 raft_consensus.cc:697] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 15 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:56.942499 15347 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:17:56.958971 15402 raft_consensus.cc:2804] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:56.959450 15402 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:56.959831 15402 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:56.963002 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:56.964524 15101 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:56.965499 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:56.967362 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:56.955726 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:56.954957 15231 consensus_queue.cc:1048] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 13:17:56.972158 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:56.941124 15413 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:56.978826 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:56.954808 10915 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:56.956877 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 67862us
W20260709 13:17:56.974870 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:56.978125 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:56.946529 15412 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:56.980875 10868 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 15.
I20260709 13:17:56.951427 15213 raft_consensus.cc:2749] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:56.950682 15358 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:56.984628 10870 raft_consensus.cc:1275] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:17:56.984323 10916 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:17:56.987025 15192 consensus_meta.cc:287] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.754s	user 0.005s	sys 0.008s
I20260709 13:17:56.991850 15192 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:56.992969 15412 raft_consensus.cc:697] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 13 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:56.993432 10871 raft_consensus.cc:1240] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 11. Current term is 12. Ops: []
I20260709 13:17:56.994006 15412 consensus_queue.cc:237] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:56.986549 11478 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:56.987677 10874 raft_consensus.cc:1275] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:17:57.000844 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:57.002225 10842 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:56.953881 15348 raft_consensus.cc:697] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 14 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:57.006040 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:57.015769 15349 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:17:57.016453 15349 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:57.016943 15349 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:57.010236 10916 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:57.009922 15262 consensus_queue.cc:1048] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 13:17:57.020309 11478 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:56.995285 15192 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:57.021571 11478 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:17:57.005755 15213 raft_consensus.cc:2749] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:17:57.043810 10842 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:56.983982 15410 consensus_queue.cc:1048] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:17:57.055110 10842 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:57.059883 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 105139us
I20260709 13:17:57.060474 11478 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:17:57.060858 10915 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:57.030943 15101 raft_consensus.cc:2749] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:57.032635 15158 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:17:57.041607 15404 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:17:57.042140 15347 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:57.071671 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:57.071413 10867 raft_consensus.cc:1275] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:17:57.074255 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:57.049029 15192 raft_consensus.cc:2764] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:17:57.076493 10842 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:57.081193 10874 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 12.
I20260709 13:17:57.075627 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:57.084036 10842 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:57.087180 10798 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:57.029048 15413 consensus_queue.cc:1048] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:17:57.069015 15267 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:57.074393 10801 raft_consensus.cc:2955] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Committing config change with OpId 10.8: config changed from index 4 to 8, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:17:57.074286 15213 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:17:57.115293 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:56.904843  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:57.115974 10868 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 10.
W20260709 13:17:57.112042 15416 log.cc:927] Time spent T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:17:57.079389 15361 raft_consensus.cc:2764] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:17:57.086256 15158 raft_consensus.cc:697] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 12 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:57.090976 15295 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:17:57.094276 15251 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:57.152556 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 82128.1us
I20260709 13:17:57.157238 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:57.149442 10798 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 14.
I20260709 13:17:57.093502 15404 raft_consensus.cc:697] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 14 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:57.106904 15347 raft_consensus.cc:697] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:57.126529 15213 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:57.127326 15116 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:57.204246 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:57.107532 15329 consensus_queue.cc:1048] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.069s
I20260709 13:17:57.205819 10799 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 10.
I20260709 13:17:57.153903 15295 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:57.071648 15348 consensus_queue.cc:237] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:57.067329 15409 consensus_queue.cc:237] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:57.203685 15213 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:17:57.239430 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 158597us
I20260709 13:17:57.172024 15262 consensus_queue.cc:1048] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 13:17:57.259473 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:57.220198 15295 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:57.219666 15231 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:57.236287 15101 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:57.269663 10916 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:57.277920 10915 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:57.284977 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:57.288632 11503 raft_consensus.cc:1240] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:17:57.256855 15410 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:57.300923 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:17:57.295567 11477 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.687s	user 0.015s	sys 0.011s
I20260709 13:17:57.307272 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:57.323248 11477 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.714s	user 0.015s	sys 0.011s
W20260709 13:17:57.315124 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 231186us
I20260709 13:17:57.327149 10842 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:57.328694 10842 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:57.325680 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 716779us
I20260709 13:17:57.345252 10798 raft_consensus.cc:1275] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 13:17:57.344887 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 81014.1us
I20260709 13:17:57.293702 15406 raft_consensus.cc:2749] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:57.273964 15399 consensus_queue.cc:1048] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 13:17:57.311324 15326 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:57.275898 15101 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:57.358464 10916 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:57.306967 15251 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:57.267342 15158 consensus_queue.cc:237] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:57.390450 15432 raft_consensus.cc:2749] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:57.311297 15410 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:57.433640 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 133385us
W20260709 13:17:57.445950 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 87587.6us
I20260709 13:17:57.446895 10916 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:57.448246 10916 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
W20260709 13:17:57.453794 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 64971.5us
I20260709 13:17:57.315431 15347 consensus_queue.cc:237] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:57.460137 10801 raft_consensus.cc:1275] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:17:57.509372 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:57.513226 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 65046.8us
I20260709 13:17:57.408746 15326 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:57.306397 15404 consensus_queue.cc:237] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:57.418681 15251 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:57.416077 15406 consensus_queue.cc:1059] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.100s
I20260709 13:17:57.515482 15432 raft_consensus.cc:2804] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:57.538511 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:57.528676 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:57.519711 15361 raft_consensus.cc:2804] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:57.423369 15434 consensus_queue.cc:1048] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.119s
W20260709 13:17:57.571267 15422 log.cc:927] Time spent T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
W20260709 13:17:57.571708 15428 log.cc:927] Time spent T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.218s	user 0.000s	sys 0.001s
I20260709 13:17:57.573846 15400 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:57.575433 15426 log.cc:927] Time spent T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.250s	user 0.000s	sys 0.001s
W20260709 13:17:57.576663 15419 log.cc:927] Time spent T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
W20260709 13:17:57.583762 15425 log.cc:927] Time spent T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.277s	user 0.001s	sys 0.000s
I20260709 13:17:57.562973 15399 raft_consensus.cc:2804] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:57.559437 15432 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:57.533236 14858 consensus_queue.cc:1059] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.029s
W20260709 13:17:57.625448 10800 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.009s	sys 0.007s
W20260709 13:17:57.633049 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 227269us
I20260709 13:17:57.635785 10841 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:57.639276 10841 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:57.560776 15038 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:57.560065 15361 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:57.565790 15406 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:17:57.650661 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 135546us
I20260709 13:17:57.658313 10916 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:57.626009 15399 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:57.685878 14858 raft_consensus.cc:3055] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:17:57.663763 15038 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:57.647316 15432 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:17:57.700704 10870 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.005s	sys 0.014s
I20260709 13:17:57.657683 15361 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:17:57.703032 10870 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.522s	user 0.006s	sys 0.014s
I20260709 13:17:57.657868 15406 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:57.676715 10841 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:57.681282 15351 consensus_meta.cc:287] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 1.434s	user 0.009s	sys 0.006s
W20260709 13:17:57.705646 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 47428.4us
I20260709 13:17:57.714798 10916 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:57.706328 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 377677us
I20260709 13:17:57.684811 15320 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:57.699793 15399 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:57.717617 10916 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:57.700093 14858 raft_consensus.cc:740] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:57.702389 15326 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:57.715972 15427 log.cc:927] Time spent T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.301s	user 0.001s	sys 0.000s
I20260709 13:17:57.709242 15406 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:57.729296 15435 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:17:57.729642 15116 consensus_meta.cc:287] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.522s	user 0.013s	sys 0.002s
I20260709 13:17:57.729811 15435 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:57.730156 15116 raft_consensus.cc:2749] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:57.730299 15435 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:57.728559 15251 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:57.716174 15101 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:57.728600 15464 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:57.627163 15400 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:57.739069 10916 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:57.742861 10916 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:57.743955 15455 raft_consensus.cc:2764] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:17:57.722623 15447 consensus_queue.cc:1048] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 13:17:57.744853 15455 raft_consensus.cc:2804] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:17:57.745074 15442 log.cc:927] Time spent T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.062s	user 0.000s	sys 0.005s
I20260709 13:17:57.745417 15455 raft_consensus.cc:697] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 14 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:17:57.734688 15358 consensus_meta.cc:287] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.740s	user 0.006s	sys 0.005s
I20260709 13:17:57.722764 15412 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:57.716063 15320 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:57.744870 15448 log.cc:927] Time spent T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:17:57.763835 15358 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:57.724462 15351 consensus_queue.cc:1059] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:17:57.742071 15410 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:57.770638 10916 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:57.783320 10916 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:57.759224 15116 raft_consensus.cc:2749] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:57.788121 15467 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:17:57.832835 15267 consensus_meta.cc:287] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.715s	user 0.008s	sys 0.004s
I20260709 13:17:57.832973 15038 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:57.833449 15267 raft_consensus.cc:2749] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:17:57.834959 15407 consensus_meta.cc:287] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.922s	user 0.011s	sys 0.002s
I20260709 13:17:57.835510 15407 raft_consensus.cc:2749] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:57.838474 15320 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:57.747942 15295 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:57.841800 15295 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:57.771111 15447 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:57.778136 15461 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:57.846735 15460 consensus_queue.cc:1048] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:17:57.847242 15101 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:17:57.847662 15351 raft_consensus.cc:3055] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:17:57.848060 15351 raft_consensus.cc:740] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:57.848883 15351 consensus_queue.cc:260] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:57.788837 15449 log.cc:927] Time spent T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 13:17:57.762346 15158 consensus_queue.cc:1059] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.092s
I20260709 13:17:57.850525 15467 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:57.804263 15412 raft_consensus.cc:2804] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:57.746407 15455 consensus_queue.cc:237] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:57.899381 15213 consensus_meta.cc:287] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.643s	user 0.011s	sys 0.000s
W20260709 13:17:57.900566 15402 consensus_meta.cc:287] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.940s	user 0.007s	sys 0.006s
I20260709 13:17:57.901584 15361 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:57.903796 15406 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:57.934531 10792 raft_consensus.cc:1275] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 13:17:57.936822 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 260196us
W20260709 13:17:57.938123 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 243234us
I20260709 13:17:57.913458 15412 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:57.913249 15467 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:57.939872 10841 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
W20260709 13:17:57.943261 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 223540us
W20260709 13:17:57.941649 15222 consensus_meta.cc:287] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 1.190s	user 0.013s	sys 0.000s
W20260709 13:17:57.943154 15349 consensus_meta.cc:287] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.914s	user 0.008s	sys 0.004s
I20260709 13:17:57.956002 10798 raft_consensus.cc:1240] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:17:57.956542 15472 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:57.958056 15358 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:57.941622 10767 leader_election.cc:336] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Timed out: RequestConsensusVote RPC to 127.9.116.130:37895 timed out after 1.667s (SENT)
I20260709 13:17:57.953035 15432 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:57.944036 15412 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:57.992053 10842 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:57.993878 10767 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:17:57.998917 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 148868us
I20260709 13:17:57.952014 15347 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:17:58.002836 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:57.943660 15409 consensus_queue.cc:1048] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 13:17:57.935153 15464 consensus_queue.cc:1048] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:17:57.936379 15213 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:57.936271 15402 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:57.989936 15116 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:57.990648 15320 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:58.011343 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 69759us
I20260709 13:17:58.015228 10948 raft_consensus.cc:1240] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:17:57.990317 15326 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:58.017997 10842 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:58.019798 10842 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:58.007043 15453 raft_consensus.cc:2749] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:17:58.032114 15326 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:58.034709 15326 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:58.036057 10796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 585) took 1067 ms. Trace:
I20260709 13:17:58.038928 15402 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:58.039878 10943 raft_consensus.cc:2393] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 12: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:57.997267 15472 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:58.011447 15447 consensus_queue.cc:1059] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:17:58.042809 15472 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:58.043128 15400 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:58.017792 15116 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:58.046180 15116 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:58.048425 15457 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:58.048846 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:58.049095 15457 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:58.051338 10795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 584) took 1082 ms. Trace:
I20260709 13:17:58.052294 15361 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:58.053987 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:57.996762 15473 consensus_queue.cc:1048] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 13:17:58.022328 15320 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:58.052431 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:57.978679 15222 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:58.051347 10949 raft_consensus.cc:1240] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 13. Current term is 14. Ops: []
I20260709 13:17:58.054770 10799 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:58.036461 10796 rpcz_store.cc:276] 0709 13:17:56.968995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:56.970662 (+  1667us) service_pool.cc:224] Handling call
0709 13:17:56.975348 (+  4686us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:56.983579 (+  8231us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:56.983608 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:56.988668 (+  5060us) log.cc:844] Serialized 26 byte log entry
0709 13:17:57.080209 (+ 91541us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:17:58.035236 (+955027us) spinlock_profiling.cc:243] Waited 4.05 ms on lock 0x7b1800469d80. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c80a5d4bc 00007f9c7ddab48b 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc5455 00007f9c848ad3c7 00007f9c80ebf9a7 00007f9c80ecf0bf
0709 13:17:58.035389 (+   153us) log.cc:844] Serialized 14 byte log entry
0709 13:17:58.035608 (+   219us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:58.035685 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:58.035727 (+    42us) raft_consensus.cc:1685] finished
0709 13:17:58.035820 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:58.035973 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31123,"prepare.queue_time_us":344025,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":3309,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":107,"spinlock_wait_cycles":9320832,"thread_start_us":155729,"threads_started":2,"wal-append.queue_time_us":245028}
I20260709 13:17:57.998745 15349 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:58.078377 11477 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:58.051751 10795 rpcz_store.cc:276] 0709 13:17:56.968626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:56.968753 (+   127us) service_pool.cc:224] Handling call
0709 13:17:56.969050 (+   297us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:56.969244 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:56.969284 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:56.969698 (+   414us) log.cc:844] Serialized 26 byte log entry
0709 13:17:57.168913 (+199215us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:17:57.359131 (+190218us) log.cc:844] Serialized 14 byte log entry
0709 13:17:57.384782 (+ 25651us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:57.384877 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:57.958721 (+573844us) spinlock_profiling.cc:243] Waited 37.6 ms on lock 0x7b24001384b8. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dc4e9c4 00007f9c7dc4cacb 00007f9c7de07dc9 00007f9c7de07a55 00007f9c7de06fb0 00007f9c80a7d488 00007f9c80a7dbe8 00007f9c80a55398 00007f9c80a5076a 00007f9c80a4fbb6
0709 13:17:58.051044 (+ 92323us) raft_consensus.cc:1685] finished
0709 13:17:58.051145 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:58.051279 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7732,"prepare.queue_time_us":348684,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":200,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":6645,"spinlock_wait_cycles":86457216,"thread_start_us":330754,"threads_started":2,"wal-append.queue_time_us":383740}
I20260709 13:17:58.018201 15435 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:58.081804 15471 log.cc:927] Time spent T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.123s	user 0.005s	sys 0.000s
W20260709 13:17:58.092449 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 72722.1us
I20260709 13:17:58.089892 10795 raft_consensus.cc:1275] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:17:58.132913 10795 raft_consensus.cc:1240] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 13:17:58.072115 15347 raft_consensus.cc:697] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:17:58.128952 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 44830.6us
I20260709 13:17:58.076624 15358 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:58.137630 10871 raft_consensus.cc:1275] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:17:58.138422 10945 raft_consensus.cc:1240] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 15. Current term is 16. Ops: []
I20260709 13:17:58.144475 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:58.145648 10949 raft_consensus.cc:2410] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:17:58.149304 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:58.142009 15358 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:58.151105 15358 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:58.089282 15407 raft_consensus.cc:2749] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:58.096138 15460 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:58.152269 15349 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:58.152144 15460 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:58.153028 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:58.154277 15460 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:58.091889 15231 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:58.090692 15480 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:58.155050 15231 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:58.155462 10871 raft_consensus.cc:1275] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 13:17:58.155051 15457 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:58.156391 15413 consensus_queue.cc:1059] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:17:58.157215 15413 raft_consensus.cc:3055] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:17:58.157516 15413 raft_consensus.cc:740] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:17:58.100206 15435 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:58.158907 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:58.159690 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:58.161413 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:58.155325 15480 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:58.162142 15480 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:58.158277 15413 consensus_queue.cc:260] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:58.146425 15403 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:58.144021 15400 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:58.094318 15192 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:17:58.183321 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:58.138106 15326 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:58.177220 10875 raft_consensus.cc:1275] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:17:58.185014 10801 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 14.
I20260709 13:17:58.186128 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:58.177237 10868 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 9.
I20260709 13:17:58.182943 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:58.097851 15320 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:58.176983 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:58.189869 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:58.177645 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:58.190132 10869 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 13.
W20260709 13:17:58.194995 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:58.195456 15406 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:58.196657 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:58.197985 10916 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:58.188391 10871 raft_consensus.cc:1275] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 13:17:58.198998 10946 raft_consensus.cc:2410] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 14 because replica has last-logged OpId of term: 13 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 5.
I20260709 13:17:58.199401 10766 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:58.202843 10867 raft_consensus.cc:1240] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 11. Current term is 12. Ops: [9.7-11.8]
I20260709 13:17:58.089421 15478 consensus_queue.cc:1048] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.108s
I20260709 13:17:58.184628 15252 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:58.205283 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:58.197502 15400 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:58.207003 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:58.207787 10799 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 12.
I20260709 13:17:58.210481 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:58.214558 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:58.196682 15402 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:58.185921 15349 consensus_queue.cc:1048] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:17:58.248224 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:58.196615 15192 raft_consensus.cc:697] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:58.206028 15413 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:58.213785 15124 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:17:58.206643 15320 raft_consensus.cc:2749] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:58.204010 15406 consensus_queue.cc:1059] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:17:58.214186 15252 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:58.185792 15403 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:58.235262 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:58.282706 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:58.283561 10949 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 14.
I20260709 13:17:58.231263 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:58.234637 10867 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 13.
I20260709 13:17:58.285063 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:58.285748 10867 raft_consensus.cc:2376] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 13. Re-sending same reply.
I20260709 13:17:58.235066 15432 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:58.222138 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:58.252245 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:58.243286 11511 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
I20260709 13:17:58.223120 10916 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:58.189750 15326 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:58.291359 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 68280.9us
I20260709 13:17:58.291852 10949 raft_consensus.cc:1240] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:17:58.260092 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:58.259423 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:58.253685 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:58.296700 10870 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:58.235381 15213 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:58.228559 15222 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:58.240263 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:58.290511 10947 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10.
I20260709 13:17:58.288825 10868 raft_consensus.cc:1275] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:17:58.267398 15124 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:58.301935 10801 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 10.
I20260709 13:17:58.300101 10869 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
I20260709 13:17:58.304742 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:58.306454 10867 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 13.
I20260709 13:17:58.307674 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:58.308427 10867 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
I20260709 13:17:58.309507 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:58.310999 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:58.312564 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:58.314601 10867 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
I20260709 13:17:58.314860 15358 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:58.315936 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:58.317009 10867 raft_consensus.cc:2393] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 12: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:58.318148 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:58.321213 10796 raft_consensus.cc:1275] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:17:58.323334 15432 consensus_queue.cc:1048] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:17:58.325752 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:58.324826 10867 raft_consensus.cc:1275] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:17:58.226737 15231 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:58.250402 15402 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:58.226997 15399 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:58.335088 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 115501us
I20260709 13:17:58.336056 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:58.338152 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:58.240249 14858 consensus_queue.cc:260] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:58.343942 14858 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:58.345614 15252 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:58.301402 15406 raft_consensus.cc:2804] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:17:58.319057 15124 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:58.375634 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:58.379870 10796 raft_consensus.cc:1275] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:17:58.381768 10796 raft_consensus.cc:1240] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:17:58.346292 15412 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:58.260558 15347 consensus_queue.cc:237] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:58.676738 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:58.722342 10867 raft_consensus.cc:2393] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 10: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:58.384835 10945 raft_consensus.cc:1275] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:17:58.389252 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:58.436329 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:58.745337 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:58.684075 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:58.593864 10870 raft_consensus.cc:2410] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:17:58.455281 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:58.756867 10869 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
I20260709 13:17:58.747998 10766 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:17:58.767323 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:58.769824 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:58.420501 15488 log.cc:927] Time spent T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
W20260709 13:17:58.458048 15490 log.cc:927] Time spent T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
W20260709 13:17:58.419601 15487 log.cc:927] Time spent T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.199s	user 0.000s	sys 0.001s
W20260709 13:17:58.348635 15213 consensus_queue.cc:1175] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "cb0428cc55984daba1ba63744a5102a2" responder_term: 13 status { last_received { term: 10 index: 8 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 12. Current term is 13. Ops: [12.9-12.9]" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:17:58.683987 15480 consensus_meta.cc:287] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.521s	user 0.009s	sys 0.000s
I20260709 13:17:58.792794 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:58.797938 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:58.790719 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 373826us
I20260709 13:17:58.677301 15446 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:58.679565 15402 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:58.680191 15358 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:58.806090 15475 log.cc:927] Time spent T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.301s	user 0.000s	sys 0.000s
I20260709 13:17:58.760025 15399 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:58.448734 15403 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:58.389513 15406 raft_consensus.cc:697] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 12 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:17:58.814703 10950 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.000s	sys 0.022s
W20260709 13:17:58.842255 10950 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.001s	sys 0.022s
I20260709 13:17:58.769814 15116 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:58.720827 15346 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:58.793308 15435 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:58.419914 15467 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:58.384023 15192 consensus_queue.cc:237] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:58.852984 10796 raft_consensus.cc:1240] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 9. Current term is 10. Ops: []
W20260709 13:17:58.849529 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 123801us
I20260709 13:17:58.860778 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:58.862360 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:58.867540 10796 raft_consensus.cc:2393] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 12: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:58.839959 15447 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:17:58.869336 11478 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.022s	sys 0.051s
W20260709 13:17:58.871145 11478 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.023s	sys 0.051s
I20260709 13:17:58.845222 15231 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:58.849089 15473 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:58.873097 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:58.871519 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 523039us
I20260709 13:17:58.874742 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:58.849357 15358 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:58.821101 15505 consensus_queue.cc:1059] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:17:58.848309 15403 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:58.878258 10876 raft_consensus.cc:2368] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 13. Current term is 14.
I20260709 13:17:58.885715 10794 raft_consensus.cc:2393] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 12: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:58.886118 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:58.886986 10796 raft_consensus.cc:2393] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 14: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:58.887019 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:58.887959 10794 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
I20260709 13:17:58.889055 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:58.897153 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:58.903550 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:58.904207 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:58.904264 11512 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10.
I20260709 13:17:58.827958 15480 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:58.906679 10794 raft_consensus.cc:1275] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:17:58.890645 14970 raft_consensus.cc:2804] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:17:58.872793 15326 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:58.846288 15446 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:58.891093 15358 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:58.925777 11510 raft_consensus.cc:1240] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:17:58.925648 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:58.926394 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:58.927929 15513 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:58.928156 15467 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:58.921226 15501 consensus_queue.cc:1048] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:17:58.877496 15447 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:58.935298 15438 log.cc:927] Time spent T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 1.396s	user 0.000s	sys 0.013s
I20260709 13:17:58.854674 15346 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:58.863361 15435 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:58.861392 15116 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:58.948736 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:58.948287 14970 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:58.922355 15403 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:58.882974 15473 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:59.014777 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 266792us
W20260709 13:17:58.996910 14858 consensus_meta.cc:287] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.652s	user 0.010s	sys 0.002s
I20260709 13:17:58.988664 14970 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:58.989012 15467 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:59.070052 15467 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:59.031816 15349 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:59.071362 15349 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:59.073643 15349 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:59.039572 10766 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:58.962661 15406 consensus_queue.cc:237] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:59.068034 15403 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:59.027040 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 150820us
I20260709 13:17:58.979207 15513 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:58.985172 15509 consensus_queue.cc:1048] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.115s
I20260709 13:17:59.124132 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:59.125130 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:59.088929 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 49470.7us
W20260709 13:17:59.068511 15124 consensus_meta.cc:287] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.523s	user 0.009s	sys 0.000s
I20260709 13:17:59.126466 10766 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:59.127074 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:17:59.128346 10950 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:59.125921 10794 raft_consensus.cc:2393] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 12: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:59.033197 15412 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:59.131750 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:59.137758 10796 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:59.116425 15335 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:59.023310 15326 raft_consensus.cc:3055] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:17:59.122598 15447 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:59.125411 10767 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:17:59.154604 10767 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:59.020803 15510 consensus_queue.cc:1048] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.118s
I20260709 13:17:59.155987 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:59.160574 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:59.161399 10946 raft_consensus.cc:2393] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 16: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:59.162822 15447 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:17:59.163288 15498 log.cc:927] Time spent T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 13:17:59.163637 15447 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:59.166186 15447 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:59.166342 15513 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:59.130443 15124 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:59.169078 15124 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:59.172976 10795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1921) took 1064 ms. Trace:
I20260709 13:17:59.173463 10795 rpcz_store.cc:276] 0709 13:17:58.109106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:58.133783 (+ 24677us) service_pool.cc:224] Handling call
0709 13:17:58.137004 (+  3221us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:58.178580 (+ 41576us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:58.178610 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:58.185903 (+  7293us) log.cc:844] Serialized 52 byte log entry
0709 13:17:58.348319 (+162416us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:58.348360 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:58.348431 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:59.172695 (+824264us) raft_consensus.cc:1685] finished
0709 13:17:59.172719 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:59.172903 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42191,"raft.queue_time_us":43887,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":13272,"thread_start_us":134922,"threads_started":1,"wal-append.queue_time_us":927082}
I20260709 13:17:59.175446 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:59.142975 15326 raft_consensus.cc:740] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:17:59.176649 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:59.176590 15326 consensus_queue.cc:260] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:59.177630 15326 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:59.177845 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:59.181703 10949 raft_consensus.cc:2393] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 15: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:17:59.186041 15467 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:59.142606 15335 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:59.153138 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:17:59.142436 15413 consensus_meta.cc:287] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.869s	user 0.010s	sys 0.001s
I20260709 13:17:59.190659 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:17:59.192186 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:59.192231 10794 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:59.197088 10767 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:59.199368 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:59.128263 15349 consensus_queue.cc:1048] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:17:59.132054 15403 consensus_queue.cc:1048] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 13:17:59.132002 15399 consensus_queue.cc:1059] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 13:17:59.204691 10950 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 14.
I20260709 13:17:59.212700 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:59.226091 10841 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:17:59.227777 10794 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 10.
I20260709 13:17:59.188951 15446 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:59.198226 15335 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:59.237895 10946 raft_consensus.cc:1275] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:17:59.240224 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:59.244663 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:17:59.257380 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:59.258587 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:59.257513 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:59.301103 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:59.304435 10796 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 12.
I20260709 13:17:59.223235 15473 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:17:59.228864 15378 raft_consensus.cc:2804] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:59.232933 15467 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:17:59.327764 11508 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 14.
I20260709 13:17:59.317118 15509 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:17:59.331594 15509 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:59.331982 15509 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:59.317199 15317 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:59.335568 15317 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:59.335391 14970 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:59.335999 15317 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:59.337684 14970 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:59.299841 15446 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:59.363960 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:59.364929 10949 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 11.
W20260709 13:17:59.367718 15414 log.cc:927] Time spent T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 2.437s	user 0.011s	sys 0.028s
I20260709 13:17:59.307462 15412 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:59.307803 15399 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:59.380153 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:59.328452 15378 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:17:59.350270 15518 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:59.350692 14970 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:59.372605 15467 raft_consensus.cc:697] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:59.383194 15511 raft_consensus.cc:2749] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:59.383476 15358 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:59.376664 15446 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:59.389981 15378 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:59.466377 10842 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:59.505338 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 377924us
I20260709 13:17:59.506920 10842 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:59.399253 15473 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:59.385433 15399 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:59.509150 10842 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:59.383391 15412 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:59.510421 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:59.512387 10795 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:59.517091 10842 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:59.441910 15480 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:59.519423 10941 raft_consensus.cc:3055] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:17:59.466377 15328 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:59.522277 10941 raft_consensus.cc:740] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:59.402694 15515 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:59.397336 15435 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:59.517686 15192 raft_consensus.cc:2749] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:59.523069 15328 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:59.529814 10872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1918) took 3929 ms. Trace:
I20260709 13:17:59.542593 10941 consensus_queue.cc:260] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:59.532298 15446 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:59.533746 15405 raft_consensus.cc:3055] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:17:59.534986 10876 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 1990) took 1375 ms (client timeout 1667 ms). Trace:
W20260709 13:17:59.589419 10876 rpcz_store.cc:269] 0709 13:17:58.159156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:58.159439 (+   283us) service_pool.cc:224] Handling call
0709 13:17:59.534739 (+1375300us) spinlock_profiling.cc:243] Waited 699 ms on lock 0x7b54002d4954. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:59.534913 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1607448320}
I20260709 13:17:59.590739 10871 raft_consensus.cc:1240] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:17:59.400591 15346 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:59.511080 15519 raft_consensus.cc:2749] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:59.443728 15320 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
W20260709 13:17:59.543916 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 346920us
I20260709 13:17:59.402052 14970 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:59.400233 15518 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:59.554680 10872 rpcz_store.cc:276] 0709 13:17:55.600657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:55.600872 (+   215us) service_pool.cc:224] Handling call
0709 13:17:55.620522 (+ 19650us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:55.653311 (+ 32789us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:55.653358 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:55.693228 (+ 39870us) log.cc:844] Serialized 26 byte log entry
0709 13:17:56.053774 (+360546us) spinlock_profiling.cc:243] Waited 3.53 ms on lock 0x7b180019d620. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7dd962a6 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9
0709 13:17:56.055405 (+  1631us) spinlock_profiling.cc:243] Waited 7.64 ms on lock 0x7b180019d620. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a550d1 00007f9c80e70a0f
0709 13:17:56.160731 (+105326us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:56.161105 (+   374us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:56.161257 (+   152us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:59.525522 (+3364265us) spinlock_profiling.cc:243] Waited 49 ms on lock 0x7b2400124028. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dc4e9c4 00007f9c7dc4cacb 00007f9c7de07dc9 00007f9c7de07a55 00007f9c7de06fb0 00007f9c80a7d488 00007f9c80a7dbe8 00007f9c80a55398 00007f9c80a5076a 00007f9c80a4fbb6
0709 13:17:59.529499 (+  3977us) raft_consensus.cc:1685] finished
0709 13:17:59.529524 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:59.529695 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":44335,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":14408,"spinlock_wait_cycles":138323456,"thread_start_us":105140,"threads_started":1,"wal-append.queue_time_us":596270}
I20260709 13:17:59.602116 10867 raft_consensus.cc:1275] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 13:17:59.558525 15480 consensus_queue.cc:1048] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:59.559774 15513 raft_consensus.cc:2749] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:59.559479 15429 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:59.565563 15317 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:59.567140 15517 log.cc:927] Time spent T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:17:59.569023 15399 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:59.569199 15508 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:59.589243 15515 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:59.574235 15328 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:59.511219 15358 raft_consensus.cc:2749] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:59.542223 15467 consensus_queue.cc:237] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:59.584178 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:59.515870 15473 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:59.589221 15405 raft_consensus.cc:740] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:59.624795 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:59.626430 10800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 587) took 2656 ms. Trace:
I20260709 13:17:59.615859 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:59.635203 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:17:59.635677 10875 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:59.628427 10948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 762) took 2163 ms. Trace:
I20260709 13:17:59.636917 10948 rpcz_store.cc:276] 0709 13:17:57.464705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:57.522394 (+ 57689us) service_pool.cc:224] Handling call
0709 13:17:57.530579 (+  8185us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:58.015857 (+485278us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:59.628178 (+1612321us) spinlock_profiling.cc:243] Waited 484 ms on lock 0x7b54000ab9d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:17:59.628365 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1113687424}
I20260709 13:17:59.587889 10870 raft_consensus.cc:1240] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:17:59.588559 10795 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 14.
I20260709 13:17:59.594174 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:59.631693 10800 rpcz_store.cc:276] 0709 13:17:56.969594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:56.972745 (+  3151us) service_pool.cc:224] Handling call
0709 13:17:56.973087 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:56.973517 (+   430us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:56.973552 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:56.974053 (+   501us) log.cc:844] Serialized 26 byte log entry
0709 13:17:57.080209 (+106156us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:17:57.626956 (+546747us) log.cc:844] Serialized 14 byte log entry
0709 13:17:57.781169 (+154213us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:57.781270 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:59.624699 (+1843429us) spinlock_profiling.cc:243] Waited 4.74 ms on lock 0x7b24001384b8. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dc4e9c4 00007f9c7dc4cacb 00007f9c7de07dc9 00007f9c7de07a55 00007f9c7de06fb0 00007f9c80a7d488 00007f9c80a7dbe8 00007f9c80a55398 00007f9c80a5076a 00007f9c80a4fbb6
0709 13:17:59.626138 (+  1439us) raft_consensus.cc:1685] finished
0709 13:17:59.626229 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:59.626369 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52770,"prepare.queue_time_us":326045,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":57,"raft.queue_time_us":97914,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":10860,"spinlock_wait_cycles":10909440,"thread_start_us":777903,"threads_started":3,"wal-append.queue_time_us":131589}
I20260709 13:17:59.640030 10876 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 12.
I20260709 13:17:59.652737 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:59.596244 10767 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:59.557658 10941 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:59.543056 14858 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:59.662099 10871 raft_consensus.cc:2393] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 12: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:17:59.663117 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:59.663558 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:59.664132 10800 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:59.664404 10795 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 10.
I20260709 13:17:59.669410 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:17:59.669534 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:59.670032 10871 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:59.670238 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:59.670583 10870 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 14.
I20260709 13:17:59.671061 10795 raft_consensus.cc:2393] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 11: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:17:59.608989 15510 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:59.608620 14446 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:59.673166 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:59.613245 15412 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:59.628948 14970 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:59.594774 15335 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:59.617686 15518 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:17:59.704674 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 108518us
I20260709 13:17:59.675212 15378 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:59.674211 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:59.688539 15492 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:59.691478 15473 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:59.735338 11506 raft_consensus.cc:1240] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 11. Current term is 12. Ops: []
I20260709 13:17:59.602444 15320 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:59.691468 10801 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39442 (request call id 599) took 1504 ms (client timeout 1938 ms). Trace:
W20260709 13:17:59.744081 10801 rpcz_store.cc:269] 0709 13:17:58.187470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:58.196405 (+  8935us) service_pool.cc:224] Handling call
0709 13:17:59.691232 (+1494827us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b5400387a54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:59.691366 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":239518976}
I20260709 13:17:59.696398 10870 raft_consensus.cc:1240] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:59.747074 10870 raft_consensus.cc:1240] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:17:59.749914 10870 raft_consensus.cc:1275] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:17:59.674867 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:17:59.698604 10941 raft_consensus.cc:1275] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 13:17:59.701514 10796 raft_consensus.cc:1240] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 14. Current term is 15. Ops: []
I20260709 13:17:59.668677 14858 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:59.706189 10767 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:17:59.755465 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 49377.2us
I20260709 13:17:59.757472 10767 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:59.711310 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:17:59.712332 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 198903us
I20260709 13:17:59.693073 10798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 598) took 1510 ms. Trace:
I20260709 13:17:59.766078 10842 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:59.768018 10842 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:59.768553 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:59.742286 10871 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
I20260709 13:17:59.753764 15335 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:59.754446 10941 pending_rounds.cc:85] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 10
I20260709 13:17:59.770972 10941 pending_rounds.cc:107] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:17:59.771348 10941 raft_consensus.cc:2889] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:59.773689 10842 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:59.702610 15515 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:59.777822 11508 raft_consensus.cc:1275] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:17:59.784988 10767 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:17:59.765508 10798 rpcz_store.cc:276] 0709 13:17:58.182912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:58.228014 (+ 45102us) service_pool.cc:224] Handling call
0709 13:17:58.228330 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:58.228525 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:17:58.238903 (+ 10378us) log.cc:844] Serialized 14 byte log entry
0709 13:17:58.330977 (+ 92074us) log.cc:844] Serialized 14 byte log entry
0709 13:17:59.692623 (+1361646us) spinlock_profiling.cc:243] Waited 19.9 ms on lock 0x7b180030a5a0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c80a5d4bc 00007f9c7ddab48b 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc5455 00007f9c80ead060 00007f9c80ebf716 00007f9c80ed0287
0709 13:17:59.692731 (+   108us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:59.692786 (+    55us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:17:59.692830 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:59.692899 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:59.693016 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":83305,"prepare.queue_time_us":12040,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":88,"raft.queue_time_us":40046,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":12984,"spinlock_wait_cycles":45834496,"thread_start_us":129907,"threads_started":2,"wal-append.queue_time_us":86427,"wal-append.run_cpu_time_us":2447,"wal-append.run_wall_time_us":1104763}
I20260709 13:17:59.780601 10794 raft_consensus.cc:1240] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 10. Current term is 11. Ops: []
I20260709 13:17:59.780637 10941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2417) took 3374 ms. Trace:
I20260709 13:17:59.629173 15429 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:59.734503 14970 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:59.788028 10941 rpcz_store.cc:276] 0709 13:17:56.406377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:56.406498 (+   121us) service_pool.cc:224] Handling call
0709 13:17:56.430885 (+ 24387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:59.519313 (+3088428us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b5400316120. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80e2cdc0 00007f9c80eb71c0 00007f9c80eb8231 00007f9c80eb4d8f 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c
0709 13:17:59.780282 (+260969us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:59.780565 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":343061760}
I20260709 13:17:59.765027 15518 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:59.789965 10842 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:59.790199 10800 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
W20260709 13:17:59.788040 15326 consensus_meta.cc:287] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.610s	user 0.012s	sys 0.001s
I20260709 13:17:59.791508 10767 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:59.804095 10799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1922) took 1663 ms. Trace:
I20260709 13:17:59.777269 15410 raft_consensus.cc:3055] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:17:59.778018 15513 raft_consensus.cc:2749] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:17:59.661695 15508 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:59.694768 14446 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:59.780098 15358 raft_consensus.cc:2749] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:17:59.695096 15510 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:59.783170 15359 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 13:17:59.795169 15223 raft_consensus.cc:2749] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:17:59.792668 10875 raft_consensus.cc:2410] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:17:59.783099 15539 raft_consensus.cc:2749] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:17:59.743534 15492 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:59.810628 10795 raft_consensus.cc:2410] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:17:59.781417 15541 raft_consensus.cc:2749] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:59.794893 14970 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:59.801292 14858 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:17:59.826445 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 91544.9us
I20260709 13:17:59.804538 10799 rpcz_store.cc:276] 0709 13:17:58.140778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:58.223528 (+ 82750us) service_pool.cc:224] Handling call
0709 13:17:58.281582 (+ 58054us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:17:58.767565 (+485983us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:58.767634 (+    69us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:17:58.805309 (+ 37675us) log.cc:844] Serialized 78 byte log entry
0709 13:17:59.028243 (+222934us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:59.440720 (+412477us) log.cc:844] Serialized 14 byte log entry
0709 13:17:59.443325 (+  2605us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:59.443431 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:59.803869 (+360438us) raft_consensus.cc:1685] finished
0709 13:17:59.803889 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:59.804030 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2309,"prepare.queue_time_us":527398,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":108628,"raft.run_cpu_time_us":322,"raft.run_wall_time_us":32717,"thread_start_us":956406,"threads_started":3,"wal-append.queue_time_us":557652}
I20260709 13:17:59.794952 15529 raft_consensus.cc:2749] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:17:59.810078 15020 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:17:59.822778 14970 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:59.824784 15508 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:59.807099 15410 raft_consensus.cc:740] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:17:59.810017 15499 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:59.839035 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:17:59.840174 11506 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:59.742694 15405 consensus_queue.cc:260] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:59.808627 15518 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:59.850163 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 58756.3us
W20260709 13:17:59.841300 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 51435us
I20260709 13:17:59.797173 15509 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:17:59.826532 15326 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:59.853735 10767 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:17:59.844156 15335 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:17:59.841812 15358 raft_consensus.cc:2749] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:59.854029 10842 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:17:59.846041 15020 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:17:59.873404 15544 log.cc:927] Time spent T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:17:59.873598 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:17:59.846997 15223 consensus_queue.cc:1059] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 13:17:59.890053 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:59.891654 14446 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:59.846534 15499 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:17:59.894132 10796 raft_consensus.cc:1240] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 11. Current term is 12. Ops: []
I20260709 13:17:59.937096 10801 raft_consensus.cc:1240] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 12. Current term is 13. Ops: []
I20260709 13:17:59.937800 11506 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
I20260709 13:17:59.897615 10842 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:17:59.938675 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:59.944057 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:59.888947 15223 raft_consensus.cc:3055] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:17:59.859476 15405 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:59.945686 10766 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:17:59.928766 15548 log.cc:927] Time spent T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:17:59.890981 15378 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:59.873721 15513 raft_consensus.cc:2804] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:59.967808 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:17:59.859160 15326 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:00.031833 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:17:59.864446 15541 raft_consensus.cc:2749] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:59.884544 15335 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:17:59.972617 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:59.972728 10797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 591) took 2982 ms. Trace:
W20260709 13:17:59.975798 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 78307.1us
I20260709 13:18:00.052764 10842 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:17:59.976394 15461 raft_consensus.cc:2749] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:17:59.979806 15429 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:59.949474 15223 raft_consensus.cc:740] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:17:59.983767 15545 log.cc:927] Time spent T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 13:17:59.968641 15500 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:17:59.984591 15513 raft_consensus.cc:697] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:17:59.963178 15317 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:59.970232 15020 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:59.965544 15453 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:17:59.973227 15510 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:17:59.990840 10793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 592) took 2979 ms. Trace:
I20260709 13:17:59.990746 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:17:59.996418 10943 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41020 (request call id 2426) took 2696 ms (client timeout 1913 ms). Trace:
W20260709 13:18:00.126904 10943 rpcz_store.cc:269] 0709 13:17:57.300089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:57.300642 (+   553us) service_pool.cc:224] Handling call
0709 13:17:59.996150 (+2695508us) spinlock_profiling.cc:243] Waited 717 ms on lock 0x7b54003589d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:17:59.996312 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1648210176}
I20260709 13:17:59.999801 15492 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:17:59.947330 15410 consensus_queue.cc:260] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:00.002449 10766 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.116.131:45553 timed out after 1.736s (SENT) [suppressed 25 similar messages]
W20260709 13:18:00.128365 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 125942us
I20260709 13:18:00.012074 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:00.010558 15509 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:00.013410 14446 consensus_queue.cc:1059] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:18:00.014096 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:00.014360 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:00.016093 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:00.150167 10941 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:00.018430 15409 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:00.023867 10796 raft_consensus.cc:1240] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:18:00.049934 10797 rpcz_store.cc:276] 0709 13:17:56.990237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:56.990356 (+   119us) service_pool.cc:224] Handling call
0709 13:17:56.990641 (+   285us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:56.991031 (+   390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:56.991059 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:56.991518 (+   459us) log.cc:844] Serialized 26 byte log entry
0709 13:17:57.181475 (+189957us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:17:57.707159 (+525684us) log.cc:844] Serialized 14 byte log entry
0709 13:17:59.964811 (+2257652us) spinlock_profiling.cc:243] Waited 17.7 ms on lock 0x7b5400305aa0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80e30733 00007f9c80eb5dbf 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c
0709 13:17:59.964892 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:59.965013 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:59.965082 (+    69us) raft_consensus.cc:1685] finished
0709 13:17:59.972524 (+  7442us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:59.972674 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45390,"prepare.queue_time_us":352646,"prepare.run_cpu_time_us":104,"prepare.run_wall_time_us":2564,"raft.queue_time_us":26855,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":163,"spinlock_wait_cycles":40749440,"thread_start_us":643253,"threads_started":2,"wal-append.queue_time_us":275215,"wal-append.run_cpu_time_us":4506,"wal-append.run_wall_time_us":1466066}
I20260709 13:18:00.084084 10800 raft_consensus.cc:1240] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:00.048728 15326 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:00.124347 10793 rpcz_store.cc:276] 0709 13:17:57.011011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:57.011158 (+   147us) service_pool.cc:224] Handling call
0709 13:17:57.029041 (+ 17883us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:17:57.033468 (+  4427us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:17:57.223120 (+189652us) log.cc:844] Serialized 14 byte log entry
0709 13:17:59.576316 (+2353196us) spinlock_profiling.cc:243] Waited 4.88 ms on lock 0x7b180049c4e0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c80a5d4bc 00007f9c7ddab48b 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a5523a
0709 13:17:59.576538 (+   222us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:17:59.577295 (+   757us) log.cc:844] Serialized 78 byte log entry
0709 13:17:59.807709 (+230414us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:59.807751 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:59.807824 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:59.990222 (+182398us) raft_consensus.cc:1685] finished
0709 13:17:59.990373 (+   151us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:59.990622 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":342859,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":56,"raft.queue_time_us":6908,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":11513,"spinlock_wait_cycles":11227392,"thread_start_us":495612,"threads_started":3,"wal-append.queue_time_us":517180,"wal-append.run_cpu_time_us":2944,"wal-append.run_wall_time_us":1266741}
I20260709 13:18:00.067535 15274 raft_consensus.cc:2749] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:00.153630 10948 raft_consensus.cc:2393] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 11: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:00.158651 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:00.159389 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:00.159417 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:00.159741 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 107136us
I20260709 13:18:00.161048 10842 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:00.161264 10946 raft_consensus.cc:1275] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:18:00.161319 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:00.163484 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:00.163949 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:00.162561 10842 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:00.168082 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 139652us
I20260709 13:18:00.169744 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:00.170439 10948 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 13.
I20260709 13:18:00.168028 10842 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:00.175170 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:00.175199 10842 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:00.176148 10948 raft_consensus.cc:2393] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 16: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:00.183321 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:00.184203 10793 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:18:00.184612 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:00.187425 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:00.188100 10943 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 14.
I20260709 13:18:00.192981 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:00.193672 10801 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 11.
I20260709 13:18:00.152063 15453 raft_consensus.cc:697] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:00.198931 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:00.199584 10799 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:00.158649 15541 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:00.201920 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:00.196657 10941 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
I20260709 13:18:00.163213 15461 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:00.204540 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:00.102878 15500 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:00.237396 10948 raft_consensus.cc:1240] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:00.174062 15295 raft_consensus.cc:2804] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:18:00.198191 15393 log.cc:927] Time spent T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 3.691s	user 0.018s	sys 0.021s
I20260709 13:18:00.187593 15359 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:00.246989 10796 raft_consensus.cc:1275] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:18:00.256275 15335 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:00.187327 15432 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:00.212283 10793 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 15.
I20260709 13:18:00.191476 15543 raft_consensus.cc:2804] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:00.194764 15457 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:00.173759 15549 raft_consensus.cc:2749] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:00.158267 15409 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:00.265784 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:00.243191 15513 consensus_queue.cc:237] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:00.268862 15432 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:00.292474 15432 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:00.268213 15543 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:00.293318 15543 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:00.270526 15453 consensus_queue.cc:237] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:00.259768 15223 consensus_queue.cc:260] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:00.302117 15223 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:00.272436 15351 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:00.275763 10793 raft_consensus.cc:1240] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:18:00.281746 10799 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 11.
I20260709 13:18:00.309980 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:00.282909 10946 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 10.
I20260709 13:18:00.266772 15359 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:00.268354 15295 raft_consensus.cc:697] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:00.267215 14970 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:00.283543 15467 raft_consensus.cc:2749] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:00.198375 15410 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:00.322420 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:00.238305 15541 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:00.260216 15461 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:00.276645 15457 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:00.329993 10793 raft_consensus.cc:1275] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:18:00.331240 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:00.337544 10942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 2051) took 3179 ms. Trace:
I20260709 13:18:00.338352 10941 raft_consensus.cc:2410] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:18:00.339214 11504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56416 (request call id 761) took 3386 ms. Trace:
I20260709 13:18:00.338047 10942 rpcz_store.cc:276] 0709 13:17:57.158467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:57.158645 (+   178us) service_pool.cc:224] Handling call
0709 13:17:57.159067 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:58.177568 (+1018501us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:58.177600 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:58.177641 (+    41us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:17:58.177680 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:00.337297 (+2159617us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b5400015ed4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:00.337343 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:00.337486 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2307533568}
I20260709 13:18:00.339627 11504 rpcz_store.cc:276] 0709 13:17:56.952476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:56.952608 (+   132us) service_pool.cc:224] Handling call
0709 13:17:56.955623 (+  3015us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:56.994550 (+ 38927us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:56.994584 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:56.994627 (+    43us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:17:56.994689 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:00.338953 (+3344264us) spinlock_profiling.cc:243] Waited 38.7 ms on lock 0x7b54005a37d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:00.339002 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:00.339127 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":89016064}
I20260709 13:18:00.367873 10946 raft_consensus.cc:2410] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:18:00.369750 10941 raft_consensus.cc:1240] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:00.377929 15541 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:00.378274 15461 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:00.379927 15295 consensus_queue.cc:237] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:00.383136 15409 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:00.384135 15326 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:00.385381 10798 raft_consensus.cc:1240] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 12. Current term is 13. Ops: []
W20260709 13:18:00.391842 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 294973us
I20260709 13:18:00.398000 14970 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:00.400729 15555 log.cc:927] Time spent T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:18:00.404126 15500 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:00.405599 15457 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:00.421366 15400 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:00.422441 10794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2468) took 1459 ms. Trace:
I20260709 13:18:00.419759 15326 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:00.425464 15116 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:00.430616 10947 raft_consensus.cc:3055] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:00.458838 10947 raft_consensus.cc:740] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:18:00.430608 10944 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36714 (request call id 756) took 4098 ms (client timeout 1797 ms). Trace:
W20260709 13:18:00.432111 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 256953us
I20260709 13:18:00.439469 10794 rpcz_store.cc:276] 0709 13:17:58.963502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:59.370421 (+406919us) service_pool.cc:224] Handling call
0709 13:17:59.370854 (+   433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:59.787986 (+417132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:00.422176 (+634190us) spinlock_profiling.cc:243] Waited 410 ms on lock 0x7b54000bd6d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:00.422372 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":941904640}
W20260709 13:18:00.462430 10944 rpcz_store.cc:269] 0709 13:17:56.332595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:56.332779 (+   184us) service_pool.cc:224] Handling call
0709 13:18:00.430313 (+4097534us) spinlock_profiling.cc:243] Waited 1.79 s on lock 0x7b540039a654. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:00.430508 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4127119104}
W20260709 13:18:00.475317 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:00.483537 10868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 1998) took 2247 ms. Trace:
W20260709 13:18:00.484097 11511 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56416 (request call id 762) took 3530 ms (client timeout 2358 ms). Trace:
W20260709 13:18:00.484508 11511 rpcz_store.cc:269] 0709 13:17:56.953616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:56.954324 (+   708us) service_pool.cc:224] Handling call
0709 13:18:00.483903 (+3529579us) spinlock_profiling.cc:243] Waited 883 ms on lock 0x7b540055e6d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:00.484020 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2030722432}
I20260709 13:18:00.460027 10947 consensus_queue.cc:260] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:00.483958 10868 rpcz_store.cc:276] 0709 13:17:58.236304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:58.236421 (+   117us) service_pool.cc:224] Handling call
0709 13:17:58.287016 (+ 50595us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:58.300497 (+ 13481us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:00.483315 (+2182818us) spinlock_profiling.cc:243] Waited 50.2 ms on lock 0x7b54000983d0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:00.483478 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":115567360}
I20260709 13:18:00.491730 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:00.493011 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:00.493619 10801 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
I20260709 13:18:00.496590 10796 raft_consensus.cc:2410] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:18:00.501037 11509 raft_consensus.cc:1275] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 13:18:00.494604 15555 log.cc:927] Time spent T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:18:00.483012 15351 consensus_queue.cc:1059] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:18:00.520169 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:00.489400 15543 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:00.503855 15116 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:00.503578 15549 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:00.529333 15543 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:00.544258 15543 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:00.544126 15549 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:00.544806 15543 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:00.546741 15543 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:00.514333 15358 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:00.547478 15543 raft_consensus.cc:3055] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:00.547859 15543 raft_consensus.cc:740] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:00.548659 15543 consensus_queue.cc:260] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:00.505678 15480 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:00.550712 15480 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:00.547479 15358 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:00.554210 15358 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:00.524645 15562 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:00.557298 15562 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:00.536230 15116 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:00.561323 15549 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:00.561672 15116 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:00.556270 15480 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:00.563041 15562 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:00.549837 15543 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:00.569202 14446 raft_consensus.cc:3055] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:00.569706 14446 raft_consensus.cc:740] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
W20260709 13:18:00.571187 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 328299us
I20260709 13:18:00.570608 14446 consensus_queue.cc:260] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:00.572389 14446 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:00.572743 11477 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:00.575289 15262 consensus_queue.cc:1048] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:00.577132 15262 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:00.577479 15262 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:00.577814 15262 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:00.582335 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:00.583137 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:00.584097 10697 catalog_manager.cc:5697] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 9 to 10, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 10 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:00.590042 15499 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:00.592617 10796 raft_consensus.cc:1240] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:00.586129 10697 catalog_manager.cc:5697] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:00.574575 11477 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:00.598078 15461 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:00.598615 15461 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:00.596691 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:00.600790 15461 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:00.600888 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:00.601613 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 110577us
I20260709 13:18:00.602957 15158 consensus_queue.cc:1048] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:18:00.604405 11478 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:00.604367 15158 consensus_queue.cc:1059] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:00.605300 15158 raft_consensus.cc:3055] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:00.605652 15158 raft_consensus.cc:740] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:18:00.605813 10698 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42506 (request call id 155) took 22701 ms (client timeout 15000 ms). Trace:
I20260709 13:18:00.622911 10945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 2057) took 2424 ms. Trace:
I20260709 13:18:00.624022 10945 rpcz_store.cc:276] 0709 13:17:58.198357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:58.198476 (+   119us) service_pool.cc:224] Handling call
0709 13:17:58.211093 (+ 12617us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:58.724856 (+513763us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:00.622649 (+1897793us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b54003aff54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:00.622848 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":398523136}
I20260709 13:18:00.610656 10697 catalog_manager.cc:5697] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 6 to 9, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 9 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:18:00.500604 15467 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:18:00.630314 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:00.628448 10697 catalog_manager.cc:5697] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:18:00.610905 15325 raft_consensus.cc:2804] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:00.631603 10916 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:00.643085 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:00.643626 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:00.609241 15429 raft_consensus.cc:2749] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:00.644243 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:00.649406 10874 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 14.
I20260709 13:18:00.647097 10869 raft_consensus.cc:2376] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 13. Re-sending same reply.
I20260709 13:18:00.650306 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:00.651453 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:00.652136 10869 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
I20260709 13:18:00.652441 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:00.653061 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:00.653824 10868 raft_consensus.cc:2393] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 13: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:00.620663 15402 consensus_queue.cc:1059] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:18:00.654912 15402 raft_consensus.cc:3055] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:00.655282 15402 raft_consensus.cc:740] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:00.656177 15402 consensus_queue.cc:260] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:00.657444 15402 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:00.659777 15356 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:00.660353 15356 raft_consensus.cc:697] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:00.661156 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:00.619207 15412 consensus_queue.cc:1048] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:18:00.663470 15563 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:00.631784 15124 consensus_queue.cc:1048] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 13:18:00.664973 15563 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:00.667197 15563 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:00.668527 15518 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:00.620497 15158 consensus_queue.cc:260] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:00.672955 15158 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:00.661403 10697 catalog_manager.cc:5697] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 6 to 9, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 9 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:18:00.661378 15356 consensus_queue.cc:237] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:00.609092 14858 consensus_queue.cc:1059] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:18:00.609113 15492 consensus_queue.cc:1059] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 13:18:00.681468 10868 raft_consensus.cc:1240] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:00.684751 10871 raft_consensus.cc:1240] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 LEADER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:00.645037 15325 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:00.687062 15325 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:00.690189 10949 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 11.
I20260709 13:18:00.640627 15467 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:00.692152 10872 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
I20260709 13:18:00.693329 15467 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:00.703251 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:00.703354 15503 consensus_queue.cc:1059] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:00.704100 10871 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:00.705983 15492 raft_consensus.cc:3055] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:00.706431 15492 raft_consensus.cc:740] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:00.676826 10697 catalog_manager.cc:5697] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 4 to 12. New cstate: current_term: 12 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:18:00.676347 15251 consensus_queue.cc:1048] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 13:18:00.611642 15398 consensus_queue.cc:1059] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:18:00.718897 10697 catalog_manager.cc:5697] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:00.711280 10916 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:00.711057 10915 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:00.724978 10871 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 16.
I20260709 13:18:00.723110 10916 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:00.727186 10916 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:00.734642 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:00.740336 10915 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:00.680197 15413 raft_consensus.cc:3055] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:00.747965 10841 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:00.752977 10697 catalog_manager.cc:5697] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 8 to 12, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 12 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:18:00.753543 10916 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:00.755177 10916 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:00.712034 15412 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:00.726135 15124 raft_consensus.cc:2749] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:00.726935 15188 raft_consensus.cc:2749] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:00.766741 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:00.782239 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:00.782546 10871 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:00.741735 14987 raft_consensus.cc:2749] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:00.779771 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:00.785072 10872 raft_consensus.cc:2410] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:18:00.776270 10950 raft_consensus.cc:1275] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:18:00.784610 10697 catalog_manager.cc:5697] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 8 to 11, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 11 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:18:00.788762 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:00.790591 10950 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:00.751258 15457 raft_consensus.cc:2804] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:00.747803 15413 raft_consensus.cc:740] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:00.727885 15410 consensus_queue.cc:1059] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:18:00.811990 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:00.812942 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:00.823608 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:00.766316 15409 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:00.825327 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:00.766487 15513 raft_consensus.cc:2749] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:00.764766 15020 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:00.815342 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:00.832093 10943 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
I20260709 13:18:00.815032 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:00.815701 10947 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:00.811787 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:00.834239 11508 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
W20260709 13:18:00.821672 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:00.814687 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:00.776805 15499 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:00.854885 15518 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:00.855214 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:00.855027 15499 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:00.855831 10944 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 11.
I20260709 13:18:00.855548 15518 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:00.856762 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:00.857170 15499 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:00.857765 15518 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:00.860729 11509 raft_consensus.cc:2393] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 12: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:00.857877 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:00.859705 10767 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:00.866381 10842 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:00.867344 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:00.868798 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:00.870978 10794 raft_consensus.cc:1240] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:00.858033 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:00.872826 10871 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 13.
I20260709 13:18:00.873256 10870 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:00.878401 10792 raft_consensus.cc:1240] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:00.868153 10798 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 10.
I20260709 13:18:00.839581 10950 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
I20260709 13:18:00.840713 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:00.830483 15492 consensus_queue.cc:260] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:00.766778 15348 consensus_queue.cc:1048] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.063s
I20260709 13:18:00.898176 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:00.898916 11504 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 14.
I20260709 13:18:00.880549 10767 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:00.904446 10767 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:00.906279 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:00.906989 10945 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 10.
I20260709 13:18:00.908529 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:00.910672 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:00.911343 10943 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
I20260709 13:18:00.911823 10767 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:00.842185 15530 consensus_queue.cc:1059] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:18:00.844779 15262 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:00.845847 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:00.916944 10872 raft_consensus.cc:2393] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 11: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:00.796365 15267 raft_consensus.cc:2804] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:00.812484 15515 consensus_queue.cc:1048] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:18:00.812703 15457 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:00.812312 15455 consensus_queue.cc:1059] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 13:18:00.784003 15467 consensus_queue.cc:1048] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.068s
W20260709 13:18:00.606211 10698 rpcz_store.cc:269] 0709 13:17:37.904280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:37.917616 (+ 13336us) service_pool.cc:224] Handling call
0709 13:18:00.605620 (+22688004us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:51.359050 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:51.359798 (+   748us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:51.359855 (+    57us) write_op.cc:435] Acquired schema lock
0709 13:17:51.359880 (+    25us) tablet.cc:662] Decoding operations
0709 13:17:51.361036 (+  1156us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:51.361125 (+    89us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:51.361156 (+    31us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:17:51.362149 (+   993us) tablet.cc:701] Row locks acquired
0709 13:17:51.362198 (+    49us) write_op.cc:260] PREPARE: finished
0709 13:17:51.362378 (+   180us) write_op.cc:270] Start()
0709 13:17:51.362522 (+   144us) write_op.cc:276] Timestamp: P: 1783603071362314 usec, L: 0
0709 13:17:51.362549 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 13:17:51.407651 (+ 45102us) log.cc:844] Serialized 3980 byte log entry
0709 13:17:52.669393 (+1261742us) op_driver.cc:464] REPLICATION: finished
0709 13:17:53.449697 (+780304us) write_op.cc:301] APPLY: starting
0709 13:17:53.464367 (+ 14670us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:53.726347 (+261980us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:53.726493 (+   146us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:54.292153 (+565660us) spinlock_profiling.cc:243] Waited 1.92 ms on lock 0x7b180029e780. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c80bf40ec 00007f9c80bf6af7 00007f9c80bf7fb0 00007f9c8490c1bb 00007f9c849535d4 00007f9c847ff8da 00007f9c848023cb 00007f9c84801fd3 00007f9c84900e34
0709 13:17:54.363217 (+ 71064us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:54.363576 (+   359us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=20,mrs_lookups=0
0709 13:17:54.363626 (+    50us) write_op.cc:312] APPLY: finished
0709 13:17:54.364209 (+   583us) log.cc:844] Serialized 97 byte log entry
0709 13:17:54.437730 (+ 73521us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:54.593418 (+155688us) write_op.cc:454] Released schema lock
0709 13:17:54.631948 (+ 38530us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:54.740598 (+108650us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:54.741412 (+   814us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5586589,"reported_tablets":31,"child_traces":[["op",{"apply.queue_time_us":772126,"apply.run_cpu_time_us":45793,"apply.run_wall_time_us":1266366,"cfile_cache_hit":38,"cfile_cache_hit_bytes":39729,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6218,"cfile_init":2,"lbm_read_time_us":19737,"lbm_reads_1-10_ms":3,"lbm_reads_lt_1ms":9,"num_ops":10,"prepare.queue_time_us":410020,"prepare.run_cpu_time_us":29289,"prepare.run_wall_time_us":260926,"raft.queue_time_us":766132,"raft.run_cpu_time_us":62396,"raft.run_wall_time_us":890833,"replication_time_us":1239498,"spinlock_wait_cycles":4413312,"thread_start_us":1890067,"threads_started":5,"wal-append.queue_time_us":411098,"wal-append.run_cpu_time_us":49229,"wal-append.run_wall_time_us":3203819}]]}
I20260709 13:18:00.910341 15251 raft_consensus.cc:2804] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:00.883185 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:00.883792 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:00.946483 10950 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 14.
I20260709 13:18:00.947459 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:00.948539 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:00.884539 10947 raft_consensus.cc:1275] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:18:00.894726 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:00.895479 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:00.912993 10841 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:00.841614 10697 catalog_manager.cc:5697] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 9 to 10, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 10 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
W20260709 13:18:00.915009 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 48721.7us
I20260709 13:18:00.922053 15543 raft_consensus.cc:2749] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:00.908172 15518 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:00.920624 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:18:00.958310 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 45384.8us
W20260709 13:18:00.920475 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 313970us
I20260709 13:18:00.921073 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:00.964514 10871 raft_consensus.cc:2393] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 11: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:00.917927 10767 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:00.861979 15429 raft_consensus.cc:3055] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:00.863723 15359 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:00.924333 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:00.927089 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:00.997450 10796 raft_consensus.cc:2410] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:18:00.858399 15409 raft_consensus.cc:697] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 15 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:00.916183 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:00.937247 10870 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
I20260709 13:18:00.851470 15413 consensus_queue.cc:260] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:00.894073 15020 raft_consensus.cc:697] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 14 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:00.939278 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:00.859653 15500 consensus_queue.cc:1048] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 13:18:00.850615 15325 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:00.922235 15400 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:00.953413 10799 raft_consensus.cc:2393] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 11: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:00.954079 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:00.954156 10842 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:18:01.029984 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 75888us
I20260709 13:18:00.992837 15124 raft_consensus.cc:3055] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:01.036432 11509 raft_consensus.cc:1240] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:01.037106 10792 raft_consensus.cc:1275] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:18:00.992353 15429 raft_consensus.cc:740] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:00.924397 15541 raft_consensus.cc:2749] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:00.955514 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:00.958087 15295 consensus_queue.cc:1059] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 13:18:00.926903 15434 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.022742 15500 consensus_queue.cc:1243] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer 468a8fd46e824f62b9c2c0d3532a6d26 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:18:00.930847 15457 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:01.066154 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:00.979517 15403 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:00.980226 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:00.983390 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:00.983718 15529 consensus_queue.cc:1048] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:18:00.984231 10697 catalog_manager.cc:5697] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 8 to 9, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 9 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:00.974392 10841 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:00.953507 15251 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:01.036770 15124 raft_consensus.cc:740] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:01.000818 10793 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 12.
I20260709 13:18:01.007448 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:01.007900 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:01.009197 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:01.009865 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:18:01.074586 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 100325us
I20260709 13:18:01.075160 11504 raft_consensus.cc:2393] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 11: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:01.010629 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:00.958626 15492 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:01.015144 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:01.016347 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:01.021478 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 237555us
I20260709 13:18:00.958237 15518 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:01.092711 15295 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:01.092914 15518 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:01.093488 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:01.093320 10697 catalog_manager.cc:5697] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 6 to 10, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 10 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:01.094296 11508 raft_consensus.cc:1240] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 14 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:01.095379 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:01.095996 15020 consensus_queue.cc:237] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:01.096786 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:01.095391 10697 catalog_manager.cc:5697] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 6 to 10, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 10 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:18:00.777462 15412 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:01.034945 10801 raft_consensus.cc:1275] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:18:00.995422 15563 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:01.103143 15563 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:01.103600 15563 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:01.103430 15409 consensus_queue.cc:237] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:01.103806 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:01.104545 10871 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 13.
I20260709 13:18:01.069643 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:01.105691 11509 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 14.
I20260709 13:18:00.954453 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:00.931313 15267 raft_consensus.cc:697] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 11 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:01.107281 10944 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 13: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:01.108193 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:01.051999 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:01.108937 10944 raft_consensus.cc:2393] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 11: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:01.109145 10950 raft_consensus.cc:2393] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 13: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:01.109813 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:01.110020 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:01.111209 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:18:01.053311 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:01.052206 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:01.059052 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:01.112340 15429 consensus_queue.cc:260] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:01.014474 14970 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.113592 15429 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:01.113513 14970 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:01.114284 15351 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.114720 15190 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:01.062541 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:18:01.062886 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:01.065177 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:00.957778 15575 consensus_queue.cc:1059] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:18:01.072603 10873 raft_consensus.cc:2393] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 12: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:01.073077 10943 raft_consensus.cc:2393] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 12: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:01.018497 15503 consensus_queue.cc:1059] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:18:01.075632 11512 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 15.
I20260709 13:18:01.005162 15508 consensus_queue.cc:1059] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:18:01.039397 15359 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:01.047945 15574 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:01.075977 10841 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:01.099097 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:01.030958 10947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 773) took 1002 ms. Trace:
I20260709 13:18:01.092727 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:01.100414 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:01.101670 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:01.164601 10947 rpcz_store.cc:276] 0709 13:18:00.028574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:00.028740 (+   166us) service_pool.cc:224] Handling call
0709 13:18:00.430477 (+401737us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:00.566127 (+135650us) spinlock_profiling.cc:243] Waited 76.6 ms on lock 0x7b180018a5c0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80ef0646 00007f9c80e2a58e 00007f9c80eabf9e 00007f9c80ead7de 00007f9c80eb7b49 00007f9c80eb8416 00007f9c80eb4d8f 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4
0709 13:18:00.609234 (+ 43107us) spinlock_profiling.cc:243] Waited 42 ms on lock 0x7b580087add0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80e110aa 00007f9c80e93447 00007f9c80eabfe7 00007f9c80ead7de 00007f9c80eb7b49 00007f9c80eb8416 00007f9c80eb4d8f 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4
0709 13:18:00.815533 (+206299us) spinlock_profiling.cc:243] Waited 88.3 ms on lock 0x7b580087b0d0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80e110aa 00007f9c80e93447 00007f9c80eabfe7 00007f9c80ead7de 00007f9c80eb7b49 00007f9c80eb8416 00007f9c80eb4d8f 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4
0709 13:18:00.949311 (+133778us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:01.030736 (+ 81425us) spinlock_profiling.cc:243] Waited 401 ms on lock 0x7b540039a650. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:01.030903 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1398815616}
I20260709 13:18:01.107822 15267 consensus_queue.cc:237] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:01.049019 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:01.067230 15434 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:00.970707 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 52846.2us
I20260709 13:18:01.179306 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:01.180233 10870 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 13.
I20260709 13:18:01.181367 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:01.116796 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:01.110553 15325 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:01.182600 10767 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:01.102392 15518 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:01.126453 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:01.187057 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:01.127143 11506 raft_consensus.cc:1240] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:01.046511 15513 consensus_queue.cc:1059] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.070s
I20260709 13:18:01.058671 15543 consensus_queue.cc:1048] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 13:18:01.120384 15190 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:01.088804 15251 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:01.130745 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:01.243572 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:01.181720 15512 consensus_queue.cc:1048] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 13:18:01.244549 10945 raft_consensus.cc:2410] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:18:01.245762 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:01.246426 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:01.248731 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:01.249430 10872 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 12.
W20260709 13:18:01.250241 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:01.131876 15320 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:01.135093 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:01.146138 15575 raft_consensus.cc:3055] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:18:01.118021 15582 log.cc:927] Time spent T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:18:01.147048 15478 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.147032 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:01.181300 15503 raft_consensus.cc:3055] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:01.255343 10867 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 11.
I20260709 13:18:01.149816 15432 raft_consensus.cc:3055] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:01.150671 15358 raft_consensus.cc:3055] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:01.153623 15262 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:01.145720 15563 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:01.156497 15412 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:01.145788 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:01.181478 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:01.182291 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:01.114836 15351 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:01.177829 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:01.294018 11507 raft_consensus.cc:2393] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 14: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:01.111999 10950 raft_consensus.cc:2410] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 15 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
W20260709 13:18:01.170523 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 94623.9us
I20260709 13:18:01.167428 11505 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 11.
I20260709 13:18:01.185633 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:01.294898 10841 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:01.184043 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:01.295635 10870 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 12.
I20260709 13:18:01.187840 10767 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:01.254812 15513 consensus_queue.cc:1243] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer e01ecd008edd4aa2b6b2b47efbc2ea87 log is divergent from this leader: its last log entry 11.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:18:01.296520 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:01.251757 11509 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 13.
W20260709 13:18:01.300311 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:01.090545 15403 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:01.176347 10697 catalog_manager.cc:5697] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 12 to 13, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 13 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:01.148064 15529 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.255299 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:01.314607 10871 raft_consensus.cc:2393] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 12: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:01.164374 15407 raft_consensus.cc:3055] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:01.256297 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:01.256405 11503 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
I20260709 13:18:01.176168 15574 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:01.167891 15359 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:01.207242 15274 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:01.262459 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:01.263654 10945 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:01.240890 15505 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:01.252662 15575 raft_consensus.cc:740] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:01.138200 15457 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:01.346858 15575 consensus_queue.cc:260] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:01.348109 15575 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:01.349262 15457 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:01.350445 10946 raft_consensus.cc:2393] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 12: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:01.352718 15413 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:18:01.354388 15583 log.cc:927] Time spent T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.232s	user 0.001s	sys 0.000s
W20260709 13:18:01.294029 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:01.309271 10841 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:01.310518 10869 raft_consensus.cc:2393] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 15: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:01.358196 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:01.358953 10869 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 15.
I20260709 13:18:01.359874 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:01.312063 10949 raft_consensus.cc:1240] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 13:18:01.313098 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:01.362375 10867 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 16.
I20260709 13:18:01.363471 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:01.292707 10916 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:01.258148 10793 raft_consensus.cc:1240] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:01.257854 14970 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:01.257186 15503 raft_consensus.cc:740] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:01.260516 15158 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.319370 15407 raft_consensus.cc:740] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:01.322448 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:01.266824 15358 raft_consensus.cc:740] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:01.335228 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:01.388458 10874 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 13.
I20260709 13:18:01.254554 15408 consensus_queue.cc:1059] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:18:01.338253 10873 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
I20260709 13:18:01.269006 14987 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:01.334195 11478 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:01.297715 15492 consensus_queue.cc:1059] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.071s
I20260709 13:18:01.345284 10697 catalog_manager.cc:5697] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:18:01.356176 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:01.393760 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 101126us
W20260709 13:18:01.391312 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 57244.6us
I20260709 13:18:01.316686 15403 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:01.399696 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:01.401265 10794 raft_consensus.cc:1240] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:18:01.369256 10867 raft_consensus.cc:2410] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:18:01.260064 15432 raft_consensus.cc:740] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:01.318656 15222 raft_consensus.cc:2749] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:01.260978 15478 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:01.322019 15574 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:01.372741 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:01.404772 10792 raft_consensus.cc:1240] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:18:01.404249 10697 catalog_manager.cc:5697] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 5 to 9, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 9 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:18:01.267333 15124 consensus_queue.cc:260] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:01.322566 15513 consensus_queue.cc:1048] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.066s
I20260709 13:18:01.371279 15516 raft_consensus.cc:3055] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:01.364423 15274 raft_consensus.cc:697] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:01.323271 15529 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:01.417367 10945 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 15.
I20260709 13:18:01.418574 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:01.381481 15347 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:01.430434 10697 catalog_manager.cc:5697] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 9 to 10, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 10 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:01.443132 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:01.433772 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:01.435335 10915 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:01.442143 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:01.446251 10943 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
I20260709 13:18:01.448156 10871 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:01.403765 15405 raft_consensus.cc:2749] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:01.450366 10915 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:01.450407 10916 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:01.410898 15577 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:01.459867 10915 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:01.460755 10916 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
W20260709 13:18:01.477531 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:01.408910 14987 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:01.390626 15158 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:01.506984 10941 raft_consensus.cc:1240] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 15 LEADER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 14. Current term is 15. Ops: []
W20260709 13:18:01.508250 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 48463.1us
I20260709 13:18:01.561187 10915 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:01.538237 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:01.432628 15516 raft_consensus.cc:740] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:01.452960 15503 consensus_queue.cc:260] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:01.458760 15492 consensus_queue.cc:1243] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer 468a8fd46e824f62b9c2c0d3532a6d26 log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:18:01.459877 15412 consensus_queue.cc:1059] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.116s
I20260709 13:18:01.460387 15577 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:01.508875 15447 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:01.444231 15188 consensus_queue.cc:1059] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.132s
I20260709 13:18:01.575621 15190 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:01.573378 15432 consensus_queue.cc:260] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:01.695444 15432 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:01.577144 15478 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:01.448277 15434 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:01.699069 15348 consensus_queue.cc:1175] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" responder_term: 14 status { last_received { term: 12 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 13. Current term is 14. Ops: [13.7-13.7]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:18:01.579301 15589 raft_consensus.cc:2749] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:01.580314 10915 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:01.580717 10697 catalog_manager.cc:5697] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
W20260709 13:18:01.700695 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 120449us
I20260709 13:18:01.701728 10915 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:18:01.584442 15593 log.cc:927] Time spent T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.239s	user 0.003s	sys 0.001s
I20260709 13:18:01.591082 15508 raft_consensus.cc:3055] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:01.704736 15508 raft_consensus.cc:740] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:01.598704 10871 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 14.
I20260709 13:18:01.706722 15589 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:01.707170 15589 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:01.707510 15589 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:01.711153 15515 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:01.711774 15515 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:01.712184 15515 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:01.715528 10915 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:01.569147 15455 raft_consensus.cc:2749] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:01.717214 15356 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:01.572090 15358 consensus_queue.cc:260] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:01.718724 15358 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:01.719964 15188 consensus_queue.cc:1243] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer 12682eb70bc84c76879e04a8628ee586 log is divergent from this leader: its last log entry 14.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:18:01.720556 15188 raft_consensus.cc:3055] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:01.720885 15188 raft_consensus.cc:740] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:01.721827 15188 consensus_queue.cc:260] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:01.570761 15588 log.cc:927] Time spent T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.245s	user 0.001s	sys 0.000s
I20260709 13:18:01.724036 14987 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:01.724845 15274 consensus_queue.cc:237] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:01.726202 10868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2023) took 1061 ms. Trace:
I20260709 13:18:01.726578 10868 rpcz_store.cc:276] 0709 13:18:00.683261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:00.685611 (+  2350us) service_pool.cc:224] Handling call
0709 13:18:00.686000 (+   389us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:00.726112 (+ 40112us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:00.726150 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:00.726872 (+   722us) log.cc:844] Serialized 52 byte log entry
0709 13:18:01.020075 (+293203us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:01.392844 (+372769us) log.cc:844] Serialized 14 byte log entry
0709 13:18:01.393333 (+   489us) log.cc:844] Serialized 14 byte log entry
0709 13:18:01.394220 (+   887us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:01.394304 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:01.589739 (+195435us) spinlock_profiling.cc:243] Waited 13.2 ms on lock 0x7b1800437980. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc8522 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:18:01.593096 (+  3357us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b1800437980. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc8522 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:18:01.725988 (+132892us) raft_consensus.cc:1685] finished
0709 13:18:01.726009 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:01.726144 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21822,"prepare.queue_time_us":134485,"prepare.run_cpu_time_us":121,"prepare.run_wall_time_us":67,"raft.queue_time_us":38299,"raft.run_cpu_time_us":402,"raft.run_wall_time_us":42756,"spinlock_wait_cycles":70140928,"thread_start_us":583147,"threads_started":2,"wal-append.queue_time_us":504463}
W20260709 13:18:01.728579 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:01.679761 15577 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:01.649112 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:01.652869 15415 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:01.661468 10869 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 11.
W20260709 13:18:01.671875 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 484049us
I20260709 13:18:01.672570 15505 raft_consensus.cc:697] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:01.742372 15478 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.750367 15505 consensus_queue.cc:237] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:01.751200 15478 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:01.753489 15478 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:01.672741 14970 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:01.680787 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:01.755381 10875 raft_consensus.cc:2376] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 16. Re-sending same reply.
I20260709 13:18:01.755431 14987 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:01.756494 15529 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:01.756286 14987 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:01.756943 15158 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:01.757651 15359 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:01.757831 15351 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:01.758664 15563 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:01.683709 10872 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 14.
I20260709 13:18:01.762903 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:01.763728 10872 raft_consensus.cc:2393] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 11: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:01.764150 10872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 1949) took 1032 ms. Trace:
I20260709 13:18:01.692409 15518 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:01.764756 15251 consensus_meta.cc:287] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.509s	user 0.007s	sys 0.001s
I20260709 13:18:01.717742 15356 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:01.765319 15356 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:01.765206 15251 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:01.767079 10799 raft_consensus.cc:2393] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 12: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:01.734951 15503 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:01.772003 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 1952) took 1005 ms. Trace:
I20260709 13:18:01.772418 10869 rpcz_store.cc:276] 0709 13:18:00.766361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.372426 (+606065us) service_pool.cc:224] Handling call
0709 13:18:01.771822 (+399396us) spinlock_profiling.cc:243] Waited 247 ms on lock 0x7b54000f69d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:01.771941 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":567820672}
I20260709 13:18:01.772984 14987 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:01.773602 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:01.774493 10869 raft_consensus.cc:2410] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:18:01.559747 15402 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:01.560861 15590 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:01.775210 15590 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:01.756543 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:01.775651 15590 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:01.599175 15530 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:01.744560 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:01.779096 10868 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:01.612983 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:01.753504 15589 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:01.776268 10875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49290 (request call id 691) took 1067 ms. Trace:
I20260709 13:18:01.764566 10872 rpcz_store.cc:276] 0709 13:18:00.731643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.762594 (+1030951us) service_pool.cc:224] Handling call
0709 13:18:01.764089 (+  1495us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:01.631359 15295 raft_consensus.cc:3055] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:01.781123 10875 rpcz_store.cc:276] 0709 13:18:00.708608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.756272 (+1047664us) service_pool.cc:224] Handling call
0709 13:18:01.776207 (+ 19935us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:01.783533 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49290 (request call id 692) took 1074 ms. Trace:
I20260709 13:18:01.785378 10869 rpcz_store.cc:276] 0709 13:18:00.708912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.773345 (+1064433us) service_pool.cc:224] Handling call
0709 13:18:01.783460 (+ 10115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:01.786767 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:01.774266 15356 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:01.762183 15515 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:01.790413 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 481245us
I20260709 13:18:01.804334 10841 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:01.790264 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:01.793310 10946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41014 (request call id 2070) took 1033 ms. Trace:
I20260709 13:18:01.805709 10946 rpcz_store.cc:276] 0709 13:18:00.765138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:00.765697 (+   559us) service_pool.cc:224] Handling call
0709 13:18:01.793014 (+1027317us) spinlock_profiling.cc:243] Waited 527 ms on lock 0x7b5400016dd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:01.793226 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1211147520}
I20260709 13:18:01.806071 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:01.807804 10841 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:01.808491 10875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49290 (request call id 690) took 1100 ms. Trace:
I20260709 13:18:01.809239 10875 rpcz_store.cc:276] 0709 13:18:00.708324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.785172 (+1076848us) service_pool.cc:224] Handling call
0709 13:18:01.808426 (+ 23254us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:01.795642 10842 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.011s	sys 0.026s
I20260709 13:18:01.791448 15188 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:18:01.782994 15020 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.091s	sys 0.179s
I20260709 13:18:01.799571 15251 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:01.801926 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:01.806195 10872 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 14.
W20260709 13:18:01.795204 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 427431us
I20260709 13:18:01.810356 10841 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:18:01.811923 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:01.817534 10868 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 12.
I20260709 13:18:01.784664 15295 raft_consensus.cc:740] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:01.823609 10868 raft_consensus.cc:1275] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:18:01.810494 10875 raft_consensus.cc:1240] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 11. Current term is 12. Ops: []
W20260709 13:18:01.816316 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:01.845741 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:01.845856 10875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 682) took 1161 ms. Trace:
I20260709 13:18:01.846226 10875 rpcz_store.cc:276] 0709 13:18:00.684792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.810024 (+1125232us) service_pool.cc:224] Handling call
0709 13:18:01.810331 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:01.845585 (+ 35254us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:01.845801 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:01.847906 11478 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:01.781136 15530 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:01.848726 10868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 686) took 1163 ms. Trace:
W20260709 13:18:01.818723 15020 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.091s	sys 0.180s
I20260709 13:18:01.770510 15508 consensus_queue.cc:260] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:01.827863 10841 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:01.825649 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 1954) took 1010 ms. Trace:
I20260709 13:18:01.842263 10867 raft_consensus.cc:1240] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:01.848958 14446 raft_consensus.cc:2749] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:01.850044 15508 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:01.805992 10876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2024) took 1121 ms. Trace:
I20260709 13:18:01.849555 10868 rpcz_store.cc:276] 0709 13:18:00.685853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.822843 (+1136990us) service_pool.cc:224] Handling call
0709 13:18:01.823195 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:01.848251 (+ 25056us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:01.848669 (+   418us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:01.864420 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:01.781448 15415 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:02.000521 10767 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.9.116.190:36007, user_credentials={real_user=slave}} blocked reactor thread for 144663us
I20260709 13:18:01.822810 15541 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:01.850376 10869 rpcz_store.cc:276] 0709 13:18:00.814877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.801630 (+986753us) service_pool.cc:224] Handling call
0709 13:18:01.825574 (+ 23944us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:01.815544 15351 raft_consensus.cc:2749] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:01.860009 10841 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:01.851042 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 687) took 1164 ms. Trace:
W20260709 13:18:02.006258 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 146330us
W20260709 13:18:01.927134 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:02.007468 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:01.962059 14446 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:02.008447 14446 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:02.002375 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:01.814950 10842 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.011s	sys 0.026s
I20260709 13:18:01.755276 15516 consensus_queue.cc:260] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:02.058264 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:02.058427 15516 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:01.822198 14970 raft_consensus.cc:697] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:02.005151 11509 raft_consensus.cc:1275] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 13:18:02.010882 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:02.067368 14970 consensus_queue.cc:237] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:02.080219 15356 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:02.088272 15515 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:02.089815 15518 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:02.090587 15518 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:02.093797 15518 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:02.096517 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:02.097635 10793 raft_consensus.cc:2393] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 14: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:01.837574 15597 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:02.100055 15597 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:02.104635 15598 log.cc:927] Time spent T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:18:02.108486 15447 raft_consensus.cc:2749] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:01.855767 15530 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:01.839939 15529 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:02.115213 15529 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:02.115752 15529 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:02.006247 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:02.119400 10869 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 11.
I20260709 13:18:02.121290 10869 raft_consensus.cc:1275] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:18:02.122074 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1965) took 1160 ms. Trace:
I20260709 13:18:02.015272 10801 raft_consensus.cc:1240] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:02.124671 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:02.016882 10800 raft_consensus.cc:1240] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:02.133275 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:02.135119 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:02.018126 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:01.860852 10876 rpcz_store.cc:276] 0709 13:18:00.684210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:00.686891 (+  2681us) service_pool.cc:224] Handling call
0709 13:18:00.687192 (+   301us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:00.689204 (+  2012us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:18:01.003990 (+314786us) log.cc:844] Serialized 14 byte log entry
0709 13:18:01.796031 (+792041us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:01.796586 (+   555us) log.cc:844] Serialized 26 byte log entry
0709 13:18:01.801672 (+  5086us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:01.803709 (+  2037us) log.cc:844] Serialized 14 byte log entry
0709 13:18:01.805571 (+  1862us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:01.805678 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:01.805733 (+    55us) raft_consensus.cc:1685] finished
0709 13:18:01.805752 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:01.805918 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1586,"prepare.queue_time_us":466926,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":86,"raft.queue_time_us":16558,"raft.run_cpu_time_us":320,"raft.run_wall_time_us":8207,"thread_start_us":1068085,"threads_started":3,"wal-append.queue_time_us":465772}
I20260709 13:18:02.008287 10841 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:02.021157 15402 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:02.021524 15563 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:02.022378  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:02.006141 10867 rpcz_store.cc:276] 0709 13:18:00.686129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.841654 (+1155525us) service_pool.cc:224] Handling call
0709 13:18:01.842071 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:01.850614 (+  8543us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:01.850972 (+   358us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:02.022567 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:02.024178 14446 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:02.026003 15543 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:02.027403 10873 raft_consensus.cc:1275] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 13:18:02.142731 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 134513us
I20260709 13:18:02.143352 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1953) took 1347 ms. Trace:
I20260709 13:18:02.143796 10873 rpcz_store.cc:276] 0709 13:18:00.796012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:02.026712 (+1230700us) service_pool.cc:224] Handling call
0709 13:18:02.027064 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:02.143060 (+115996us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:02.143297 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:02.147157 10948 raft_consensus.cc:1240] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:18:02.028868 15295 consensus_queue.cc:260] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:02.033725 10874 raft_consensus.cc:1240] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:02.035113 15596 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:02.157226 10876 raft_consensus.cc:1240] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:02.035753 10871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 2033) took 1216 ms. Trace:
W20260709 13:18:02.036139 15602 log.cc:927] Time spent T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.197s	user 0.000s	sys 0.000s
I20260709 13:18:02.037128 10945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 2094) took 1029 ms. Trace:
I20260709 13:18:02.168599 10945 rpcz_store.cc:276] 0709 13:18:01.007883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.439497 (+431614us) service_pool.cc:224] Handling call
0709 13:18:01.454184 (+ 14687us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:01.682818 (+228634us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:01.682866 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:01.682926 (+    60us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:01.682977 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:02.036845 (+353868us) spinlock_profiling.cc:243] Waited 228 ms on lock 0x7b54001e1c54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:02.036903 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:02.037064 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":525111680}
I20260709 13:18:02.037827 15407 consensus_queue.cc:260] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:02.040722 15589 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:02.169975 10798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1959) took 1013 ms. Trace:
I20260709 13:18:02.043876 15124 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:02.170411 15407 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:02.045634 11478 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:02.136170 10795 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 13.
W20260709 13:18:02.175478 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 129931us
I20260709 13:18:02.170406 10798 rpcz_store.cc:276] 0709 13:18:01.156525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.160025 (+  3500us) service_pool.cc:224] Handling call
0709 13:18:01.160392 (+   367us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:01.160751 (+   359us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:01.160776 (+    25us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:01.161353 (+   577us) log.cc:844] Serialized 52 byte log entry
0709 13:18:01.444963 (+283610us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:01.615991 (+171028us) log.cc:844] Serialized 14 byte log entry
0709 13:18:01.673512 (+ 57521us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:01.673595 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:02.169745 (+496150us) raft_consensus.cc:1685] finished
0709 13:18:02.169767 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:02.169913 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":331482,"prepare.queue_time_us":171038,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":31,"raft.queue_time_us":58466,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":165,"thread_start_us":199529,"threads_started":1,"wal-append.queue_time_us":556043}
I20260709 13:18:02.176472 11478 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:02.176683 15503 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:02.177260 15503 raft_consensus.cc:697] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 16 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:02.177764 11478 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:02.178982 15429 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:02.179208 14446 raft_consensus.cc:2749] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:02.122485 10869 rpcz_store.cc:276] 0709 13:18:00.961175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:02.120398 (+1159223us) service_pool.cc:224] Handling call
0709 13:18:02.120845 (+   447us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:02.121743 (+   898us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:02.122000 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:02.141281 10867 raft_consensus.cc:1240] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 11. Current term is 12. Ops: []
W20260709 13:18:02.141817 10916 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.016s	sys 0.047s
I20260709 13:18:01.823132 15403 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:02.002167 15222 raft_consensus.cc:2804] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:01.863991 10766 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:02.194872 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 330930us
I20260709 13:18:02.195073 10943 raft_consensus.cc:1275] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:18:02.196614 10766 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:02.154132 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:02.155365 10874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1964) took 1194 ms. Trace:
I20260709 13:18:02.157931 10871 rpcz_store.cc:276] 0709 13:18:00.819633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.319260 (+499627us) service_pool.cc:224] Handling call
0709 13:18:02.035497 (+716237us) spinlock_profiling.cc:243] Waited 60.3 ms on lock 0x7b5400077454. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:02.035677 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":138601216}
I20260709 13:18:02.146502 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:02.151486 15402 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:02.151854 15563 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:02.165604 10876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1970) took 1202 ms. Trace:
I20260709 13:18:02.166395 15467 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:02.166965 15596 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:02.167972 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:02.149920 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:02.221240 10800 raft_consensus.cc:2410] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:18:02.139166 15320 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:02.181787 10916 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.016s	sys 0.048s
W20260709 13:18:02.222844 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 762113us
I20260709 13:18:02.224495 10916 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:02.163597 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:02.225410 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2039) took 1192 ms. Trace:
I20260709 13:18:02.226032 10948 raft_consensus.cc:2410] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:18:02.184341 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1969) took 1221 ms. Trace:
I20260709 13:18:02.226934 10867 rpcz_store.cc:276] 0709 13:18:00.963068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:02.140804 (+1177736us) service_pool.cc:224] Handling call
0709 13:18:02.141100 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:02.184061 (+ 42961us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:02.184285 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:02.183732 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:02.184743 10841 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:02.190183 14446 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:02.193120 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:02.229700 14446 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:02.204401 10876 rpcz_store.cc:276] 0709 13:18:00.963460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:02.151708 (+1188248us) service_pool.cc:224] Handling call
0709 13:18:02.157041 (+  5333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:02.165314 (+  8273us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:02.165541 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:02.229971 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 45515.7us
I20260709 13:18:02.200029 10874 rpcz_store.cc:276] 0709 13:18:00.960766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:02.033138 (+1072372us) service_pool.cc:224] Handling call
0709 13:18:02.033524 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:02.155004 (+121480us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:02.155288 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:02.210815 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:02.152420 15543 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:02.217895 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:02.218727 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:02.229907 11509 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:02.211053 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:02.246203 10946 raft_consensus.cc:2410] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
W20260709 13:18:02.193982 15494 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.032s	sys 0.061s
I20260709 13:18:02.231141 10841 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:02.232542 10916 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:18:02.281209 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 48738.9us
I20260709 13:18:02.232995 10950 raft_consensus.cc:1240] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 16. Current term is 17. Ops: [14.12-16.13]
I20260709 13:18:02.288280 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:02.291409 10767 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:02.157323 15347 raft_consensus.cc:2749] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:02.236574 10797 raft_consensus.cc:1240] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:02.213341 15251 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:02.301098 10796 raft_consensus.cc:1275] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:18:02.203267 15408 consensus_queue.cc:1059] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 13:18:02.184091 15512 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:02.254878 15402 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:02.194200 15222 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:02.237337 15351 raft_consensus.cc:2749] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:02.256349 15320 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:02.259093 15515 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:02.221822 15295 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:02.271945 15591 log.cc:927] Time spent T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.867s	user 0.000s	sys 0.005s
I20260709 13:18:02.272395 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:02.274111 14446 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:02.178186 15503 consensus_queue.cc:237] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:02.187088 15597 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:02.194947 15478 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:02.235270 15467 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:02.225961 10869 rpcz_store.cc:276] 0709 13:18:01.032746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:02.224565 (+1191819us) service_pool.cc:224] Handling call
0709 13:18:02.224928 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:02.225093 (+   165us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:02.225116 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:02.225156 (+    40us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:02.225193 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:02.225245 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:02.225349 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:02.294397 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 63375us
I20260709 13:18:02.342914 10869 raft_consensus.cc:1240] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:02.343168 10841 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:02.344075 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1975) took 1207 ms. Trace:
I20260709 13:18:02.296386 10916 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:02.203075 15325 consensus_queue.cc:1048] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 13:18:02.344504 10869 rpcz_store.cc:276] 0709 13:18:01.136227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:02.342206 (+1205979us) service_pool.cc:224] Handling call
0709 13:18:02.342587 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:02.343555 (+   968us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:02.343910 (+   355us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:02.345733 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 49448.8us
I20260709 13:18:02.346854 10869 raft_consensus.cc:1275] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:18:02.347711 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1977) took 1076 ms. Trace:
I20260709 13:18:02.348330 10869 rpcz_store.cc:276] 0709 13:18:01.271123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:02.346140 (+1075017us) service_pool.cc:224] Handling call
0709 13:18:02.346505 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:02.347446 (+   941us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:02.347657 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:02.304569 11509 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 15.
I20260709 13:18:02.304062 10767 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:18:02.354085 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 50107.9us
I20260709 13:18:02.355006 10767 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:02.303372 11478 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:02.153070 15541 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:02.281675 15543 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:02.319044 10876 raft_consensus.cc:1240] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:18:02.367216 15251 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:02.367509 10876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1973) took 1307 ms. Trace:
I20260709 13:18:02.367926 15251 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:02.324249 10913 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.725s	user 0.033s	sys 0.058s
I20260709 13:18:02.368502 15251 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:18:02.368963 10913 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.808s	user 0.033s	sys 0.058s
W20260709 13:18:02.369994 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:02.203398 15398 consensus_queue.cc:1059] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.103s
I20260709 13:18:02.373193 15398 raft_consensus.cc:3055] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:02.373704 15398 raft_consensus.cc:740] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:02.281410 15500 raft_consensus.cc:2749] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:02.279706 15415 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:02.306993 15562 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:02.375757 15562 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:02.318084 15505 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.027s	sys 0.090s
I20260709 13:18:02.377882 15562 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:02.381330 15429 raft_consensus.cc:2749] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:02.285429 15409 raft_consensus.cc:2749] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:18:02.327515 15274 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.021s	sys 0.103s
I20260709 13:18:02.300994 15518 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:18:02.334344 10697 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.088s	sys 0.082s
I20260709 13:18:02.231042 10766 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:02.411579 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 180608us
W20260709 13:18:02.412200 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:02.362524 15541 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:02.325073 15222 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:02.315395 15192 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:02.382000 15597 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:02.324380 15347 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:02.319429 15512 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:02.331475 15413 consensus_meta.cc:287] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.978s	user 0.010s	sys 0.001s
I20260709 13:18:02.368136 10876 rpcz_store.cc:276] 0709 13:18:01.060587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:02.294287 (+1233700us) service_pool.cc:224] Handling call
0709 13:18:02.318738 (+ 24451us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:02.367121 (+ 48383us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:02.367433 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:02.386714 15577 consensus_meta.cc:287] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.627s	user 0.008s	sys 0.004s
I20260709 13:18:02.346761 10916 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:02.388877 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 86364.4us
I20260709 13:18:02.405520 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:02.410650 10697 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.088s	sys 0.083s
I20260709 13:18:02.319021 15606 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:02.334444 15515 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:02.413836 10766 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:02.415551 11505 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:18:02.418222 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 75135.7us
I20260709 13:18:02.405006 15407 raft_consensus.cc:2749] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:18:02.403720 15358 consensus_meta.cc:287] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.685s	user 0.009s	sys 0.000s
W20260709 13:18:02.405742 15574 consensus_meta.cc:287] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.965s	user 0.008s	sys 0.004s
I20260709 13:18:02.422349 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:02.410261 15274 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.021s	sys 0.104s
W20260709 13:18:02.280093 15494 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.636s	user 0.033s	sys 0.061s
I20260709 13:18:02.359539 15402 raft_consensus.cc:2749] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:02.173957 15530 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:02.420934 15591 log.cc:927] Time spent T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:18:02.351953 15478 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:02.463265 11478 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:18:02.626513 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 163361us
W20260709 13:18:02.627009 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:02.374683 15398 consensus_queue.cc:260] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:02.410327 14987 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:02.556881 10871 raft_consensus.cc:1240] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:02.416216 15501 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:02.617192 10841 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:02.640300 10841 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:02.489886 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:02.559150 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 212481us
W20260709 13:18:02.642118 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 228341us
W20260709 13:18:02.560024 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:02.628738 10949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 2093) took 1621 ms. Trace:
W20260709 13:18:02.591392 15262 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.022s	user 0.045s	sys 0.110s
I20260709 13:18:02.424139 15608 raft_consensus.cc:2749] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:02.423800 15597 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:02.643143 10949 rpcz_store.cc:276] 0709 13:18:01.007598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.361314 (+353716us) service_pool.cc:224] Handling call
0709 13:18:01.361670 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:01.372979 (+ 11309us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:18:01.373013 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:01.390048 (+ 17035us) log.cc:844] Serialized 26 byte log entry
0709 13:18:01.812348 (+422300us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:01.812386 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:01.812467 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:02.628427 (+815960us) raft_consensus.cc:1685] finished
0709 13:18:02.628458 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:02.628653 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45202,"raft.queue_time_us":17954,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":2834,"thread_start_us":367698,"threads_started":1,"wal-append.queue_time_us":918502}
I20260709 13:18:02.644608 10916 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:02.654495 10841 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:02.663789 10944 raft_consensus.cc:1240] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:02.669117 10916 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:18:02.644166 15262 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.084s	user 0.046s	sys 0.115s
W20260709 13:18:02.673043 11477 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.067s	user 0.010s	sys 0.018s
W20260709 13:18:02.673559 11477 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.068s	user 0.010s	sys 0.018s
I20260709 13:18:02.676695 10915 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:18:02.677206 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:02.567724 15409 raft_consensus.cc:2804] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:02.630229 15515 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:02.653914 15400 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:02.667577 15358 raft_consensus.cc:2749] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:02.668639 15563 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:02.669728 15403 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:02.617174 15574 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:02.672717 15500 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:02.672430 15597 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:18:02.700934 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:02.482864 15577 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:02.426520 15192 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:02.681226 15274 raft_consensus.cc:2749] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:02.680413 15409 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:02.679338 15415 consensus_queue.cc:1059] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.079s
I20260709 13:18:02.764979 15500 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:02.691931 15608 consensus_queue.cc:1059] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.075s
I20260709 13:18:02.751961 15596 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:02.768673 15574 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:02.654443 14987 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:02.770685 15429 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:02.798997 10944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 796) took 1192 ms. Trace:
I20260709 13:18:02.799606 10944 rpcz_store.cc:276] 0709 13:18:01.606546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.742832 (+136286us) service_pool.cc:224] Handling call
0709 13:18:01.743138 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:02.669384 (+926246us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:02.798618 (+129234us) spinlock_profiling.cc:243] Waited 920 ms on lock 0x7b540018ddd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:02.798912 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2116676864}
W20260709 13:18:02.780284 10801 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.009s	sys 0.000s
W20260709 13:18:02.802181 10801 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.010s	sys 0.000s
I20260709 13:18:02.638126 15606 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:02.686721 15402 consensus_queue.cc:1048] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.080s
I20260709 13:18:02.684649 15447 consensus_queue.cc:1048] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.108s
I20260709 13:18:02.678721 15612 raft_consensus.cc:2749] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:18:02.816499 15617 log.cc:927] Time spent T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 13:18:02.819568 15478 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:02.809378 15429 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:02.685456 15351 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:02.687584 15398 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:02.751088 15541 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
W20260709 13:18:02.792415 11507 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.016s	sys 0.042s
I20260709 13:18:02.680660 15589 consensus_queue.cc:1048] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.111s
I20260709 13:18:02.782357 15467 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:02.678529 15613 raft_consensus.cc:2804] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:02.785156 15409 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:18:02.810420 10872 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.009s	sys 0.003s
W20260709 13:18:02.857429 10872 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.612s	user 0.009s	sys 0.004s
I20260709 13:18:02.743436 10766 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:02.790760 14970 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.036s	sys 0.085s
W20260709 13:18:02.860294 14970 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.036s	sys 0.085s
I20260709 13:18:02.850498 15541 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:02.830505 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:02.829800 15512 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:02.850813 11507 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.016s	sys 0.042s
I20260709 13:18:02.861347 15589 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:02.862300 15541 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:02.861919 15589 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:02.862962 15325 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:02.863897 14987 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:02.863948 15505 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:18:02.858666 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 115348us
I20260709 13:18:02.867990 15613 raft_consensus.cc:697] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 12 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:02.871493 15612 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:02.818262 15543 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:02.910023 10766 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:18:02.885293 10873 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.004s	sys 0.015s
I20260709 13:18:02.828130 15574 consensus_queue.cc:1048] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
W20260709 13:18:02.911358 10873 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.004s	sys 0.015s
W20260709 13:18:02.890532 15124 consensus_meta.cc:287] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.718s	user 0.012s	sys 0.000s
I20260709 13:18:02.894160 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:02.873062 11505 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 17.
I20260709 13:18:02.900869 15530 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:02.920370 15508 consensus_meta.cc:287] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.920s	user 0.013s	sys 0.000s
I20260709 13:18:02.917249 15612 raft_consensus.cc:697] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 15 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:02.925460 14970 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:18:02.942459 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:02.920053 15429 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:02.948385 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 697) took 1072 ms. Trace:
W20260709 13:18:02.948137 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 273664us
I20260709 13:18:02.949515 10869 rpcz_store.cc:276] 0709 13:18:01.876083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:02.463190 (+587107us) service_pool.cc:224] Handling call
0709 13:18:02.463581 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:02.463862 (+   281us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:18:02.494655 (+ 30793us) log.cc:844] Serialized 14 byte log entry
0709 13:18:02.948008 (+453353us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:02.948063 (+    55us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:02.948117 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:02.948186 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:02.948325 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6586,"prepare.queue_time_us":41758,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"thread_start_us":370554,"threads_started":1}
I20260709 13:18:02.954135 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:02.949537 10793 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.011s	sys 0.007s
W20260709 13:18:02.956847 10793 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.011s	sys 0.007s
W20260709 13:18:02.944722 15618 log.cc:927] Time spent T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
W20260709 13:18:02.947237 15590 consensus_meta.cc:287] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 1.171s	user 0.013s	sys 0.002s
I20260709 13:18:02.950368 15467 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:02.945072 14970 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:02.964615 15447 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:18:02.970705 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 287199us
I20260709 13:18:02.966738 15192 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:02.962888 15590 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:03.064182 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:02.966049 15613 consensus_queue.cc:237] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:03.048919 10916 connection.cc:256] RPC call timeout handler was delayed by -1.0483138289998806s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2094s for a response
I20260709 13:18:03.091624 10876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 698) took 1215 ms. Trace:
I20260709 13:18:03.015861 10915 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:03.012799 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:03.013336 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:18:03.006909 15623 log.cc:927] Time spent T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:18:03.157346 10797 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 14
W20260709 13:18:03.162329 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:02.970983 15500 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:03.158865 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 226242us
W20260709 13:18:03.006163 15621 log.cc:927] Time spent T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:18:02.970463 14970 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:18:02.997498 15295 consensus_meta.cc:287] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.657s	user 0.008s	sys 0.000s
I20260709 13:18:03.155543 10876 rpcz_store.cc:276] 0709 13:18:01.876407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:02.462367 (+585960us) service_pool.cc:224] Handling call
0709 13:18:02.555094 (+ 92727us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:02.684470 (+129376us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:18:02.748831 (+ 64361us) log.cc:844] Serialized 14 byte log entry
0709 13:18:02.927142 (+178311us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:02.927204 (+    62us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:02.927255 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:03.091366 (+164111us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b54001095d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:03.091416 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:03.091557 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":751,"prepare.queue_time_us":119162,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":9571,"raft.queue_time_us":90,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":5936,"spinlock_wait_cycles":296873216,"thread_start_us":177875,"threads_started":1,"wal-append.queue_time_us":302333}
I20260709 13:18:03.000311 15407 raft_consensus.cc:3055] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:03.181531 10766 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:18:02.993965 15624 log.cc:927] Time spent T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:18:03.081818 10841 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:03.095455 15622 log.cc:927] Time spent T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
W20260709 13:18:03.131408 15222 consensus_meta.cc:287] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.705s	user 0.008s	sys 0.004s
I20260709 13:18:02.977404 15124 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:03.193885 15407 raft_consensus.cc:740] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:03.201040 15295 raft_consensus.cc:2749] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:18:03.201201 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 119478us
I20260709 13:18:03.099483 15634 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:03.202191 10841 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:03.202103 15634 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:03.203229 15295 raft_consensus.cc:2749] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:03.201509 15407 consensus_queue.cc:260] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:03.204255 15407 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:03.100768 15325 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:03.092125 15409 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:03.193009 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:03.193328 10867 raft_consensus.cc:1275] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 11. (term mismatch)
I20260709 13:18:03.309692 10867 pending_rounds.cc:85] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 10
I20260709 13:18:03.310024 10867 pending_rounds.cc:107] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 10.11
I20260709 13:18:03.310329 10867 raft_consensus.cc:2889] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:03.005321 15543 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:03.100484 15320 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:03.002628 15577 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:03.040921 15402 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:03.088608 15492 raft_consensus.cc:3055] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:03.015026 15403 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:03.122263 15589 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:03.084055 15606 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:03.200973 15124 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:03.001937 15516 consensus_meta.cc:287] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.943s	user 0.007s	sys 0.003s
I20260709 13:18:03.205297 15634 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:03.206490 10872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2025) took 2521 ms. Trace:
I20260709 13:18:03.206379 15222 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:03.376840 15222 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:03.065604 14987 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:03.377647 14987 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:03.379923 14987 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:03.380503 14987 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:03.380939 14987 raft_consensus.cc:697] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 17 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:03.381843 14987 consensus_queue.cc:237] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:03.383343 10869 raft_consensus.cc:2410] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 12 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 4.
I20260709 13:18:03.385792 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:03.211745 10875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2027) took 2525 ms. Trace:
I20260709 13:18:03.387744 10875 rpcz_store.cc:276] 0709 13:18:00.686248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.846949 (+1160701us) service_pool.cc:224] Handling call
0709 13:18:01.847273 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:01.847615 (+   342us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:01.847643 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:02.023265 (+175622us) log.cc:844] Serialized 26 byte log entry
0709 13:18:02.270185 (+246920us) spinlock_profiling.cc:243] Waited 94.4 ms on lock 0x7b1800085240. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a550d1 00007f9c80e70a0f
0709 13:18:02.683678 (+413493us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:03.041096 (+357418us) log.cc:844] Serialized 14 byte log entry
0709 13:18:03.041414 (+   318us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:03.041521 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:03.211274 (+169753us) raft_consensus.cc:1685] finished
0709 13:18:03.211435 (+   161us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:03.211657 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":251265,"prepare.queue_time_us":137699,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":4700,"raft.run_cpu_time_us":350,"raft.run_wall_time_us":147690,"spinlock_wait_cycles":217139456,"thread_start_us":641875,"threads_started":2,"wal-append.queue_time_us":882782}
I20260709 13:18:03.219202 15400 raft_consensus.cc:2749] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:18:03.223376 15627 log.cc:927] Time spent T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.249s	user 0.001s	sys 0.000s
W20260709 13:18:03.224169 15620 log.cc:927] Time spent T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.259s	user 0.000s	sys 0.001s
W20260709 13:18:03.225221 15518 consensus_meta.cc:287] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.807s	user 0.009s	sys 0.002s
W20260709 13:18:03.226183 15636 log.cc:927] Time spent T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:18:03.229059 10874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1974) took 2109 ms. Trace:
W20260709 13:18:03.235812 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 54392.5us
I20260709 13:18:03.236596 15642 consensus_queue.cc:1048] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:03.237488 15637 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:03.237746 15631 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:18:03.241108 10870 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 1957) took 2324 ms (client timeout 2094 ms). Trace:
I20260709 13:18:03.406105 15634 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:03.253095 10793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2500) took 1181 ms. Trace:
I20260709 13:18:03.260046 15478 raft_consensus.cc:3055] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:03.442813 10793 rpcz_store.cc:276] 0709 13:18:02.071568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:02.137560 (+ 65992us) service_pool.cc:224] Handling call
0709 13:18:02.188815 (+ 51255us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:02.359148 (+170333us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:02.359177 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:02.359655 (+   478us) log.cc:844] Serialized 26 byte log entry
0709 13:18:02.958549 (+598894us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:02.958614 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:03.252634 (+294020us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b54000d7354. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:03.252711 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:03.252754 (+    43us) raft_consensus.cc:1685] finished
0709 13:18:03.252767 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:03.253010 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2294,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":115,"spinlock_wait_cycles":390759808,"thread_start_us":532336,"threads_started":1,"wal-append.queue_time_us":742984}
W20260709 13:18:03.261436 15529 consensus_meta.cc:287] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 1.145s	user 0.010s	sys 0.003s
I20260709 13:18:02.970702 15467 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:03.266287 10871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1972) took 2248 ms. Trace:
I20260709 13:18:03.187410 15500 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:03.479478 15500 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:03.480834 15467 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:03.203890 10841 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:03.483465 15647 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:18:03.483810 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 279965us
I20260709 13:18:03.484266 15647 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:03.484988 10841 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:03.271556 10801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2501) took 1199 ms. Trace:
I20260709 13:18:03.487154 10801 rpcz_store.cc:276] 0709 13:18:02.071917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:02.147315 (+ 75398us) service_pool.cc:224] Handling call
0709 13:18:02.147608 (+   293us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:02.148040 (+   432us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:18:02.148069 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:02.148508 (+   439us) log.cc:844] Serialized 26 byte log entry
0709 13:18:02.267923 (+119415us) spinlock_profiling.cc:243] Waited 3.22 ms on lock 0x7b18001762c0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a550d1 00007f9c80e70a0f
0709 13:18:02.803101 (+535178us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:02.803161 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:02.803257 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:03.271074 (+467817us) raft_consensus.cc:1685] finished
0709 13:18:03.271109 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:03.271458 (+   349us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":92,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":5076,"spinlock_wait_cycles":7409024,"thread_start_us":534800,"threads_started":1,"wal-append.queue_time_us":786605}
I20260709 13:18:03.488714 15645 consensus_queue.cc:1059] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:03.271909 10800 raft_consensus.cc:1240] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:03.491662 10800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 627) took 1235 ms. Trace:
I20260709 13:18:03.492236 10800 rpcz_store.cc:276] 0709 13:18:02.256335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:02.263315 (+  6980us) service_pool.cc:224] Handling call
0709 13:18:03.271652 (+1008337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:03.489782 (+218130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:03.491380 (+  1598us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b540026b950. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:03.491577 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2317990272}
I20260709 13:18:03.493958 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:03.280910 11477 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:03.284724 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:03.050851 15530 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:03.303205 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:03.305212 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:03.496688 10795 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:02.973869 15447 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:03.307855 11507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56404 (request call id 822) took 1091 ms. Trace:
I20260709 13:18:03.313560 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:03.314112 15512 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:03.272451 15251 consensus_meta.cc:287] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.903s	user 0.008s	sys 0.005s
I20260709 13:18:03.281117 15190 consensus_queue.cc:1048] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.034s
W20260709 13:18:03.272780 15597 consensus_meta.cc:287] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.508s	user 0.012s	sys 0.000s
W20260709 13:18:03.277825 15501 consensus_meta.cc:287] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.634s	user 0.011s	sys 0.000s
I20260709 13:18:03.283028 15274 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:03.304417 15513 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:03.283737 15562 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:03.276526 15625 log.cc:927] Time spent T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.260s	user 0.000s	sys 0.001s
W20260709 13:18:03.278834 15347 consensus_meta.cc:287] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.817s	user 0.009s	sys 0.003s
I20260709 13:18:03.317648 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:03.305011 15616 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:03.324575 15515 consensus_meta.cc:287] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.636s	user 0.011s	sys 0.000s
I20260709 13:18:03.325033 15325 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:03.327561 15409 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:03.327826 15543 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:03.337056 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:03.338479 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2040) took 2305 ms. Trace:
W20260709 13:18:03.339881 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:03.120110 15608 consensus_queue.cc:1048] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.186s
W20260709 13:18:03.347103 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 331308us
W20260709 13:18:03.334556 15640 log.cc:927] Time spent T 00000000000000000000000000000000 P ae957c8fc66c4b6ba063d0c74cbf5484: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:18:03.355526 10797 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 14.
I20260709 13:18:03.266763 15505 raft_consensus.cc:2749] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:03.372640 15577 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:03.371313 10872 rpcz_store.cc:276] 0709 13:18:00.685480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.819998 (+1134518us) service_pool.cc:224] Handling call
0709 13:18:01.820427 (+   429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:01.827233 (+  6806us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:18:01.836365 (+  9132us) log.cc:844] Serialized 14 byte log entry
0709 13:18:02.049925 (+213560us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b18003b1fa0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc8522 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:18:02.244942 (+195017us) spinlock_profiling.cc:243] Waited 94.2 ms on lock 0x7b18003b1fa0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a5523a 00007f9c80ebfa2a
0709 13:18:02.865012 (+620070us) log.cc:844] Serialized 14 byte log entry
0709 13:18:02.865130 (+   118us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:02.865644 (+   514us) log.cc:844] Serialized 26 byte log entry
0709 13:18:02.865819 (+   175us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:02.866017 (+   198us) log.cc:844] Serialized 14 byte log entry
0709 13:18:02.866183 (+   166us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:02.866271 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:03.206179 (+339908us) raft_consensus.cc:1685] finished
0709 13:18:03.206208 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:03.206406 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15563,"prepare.queue_time_us":11823,"prepare.run_cpu_time_us":109,"prepare.run_wall_time_us":18064,"raft.queue_time_us":19671,"raft.run_cpu_time_us":273,"raft.run_wall_time_us":3873,"spinlock_wait_cycles":241990400,"thread_start_us":612773,"threads_started":1,"wal-append.queue_time_us":940084}
I20260709 13:18:03.589035 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:03.589746 10872 raft_consensus.cc:2393] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 15: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:03.373658 15516 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:03.348631 15320 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:03.351641 15492 raft_consensus.cc:740] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
W20260709 13:18:03.398774 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:03.592031 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:03.371075 15403 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:03.371804 15589 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:03.401430 15641 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:03.356089 15402 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:03.029482 15612 consensus_queue.cc:237] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:03.403753 15590 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:03.585933 15408 raft_consensus.cc:2749] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:03.408882 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:03.617305 10869 raft_consensus.cc:2393] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 15: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:03.430505 10766 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:03.429714 10874 rpcz_store.cc:276] 0709 13:18:01.119906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:02.308255 (+1188349us) service_pool.cc:224] Handling call
0709 13:18:02.327496 (+ 19241us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:02.327825 (+   329us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:02.327865 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:02.466607 (+138742us) spinlock_profiling.cc:243] Waited 58.4 ms on lock 0x7b2c00029f28. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7f7aea11 00007f9c7f7be372 00007f9c80ef0a01 00007f9c80eb543a 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2
0709 13:18:02.506884 (+ 40277us) log.cc:844] Serialized 26 byte log entry
0709 13:18:02.971933 (+465049us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:02.995036 (+ 23103us) log.cc:844] Serialized 14 byte log entry
0709 13:18:02.995278 (+   242us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:02.995366 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:03.228732 (+233366us) raft_consensus.cc:1685] finished
0709 13:18:03.228837 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:03.228987 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7388,"prepare.queue_time_us":116433,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":31438,"raft.run_cpu_time_us":257,"raft.run_wall_time_us":42576,"spinlock_wait_cycles":134256384,"thread_start_us":352169,"threads_started":1,"wal-append.queue_time_us":617282}
W20260709 13:18:03.441092 10870 rpcz_store.cc:269] 0709 13:18:00.916705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.296283 (+379578us) service_pool.cc:224] Handling call
0709 13:18:03.240809 (+1944526us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b540037b254. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:03.241022 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3080770432}
I20260709 13:18:03.589689 15409 raft_consensus.cc:3055] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:03.433911 15518 raft_consensus.cc:2749] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:03.458757 15124 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:03.469457 14970 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:03.479045 10871 rpcz_store.cc:276] 0709 13:18:01.017493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:02.200468 (+1182975us) service_pool.cc:224] Handling call
0709 13:18:02.200911 (+   443us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:02.639022 (+438111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:03.265950 (+626928us) spinlock_profiling.cc:243] Waited 356 ms on lock 0x7b540009abd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:03.266197 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":818162048}
I20260709 13:18:03.497840 11507 rpcz_store.cc:276] 0709 13:18:02.193621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:02.193740 (+   119us) service_pool.cc:224] Handling call
0709 13:18:02.194071 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:02.194341 (+   270us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:02.194362 (+    21us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:02.202629 (+  8267us) log.cc:844] Serialized 26 byte log entry
0709 13:18:02.861605 (+658976us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:02.861646 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:02.861724 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:03.284324 (+422600us) raft_consensus.cc:1685] finished
0709 13:18:03.284359 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:03.284548 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":658604,"threads_started":1,"wal-append.queue_time_us":726818}
I20260709 13:18:03.507128 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:03.492040 15647 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:03.514758 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:03.548321 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:03.549247 10795 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 11.
I20260709 13:18:03.652362 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:03.508996 15190 consensus_queue.cc:1243] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer cb0428cc55984daba1ba63744a5102a2 log is divergent from this leader: its last log entry 13.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:18:03.560249 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:03.446699 15634 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:03.564054 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:03.699996 10798 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 12.
I20260709 13:18:03.442152 15637 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:03.576848 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:03.710695 10949 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:03.509706 15606 raft_consensus.cc:2749] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:03.739383 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:03.577996 10794 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:18:03.584174 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:03.578475 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:03.578171 10867 rpcz_store.cc:276] 0709 13:18:01.033072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:02.227962 (+1194890us) service_pool.cc:224] Handling call
0709 13:18:02.228251 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:03.310931 (+1082680us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:03.338110 (+ 27179us) spinlock_profiling.cc:243] Waited 965 ms on lock 0x7b54000f78d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:03.338392 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2218605824}
I20260709 13:18:03.579749 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:03.744236 10945 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 14.
I20260709 13:18:03.586645 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:03.745388 11509 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 12.
I20260709 13:18:03.586828 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:03.746959 11505 raft_consensus.cc:2410] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:18:03.587338 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:03.578158 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:03.509369 15222 raft_consensus.cc:2749] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:03.579104 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:03.756726 10950 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
I20260709 13:18:03.589668 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:03.513867 15501 raft_consensus.cc:2749] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:03.495110 15529 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:03.593540 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:03.510461 15574 consensus_queue.cc:1059] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.081s
I20260709 13:18:03.507457 15530 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:03.506908 15512 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:03.507094 15251 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:03.557353 15351 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:03.555732 15626 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:03.602465 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:03.517377 15274 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:03.407779 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:03.511044 15597 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:03.547593 15513 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:03.619275 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:03.245883 10868 raft_consensus.cc:1240] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 LEADER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:03.577287 15347 raft_consensus.cc:2749] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:03.443063 15478 raft_consensus.cc:740] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:18:03.622123 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 191742us
I20260709 13:18:03.809309 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:03.809880 10766 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:03.576112 15515 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:03.644133 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:03.647899 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:03.824405 10874 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 14.
I20260709 13:18:03.600589 15262 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:03.591446 15639 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:03.576663 15616 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:03.651387 10793 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 12.
I20260709 13:18:03.671423 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:03.837903 10871 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 12.
I20260709 13:18:03.672749 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:03.670692 14970 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:03.838610 10766 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:03.679284 10792 raft_consensus.cc:2410] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:18:03.680251 15596 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:03.652989 10795 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
I20260709 13:18:03.681953 15403 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:03.822609 15625 log.cc:927] Time spent T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:18:03.652962 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:03.688645 10801 raft_consensus.cc:2393] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 11: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
W20260709 13:18:03.690274 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:03.580080 15543 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:03.701009 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 256109us
I20260709 13:18:03.599499 15635 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:03.623916 15409 raft_consensus.cc:740] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:03.598048 15577 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:03.579787 15562 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:03.600400 15516 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:03.743518 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:03.640828 15643 raft_consensus.cc:2749] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:03.611446 15589 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:03.773085 10947 raft_consensus.cc:2410] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:18:03.601873 15320 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:03.764927 10915 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:03.611639 15641 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:03.787139 10875 raft_consensus.cc:2393] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 12: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:03.757997 15647 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:03.810549 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:03.786345 15351 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:03.815610 15251 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:03.815950 15530 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:03.817422 15513 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:03.791841 10796 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 11.
I20260709 13:18:03.817634 15512 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:03.818277 15447 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:03.773669 15429 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:03.819047 15651 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:03.819891 15529 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:03.809401 15649 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:03.820686 15478 consensus_queue.cc:260] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:03.830502 10797 raft_consensus.cc:1240] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:03.766412 15650 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:03.834046 10949 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 11.
I20260709 13:18:03.835618 10876 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
I20260709 13:18:03.861897 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:03.864521 10868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 688) took 3178 ms. Trace:
W20260709 13:18:03.685899 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 405061us
I20260709 13:18:03.777799 15500 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:03.861810 10798 raft_consensus.cc:1240] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:03.934082 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:04.082922 10874 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
I20260709 13:18:04.083796 10869 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:03.777338 15492 consensus_queue.cc:260] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:04.093060 10916 leader_election.cc:336] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Timed out: RequestConsensusVote RPC to 127.9.116.130:37895 timed out after 2.094s (SENT)
W20260709 13:18:03.977975 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 373204us
W20260709 13:18:04.093842 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:03.980931 10842 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:04.083928 10874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 1982) took 1128 ms. Trace:
I20260709 13:18:04.084479 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:03.935976 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:04.096606 10871 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 14: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:04.096786 10875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 2046) took 1340 ms. Trace:
I20260709 13:18:04.097335 10875 rpcz_store.cc:276] 0709 13:18:02.756417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:03.390530 (+634113us) service_pool.cc:224] Handling call
0709 13:18:04.096730 (+706200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:04.097370 10871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49290 (request call id 708) took 1087 ms. Trace:
I20260709 13:18:04.098265 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:03.840960 15262 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:03.916172 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:03.917586 10800 raft_consensus.cc:1275] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:18:04.086035 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 2042) took 1340 ms. Trace:
I20260709 13:18:04.140268 10867 rpcz_store.cc:276] 0709 13:18:02.745690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:03.743210 (+997520us) service_pool.cc:224] Handling call
0709 13:18:04.085965 (+342755us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:04.141594 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:04.085672 10868 rpcz_store.cc:276] 0709 13:18:00.707303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.864636 (+1157333us) service_pool.cc:224] Handling call
0709 13:18:03.245595 (+1380959us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:03.808789 (+563194us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:03.864322 (+ 55533us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b540037b250. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:03.864463 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3175110400}
I20260709 13:18:04.142905 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:04.002894 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:03.933229 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:04.087714 10870 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:03.807792 15660 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.127890 10875 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 9.
I20260709 13:18:04.146564 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:04.147224 10875 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 14.
I20260709 13:18:03.900298 15659 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.098016 10871 rpcz_store.cc:276] 0709 13:18:03.010372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:03.901426 (+891054us) service_pool.cc:224] Handling call
0709 13:18:04.097299 (+195873us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:04.094043 11477 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:03.866909 15642 raft_consensus.cc:2749] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:03.861434 15501 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:04.155381 10799 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 11.
I20260709 13:18:03.949808 15637 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:04.153213 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 1984) took 1195 ms. Trace:
I20260709 13:18:04.123392 10874 rpcz_store.cc:276] 0709 13:18:02.955680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:03.825082 (+869402us) service_pool.cc:224] Handling call
0709 13:18:04.083870 (+258788us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:04.157420 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:04.158420 15613 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:03.995230 15402 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:04.174576 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:18:04.174875 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 194054us
I20260709 13:18:04.175256 10875 raft_consensus.cc:2376] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 14. Re-sending same reply.
W20260709 13:18:04.165727 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 205544us
I20260709 13:18:03.880827 14987 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:04.179680 10842 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:03.792037 15626 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:04.168853 10949 raft_consensus.cc:1240] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:18:04.155534 10868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 1983) took 1186 ms. Trace:
I20260709 13:18:04.186864 10870 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
I20260709 13:18:04.187005 10868 rpcz_store.cc:276] 0709 13:18:02.956816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.142639 (+1185823us) service_pool.cc:224] Handling call
0709 13:18:04.143572 (+   933us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:04.187502 10870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 1991) took 1198 ms. Trace:
I20260709 13:18:04.092393 15653 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.188269 10946 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 11.
I20260709 13:18:04.173123 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:04.173833 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:04.170578 10867 rpcz_store.cc:276] 0709 13:18:02.957694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.141294 (+1183600us) service_pool.cc:224] Handling call
0709 13:18:04.153128 (+ 11834us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:04.177650 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:04.150385 15124 raft_consensus.cc:2804] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:04.168678 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1971) took 3204 ms. Trace:
I20260709 13:18:04.189642 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:04.177852 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:04.190822 10875 raft_consensus.cc:2393] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 15: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:04.188125 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:03.866604 15518 consensus_queue.cc:1059] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.306s
W20260709 13:18:04.182124 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 53575.9us
I20260709 13:18:04.193771 10916 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:04.194692 10842 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:04.165396 15478 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:04.197412 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:04.187897 10870 rpcz_store.cc:276] 0709 13:18:02.989514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:03.810310 (+820796us) service_pool.cc:224] Handling call
0709 13:18:04.187443 (+377133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:04.096539 15635 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:04.190039 10873 rpcz_store.cc:276] 0709 13:18:00.963793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:02.186594 (+1222801us) service_pool.cc:224] Handling call
0709 13:18:02.217853 (+ 31259us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:02.218287 (+   434us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:18:02.271459 (+ 53172us) log.cc:844] Serialized 14 byte log entry
0709 13:18:02.911950 (+640491us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:02.921077 (+  9127us) log.cc:844] Serialized 26 byte log entry
0709 13:18:02.921269 (+   192us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:02.921650 (+   381us) log.cc:844] Serialized 14 byte log entry
0709 13:18:02.921950 (+   300us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:02.922051 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:03.263396 (+341345us) spinlock_profiling.cc:243] Waited 2.87 ms on lock 0x7b1800389620. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc8522 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:18:03.282756 (+ 19360us) spinlock_profiling.cc:243] Waited 3.7 ms on lock 0x7b1800389620. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc8522 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:18:03.336243 (+ 53487us) spinlock_profiling.cc:243] Waited 3.12 ms on lock 0x7b1800389620. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc8522 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:18:04.168409 (+832166us) raft_consensus.cc:1685] finished
0709 13:18:04.168439 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:04.168614 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":53660,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":94,"raft.queue_time_us":590,"raft.run_cpu_time_us":489,"raft.run_wall_time_us":61227,"spinlock_wait_cycles":22293248,"thread_start_us":640013,"threads_started":1,"wal-append.queue_time_us":686651}
I20260709 13:18:03.902359 15596 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:04.199210 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:04.200099 10870 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 11.
I20260709 13:18:04.198148 10874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 2050) took 1440 ms. Trace:
I20260709 13:18:04.201251 10874 rpcz_store.cc:276] 0709 13:18:02.757579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.180674 (+1423095us) service_pool.cc:224] Handling call
0709 13:18:04.198087 (+ 17413us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:04.201440 10841 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:04.201577 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:04.202144 10874 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:18:04.096925 15429 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:04.206322 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:04.096428 15647 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:04.207415 10875 raft_consensus.cc:2393] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 12: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:18:04.198107 15631 consensus_meta.cc:287] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.752s	user 0.009s	sys 0.003s
I20260709 13:18:04.185479 15637 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:04.189885 15513 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:04.194005 15124 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:04.185935 15402 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:04.167248 15660 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:04.125473 15650 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:04.095938 15649 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:03.840936 15639 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:04.099517 15651 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:04.188656 15653 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:04.184481 15613 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:04.191152 14987 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:04.088819 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 387261us
I20260709 13:18:03.843212 10766 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:04.796074 15262 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:18:04.796953 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 953635us
I20260709 13:18:04.209735 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:04.209983 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:04.798769 15429 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:04.799643 10873 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:04.799903 10871 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
W20260709 13:18:04.801318 15652 log.cc:927] Time spent T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.705s	user 0.000s	sys 0.002s
I20260709 13:18:04.802124 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:04.541927 10875 raft_consensus.cc:1240] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:04.803196 15518 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:04.803869 10875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 701) took 2926 ms. Trace:
W20260709 13:18:04.804559 15663 log.cc:927] Time spent T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.826s	user 0.001s	sys 0.000s
I20260709 13:18:04.804390 10875 rpcz_store.cc:276] 0709 13:18:01.877256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.208736 (+2331480us) service_pool.cc:224] Handling call
0709 13:18:04.541216 (+332480us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:04.803486 (+262270us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:04.803747 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:04.806322 15347 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:04.806777 10875 raft_consensus.cc:1240] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:04.531523 10868 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:04.807226 15402 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:04.808444 11478 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.008s	sys 0.015s
W20260709 13:18:04.810403 15655 log.cc:927] Time spent T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.819s	user 0.000s	sys 0.000s
W20260709 13:18:04.499518 10794 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.002s	sys 0.003s
W20260709 13:18:04.811226 10794 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.899s	user 0.002s	sys 0.009s
W20260709 13:18:04.807408 10915 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.058s	sys 0.082s
I20260709 13:18:04.327741 15320 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:04.488248 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:04.813632 10870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 699) took 2936 ms. Trace:
I20260709 13:18:04.814683 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:04.498019 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:18:04.530683 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 329102us
I20260709 13:18:04.492406 10799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 1973) took 1174 ms. Trace:
I20260709 13:18:04.352258 15516 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:04.548163 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:04.572831 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:04.576465 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:04.169876 15659 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:04.193415 15512 raft_consensus.cc:2804] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:04.815583 15513 raft_consensus.cc:697] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:04.815986 10799 rpcz_store.cc:276] 0709 13:18:03.317445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:03.506848 (+189403us) service_pool.cc:224] Handling call
0709 13:18:04.491730 (+984882us) spinlock_profiling.cc:243] Waited 507 ms on lock 0x7b54003be2d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:04.492116 (+   386us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1166136192}
I20260709 13:18:04.820083 15512 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:04.820806 15512 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:04.821816 15659 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:04.812515 10794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2513) took 1506 ms. Trace:
I20260709 13:18:04.821539 15513 consensus_queue.cc:237] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:04.823779 10793 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 12.
W20260709 13:18:04.824622 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:04.825605 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:04.827364 10875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 702) took 2939 ms. Trace:
I20260709 13:18:04.827800 10875 rpcz_store.cc:276] 0709 13:18:01.877523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.805770 (+2928247us) service_pool.cc:224] Handling call
0709 13:18:04.806387 (+   617us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:04.816921 (+ 10534us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:04.817148 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:04.828061 15653 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:04.803252 10871 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 11.
I20260709 13:18:04.821566 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:04.831050 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:04.831139 10948 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:04.832419 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:04.833489 10799 raft_consensus.cc:2410] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:18:04.835507 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:04.837069 15651 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:04.837882 15631 raft_consensus.cc:2749] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:04.838042 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:04.839452 10872 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:04.826602 10793 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 11.
I20260709 13:18:04.840766 15639 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:04.841037 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:04.839109 10875 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:04.196003 15634 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:04.417903 15492 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:04.407873 15642 raft_consensus.cc:3055] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:04.845546 15642 raft_consensus.cc:740] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:04.846046 10841 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:04.204944 15605 raft_consensus.cc:2804] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:04.337472 15670 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:04.206641 15295 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.218192 15669 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.779234 15637 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:04.780361 10916 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.000s	sys 0.018s
W20260709 13:18:04.780727 10944 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.008s	sys 0.008s
W20260709 13:18:04.781023 15590 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.805s	user 0.002s	sys 0.011s
I20260709 13:18:04.781881 15684 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.350699 15641 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:04.783284 15667 log.cc:927] Time spent T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.555s	user 0.001s	sys 0.000s
I20260709 13:18:04.784186 15222 raft_consensus.cc:2804] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:04.784667 15124 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:18:04.785132 10913 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.797s	user 0.001s	sys 0.009s
W20260709 13:18:04.863144 10913 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.083s	user 0.004s	sys 0.011s
I20260709 13:18:04.786172 15660 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:04.786899 15616 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:04.258428 15562 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:04.787863 15589 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:04.864029 10874 raft_consensus.cc:606] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Received request to transfer leadership to TS e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:04.789809 15543 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:04.351574 15596 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:04.489673 10792 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 14: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
W20260709 13:18:04.810755 11478 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.008s	sys 0.016s
I20260709 13:18:04.516829 10800 raft_consensus.cc:2393] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 11: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:04.809036 14970 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:04.544366 10876 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 2048) took 1787 ms (client timeout 1556 ms). Trace:
I20260709 13:18:04.542548 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:04.877429 10874 raft_consensus.cc:614] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:04.209681 15274 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:04.814843 15613 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:18:04.499094 10796 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39442 (request call id 629) took 2141 ms (client timeout 1666 ms). Trace:
I20260709 13:18:04.496409 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:04.349776 15647 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:18:04.768903 11477 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.044s	sys 0.093s
W20260709 13:18:04.819429 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:04.823917 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:04.822379 10794 rpcz_store.cc:276] 0709 13:18:03.305758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:03.741177 (+435419us) service_pool.cc:224] Handling call
0709 13:18:03.781282 (+ 40105us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:03.781606 (+   324us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:03.781636 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:03.797424 (+ 15788us) log.cc:844] Serialized 26 byte log entry
0709 13:18:04.811888 (+1014464us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:04.811957 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:04.812122 (+   165us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:04.812200 (+    78us) raft_consensus.cc:1685] finished
0709 13:18:04.812212 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:04.812421 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":218,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":22169,"thread_start_us":899034,"threads_started":1,"wal-append.queue_time_us":688582}
I20260709 13:18:04.841996 10793 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 15.
I20260709 13:18:04.862843 15222 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:18:04.885764 11477 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.732s	user 0.045s	sys 0.093s
I20260709 13:18:04.834645 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:04.439368 15325 consensus_meta.cc:287] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.853s	user 0.013s	sys 0.000s
I20260709 13:18:04.798547 10766 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:04.815783 10915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.058s	sys 0.082s
I20260709 13:18:04.332795 15501 raft_consensus.cc:2749] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:04.350129 15626 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:04.271929 15407 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:04.275770 15530 consensus_queue.cc:1059] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.576s
I20260709 13:18:04.851591 15670 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:04.852154 15295 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:04.888613 14970 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:04.901450 15295 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:04.845928 15577 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:04.852821 10868 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
I20260709 13:18:04.210058 15656 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:04.259483 15515 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:04.891930 15639 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:04.400194 15409 consensus_queue.cc:260] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:04.858770 10916 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.000s	sys 0.019s
I20260709 13:18:04.810109 15402 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:04.258806 15597 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:04.783738 15650 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:04.780727 14987 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:04.861521 10944 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.009s	sys 0.008s
I20260709 13:18:04.787456 15351 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:04.861261 15590 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.052s	user 0.003s	sys 0.011s
W20260709 13:18:04.876840 10876 rpcz_store.cc:269] 0709 13:18:02.757021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:03.192712 (+435691us) service_pool.cc:224] Handling call
0709 13:18:04.543634 (+1350922us) spinlock_profiling.cc:243] Waited 526 ms on lock 0x7b5400078854. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:04.543972 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1210004352}
I20260709 13:18:04.807248 15347 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:04.445827 15574 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.014s	sys 0.010s
I20260709 13:18:04.930606 10799 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
W20260709 13:18:04.921871 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 1.15697e+06us
I20260709 13:18:04.887727 15605 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:04.880828 10801 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 12.
W20260709 13:18:04.893348 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:04.881611 10796 rpcz_store.cc:269] 0709 13:18:02.357444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:02.390205 (+ 32761us) service_pool.cc:224] Handling call
0709 13:18:04.498605 (+2108400us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b5400294d54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:04.498843 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2706667264}
W20260709 13:18:04.894958 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:04.896634 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:04.902395 10798 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 11.
I20260709 13:18:04.901872 14970 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:04.906730 15596 raft_consensus.cc:2764] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:18:04.905414 15684 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:04.889469 15222 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:04.854846 15642 consensus_queue.cc:260] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:04.351912 15635 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:04.912057 10792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2521) took 1010 ms. Trace:
I20260709 13:18:04.911192 15639 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:04.944383 10792 rpcz_store.cc:276] 0709 13:18:03.901989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:03.912123 (+ 10134us) service_pool.cc:224] Handling call
0709 13:18:04.903529 (+991406us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:04.944731 15639 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:04.925043 15656 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:04.936445 15662 log.cc:927] Time spent T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.896s	user 0.000s	sys 0.004s
I20260709 13:18:04.938280 15605 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:04.886174 15447 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:04.947554 15407 raft_consensus.cc:697] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 15 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:04.948030 15447 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:04.948971 15407 consensus_queue.cc:237] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:04.950877 10869 consensus_meta.cc:287] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.819s	user 0.006s	sys 0.017s
I20260709 13:18:04.951367 10869 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
W20260709 13:18:04.952003 10869 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 2049) took 2194 ms (client timeout 2016 ms). Trace:
I20260709 13:18:04.952510 15347 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:04.952669 10871 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:04.903440 15670 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:18:04.955917 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 762290us
I20260709 13:18:04.957093 10916 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:04.957362 15577 consensus_queue.cc:1059] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:18:04.958895 15577 raft_consensus.cc:3055] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:04.959028 10800 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
I20260709 13:18:04.959446 15577 raft_consensus.cc:740] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:04.960393 10842 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:04.961910 15684 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:04.962376 15351 raft_consensus.cc:2804] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:04.962852 15351 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:04.963254 15351 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:04.963985 15402 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:18:04.964335 15574 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.131s	user 0.024s	sys 0.011s
I20260709 13:18:04.964856 15402 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:04.960768 15577 consensus_queue.cc:260] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:04.966002 15325 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:04.933369 10767 connection.cc:256] RPC call timeout handler was delayed by -1.0732138410000971s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2191s for a response
I20260709 13:18:04.966830 15683 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.967586 15683 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:04.968592 15325 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:04.970018 15683 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:04.966831 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:04.974756 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:04.975642 10796 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 11.
I20260709 13:18:04.976696 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:04.977520 10796 raft_consensus.cc:2393] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 15: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:04.962009 15515 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:04.979187 15515 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:04.979628 15515 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:18:04.981896 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 887862us
W20260709 13:18:04.982982 15661 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.754s	user 0.001s	sys 0.007s
W20260709 13:18:04.983459 15661 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.810s	user 0.002s	sys 0.007s
I20260709 13:18:04.983755 15686 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:04.952350 10869 rpcz_store.cc:269] 0709 13:18:02.757319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:03.652712 (+895393us) service_pool.cc:224] Handling call
0709 13:18:04.951900 (+1299188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:04.984508 15686 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:04.985507 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:04.987560 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:04.942862 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:04.943660 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:04.958457 10916 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:04.944984 15642 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:04.966390 15637 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:04.966463 15577 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:04.970603 15649 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:04.972170 15402 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:04.972702 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 174505us
I20260709 13:18:04.973201 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:04.934760 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:04.940043 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 94186.2us
I20260709 13:18:04.988487 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:05.014567 10873 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 14.
I20260709 13:18:04.947556 15656 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:04.990203 11477 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:04.991036 10874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:53748 (request call id 39) took 1410 ms. Trace:
I20260709 13:18:04.991256 15686 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:04.929666 15295 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.950695 15447 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:04.994891 15669 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:05.000000 15409 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:05.004558 10869 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
I20260709 13:18:04.931097 10948 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 14.
W20260709 13:18:05.016146 15190 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.726s	user 0.013s	sys 0.013s
W20260709 13:18:05.042718 15190 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.834s	user 0.013s	sys 0.013s
I20260709 13:18:05.044023 15512 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:05.046092 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:18:05.049410 15478 consensus_meta.cc:287] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.845s	user 0.006s	sys 0.006s
I20260709 13:18:05.052100 15645 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:05.052657 15645 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:05.053196 15645 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:05.054482 15669 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:18:05.055871 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:05.058919 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:05.059321 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:05.060465 10950 raft_consensus.cc:2410] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 12 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 4.
I20260709 13:18:05.060727 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:05.061780 15325 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:05.030997 10874 rpcz_store.cc:276] 0709 13:18:03.580784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.202029 (+621245us) service_pool.cc:224] Handling call
0709 13:18:04.990770 (+788741us) spinlock_profiling.cc:243] Waited 647 ms on lock 0x7b540009abd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eaf853 00007f9c85e7d60a 00007f9c7f4037d4 00007f9c7f403762 00007f9c7f403691 00007f9c7f40360c 00007f9c7f4028c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:04.990932 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1488583552}
I20260709 13:18:05.064030 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:05.019393 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:05.020917 11503 raft_consensus.cc:1240] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:05.021894 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:05.074750 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:05.022401 15605 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:05.091863 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:05.023044 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:05.023530 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:05.024083 10798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1976) took 1705 ms. Trace:
I20260709 13:18:05.024379 10799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1977) took 1705 ms. Trace:
I20260709 13:18:05.113050 10799 rpcz_store.cc:276] 0709 13:18:03.318638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.023248 (+1704610us) service_pool.cc:224] Handling call
0709 13:18:05.023698 (+   450us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:05.024022 (+   324us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:05.024053 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:05.024095 (+    42us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:05.024139 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:05.024194 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:05.024293 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:05.024204 15222 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:05.025169 10797 raft_consensus.cc:2393] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 13: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:05.027199 15192 raft_consensus.cc:3055] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:05.027572 10792 raft_consensus.cc:2393] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 14: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
W20260709 13:18:05.031375 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 73161.5us
I20260709 13:18:05.031870 10796 raft_consensus.cc:1275] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:18:05.035051 10872 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
I20260709 13:18:05.034847 10871 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 14.
I20260709 13:18:05.066046 10800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2522) took 1157 ms. Trace:
I20260709 13:18:05.118443 10800 rpcz_store.cc:276] 0709 13:18:03.908918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.064932 (+1156014us) service_pool.cc:224] Handling call
0709 13:18:05.065366 (+   434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:05.065636 (+   270us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:05.065665 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:05.065710 (+    45us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:05.065790 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:05.065859 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:05.065980 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:05.052634 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:05.120483 10873 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 11.
I20260709 13:18:05.068029 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:05.069428 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:05.070852 11504 raft_consensus.cc:1275] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:18:05.072085 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:04.814105 10870 rpcz_store.cc:276] 0709 13:18:01.876701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.206263 (+2329562us) service_pool.cc:224] Handling call
0709 13:18:04.206564 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:04.206854 (+   290us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:18:04.531579 (+324725us) log.cc:844] Serialized 14 byte log entry
0709 13:18:04.812958 (+281379us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:04.813069 (+   111us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:04.813180 (+   111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:04.813304 (+   124us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:04.813447 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":64145,"prepare.queue_time_us":388702,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":5123,"thread_start_us":92749,"threads_started":1,"wal-append.queue_time_us":284023}
I20260709 13:18:05.018154 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:05.044431 14970 consensus_queue.cc:1048] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:18:05.092291 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:05.094770 10942 raft_consensus.cc:1240] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:05.098528 10945 raft_consensus.cc:1240] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 11. Current term is 12. Ops: []
W20260709 13:18:05.075232 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 45059.5us
I20260709 13:18:05.086012 10794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2523) took 1175 ms. Trace:
I20260709 13:18:05.124905 10794 rpcz_store.cc:276] 0709 13:18:03.910301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.084563 (+1174262us) service_pool.cc:224] Handling call
0709 13:18:05.085098 (+   535us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:05.085563 (+   465us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:05.085603 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:05.085662 (+    59us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:05.085705 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:05.085781 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:05.085930 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:05.075469 11508 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 11.
I20260709 13:18:05.112584 10798 rpcz_store.cc:276] 0709 13:18:03.318340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.017634 (+1699294us) service_pool.cc:224] Handling call
0709 13:18:05.018161 (+   527us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:05.018511 (+   350us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:05.023667 (+  5156us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:05.023718 (+    51us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:05.023776 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:05.023871 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:05.024004 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:05.096169 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:05.127794 10793 raft_consensus.cc:1240] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:05.093369 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:05.129060 10796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1971) took 1801 ms. Trace:
I20260709 13:18:05.133093 10796 rpcz_store.cc:276] 0709 13:18:03.316826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.978143 (+1661317us) service_pool.cc:224] Handling call
0709 13:18:04.978568 (+   425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:05.117956 (+139388us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:05.118267 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:05.135428 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:05.092960 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:05.155972 10873 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 12.
I20260709 13:18:05.087179 15647 raft_consensus.cc:3055] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:05.160373 10873 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:18:05.161216 10873 raft_consensus.cc:606] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Received request to transfer leadership to TS e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:05.161614 10873 raft_consensus.cc:614] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:05.161756 14970 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:05.162173 11478 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:05.163094 10876 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 12.
I20260709 13:18:05.164601 10876 raft_consensus.cc:1217] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 LEADER]: Deduplicated request from leader. Original: 7.8->[9.9-9.9]   Dedup: 9.9->[]
W20260709 13:18:05.164772 15612 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.754s	user 0.035s	sys 0.050s
I20260709 13:18:05.165143 10876 raft_consensus.cc:1240] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 LEADER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
W20260709 13:18:05.165246 15612 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.014s	user 0.035s	sys 0.050s
I20260709 13:18:05.165931 10876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 703) took 3134 ms. Trace:
I20260709 13:18:05.167097 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:05.168644 10870 raft_consensus.cc:1240] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:05.168459 15670 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:05.169317 10870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 704) took 2977 ms. Trace:
I20260709 13:18:05.137249 15325 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:05.169679 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 143451us
I20260709 13:18:05.171921 10841 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:05.174005 10916 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:05.174803 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:05.175163 15590 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:05.175897 15590 raft_consensus.cc:697] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 13 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:05.176365 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:05.177209 11504 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
I20260709 13:18:05.177321 15590 consensus_queue.cc:237] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:05.178256 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:05.178869 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:05.180236 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:05.166317 10876 rpcz_store.cc:276] 0709 13:18:02.031621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.163959 (+3132338us) service_pool.cc:224] Handling call
0709 13:18:05.164393 (+   434us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:05.165635 (+  1242us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:05.165843 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:05.169695 10870 rpcz_store.cc:276] 0709 13:18:02.192141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.168087 (+2975946us) service_pool.cc:224] Handling call
0709 13:18:05.168435 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:05.169073 (+   638us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:05.169262 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:05.182801 10943 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 11.
I20260709 13:18:05.182926 11506 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 15.
I20260709 13:18:05.183087 15513 raft_consensus.cc:2804] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:05.123587 15192 raft_consensus.cc:740] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:05.184720 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:05.156970 11507 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 12.
W20260709 13:18:05.182777 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 52188.7us
I20260709 13:18:05.163139 10874 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 12.
I20260709 13:18:05.183939 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:05.170074 15325 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:05.184158 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:18:05.187959 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 220301us
I20260709 13:18:05.077559 15124 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:05.194283 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:05.194412 10869 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 15.
I20260709 13:18:05.191592 11510 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 11.
I20260709 13:18:05.077826 15295 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:05.195456 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:05.162454 14970 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:05.195353 11511 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
I20260709 13:18:05.200359 11504 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
I20260709 13:18:05.189358 10947 raft_consensus.cc:2376] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15. Re-sending same reply.
I20260709 13:18:05.197741 10842 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:05.201849 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:05.202687 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:05.199813 10946 raft_consensus.cc:2393] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 13: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:05.203627 15641 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:05.204422 15641 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:05.207916 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 1998) took 1027 ms. Trace:
I20260709 13:18:05.207921 10874 raft_consensus.cc:1240] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:05.208379 10867 rpcz_store.cc:276] 0709 13:18:04.180206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.189398 (+  9192us) service_pool.cc:224] Handling call
0709 13:18:05.207533 (+1018135us) spinlock_profiling.cc:243] Waited 907 ms on lock 0x7b54002f0654. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:05.207722 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2086365568}
I20260709 13:18:05.209108 10874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 706) took 2447 ms. Trace:
I20260709 13:18:05.209504 10874 rpcz_store.cc:276] 0709 13:18:02.762071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.194406 (+2432335us) service_pool.cc:224] Handling call
0709 13:18:05.207538 (+ 13132us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:05.208857 (+  1319us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:05.209053 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:05.210263 10948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41020 (request call id 2480) took 1036 ms. Trace:
I20260709 13:18:05.210506 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:05.210708 10948 rpcz_store.cc:276] 0709 13:18:04.173412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.173537 (+   125us) service_pool.cc:224] Handling call
0709 13:18:05.210024 (+1036487us) spinlock_profiling.cc:243] Waited 642 ms on lock 0x7b5400173754. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:05.210204 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1476147328}
I20260709 13:18:05.211422 11509 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10.
I20260709 13:18:05.195127 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:05.211027 11512 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 13.
I20260709 13:18:05.212357 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:05.082880 15645 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:05.200342 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:05.213188 10950 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
W20260709 13:18:05.217051 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 45274.5us
I20260709 13:18:05.185712 15647 raft_consensus.cc:740] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:05.218469 10869 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:18:05.220554 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:05.221850 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:05.222620 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:05.216595 10916 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:05.226600 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:05.226918 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:05.227384 15674 raft_consensus.cc:2804] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:05.228215 15492 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:05.228998 15518 raft_consensus.cc:2749] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:05.229053 15124 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:05.205111 15351 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:05.235466 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:05.236835 14970 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:05.243010 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:05.199558 15684 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:05.220196 10766 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:05.272085 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:05.248024 10942 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
I20260709 13:18:05.198977 15513 raft_consensus.cc:697] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:05.250573 10948 raft_consensus.cc:2393] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 12: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:05.249524 10842 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:05.255405 10943 raft_consensus.cc:2393] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 13: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:05.270462 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:05.260855 15492 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:05.261827 11503 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
I20260709 13:18:05.272941 10947 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 13.
W20260709 13:18:05.259003 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 59262.5us
I20260709 13:18:05.251653 10869 raft_consensus.cc:606] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Received request to transfer leadership to TS e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:05.297804 10869 raft_consensus.cc:614] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:05.225327 15192 consensus_queue.cc:260] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:05.270583 10945 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 14: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:18:05.257618 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 59797.7us
W20260709 13:18:05.311159 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:05.311349 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:05.294126 10841 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:05.314451 15501 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:05.257918 15612 raft_consensus.cc:2804] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:05.314939 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:05.256146 15674 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:05.324154 10950 raft_consensus.cc:2410] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:18:05.324085 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:05.321367 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:05.322248 11503 raft_consensus.cc:1240] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:05.319610 10946 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 13.
I20260709 13:18:05.274689 15684 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:05.336385 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:05.312454 15407 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:05.338377 10793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1978) took 2003 ms. Trace:
I20260709 13:18:05.327800 10949 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 11.
I20260709 13:18:05.323817 15674 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:05.335682 11478 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
W20260709 13:18:05.371176 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 156889us
I20260709 13:18:05.366909 10941 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10.
I20260709 13:18:05.325371 15612 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:05.379994 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 154359us
I20260709 13:18:05.381963 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 705) took 2663 ms. Trace:
I20260709 13:18:05.341459 11507 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 14: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:05.370092 10943 raft_consensus.cc:1240] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:05.382383 10873 rpcz_store.cc:276] 0709 13:18:02.678003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.197348 (+2519345us) service_pool.cc:224] Handling call
0709 13:18:05.210117 (+ 12769us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:05.229638 (+ 19521us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:05.229683 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:05.230412 (+   729us) log.cc:844] Serialized 52 byte log entry
0709 13:18:05.254421 (+ 24009us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:05.254472 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:05.254544 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:05.340946 (+ 86402us) raft_consensus.cc:1685] finished
0709 13:18:05.340966 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:05.341117 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5607,"raft.queue_time_us":11559,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":6712,"thread_start_us":23521,"threads_started":1,"wal-append.queue_time_us":23984}
I20260709 13:18:05.341550 10793 rpcz_store.cc:276] 0709 13:18:03.327696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.046534 (+1718838us) service_pool.cc:224] Handling call
0709 13:18:05.127529 (+ 80995us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:05.128369 (+   840us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:05.330212 (+201843us) spinlock_profiling.cc:243] Waited 78 ms on lock 0x7b54002116d0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:05.330377 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":179307776}
I20260709 13:18:05.334129 15684 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:18:05.344578 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:05.392385 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 56794.8us
I20260709 13:18:05.342417 10799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1987) took 1249 ms. Trace:
I20260709 13:18:05.387718 10795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2518) took 1500 ms. Trace:
I20260709 13:18:05.379279 15641 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:05.366179 15596 raft_consensus.cc:2749] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:05.430063 11478 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:05.366708 15635 raft_consensus.cc:2749] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:05.436110 11503 raft_consensus.cc:1240] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 15 LEADER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
W20260709 13:18:05.431838 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 175962us
I20260709 13:18:05.369166 15351 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:05.439564 10870 raft_consensus.cc:1240] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 13. Current term is 14. Ops: [11.6-13.7]
I20260709 13:18:05.439790 11508 raft_consensus.cc:1275] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:18:05.367566 15645 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:05.445914 10874 raft_consensus.cc:1240] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:05.388269 10944 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
W20260709 13:18:05.434659 10875 consensus_meta.cc:287] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.567s	user 0.009s	sys 0.002s
W20260709 13:18:05.436844 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 53244.9us
I20260709 13:18:05.438939 15320 consensus_queue.cc:1048] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 13:18:05.443568 15192 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:05.390954 15478 raft_consensus.cc:2804] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:05.391045 15612 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:05.481432 10873 raft_consensus.cc:1240] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:05.449070 10875 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 17.
W20260709 13:18:05.449172 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 61207.2us
I20260709 13:18:05.456296 10950 raft_consensus.cc:1240] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 14. Current term is 15. Ops: [12.12-14.13]
I20260709 13:18:05.467638 10874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1981) took 2513 ms. Trace:
I20260709 13:18:05.427213 15190 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:05.469126 10945 raft_consensus.cc:1240] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:05.477694 10870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2044) took 2711 ms. Trace:
I20260709 13:18:05.486307 10870 rpcz_store.cc:276] 0709 13:18:02.746359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.331891 (+2585532us) service_pool.cc:224] Handling call
0709 13:18:05.430380 (+ 98489us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:05.457817 (+ 27437us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:05.458159 (+   342us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:05.443881 15407 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:05.444033 15642 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:05.482163 10916 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:05.380846 15501 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:05.482177 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 707) took 2473 ms. Trace:
I20260709 13:18:05.493803 10873 rpcz_store.cc:276] 0709 13:18:03.008909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.399465 (+2390556us) service_pool.cc:224] Handling call
0709 13:18:05.481263 (+ 81798us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:05.481920 (+   657us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:05.482110 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:05.456461 15222 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:05.367640 15513 consensus_queue.cc:237] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:05.434870 10795 rpcz_store.cc:276] 0709 13:18:03.887477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.055409 (+1167932us) service_pool.cc:224] Handling call
0709 13:18:05.055965 (+   556us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:05.056445 (+   480us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:18:05.156253 (+ 99808us) log.cc:844] Serialized 14 byte log entry
0709 13:18:05.330685 (+174432us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:05.330738 (+    53us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:05.386935 (+ 56197us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:05.387217 (+   282us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:05.387627 (+   410us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11099,"prepare.queue_time_us":144012,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":3285,"raft.queue_time_us":56846,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":47542,"thread_start_us":186233,"threads_started":2}
I20260709 13:18:05.489904 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:05.483379 10874 rpcz_store.cc:276] 0709 13:18:02.954069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.339390 (+2385321us) service_pool.cc:224] Handling call
0709 13:18:05.445681 (+106291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:05.467286 (+ 21605us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:05.467545 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:05.477352 15478 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:05.487787 10869 raft_consensus.cc:1240] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 16. Current term is 17. Ops: []
W20260709 13:18:05.502678 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 125345us
I20260709 13:18:05.492280 10767 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:05.505049 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 68098.6us
I20260709 13:18:05.485546 15686 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:05.505460 10841 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:05.362201 15647 consensus_queue.cc:260] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:05.488301 15190 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:05.500394 10875 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:18:05.517071 11508 raft_consensus.cc:1275] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:18:05.517554 11478 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:05.505417 15478 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:05.524003 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2056) took 2236 ms. Trace:
W20260709 13:18:05.524331 10872 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 2045) took 2777 ms (client timeout 1946 ms). Trace:
I20260709 13:18:05.524452 10873 rpcz_store.cc:276] 0709 13:18:03.283557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.494875 (+2211318us) service_pool.cc:224] Handling call
0709 13:18:05.501602 (+  6727us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:05.515249 (+ 13647us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:05.515281 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:05.520037 (+  4756us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:05.520107 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:05.520185 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:05.520317 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:05.526057 10916 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:18:05.530656 15515 consensus_meta.cc:287] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.551s	user 0.007s	sys 0.005s
I20260709 13:18:05.530944 15320 raft_consensus.cc:2749] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:05.533059 15320 consensus_queue.cc:1059] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:05.533402 15647 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:05.534373 15320 raft_consensus.cc:3055] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:18:05.535101 15692 log.cc:927] Time spent T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
W20260709 13:18:05.535326 15697 log.cc:927] Time spent T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 13:18:05.531177 15515 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:05.538880 10871 raft_consensus.cc:1275] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 13:18:05.517107 15645 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:18:05.539674 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 52198.6us
I20260709 13:18:05.541323 15492 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:05.544281 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:05.546928 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:05.513947 15686 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:05.547227 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:05.523839 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:18:05.656457 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 142705us
I20260709 13:18:05.652649 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:05.547822 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:05.547974 15577 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:18:05.548938 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:05.660902 11509 raft_consensus.cc:2393] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 15: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:05.661103 11506 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 14: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:05.550424 10876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2051) took 2786 ms. Trace:
I20260709 13:18:05.552418 10874 raft_consensus.cc:1275] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:18:05.552248 10915 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:05.553014 15124 consensus_queue.cc:1048] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:18:05.553939 15409 consensus_queue.cc:1059] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:18:05.555078 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:18:05.524850 10872 rpcz_store.cc:269] 0709 13:18:02.755939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:03.593321 (+837382us) service_pool.cc:224] Handling call
0709 13:18:05.519953 (+1926632us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b5400078d54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:05.524265 (+  4312us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2470248832}
I20260709 13:18:05.523048 10875 raft_consensus.cc:606] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Received request to transfer leadership to TS e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:05.662030 10876 rpcz_store.cc:276] 0709 13:18:02.764102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.549394 (+2785292us) service_pool.cc:224] Handling call
0709 13:18:05.549750 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:05.550037 (+   287us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:05.550061 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:05.550117 (+    56us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:05.550152 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:05.550211 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:05.550337 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:05.668366 10875 raft_consensus.cc:614] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:05.434878 10799 rpcz_store.cc:276] 0709 13:18:04.092591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.159032 (+1066441us) service_pool.cc:224] Handling call
0709 13:18:05.213016 (+ 53984us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:05.235084 (+ 22068us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:18:05.243106 (+  8022us) log.cc:844] Serialized 14 byte log entry
0709 13:18:05.341992 (+ 98886us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:05.342052 (+    60us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:05.342114 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:05.342210 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:05.342345 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43253,"prepare.queue_time_us":8321,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":19825,"raft.queue_time_us":79029,"spinlock_wait_cycles":6444288,"thread_start_us":57386,"threads_started":1,"wal-append.queue_time_us":84754}
I20260709 13:18:05.637670 10867 raft_consensus.cc:1240] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 14. Current term is 15. Ops: [11.6-14.8]
I20260709 13:18:05.556749 11477 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:18:05.639202 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 91960.5us
I20260709 13:18:05.517377 15190 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:05.630362 15320 raft_consensus.cc:740] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:05.641921 10871 pending_rounds.cc:85] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 7
I20260709 13:18:05.505096 15642 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:05.663844 11478 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:05.666805 11512 raft_consensus.cc:1275] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:18:05.666999 11503 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
I20260709 13:18:05.667361 10874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2057) took 2383 ms. Trace:
I20260709 13:18:05.673410 10874 rpcz_store.cc:276] 0709 13:18:03.283829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.501355 (+2217526us) service_pool.cc:224] Handling call
0709 13:18:05.552061 (+ 50706us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:05.662112 (+110051us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:05.667295 (+  5183us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:05.649622 15645 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:05.654140 15295 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:05.638298 15347 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:05.694512 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 137855us
I20260709 13:18:05.670343 10871 pending_rounds.cc:107] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 13:18:05.660674 15686 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:05.701758 10871 raft_consensus.cc:2889] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:05.660935 15577 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:05.710386 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 709) took 1970 ms. Trace:
I20260709 13:18:05.712575 10873 rpcz_store.cc:276] 0709 13:18:03.739620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.630104 (+1890484us) service_pool.cc:224] Handling call
0709 13:18:05.689821 (+ 59717us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:05.702200 (+ 12379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:05.702247 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:05.709989 (+  7742us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:05.710054 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:05.710141 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:05.710307 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:05.662994 15222 raft_consensus.cc:2749] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:18:05.720523 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 230275us
I20260709 13:18:05.715384 10767 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:05.717262 10916 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:05.722867 11477 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:05.714409 15492 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:05.724349 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1980) took 2763 ms. Trace:
I20260709 13:18:05.724449 15492 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:05.724833 10869 rpcz_store.cc:276] 0709 13:18:02.952687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.327971 (+2375284us) service_pool.cc:224] Handling call
0709 13:18:05.483328 (+155357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:05.502277 (+ 18949us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:05.715764 (+213487us) spinlock_profiling.cc:243] Waited 51.2 ms on lock 0x7b540006d6d0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:05.715911 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":117854464}
I20260709 13:18:05.725561 15642 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:05.724990 15492 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:05.724395 15447 raft_consensus.cc:3055] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:05.727135 15447 raft_consensus.cc:740] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:05.728176 15447 consensus_queue.cc:260] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:05.728981 15325 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:05.729470 15447 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:05.730994 15700 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:05.731662 15320 consensus_queue.cc:260] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:05.733044 15320 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:05.737273 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:05.740272 10875 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:53748
I20260709 13:18:05.695860 15402 raft_consensus.cc:2804] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:05.702381 15645 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:05.736846 11504 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 11.
W20260709 13:18:05.744757 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 47644.1us
I20260709 13:18:05.716928 15577 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:18:05.746194 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:05.771344 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:05.753975 10801 raft_consensus.cc:1275] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:18:05.757066 10876 raft_consensus.cc:1240] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:05.775547 10872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 710) took 1943 ms. Trace:
I20260709 13:18:05.776113 10872 rpcz_store.cc:276] 0709 13:18:03.832270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.694117 (+1861847us) service_pool.cc:224] Handling call
0709 13:18:05.714921 (+ 20804us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:05.775068 (+ 60147us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:05.775111 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:05.775171 (+    60us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:05.775236 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:05.775316 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:05.775474 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:05.780076 15478 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:05.782078 15222 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:05.783018 15222 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:05.785187 10945 raft_consensus.cc:1275] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 13:18:05.786353 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 113469us
I20260709 13:18:05.787102 15501 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:05.787462 10767 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:05.786388 15124 raft_consensus.cc:2804] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:05.788739 15325 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:05.788990 15124 raft_consensus.cc:697] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 14 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:05.745767 10875 raft_consensus.cc:606] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Received request to transfer leadership to TS cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:05.789767 10875 raft_consensus.cc:614] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:05.791561 15222 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:05.791857  9682 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:18:05.819159  9682 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:05.782511 15478 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:05.795655 15407 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:05.790230 15124 consensus_queue.cc:237] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:05.823784 15596 raft_consensus.cc:2749] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:05.799541 10916 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:05.824549 15596 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:05.747869 15492 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:05.825110 15596 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:05.827486 15492 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:05.828261 15512 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:05.736694 15347 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:05.731659 15700 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:05.744102 15402 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:05.807199 10876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2060) took 1827 ms. Trace:
W20260709 13:18:05.837033 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 49705.2us
I20260709 13:18:05.837405 10876 rpcz_store.cc:276] 0709 13:18:03.980272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.693903 (+1713631us) service_pool.cc:224] Handling call
0709 13:18:05.756822 (+ 62919us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:05.806905 (+ 50083us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:05.807133 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:05.838469 10871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1990) took 2875 ms. Trace:
I20260709 13:18:05.753774 15683 raft_consensus.cc:2804] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:05.810216 15515 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:05.789215 15325 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:05.838941 10871 rpcz_store.cc:276] 0709 13:18:02.962634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.335776 (+2373142us) service_pool.cc:224] Handling call
0709 13:18:05.389268 (+ 53492us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:05.735908 (+346640us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:05.838204 (+102296us) spinlock_profiling.cc:243] Waited 95.9 ms on lock 0x7b54000f64d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:05.838400 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":220468608}
I20260709 13:18:05.843302 15501 raft_consensus.cc:2804] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:05.844012 10767 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:05.834223 15639 consensus_meta.cc:287] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.889s	user 0.014s	sys 0.000s
W20260709 13:18:05.846760 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 68584.4us
I20260709 13:18:05.832767 15347 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:05.850123 10794 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.007s	sys 0.016s
I20260709 13:18:05.817900 15684 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:05.845829 15700 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:05.841485 15402 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:05.844265 15683 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:05.851481 10794 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.008s	sys 0.016s
I20260709 13:18:05.848519 15325 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:05.871356 10796 raft_consensus.cc:1275] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:18:05.872998 15515 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:05.855185 14970 raft_consensus.cc:2804] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:05.848311 15501 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:05.873273 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:05.874415 11510 raft_consensus.cc:1240] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 10. Current term is 12. Ops: []
I20260709 13:18:05.874171 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:05.879518 10795 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:05.875947 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:05.851157 15674 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:05.884702 10793 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 14.
I20260709 13:18:05.863258 15683 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:05.887792 10868 raft_consensus.cc:1240] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 11. Current term is 12. Ops: []
W20260709 13:18:05.891543 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:05.895862 15670 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:05.891444 15674 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:05.850770 15639 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:05.882366 15501 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:05.877887 15515 raft_consensus.cc:697] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 13 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:05.877022 14970 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:05.903872 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:05.941968 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:05.946966 10793 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:05.894297 15612 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:05.942516 10767 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:05.899387 15596 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:05.973241 10948 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.018s	sys 0.048s
W20260709 13:18:05.973658 10948 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.018s	sys 0.048s
I20260709 13:18:05.948589 14970 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:05.981716 10795 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 16.
I20260709 13:18:05.996698 15686 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:05.998097 11507 raft_consensus.cc:2368] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 12. Current term is 13.
I20260709 13:18:05.999182 15686 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:05.999928 15686 consensus_queue.cc:1059] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:05.988948 15596 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:05.983733 15605 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:06.008033 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 107453us
I20260709 13:18:05.987697 15190 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:06.036943 11506 raft_consensus.cc:1240] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 13:18:06.041877 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:06.042593 10942 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
I20260709 13:18:06.026505 15596 raft_consensus.cc:697] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:06.045583 10767 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:06.046566 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 494385us
W20260709 13:18:06.055425 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 343494us
I20260709 13:18:06.060235 11477 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:06.061388 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 262075us
I20260709 13:18:06.055924 15642 raft_consensus.cc:2749] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:06.071732 10915 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:06.039124 15515 consensus_queue.cc:237] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:06.074123 11477 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:06.055418 15684 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:06.084405 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:06.074671 15639 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:06.096926 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:06.098959 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:06.099620 10944 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 11.
I20260709 13:18:06.102675 15577 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:06.106891 15684 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:06.085621 15612 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:06.112246 10767 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:06.109759 15402 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:06.121593 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1992) took 3131 ms. Trace:
I20260709 13:18:06.153431 10867 rpcz_store.cc:276] 0709 13:18:02.990435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.386317 (+2395882us) service_pool.cc:224] Handling call
0709 13:18:05.630232 (+243915us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:18:05.672306 (+ 42074us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:06.119524 (+447218us) spinlock_profiling.cc:243] Waited 169 ms on lock 0x7b5400078d50. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:06.121529 (+  2005us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":388221184}
I20260709 13:18:06.135299 15409 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:18:06.150285 15647 consensus_meta.cc:287] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.602s	user 0.012s	sys 0.000s
W20260709 13:18:06.161847 10841 connection.cc:256] RPC call timeout handler was delayed by -1.745452226999987s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22210000000000002s for a response
I20260709 13:18:06.093992 15124 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:06.119746 10948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 2117) took 1022 ms. Trace:
W20260709 13:18:06.122812 15703 log.cc:927] Time spent T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:18:06.184934 10942 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 11.
I20260709 13:18:06.097692 15712 raft_consensus.cc:3055] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:06.166307 10870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1994) took 3132 ms. Trace:
I20260709 13:18:06.166657 15717 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:18:06.163293 10766 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:18:06.180500 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 89376.7us
I20260709 13:18:06.183655 10948 rpcz_store.cc:276] 0709 13:18:05.097652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.341718 (+244066us) service_pool.cc:224] Handling call
0709 13:18:05.389763 (+ 48045us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:05.428581 (+ 38818us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:18:05.977720 (+549139us) log.cc:844] Serialized 14 byte log entry
0709 13:18:06.113136 (+135416us) log.cc:844] Serialized 14 byte log entry
0709 13:18:06.119190 (+  6054us) log.cc:844] Serialized 14 byte log entry
0709 13:18:06.119388 (+   198us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:06.119448 (+    60us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:06.119512 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:06.119580 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:06.119680 (+   100us) inbound_call.cc:177] Queueing success response
0709 13:18:06.176197 (+ 56517us) spinlock_profiling.cc:243] Waited 5.69 ms on lock 0x7b180044fc20. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc83f3 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
Metrics: {"mutex_wait_us":21734,"prepare.queue_time_us":187745,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":92,"raft.queue_time_us":16825,"raft.run_cpu_time_us":221,"raft.run_wall_time_us":224,"spinlock_wait_cycles":13075712,"thread_start_us":614724,"threads_started":2}
W20260709 13:18:06.163728 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 51694.6us
I20260709 13:18:06.121309 10793 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 12.
I20260709 13:18:06.193738 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:06.183396 15124 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:18:06.195539 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 82794.8us
I20260709 13:18:06.195820 15124 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:06.152320 15596 consensus_queue.cc:237] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:06.196324 10766 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:06.197547 10766 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:06.188722 15712 raft_consensus.cc:740] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:06.200024 15712 consensus_queue.cc:260] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:06.201910 15712 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:06.203003 15709 consensus_queue.cc:1059] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:06.186064 10870 rpcz_store.cc:276] 0709 13:18:03.034029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.497210 (+2463181us) service_pool.cc:224] Handling call
0709 13:18:05.498229 (+  1019us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:05.651831 (+153602us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:05.651880 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:05.660707 (+  8827us) log.cc:844] Serialized 26 byte log entry
0709 13:18:05.987902 (+327195us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:05.987946 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:05.988023 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:06.166046 (+178023us) raft_consensus.cc:1685] finished
0709 13:18:06.166073 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:06.166235 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":120679,"raft.queue_time_us":137762,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":28427,"thread_start_us":278992,"threads_started":1,"wal-append.queue_time_us":395553}
I20260709 13:18:06.203029 15320 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:06.206204 15642 raft_consensus.cc:2804] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:06.206354 15320 raft_consensus.cc:697] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:06.206668 15642 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:06.207398 10950 raft_consensus.cc:2393] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 16: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:06.209512 10874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1995) took 2030 ms. Trace:
W20260709 13:18:06.178241 15704 log.cc:927] Time spent T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.165s	user 0.001s	sys 0.001s
I20260709 13:18:06.213997 15683 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:06.216475 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:06.218627 15402 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:06.203389 15708 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:06.210845 15642 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:06.214475 15597 raft_consensus.cc:2804] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:06.208979 11511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56416 (request call id 796) took 1048 ms. Trace:
W20260709 13:18:06.221494 10841 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.116.132:40555 timed out after 1.519s (SENT) [suppressed 16 similar messages]
I20260709 13:18:06.223807 15708 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:06.227748 15708 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:06.227880 10767 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:06.228878 10767 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:06.229790 10767 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:06.215060 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:18:06.209609 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 138019us
I20260709 13:18:06.222184 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:06.231420 10915 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:06.232376 10915 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:06.232604 15713 raft_consensus.cc:2804] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:06.233053 15713 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:06.233327 14970 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:06.211158 10874 rpcz_store.cc:276] 0709 13:18:04.179382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.704266 (+1524884us) service_pool.cc:224] Handling call
0709 13:18:05.724665 (+ 20399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:05.725159 (+   494us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:05.725186 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:05.725787 (+   601us) log.cc:844] Serialized 26 byte log entry
0709 13:18:06.020504 (+294717us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:06.020543 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:06.020614 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:06.209310 (+188696us) raft_consensus.cc:1685] finished
0709 13:18:06.209328 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:06.209452 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10354,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":103,"thread_start_us":294317,"threads_started":1,"wal-append.queue_time_us":423843}
I20260709 13:18:06.233373 15713 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:06.215119 10794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2530) took 1641 ms. Trace:
I20260709 13:18:06.235505 15597 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:06.236011 15597 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:06.236023 15402 raft_consensus.cc:2804] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:06.236651 15402 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:06.237002 15402 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:06.237030 15501 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:06.238770 15715 consensus_queue.cc:1048] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:18:06.226302 11511 rpcz_store.cc:276] 0709 13:18:05.161025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.216771 (+ 55746us) service_pool.cc:224] Handling call
0709 13:18:05.273941 (+ 57170us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:05.381388 (+107447us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:05.381422 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:05.459065 (+ 77643us) log.cc:844] Serialized 52 byte log entry
0709 13:18:05.849900 (+390835us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:06.078391 (+228491us) log.cc:844] Serialized 14 byte log entry
0709 13:18:06.179343 (+100952us) log.cc:844] Serialized 14 byte log entry
0709 13:18:06.179579 (+   236us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:06.179658 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:06.208762 (+ 29104us) raft_consensus.cc:1685] finished
0709 13:18:06.208782 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:06.208921 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24318,"prepare.queue_time_us":327128,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":68,"raft.queue_time_us":154643,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":269,"thread_start_us":598128,"threads_started":3,"wal-append.queue_time_us":503496}
I20260709 13:18:06.202772 15645 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:06.242133 15645 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:06.208432 15320 consensus_queue.cc:237] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:06.243698 15684 raft_consensus.cc:2749] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:06.244081 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:06.233625 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:06.244951 10944 raft_consensus.cc:2393] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 15: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:06.234597 10794 rpcz_store.cc:276] 0709 13:18:04.574020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.225403 (+651383us) service_pool.cc:224] Handling call
0709 13:18:05.255197 (+ 29794us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:05.268668 (+ 13471us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:05.268709 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:05.269382 (+   673us) log.cc:844] Serialized 26 byte log entry
0709 13:18:05.860339 (+590957us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:05.860379 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:05.860445 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:06.214867 (+354422us) raft_consensus.cc:1685] finished
0709 13:18:06.214891 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:06.215031 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":590470,"threads_started":1,"wal-append.queue_time_us":713619}
W20260709 13:18:06.246670 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 48156.8us
W20260709 13:18:06.253788 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 179795us
I20260709 13:18:06.256049 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:06.257802 11505 raft_consensus.cc:1275] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:18:06.236187 15709 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:06.264793 11477 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:06.266690 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:06.268219 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:06.268950 10947 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 11.
I20260709 13:18:06.269984 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:06.269829 15402 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:06.275110 10945 raft_consensus.cc:2410] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 13:18:06.274641 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:06.234452 15325 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:06.278712 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:06.278314 10916 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:06.283470 10946 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:18:06.282544 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 52902.1us
I20260709 13:18:06.285249 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:06.262121 15124 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:06.290895 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:06.275344 15709 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:06.292320 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:06.294112 10944 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:06.277011 15686 raft_consensus.cc:2749] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:06.297387 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:06.303251 15713 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:06.309084 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 44430.9us
I20260709 13:18:06.309971 15708 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:06.310554 11477 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:06.310771 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:06.311491 10948 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
I20260709 13:18:06.291808 15725 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:06.312126 15708 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:06.312893 15708 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:18:06.312929 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 57571.2us
I20260709 13:18:06.313505 15708 raft_consensus.cc:697] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:06.313552 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:06.314983 10916 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:06.315788 15402 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:06.316078 15720 raft_consensus.cc:2749] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:06.316926 15492 consensus_queue.cc:1059] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:18:06.317517 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:06.317773 15492 consensus_queue.cc:1243] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer cb0428cc55984daba1ba63744a5102a2 log is divergent from this leader: its last log entry 13.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:18:06.319113 15717 raft_consensus.cc:3055] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:06.319525 15717 raft_consensus.cc:740] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:06.316835 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:06.320415 15717 consensus_queue.cc:260] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:06.321360 10948 raft_consensus.cc:2410] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 12 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:18:06.321837 15717 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:06.295141 15515 raft_consensus.cc:2749] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:06.324712 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:06.325080 15713 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:06.326009 10948 raft_consensus.cc:2393] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 13: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:06.328801 10947 pending_rounds.cc:85] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 8
I20260709 13:18:06.329083 10947 pending_rounds.cc:107] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 13:18:06.329313 10947 raft_consensus.cc:2889] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:06.329947 15642 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:06.296241 15709 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:06.357218 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:06.358085 11505 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:06.361493 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:06.366184 10941 raft_consensus.cc:1240] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:06.368096 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:06.366930 10946 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
I20260709 13:18:06.308138 15501 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:06.365219 10950 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
W20260709 13:18:06.371830 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 63681.7us
I20260709 13:18:06.374061 10944 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 16.
I20260709 13:18:06.377396 10801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 1974) took 3059 ms. Trace:
I20260709 13:18:06.378223 10949 raft_consensus.cc:1240] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:06.379132 10950 raft_consensus.cc:1240] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:06.342216 14970 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:06.380314 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:06.381350 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:06.377890 10801 rpcz_store.cc:276] 0709 13:18:03.317745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.016224 (+1698479us) service_pool.cc:224] Handling call
0709 13:18:05.016811 (+   587us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:05.773402 (+756591us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:06.374990 (+601588us) spinlock_profiling.cc:243] Waited 735 ms on lock 0x7b540024efd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:06.377339 (+  2349us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1689353472}
I20260709 13:18:06.382439 10941 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:06.383539 10766 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
W20260709 13:18:06.388240 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 73326.3us
I20260709 13:18:06.354326 15597 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:06.377386 15612 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:06.396528 15597 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:06.396450 10766 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:06.393193 15730 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:06.400350 15683 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:06.401577 11505 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 12.
W20260709 13:18:06.402400 15711 log.cc:927] Time spent T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.250s	user 0.002s	sys 0.000s
I20260709 13:18:06.387398 15725 raft_consensus.cc:697] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:06.439635 15725 consensus_queue.cc:237] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:06.408887 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:06.428200 15612 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:06.430649 15730 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:18:06.431725 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 210316us
I20260709 13:18:06.391757 15325 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:06.358836 15721 consensus_queue.cc:1048] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:18:06.434574 10916 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:06.448166 10841 connection.cc:256] RPC call timeout handler was delayed by -1.1073076360000869s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2335s for a response
I20260709 13:18:06.374745 15726 consensus_queue.cc:1059] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 13:18:06.447966 15730 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:06.371417 15708 consensus_queue.cc:237] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:06.450363 15124 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:06.450140 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 53778us
I20260709 13:18:06.452072 15642 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:06.454321 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:06.456805 10766 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:06.448100 15612 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:06.460911 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:06.463460 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:06.464133 10870 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:06.452402 15320 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:06.464298 10874 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 12.
I20260709 13:18:06.450711 15645 raft_consensus.cc:2749] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:06.468600 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:06.459847 15726 consensus_queue.cc:1243] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer 468a8fd46e824f62b9c2c0d3532a6d26 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:18:06.479504 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:06.482990 15641 consensus_queue.cc:1059] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:06.484658 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:18:06.485255 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:06.450089 15733 consensus_queue.cc:1048] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 13:18:06.471057 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:06.492645 10869 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 11.
W20260709 13:18:06.503523 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:06.503530 10867 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:06.504593 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:06.504853 10766 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:06.505246 10872 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:06.515697 10941 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 13.
I20260709 13:18:06.469141 15642 raft_consensus.cc:2749] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:06.518415 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:06.519311 10874 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:06.506868 15325 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.552698 11478 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
W20260709 13:18:06.577134 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 55335.8us
W20260709 13:18:06.578625 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:06.579524 15639 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.580053 15639 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:06.582122 15639 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:06.585530 15735 log.cc:927] Time spent T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 13:18:06.586370 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 92813.4us
I20260709 13:18:06.587844 10766 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:06.589780 10766 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:06.590477 15725 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:06.590955 15642 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:06.591183 15725 raft_consensus.cc:697] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:06.593596 10766 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:06.583026 11478 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:06.574811 15686 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:06.598430 15686 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:06.598496 15642 consensus_queue.cc:1059] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:06.598969 15686 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:06.531555 15726 raft_consensus.cc:2764] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader election decision vote started in defunct term 15: won
I20260709 13:18:06.612092 10872 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 12.
I20260709 13:18:06.618443 11478 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:06.615970 15730 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:06.631827 11506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56416 (request call id 800) took 1069 ms. Trace:
I20260709 13:18:06.739009 11506 rpcz_store.cc:276] 0709 13:18:05.562960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.667887 (+104927us) service_pool.cc:224] Handling call
0709 13:18:06.035095 (+367208us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:06.037513 (+  2418us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:06.631523 (+594010us) spinlock_profiling.cc:243] Waited 336 ms on lock 0x7b54005f0350. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:06.631734 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":772308736}
I20260709 13:18:06.599732 15641 raft_consensus.cc:2764] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:18:06.635341 10766 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:06.741022 15641 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:18:06.741218 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 105972us
I20260709 13:18:06.741658 15641 raft_consensus.cc:697] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:06.742856 10766 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:06.582939 15635 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.743692 15635 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:06.744401 15642 raft_consensus.cc:2804] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:06.744853 15642 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:06.745206 15642 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:06.636003 15726 consensus_queue.cc:1059] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:06.639253 10916 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:06.746014 15635 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:06.746296 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 107135us
I20260709 13:18:06.554145 15325 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:06.526237 15124 consensus_queue.cc:1059] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.075s
I20260709 13:18:06.639884 15739 raft_consensus.cc:2804] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:06.643613 15709 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:06.646957 15683 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:06.647814 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:06.648573 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:06.649243 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:06.649897 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:06.650619 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:06.651324 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:06.651973 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:06.593753 15725 consensus_queue.cc:237] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:06.656761 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:06.661680 15320 raft_consensus.cc:3055] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:18:06.663106 11478 leader_election.cc:336] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Timed out: RequestConsensusVote RPC to 127.9.116.129:39685 timed out after 1.666s (SENT)
I20260709 13:18:06.667078 11507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56410 (request call id 776) took 1222 ms. Trace:
I20260709 13:18:06.668707 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:06.671126 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:06.671432 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:06.672259 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:06.672978 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:06.673144 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:06.674546 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:06.675932 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:06.677191 15647 raft_consensus.cc:3055] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:06.679687 15190 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:06.682649 10868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 1987) took 3722 ms. Trace:
I20260709 13:18:06.691422 15597 raft_consensus.cc:3055] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:06.694826 11510 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
I20260709 13:18:06.700914 15409 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:18:06.839792 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 176736us
I20260709 13:18:06.705703 15577 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:06.717424 10867 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 13.
I20260709 13:18:06.724756 15612 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:06.843649 10793 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 11.
I20260709 13:18:06.729941 10870 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 16.
I20260709 13:18:06.742537 15641 consensus_queue.cc:237] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:06.604246 15639 consensus_queue.cc:1048] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 13:18:06.851877 10801 raft_consensus.cc:1240] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:06.829692 15683 raft_consensus.cc:697] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 12 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:06.857275 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:06.857127 15683 consensus_queue.cc:237] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:06.858528 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:06.859005 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:06.859690 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:06.860116 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:06.750291 15596 consensus_queue.cc:1059] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 15.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.104s
I20260709 13:18:06.750586 15713 consensus_queue.cc:1059] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.103s
I20260709 13:18:06.861928 15709 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:06.748555 15402 consensus_queue.cc:1048] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.114s
I20260709 13:18:06.818668 15647 raft_consensus.cc:740] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:06.863544 15647 consensus_queue.cc:260] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:06.864539 15647 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:06.746594 15635 consensus_queue.cc:1048] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.093s
I20260709 13:18:06.867115 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:06.750633 15518 consensus_queue.cc:1059] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.105s
I20260709 13:18:06.868475 15730 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:06.818866 15320 raft_consensus.cc:740] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:06.748109 10916 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:06.800001 10794 raft_consensus.cc:2410] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:18:06.800990 10797 raft_consensus.cc:2393] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 13: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
W20260709 13:18:06.776443 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:06.743191 10874 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 13.
I20260709 13:18:06.809791 10799 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 13.
I20260709 13:18:06.819535 10795 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 15.
I20260709 13:18:06.832199 15720 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.831513 15124 consensus_queue.cc:1243] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer e01ecd008edd4aa2b6b2b47efbc2ea87 log is divergent from this leader: its last log entry 15.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:18:06.802428 10796 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 12.
I20260709 13:18:06.829790 10868 rpcz_store.cc:276] 0709 13:18:02.959885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.339200 (+2379315us) service_pool.cc:224] Handling call
0709 13:18:05.436480 (+ 97280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:05.889797 (+453317us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:06.682405 (+792608us) spinlock_profiling.cc:243] Waited 451 ms on lock 0x7b5400098dd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:06.682559 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1037436800}
I20260709 13:18:06.831990 10945 raft_consensus.cc:1275] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:18:06.833000 11477 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:18:06.702389 15706 log.cc:927] Time spent T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.689s	user 0.000s	sys 0.004s
I20260709 13:18:06.748322 15447 raft_consensus.cc:2749] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:06.844527 10948 raft_consensus.cc:1240] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:06.752063 15700 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.801436 15739 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:06.869951 10800 raft_consensus.cc:2393] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 13: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:06.876684 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:06.906632 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:06.808470 11507 rpcz_store.cc:276] 0709 13:18:05.444792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.477260 (+ 32468us) service_pool.cc:224] Handling call
0709 13:18:06.666853 (+1189593us) spinlock_profiling.cc:243] Waited 497 ms on lock 0x7b540021e654. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:06.666994 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1142659328}
I20260709 13:18:06.799508 10792 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:18:06.905388 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:06.891326 11512 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
I20260709 13:18:06.842964 15409 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:18:06.893528 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 60597.7us
I20260709 13:18:06.898864 10915 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:06.832506 15597 raft_consensus.cc:740] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:06.871778 15325 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:06.912344 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:06.913823 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:06.941499 10947 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:06.915333 10795 raft_consensus.cc:1240] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:06.912117 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:06.912070 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:06.755393 15642 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:06.940346 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:06.949210 10869 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:06.943392 10796 raft_consensus.cc:1240] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:06.944253 10799 raft_consensus.cc:2393] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 12: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:06.941847 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:06.951010 10875 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 13.
I20260709 13:18:06.953034 10797 raft_consensus.cc:2410] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 13 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 13:18:06.958516 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 338143us
W20260709 13:18:06.959298 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:06.948513 10867 raft_consensus.cc:1275] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:18:06.959378 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:06.905345 15590 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:06.914655 15492 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:06.919186 15124 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.911883 15739 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:06.967722 10870 raft_consensus.cc:1275] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 13:18:06.971905 15612 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:06.973335 15686 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:06.942984 15715 raft_consensus.cc:2804] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:06.976552 15642 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:06.979908 10793 raft_consensus.cc:2393] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 13: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
W20260709 13:18:06.980654 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 232619us
I20260709 13:18:06.985038 15124 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:07.001518 15124 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:06.953892 15700 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:07.008008 15715 raft_consensus.cc:697] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 12 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:07.008853 15700 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:07.009146 15726 consensus_queue.cc:1059] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:18:07.010540 15726 raft_consensus.cc:3055] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:07.010970 15726 raft_consensus.cc:740] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:07.011890 15726 consensus_queue.cc:260] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:07.012953 15726 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:06.983640 10870 pending_rounds.cc:85] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 10
I20260709 13:18:06.896265 15720 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:06.993570 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.015012 15320 consensus_queue.cc:260] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:07.016652 15320 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:07.016701 10916 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:06.958516 15712 consensus_queue.cc:1048] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.059s
I20260709 13:18:07.017740 15712 consensus_queue.cc:1243] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer cb0428cc55984daba1ba63744a5102a2 log is divergent from this leader: its last log entry 15.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:18:06.912869 15402 consensus_queue.cc:1048] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.070s
I20260709 13:18:07.014967 10870 pending_rounds.cc:107] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 13:18:06.966890 15590 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:07.009521 10798 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 11.
I20260709 13:18:07.016932 15720 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:07.022996 10948 pending_rounds.cc:85] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 12
I20260709 13:18:07.019217 15447 raft_consensus.cc:2749] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:07.023473 10948 pending_rounds.cc:107] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 15.13
I20260709 13:18:07.023823 10948 raft_consensus.cc:2889] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:07.019218 10870 raft_consensus.cc:2889] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:07.019922 15590 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:07.028308 15645 raft_consensus.cc:3055] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:07.028748 15645 raft_consensus.cc:740] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:07.029699 15645 consensus_queue.cc:260] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:07.032191 10792 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 15.
I20260709 13:18:06.963047 15747 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.035648 15747 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:07.038430 15747 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:07.018712 10916 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:06.984714 15492 raft_consensus.cc:697] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 13 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:07.046041 11504 raft_consensus.cc:2368] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 13. Current term is 14.
I20260709 13:18:06.952225  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:07.057700 10869 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 13.
I20260709 13:18:07.057705 10947 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 12.
W20260709 13:18:07.061445 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.045535 15739 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:07.047052 15686 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:07.026530 15597 consensus_queue.cc:260] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:07.050140 10915 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:18:07.108767 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:07.132750 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 51461.2us
W20260709 13:18:07.133592 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 83533.7us
I20260709 13:18:07.077623 15124 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:18:07.141690 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:07.142496 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.144268 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:18:07.141837 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.147197 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:07.147799 10947 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:18:07.149585 15740 log.cc:927] Time spent T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:18:07.138803 15713 consensus_queue.cc:1243] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer cb0428cc55984daba1ba63744a5102a2 log is divergent from this leader: its last log entry 12.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:18:07.150615 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:07.151692 10869 raft_consensus.cc:2393] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 12: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:07.152216 15590 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:07.139424 15124 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:07.153270 15124 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:07.154454 15590 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:07.157505 10947 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 13.
I20260709 13:18:07.158401 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:07.009078 15715 consensus_queue.cc:237] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:07.159611 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:07.159759 15597 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:07.159104 10699 catalog_manager.cc:5697] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 8 to 11. New cstate: current_term: 11 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:07.162567 10869 raft_consensus.cc:2393] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 13: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:07.164572 11477 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:07.086987 15645 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:18:07.171151 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 209576us
I20260709 13:18:07.171852 10950 raft_consensus.cc:1275] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 13:18:07.172219 10950 pending_rounds.cc:85] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 6
I20260709 13:18:07.172441 10950 pending_rounds.cc:107] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 13:18:07.172662 10950 raft_consensus.cc:2889] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:07.176389 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:07.178367 10766 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:07.178579 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:07.181197 10792 raft_consensus.cc:2393] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 13: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:07.181646 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:07.182260 10870 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
I20260709 13:18:07.183427 10945 raft_consensus.cc:2393] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 12: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:07.184839 10916 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:07.185019 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:07.139055 15647 raft_consensus.cc:2804] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:07.080201 15409 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:07.187752 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 286523us
I20260709 13:18:07.258720 11478 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:07.259783 11478 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:07.261757 10869 raft_consensus.cc:1240] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
W20260709 13:18:07.262329 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 84018.9us
I20260709 13:18:07.262907 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:07.262333 11478 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:07.263545 10793 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:07.262329 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:07.266291 10870 raft_consensus.cc:2393] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 13: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:07.273921 15398 raft_consensus.cc:2804] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:07.276718 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:07.294226 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 109467us
I20260709 13:18:07.294458 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:07.275121 11478 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:07.295886 10871 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
I20260709 13:18:07.297958 15577 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:07.298115 10874 raft_consensus.cc:1240] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:07.298535 15577 raft_consensus.cc:697] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 13 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:07.277522 15736 consensus_queue.cc:1048] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 13:18:07.271986 15492 consensus_queue.cc:237] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:07.299497 15577 consensus_queue.cc:237] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:07.293861 15755 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:07.301062 15755 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:07.274250 15684 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:07.256553 15717 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:07.271505 15720 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:07.308482 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:07.271627 15700 raft_consensus.cc:2804] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:07.319154 10870 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
I20260709 13:18:07.268941 15590 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:07.267189 15686 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:07.188948 15726 consensus_queue.cc:1059] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.114s
I20260709 13:18:07.268711 15647 raft_consensus.cc:697] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 13 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:07.336597 10870 raft_consensus.cc:1275] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:18:07.330408 10793 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 12.
I20260709 13:18:07.276886 14970 raft_consensus.cc:3055] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:18:07.330875 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.331461 10916 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
W20260709 13:18:07.333288 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 58273.9us
I20260709 13:18:07.276614 15190 consensus_queue.cc:1048] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
W20260709 13:18:07.331076 15735 log.cc:927] Time spent T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:18:07.308130 15684 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:07.318080 15398 raft_consensus.cc:697] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 12 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:07.354776 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:07.357417 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:07.330024 15590 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:07.359261 11508 raft_consensus.cc:1240] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:07.355805 10841 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:07.332386 15717 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:07.361476 11507 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:07.358451 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:07.367754 11510 raft_consensus.cc:2393] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 12: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:07.366087 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:18:07.369294 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.369205 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:07.370378 10870 raft_consensus.cc:2393] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 10: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:07.336056 15700 raft_consensus.cc:697] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:07.373818 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:07.374084 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:18:07.387307 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.375094 15744 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:18:07.379216 15739 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:07.379621 15755 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:07.338867 14970 raft_consensus.cc:740] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:07.380079 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:07.379868 15647 consensus_queue.cc:237] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:07.373907 15733 raft_consensus.cc:3055] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:07.380965 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:18:07.402504 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.341424 15720 raft_consensus.cc:697] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 13 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:07.399451 11512 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 15.
I20260709 13:18:07.403255 10874 raft_consensus.cc:2393] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 12: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:07.382515 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:07.383330 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:07.384497 10867 raft_consensus.cc:1275] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:18:07.364519 15684 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:07.347229 15709 raft_consensus.cc:2804] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:07.386080 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:07.369421 15590 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:07.387825 11510 raft_consensus.cc:2393] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 14: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:07.348591 15402 consensus_queue.cc:1048] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:18:07.398663 10841 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:07.422525 10841 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:07.423795 10841 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:07.426023 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:07.426746 15754 raft_consensus.cc:2749] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:18:07.381788 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:07.427795 15754 raft_consensus.cc:2749] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:07.428150 15757 raft_consensus.cc:2764] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:18:07.428371 10871 raft_consensus.cc:2393] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 13: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:07.366348 15612 consensus_queue.cc:1059] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:18:07.429090 15612 raft_consensus.cc:3055] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:07.429476 15612 raft_consensus.cc:740] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:07.399430 10875 raft_consensus.cc:1240] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:07.374619 15409 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:07.373829 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:07.431438 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:07.435251 10870 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:07.438246 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:07.439460 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:07.441066 10793 raft_consensus.cc:2393] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 11: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:07.414379 15709 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:07.418159 15720 consensus_queue.cc:237] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:07.346045 15124 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:07.420255 15700 consensus_queue.cc:237] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:07.425447 10841 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:07.411818 11503 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:07.339515 15686 raft_consensus.cc:697] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 16 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:07.454916 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:07.407590 15398 consensus_queue.cc:237] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:07.410799 15717 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:07.455579 10799 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 12.
I20260709 13:18:07.456452 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:07.457448 10797 raft_consensus.cc:2410] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 13:18:07.447614 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:07.458563 10796 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:07.448550 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:07.460065 10874 raft_consensus.cc:2393] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 13: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:07.448310 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:07.409543 15733 raft_consensus.cc:740] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:07.463341 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:07.448017 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:07.464112 10869 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 11.
I20260709 13:18:07.464308 10875 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:18:07.461122 10792 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:07.462225 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:07.470775 10868 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:07.475081 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:07.474439 11507 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 13.
I20260709 13:18:07.475756 11478 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:07.398722 14970 consensus_queue.cc:260] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:07.477064 15717 raft_consensus.cc:2749] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:07.477434 14970 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:18:07.478222 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 52845.1us
I20260709 13:18:07.462462 15686 consensus_queue.cc:237] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:07.480391 10841 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:07.481400 10841 leader_election.cc:400] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 election: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 12. Current term is 13.
I20260709 13:18:07.481652 10841 leader_election.cc:403] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:18:07.483795 15733 consensus_queue.cc:260] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:07.484719 15744 raft_consensus.cc:2804] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:07.484935 15733 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:07.485296 15744 raft_consensus.cc:697] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:07.486255 15744 consensus_queue.cc:237] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:07.488761 15754 raft_consensus.cc:2749] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 12. Current term is 13.
I20260709 13:18:07.488648 15612 consensus_queue.cc:260] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:07.490000 15612 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:07.453446 15709 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:18:07.495924 15761 log.cc:927] Time spent T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.098s	user 0.000s	sys 0.002s
I20260709 13:18:07.496138 11503 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
W20260709 13:18:07.496973 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.498024 10915 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:07.498937 15765 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:07.499069 15730 raft_consensus.cc:3055] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:07.499428 15765 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:07.500275 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.500211 10795 raft_consensus.cc:1275] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 11. (term mismatch)
I20260709 13:18:07.503509 10795 pending_rounds.cc:85] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 10
I20260709 13:18:07.503780 10795 pending_rounds.cc:107] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 11.11
I20260709 13:18:07.504009 10795 raft_consensus.cc:2889] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:07.482692 10841 leader_election.cc:400] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 13. Current term is 14.
I20260709 13:18:07.506222 10841 leader_election.cc:403] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:07.506225 10868 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 12.
I20260709 13:18:07.508297 10792 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 12.
I20260709 13:18:07.512106 11478 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:07.510341 11511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56404 (request call id 847) took 1167 ms. Trace:
I20260709 13:18:07.528143 11511 rpcz_store.cc:276] 0709 13:18:06.333657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:06.333844 (+   187us) service_pool.cc:224] Handling call
0709 13:18:06.363479 (+ 29635us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:06.694732 (+331253us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:06.694763 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:06.695277 (+   514us) log.cc:844] Serialized 26 byte log entry
0709 13:18:07.085635 (+390358us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:07.085685 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:07.267861 (+182176us) spinlock_profiling.cc:243] Waited 331 ms on lock 0x7b54005d66d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:07.267903 (+    42us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:07.501125 (+233222us) raft_consensus.cc:1685] finished
0709 13:18:07.501146 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:07.501347 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12509,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":92,"spinlock_wait_cycles":760825088,"thread_start_us":389969,"threads_started":1,"wal-append.queue_time_us":791525}
I20260709 13:18:07.554770 10942 raft_consensus.cc:1275] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:18:07.562059 10875 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 14.
I20260709 13:18:07.499624 15730 raft_consensus.cc:740] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:07.533747 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:07.634506 10794 raft_consensus.cc:1240] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:07.596241 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:07.513041 15765 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:07.510972 15757 raft_consensus.cc:2764] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
W20260709 13:18:07.591143 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 108587us
I20260709 13:18:07.581377 10870 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
I20260709 13:18:07.643239 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:07.538731 11478 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:07.668303 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 129685us
I20260709 13:18:07.669550 10801 raft_consensus.cc:2393] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 16: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:07.533010 15325 raft_consensus.cc:2749] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:18:07.601187 10869 raft_consensus.cc:1275] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:18:07.582262 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:07.622481 10842 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:18:07.577010 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.673851 10875 raft_consensus.cc:1275] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:18:07.634685 15124 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:07.691876 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:07.565896 15742 raft_consensus.cc:2749] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 13. Current term is 14.
I20260709 13:18:07.695678 10874 raft_consensus.cc:2393] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 12: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:07.590654 15684 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:07.703950 11511 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:18:07.702119 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 79743.8us
W20260709 13:18:07.725739 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.530596 15683 consensus_queue.cc:1059] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.163s
I20260709 13:18:07.709558 15755 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:07.737388 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:07.674607 15577 raft_consensus.cc:2749] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:07.642055 15709 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:07.755817 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:07.731527 10842 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:07.762334 11508 raft_consensus.cc:1240] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:18:07.772106 15709 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:07.773407 10950 raft_consensus.cc:1240] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 13. Current term is 14. Ops: []
W20260709 13:18:08.018059 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 286586us
I20260709 13:18:07.737498 15755 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:08.018879 15755 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:08.019111 15686 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.019933 15686 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:08.022452 15686 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:07.778503 15642 raft_consensus.cc:3055] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:07.711407 15639 consensus_queue.cc:1048] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:18:08.023440 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:07.789058 11511 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
I20260709 13:18:07.795691 15684 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:08.023474 15642 raft_consensus.cc:740] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:08.018900 10842 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:08.025983 15642 consensus_queue.cc:260] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:08.027305 15642 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:08.029006 10915 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:08.029996 15709 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:08.030494 15709 raft_consensus.cc:697] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:07.796933 15124 consensus_queue.cc:1048] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:08.031471 15709 consensus_queue.cc:237] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:08.032747 15742 raft_consensus.cc:2804] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:08.033564 15755 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:07.799430 15612 consensus_queue.cc:1059] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:07.785130 15765 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:07.820592 15776 log.cc:927] Time spent T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:18:08.034837 15647 raft_consensus.cc:3055] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:08.035236 15647 raft_consensus.cc:740] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:07.823313 15590 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:08.036176 15647 consensus_queue.cc:260] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:08.037441 15647 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:08.037678 15590 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:07.871925 10796 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 14.
I20260709 13:18:08.039131 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:07.928349 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:08.045292 10796 raft_consensus.cc:2368] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 11. Current term is 12.
I20260709 13:18:08.046005 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:08.046185 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:08.046343 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:07.947099 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:08.046942 11506 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:08.047241 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:08.047453 10800 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:08.047884 10793 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:07.965101 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:08.051082 10799 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:07.967821 15492 raft_consensus.cc:3055] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:07.971721 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:07.972458 11510 raft_consensus.cc:1275] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:18:07.974555 11507 raft_consensus.cc:1275] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:18:08.001221 15726 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.056149 15726 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:08.054495 15492 raft_consensus.cc:740] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:08.058301 15726 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:08.011608 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:08.059302 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:07.776053 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:08.051301 15730 consensus_queue.cc:260] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:08.060670 15730 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:18:07.776563 15743 log.cc:927] Time spent T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.479s	user 0.001s	sys 0.001s
I20260709 13:18:08.067807 10943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 820) took 1387 ms. Trace:
I20260709 13:18:08.068243 10943 rpcz_store.cc:276] 0709 13:18:06.679967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:06.680107 (+   140us) service_pool.cc:224] Handling call
0709 13:18:06.680444 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:06.680858 (+   414us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:18:06.681143 (+   285us) log.cc:844] Serialized 14 byte log entry
0709 13:18:07.062357 (+381214us) log.cc:844] Serialized 14 byte log entry
0709 13:18:07.079814 (+ 17457us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:07.080388 (+   574us) log.cc:844] Serialized 26 byte log entry
0709 13:18:07.081929 (+  1541us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:07.081965 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:07.082033 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:07.345621 (+263588us) spinlock_profiling.cc:243] Waited 2.79 ms on lock 0x7b18003e7b80. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc83f3 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:18:07.775944 (+430323us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:18:08.067546 (+291602us) raft_consensus.cc:1685] finished
0709 13:18:08.067568 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:08.067722 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19879,"prepare.queue_time_us":14737,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":86,"raft.queue_time_us":31838,"raft.run_cpu_time_us":343,"raft.run_wall_time_us":4478,"spinlock_wait_cycles":29979648,"thread_start_us":380389,"threads_started":1,"wal-append.queue_time_us":500601}
I20260709 13:18:07.926990 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:18:08.075500 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:08.040139 10948 raft_consensus.cc:1240] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:08.066526 10799 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
I20260709 13:18:08.041184 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:08.039446 15639 consensus_queue.cc:1059] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 13:18:08.041903 15765 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:08.042273 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:08.090739 11478 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:08.091003 10801 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:08.091014 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:08.024766 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:08.046906 10796 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 15.
I20260709 13:18:08.052520 10916 leader_election.cc:400] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 election: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 11. Current term is 12.
I20260709 13:18:08.093225 10916 leader_election.cc:403] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:18:08.090519 10800 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 12.
I20260709 13:18:08.019158 15744 raft_consensus.cc:2804] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:08.058902 15492 consensus_queue.cc:260] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:08.084399 10794 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 14.
I20260709 13:18:08.085323 11506 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
W20260709 13:18:08.085616 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:08.108811 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:08.108942 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:18:08.101687 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 49294.9us
I20260709 13:18:08.109762 10799 raft_consensus.cc:2393] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 13: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:08.110589 10916 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:08.090533 15686 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.104698 11478 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:08.092797 10797 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:18:08.102833 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:08.114745 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:08.114578 11478 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:08.116328 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:08.116875 10799 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 11.
I20260709 13:18:08.112920 10793 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 12.
I20260709 13:18:08.117799 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:08.118219 10793 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:08.103685 15612 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:08.120520 11478 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:18:08.113322 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 86267.6us
I20260709 13:18:08.121623 10915 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:08.111269 15763 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:08.128350 10793 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 12.
I20260709 13:18:08.124045 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:08.129454 10799 raft_consensus.cc:2393] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 13: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:08.112349 15744 raft_consensus.cc:697] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:08.131865 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:08.130594 15124 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:08.111579 15742 raft_consensus.cc:697] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:08.122196 15750 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.114945 15765 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:08.122031 15492 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:08.132618 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:08.110343 15590 consensus_queue.cc:1048] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 13:18:08.107507 15645 consensus_queue.cc:1048] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 13:18:08.122323 15684 raft_consensus.cc:2804] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:08.123008 15398 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:08.140892 10793 raft_consensus.cc:2410] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:18:08.141754 10801 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 14.
I20260709 13:18:08.126734 15739 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:08.130352 15726 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.130882 15763 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:08.122519 15686 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:08.123293 15190 consensus_queue.cc:1048] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 13:18:08.163667 10797 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 14.
I20260709 13:18:08.149700 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:08.152577 10767 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:08.144954 10799 raft_consensus.cc:2376] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 15. Re-sending same reply.
I20260709 13:18:08.151350 15684 raft_consensus.cc:697] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 12 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:08.151552 15783 consensus_queue.cc:1059] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 13:18:08.152041 15763 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:08.144155 15124 raft_consensus.cc:697] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:08.209741 10867 raft_consensus.cc:1275] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:18:08.171042 10801 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:08.205677 15708 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.287787 15708 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:08.289671 15708 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:08.174953 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:08.204667 15783 raft_consensus.cc:3055] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:08.205507 15784 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.291214 15783 raft_consensus.cc:740] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:08.291474 15784 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:08.291945 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:08.292347 15783 consensus_queue.cc:260] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:08.293220 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:08.293787 15783 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:08.293874 15784 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:08.207614 10874 raft_consensus.cc:1275] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:18:08.174594 15713 raft_consensus.cc:2749] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:08.145391 15750 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:08.155510 15758 log.cc:927] Time spent T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.879s	user 0.004s	sys 0.000s
I20260709 13:18:08.160681 15726 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:08.174527 15744 consensus_queue.cc:237] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:08.147543 15765 raft_consensus.cc:697] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 13 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:08.216677 10767 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:08.696748 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 480135us
I20260709 13:18:08.698604 10767 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:08.207253 15684 consensus_queue.cc:237] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:08.218506 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:08.219818 10801 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
I20260709 13:18:08.703255 10793 raft_consensus.cc:2393] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 15: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:08.703912 10801 raft_consensus.cc:1240] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:08.705374 10801 raft_consensus.cc:1275] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:18:08.705575 10793 raft_consensus.cc:1275] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:18:08.249341 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:08.707129 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:08.707464 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:08.707742 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:08.250087 10795 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
I20260709 13:18:08.710336 10795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 1992) took 1122 ms. Trace:
I20260709 13:18:08.259578 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:08.261291 10867 raft_consensus.cc:1275] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:18:08.711112 10871 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:08.264127 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:08.265616 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:08.712306 10793 raft_consensus.cc:1275] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:18:08.268149 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:08.269229 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:08.270363 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:08.273180 15788 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:08.285048 15701 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.295969 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:08.210494 15124 consensus_queue.cc:237] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:08.310717 15686 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:08.328567 10699 catalog_manager.cc:5697] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 11 to 12, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 12 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:08.338971 11511 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
I20260709 13:18:08.357630 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:08.404613 15647 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:08.721194 15763 consensus_meta.cc:287] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.511s	user 0.018s	sys 0.009s
I20260709 13:18:08.721308 15647 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:08.721750 15763 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:08.416026 15742 consensus_queue.cc:237] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:08.428160 15713 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.428666 15747 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:08.723691 15701 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:08.724270 15747 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:08.719180 14970 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:08.725248 14970 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:08.726281 15747 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:08.727102 10797 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 14: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:08.437229 15754 raft_consensus.cc:3055] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:08.727830 14970 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:08.450645 10950 raft_consensus.cc:1240] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:08.728080 10797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2000) took 1086 ms. Trace:
I20260709 13:18:08.450925 10943 raft_consensus.cc:1217] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Deduplicated request from leader. Original: 8.6->[9.7-13.9]   Dedup: 11.8->[13.9-13.9]
I20260709 13:18:08.456439 11503 raft_consensus.cc:1240] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:08.451191 10942 raft_consensus.cc:1240] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:08.733260 10943 raft_consensus.cc:1240] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 13. Current term is 14. Ops: [9.7-13.9]
I20260709 13:18:08.457525 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:08.459038 15447 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.736743 15447 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:08.461619 10800 raft_consensus.cc:1275] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:18:08.739871 10800 raft_consensus.cc:1240] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:08.463860 10945 raft_consensus.cc:1275] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:18:08.464895 10941 raft_consensus.cc:1275] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:18:08.466868 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:08.468482 10947 raft_consensus.cc:1240] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:08.466420 15715 consensus_queue.cc:1059] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 13:18:08.445405 15612 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 11. Current term is 12.
I20260709 13:18:08.476367 10792 raft_consensus.cc:2410] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:18:08.476944 10799 raft_consensus.cc:1275] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:18:08.477016 15750 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:08.162252 15739 raft_consensus.cc:697] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 14 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:08.160421 15755 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:08.505863 10798 raft_consensus.cc:1275] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 13:18:08.517261 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:08.761504 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:08.518131 15645 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:08.520759 10796 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 14.
I20260709 13:18:08.765291 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:08.524044 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:08.766536 11504 raft_consensus.cc:2393] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 14: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:08.524818 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:08.526998 15639 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:08.767941 15639 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:08.770098 15639 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:08.530007 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:08.640699 10794 raft_consensus.cc:1240] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
W20260709 13:18:08.667606 15790 log.cc:927] Time spent T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 13:18:08.670578 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:18:08.675598 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:08.676862 11509 raft_consensus.cc:1240] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 13 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:08.679262 15726 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:08.780764 10915 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:08.751502 15715 raft_consensus.cc:3055] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:08.711623 10767 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:08.712285 10869 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 15.
I20260709 13:18:08.783922 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:08.784716 10869 raft_consensus.cc:2393] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 15: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:08.266904 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:08.710640 10795 rpcz_store.cc:276] 0709 13:18:07.588313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.588507 (+   194us) service_pool.cc:224] Handling call
0709 13:18:08.710092 (+1121585us) spinlock_profiling.cc:243] Waited 583 ms on lock 0x7b5400294854. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:08.710241 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1340551552}
I20260709 13:18:08.706588 10801 raft_consensus.cc:1240] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:08.720327 10875 raft_consensus.cc:2393] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 14: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:08.786105 10872 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:08.720371 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:08.721921 15683 consensus_queue.cc:1048] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:08.727880 15754 raft_consensus.cc:740] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:08.443600 10916 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:08.765326 15398 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:08.789844 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 346303us
W20260709 13:18:08.790033 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 78506.8us
I20260709 13:18:08.790548 10916 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:08.785475 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:18:08.446869 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:08.791599 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:08.791972 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:18:08.792493 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:08.718979 15730 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.718910 15708 raft_consensus.cc:2749] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:08.800884 10872 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
I20260709 13:18:08.802132 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:08.751693 15757 consensus_queue.cc:1059] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:18:08.803452 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:08.804066 10872 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:08.795003 15398 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:08.806399 15398 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:08.811941 10872 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 14.
I20260709 13:18:08.813010 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:08.729242 15647 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:08.724017 15713 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:08.729712 15747 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.718901 15642 raft_consensus.cc:2804] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:08.724018 15763 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:08.828621 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:08.829692 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 47626.6us
I20260709 13:18:08.830745 10841 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:08.730619 15788 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:08.734305 11503 raft_consensus.cc:1240] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:18:08.831846 10868 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 14.
I20260709 13:18:08.732720 10797 rpcz_store.cc:276] 0709 13:18:07.641165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:08.292951 (+651786us) service_pool.cc:224] Handling call
0709 13:18:08.727907 (+434956us) spinlock_profiling.cc:243] Waited 433 ms on lock 0x7b54003ecbd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:08.728026 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":996005248}
I20260709 13:18:08.736043 10793 raft_consensus.cc:1240] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:08.726544 15701 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:08.833038 14970 consensus_queue.cc:1048] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:18:08.745463 11506 raft_consensus.cc:1275] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:18:08.773845 10798 raft_consensus.cc:1240] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:08.691515 11505 raft_consensus.cc:1240] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:18:08.834863 14970 consensus_queue.cc:1048] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:08.835479 14970 consensus_queue.cc:1243] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer cb0428cc55984daba1ba63744a5102a2 log is divergent from this leader: its last log entry 16.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:18:08.781493 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:08.759702 15447 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:08.782475 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:08.786950 10867 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:08.788772 10871 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 14.
I20260709 13:18:08.792868 10869 raft_consensus.cc:2368] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 12. Current term is 13.
I20260709 13:18:08.844724 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:08.845474 10869 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
I20260709 13:18:08.794266 15709 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:08.794981 15739 consensus_queue.cc:237] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:08.795480 10767 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:08.776134 15645 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:08.783111 15715 raft_consensus.cc:740] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:08.696734 15765 consensus_queue.cc:237] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:08.796972 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:08.817060 15726 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:08.825284 10795 raft_consensus.cc:1240] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:08.824666 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:08.878268 11512 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 12.
I20260709 13:18:08.831710 15590 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:08.846589 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:08.845638 10872 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 14.
I20260709 13:18:08.848948 11478 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:08.883638 10876 raft_consensus.cc:1275] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 13:18:08.867658 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 72274.2us
I20260709 13:18:08.869108 10875 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:08.829149 15642 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:08.816838 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:08.874918 15744 raft_consensus.cc:2804] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:08.874478 10867 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
I20260709 13:18:08.837285 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:08.879297 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:08.829603 15124 raft_consensus.cc:2804] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:08.884960 10767 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:08.841874 15747 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:08.888592 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:08.888630 10871 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 14: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:08.829344 15683 consensus_queue.cc:1059] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.055s
I20260709 13:18:08.901315 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:08.921056 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:08.901445 10872 raft_consensus.cc:1240] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:08.921690 10948 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:08.921787 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:08.896132 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:08.903877 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:08.876971 15709 raft_consensus.cc:697] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 13 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:08.921706 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:08.955979 10945 raft_consensus.cc:2393] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 13: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:08.816999 15730 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:08.914912 10799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2004) took 1178 ms. Trace:
I20260709 13:18:08.970257 10799 rpcz_store.cc:276] 0709 13:18:07.736804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:08.332682 (+595878us) service_pool.cc:224] Handling call
0709 13:18:08.333012 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:08.752387 (+419375us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:08.914576 (+162189us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b5400386b54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:08.914829 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":320990464}
I20260709 13:18:08.905334 11509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56416 (request call id 807) took 1548 ms. Trace:
I20260709 13:18:08.877339 15726 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:08.953898 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:08.960002 10915 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:08.920849 15124 raft_consensus.cc:697] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 12 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:08.963136 15801 consensus_queue.cc:1059] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:18:08.901095 15708 raft_consensus.cc:2764] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:18:08.882156 15754 consensus_queue.cc:260] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:08.968695 14970 consensus_queue.cc:1048] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:18:08.995381 10792 raft_consensus.cc:1240] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:08.973327 10875 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
I20260709 13:18:08.973555 10946 raft_consensus.cc:1275] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:18:08.971832 11509 rpcz_store.cc:276] 0709 13:18:07.357363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.357529 (+   166us) service_pool.cc:224] Handling call
0709 13:18:07.371494 (+ 13965us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:08.781174 (+1409680us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:08.905094 (+123920us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54001f90d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:08.905279 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3001575168}
I20260709 13:18:08.900852 15763 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:08.900701 15744 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:09.010039 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:09.012517 10946 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 14.
I20260709 13:18:08.997020 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:18:08.993352 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 108489us
I20260709 13:18:09.014053 10941 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 15.
I20260709 13:18:09.014648 10767 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:09.015233 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 224740us
I20260709 13:18:09.016857 10916 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:09.017823 15402 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:09.018322 15402 raft_consensus.cc:697] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 12 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:09.019593 15402 consensus_queue.cc:237] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:09.023595 10872 raft_consensus.cc:1240] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:09.025146 10872 raft_consensus.cc:1275] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:18:09.027774 10867 raft_consensus.cc:1240] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:09.026460 10916 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:09.029956 15409 raft_consensus.cc:2804] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:09.030386 10916 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:18:08.993942 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 163316us
I20260709 13:18:09.031507 10870 raft_consensus.cc:1275] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:18:09.031409 10841 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:09.031823 11478 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:09.032668 15683 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:09.033042 15683 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:09.033385 15683 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:09.033761 10870 raft_consensus.cc:1275] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:18:08.885659 15642 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:09.034848 15684 consensus_queue.cc:1048] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:09.037904 15647 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:18:09.007422 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 47549.7us
I20260709 13:18:09.038627 15647 raft_consensus.cc:697] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 14 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:09.030488 15409 raft_consensus.cc:697] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 14 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:09.039806 15647 consensus_queue.cc:237] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:09.040859 15409 consensus_queue.cc:237] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:09.044656 15755 raft_consensus.cc:3055] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:09.044480 15612 consensus_queue.cc:1048] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:18:09.045130 15755 raft_consensus.cc:740] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:09.049371 15686 raft_consensus.cc:3055] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:09.049855 15686 raft_consensus.cc:740] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:08.902913 15590 raft_consensus.cc:697] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 14 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:09.051534 15590 consensus_queue.cc:237] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:09.056902 10874 raft_consensus.cc:1275] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:18:09.059325 10869 raft_consensus.cc:1240] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 11. Current term is 13. Ops: []
I20260709 13:18:09.061153 10876 raft_consensus.cc:1240] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:09.063949 10945 raft_consensus.cc:1275] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:18:08.991153 15726 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:09.070914 10875 raft_consensus.cc:1275] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
I20260709 13:18:09.075327 10875 pending_rounds.cc:85] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 11
I20260709 13:18:09.075696 10875 pending_rounds.cc:107] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 13:18:09.076009 10875 raft_consensus.cc:2889] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:09.076404 10875 pending_rounds.cc:107] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 13:18:09.076692 10875 raft_consensus.cc:2889] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:18:09.078830 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 190833us
I20260709 13:18:09.072926 10869 raft_consensus.cc:1275] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:18:09.006567 15701 raft_consensus.cc:2749] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:18:09.120407 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 87645.2us
I20260709 13:18:09.123277 10841 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:09.123484 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 83946.5us
I20260709 13:18:09.128868 15750 consensus_queue.cc:1048] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:18:09.131193 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 50671.9us
I20260709 13:18:09.055099 15744 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:09.109411 15801 consensus_queue.cc:1048] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:18:09.046028 15755 consensus_queue.cc:260] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:09.005918 15757 consensus_queue.cc:1059] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.122s
W20260709 13:18:09.139650 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 86813us
I20260709 13:18:09.137450 10766 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:09.061132 15763 raft_consensus.cc:697] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 14 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:09.055845 15800 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:09.152626 10948 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
I20260709 13:18:09.149313 10766 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:09.150054 14970 raft_consensus.cc:3055] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:09.154326 14970 raft_consensus.cc:740] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:09.155469 14970 consensus_queue.cc:260] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:09.156877 14970 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:09.159456 10699 catalog_manager.cc:5697] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 9 to 12, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 12 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:09.055159 15783 consensus_queue.cc:1059] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.084s
I20260709 13:18:09.167519 10766 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:09.172940 10766 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:09.182080 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:09.183419 10766 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:09.055433 15794 consensus_queue.cc:1059] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.147s
I20260709 13:18:09.186576 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:09.187505 10946 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 15.
I20260709 13:18:09.113826 15717 consensus_queue.cc:1048] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 13:18:08.979216 15715 consensus_queue.cc:260] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:09.186582 10945 raft_consensus.cc:1275] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:18:09.163164 15701 raft_consensus.cc:2749] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:09.137432 15713 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:09.164091 15800 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:09.134076 15804 raft_consensus.cc:2749] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:09.191742 15755 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:18:09.198066 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 183476us
I20260709 13:18:09.152038 15788 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:09.210803 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:09.176986 15750 raft_consensus.cc:2749] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:09.174777 15639 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:09.242695 10767 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:09.263197 14970 raft_consensus.cc:2804] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:09.266927 14970 raft_consensus.cc:697] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:09.275978 14970 consensus_queue.cc:237] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:09.287482 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:09.269428 15713 raft_consensus.cc:2804] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:09.287392 15645 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:09.290365 15645 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.290886 15645 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:09.264115 15715 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:18:09.293107 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:09.284884 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 161686us
I20260709 13:18:09.263952 15800 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:09.239830 15720 consensus_queue.cc:1048] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:18:09.305931 10841 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:09.288832 10897 debug-util.cc:398] Leaking SignalData structure 0x7b08002f91e0 after lost signal to thread 9685
W20260709 13:18:09.304646 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 154670us
I20260709 13:18:09.273492 15788 raft_consensus.cc:2749] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:09.188683 15124 consensus_queue.cc:237] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:09.281342 15708 raft_consensus.cc:2749] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:18:09.316509 10897 debug-util.cc:398] Leaking SignalData structure 0x7b08002fb7a0 after lost signal to thread 10733
I20260709 13:18:09.317106 10841 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:09.318207 10897 debug-util.cc:398] Leaking SignalData structure 0x7b08000dda80 after lost signal to thread 10825
W20260709 13:18:09.324582 10897 debug-util.cc:398] Leaking SignalData structure 0x7b080037b8a0 after lost signal to thread 10900
W20260709 13:18:09.325186 10897 debug-util.cc:398] Leaking SignalData structure 0x7b080037b8c0 after lost signal to thread 10901
W20260709 13:18:09.326622 10897 debug-util.cc:398] Leaking SignalData structure 0x7b080035bd60 after lost signal to thread 10974
W20260709 13:18:09.328419 10897 debug-util.cc:398] Leaking SignalData structure 0x7b08001acce0 after lost signal to thread 10975
W20260709 13:18:09.333573 10897 debug-util.cc:398] Leaking SignalData structure 0x7b080023db60 after lost signal to thread 11536
W20260709 13:18:09.334106 10897 debug-util.cc:398] Leaking SignalData structure 0x7b0800401020 after lost signal to thread 11537
I20260709 13:18:09.338032 10916 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:09.303031 15798 consensus_queue.cc:1059] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:18:09.190852 15709 consensus_queue.cc:237] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:09.311187 15713 raft_consensus.cc:697] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:18:09.346843 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 195720us
W20260709 13:18:09.348901 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 81551.9us
I20260709 13:18:09.375422 10767 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:09.318276 10841 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:09.305974 15812 consensus_queue.cc:1059] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.079s
I20260709 13:18:09.383236 15686 consensus_queue.cc:260] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:09.384832 15686 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:09.330667 15747 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:09.389537 10800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2552) took 1183 ms. Trace:
I20260709 13:18:09.392521 10800 rpcz_store.cc:276] 0709 13:18:08.206007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:08.740543 (+534536us) service_pool.cc:224] Handling call
0709 13:18:08.740863 (+   320us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:08.876956 (+136093us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:08.876989 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:08.883509 (+  6520us) log.cc:844] Serialized 52 byte log entry
0709 13:18:09.056222 (+172713us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:09.056260 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:09.107353 (+ 51093us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b5400211bd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:09.107403 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:09.348142 (+240739us) raft_consensus.cc:1685] finished
0709 13:18:09.348162 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:09.389471 (+ 41309us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1011,"raft.queue_time_us":1887,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":11380,"spinlock_wait_cycles":287357184,"thread_start_us":93371,"threads_started":1,"wal-append.queue_time_us":287412}
I20260709 13:18:09.338989 15803 raft_consensus.cc:2749] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:09.302810 15763 consensus_queue.cc:237] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:09.403674 15822 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:09.337759 15402 consensus_queue.cc:1048] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 13:18:09.390899 15730 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:09.342306 15817 consensus_queue.cc:1048] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
W20260709 13:18:09.421350 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 238029us
W20260709 13:18:09.433341 10873 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.007s	sys 0.003s
W20260709 13:18:09.435084 10873 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.007s	sys 0.003s
I20260709 13:18:09.436144 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 2009) took 1164 ms. Trace:
I20260709 13:18:09.389470 15447 consensus_queue.cc:1048] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:18:09.390465 15818 raft_consensus.cc:2749] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:09.347761 15804 consensus_queue.cc:1059] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.087s
W20260709 13:18:09.468902 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 140370us
I20260709 13:18:09.472221 10916 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
W20260709 13:18:09.454444 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:09.433993 15822 raft_consensus.cc:697] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:18:09.467549 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 89910.4us
I20260709 13:18:09.483498 10767 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:09.416393 15826 raft_consensus.cc:2804] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:09.484794 15829 raft_consensus.cc:3055] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:09.485327 15829 raft_consensus.cc:740] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:09.399893 15815 consensus_queue.cc:1048] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:18:09.436933 10873 rpcz_store.cc:276] 0709 13:18:08.271434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:08.868782 (+597348us) service_pool.cc:224] Handling call
0709 13:18:08.869112 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:09.435625 (+566513us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:09.435680 (+    55us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:09.435741 (+    61us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:09.435788 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:09.435879 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:09.436070 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":99551,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":14004,"thread_start_us":566109,"threads_started":1}
I20260709 13:18:09.436728 15828 raft_consensus.cc:3055] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:09.420758 15747 consensus_queue.cc:1048] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 13:18:09.439869 15798 raft_consensus.cc:2749] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:09.467193 15398 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.517366 15398 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:09.490332 10916 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:09.495690 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:09.519846 15398 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:09.497071 10946 raft_consensus.cc:1275] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:18:09.520097 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:09.520852 10945 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 16.
I20260709 13:18:09.521322 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:09.521756 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:09.522002 10946 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 14.
I20260709 13:18:09.522426 10945 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 13.
I20260709 13:18:09.524804 10946 raft_consensus.cc:1240] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 14. Current term is 15. Ops: [14.7-14.7]
I20260709 13:18:09.525467 10945 pending_rounds.cc:85] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 7
W20260709 13:18:09.437207 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 313652us
I20260709 13:18:09.498683 10699 catalog_manager.cc:5697] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 6 to 7, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 7 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:09.499480 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:09.433053 15803 consensus_queue.cc:1048] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.068s
W20260709 13:18:09.501070 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:09.354838  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:09.500984 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:09.501765 15801 raft_consensus.cc:2749] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:09.503523 15804 consensus_queue.cc:1048] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:09.503930 10792 raft_consensus.cc:1275] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:18:09.507287 15730 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.491671 15826 raft_consensus.cc:697] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:18:09.516248 15813 log.cc:927] Time spent T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
I20260709 13:18:09.525811 10945 pending_rounds.cc:107] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 16.8
I20260709 13:18:09.640094 10945 raft_consensus.cc:2889] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:18:09.566203 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 75955.4us
I20260709 13:18:09.568900 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:09.492111 15807 log.cc:927] Time spent T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.209s	user 0.000s	sys 0.001s
W20260709 13:18:09.571856 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 253642us
I20260709 13:18:09.513947 15822 consensus_queue.cc:237] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:09.631765 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:09.636476 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:09.637216 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:09.648983 10873 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 13.
I20260709 13:18:09.649053 10876 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
I20260709 13:18:09.637912 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:09.650156 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:09.492398 15828 raft_consensus.cc:740] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:18:09.526983 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:09.640853 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:09.640151 10792 raft_consensus.cc:1275] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:18:09.644049 10799 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 14.
I20260709 13:18:09.645131 10841 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:09.523329 15645 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:09.650161 10869 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 14.
I20260709 13:18:09.660262 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:09.639201 10872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 2035) took 1351 ms. Trace:
I20260709 13:18:09.508159 15765 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:09.655965 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:09.660327 10767 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:09.659192 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:09.665402 10876 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
I20260709 13:18:09.666283 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:09.666890 10876 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 17.
I20260709 13:18:09.660468 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:09.661002 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:09.669629 10871 raft_consensus.cc:1275] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:18:09.670166 14970 consensus_queue.cc:1048] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:09.671810 10873 raft_consensus.cc:2393] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 16: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:09.661478 10872 rpcz_store.cc:276] 0709 13:18:08.287748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.025788 (+738040us) service_pool.cc:224] Handling call
0709 13:18:09.026036 (+   248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:09.026217 (+   181us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:18:09.281967 (+255750us) log.cc:844] Serialized 14 byte log entry
0709 13:18:09.638833 (+356866us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:09.638886 (+    53us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:09.638943 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:09.639020 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:09.639144 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":288208,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"thread_start_us":611792,"threads_started":2}
I20260709 13:18:09.676074 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:09.670719 11478 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:09.665071 10767 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:09.679879 11478 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:09.680343 10767 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:18:09.682330 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:09.682830 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:09.683439 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:09.475435 15713 consensus_queue.cc:237] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:09.683040 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:09.684362 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:09.687685 11509 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 14.
I20260709 13:18:09.577934 15837 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:09.640760 15788 raft_consensus.cc:2749] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:09.685228 10797 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:09.652240 15835 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:09.662922 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 386733us
I20260709 13:18:09.648185 15801 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.486218 15829 consensus_queue.cc:260] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:09.695154 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 166961us
I20260709 13:18:09.686223 10869 raft_consensus.cc:1240] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 14. Current term is 15. Ops: [14.7-14.7]
I20260709 13:18:09.708115 10766 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:09.697692 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:09.713110 10766 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:09.698237 11477 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:09.703840 15839 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:09.704768 15827 log.cc:927] Time spent T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.218s	user 0.000s	sys 0.000s
W20260709 13:18:09.705456 15810 log.cc:927] Time spent T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.264s	user 0.000s	sys 0.001s
W20260709 13:18:09.720973 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:09.725453 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:09.802137 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:09.727438 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:09.729401 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:09.697165 15823 raft_consensus.cc:2764] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader election decision vote started in defunct term 15: won
I20260709 13:18:09.807613 10916 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:18:09.762317 15807 log.cc:927] Time spent T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:18:09.683655 15833 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:09.698758 15798 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.792498 10797 raft_consensus.cc:1275] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:18:09.655355 15730 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:09.775012 10948 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 14.
I20260709 13:18:09.769522 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:09.852519 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:09.823346 10916 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:09.815721 15590 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.813005 10795 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:09.793342 15398 consensus_queue.cc:1048] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 13:18:09.873148 15754 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:09.876078 15804 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:09.876617 15804 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:09.795166 15829 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:18:09.843637 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:09.801187 15402 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.729699 15843 consensus_queue.cc:1048] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.116s
I20260709 13:18:09.814837 15805 raft_consensus.cc:2804] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:09.716617 15801 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:09.801442 15747 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:09.758352 15839 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:09.716502 15835 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:09.794533 15828 consensus_queue.cc:260] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:09.865418 15831 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:09.865828 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 2037) took 1577 ms. Trace:
I20260709 13:18:09.925585 10867 rpcz_store.cc:276] 0709 13:18:08.288315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.028552 (+740237us) service_pool.cc:224] Handling call
0709 13:18:09.028833 (+   281us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:09.029125 (+   292us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:09.029155 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:09.029583 (+   428us) log.cc:844] Serialized 26 byte log entry
0709 13:18:09.432003 (+402420us) spinlock_profiling.cc:243] Waited 22.3 ms on lock 0x7b180031e1c0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c80a5d4bc 00007f9c7ddab48b 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a550d1
0709 13:18:09.432215 (+   212us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:09.432268 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:09.432363 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:09.865435 (+433072us) raft_consensus.cc:1685] finished
0709 13:18:09.865475 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:09.865741 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":141070,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":62669,"spinlock_wait_cycles":51323648,"thread_start_us":211632,"threads_started":1,"wal-append.queue_time_us":363232}
W20260709 13:18:09.790463 15708 consensus_queue.cc:1175] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "12682eb70bc84c76879e04a8628ee586" responder_term: 13 status { last_received { term: 13 index: 10 } last_committed_idx: 9 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 12. Current term is 13. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:18:09.812026 15447 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.828074 15765 raft_consensus.cc:697] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 14 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:09.881342 10795 raft_consensus.cc:1275] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:18:09.934084 10942 raft_consensus.cc:1240] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 12 LEADER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 11. Current term is 12. Ops: []
W20260709 13:18:09.810380 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 129762us
W20260709 13:18:09.821231 15642 consensus_meta.cc:287] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.786s	user 0.006s	sys 0.005s
I20260709 13:18:09.817214 15639 raft_consensus.cc:2749] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:09.822903 15841 raft_consensus.cc:2749] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:09.820257 15837 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.855743 15823 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:09.883919 15804 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:18:09.931174 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 107927us
I20260709 13:18:09.906612 15805 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:09.850723 15798 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:09.943498 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 230340us
I20260709 13:18:09.731961 15826 consensus_queue.cc:237] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:09.912814 15747 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:09.882565 15590 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:09.962692 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:09.917599 15842 raft_consensus.cc:2749] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:09.907160 15402 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:09.955281 15823 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:09.961822 10916 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:09.967209 10870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 747) took 1478 ms. Trace:
W20260709 13:18:09.977389 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 332298us
I20260709 13:18:09.978348 10870 rpcz_store.cc:276] 0709 13:18:08.488903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.034517 (+545614us) service_pool.cc:224] Handling call
0709 13:18:09.034756 (+   239us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:09.063671 (+ 28915us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:18:09.265518 (+201847us) log.cc:844] Serialized 14 byte log entry
0709 13:18:09.966721 (+701203us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:09.966814 (+    93us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:09.966881 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:09.966974 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:09.967139 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":307880,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":41,"raft.queue_time_us":117729,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":32059,"thread_start_us":836196,"threads_started":3,"wal-append.queue_time_us":705578}
I20260709 13:18:09.978878 10841 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:09.985611 10699 catalog_manager.cc:5697] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 10 to 11, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 11 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:09.959837 15805 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:09.949237 15742 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.965853 15747 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:09.994570 10916 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:09.970052 15828 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:09.951045 15447 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:10.003098 10841 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:09.965919 15837 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:09.937314 15831 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:09.987505 15823 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:09.955308 15642 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:10.011870 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:10.021173 10697 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42984 (request call id 246) took 25073 ms (client timeout 15000 ms). Trace:
I20260709 13:18:09.988021 15639 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.989666 15645 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:18:10.032189 10697 rpcz_store.cc:269] 0709 13:17:44.947689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:44.956263 (+  8574us) service_pool.cc:224] Handling call
0709 13:18:10.020920 (+25064657us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:02.769514 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:02.770301 (+   787us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:02.770361 (+    60us) write_op.cc:435] Acquired schema lock
0709 13:18:02.770383 (+    22us) tablet.cc:662] Decoding operations
0709 13:18:02.772132 (+  1749us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:02.772220 (+    88us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:02.772254 (+    34us) tablet.cc:685] Acquiring locks for 18 operations
0709 13:18:02.773834 (+  1580us) tablet.cc:701] Row locks acquired
0709 13:18:02.773882 (+    48us) write_op.cc:260] PREPARE: finished
0709 13:18:02.774078 (+   196us) write_op.cc:270] Start()
0709 13:18:02.774243 (+   165us) write_op.cc:276] Timestamp: P: 1783603082774006 usec, L: 0
0709 13:18:02.774265 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:18:02.812927 (+ 38662us) log.cc:844] Serialized 7171 byte log entry
0709 13:18:04.156436 (+1343509us) op_driver.cc:464] REPLICATION: finished
0709 13:18:04.853394 (+696958us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c7ddab0ac 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c848ebef0
0709 13:18:05.047904 (+194510us) write_op.cc:301] APPLY: starting
0709 13:18:05.048043 (+   139us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:05.432202 (+384159us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:05.444619 (+ 12417us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:06.081548 (+636929us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:06.177002 (+ 95454us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=36,key_file_lookups=36,delta_file_lookups=36,mrs_lookups=0
0709 13:18:06.177051 (+    49us) write_op.cc:312] APPLY: finished
0709 13:18:06.177626 (+   575us) log.cc:844] Serialized 164 byte log entry
0709 13:18:06.247238 (+ 69612us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:06.291713 (+ 44475us) write_op.cc:454] Released schema lock
0709 13:18:06.307548 (+ 15835us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:06.319742 (+ 12194us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:06.319774 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":784035,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":874096,"apply.run_cpu_time_us":48375,"apply.run_wall_time_us":1271877,"cfile_cache_hit":74,"cfile_cache_hit_bytes":79403,"num_ops":18,"prepare.queue_time_us":970590,"prepare.run_cpu_time_us":55124,"prepare.run_wall_time_us":402663,"raft.queue_time_us":276094,"raft.run_cpu_time_us":19196,"raft.run_wall_time_us":1007087,"replication_time_us":1237627,"spinlock_wait_cycles":348083456,"thread_start_us":2011057,"threads_started":5,"wal-append.queue_time_us":529272,"wal-append.run_cpu_time_us":33287,"wal-append.run_wall_time_us":2620120}]]}
I20260709 13:18:10.015377 15835 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:10.045063 15838 raft_consensus.cc:2749] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:18:10.046386 15844 log.cc:927] Time spent T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.276s	user 0.000s	sys 0.001s
W20260709 13:18:10.036538 15832 log.cc:927] Time spent T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.321s	user 0.000s	sys 0.001s
W20260709 13:18:10.048460 15846 log.cc:927] Time spent T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
I20260709 13:18:10.048840 15861 consensus_queue.cc:1048] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:18:10.050431 15755 consensus_meta.cc:287] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.785s	user 0.007s	sys 0.001s
W20260709 13:18:10.052347 15836 log.cc:927] Time spent T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 13:18:10.038676 15639 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:10.053778 15686 consensus_meta.cc:287] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.668s	user 0.008s	sys 0.001s
I20260709 13:18:09.999994 15841 raft_consensus.cc:2749] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:18:10.072293 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:10.072770 10868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 748) took 1557 ms. Trace:
I20260709 13:18:10.141654 10868 rpcz_store.cc:276] 0709 13:18:08.515481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.047561 (+532080us) service_pool.cc:224] Handling call
0709 13:18:09.047876 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:09.048075 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:18:09.192239 (+144164us) log.cc:844] Serialized 14 byte log entry
0709 13:18:09.701757 (+509518us) log.cc:844] Serialized 14 byte log entry
0709 13:18:10.070927 (+369170us) log.cc:844] Serialized 14 byte log entry
0709 13:18:10.071972 (+  1045us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:10.072231 (+   259us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:18:10.072376 (+   145us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:10.072520 (+   144us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:10.072703 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12029,"prepare.queue_time_us":306248,"prepare.run_cpu_time_us":172,"prepare.run_wall_time_us":7157,"raft.queue_time_us":503512,"raft.run_cpu_time_us":344,"raft.run_wall_time_us":1015,"thread_start_us":1000613,"threads_started":4,"wal-append.queue_time_us":673001}
I20260709 13:18:10.007453 15853 raft_consensus.cc:2804] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:10.077472 10794 raft_consensus.cc:1240] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:10.088637 10699 catalog_manager.cc:5697] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
W20260709 13:18:10.090049 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 391875us
W20260709 13:18:10.145318 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 182740us
W20260709 13:18:10.122195 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 127697us
W20260709 13:18:10.139900 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 203363us
I20260709 13:18:09.996457 15742 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:10.147924 11477 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:10.155885 10916 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:10.156519 10792 raft_consensus.cc:1275] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 13:18:10.156934 10792 pending_rounds.cc:85] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 9
I20260709 13:18:10.157162 10792 pending_rounds.cc:107] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 13:18:10.157404 10792 raft_consensus.cc:2889] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:18:10.159415 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 127092us
I20260709 13:18:10.160211 10841 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:10.161310 10841 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:10.161983 10699 catalog_manager.cc:5697] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 7 to 8, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 8 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
W20260709 13:18:10.164443 10800 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.004s	sys 0.024s
W20260709 13:18:10.165673 10800 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.005s	sys 0.024s
I20260709 13:18:10.022315 15834 consensus_queue.cc:1048] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.228s
I20260709 13:18:10.171172 10916 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:10.038956 15645 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:10.174396 10916 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:10.151633 15853 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:10.151602 15755 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:10.178303 10699 catalog_manager.cc:5697] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 9 to 12, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 12 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:10.186517 10699 catalog_manager.cc:5697] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 7 to 8, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 8 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:18:10.203819 10950 raft_consensus.cc:1240] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 14 LEADER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:10.205451 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:10.214268 10916 leader_election.cc:400] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 12. Current term is 13.
I20260709 13:18:10.214715 10916 leader_election.cc:403] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:18:10.215907 10916 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:10.217896 10916 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:10.227931 10916 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:10.229753 10916 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:10.178453 15838 raft_consensus.cc:2749] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:10.056098 15765 consensus_queue.cc:237] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:10.184620 15841 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:10.224617 10699 catalog_manager.cc:5697] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 5 to 8, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 8 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:10.252735 10874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 753) took 1440 ms. Trace:
I20260709 13:18:10.253223 10874 rpcz_store.cc:276] 0709 13:18:08.812342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.074920 (+262578us) service_pool.cc:224] Handling call
0709 13:18:09.077270 (+  2350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:09.243272 (+166002us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:09.243303 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:09.243792 (+   489us) log.cc:844] Serialized 26 byte log entry
0709 13:18:09.424035 (+180243us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:09.424078 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:09.424155 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:10.252500 (+828345us) raft_consensus.cc:1685] finished
0709 13:18:10.252521 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:10.252669 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":186345,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":7081,"thread_start_us":228917,"threads_started":2,"wal-append.queue_time_us":645955}
I20260709 13:18:10.257886 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:10.259989 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:10.260725 10875 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 14.
I20260709 13:18:10.192866 15713 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:10.265138 10916 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:10.193549 15853 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:18:10.352460 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 87439.4us
I20260709 13:18:10.357316 15859 raft_consensus.cc:2749] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:10.360420 15742 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:10.361290 10798 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 12.
I20260709 13:18:10.242841 15817 raft_consensus.cc:2749] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:10.362351 15409 raft_consensus.cc:2749] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:10.364691 10875 raft_consensus.cc:1275] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:18:10.238603 15124 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:10.233904 15709 raft_consensus.cc:2749] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 12. Current term is 13.
I20260709 13:18:10.238452 15843 raft_consensus.cc:2804] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:10.184425 15398 consensus_queue.cc:1048] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.075s
I20260709 13:18:10.243908 15763 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
W20260709 13:18:10.380870 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 219596us
I20260709 13:18:10.277884 15842 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:10.266579 15801 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:10.258700 10874 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 14.
I20260709 13:18:10.351316 15833 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:10.239019 15812 consensus_queue.cc:1048] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.141s
W20260709 13:18:10.442013 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:10.401188 15851 log.cc:927] Time spent T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
W20260709 13:18:10.442891 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:10.382000 15829 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
W20260709 13:18:10.398164 15683 consensus_meta.cc:287] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 1.364s	user 0.015s	sys 0.000s
I20260709 13:18:10.384004 15755 raft_consensus.cc:2749] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:10.403136 15402 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:10.439731 15839 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:10.443423 15852 consensus_queue.cc:1059] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.459s
W20260709 13:18:10.613848 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 896526us
I20260709 13:18:10.559590 10841 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
W20260709 13:18:10.620203 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:10.615813 10948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41014 (request call id 2161) took 1114 ms. Trace:
I20260709 13:18:10.394232 15590 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:10.405174 15854 raft_consensus.cc:2804] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:10.778350 15874 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:10.778853 15854 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:10.779937 15892 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:10.780004 15854 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:10.780542 15892 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:10.781543 15747 consensus_meta.cc:287] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.752s	user 0.010s	sys 0.006s
I20260709 13:18:10.781991 15747 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:10.712143 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:10.437726 15713 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:10.404959 15802 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:10.784055 15802 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:10.467327 15843 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:10.785456 15843 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:10.785696 15747 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:10.786463 15837 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:10.597935 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:10.467067 15124 consensus_queue.cc:1059] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.321s
W20260709 13:18:10.790256 15804 consensus_meta.cc:287] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.824s	user 0.003s	sys 0.003s
I20260709 13:18:10.778997 15874 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:10.592383 15833 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:10.579145 15726 consensus_meta.cc:287] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 1.457s	user 0.012s	sys 0.002s
W20260709 13:18:10.593784 15862 log.cc:927] Time spent T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.432s	user 0.005s	sys 0.000s
I20260709 13:18:10.784767 15713 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:10.592702 15887 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:10.776808 10948 rpcz_store.cc:276] 0709 13:18:09.501809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.532125 (+ 30316us) service_pool.cc:224] Handling call
0709 13:18:10.615612 (+1083487us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b5400173754. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:10.615715 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":284443392}
I20260709 13:18:10.796070 11478 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:10.478927 15845 raft_consensus.cc:2749] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:18:10.593926 15871 log.cc:927] Time spent T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.098s	user 0.002s	sys 0.003s
W20260709 13:18:10.799527 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 240328us
I20260709 13:18:10.482184 15870 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:10.597632 15863 log.cc:927] Time spent T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.308s	user 0.001s	sys 0.000s
I20260709 13:18:10.609791 15886 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:10.820429 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:10.478037 15642 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:10.479102 15865 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:10.592180 15831 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:10.715484 15803 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:10.592971 15856 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:10.594159 15860 log.cc:927] Time spent T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.205s	user 0.005s	sys 0.000s
I20260709 13:18:10.479156 15878 raft_consensus.cc:2804] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:10.480046 15875 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:10.591125 15842 raft_consensus.cc:697] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:18:10.792771 15805 consensus_meta.cc:287] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.795s	user 0.010s	sys 0.002s
W20260709 13:18:10.790256 15744 consensus_meta.cc:287] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 1.626s	user 0.019s	sys 0.001s
I20260709 13:18:10.554658 15686 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:10.480187 15858 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:10.836361 15639 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:10.822628 10871 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.007s	sys 0.013s
W20260709 13:18:10.896020 10871 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.007s	sys 0.013s
I20260709 13:18:10.896109 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:10.482023 15798 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:10.808934 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:10.833856 11510 raft_consensus.cc:1240] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 14 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:18:10.792577 15874 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:10.571292 15763 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:10.855582 10868 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.010s	sys 0.019s
I20260709 13:18:10.782989 15720 consensus_queue.cc:1048] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 13:18:10.594977 15829 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:10.479190 15847 consensus_queue.cc:1048] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.676s
I20260709 13:18:10.554878 15859 consensus_queue.cc:1048] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.279s
I20260709 13:18:10.867818 10841 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:10.726424 15683 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:10.882284 10766 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:18:10.856911 15794 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.589s	user 0.009s	sys 0.015s
I20260709 13:18:10.804544 15726 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:11.002444 15794 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.760s	user 0.010s	sys 0.016s
I20260709 13:18:11.003798 15794 consensus_queue.cc:1243] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer cb0428cc55984daba1ba63744a5102a2 log is divergent from this leader: its last log entry 16.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:18:11.003026 10801 raft_consensus.cc:1240] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:11.005012 15852 raft_consensus.cc:2749] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:10.715041 15801 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.006093 15801 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:10.860778 15803 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:11.007376 15701 consensus_queue.cc:1059] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:18:11.008509 15801 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:10.898226 10874 raft_consensus.cc:1275] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:18:10.846935 15841 raft_consensus.cc:697] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 14 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:10.852726 15878 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:11.011695 15878 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:11.011523 15841 consensus_queue.cc:237] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:10.790766 15804 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:10.831893 15870 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:10.836180 15865 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:10.836074 15877 consensus_queue.cc:1059] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.093s
I20260709 13:18:10.902187 15686 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:10.909498 10942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2510) took 2016 ms. Trace:
I20260709 13:18:10.834410 15886 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:10.908461 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:10.836542 15398 consensus_queue.cc:1048] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 13:18:10.907100 15883 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:10.892645 15642 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:10.891315 15831 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:18:10.985275 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 544303us
W20260709 13:18:10.985700 10916 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.004s	sys 0.012s
W20260709 13:18:11.029759 10916 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.004s	sys 0.012s
W20260709 13:18:11.030107 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 592971us
W20260709 13:18:10.986151 10867 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.500s	user 0.006s	sys 0.009s
I20260709 13:18:11.031822 10875 raft_consensus.cc:1275] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:18:10.987802 10944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 2168) took 1482 ms. Trace:
W20260709 13:18:10.992969 10868 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.777s	user 0.010s	sys 0.019s
W20260709 13:18:10.999205 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 117006us
W20260709 13:18:11.020612 15818 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.316s	user 0.001s	sys 0.014s
I20260709 13:18:10.887238 10798 raft_consensus.cc:1275] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:18:11.008580 15803 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:10.875258 15875 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:11.044831 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:10.984571 15754 consensus_meta.cc:287] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 1.110s	user 0.012s	sys 0.007s
W20260709 13:18:10.985983 15788 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.831s	user 0.005s	sys 0.013s
I20260709 13:18:11.015461 10795 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 14.
W20260709 13:18:11.018479 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 150746us
W20260709 13:18:11.017143 10699 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.015s	sys 0.024s
W20260709 13:18:11.098167 10699 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.735s	user 0.015s	sys 0.024s
I20260709 13:18:10.889993 15744 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:10.860450 15856 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:10.871106 15805 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:10.998114 15715 consensus_meta.cc:287] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 1.691s	user 0.018s	sys 0.003s
W20260709 13:18:11.002055 15800 consensus_meta.cc:287] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 1.679s	user 0.014s	sys 0.005s
I20260709 13:18:11.099763 15800 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:11.025760 10942 rpcz_store.cc:276] 0709 13:18:08.893185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:08.903552 (+ 10367us) service_pool.cc:224] Handling call
0709 13:18:08.964140 (+ 60588us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:09.934674 (+970534us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:10.908388 (+973714us) spinlock_profiling.cc:243] Waited 970 ms on lock 0x7b54000159d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:10.909436 (+  1048us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2230254976}
I20260709 13:18:10.997360 15889 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:10.987121 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:11.035094 10766 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:11.005080 15794 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:11.032835 10944 rpcz_store.cc:276] 0709 13:18:09.505526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.563843 (+ 58317us) service_pool.cc:224] Handling call
0709 13:18:09.633470 (+ 69627us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:09.661864 (+ 28394us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:09.661891 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:09.675848 (+ 13957us) log.cc:844] Serialized 26 byte log entry
0709 13:18:09.977944 (+302096us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:09.978116 (+   172us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:09.978213 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:10.987570 (+1009357us) raft_consensus.cc:1685] finished
0709 13:18:10.987593 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:10.987739 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":117176,"raft.queue_time_us":48125,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":9879,"spinlock_wait_cycles":21248,"thread_start_us":219688,"threads_started":1,"wal-append.queue_time_us":520627}
I20260709 13:18:11.004280 15726 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:11.041857 10796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2036) took 1314 ms. Trace:
I20260709 13:18:11.022954 15801 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.014286 15854 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:10.815392 15887 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:11.025506 15892 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:11.143877 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:11.144846 10942 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 11.
I20260709 13:18:11.036681 15642 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:11.031706 15883 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:18:11.037547 15818 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.334s	user 0.002s	sys 0.014s
I20260709 13:18:11.103070 10841 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:11.104003 10943 raft_consensus.cc:2410] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:18:11.047906 15831 raft_consensus.cc:697] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 12 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
W20260709 13:18:11.123353 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 88353.6us
I20260709 13:18:11.116184 10916 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:18:11.167184 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 51116.3us
I20260709 13:18:11.167280 10766 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:18:11.088795 15788 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.927s	user 0.005s	sys 0.013s
I20260709 13:18:11.138971 10941 raft_consensus.cc:2368] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for earlier term 12. Current term is 13.
W20260709 13:18:11.068493 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 173669us
I20260709 13:18:11.130439 10796 rpcz_store.cc:276] 0709 13:18:09.727908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.803719 (+ 75811us) service_pool.cc:224] Handling call
0709 13:18:09.804070 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:09.804467 (+   397us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:09.804509 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:09.813704 (+  9195us) log.cc:844] Serialized 26 byte log entry
0709 13:18:09.917514 (+103810us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:09.969667 (+ 52153us) log.cc:844] Serialized 14 byte log entry
0709 13:18:10.264635 (+294968us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:10.264743 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:11.041638 (+776895us) raft_consensus.cc:1685] finished
0709 13:18:11.041659 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:11.041801 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43765,"prepare.queue_time_us":673974,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":262517,"raft.run_cpu_time_us":377,"raft.run_wall_time_us":49623,"thread_start_us":384551,"threads_started":3,"wal-append.queue_time_us":143280}
I20260709 13:18:11.150882 10869 raft_consensus.cc:1240] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:11.150997 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:18:11.167873 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 64869.3us
I20260709 13:18:11.172333 10841 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:11.087762 15765 consensus_queue.cc:1048] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.085s
I20260709 13:18:11.130151 15826 raft_consensus.cc:2749] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:11.182757 10766 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:11.174515 10875 raft_consensus.cc:1240] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:11.114900 15803 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:11.185115 10841 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:11.163710 15883 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:11.172670 15833 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:11.172309 15754 raft_consensus.cc:2749] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:11.135679 15794 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:11.155280 15801 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:11.257323 10841 connection.cc:256] RPC call timeout handler was delayed by -2.1309714559999975s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1519s for a response
I20260709 13:18:11.186467 15757 raft_consensus.cc:2749] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:11.137850 15843 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:11.261376 10841 connection.cc:256] RPC call timeout handler was delayed by -2.0175714560000415s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1645s for a response
W20260709 13:18:11.264097 10841 connection.cc:256] RPC call timeout handler was delayed by -1.9266714559998945s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1746s for a response
I20260709 13:18:11.265158 10841 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:11.192097 15726 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:11.268151 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:18:11.277730 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:11.264355 10767 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:11.289084 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:11.130064 15889 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:11.036552 15842 consensus_queue.cc:237] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:11.295861 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:11.295970 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:11.314956 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:11.316054 11503 raft_consensus.cc:2368] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 15. Current term is 16.
I20260709 13:18:11.316540 10767 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:11.258826 15893 raft_consensus.cc:2749] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:11.317402 15899 raft_consensus.cc:2804] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:11.317924 15899 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:11.317981 15893 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.318424 15899 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:11.261642 15834 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:11.291790 15828 consensus_meta.cc:287] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 1.281s	user 0.017s	sys 0.000s
I20260709 13:18:11.269783 15726 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:11.293072 15823 consensus_meta.cc:287] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 1.237s	user 0.007s	sys 0.010s
I20260709 13:18:11.294497 15754 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.243563 15803 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:11.327522 11507 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
I20260709 13:18:11.327651 10872 raft_consensus.cc:1275] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:18:11.329839 15896 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:11.318733 15893 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:11.335409 15855 consensus_queue.cc:1048] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.142s
I20260709 13:18:11.336128 15872 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:11.336726 15857 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.317112 15757 raft_consensus.cc:2749] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:11.325218 15726 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:18:11.338356 15890 log.cc:927] Time spent T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.347s	user 0.000s	sys 0.001s
I20260709 13:18:11.326664 15763 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:11.329167 15826 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:18:11.363265 15897 log.cc:927] Time spent T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.339s	user 0.001s	sys 0.000s
I20260709 13:18:11.337792 15845 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.324816 15852 consensus_queue.cc:1048] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:18:11.328578 15834 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:11.374343 15826 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:11.373512 15829 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:11.326889 15891 consensus_queue.cc:1048] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.334s
I20260709 13:18:11.355592 15754 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:11.332417 15823 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:11.404522 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 139427us
W20260709 13:18:11.395609 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 224711us
I20260709 13:18:11.405674 10841 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:11.384842 15896 raft_consensus.cc:697] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:11.338112 15831 consensus_queue.cc:237] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:11.399955 15894 log.cc:927] Time spent T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.254s	user 0.000s	sys 0.001s
I20260709 13:18:11.392692 15865 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:11.422222 10800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 670) took 1854 ms. Trace:
I20260709 13:18:11.423844 10841 leader_election.cc:400] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 15. Current term is 16.
I20260709 13:18:11.400383 15826 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:11.395015 15757 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
W20260709 13:18:11.401340 15904 log.cc:927] Time spent T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 13:18:11.425848 15794 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:11.434281 15642 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:11.384124 15845 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:11.438670 15893 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:11.439913 15803 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:11.440538 15865 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.404472 15804 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:11.429303 11478 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:11.422643 10800 rpcz_store.cc:276] 0709 13:18:09.567839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.568066 (+   227us) service_pool.cc:224] Handling call
0709 13:18:09.574137 (+  6071us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:10.166735 (+592598us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:10.166766 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:10.175301 (+  8535us) log.cc:844] Serialized 26 byte log entry
0709 13:18:10.417225 (+241924us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:10.417273 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:10.417343 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:11.422010 (+1004667us) raft_consensus.cc:1685] finished
0709 13:18:11.422029 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:11.422158 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":101819,"raft.queue_time_us":137916,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":13917,"thread_start_us":715266,"threads_started":2,"wal-append.queue_time_us":688039}
I20260709 13:18:11.367195 15857 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:11.408453 15891 consensus_queue.cc:1243] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer 468a8fd46e824f62b9c2c0d3532a6d26 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:18:11.463804 15893 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:11.464444 15893 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:11.465615 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:11.466647 15686 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:11.467545 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:11.468405 10800 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 12.
I20260709 13:18:11.468935 10868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2107) took 1310 ms. Trace:
I20260709 13:18:11.469594 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:11.471014 15923 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:11.469952 15896 consensus_queue.cc:237] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:11.469321 10868 rpcz_store.cc:276] 0709 13:18:10.158295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:10.158482 (+   187us) service_pool.cc:224] Handling call
0709 13:18:10.158882 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:10.159138 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:18:10.214702 (+ 55564us) log.cc:844] Serialized 14 byte log entry
0709 13:18:11.468469 (+1253767us) spinlock_profiling.cc:243] Waited 4.28 ms on lock 0x7b1800227740. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c80a5d4bc 00007f9c7ddab48b 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a5523a
0709 13:18:11.468642 (+   173us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:11.468688 (+    46us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:11.468728 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:11.468787 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:11.468883 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5828,"prepare.queue_time_us":53223,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":43,"raft.queue_time_us":3601,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":110,"spinlock_wait_cycles":9838976,"thread_start_us":818854,"threads_started":1,"wal-append.queue_time_us":816646}
I20260709 13:18:11.404547 15800 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:11.424401 10841 leader_election.cc:403] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:11.441901 10945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 2165) took 1937 ms. Trace:
I20260709 13:18:11.408368 15854 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:11.477072 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 53429.6us
I20260709 13:18:11.452171 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2109) took 1280 ms. Trace:
I20260709 13:18:11.477514 15854 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:18:11.477854 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:11.457684 15801 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:11.422255 15398 consensus_queue.cc:1048] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 13:18:11.477216 10945 rpcz_store.cc:276] 0709 13:18:09.504273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.524267 (+ 19994us) service_pool.cc:224] Handling call
0709 13:18:09.525270 (+  1003us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:09.724325 (+199055us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:09.724355 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:11.336777 (+1612422us) spinlock_profiling.cc:243] Waited 26.3 ms on lock 0x7b18003b7820. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7dd962a6 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7f7aea11 00007f9c7f7be372 00007f9c80ef0a01 00007f9c80eb543a
0709 13:18:11.337304 (+   527us) spinlock_profiling.cc:243] Waited 7.39 ms on lock 0x7b2c0015d218. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7f7aea11 00007f9c7f7be372 00007f9c80ef0a01 00007f9c80eb543a 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2
0709 13:18:11.365648 (+ 28344us) log.cc:844] Serialized 52 byte log entry
0709 13:18:11.426556 (+ 60908us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:11.426608 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:11.427437 (+   829us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:11.441666 (+ 14229us) raft_consensus.cc:1685] finished
0709 13:18:11.441687 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:11.441838 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":116121,"raft.queue_time_us":76233,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":35652,"spinlock_wait_cycles":77460096,"thread_start_us":35857,"threads_started":1,"wal-append.queue_time_us":62814}
W20260709 13:18:11.479660 10841 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.116.131:45553 timed out after 1.746s (SENT) [suppressed 7 similar messages]
I20260709 13:18:11.479759 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:11.481027 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:11.458307 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2111) took 1284 ms. Trace:
I20260709 13:18:11.432368 15859 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:11.482967 10869 rpcz_store.cc:276] 0709 13:18:10.173991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:10.174098 (+   107us) service_pool.cc:224] Handling call
0709 13:18:10.222079 (+ 47981us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:11.171390 (+949311us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:11.458052 (+286662us) spinlock_profiling.cc:243] Waited 924 ms on lock 0x7b540006c7d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:11.458239 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2125935872}
I20260709 13:18:11.484375 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:11.484844 15859 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:11.484995 10869 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 14.
I20260709 13:18:11.485351 15859 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:11.436127 15911 consensus_queue.cc:1048] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:18:11.487075 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:11.421686 15886 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:11.445380 15757 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:11.487126 15843 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:11.490053 15757 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:11.459496 15928 raft_consensus.cc:2749] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:11.490439 15843 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.482231 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:11.490294 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:11.491042 15843 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:11.491503 10875 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 12.
I20260709 13:18:11.491703 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:11.492362 10950 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:11.492424 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:11.493222 10875 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:11.493353 15843 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:11.495071 10868 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:11.459517 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:11.517560 10876 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
I20260709 13:18:11.518836 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:11.519829 10875 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 16.
I20260709 13:18:11.460780 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:18:11.497852 15914 log.cc:927] Time spent T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 13:18:11.478157 10867 rpcz_store.cc:276] 0709 13:18:10.171581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:10.172741 (+  1160us) service_pool.cc:224] Handling call
0709 13:18:10.173065 (+   324us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:18:10.380129 (+207064us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:18:10.437629 (+ 57500us) log.cc:844] Serialized 14 byte log entry
0709 13:18:10.906541 (+468912us) spinlock_profiling.cc:243] Waited 3.74 ms on lock 0x7b48001602d8. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c85ddeaa1 00007f9c85dd5b37 00007f9c85dd59ca 00007f9c85ecd26f 00007f9c85eccf37 00007f9c85ee155c 00007f9c85ee1508 00007f9c85ee1488 00007f9c85ee1443 00007f9c85ee06ec 00007f9c84e0aa96
0709 13:18:11.031133 (+124592us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:18:11.048354 (+ 17221us) log.cc:844] Serialized 78 byte log entry
0709 13:18:11.048518 (+   164us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:11.048722 (+   204us) log.cc:844] Serialized 14 byte log entry
0709 13:18:11.048920 (+   198us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:11.048989 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:11.450579 (+401590us) raft_consensus.cc:1685] finished
0709 13:18:11.450607 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:11.452084 (+  1477us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":93439,"prepare.queue_time_us":57519,"raft.queue_time_us":272347,"raft.run_cpu_time_us":2676,"raft.run_wall_time_us":494768,"spinlock_wait_cycles":20381824,"thread_start_us":738502,"threads_started":2,"wal-append.queue_time_us":549046}
I20260709 13:18:11.526669 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:11.527477 10867 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 17.
I20260709 13:18:11.528615 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:11.529348 10867 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 16.
W20260709 13:18:11.499033 15901 log.cc:927] Time spent T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.182s	user 0.000s	sys 0.001s
I20260709 13:18:11.499447 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:11.441094 15865 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:11.499903 10794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2029) took 3094 ms. Trace:
I20260709 13:18:11.535475 10794 rpcz_store.cc:276] 0709 13:18:08.405260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:08.772339 (+367079us) service_pool.cc:224] Handling call
0709 13:18:08.772590 (+   251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:10.146074 (+1373484us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:11.499600 (+1353526us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b54000d8c54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:11.499823 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3000448256}
I20260709 13:18:11.537077 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:11.500855 10792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 676) took 1841 ms. Trace:
I20260709 13:18:11.500561 15826 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:11.500990 10793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2039) took 1772 ms. Trace:
I20260709 13:18:11.501250 15754 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:11.504989 15893 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:11.470499 10800 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 10.
I20260709 13:18:11.510596 15757 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:11.510907 15913 log.cc:927] Time spent T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 13:18:11.554194 15447 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:11.511307 15887 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:11.434682 15879 log.cc:927] Time spent T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.323s	user 0.001s	sys 0.000s
I20260709 13:18:11.556298 10871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2110) took 1384 ms. Trace:
I20260709 13:18:11.427273 15870 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:11.496260 15783 raft_consensus.cc:3055] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:11.557257 15783 raft_consensus.cc:740] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:11.556679 10871 rpcz_store.cc:276] 0709 13:18:10.171867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:10.172598 (+   731us) service_pool.cc:224] Handling call
0709 13:18:10.213679 (+ 41081us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:18:10.896412 (+682733us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:10.896444 (+    32us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 13:18:10.904540 (+  8096us) log.cc:844] Serialized 104 byte log entry
0709 13:18:11.135597 (+231057us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:11.324257 (+188660us) log.cc:844] Serialized 14 byte log entry
0709 13:18:11.358498 (+ 34241us) log.cc:844] Serialized 14 byte log entry
0709 13:18:11.364233 (+  5735us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:11.364444 (+   211us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:11.556051 (+191607us) raft_consensus.cc:1685] finished
0709 13:18:11.556072 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:11.556228 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48123,"prepare.queue_time_us":292335,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":62,"raft.queue_time_us":319881,"raft.run_cpu_time_us":445,"raft.run_wall_time_us":13918,"thread_start_us":923216,"threads_started":3,"wal-append.queue_time_us":319628}
I20260709 13:18:11.558352 15783 consensus_queue.cc:260] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:11.559103 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:11.559695 10871 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 14.
I20260709 13:18:11.559770 15783 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:11.560629 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:11.561146 10871 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:11.561866 15854 raft_consensus.cc:2749] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 15. Current term is 16.
I20260709 13:18:11.475890 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:11.524580 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:11.524672 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:11.531749 11478 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:11.571174 10948 raft_consensus.cc:2393] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 14: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:11.530368 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:11.571404 10875 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 10.
I20260709 13:18:11.537866 10792 rpcz_store.cc:276] 0709 13:18:09.659263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.659623 (+   360us) service_pool.cc:224] Handling call
0709 13:18:09.659925 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:10.157830 (+497905us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:11.500632 (+1342802us) spinlock_profiling.cc:243] Waited 496 ms on lock 0x7b540024ead4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:11.500801 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1141263616}
I20260709 13:18:11.536770 15919 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:11.538055 10793 rpcz_store.cc:276] 0709 13:18:09.728759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.868166 (+139407us) service_pool.cc:224] Handling call
0709 13:18:09.868658 (+   492us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:10.088085 (+219427us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:10.088128 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:10.134699 (+ 46571us) log.cc:844] Serialized 26 byte log entry
0709 13:18:10.207203 (+ 72504us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:10.207238 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:11.500628 (+1293390us) spinlock_profiling.cc:243] Waited 64.5 ms on lock 0x7b54000e9054. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:11.500682 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:11.500724 (+    42us) raft_consensus.cc:1685] finished
0709 13:18:11.500734 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:11.500938 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21415,"raft.queue_time_us":74752,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":22860,"spinlock_wait_cycles":148413184,"thread_start_us":191155,"threads_started":2,"wal-append.queue_time_us":89710,"wal-append.run_cpu_time_us":2057,"wal-append.run_wall_time_us":1120408}
I20260709 13:18:11.543490 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:11.587785 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:11.588444 10794 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
I20260709 13:18:11.549578 15757 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:11.550829 10947 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41020 (request call id 2507) took 2662 ms (client timeout 1746 ms). Trace:
I20260709 13:18:11.588882 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:11.589387 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:11.552222 15826 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:11.537662 15911 consensus_queue.cc:1059] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 13:18:11.552609 15726 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:11.553124 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:11.597875 10800 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 16.
I20260709 13:18:11.597772 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:11.573156 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:11.573647 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:11.589954 10874 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 13.
I20260709 13:18:11.599368 10792 raft_consensus.cc:2393] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 13: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
W20260709 13:18:11.589752 10947 rpcz_store.cc:269] 0709 13:18:08.888246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:08.888365 (+   119us) service_pool.cc:224] Handling call
0709 13:18:11.550527 (+2662162us) spinlock_profiling.cc:243] Waited 772 ms on lock 0x7b54001e1c54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:11.550688 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1776368000}
I20260709 13:18:11.589077 10875 raft_consensus.cc:2410] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
W20260709 13:18:11.590451 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 58802us
I20260709 13:18:11.589963 10794 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 12.
W20260709 13:18:11.601696 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 417167us
I20260709 13:18:11.592494 10842 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:18:11.595067 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 115416us
I20260709 13:18:11.596267 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:11.603133 10793 raft_consensus.cc:2393] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 15: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:11.569957 10871 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 15.
I20260709 13:18:11.607044 10868 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 13.
I20260709 13:18:11.609329 10868 raft_consensus.cc:1275] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:18:11.611922 10867 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:18:11.597771 15914 log.cc:927] Time spent T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:18:11.615372 15927 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:11.597822 15816 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:11.619387 10842 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:18:11.604696 15876 log.cc:927] Time spent T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.932s	user 0.001s	sys 0.001s
I20260709 13:18:11.627064 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:11.628006 10800 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 16.
I20260709 13:18:11.634425 10867 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 14.
W20260709 13:18:11.635275 15902 log.cc:927] Time spent T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.459s	user 0.000s	sys 0.003s
I20260709 13:18:11.637387 10797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2038) took 1908 ms. Trace:
I20260709 13:18:11.638631 15713 raft_consensus.cc:3055] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:11.639068 15713 raft_consensus.cc:740] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:11.637807 10797 rpcz_store.cc:276] 0709 13:18:09.728481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.822619 (+ 94138us) service_pool.cc:224] Handling call
0709 13:18:09.823025 (+   406us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:09.860776 (+ 37751us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:09.860822 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:09.904690 (+ 43868us) log.cc:844] Serialized 26 byte log entry
0709 13:18:10.087397 (+182707us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:10.167417 (+ 80020us) log.cc:844] Serialized 14 byte log entry
0709 13:18:10.180320 (+ 12903us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:10.180423 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:11.513489 (+1333066us) spinlock_profiling.cc:243] Waited 84.8 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:18:11.637116 (+123627us) raft_consensus.cc:1685] finished
0709 13:18:11.637212 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:11.637328 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45493,"prepare.queue_time_us":504318,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":32,"raft.queue_time_us":200025,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":32024,"spinlock_wait_cycles":194979200,"thread_start_us":226597,"threads_started":2,"wal-append.queue_time_us":459536}
I20260709 13:18:11.639436 15858 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:11.639986 15909 consensus_queue.cc:1048] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.126s
I20260709 13:18:11.639887 15713 consensus_queue.cc:260] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:11.641364 15713 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:18:11.642518 10941 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36714 (request call id 840) took 2463 ms (client timeout 1707 ms). Trace:
W20260709 13:18:11.642954 10941 rpcz_store.cc:269] 0709 13:18:09.179426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.179710 (+   284us) service_pool.cc:224] Handling call
0709 13:18:11.642291 (+2462581us) spinlock_profiling.cc:243] Waited 1.96 s on lock 0x7b5400015ed4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:11.642436 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4498085760}
I20260709 13:18:11.644039 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:11.639981 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:11.645448 10797 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
I20260709 13:18:11.646907 10797 raft_consensus.cc:1240] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:18:11.648101 10950 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 16.
I20260709 13:18:11.621416 15816 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:11.649996 10870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2108) took 1478 ms. Trace:
I20260709 13:18:11.650053 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:11.650403 10841 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:11.605464 15859 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:11.654179 10876 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 16.
I20260709 13:18:11.654757 10941 raft_consensus.cc:1275] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:18:11.656474 10870 rpcz_store.cc:276] 0709 13:18:10.171245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:10.172006 (+   761us) service_pool.cc:224] Handling call
0709 13:18:10.172344 (+   338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:10.377970 (+205626us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:10.378026 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:10.396994 (+ 18968us) log.cc:844] Serialized 26 byte log entry
0709 13:18:11.016544 (+619550us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:11.016587 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:11.016658 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:11.550583 (+533925us) spinlock_profiling.cc:243] Waited 6.13 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:18:11.643941 (+ 93358us) raft_consensus.cc:1685] finished
0709 13:18:11.643966 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:11.649935 (+  5969us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18896,"raft.queue_time_us":263186,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":7165,"spinlock_wait_cycles":14105984,"thread_start_us":625566,"threads_started":2,"wal-append.queue_time_us":585001}
I20260709 13:18:11.624178 15927 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:11.659237 10870 raft_consensus.cc:1275] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:18:11.663267 10797 raft_consensus.cc:1240] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
W20260709 13:18:11.645962 15853 consensus_meta.cc:287] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 1.207s	user 0.010s	sys 0.002s
I20260709 13:18:11.671615 15754 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:11.672555 15859 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:11.601004 15919 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:11.639896 15858 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:11.683769 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:11.685209 10800 raft_consensus.cc:1275] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:18:11.678356 10797 raft_consensus.cc:1275] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:18:11.648425 15788 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:11.660399 15816 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:11.693863 11506 raft_consensus.cc:1275] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:18:11.695116 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:11.697467 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:11.698873 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:11.666222 15927 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:11.701320 10869 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 15.
I20260709 13:18:11.704214 11505 raft_consensus.cc:1275] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:18:11.706588 11506 raft_consensus.cc:1275] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:18:11.690277 15788 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:11.692922 15647 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.683418 15834 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:11.670465 15883 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:11.734403 15402 consensus_queue.cc:1048] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:18:11.721992 15924 consensus_queue.cc:1048] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.014s
I20260709 13:18:11.735942 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:11.681825 15853 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:11.741384 10868 raft_consensus.cc:1275] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:18:11.699012 15859 consensus_queue.cc:1048] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:18:11.739064 15788 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:11.752395 10868 raft_consensus.cc:1240] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:11.755942 10950 raft_consensus.cc:1240] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:11.761528 10800 raft_consensus.cc:1240] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 16. Current term is 17. Ops: [14.8-16.9]
I20260709 13:18:11.765419 15805 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:11.769497 15905 log.cc:927] Time spent T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.444s	user 0.001s	sys 0.000s
I20260709 13:18:11.756150 15865 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:11.779111 15730 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:11.783411 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:11.784349 10796 raft_consensus.cc:2393] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 14: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:11.785491 10875 raft_consensus.cc:1275] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:18:11.786157 15919 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:11.787499 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:11.785547 10796 raft_consensus.cc:1240] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:11.790534 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:11.742332 15647 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:11.793852 10872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 2040) took 2112 ms. Trace:
I20260709 13:18:11.794245 10872 rpcz_store.cc:276] 0709 13:18:09.678492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.689075 (+ 10583us) service_pool.cc:224] Handling call
0709 13:18:09.710252 (+ 21177us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:11.333352 (+1623100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:11.790346 (+456994us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b540006ccd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:11.790503 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3718888192}
I20260709 13:18:11.795673 15883 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:11.796648 10799 raft_consensus.cc:3055] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:11.797004 10799 raft_consensus.cc:740] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:11.800370 15899 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:11.801944 15856 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:11.802088 15845 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:11.828778 10942 raft_consensus.cc:1275] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:18:11.798827 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:11.797767 10799 consensus_queue.cc:260] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:11.831250 10949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 2171) took 1036 ms. Trace:
I20260709 13:18:11.833076 10799 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:11.836804 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:11.831676 10949 rpcz_store.cc:276] 0709 13:18:10.794723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:10.794891 (+   168us) service_pool.cc:224] Handling call
0709 13:18:10.814887 (+ 19996us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:10.815306 (+   419us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:10.815339 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:10.836661 (+ 21322us) log.cc:844] Serialized 26 byte log entry
0709 13:18:11.037221 (+200560us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:11.037272 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:11.037342 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:11.768879 (+731537us) spinlock_profiling.cc:243] Waited 9.39 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:18:11.827474 (+ 58595us) raft_consensus.cc:1685] finished
0709 13:18:11.827506 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:11.831182 (+  3676us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":111,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":13263,"spinlock_wait_cycles":21590400,"thread_start_us":144444,"threads_started":1,"wal-append.queue_time_us":379804}
I20260709 13:18:11.838981 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:11.839900 10793 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 12.
I20260709 13:18:11.851759 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:18:11.872722 15937 log.cc:927] Time spent T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 13:18:11.873536 15647 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:11.874423 15857 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:11.850587 15757 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:11.845567 15878 consensus_meta.cc:287] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.833s	user 0.013s	sys 0.000s
I20260709 13:18:11.877382 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:11.878639 10801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2034) took 2151 ms. Trace:
I20260709 13:18:11.880689 10801 rpcz_store.cc:276] 0709 13:18:09.727319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.727429 (+   110us) service_pool.cc:224] Handling call
0709 13:18:09.769799 (+ 42370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:11.004661 (+1234862us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:11.878406 (+873745us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b54000bc7d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:11.878585 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2835728256}
I20260709 13:18:11.880126 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:11.845580 15726 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:11.884991 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:11.886404 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:11.879683 15823 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:11.884713 15739 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.887480 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:11.910810 10799 raft_consensus.cc:1275] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:18:11.914016 10799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2030) took 2189 ms. Trace:
I20260709 13:18:11.914674 10799 rpcz_store.cc:276] 0709 13:18:09.724637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.724772 (+   135us) service_pool.cc:224] Handling call
0709 13:18:09.725180 (+   408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:11.796553 (+2071373us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b5400304ba0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80e2cdc0 00007f9c80eb71c0 00007f9c80eb8231 00007f9c80eb4d8f 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c
0709 13:18:11.911353 (+114800us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:11.913946 (+  2593us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2872091904}
I20260709 13:18:11.918090 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:11.883823 15878 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:11.924341 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:11.927651 10948 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:11.939857 10795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39442 (request call id 678) took 1053 ms. Trace:
I20260709 13:18:11.940442 10795 rpcz_store.cc:276] 0709 13:18:10.886619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:10.895856 (+  9237us) service_pool.cc:224] Handling call
0709 13:18:11.939606 (+1043750us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b54002120d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:11.939773 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":272918912}
W20260709 13:18:11.919298 15808 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 13:18:11.913513 15739 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:11.953651 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:11.954169 15799 raft_consensus.cc:3055] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:11.954618 15799 raft_consensus.cc:740] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:11.955564 15799 consensus_queue.cc:260] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:11.956498 15927 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:11.958958 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:18:11.927896 15941 log.cc:927] Time spent T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 13:18:11.960289 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:11.961655 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:11.963398 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:11.969117 10948 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 15.
I20260709 13:18:11.969997 11510 raft_consensus.cc:1240] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:11.964473 10872 raft_consensus.cc:1240] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:11.970398 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:11.971776 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:11.974920 15799 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:11.982985 10946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 2166) took 2473 ms. Trace:
I20260709 13:18:11.985476 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:11.987999 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:18:11.977874 15939 log.cc:927] Time spent T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:18:11.994890 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:11.983444 10946 rpcz_store.cc:276] 0709 13:18:09.504912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.563837 (+ 58925us) service_pool.cc:224] Handling call
0709 13:18:09.564280 (+   443us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:09.681986 (+117706us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:09.682025 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:11.789606 (+2107581us) spinlock_profiling.cc:243] Waited 3.5 ms on lock 0x7b2c0015d218. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7f7aea11 00007f9c7f7be372 00007f9c80ef0a01 00007f9c80eb543a 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2
0709 13:18:11.792222 (+  2616us) log.cc:844] Serialized 26 byte log entry
0709 13:18:11.905562 (+113340us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:11.905618 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:11.905722 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:11.962852 (+ 57130us) raft_consensus.cc:1685] finished
0709 13:18:11.962873 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:11.978521 (+ 15648us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":107020,"raft.queue_time_us":141015,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":45300,"spinlock_wait_cycles":8048000,"thread_start_us":112904,"threads_started":1,"wal-append.queue_time_us":150570}
I20260709 13:18:11.996907 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:11.953758 15853 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:11.994282 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:12.000319 15878 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:12.000743 15840 log.cc:927] Time spent T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 2.295s	user 0.003s	sys 0.002s
I20260709 13:18:12.001358 15739 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:12.014408 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:12.019373 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:12.020366 10946 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:12.020398 10941 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 15: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:12.029515 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:12.031737 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:12.014387 15927 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:12.022174 15899 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:12.049788 10946 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 16.
I20260709 13:18:12.051692 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:12.051944 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:12.063868 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:12.064512 10946 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 15: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:12.067802 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:12.072750 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:12.073364 10946 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:12.074497 10950 raft_consensus.cc:1275] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 13:18:12.075250 10950 pending_rounds.cc:85] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 10
I20260709 13:18:12.075528 10950 pending_rounds.cc:107] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:18:12.075759 10950 raft_consensus.cc:2889] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:18:12.086328 10943 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41020 (request call id 2509) took 3197 ms (client timeout 3313 ms). Trace:
W20260709 13:18:12.086958 10943 rpcz_store.cc:269] 0709 13:18:08.888831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:08.903625 (+ 14794us) service_pool.cc:224] Handling call
0709 13:18:12.073678 (+3170053us) spinlock_profiling.cc:243] Waited 2.18 s on lock 0x7b540018ddd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:12.086234 (+ 12556us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5009140096}
I20260709 13:18:12.088541 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:12.115880 15816 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:12.111675 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:12.120201 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:12.121784 10948 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
I20260709 13:18:12.122138 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:12.203291 10946 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 17.
W20260709 13:18:12.204794 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:12.206092 10946 raft_consensus.cc:1240] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:12.206548 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:18:12.192099 15713 consensus_meta.cc:287] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.550s	user 0.014s	sys 0.000s
I20260709 13:18:12.216750 10767 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:12.220014 10767 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:12.221637 10767 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:12.227280 10767 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:12.242087 10767 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:12.249166 10767 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:12.250451 10767 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:12.237192 15899 raft_consensus.cc:2804] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:12.236469 15870 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:12.236994 15853 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:12.239303 15816 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:12.278929 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:12.219233 15701 consensus_queue.cc:1059] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 13:18:12.251897 15805 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:12.238390 15713 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:12.277225 15801 raft_consensus.cc:2804] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:12.295665 10946 raft_consensus.cc:1240] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 17. Current term is 18. Ops: []
W20260709 13:18:12.303012 15949 log.cc:927] Time spent T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 13:18:12.304459 10797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2567) took 1037 ms. Trace:
I20260709 13:18:12.304883 10797 rpcz_store.cc:276] 0709 13:18:11.267007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:11.692564 (+425557us) service_pool.cc:224] Handling call
0709 13:18:11.692964 (+   400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:11.693321 (+   357us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:11.693358 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:11.693881 (+   523us) log.cc:844] Serialized 26 byte log entry
0709 13:18:11.991655 (+297774us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:11.991728 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:11.991821 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:12.304204 (+312383us) raft_consensus.cc:1685] finished
0709 13:18:12.304227 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:12.304391 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":257515,"threads_started":1,"wal-append.queue_time_us":351915}
I20260709 13:18:12.313225 15889 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:12.317682 15683 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:12.326165 11510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56410 (request call id 797) took 1068 ms. Trace:
I20260709 13:18:12.326819 11510 rpcz_store.cc:276] 0709 13:18:11.258024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:11.258191 (+   167us) service_pool.cc:224] Handling call
0709 13:18:11.294953 (+ 36762us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:11.979485 (+684532us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:12.325824 (+346339us) spinlock_profiling.cc:243] Waited 675 ms on lock 0x7b540017c5d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:12.326080 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1552054528}
W20260709 13:18:12.328074 15754 consensus_meta.cc:287] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.645s	user 0.006s	sys 0.006s
I20260709 13:18:12.286131 15853 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:12.255475 15852 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:12.357534 15802 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:12.334640 15788 consensus_meta.cc:287] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.563s	user 0.006s	sys 0.006s
I20260709 13:18:12.367202 15788 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:12.369267 15788 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:12.328572 15754 raft_consensus.cc:2749] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:12.289176 15805 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:12.294008 15713 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:12.291509 15899 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
W20260709 13:18:12.380584 15858 consensus_meta.cc:287] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.684s	user 0.010s	sys 0.004s
I20260709 13:18:12.351006 15853 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:12.397081 15899 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:12.384048 15805 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:12.432740 15645 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:18:12.434126 15950 log.cc:927] Time spent T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
I20260709 13:18:12.429432 15858 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:12.436439 15948 log.cc:927] Time spent T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.227s	user 0.001s	sys 0.000s
I20260709 13:18:12.396176 15713 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:12.438632 15858 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:12.430192 15856 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:12.467545 15856 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:12.470268 15856 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:12.472173 15887 raft_consensus.cc:3055] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:12.472630 15887 raft_consensus.cc:740] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:12.473507 15887 consensus_queue.cc:260] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:12.381098 15852 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:12.476936 10950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 856) took 1084 ms. Trace:
I20260709 13:18:12.477526 10950 rpcz_store.cc:276] 0709 13:18:11.392901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:11.756609 (+363708us) service_pool.cc:224] Handling call
0709 13:18:12.074127 (+317518us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:12.182349 (+108222us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:12.476634 (+294285us) spinlock_profiling.cc:243] Waited 313 ms on lock 0x7b540018ddd0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:12.476849 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":720687488}
W20260709 13:18:12.483173 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 232822us
I20260709 13:18:12.474519 15887 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:12.474628 15852 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:12.490626 15645 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:12.491485 15645 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:12.493140 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:12.493809 15801 raft_consensus.cc:697] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:12.497643 10841 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:12.498364 15857 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:12.498883 15757 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:12.521567 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:12.501899 15801 consensus_queue.cc:237] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:12.505395 15845 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:12.506021 15883 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:12.506484 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:12.511861 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:12.524971 10766 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:12.516577 10941 raft_consensus.cc:1240] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:12.521783 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:12.528460 10794 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 13.
I20260709 13:18:12.500514 15853 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:12.528879 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:12.524741 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:18:12.515321 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:12.531003 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:12.525388 10874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 760) took 1641 ms. Trace:
I20260709 13:18:12.518312 11509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56404 (request call id 869) took 1703 ms. Trace:
I20260709 13:18:12.533170 11509 rpcz_store.cc:276] 0709 13:18:10.814855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:10.831133 (+ 16278us) service_pool.cc:224] Handling call
0709 13:18:10.857580 (+ 26447us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:11.028771 (+171191us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:18:11.296756 (+267985us) log.cc:844] Serialized 14 byte log entry
0709 13:18:11.625374 (+328618us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:11.625444 (+    70us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:11.625511 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:12.518009 (+892498us) spinlock_profiling.cc:243] Waited 171 ms on lock 0x7b54004290d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:12.518075 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:12.518239 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":109,"prepare.queue_time_us":300747,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":745,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":14435,"spinlock_wait_cycles":405260800,"thread_start_us":544610,"threads_started":2,"wal-append.queue_time_us":351846}
I20260709 13:18:12.536767 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:12.537554 10799 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:12.531757 10874 rpcz_store.cc:276] 0709 13:18:10.883907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:11.599572 (+715665us) service_pool.cc:224] Handling call
0709 13:18:11.601380 (+  1808us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:11.601751 (+   371us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:18:11.631398 (+ 29647us) log.cc:844] Serialized 14 byte log entry
0709 13:18:11.738728 (+107330us) log.cc:844] Serialized 14 byte log entry
0709 13:18:11.745005 (+  6277us) log.cc:844] Serialized 14 byte log entry
0709 13:18:11.745258 (+   253us) log.cc:844] Serialized 14 byte log entry
0709 13:18:12.330155 (+584897us) spinlock_profiling.cc:243] Waited 2.57 ms on lock 0x7b180045aae0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc83f3 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:18:12.358957 (+ 28802us) spinlock_profiling.cc:243] Waited 3.97 ms on lock 0x7b180045aae0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc8522 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:18:12.487518 (+128561us) spinlock_profiling.cc:243] Waited 4.01 ms on lock 0x7b180045aae0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7dd42c0a 00007f9c87482219 00007f9c7ddcaa40 00007f9c7ddc54e4 00007f9c7ddc5455 00007f9c80ead060 00007f9c80ebf716 00007f9c80ed0287 00007f9c80ed0208
0709 13:18:12.487672 (+   154us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:12.488187 (+   515us) log.cc:844] Serialized 26 byte log entry
0709 13:18:12.488350 (+   163us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:12.488384 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:12.488452 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:12.525167 (+ 36715us) raft_consensus.cc:1685] finished
0709 13:18:12.525191 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:12.525330 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":901371,"prepare.queue_time_us":599479,"prepare.run_cpu_time_us":138,"prepare.run_wall_time_us":143,"raft.queue_time_us":754511,"raft.run_cpu_time_us":473,"raft.run_wall_time_us":15124,"spinlock_wait_cycles":24263040,"thread_start_us":71002,"threads_started":1,"wal-append.queue_time_us":163075}
W20260709 13:18:12.545694 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 48148.5us
I20260709 13:18:12.546532 10841 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:12.549080 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:12.550964 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:12.554132 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:12.555542 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:12.556939 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:12.558521 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:12.558640 10766 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:12.571856 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:12.572098 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:12.572556 10876 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 14.
I20260709 13:18:12.572659 10801 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 12.
I20260709 13:18:12.573444 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:12.524230 15918 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:12.578583 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:12.589170 10871 raft_consensus.cc:2393] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 17: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:12.584934 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:12.591293 10874 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:12.587622 10799 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
I20260709 13:18:12.594656 15645 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:12.555764 15928 raft_consensus.cc:2804] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:12.595481 15928 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:12.595871 15928 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:12.595885 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:12.596895 10801 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 15: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:12.598237 15726 consensus_queue.cc:1059] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:12.598896 15713 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:12.600219 15750 raft_consensus.cc:3055] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:18:12.600606 15750 raft_consensus.cc:740] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:12.601150 15713 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:12.601438 15750 consensus_queue.cc:260] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:12.602720 15750 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:12.607627 15826 raft_consensus.cc:2804] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:12.608287 15826 raft_consensus.cc:697] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:12.609028 10872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 770) took 1092 ms. Trace:
I20260709 13:18:12.609406 10872 rpcz_store.cc:276] 0709 13:18:11.516426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:11.795212 (+278786us) service_pool.cc:224] Handling call
0709 13:18:11.795506 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:11.978406 (+182900us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:12.608745 (+630339us) spinlock_profiling.cc:243] Waited 169 ms on lock 0x7b5400078854. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:12.608964 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":388128000}
I20260709 13:18:12.527200 15757 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:12.610924 15826 consensus_queue.cc:237] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:12.611727 10842 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:12.612687 10800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2564) took 1346 ms. Trace:
I20260709 13:18:12.554610 15865 consensus_queue.cc:1048] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 13:18:12.613085 10800 rpcz_store.cc:276] 0709 13:18:11.265892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:11.686296 (+420404us) service_pool.cc:224] Handling call
0709 13:18:11.700217 (+ 13921us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:11.762023 (+ 61806us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:12.612432 (+850409us) spinlock_profiling.cc:243] Waited 61 ms on lock 0x7b5400385c54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:12.612629 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":140366592}
I20260709 13:18:12.558305 15798 raft_consensus.cc:2749] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:12.549027 15883 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:12.614552 15645 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:12.615509 15645 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:12.616403 15883 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:12.617744 15645 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:12.619302 15883 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:12.619861 15883 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:12.622071 15883 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:12.588701 15870 raft_consensus.cc:697] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:12.613504 10842 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:12.624673 10876 raft_consensus.cc:2410] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 4.
I20260709 13:18:12.610574 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:12.632164 10841 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:12.634385 10872 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 15: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:12.647594 10842 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:12.648919 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:12.540683 15845 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:12.648788 15798 consensus_queue.cc:1048] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 13:18:12.650961 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:12.651490 10872 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:12.642207 15717 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:12.652122 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:12.652396 15845 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:12.652968 10950 raft_consensus.cc:2393] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 18: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
W20260709 13:18:12.633744 15946 log.cc:927] Time spent T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:18:12.641903 10841 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:12.612721 15757 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:12.655766 10842 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:12.657049 10949 raft_consensus.cc:1275] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 13:18:12.657951 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:12.657011 15717 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:12.676749 15854 raft_consensus.cc:2749] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:12.677383 15717 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:12.677809 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:12.679150 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:12.679948 10871 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:12.679014 10841 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:12.659303 15645 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:12.683458 15798 raft_consensus.cc:2749] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:18:12.683305 15645 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:12.685472 10872 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 18.
I20260709 13:18:12.686314 15645 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:12.686743 15875 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:12.653592 15889 consensus_queue.cc:1048] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:18:12.540881 15857 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:12.689337 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 2120) took 1428 ms. Trace:
I20260709 13:18:12.689765 10867 rpcz_store.cc:276] 0709 13:18:11.260551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:11.530097 (+269546us) service_pool.cc:224] Handling call
0709 13:18:12.689136 (+1159039us) spinlock_profiling.cc:243] Waited 40.2 ms on lock 0x7b5400316ad4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:12.689276 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":92535040}
I20260709 13:18:12.691133 10873 raft_consensus.cc:1240] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:18:12.691558 15857 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:12.692361 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 2047) took 1506 ms. Trace:
I20260709 13:18:12.692896 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:12.692850 10873 rpcz_store.cc:276] 0709 13:18:11.186320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:11.637137 (+450817us) service_pool.cc:224] Handling call
0709 13:18:12.690902 (+1053765us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:12.691569 (+   667us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:12.692176 (+   607us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400316ad0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:12.692301 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2422610304}
I20260709 13:18:12.694149 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:12.694669 15845 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:12.695163 15845 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:12.695551 15845 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:12.694963 10873 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
I20260709 13:18:12.697335 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:12.698817 10867 raft_consensus.cc:1275] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:18:12.692256 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:12.700438 10868 raft_consensus.cc:1240] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:12.641119 15883 raft_consensus.cc:2749] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:12.702757 10868 raft_consensus.cc:1240] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:12.689520 10841 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:12.672801 15955 log.cc:927] Time spent T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:18:12.712913 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:12.713586 10801 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 16.
I20260709 13:18:12.704039 10842 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:12.718427 10800 raft_consensus.cc:1240] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:12.721055 10841 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:12.731154 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:12.739653 10842 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:12.741160 15802 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:12.741861 15802 raft_consensus.cc:697] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:12.743124 15802 consensus_queue.cc:237] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:12.745360 15754 raft_consensus.cc:2749] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:18:12.749944 15754 consensus_queue.cc:1059] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:12.751057 15896 raft_consensus.cc:3055] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:12.751570 15896 raft_consensus.cc:740] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:12.752647 15896 consensus_queue.cc:260] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:12.753718 15896 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:12.756392 10797 raft_consensus.cc:1275] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 7. (term mismatch)
I20260709 13:18:12.756899 10797 pending_rounds.cc:85] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 6
I20260709 13:18:12.757156 10797 pending_rounds.cc:107] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 14.7
I20260709 13:18:12.757428 10797 raft_consensus.cc:2889] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:12.710273 15854 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:12.633996 15918 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:12.712493 15893 raft_consensus.cc:2804] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:12.716248 15857 raft_consensus.cc:2749] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:12.709878 15883 consensus_queue.cc:1059] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:18:12.761206 15798 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:18:12.801787 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 86177.5us
I20260709 13:18:12.803709 10841 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:12.812629 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:12.816818 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:12.819854 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:12.844087 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:12.846230 10801 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
I20260709 13:18:12.849359 10767 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:12.877588 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:12.878539 10800 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 12.
I20260709 13:18:12.709929 15870 consensus_queue.cc:237] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:12.764851 15805 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:12.803200 15798 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:12.889372 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 51532.1us
I20260709 13:18:12.864199 10842 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:12.893414 10871 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
I20260709 13:18:12.894837 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:12.895642 10874 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
I20260709 13:18:12.898139 10946 raft_consensus.cc:1240] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 16. Current term is 17. Ops: [14.8-16.9]
I20260709 13:18:12.900262 10872 raft_consensus.cc:1240] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:12.834962 15788 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:12.898748 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:12.852862 15818 raft_consensus.cc:2749] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:12.901705 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:12.905328 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:12.905524 10874 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:18:12.911163 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:12.919696 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:12.874269 15855 consensus_queue.cc:1059] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 13:18:12.898341 15798 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:12.905365 15852 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:12.933001 15852 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:12.936455 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:12.938575 10870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2119) took 1678 ms. Trace:
I20260709 13:18:12.939054 10870 rpcz_store.cc:276] 0709 13:18:11.260316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:11.660451 (+400135us) service_pool.cc:224] Handling call
0709 13:18:12.003635 (+343184us) spinlock_profiling.cc:243] Waited 5.89 ms on lock 0x7b5800177098. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd90a22 00007f9c86ca9c09 00007f9c85e03ee0 00007f9c85ecd0ab 00007f9c85ecd131 00007f9c85e8efc0 00007f9c85e79929 00007f9c85e793d2 00007f9c7f3f0bd4
0709 13:18:12.003870 (+   235us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:12.509178 (+505308us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:18:12.509541 (+   363us) log.cc:844] Serialized 14 byte log entry
0709 13:18:12.712451 (+202910us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:12.719219 (+  6768us) log.cc:844] Serialized 26 byte log entry
0709 13:18:12.719420 (+   201us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:12.719468 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:12.719558 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:12.938241 (+218683us) raft_consensus.cc:1685] finished
0709 13:18:12.938355 (+   114us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:12.938511 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":510197,"prepare.queue_time_us":10925,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":505086,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":5867,"spinlock_wait_cycles":13543168,"thread_start_us":202257,"threads_started":1,"wal-append.queue_time_us":371279}
W20260709 13:18:12.945125 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 95882.7us
I20260709 13:18:12.947037 10767 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:12.882417 15857 consensus_queue.cc:1048] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:18:12.902139 15927 raft_consensus.cc:2749] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:12.954866 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:18:13.043433 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 96615.1us
I20260709 13:18:13.044422 10767 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:12.906143 15853 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:13.046607 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:13.047251 10871 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:12.950798 15730 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:18:13.052110 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 100569us
I20260709 13:18:12.915823 15788 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:13.041363 15852 raft_consensus.cc:2749] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:12.922732 15928 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:13.056437 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 406941us
I20260709 13:18:13.065812 11478 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:13.067065 11478 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:13.058372 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:13.062273 10767 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:13.049214 10916 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.005s	sys 0.026s
W20260709 13:18:13.069958 10916 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.006s	sys 0.026s
I20260709 13:18:13.071420 10916 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:13.076756 10767 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:13.077090 11478 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:13.049858 15818 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:13.082990 10873 raft_consensus.cc:1240] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:18:13.053836 15967 raft_consensus.cc:2749] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:18:13.066478 15875 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:13.080648 10767 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:13.079111 11478 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:13.101413 10767 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:13.074962 15847 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:13.073134 10874 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
I20260709 13:18:13.081226 15877 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:13.084167 15684 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:13.102813 11478 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:13.081365 15962 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:13.080447 10871 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 17.
I20260709 13:18:13.110345 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:13.083756 15887 raft_consensus.cc:2749] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:13.120389 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:13.122485 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:13.108014 15684 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:13.124166 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:13.104768 15847 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:13.103708 15875 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:18:13.121208 11504 raft_consensus.cc:1275] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:18:13.131289 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:13.106420 15877 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:13.122206 10767 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:13.138595 11505 raft_consensus.cc:1275] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:18:13.142473 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:13.105929 15892 raft_consensus.cc:2804] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:13.087198 15818 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:13.112639 15889 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:13.109263 15852 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:13.143405 11507 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 15: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:13.139243 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:13.113529 15918 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:13.151013 10767 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:13.123214 15684 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:13.181836 15889 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:13.207168 15788 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:13.209224 15893 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:13.209965 15893 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:13.142453 15857 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:13.211517 15857 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:13.214027 15857 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:13.215142 15966 log.cc:927] Time spent T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.270s	user 0.001s	sys 0.000s
I20260709 13:18:13.215654 15928 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:13.145705 15892 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:13.216434 15892 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:18:13.219084 15970 log.cc:927] Time spent T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.291s	user 0.000s	sys 0.001s
I20260709 13:18:13.140614 15847 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:13.220480 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 772) took 1703 ms. Trace:
I20260709 13:18:13.220847 10869 rpcz_store.cc:276] 0709 13:18:11.517067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:11.993823 (+476756us) service_pool.cc:224] Handling call
0709 13:18:11.999458 (+  5635us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:12.506602 (+507144us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:12.506638 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:12.507148 (+   510us) log.cc:844] Serialized 26 byte log entry
0709 13:18:12.710009 (+202861us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:12.710051 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:12.712312 (+  2261us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:13.220229 (+507917us) raft_consensus.cc:1685] finished
0709 13:18:13.220253 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:13.220420 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":519146,"raft.queue_time_us":508650,"raft.run_cpu_time_us":60,"raft.run_wall_time_us":60,"thread_start_us":202481,"threads_started":1,"wal-append.queue_time_us":366765}
I20260709 13:18:13.112625 15686 consensus_queue.cc:1059] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 13:18:13.182967 11510 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 14.
I20260709 13:18:13.207362 15889 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:18:13.265748 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 114843us
I20260709 13:18:13.145501 15877 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:18:13.193657 15981 log.cc:927] Time spent T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:18:13.139952 15805 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:13.180821 15899 consensus_meta.cc:287] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.679s	user 0.008s	sys 0.003s
I20260709 13:18:13.266703 10767 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:18:13.200677 15969 log.cc:927] Time spent T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:18:13.187491 15963 consensus_queue.cc:1048] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.086s
I20260709 13:18:13.305955 10767 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:13.147109 15744 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:13.191636 15975 log.cc:927] Time spent T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:18:13.192509 15818 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:13.284160 15875 raft_consensus.cc:2804] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:13.291093 15899 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:13.303499 15845 consensus_meta.cc:287] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.607s	user 0.007s	sys 0.004s
I20260709 13:18:13.304427 10875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 771) took 1787 ms. Trace:
W20260709 13:18:13.199790 15973 log.cc:927] Time spent T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:18:13.200069 15852 raft_consensus.cc:697] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:13.179611 15962 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:13.265415 15928 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:13.320912 15726 raft_consensus.cc:2749] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:13.283684 15683 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:13.332487 10875 rpcz_store.cc:276] 0709 13:18:11.516699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:11.963202 (+446503us) service_pool.cc:224] Handling call
0709 13:18:11.963513 (+   311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:12.503026 (+539513us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:12.503068 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:12.503659 (+   591us) log.cc:844] Serialized 26 byte log entry
0709 13:18:12.707103 (+203444us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:12.707145 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:12.707227 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:13.304156 (+596929us) raft_consensus.cc:1685] finished
0709 13:18:13.304179 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:13.304349 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":539082,"raft.queue_time_us":544580,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":86,"thread_start_us":203039,"threads_started":1,"wal-append.queue_time_us":555914}
I20260709 13:18:13.316972 15805 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:13.349972 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:13.350750 10875 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:18:13.354948 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:13.335592 15875 raft_consensus.cc:697] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:13.337148 15962 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:13.339998 15853 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
W20260709 13:18:13.375821 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:13.392144 15899 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:13.400512 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:18:13.415421 15983 log.cc:927] Time spent T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:18:13.347380 15928 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:13.417986 15928 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:13.423033 15852 consensus_queue.cc:237] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:13.399435 15962 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:18:13.371768 10868 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.014s	sys 0.011s
W20260709 13:18:13.428422 10868 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.015s	sys 0.011s
I20260709 13:18:13.430114 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:13.337666 15845 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:13.441630 10942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2523) took 1766 ms. Trace:
I20260709 13:18:13.366707 15805 raft_consensus.cc:697] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:13.394768 15818 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:13.449647 10942 rpcz_store.cc:276] 0709 13:18:11.674710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:11.764773 (+ 90063us) service_pool.cc:224] Handling call
0709 13:18:11.765100 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:11.829325 (+ 64225us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:13.441296 (+1611971us) spinlock_profiling.cc:243] Waited 63.2 ms on lock 0x7b5400174654. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:13.441542 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":145257728}
I20260709 13:18:13.349450 15683 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:13.441987 15684 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:13.453775 15845 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:13.462716 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:13.442612 15896 consensus_meta.cc:287] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.688s	user 0.013s	sys 0.000s
I20260709 13:18:13.457971 15686 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:13.459007 11507 raft_consensus.cc:2393] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 14: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:13.458457 15805 consensus_queue.cc:237] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:13.460683 15684 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:13.432272 15853 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:13.438597 10875 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
I20260709 13:18:13.479319 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:13.480845 10875 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:13.455300 15847 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:13.455636 15818 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:13.474450 15861 raft_consensus.cc:3055] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:13.471475 15686 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:13.485638 15683 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:13.486498 15980 log.cc:927] Time spent T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.136s	user 0.002s	sys 0.000s
I20260709 13:18:13.496106 15892 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:13.499483 10875 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 13.
I20260709 13:18:13.506136 11509 raft_consensus.cc:2393] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 15: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:13.508836 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:13.509488 10875 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:13.472586 15875 consensus_queue.cc:237] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:13.500219 15861 raft_consensus.cc:740] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:13.522114 11477 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:13.525287 11477 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:13.527129 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:13.509454 15798 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:13.531056 15892 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:13.531512 15686 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:13.532027 15919 raft_consensus.cc:3055] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:13.525787 15888 raft_consensus.cc:2749] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:13.557085 15962 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:13.540735 15986 consensus_queue.cc:1059] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:18:13.531924 15684 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:13.543954 15818 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:13.544544 15883 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:13.545958 15861 consensus_queue.cc:260] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:13.557132 15798 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:13.547601 15877 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:13.549701 15847 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:13.550987 15853 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:13.557583 15919 raft_consensus.cc:740] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:13.532774 11508 raft_consensus.cc:1240] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:13.558812 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:13.559764 10875 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 14.
I20260709 13:18:13.570317 11506 raft_consensus.cc:1240] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:13.574801 11478 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:13.575031 11477 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:13.577673 15847 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:13.578058 15847 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:13.578413 15847 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:13.578826 15717 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:13.579006 11477 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:13.579566 15717 raft_consensus.cc:697] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 15 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:13.579680 15976 consensus_queue.cc:1059] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:18:13.580899 15906 raft_consensus.cc:2804] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:13.580585 15717 consensus_queue.cc:237] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.9, Last appended by leader: 9, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:13.581403 15906 raft_consensus.cc:697] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 16 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:13.582226 15906 consensus_queue.cc:237] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:13.576321 11503 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 12.
I20260709 13:18:13.568138 15683 consensus_queue.cc:1059] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 13:18:13.573021 15684 raft_consensus.cc:697] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 16 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:13.594348 15861 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:13.612692 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:13.614024 10867 raft_consensus.cc:2393] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 13: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:18:13.615813 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:13.618997 15877 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:13.620697 15684 consensus_queue.cc:237] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:13.597795 15889 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:13.622681 11510 raft_consensus.cc:1240] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:18:13.623711 15889 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:13.624374 15730 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:13.624852 15730 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:13.628166 15887 raft_consensus.cc:3055] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:13.628680 15887 raft_consensus.cc:740] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:13.628351 15847 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:13.629621 15887 consensus_queue.cc:260] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:13.587885 15911 consensus_queue.cc:1048] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 13:18:13.633625 15962 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:13.635488 15816 consensus_queue.cc:1059] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 13:18:13.634676 10767 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:13.638633 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:13.652689 10872 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:13.653036 15962 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:13.653765 15962 raft_consensus.cc:697] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:13.652695 10767 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:13.639228 11512 raft_consensus.cc:1240] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:13.630705 15887 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:13.630627 15847 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:13.659283 15847 raft_consensus.cc:3055] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:13.659687 15847 raft_consensus.cc:740] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:13.660483 15847 consensus_queue.cc:260] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:13.639415 15883 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:13.663350 15883 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:13.663795 15959 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:13.664608 15959 raft_consensus.cc:697] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:13.665771 15959 consensus_queue.cc:237] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:13.631779 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2136) took 1101 ms. Trace:
I20260709 13:18:13.671114 11504 raft_consensus.cc:1240] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:18:13.672025 10767 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:13.673246 15964 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:13.673923 15964 raft_consensus.cc:697] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:13.678833 10794 raft_consensus.cc:1240] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:13.675019 15964 consensus_queue.cc:237] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:13.680130 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:13.681422 15805 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:13.681982 15805 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:13.682878 15826 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:13.683317 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:13.683203 10842 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:13.685492 15972 log.cc:927] Time spent T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.582s	user 0.008s	sys 0.009s
I20260709 13:18:13.682931 15853 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:13.694301 15853 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:13.696911 15853 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:13.683597 15826 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:13.703118 15826 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:13.687183 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:13.706485 10767 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:13.708971 10948 raft_consensus.cc:1275] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 13:18:13.687373 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:13.687947 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:13.688608 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:13.688800 10941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2531) took 1178 ms. Trace:
I20260709 13:18:13.711761 10796 raft_consensus.cc:2393] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 13: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:13.711519 10941 rpcz_store.cc:276] 0709 13:18:12.510301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:12.532103 (+ 21802us) service_pool.cc:224] Handling call
0709 13:18:12.532497 (+   394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:12.533134 (+   637us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:12.533178 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:12.551373 (+ 18195us) log.cc:844] Serialized 26 byte log entry
0709 13:18:12.838481 (+287108us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:12.889341 (+ 50860us) log.cc:844] Serialized 14 byte log entry
0709 13:18:13.688052 (+798711us) spinlock_profiling.cc:243] Waited 2.05 ms on lock 0x7b180035e540. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7dd42c0a 00007f9c87482219 00007f9c7ddcaa40 00007f9c7ddc54e4 00007f9c7ddc5455 00007f9c80ead060 00007f9c80ebf716 00007f9c80ed0287 00007f9c80ed0208
0709 13:18:13.688278 (+   226us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:13.688348 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:13.688399 (+    51us) raft_consensus.cc:1685] finished
0709 13:18:13.688528 (+   129us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:13.688722 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9044,"prepare.queue_time_us":59212,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":798811,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":19305,"spinlock_wait_cycles":4723968,"thread_start_us":263853,"threads_started":1,"wal-append.queue_time_us":511641}
I20260709 13:18:13.691227 15805 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:13.691776 15822 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:18:13.693039 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 621636us
I20260709 13:18:13.721813 10916 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:13.693638 15968 raft_consensus.cc:2749] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:13.670826 10873 rpcz_store.cc:276] 0709 13:18:12.530774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:12.696711 (+165937us) service_pool.cc:224] Handling call
0709 13:18:12.697031 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:13.083436 (+386405us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:13.631502 (+548066us) spinlock_profiling.cc:243] Waited 386 ms on lock 0x7b5400097ed4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:13.631714 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":887021440}
I20260709 13:18:13.633056 15919 consensus_queue.cc:260] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:13.706611 10841 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:13.727454 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:13.727759 10841 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:13.728066 10941 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:13.694339 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:13.716572 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:13.720103 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:13.720819 10801 raft_consensus.cc:1275] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 13:18:13.720103 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:18:13.729590 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 46482.5us
I20260709 13:18:13.729960 10792 raft_consensus.cc:2393] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 13: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:13.686677 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:13.734359 15822 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:13.734841 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:13.720736 10767 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:13.736354 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:13.736750 15822 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:13.737383 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:13.737432 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:18:13.755122 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 174680us
I20260709 13:18:13.732426 10872 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
I20260709 13:18:13.707513 15754 raft_consensus.cc:2804] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:18:13.769686 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:13.763830 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:13.771858 10950 raft_consensus.cc:1240] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:18:13.773725 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:13.708775 15907 consensus_queue.cc:1048] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 13:18:13.768926 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:13.776387 10842 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:13.767405 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:13.767660 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:13.768399 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:13.781112 11504 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 11.
I20260709 13:18:13.768720 15858 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:13.783166 15858 raft_consensus.cc:697] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:13.768692 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:13.784443 15858 consensus_queue.cc:237] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:13.764336 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:13.791996 10942 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 14.
I20260709 13:18:13.632447 15893 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:13.794987 10950 raft_consensus.cc:3055] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:13.720074 15859 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:13.720144 15877 raft_consensus.cc:2804] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:13.794982 10943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 2184) took 1876 ms. Trace:
I20260709 13:18:13.801529 10842 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:13.822472 10943 rpcz_store.cc:276] 0709 13:18:11.918531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:12.183285 (+264754us) service_pool.cc:224] Handling call
0709 13:18:12.187283 (+  3998us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:12.332599 (+145316us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:12.332627 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:12.332664 (+    37us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:12.332699 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:13.794622 (+1461923us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b5400399c54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:13.794690 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:13.794907 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":106,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":85,"spinlock_wait_cycles":333435264}
I20260709 13:18:13.743674 15709 raft_consensus.cc:2749] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:13.775440 15896 consensus_queue.cc:1048] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 13:18:13.824942 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:13.763739 15818 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:13.799278 10841 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:13.833195 10842 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:13.833516 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:13.777169 15919 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:18:13.837853 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 239263us
I20260709 13:18:13.824693 10950 raft_consensus.cc:740] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:13.777408 15754 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:13.845292 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 121165us
I20260709 13:18:13.763888 15927 raft_consensus.cc:2749] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:13.794135 15639 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:13.866848 10916 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:13.841056 15877 raft_consensus.cc:697] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:13.848893 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:13.821926 15928 raft_consensus.cc:2749] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:13.850076 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:13.850250 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:13.850745 10941 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 12.
I20260709 13:18:13.918675 10946 raft_consensus.cc:1240] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 11. Current term is 12. Ops: []
W20260709 13:18:13.856240 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:13.856482 11478 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:13.721385 15962 consensus_queue.cc:237] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:13.769814 15831 consensus_queue.cc:1048] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.077s
I20260709 13:18:13.763814 15765 consensus_queue.cc:1048] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.072s
I20260709 13:18:13.883257 10916 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:13.842303 15968 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:13.832029 15859 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:13.925057 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:13.925096 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:13.945772 10944 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
I20260709 13:18:13.824374 15918 consensus_queue.cc:1059] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
W20260709 13:18:13.935014 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 101914us
I20260709 13:18:13.951934 10842 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:13.883297 15754 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:18:13.935693 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 79280.8us
W20260709 13:18:13.961462 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:13.856302 10950 consensus_queue.cc:260] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:13.986968 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:13.987289 10950 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:13.987011 11478 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:13.863440 15861 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:13.986194 10873 raft_consensus.cc:1275] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 8. (term mismatch)
I20260709 13:18:14.014426 10873 pending_rounds.cc:85] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 7
I20260709 13:18:14.014710 10873 pending_rounds.cc:107] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 13:18:13.971906 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:13.914719 15709 raft_consensus.cc:2804] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:14.017596 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:14.017935 10767 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:14.015484 10949 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
I20260709 13:18:14.014971 10873 raft_consensus.cc:2889] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:13.923985 15818 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:14.018505 11509 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:14.019874 10873 pending_rounds.cc:107] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 13:18:14.020262 10873 raft_consensus.cc:2889] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:13.944341 15683 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:18:13.943403 15870 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.030875 10950 raft_consensus.cc:1275] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 8. (term mismatch)
I20260709 13:18:14.053732 10950 pending_rounds.cc:85] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 7
I20260709 13:18:14.054105 10950 pending_rounds.cc:107] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 16.8
I20260709 13:18:14.054586 10950 raft_consensus.cc:2889] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:14.021307 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:14.066670 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:14.068074 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:14.069975 10941 raft_consensus.cc:1240] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:13.981521 15928 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.090175 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:13.914708 15639 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:14.023291 15709 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:13.971095 15854 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:13.974131 15907 raft_consensus.cc:2749] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:13.974567 15927 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:18:14.133198 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 210686us
I20260709 13:18:14.103103 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:14.135480 10949 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 13.
W20260709 13:18:14.136952 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.112079 15853 consensus_queue.cc:1048] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:14.045719 15713 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:14.103799 16012 consensus_queue.cc:1048] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 13:18:14.144716 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:14.084704 15893 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:14.021991 15918 raft_consensus.cc:2749] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:18:14.122978 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 239829us
I20260709 13:18:14.149284 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:14.151127 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:14.163237 10874 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:13.975348 15968 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:14.046401 15958 consensus_queue.cc:1059] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.106s
I20260709 13:18:14.166810 10873 raft_consensus.cc:1240] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:14.119760 15709 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:14.130209 15854 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:14.172829 10874 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 16.
I20260709 13:18:14.175674 10950 raft_consensus.cc:3055] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:14.201376 15847 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:18:14.162752 15989 log.cc:927] Time spent T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:18:14.022998 15861 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:14.216825 15861 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:14.222766 15828 raft_consensus.cc:3055] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:14.223358 15828 raft_consensus.cc:740] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:18:14.224752 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.228191 10798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 683) took 1707 ms. Trace:
I20260709 13:18:14.228686 10798 rpcz_store.cc:276] 0709 13:18:12.521180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:12.523155 (+  1975us) service_pool.cc:224] Handling call
0709 13:18:12.523454 (+   299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:12.527183 (+  3729us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:12.527212 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:12.527662 (+   450us) log.cc:844] Serialized 26 byte log entry
0709 13:18:13.837180 (+1309518us) spinlock_profiling.cc:243] Waited 2.01 ms on lock 0x7b1800085600. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a550d1 00007f9c80e70a0f
0709 13:18:13.971472 (+134292us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:13.983204 (+ 11732us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:13.983318 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:14.227835 (+244517us) raft_consensus.cc:1685] finished
0709 13:18:14.227865 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:14.228120 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27918,"raft.queue_time_us":25611,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":9193,"spinlock_wait_cycles":4631936,"thread_start_us":134059,"threads_started":1,"wal-append.queue_time_us":1533817}
I20260709 13:18:14.230391 15822 raft_consensus.cc:2804] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:14.230903 15822 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:14.231273 15822 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:14.236166 10842 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:14.242079 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:14.244019 10794 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:18:14.205089 10950 raft_consensus.cc:740] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:14.131062 15927 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:14.090969 15686 consensus_queue.cc:1059] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.079s
I20260709 13:18:14.224263 15828 consensus_queue.cc:260] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:14.248538 10950 consensus_queue.cc:260] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:14.091394 15870 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:14.101408 15928 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:14.211660 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 45547.8us
W20260709 13:18:14.261451 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 231454us
I20260709 13:18:14.155445 15883 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:14.261768 10950 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:14.170608 15713 raft_consensus.cc:697] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:14.272738 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:14.273826 15713 consensus_queue.cc:237] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:14.274680 11511 raft_consensus.cc:1240] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:14.277288 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:14.255795 15927 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:14.285395 10872 raft_consensus.cc:1275] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:18:14.203029 15765 consensus_queue.cc:1059] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 13:18:14.302608 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:14.259589 16012 raft_consensus.cc:2749] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:14.307337 10876 raft_consensus.cc:1275] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:18:14.329882 11504 raft_consensus.cc:2410] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:18:14.342706 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:14.271807 16008 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:18:14.355846 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 193448us
I20260709 13:18:14.303773 15828 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:14.262511 15861 consensus_queue.cc:1048] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.079s
I20260709 13:18:14.158900 15877 consensus_queue.cc:237] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:14.404213 10950 raft_consensus.cc:1275] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 13:18:14.405426 10950 pending_rounds.cc:85] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 9
I20260709 13:18:14.299036 15683 raft_consensus.cc:697] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 18 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:14.406195 10950 pending_rounds.cc:107] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 13:18:14.406920 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:14.407210 15683 consensus_queue.cc:237] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:14.407006 10950 raft_consensus.cc:2889] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:18:14.424789 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.378545 15765 consensus_queue.cc:1059] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 13:18:14.378993 16017 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:14.446753 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 270304us
I20260709 13:18:14.435562 15859 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:14.368835 16008 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:14.476511 10915 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:14.455312 11478 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:14.480595 11509 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 14.
I20260709 13:18:14.471239 15744 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.470778 16012 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.452281 16017 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:14.488175 15639 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:14.482764 16001 raft_consensus.cc:2749] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:14.470324 15911 consensus_queue.cc:1059] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:18:14.492023 15717 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:14.483740 15858 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:14.485293 16024 raft_consensus.cc:3055] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:14.532857 10801 raft_consensus.cc:1275] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:18:14.563925 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:14.539453 10875 raft_consensus.cc:1240] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 12. Current term is 13. Ops: []
W20260709 13:18:14.526914 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 170152us
I20260709 13:18:14.542454 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:14.565276 10799 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 16.
I20260709 13:18:14.565375 10916 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:14.566195 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:14.566831 10799 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:14.527340 16025 raft_consensus.cc:3055] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:14.530617 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:14.567621 10794 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 14.
I20260709 13:18:14.531862 10798 raft_consensus.cc:1240] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:14.564575 10801 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 13.
I20260709 13:18:14.533428 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:14.568969 10797 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:14.538877 10792 raft_consensus.cc:1240] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:18:14.524778 15968 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:14.547597 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:18:14.540463 15887 consensus_meta.cc:287] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.883s	user 0.014s	sys 0.018s
I20260709 13:18:14.581714 10795 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:14.547248 15808 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:14.558285 15928 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:14.506690 15744 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:14.562958 16024 raft_consensus.cc:740] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:14.556636 16021 consensus_queue.cc:1048] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 13:18:14.502013 16012 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:14.628491 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:14.620285 10795 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 13.
I20260709 13:18:14.629969 10797 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 14.
I20260709 13:18:14.581422 16025 raft_consensus.cc:740] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:14.584671 15717 raft_consensus.cc:697] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 16 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:14.604278 15968 raft_consensus.cc:2804] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:18:14.621763 15730 consensus_meta.cc:287] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.996s	user 0.007s	sys 0.006s
I20260709 13:18:14.630995 10799 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 15.
I20260709 13:18:14.636971 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:14.642587 10792 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:14.642467 10842 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:18:14.643381 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 46988.8us
I20260709 13:18:14.625571 16035 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:14.568653 15858 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:14.598048 16040 consensus_queue.cc:1059] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.058s
W20260709 13:18:14.664307 15906 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.078s	user 0.018s	sys 0.019s
W20260709 13:18:14.670617 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.647140 15968 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:14.657193 16015 raft_consensus.cc:2749] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:14.654215 15928 raft_consensus.cc:2804] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:14.617869 15808 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:14.687311 15805 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.687837 15805 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:14.688580 16012 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:14.689009 15808 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:14.690824 10801 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:14.682564 16040 consensus_queue.cc:1048] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
W20260709 13:18:14.686617 16034 log.cc:927] Time spent T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 13:18:14.686836 15744 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:14.693812 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:14.695223 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:14.742412 16032 consensus_queue.cc:1048] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:18:14.743042 10799 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 10.
I20260709 13:18:14.743149 10796 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
I20260709 13:18:14.701788 10792 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 17.
W20260709 13:18:14.708353 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.746802 10842 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:14.649427 15730 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:14.751066 10797 raft_consensus.cc:1240] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 11. Current term is 12. Ops: []
W20260709 13:18:14.682291 15919 consensus_meta.cc:287] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.819s	user 0.011s	sys 0.000s
W20260709 13:18:14.679054 15906 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.093s	user 0.018s	sys 0.019s
W20260709 13:18:14.755590 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 99511.2us
I20260709 13:18:14.740159 10793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2589) took 1062 ms. Trace:
W20260709 13:18:14.756909 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.756773 10793 rpcz_store.cc:276] 0709 13:18:13.677935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:13.683396 (+  5461us) service_pool.cc:224] Handling call
0709 13:18:13.683738 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:13.700550 (+ 16812us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:13.700587 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:13.701155 (+   568us) log.cc:844] Serialized 26 byte log entry
0709 13:18:13.888930 (+187775us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:13.889803 (+   873us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:14.690447 (+800644us) spinlock_profiling.cc:243] Waited 16.4 ms on lock 0x7b54000d8754. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:14.690501 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:14.690541 (+    40us) raft_consensus.cc:1685] finished
0709 13:18:14.690553 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:14.740073 (+ 49520us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2561,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":87,"spinlock_wait_cycles":37660544,"thread_start_us":187378,"threads_started":1,"wal-append.queue_time_us":467341}
I20260709 13:18:14.758436 10875 raft_consensus.cc:1240] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:18:14.682396 15968 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:14.762979 10799 raft_consensus.cc:1240] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 12. Current term is 13. Ops: []
W20260709 13:18:14.758600 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 282159us
W20260709 13:18:14.759168 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 200503us
I20260709 13:18:14.683171 15717 consensus_queue.cc:237] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:14.664134 16036 consensus_queue.cc:1059] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.103s
I20260709 13:18:14.684755 15928 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:14.768884 15875 consensus_queue.cc:1048] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 13:18:14.779170 10867 raft_consensus.cc:1240] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:14.781311 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:14.781816 10801 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
I20260709 13:18:14.772203 10766 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:14.773358 10875 raft_consensus.cc:1240] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:18:14.691685 15859 consensus_queue.cc:1048] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.081s
I20260709 13:18:14.760988 15744 raft_consensus.cc:2804] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:14.668412 16035 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:14.792549 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.798728 10875 raft_consensus.cc:1240] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 14. Current term is 15. Ops: [11.6-14.8]
I20260709 13:18:14.801208 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:14.808640 10796 raft_consensus.cc:1275] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:18:14.797518 10766 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:14.979000 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:14.983016 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:18:14.985908 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 188308us
I20260709 13:18:14.983016 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:14.985914 10793 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:14.996116 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:14.997412 10795 raft_consensus.cc:2410] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 13 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 13:18:14.998553 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:14.999933 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:15.001186 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:14.837024 10875 raft_consensus.cc:1240] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:15.005115 10798 raft_consensus.cc:1240] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:15.006732 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:15.007689 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:15.008508 10871 raft_consensus.cc:2393] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 18: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:15.010090 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:15.010115 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:15.013064 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:15.014856 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:15.011649 10798 raft_consensus.cc:1240] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:15.020147 10798 raft_consensus.cc:1275] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:18:15.022142 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:14.908674 10945 raft_consensus.cc:1275] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:18:15.060497 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:15.032917 10950 raft_consensus.cc:1240] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:15.033463 15639 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:14.929359 15709 consensus_meta.cc:287] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.676s	user 0.013s	sys 0.003s
W20260709 13:18:14.928011 15754 consensus_meta.cc:287] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.906s	user 0.023s	sys 0.000s
I20260709 13:18:15.068928 15754 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:15.072561 15639 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:15.073617 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 274954us
I20260709 13:18:14.893141 15805 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:14.930836 15822 consensus_meta.cc:287] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.699s	user 0.015s	sys 0.010s
I20260709 13:18:15.076542 15822 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:15.073912 10799 raft_consensus.cc:1240] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:15.078277 16067 consensus_queue.cc:1059] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:18:14.906942 15858 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:15.080431 10799 raft_consensus.cc:1240] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 16. Current term is 17. Ops: [14.11-16.12]
I20260709 13:18:15.080597 16056 raft_consensus.cc:3055] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:15.081388 16056 raft_consensus.cc:740] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:15.082286 10799 raft_consensus.cc:1240] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:15.082515 15757 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:14.926265 15854 consensus_meta.cc:287] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.673s	user 0.005s	sys 0.010s
I20260709 13:18:15.083534 15757 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:15.084635 15854 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:15.085448 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:15.085924 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:14.836103 15887 consensus_queue.cc:1059] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 13:18:15.088728 15887 raft_consensus.cc:3055] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:15.088992 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:15.041265 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:15.089936 10874 raft_consensus.cc:1240] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:14.886405 15730 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:15.040920 15892 raft_consensus.cc:3055] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:15.091051 15892 raft_consensus.cc:740] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:15.091277 15730 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:15.092046 15730 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:15.092789 15892 consensus_queue.cc:260] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:14.807425 15744 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:14.871234 15818 consensus_meta.cc:287] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.846s	user 0.010s	sys 0.001s
I20260709 13:18:15.094362 15744 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:15.095147 15818 raft_consensus.cc:2749] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:18:14.789362 15831 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.019s	sys 0.036s
W20260709 13:18:15.096314 15831 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.836s	user 0.019s	sys 0.036s
W20260709 13:18:14.926265 15847 consensus_meta.cc:287] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.676s	user 0.012s	sys 0.002s
I20260709 13:18:14.795317 15906 consensus_queue.cc:1059] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:18:14.776536 15919 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:14.918713 15927 consensus_meta.cc:287] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.576s	user 0.011s	sys 0.000s
I20260709 13:18:15.099080 15927 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:15.099512 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:18:14.927598 15828 consensus_meta.cc:287] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.525s	user 0.011s	sys 0.000s
I20260709 13:18:14.769894 16008 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:14.918856 16017 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:14.914264 16001 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:15.101832 15919 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:14.738055 16024 consensus_queue.cc:260] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:15.103195 10799 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8"
dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
mode: GRACEFUL
new_leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
 from {username='slave'} at 127.0.0.1:40148
I20260709 13:18:14.910113 16035 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:15.104210 10799 raft_consensus.cc:606] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Received request to transfer leadership to TS 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:15.104771 16035 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:15.105067 15919 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:14.683102 16025 consensus_queue.cc:260] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:14.835202 16039 raft_consensus.cc:3055] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:18:14.862320 16041 log.cc:927] Time spent T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
I20260709 13:18:15.106735 16039 raft_consensus.cc:740] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:14.784790 16046 consensus_queue.cc:1059] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 13:18:15.108741 16039 consensus_queue.cc:260] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:15.109637 16046 raft_consensus.cc:3055] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:15.110276 16046 raft_consensus.cc:740] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:15.111135 10875 raft_consensus.cc:2393] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 12: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:15.111681 16046 consensus_queue.cc:260] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:14.842286 16048 consensus_queue.cc:1048] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.087s
W20260709 13:18:14.827066 16050 log.cc:927] Time spent T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:18:14.826824 16053 raft_consensus.cc:2749] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:15.160408 15854 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:15.161955 15639 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:14.924858 16064 consensus_queue.cc:1059] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:18:15.163767 16064 raft_consensus.cc:3055] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:15.164351 16064 raft_consensus.cc:740] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:15.040962 15875 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:15.052469 10871 raft_consensus.cc:2393] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 16: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:15.052639 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:15.053365 11504 raft_consensus.cc:1275] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 13:18:15.062845 10793 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
I20260709 13:18:15.067106 16049 raft_consensus.cc:2749] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:15.065649 15709 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:15.089138 15887 raft_consensus.cc:740] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:14.774844 15928 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:15.102326 10766 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:15.105613 16035 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:14.834523 16036 raft_consensus.cc:3055] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:18:15.184733 16036 raft_consensus.cc:740] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:15.175294 11504 pending_rounds.cc:85] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586: Aborting all ops after (but not including) 10
I20260709 13:18:15.076529 16040 consensus_queue.cc:1059] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 13:18:15.186687 11504 pending_rounds.cc:107] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 13:18:15.187193 11504 raft_consensus.cc:2889] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:15.116711 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:15.103226 16001 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:15.109669 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:15.119437 15828 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:15.189584 15828 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:15.114928 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:15.190625 10876 raft_consensus.cc:1240] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:15.191628 16001 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:15.192101 10875 raft_consensus.cc:1240] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:15.193645 10875 raft_consensus.cc:1240] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:15.137368 16056 consensus_queue.cc:260] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:15.194186 16035 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:15.115233 16008 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:15.194808 16056 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:15.195438 16008 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:15.126073 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 48730us
I20260709 13:18:15.104741 10799 raft_consensus.cc:614] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:18:15.198657 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 96538.9us
I20260709 13:18:15.137177 10796 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 18.
I20260709 13:18:15.139712 10868 raft_consensus.cc:2393] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 15: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:18:15.201992 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:15.202376 10868 raft_consensus.cc:1240] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:15.203877 16047 consensus_queue.cc:1059] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:18:15.204620 10792 raft_consensus.cc:1240] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:15.142031 16039 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:15.203868 10871 raft_consensus.cc:1240] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:15.206278 15875 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:15.207625 15822 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:15.192101 10876 raft_consensus.cc:1240] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:15.186445 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:15.207777 15828 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:15.208885 15875 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:15.209374 10867 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 14.
I20260709 13:18:15.209717 15875 raft_consensus.cc:2749] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:15.210466 10872 raft_consensus.cc:1240] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:15.210821 16014 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:15.186421 16036 consensus_queue.cc:260] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:15.211541 16014 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:15.213581 16036 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:15.152261 15683 consensus_queue.cc:1048] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 13:18:15.182924 15757 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:15.187458 16040 raft_consensus.cc:3055] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:15.204245 10868 raft_consensus.cc:1275] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:18:15.141114 16046 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:15.094370 15892 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:15.205025 16047 raft_consensus.cc:3055] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:15.213869 10875 raft_consensus.cc:1240] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:15.233420 11507 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 14.
I20260709 13:18:15.213678 16008 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:15.215413 10796 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a"
dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
mode: GRACEFUL
new_leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
 from {username='slave'} at 127.0.0.1:40148
I20260709 13:18:15.215765 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:15.217173 15887 consensus_queue.cc:260] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:15.220317 15709 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:15.174265 16065 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:15.208981 10793 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:15.237016 10941 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 13.
I20260709 13:18:15.237876 15754 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:15.238700 15968 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:15.239864 10798 raft_consensus.cc:1240] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:15.233700 16047 raft_consensus.cc:740] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:15.244247 15852 raft_consensus.cc:3055] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:18:15.245543 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:15.246758 10949 raft_consensus.cc:3055] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:15.251906 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:15.253789 16024 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:15.256242 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:15.213521 16064 consensus_queue.cc:260] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:15.264367 15928 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:15.226323 16040 raft_consensus.cc:740] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:15.271842 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:15.272089 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:15.265350 15730 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:15.284724 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:15.286267 10949 raft_consensus.cc:740] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:18:15.298219 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:15.299309 10944 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 18.
W20260709 13:18:15.300344 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 83352.2us
I20260709 13:18:15.300904 10944 raft_consensus.cc:3055] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:15.301326 10944 raft_consensus.cc:740] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:15.302850 15887 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:15.304559 15968 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:15.306108 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:15.285868 15852 raft_consensus.cc:740] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:15.307829 15852 consensus_queue.cc:260] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:15.309638 15852 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:15.308635 10949 consensus_queue.cc:260] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:15.310812 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:15.315982 10942 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:15.288753 16065 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:15.320660 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:15.321036 10766 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:15.321437 10799 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
I20260709 13:18:15.324311 16035 raft_consensus.cc:2749] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:18:15.325079 16004 log.cc:927] Time spent T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 1.205s	user 0.001s	sys 0.001s
W20260709 13:18:15.326987 16027 log.cc:927] Time spent T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.802s	user 0.003s	sys 0.000s
I20260709 13:18:15.327270 10868 raft_consensus.cc:1240] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 9. Current term is 10. Ops: []
W20260709 13:18:15.327679 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:15.328660 15754 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:15.329306 15754 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:15.330317 10870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 786) took 1164 ms. Trace:
I20260709 13:18:15.329861 16040 consensus_queue.cc:260] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:15.330904 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:15.302290 10944 consensus_queue.cc:260] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:15.296423 16064 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:15.331559 10766 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:15.331573 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:15.314532 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:18:15.342285 16033 log.cc:927] Time spent T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.786s	user 0.002s	sys 0.000s
I20260709 13:18:15.343864 16040 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:15.330996 15683 consensus_queue.cc:1048] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.016s
I20260709 13:18:15.347175 10949 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:15.347715 10946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 2217) took 1634 ms. Trace:
I20260709 13:18:15.342317 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:15.329334 15911 consensus_queue.cc:1059] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 13:18:15.328817 16047 consensus_queue.cc:260] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:15.351109 16055 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:15.352072 16055 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:15.354903 10944 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:15.354503 16055 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:15.362537 10945 raft_consensus.cc:1275] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:18:15.348093 10946 rpcz_store.cc:276] 0709 13:18:13.713266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:13.721579 (+  8313us) service_pool.cc:224] Handling call
0709 13:18:13.918427 (+196848us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:13.919271 (+   844us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:15.347317 (+1428046us) spinlock_profiling.cc:243] Waited 175 ms on lock 0x7b54000172d0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:15.347518 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":402978176}
I20260709 13:18:15.364550 15754 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:15.367985 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:15.369036 10876 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 15.
I20260709 13:18:15.369735 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:15.371258 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:15.371762 16060 log.cc:927] Time spent T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.446s	user 0.001s	sys 0.000s
I20260709 13:18:15.372591 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:15.373000 11478 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:15.374285 15398 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:15.374806 15398 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:15.375227 15398 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:18:15.377566 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 54928.9us
I20260709 13:18:15.379055 10766 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:15.379731 15976 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.709s	user 0.015s	sys 0.019s
W20260709 13:18:15.380162 15976 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.800s	user 0.015s	sys 0.019s
I20260709 13:18:15.380301 16032 raft_consensus.cc:2749] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:15.382391 15928 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:15.385039 10942 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
I20260709 13:18:15.392107 10766 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:15.401338 10793 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
I20260709 13:18:15.330747 10870 rpcz_store.cc:276] 0709 13:18:14.166094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.166235 (+   141us) service_pool.cc:224] Handling call
0709 13:18:14.166551 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:14.167669 (+  1118us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:14.167700 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:14.168159 (+   459us) log.cc:844] Serialized 26 byte log entry
0709 13:18:14.436600 (+268441us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:14.436642 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:14.436713 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:15.326095 (+889382us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:18:15.329846 (+  3751us) raft_consensus.cc:1685] finished
0709 13:18:15.329870 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:15.330135 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2345,"raft.queue_time_us":22201,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":45472,"spinlock_wait_cycles":30059904,"thread_start_us":222007,"threads_started":1,"wal-append.queue_time_us":264326}
I20260709 13:18:15.410482 10867 raft_consensus.cc:1275] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 13:18:15.422241 16002 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.060s	user 0.006s	sys 0.012s
W20260709 13:18:15.422703 16002 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.122s	user 0.007s	sys 0.012s
W20260709 13:18:15.423542 10916 connection.cc:256] RPC call timeout handler was delayed by -1.9089730690000124s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1855s for a response
W20260709 13:18:15.424058 10916 connection.cc:256] RPC call timeout handler was delayed by -1.7838730690000375s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19940000000000002s for a response
I20260709 13:18:15.442746 10941 raft_consensus.cc:2410] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:18:15.455394 10949 raft_consensus.cc:1275] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 7. (term mismatch)
I20260709 13:18:15.456245 10949 pending_rounds.cc:85] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 6
I20260709 13:18:15.456843 10949 pending_rounds.cc:107] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 14.7
W20260709 13:18:15.457491 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:15.458266 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:15.458756 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:15.459293 10842 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:18:15.460057 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 127920us
I20260709 13:18:15.462924 15928 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:15.462975 15854 consensus_queue.cc:1059] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:15.463461 15928 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:15.463897 15928 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:15.464015 15754 raft_consensus.cc:3055] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:15.464596 15754 raft_consensus.cc:740] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:15.458091 10944 raft_consensus.cc:1275] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:18:15.465166 10944 pending_rounds.cc:85] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 7
I20260709 13:18:15.465467 10944 pending_rounds.cc:107] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:18:15.416296 15798 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:15.466601 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:15.466424 15798 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:15.468859 15798 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:15.471552 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:15.471825 10947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2543) took 1753 ms. Trace:
I20260709 13:18:15.457263 10949 raft_consensus.cc:2889] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:15.472240 10947 rpcz_store.cc:276] 0709 13:18:13.718400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:13.764034 (+ 45634us) service_pool.cc:224] Handling call
0709 13:18:13.764462 (+   428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:13.770951 (+  6489us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:13.770984 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:13.771028 (+    44us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:13.771066 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:15.471366 (+1700300us) spinlock_profiling.cc:243] Waited 6.24 ms on lock 0x7b54001e2654. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:15.471465 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:15.471645 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14386816}
I20260709 13:18:15.465746 10944 raft_consensus.cc:2889] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:18:15.464082 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:15.502403 10842 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:15.472220 15928 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:15.450230 15744 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:15.506322 10948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 863) took 1795 ms. Trace:
I20260709 13:18:15.508320 10948 rpcz_store.cc:276] 0709 13:18:13.710626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:13.714023 (+  3397us) service_pool.cc:224] Handling call
0709 13:18:13.714317 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:14.027595 (+313278us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:14.027632 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:14.027684 (+    52us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:14.027728 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:15.505089 (+1477361us) spinlock_profiling.cc:243] Waited 313 ms on lock 0x7b5400331154. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:15.505175 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:15.506111 (+   936us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":719960832}
W20260709 13:18:15.509073 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 82393.3us
I20260709 13:18:15.506307 15801 raft_consensus.cc:2804] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:15.512219 15798 raft_consensus.cc:2804] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:15.522944 15928 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:15.523589 10796 raft_consensus.cc:606] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Received request to transfer leadership to TS 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:15.523991 10796 raft_consensus.cc:614] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:15.526939 16037 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:15.528124 16037 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:15.528548 16037 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:15.535339 16047 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:15.519404 15801 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:18:15.552042 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:15.557183 16040 consensus_queue.cc:1048] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:15.550995 15798 raft_consensus.cc:697] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:18:15.558902 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 176783us
I20260709 13:18:15.559679 15798 consensus_queue.cc:237] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:15.560036 15730 consensus_queue.cc:1048] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:15.566738 15883 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:15.551834 15801 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:15.568271  9682 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:18:15.571511 16065 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:15.577287 15888 raft_consensus.cc:3055] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:15.577950 15888 raft_consensus.cc:740] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:15.581828 15870 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:15.583276 10841 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.729s	user 0.013s	sys 0.032s
W20260709 13:18:15.583792 10841 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.729s	user 0.013s	sys 0.032s
I20260709 13:18:15.591941 15744 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:15.604272 16040 consensus_queue.cc:1048] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:18:15.606127 16040 consensus_queue.cc:1059] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:15.608464 10916 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:15.608740 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 789) took 1328 ms. Trace:
I20260709 13:18:15.611071 16037 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:15.609244 15997 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.156s	user 0.044s	sys 0.053s
W20260709 13:18:15.619985 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:15.617595 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 1.81823e+06us
I20260709 13:18:15.625993 15859 raft_consensus.cc:2749] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:18:15.621269 15997 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.275s	user 0.045s	sys 0.054s
W20260709 13:18:15.637333 16030 log.cc:927] Time spent T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 1.081s	user 0.000s	sys 0.002s
W20260709 13:18:15.640317 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:15.643285 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:15.647238 16028 log.cc:927] Time spent T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 1.108s	user 0.002s	sys 0.000s
I20260709 13:18:15.648058 15870 consensus_queue.cc:1048] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:18:15.655061 16040 consensus_queue.cc:1243] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer 468a8fd46e824f62b9c2c0d3532a6d26 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:18:15.656425 16003 log.cc:927] Time spent T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 1.311s	user 0.000s	sys 0.005s
I20260709 13:18:15.656654 16037 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:15.667116 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:15.638432 10916 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:15.668308 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:15.669263 10942 raft_consensus.cc:2393] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 11: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:15.671180 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:15.672094 10949 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 16.
I20260709 13:18:15.675153 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:15.677057 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:15.679191 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:15.679476 11503 raft_consensus.cc:1240] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:18:15.670684 15888 consensus_queue.cc:260] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:15.681247 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:15.684325 10842 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:15.684599 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:15.686160 15907 raft_consensus.cc:2804] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:15.686671 15907 raft_consensus.cc:697] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:15.687686 15907 consensus_queue.cc:237] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:15.691423 16018 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:15.691736 10874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 788) took 1412 ms. Trace:
I20260709 13:18:15.692137 10874 rpcz_store.cc:276] 0709 13:18:14.279528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.281017 (+  1489us) service_pool.cc:224] Handling call
0709 13:18:14.281296 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:15.090479 (+809183us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:15.691428 (+600949us) spinlock_profiling.cc:243] Waited 808 ms on lock 0x7b540006b8d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:15.691672 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1858775296}
I20260709 13:18:15.609248 10873 rpcz_store.cc:276] 0709 13:18:14.279870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.280351 (+   481us) service_pool.cc:224] Handling call
0709 13:18:14.280813 (+   462us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:15.114048 (+833235us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:18:15.119244 (+  5196us) log.cc:844] Serialized 14 byte log entry
0709 13:18:15.177181 (+ 57937us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:15.177257 (+    76us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:15.177315 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:15.608349 (+431034us) spinlock_profiling.cc:243] Waited 832 ms on lock 0x7b54000997d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:15.608407 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:15.608557 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":213,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":59,"raft.queue_time_us":5264,"raft.run_cpu_time_us":355,"raft.run_wall_time_us":4443,"spinlock_wait_cycles":1914569856,"thread_start_us":56732,"threads_started":1,"wal-append.queue_time_us":248375}
I20260709 13:18:15.618129 16002 raft_consensus.cc:3055] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:18:15.718007 10841 connection.cc:256] RPC call timeout handler was delayed by -1.8007139420001295s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1514s for a response
I20260709 13:18:15.724256 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:15.708230 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:15.692173 16018 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:15.723995 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:15.718757 10947 raft_consensus.cc:1240] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:15.724936 10796 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:18:15.726974 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:15.730196 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:15.699815 15888 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:15.732911 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:15.734622 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:15.727200 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:15.741650 10943 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 12.
I20260709 13:18:15.746753 10795 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:15.746948 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 64489.7us
I20260709 13:18:15.748030 10916 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:18:15.749912 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:15.751171 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:15.751639 10797 raft_consensus.cc:1240] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:15.751940 10949 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
I20260709 13:18:15.723220 16002 raft_consensus.cc:740] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:15.743991 10944 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
I20260709 13:18:15.699717 15398 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:15.759876 10841 connection.cc:256] RPC call timeout handler was delayed by -1.4627795310000238s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17470000000000002s for a response
W20260709 13:18:15.763190 10841 connection.cc:256] RPC call timeout handler was delayed by -1.3093139420000171s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20600000000000002s for a response
W20260709 13:18:15.763581 10841 connection.cc:256] RPC call timeout handler was delayed by -1.2877139419999821s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20840000000000003s for a response
W20260709 13:18:15.763926 10841 connection.cc:256] RPC call timeout handler was delayed by -1.0968372730001192s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1574s for a response
I20260709 13:18:15.736555 15875 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:15.767877 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:15.762511 16064 raft_consensus.cc:2749] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:15.768712 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:15.769357 16040 raft_consensus.cc:2749] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:15.769773 15757 consensus_queue.cc:1059] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:15.775401 10950 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
I20260709 13:18:15.780680 10766 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:15.741556 15859 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:15.783918 11504 raft_consensus.cc:2368] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 15. Current term is 16.
I20260709 13:18:15.799203 10915 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:15.800877 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:15.801708 10949 raft_consensus.cc:2393] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 11: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:15.797379 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:15.805940 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:15.766155 15798 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:15.809226 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:15.762354 15744 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:15.831693 10795 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 11.
I20260709 13:18:15.838680 10842 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:15.840346 10947 raft_consensus.cc:3055] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:15.844323 15757 raft_consensus.cc:2804] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:15.845016 15757 raft_consensus.cc:697] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:15.846055 15757 consensus_queue.cc:237] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:15.846975 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:15.847743 10949 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
I20260709 13:18:15.848237 16002 consensus_queue.cc:260] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:15.849514 16002 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:15.850572 15744 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:15.853245 15744 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:15.854038 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:15.784339 16065 consensus_queue.cc:1059] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 13:18:15.805863 16032 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:15.840855 10947 raft_consensus.cc:740] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:18:15.855310 16085 log.cc:927] Time spent T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 13:18:15.856088 10947 consensus_queue.cc:260] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:15.856652 15798 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:15.857252 10947 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:15.860646 10841 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:15.863884 15883 raft_consensus.cc:2749] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:15.865140 16064 raft_consensus.cc:2749] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:15.866915 15997 raft_consensus.cc:3055] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:15.867442 15997 raft_consensus.cc:740] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:15.868602 15997 consensus_queue.cc:260] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:15.869864 15997 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:15.870887 15859 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:15.872841 10793 raft_consensus.cc:1275] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:18:15.873577 15859 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:15.874015 10767 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:18:15.874243 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 70369.4us
I20260709 13:18:15.875231 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:15.876044 10916 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:15.876623 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:15.876740 15875 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:15.878315 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:15.879295 15875 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:15.855317 16032 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:15.881081 16032 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:18:15.882107 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 83011.6us
I20260709 13:18:15.827033 16026 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:15.884557 16026 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:15.884970 15398 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:15.862525 10841 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:15.886545 10950 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 14.
I20260709 13:18:15.890070 10796 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 16.
W20260709 13:18:15.890342 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:15.886866 10766 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:15.898465 10795 raft_consensus.cc:1240] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:15.899294 10947 raft_consensus.cc:1275] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:18:15.900686 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:15.900600  9682 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:15.907219 10842 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:15.908036 10796 raft_consensus.cc:1240] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:15.893033 15870 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:18:15.915683 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:15.917161 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:18:15.921653 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 45713.9us
I20260709 13:18:15.925004 16063 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:15.929167 16025 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:15.931919 15744 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:15.932130 16018 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:15.932562 15744 raft_consensus.cc:697] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:15.932431 15928 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:18:15.933981 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 60169.2us
I20260709 13:18:15.933578 15928 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:15.933826 15744 consensus_queue.cc:237] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:15.935050 15798 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:15.938203 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:15.939138 10945 raft_consensus.cc:2393] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 18: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:15.940338 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:15.913956 15875 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:15.941361 15875 raft_consensus.cc:697] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:15.943106 15883 raft_consensus.cc:2804] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:15.943548 15883 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:15.943944 15883 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:15.969103 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:18:15.970937 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:15.978921 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 116555us
W20260709 13:18:15.979116 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 44346.4us
I20260709 13:18:15.978017 15928 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:15.995018 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:16.003299 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:16.004866 16016 consensus_queue.cc:1048] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 13:18:16.009254 16016 consensus_queue.cc:1243] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer cb0428cc55984daba1ba63744a5102a2 log is divergent from this leader: its last log entry 16.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:18:16.016362 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:16.015777 10841 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:16.026484 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:15.978816 15798 consensus_queue.cc:1059] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 15.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
W20260709 13:18:16.015149 16052 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.880s	user 0.019s	sys 0.016s
I20260709 13:18:15.978647 15870 consensus_queue.cc:1048] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:18:16.035512 15854 raft_consensus.cc:2749] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:16.036500 16026 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:16.000165 16031 consensus_queue.cc:1048] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:18:16.018927 16039 consensus_queue.cc:1059] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:18:16.047271 15875 consensus_queue.cc:237] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:15.998332 16054 consensus_queue.cc:1048] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:18:16.052067 15801 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:16.038486 15927 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:16.055100 16052 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.127s	user 0.020s	sys 0.016s
W20260709 13:18:16.060532 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.067077 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:16.072532 10949 raft_consensus.cc:3055] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:16.076999 10949 raft_consensus.cc:740] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:16.077068 10795 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 12.
I20260709 13:18:16.078508 10916 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:16.073258 16032 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:16.073714 16039 raft_consensus.cc:3055] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:16.084358 10872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 793) took 1294 ms. Trace:
I20260709 13:18:16.085347 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:16.061890 15928 raft_consensus.cc:3055] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:16.084892 10872 rpcz_store.cc:276] 0709 13:18:14.789574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.789763 (+   189us) service_pool.cc:224] Handling call
0709 13:18:15.210092 (+420329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:15.211018 (+   926us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.082721 (+871703us) spinlock_profiling.cc:243] Waited 420 ms on lock 0x7b54002d4450. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:16.084279 (+  1558us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":965482880}
I20260709 13:18:16.091759 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:18:16.089857 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 83363.1us
I20260709 13:18:16.093171 10794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 706) took 1461 ms. Trace:
I20260709 13:18:16.093603 10801 raft_consensus.cc:2368] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 18. Current term is 19.
I20260709 13:18:16.095058 10867 raft_consensus.cc:1240] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:16.105118 10945 raft_consensus.cc:2393] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 13: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:16.110345 10949 consensus_queue.cc:260] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:16.084568 16039 raft_consensus.cc:740] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:16.089342 16065 raft_consensus.cc:2804] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:16.114828 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:16.090945 15928 raft_consensus.cc:740] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:16.114910 10947 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
W20260709 13:18:16.118422 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:16.123063 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.118050 11510 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 12.
I20260709 13:18:16.121454 10949 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:16.093616 10794 rpcz_store.cc:276] 0709 13:18:14.627688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.842672 (+1214984us) service_pool.cc:224] Handling call
0709 13:18:15.898616 (+ 55944us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:15.899042 (+   426us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:18:15.919529 (+ 20487us) log.cc:844] Serialized 14 byte log entry
0709 13:18:16.088884 (+169355us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:16.088936 (+    52us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:16.088983 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.089071 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:16.089200 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":20320,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":59,"raft.queue_time_us":3761,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":90,"thread_start_us":164936,"threads_started":1}
I20260709 13:18:16.122382 10945 raft_consensus.cc:2393] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 11: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:16.128391 10867 raft_consensus.cc:1275] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 13:18:16.125934 16090 log.cc:927] Time spent T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:18:16.133191 10872 raft_consensus.cc:1275] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 13:18:16.135058 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 101090us
I20260709 13:18:16.127182 16063 raft_consensus.cc:2749] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:16.124575 16065 raft_consensus.cc:697] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 14 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:18:16.141289 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 137843us
I20260709 13:18:16.144659 10867 raft_consensus.cc:1275] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:18:16.136191 15765 raft_consensus.cc:3055] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:16.171828 10766 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:18:16.152222 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:16.215721 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:16.196192 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 81537.4us
I20260709 13:18:16.203917 10947 raft_consensus.cc:3055] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:16.217495 10947 raft_consensus.cc:740] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:18:16.208081 16096 log.cc:927] Time spent T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 13:18:16.231352 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:16.232324 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.132539 16032 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:16.201411 15801 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:16.171088 15928 consensus_queue.cc:260] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:16.228174 10945 raft_consensus.cc:1275] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:18:16.240352 16031 raft_consensus.cc:2749] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:16.214180 16063 consensus_queue.cc:1059] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:18:16.244436 16063 consensus_queue.cc:1243] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer 468a8fd46e824f62b9c2c0d3532a6d26 log is divergent from this leader: its last log entry 11.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:18:16.245015 16063 raft_consensus.cc:3055] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:16.245355 16063 raft_consensus.cc:740] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:18:16.247210 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 101856us
I20260709 13:18:16.248108 10766 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:16.249590 10766 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:16.217029 15765 raft_consensus.cc:740] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:16.258589 10949 raft_consensus.cc:1275] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
W20260709 13:18:16.251097 16088 log.cc:927] Time spent T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:18:16.193907 16039 consensus_queue.cc:260] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:16.260520 10947 consensus_queue.cc:260] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:16.246294 16063 consensus_queue.cc:260] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:16.259101 10949 pending_rounds.cc:85] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 9
I20260709 13:18:16.266122 10949 pending_rounds.cc:107] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 13:18:16.266369 10949 raft_consensus.cc:2889] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:16.260284 15870 raft_consensus.cc:2749] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:16.274844 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:16.265739 15928 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:16.259044 16031 raft_consensus.cc:2749] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:16.281761 10941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41014 (request call id 2236) took 1032 ms. Trace:
I20260709 13:18:16.283560 10792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2059) took 1307 ms. Trace:
I20260709 13:18:16.284054 10792 rpcz_store.cc:276] 0709 13:18:14.976584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.071347 (+ 94763us) service_pool.cc:224] Handling call
0709 13:18:15.071832 (+   485us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:15.205128 (+133296us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.283239 (+1078111us) spinlock_profiling.cc:243] Waited 93.3 ms on lock 0x7b5400295254. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:16.283465 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":214594304}
W20260709 13:18:16.285768 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.294871 10947 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:16.295001 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:16.296177 15757 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:18:16.297756 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.282279 10941 rpcz_store.cc:276] 0709 13:18:15.248937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.305710 (+ 56773us) service_pool.cc:224] Handling call
0709 13:18:16.281483 (+975773us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b5400016dd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:16.281662 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":312048000}
I20260709 13:18:16.297971 10949 raft_consensus.cc:1275] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 13:18:16.298230 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.298617 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:16.299897 10945 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:18:16.301059 11507 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56410 (request call id 811) took 2540 ms (client timeout 2084 ms). Trace:
W20260709 13:18:16.301494 11507 rpcz_store.cc:269] 0709 13:18:13.760526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:13.760631 (+   105us) service_pool.cc:224] Handling call
0709 13:18:16.300799 (+2540168us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b54005ff5d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:16.300969 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3031981440}
I20260709 13:18:16.268929 15754 consensus_queue.cc:260] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:16.309684 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:16.323904 16036 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
W20260709 13:18:16.325031 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 108325us
I20260709 13:18:16.313324 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:16.296881 15757 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:16.283602 16098 log.cc:927] Time spent T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
W20260709 13:18:16.332417 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:16.333072 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.337850 10946 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:16.257966 16065 consensus_queue.cc:237] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:16.341624 10944 raft_consensus.cc:1275] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:18:16.346994 10875 raft_consensus.cc:1275] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 13:18:16.347368 10867 raft_consensus.cc:1240] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 LEADER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:16.330042 15927 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:16.349177 15927 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:16.351711 15883 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:16.353993 10841 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:16.354967 15854 raft_consensus.cc:2749] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:18:16.355940 10841 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:16.357457 10841 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:16.360065 15754 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:18:16.361308 16089 log.cc:927] Time spent T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.435s	user 0.004s	sys 0.000s
I20260709 13:18:16.362954 15765 consensus_queue.cc:260] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:16.363674 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:16.364462 16063 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:16.369056 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:18:16.369670 16097 log.cc:927] Time spent T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.191s	user 0.000s	sys 0.003s
I20260709 13:18:16.370164 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:16.370332 10871 raft_consensus.cc:1240] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:18:16.383251 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:18:16.371742 16093 log.cc:927] Time spent T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.235s	user 0.000s	sys 0.002s
I20260709 13:18:16.370222 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:16.383355 10870 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:16.387377 10942 raft_consensus.cc:1275] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:18:16.377835 10947 raft_consensus.cc:1275] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 13:18:16.378479 10876 raft_consensus.cc:2393] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 12: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:16.393045 10947 pending_rounds.cc:85] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 8
I20260709 13:18:16.393049 15765 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:16.393462 10947 pending_rounds.cc:107] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 13:18:16.393761 10947 raft_consensus.cc:2889] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:16.398062 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:16.400197 10798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2058) took 1423 ms. Trace:
I20260709 13:18:16.400403 10867 raft_consensus.cc:1240] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:18:16.400678 10798 rpcz_store.cc:276] 0709 13:18:14.976295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.021031 (+ 44736us) service_pool.cc:224] Handling call
0709 13:18:15.021428 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:15.277092 (+255664us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.399932 (+1122840us) spinlock_profiling.cc:243] Waited 218 ms on lock 0x7b54002111d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:16.400132 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":501710720}
I20260709 13:18:16.402199 10944 pending_rounds.cc:85] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 8
I20260709 13:18:16.402622 10944 pending_rounds.cc:107] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 13:18:16.402917 10944 raft_consensus.cc:2889] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:16.409904 16039 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:16.419919 10799 raft_consensus.cc:1275] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:18:16.421161 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.422701 10801 raft_consensus.cc:1240] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 14. Current term is 15. Ops: [14.7-14.7]
I20260709 13:18:16.423559 11507 raft_consensus.cc:1240] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 16 LEADER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
W20260709 13:18:16.424445 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.425658 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:16.425927 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:16.426728 10799 raft_consensus.cc:2410] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 16 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:18:16.431516 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:16.436476 15907 raft_consensus.cc:2804] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:16.437232 15907 raft_consensus.cc:697] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:16.438756 10799 raft_consensus.cc:1240] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:18:16.439137 15986 raft_consensus.cc:2804] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:16.440304 16014 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:16.441267 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:16.443224 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:16.443943 11510 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 12.
I20260709 13:18:16.455433 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:16.462064 10870 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
I20260709 13:18:16.465147 10946 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 13.
W20260709 13:18:16.469878 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:16.477273 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:16.478773 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.480649 10949 raft_consensus.cc:1240] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 12. Current term is 13. Ops: []
W20260709 13:18:16.482939 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 68828us
I20260709 13:18:16.482992 10941 raft_consensus.cc:1240] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:18:16.484658 10945 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 19.
I20260709 13:18:16.484014 10766 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:16.489116 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:16.491555 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.491132 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:16.492353 10794 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:16.492496 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:16.498071 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:16.498981 10796 raft_consensus.cc:2368] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 15. Current term is 16.
I20260709 13:18:16.504103 10797 raft_consensus.cc:1240] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 LEADER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:16.505064 10766 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:16.491173 16031 raft_consensus.cc:2804] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:16.438308 15907 consensus_queue.cc:237] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:16.506814 16031 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:16.507313 16031 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:18:16.511111 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 153790us
I20260709 13:18:16.511354 10795 raft_consensus.cc:1240] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:16.512642 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:16.513013 10841 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:18:16.514623 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.514849 16032 consensus_queue.cc:1048] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 13:18:16.515647 16032 consensus_queue.cc:1243] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer cb0428cc55984daba1ba63744a5102a2 log is divergent from this leader: its last log entry 12.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:18:16.516590 15845 consensus_queue.cc:1048] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:16.518236 16014 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:16.518725 16014 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:16.519256 16014 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:16.491233 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:16.523088 10794 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
I20260709 13:18:16.523535 10798 raft_consensus.cc:1240] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:16.534835 10792 raft_consensus.cc:2393] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 11: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:16.527304 10870 raft_consensus.cc:1240] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 14. Current term is 15. Ops: [11.6-14.8]
I20260709 13:18:16.524783 11512 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
W20260709 13:18:16.556105 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:16.557049 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.521956 16067 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:16.566112 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:16.571527 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:18:16.573010 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 68043.1us
I20260709 13:18:16.573094 10915 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:16.585610 10842 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:16.580612 10766 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:18:16.592209 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.503917 16054 consensus_queue.cc:1048] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.075s
I20260709 13:18:16.596732 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:16.524705 15875 consensus_queue.cc:1048] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.065s
W20260709 13:18:16.610708 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.617236 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:16.618211 10875 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:16.558873 16026 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:16.573100 16040 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:16.621073 10801 raft_consensus.cc:1275] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 13:18:16.627755 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:16.644740 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.611781 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:18:16.649575 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 49128.8us
W20260709 13:18:16.651506 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.654400 10842 leader_election.cc:400] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 18. Current term is 19.
I20260709 13:18:16.654831 10842 leader_election.cc:403] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:18:16.593662 16016 raft_consensus.cc:2804] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:16.559959 15997 consensus_queue.cc:1048] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.071s
I20260709 13:18:16.658334 16040 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:16.648715 11478 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.116.129:39685 timed out after 1.637s (SENT) [suppressed 9 similar messages]
I20260709 13:18:16.588305 15928 raft_consensus.cc:2764] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
W20260709 13:18:16.664805 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 91844.6us
W20260709 13:18:16.677618 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 88445.9us
W20260709 13:18:16.678889 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:16.685621 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.659914 15870 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:18:16.628005 15744 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:16.645548 16063 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:16.660552 15798 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:16.677304 15997 consensus_queue.cc:1243] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer cb0428cc55984daba1ba63744a5102a2 log is divergent from this leader: its last log entry 13.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:18:16.699834 15870 raft_consensus.cc:697] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:18:16.704778 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 183966us
I20260709 13:18:16.619954 16052 consensus_queue.cc:1059] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.085s
I20260709 13:18:16.682526 15765 raft_consensus.cc:2804] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:16.684942 15816 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:16.667287 16026 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:16.681042 16016 raft_consensus.cc:697] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:16.690946 16025 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:16.707952 15928 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:16.601572 16067 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:16.715222 15765 raft_consensus.cc:697] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 12 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:16.752573 11477 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:16.702055 16063 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:16.750440 10841 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:18:16.763074 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 247630us
I20260709 13:18:16.757257 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:16.766578 15859 consensus_queue.cc:1048] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:18:16.768107 15870 consensus_queue.cc:237] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:16.769456 10916 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:16.717108 15986 raft_consensus.cc:697] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:16.730479 15845 consensus_queue.cc:1048] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 13:18:16.773087 15986 consensus_queue.cc:237] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:16.774174 16102 log.cc:927] Time spent T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 13:18:16.760227 16054 consensus_queue.cc:1059] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
W20260709 13:18:16.775117 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.781421 15859 raft_consensus.cc:2749] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:16.783221 10916 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:16.787138 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.789057 16026 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:16.790166 15859 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:16.792359 10916 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:16.795416 16016 consensus_queue.cc:237] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:16.799064 15861 consensus_queue.cc:1048] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:16.721055 15816 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:16.800727 15883 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:16.798029 10868 raft_consensus.cc:1240] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 16. Current term is 17. Ops: []
W20260709 13:18:16.802722 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 52408.5us
I20260709 13:18:16.803550 10841 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:16.804620 10841 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:16.805823 10841 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:16.708850 15798 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:16.812577 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 60143us
W20260709 13:18:16.814586 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.815089 16047 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:16.724822 16025 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:16.711246 15744 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:16.815886 16047 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:16.832762 15883 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:16.807349 16040 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:16.839601 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 47337.3us
I20260709 13:18:16.817200 15907 raft_consensus.cc:2804] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:18:16.845187 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.817066 16001 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:16.816636 15859 raft_consensus.cc:2749] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:16.831835 10875 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 16.
I20260709 13:18:16.816783 15754 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:16.764394 16037 consensus_queue.cc:1048] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 13:18:16.855923 10916 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:16.834734 16026 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:16.853608 15754 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:16.860230 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:16.858505 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 277982us
I20260709 13:18:16.838172 15883 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:18:16.838418 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 159771us
W20260709 13:18:16.864915 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.852597 15907 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:16.905584 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.865603 16040 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:16.861202 16026 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:16.867549 15754 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:16.911405 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:16.912170 15798 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:16.858551 15997 consensus_queue.cc:1048] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 13:18:16.913632 16063 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:16.913703 15927 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:16.915141 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 59313.7us
I20260709 13:18:16.916145 15875 raft_consensus.cc:3055] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:16.916252 10916 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:16.858354 15816 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:16.920156 16107 raft_consensus.cc:2749] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:16.888356 15859 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:16.916646 15875 raft_consensus.cc:740] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:16.886147 15883 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:16.891744 15907 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:16.932518 10950 raft_consensus.cc:1275] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:18:16.924875 16026 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:16.934134 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:16.940092 10875 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
I20260709 13:18:16.940603 10868 raft_consensus.cc:2393] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 17: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:18:16.935752 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 286120us
W20260709 13:18:16.929965 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 124216us
I20260709 13:18:16.942812 10841 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:16.925877 16040 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:16.944573 15744 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:16.945158 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:16.945933 16040 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:16.945928 10872 raft_consensus.cc:2393] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 14: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:18:16.954008 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 207252us
W20260709 13:18:16.956717 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 143154us
I20260709 13:18:16.964951 10875 raft_consensus.cc:1275] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:18:16.965463 10941 raft_consensus.cc:1217] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Deduplicated request from leader. Original: 8.6->[9.7-15.10]   Dedup: 11.8->[13.9-15.10]
I20260709 13:18:16.966130 10941 pending_rounds.cc:85] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 8
I20260709 13:18:16.966432 10941 pending_rounds.cc:107] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 12.9
I20260709 13:18:16.966732 10941 raft_consensus.cc:2889] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:16.975441 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:16.893857 15765 consensus_queue.cc:237] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:16.980213 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.978879 16001 consensus_queue.cc:1048] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.005s
I20260709 13:18:16.983731 16067 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:16.985878 16001 consensus_queue.cc:1059] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:16.988747 16109 consensus_queue.cc:1048] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:18:16.990495 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:17.003261 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:17.003449 16047 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:17.004860 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:17.005404 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:17.006469 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:17.007211 10875 raft_consensus.cc:2393] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 13: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:17.007498 10876 raft_consensus.cc:1275] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:18:17.008852 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:17.011221 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:17.012050 11507 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 14.
W20260709 13:18:17.013463 16039 consensus_meta.cc:287] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.603s	user 0.010s	sys 0.002s
I20260709 13:18:17.014012 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:17.014339 10943 raft_consensus.cc:1275] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:18:16.978986 15816 consensus_queue.cc:1048] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 13:18:17.017115 15816 raft_consensus.cc:3055] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:17.017495 15816 raft_consensus.cc:740] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:17.018811 15875 consensus_queue.cc:260] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:17.020414 15875 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:17.027540 16036 raft_consensus.cc:3055] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:17.028102 16036 raft_consensus.cc:740] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:17.029096 16036 consensus_queue.cc:260] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:17.030304 16036 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:17.032160 10871 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 16.
W20260709 13:18:17.032481 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 61994.5us
I20260709 13:18:17.007576 10872 raft_consensus.cc:1275] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:18:16.982887 10767 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:18:17.048224 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:17.056202 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 74037us
I20260709 13:18:16.977674 15854 raft_consensus.cc:2804] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:17.064296 10766 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:16.998301 16031 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:17.075091 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:17.076663 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:17.077817 10870 raft_consensus.cc:2410] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
W20260709 13:18:17.079276 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 45552.5us
I20260709 13:18:17.080674 15861 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.065644 15854 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:18:17.097472 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 114731us
I20260709 13:18:17.100883 10767 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:17.102551 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:17.018471 15816 consensus_queue.cc:260] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:17.103629 10841 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:17.113322 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:17.097770 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 122883us
I20260709 13:18:17.116926 10916 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:17.085847 16109 raft_consensus.cc:2749] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:17.115291 15816 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:17.101454 15861 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:17.127282 10841 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:17.098342 16032 raft_consensus.cc:2749] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:17.104501 15854 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:17.130869 15964 raft_consensus.cc:2749] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:17.129278 16111 raft_consensus.cc:2749] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:17.138530 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:17.136157 16122 raft_consensus.cc:2749] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:17.137353 16039 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:17.141237 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:17.141849 16039 raft_consensus.cc:697] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 13 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:17.142524 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:17.143393 10942 raft_consensus.cc:2393] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 13: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:17.143157 16039 consensus_queue.cc:237] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:17.144270 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:17.135787 10946 pending_rounds.cc:85] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 11
I20260709 13:18:17.147236 10946 pending_rounds.cc:107] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 13:18:17.147572 10946 raft_consensus.cc:2889] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:18:17.149362 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:17.154474 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 140850us
I20260709 13:18:17.133395 15928 raft_consensus.cc:2749] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 18. Current term is 19.
I20260709 13:18:17.161139 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:17.162644 11507 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 16.
W20260709 13:18:17.164369 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 63588.5us
W20260709 13:18:17.170984 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:17.173193 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:17.174331 10870 raft_consensus.cc:1275] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:18:17.179379 10800 raft_consensus.cc:1275] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:18:17.180119 10868 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 19.
W20260709 13:18:17.182922 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 118731us
W20260709 13:18:17.196343 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:17.200922 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:17.183843 10766 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:18:17.175956 16014 consensus_meta.cc:287] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.645s	user 0.012s	sys 0.001s
I20260709 13:18:17.230487 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:17.222664 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:17.230341 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:17.234002 11512 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:17.237602 10873 raft_consensus.cc:1275] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:18:17.231451 10801 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
I20260709 13:18:17.238485 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:17.247527 10872 raft_consensus.cc:1275] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:18:17.248356 10792 raft_consensus.cc:1275] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:18:17.179471 16111 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:18:17.258287 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 68621.1us
I20260709 13:18:17.219744 16032 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.261260 10842 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:17.222903 16129 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:17.264923 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 48596.4us
I20260709 13:18:17.265825 10767 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:17.179010 16109 consensus_queue.cc:1048] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.075s
W20260709 13:18:17.280719 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 180823us
I20260709 13:18:17.287937 10801 raft_consensus.cc:1240] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:17.289245 11512 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 12.
I20260709 13:18:17.260476 15801 raft_consensus.cc:2749] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:18:17.290915 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 107186us
I20260709 13:18:17.222332 16122 consensus_queue.cc:1048] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:18:17.242116 16014 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:17.296518 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:17.239538 16126 consensus_queue.cc:1048] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 13:18:17.302475 10767 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:17.266961 15757 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:17.279973 15928 raft_consensus.cc:2804] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:17.308163 10876 raft_consensus.cc:1240] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:18:17.291574 15964 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:17.319489 10766 leader_election.cc:400] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 15. Current term is 16.
I20260709 13:18:17.307638 10916 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:17.267970 16032 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:17.330117 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:17.332768 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:17.303818 16031 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:17.339855 15853 consensus_queue.cc:1048] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:18:17.284261 16133 consensus_queue.cc:1048] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.056s
W20260709 13:18:17.342913 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:17.344856 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:17.293365 15986 consensus_queue.cc:1048] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:18:17.271814 16129 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:17.370847 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 75048.8us
W20260709 13:18:17.367228 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 64791us
I20260709 13:18:17.264948 16111 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:17.326017 15928 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:17.330149 15801 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:17.396034 10800 raft_consensus.cc:1275] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 12. (term mismatch)
W20260709 13:18:17.396222 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:17.391974 16128 log.cc:927] Time spent T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.239s	user 0.000s	sys 0.001s
I20260709 13:18:17.365768 16064 raft_consensus.cc:2764] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader election decision vote started in defunct term 18: won
I20260709 13:18:17.401937 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:17.398159 10800 pending_rounds.cc:85] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 11
I20260709 13:18:17.403741 10947 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:17.403749 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:17.408010 16129 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:17.408891 15854 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:17.397208 16109 consensus_queue.cc:1048] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 13:18:17.402226 15861 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:17.457875 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:18:17.410344 16134 log.cc:927] Time spent T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:18:17.401228 16141 consensus_queue.cc:1059] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 13:18:17.415661 16025 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:17.419513 10943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 2250) took 1439 ms. Trace:
I20260709 13:18:17.404093 10800 pending_rounds.cc:107] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 13:18:17.514158 10800 raft_consensus.cc:2889] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:18:17.422467 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 266976us
I20260709 13:18:17.397580 15928 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:17.457237 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:17.516068 10944 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
W20260709 13:18:17.516152 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 97875.5us
W20260709 13:18:17.467561 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:17.488729 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:17.424981 16016 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.512894 10871 raft_consensus.cc:1240] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:17.512931 10867 raft_consensus.cc:1240] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
W20260709 13:18:17.519341 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 59785.6us
I20260709 13:18:17.015249  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:17.543838 10945 raft_consensus.cc:1275] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:18:17.546391 10945 raft_consensus.cc:1275] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:18:17.452732 15754 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:17.548920 15754 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:17.549757 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:17.550431 10949 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 16.
I20260709 13:18:17.551802 16111 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:17.552152 15854 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:17.538476 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:18:17.471524 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:17.559058 16142 log.cc:927] Time spent T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
W20260709 13:18:17.559427 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:17.560325 16047 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.560937 16047 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:17.567020 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 51603.3us
I20260709 13:18:17.569995 16145 consensus_queue.cc:1059] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:17.519028 16025 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:17.571547 16014 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:17.518687 10943 rpcz_store.cc:276] 0709 13:18:15.980228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.219029 (+238801us) service_pool.cc:224] Handling call
0709 13:18:16.251639 (+ 32610us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:17.014765 (+763126us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.410717 (+395952us) spinlock_profiling.cc:243] Waited 762 ms on lock 0x7b54003f0054. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:17.419453 (+  8736us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1753147776}
I20260709 13:18:17.571468 16025 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:17.581722 16026 consensus_meta.cc:287] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.608s	user 0.010s	sys 0.001s
W20260709 13:18:17.583787 15883 consensus_meta.cc:287] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.646s	user 0.010s	sys 0.000s
I20260709 13:18:17.585254 10947 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 13.
I20260709 13:18:17.582227 15964 raft_consensus.cc:2749] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:18:17.589219 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:17.590806 10916 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:17.518327 16016 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:17.590070 16036 consensus_meta.cc:287] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.559s	user 0.009s	sys 0.004s
W20260709 13:18:17.591969 15859 consensus_meta.cc:287] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.659s	user 0.009s	sys 0.001s
W20260709 13:18:17.599606 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 354682us
I20260709 13:18:17.604239 10916 leader_election.cc:400] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 15. Current term is 16.
W20260709 13:18:17.604398 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 272097us
I20260709 13:18:17.605872 10916 leader_election.cc:403] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:17.608492 10916 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:17.590247 16047 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:18:17.617616 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:17.598733 15801 raft_consensus.cc:2749] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:17.613330 15859 raft_consensus.cc:2749] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:17.589671 16026 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:17.592173 15883 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:17.645802 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:17.656127 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:17.656958 10870 raft_consensus.cc:2410] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:18:17.647902 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:17.614636 16052 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:17.655208 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:17.621778 16039 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 16
W20260709 13:18:17.655958 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 128219us
I20260709 13:18:17.658114 10876 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 12.
I20260709 13:18:17.641889 15854 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.659962 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:17.664572 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:17.663316 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:17.661736 10800 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 14.
W20260709 13:18:17.666853 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 273223us
I20260709 13:18:17.667516 10767 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:17.668660 10767 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:17.649618 15765 raft_consensus.cc:2804] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:17.670239 10794 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 15.
I20260709 13:18:17.662552 10916 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:17.672026 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:17.746398 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 83918.8us
I20260709 13:18:17.666498 16040 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:17.741850 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:17.750337 10792 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
I20260709 13:18:17.741801 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:17.743806 10868 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
W20260709 13:18:17.747318 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 78740.3us
W20260709 13:18:17.748730 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 179305us
I20260709 13:18:17.758183 10916 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:17.666234 16067 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:17.762646 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:17.762705 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:18:17.784799 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:17.664193 15997 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.793849 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:18:17.804189 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 139940us
I20260709 13:18:17.806006 10841 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:17.749883 15859 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:17.807097 10798 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
I20260709 13:18:17.811010 10916 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:17.825944 10841 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:17.669564 15854 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:17.761981 15765 raft_consensus.cc:697] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 13 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:17.831810 10916 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:17.809520 16052 raft_consensus.cc:2749] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:17.836951 10767 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:17.791191 16157 raft_consensus.cc:2749] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:17.836171 16014 raft_consensus.cc:2749] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:18:17.836395 15744 raft_consensus.cc:2749] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:18:17.838531 15964 raft_consensus.cc:2749] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:17.841511 15854 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:17.828639 16047 raft_consensus.cc:2749] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:17.829818 16141 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.783241 16040 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:17.790407 16067 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:17.847251 10916 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:17.865746 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:17.846097 16025 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:17.859472 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 74190.1us
I20260709 13:18:17.866510 10841 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:17.795717 15997 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:17.857504 16148 log.cc:927] Time spent T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
W20260709 13:18:17.872823 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:17.849571 15859 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:17.873121 10841 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:17.841094 16158 consensus_queue.cc:1048] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:18:17.868373 16064 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:17.911811 11512 raft_consensus.cc:1275] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:18:17.874894 15859 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:17.874800 16014 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:17.921307 10841 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:17.862845 16141 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:17.882606 15907 consensus_meta.cc:287] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.914s	user 0.013s	sys 0.000s
W20260709 13:18:17.882586 16153 log.cc:927] Time spent T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:18:17.975592 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:18:17.974628 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 53455.2us
I20260709 13:18:17.980494 11510 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 11.
I20260709 13:18:17.980780 10841 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:17.884053 16157 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:17.909790 16158 consensus_queue.cc:1243] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer 468a8fd46e824f62b9c2c0d3532a6d26 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:18:17.973105 11478 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
W20260709 13:18:18.000636 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 163768us
I20260709 13:18:18.010768 10767 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:17.926543 15883 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:17.974871 16014 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:18.020478 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 93377.5us
I20260709 13:18:17.975423 15754 raft_consensus.cc:2749] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:18.024016 10767 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:17.883527 15854 raft_consensus.cc:2804] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:18.025293 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:17.978966 15859 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:18:18.029647 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:17.980316 16047 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:18.032160 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:17.927181 16111 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.909572 15765 consensus_queue.cc:237] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:17.911737 16064 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:18.045877 16172 consensus_queue.cc:1048] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:18.018011 15798 raft_consensus.cc:2749] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:18:18.053936 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 73253.3us
I20260709 13:18:18.055449 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:18:18.058539 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 233309us
I20260709 13:18:18.077142 11507 raft_consensus.cc:2376] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 14. Re-sending same reply.
I20260709 13:18:18.026993 16026 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:18.022907 16014 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:17.989851 15907 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:18.041287 16157 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:18.046425 16117 raft_consensus.cc:2749] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:18.027408 16145 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:18:18.055748 16151 log.cc:927] Time spent T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.308s	user 0.000s	sys 0.001s
I20260709 13:18:18.078804 10841 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:18.070256 16111 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:18.068250 15854 raft_consensus.cc:697] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:18.116770 10841 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:18.118089 10841 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:18.119472 16026 raft_consensus.cc:2749] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:18.120147 16026 raft_consensus.cc:2749] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:18:18.120718 16026 raft_consensus.cc:2749] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:18.121623 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:18.122989 10872 pending_rounds.cc:85] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 5
I20260709 13:18:18.123114 10873 raft_consensus.cc:1240] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 15. Current term is 16. Ops: [15.7-15.7]
I20260709 13:18:18.123602 10872 pending_rounds.cc:107] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:18:18.123852 10872 raft_consensus.cc:2889] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:18:18.124169 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 43576.1us
I20260709 13:18:18.125371 10868 raft_consensus.cc:1275] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:18:18.126614 11478 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:18.126533 10841 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:18.128212 16133 raft_consensus.cc:2804] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:18.128386 11478 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:18:18.129094 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:18.124378 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:18.128193 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:18.140072 10870 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 16: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:18.079146 16167 consensus_queue.cc:1048] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:18:18.049909 16047 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:18.104547 16036 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:18.101038 15870 consensus_queue.cc:1048] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:18:18.129806 16145 raft_consensus.cc:697] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 16 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:18.163398 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:18.109423 16016 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:18.135934 16133 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:18.105002 15883 consensus_queue.cc:1059] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:18:18.226863 16133 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:18.074740 16064 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:18.202539 11504 raft_consensus.cc:1275] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:18:18.203733 15826 consensus_queue.cc:1048] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.124s
I20260709 13:18:18.203923 15744 consensus_queue.cc:1048] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:18.205336 16182 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:18.206493 16166 log.cc:927] Time spent T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.194s	user 0.000s	sys 0.001s
I20260709 13:18:18.206861 16025 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:18.207254 16026 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:18.210965 16141 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:18:18.213441 15928 consensus_meta.cc:287] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.698s	user 0.009s	sys 0.001s
I20260709 13:18:18.214103 15854 consensus_queue.cc:237] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:18.156282 16036 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:18.215080 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:18.216403 15907 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:18.220065 15997 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:18.221225 16040 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:18.154891 16001 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:18.224050 16047 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:18.272200 16025 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:18.268537 16145 consensus_queue.cc:237] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:18.161525 15798 raft_consensus.cc:2749] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:18:18.351907 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 56826.9us
I20260709 13:18:18.156350 16175 consensus_queue.cc:1059] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.100s
W20260709 13:18:18.279723 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 151449us
I20260709 13:18:18.270264 16182 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:18.282379 10947 raft_consensus.cc:1275] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
W20260709 13:18:18.283586 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 157782us
I20260709 13:18:18.240237 16172 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:18.274375 16156 consensus_queue.cc:1059] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 13:18:18.292335 16179 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:18.293673 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:18.370810 10945 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
I20260709 13:18:18.303365 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:18.236644 16117 consensus_queue.cc:1048] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
W20260709 13:18:18.297006 16169 log.cc:927] Time spent T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.279s	user 0.001s	sys 0.000s
I20260709 13:18:18.346613 16047 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:18.111024 15754 consensus_queue.cc:1048] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
W20260709 13:18:18.350989 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:18.229924 16160 log.cc:927] Time spent T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.352s	user 0.000s	sys 0.001s
I20260709 13:18:18.339983 15927 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:18.319811 16001 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:18.371552 11504 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:18.240628 16014 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:18.344512 16040 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:18.306890 15928 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:18.342357 16157 raft_consensus.cc:697] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:18.142917 16174 consensus_queue.cc:1059] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.084s
I20260709 13:18:18.382644 10766 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:18.345980 16188 consensus_queue.cc:1059] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
W20260709 13:18:18.383327 15801 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.006s	sys 0.015s
I20260709 13:18:18.316915 16026 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:18.384764 10916 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.002s	sys 0.009s
I20260709 13:18:18.356930 15744 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:18.344599 16016 consensus_queue.cc:1048] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:18:18.390834 16047 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:18.381125 16179 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:18.399955 10944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2573) took 1248 ms. Trace:
W20260709 13:18:18.400763 16171 log.cc:927] Time spent T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 13:18:18.404605 16064 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:18.407507 16173 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:18.381122 15870 raft_consensus.cc:3055] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:18:18.423480 10916 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.002s	sys 0.009s
I20260709 13:18:18.415872 16189 raft_consensus.cc:3055] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:18.454425 10874 raft_consensus.cc:2393] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 14: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:18:18.455376 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:18.421418 15801 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.010s	sys 0.015s
I20260709 13:18:18.422479 16187 raft_consensus.cc:2804] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:18:18.484171 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 619935us
I20260709 13:18:18.397164 16001 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:18.485227 16032 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:18.485697 16180 log.cc:927] Time spent T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.205s	user 0.001s	sys 0.000s
I20260709 13:18:18.493359 15870 raft_consensus.cc:740] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:18:18.495077 16181 log.cc:927] Time spent T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.209s	user 0.000s	sys 0.001s
W20260709 13:18:18.496718 16146 log.cc:927] Time spent T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.701s	user 0.002s	sys 0.000s
I20260709 13:18:18.374488 16172 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:18.499637 16025 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:18.408808 15927 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:18.535383 15870 consensus_queue.cc:260] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:18.536568 15927 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:18.503551 11478 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:18.537719 15859 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:18.538429 11478 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:18.538995 15816 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:18.539577 15816 raft_consensus.cc:697] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 15 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:18.540488 15816 consensus_queue.cc:237] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:18.475704 15997 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:18.527827 16032 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:18.453294 15744 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:18.543094 15997 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:18.543365 16032 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:18.544909 15744 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:18.545233 15997 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:18.545702 16032 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:18.396827 16175 consensus_queue.cc:1048] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 13:18:18.509408 11504 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
I20260709 13:18:18.511128 10916 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:18.506320 16188 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:18.414959 16040 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:18.405663 16133 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:18.508445 16189 raft_consensus.cc:740] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:18:18.554471 16039 consensus_meta.cc:287] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.809s	user 0.008s	sys 0.004s
I20260709 13:18:18.453950 16111 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:18.547335 10916 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:18.577670 10916 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:18:18.392318 15861 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.814s	user 0.008s	sys 0.021s
I20260709 13:18:18.573482 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:18.546423 16168 consensus_queue.cc:1048] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:18:18.479197 10944 rpcz_store.cc:276] 0709 13:18:17.151380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.516818 (+365438us) service_pool.cc:224] Handling call
0709 13:18:17.517140 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:17.537760 (+ 20620us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:17.537804 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:17.563095 (+ 25291us) log.cc:844] Serialized 26 byte log entry
0709 13:18:17.666545 (+103450us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:17.672396 (+  5851us) log.cc:844] Serialized 14 byte log entry
0709 13:18:17.673033 (+   637us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.673132 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:18.399736 (+726604us) raft_consensus.cc:1685] finished
0709 13:18:18.399758 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:18.399893 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20170,"prepare.queue_time_us":73957,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":2837,"raft.queue_time_us":42950,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":226,"spinlock_wait_cycles":1104640,"thread_start_us":103057,"threads_started":1,"wal-append.queue_time_us":281550}
I20260709 13:18:18.543133 16123 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:18.557839 16025 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:18.579311 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:18.559527 15870 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:18.547269 16200 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:18.512836 16173 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:18.653869 11510 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
I20260709 13:18:18.564627 16036 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:18:18.622296 15861 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.044s	user 0.008s	sys 0.021s
I20260709 13:18:18.727902 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:18:18.715601 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 56643.5us
I20260709 13:18:18.717761 11511 raft_consensus.cc:1240] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:18.580734 16039 raft_consensus.cc:2749] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 15. Current term is 16.
I20260709 13:18:18.734462 11505 raft_consensus.cc:2410] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
W20260709 13:18:18.735157 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 157578us
W20260709 13:18:18.719310 16171 log.cc:927] Time spent T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
I20260709 13:18:18.740276 10841 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:18.743283 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:18.708684 15798 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:18.562074 16188 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:18.718441 16163 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:18.785557 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:18.791290 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:18.743438 16036 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:18.734615 16123 raft_consensus.cc:697] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 17 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:18.736496 16193 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:18.801579 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:18.813580 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:18.811794 10799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2076) took 1156 ms. Trace:
I20260709 13:18:18.576239 16157 consensus_queue.cc:237] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:18.722874 16025 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:18.737629 15801 consensus_queue.cc:1048] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 13:18:18.781729 16187 raft_consensus.cc:697] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:18.816677 10799 rpcz_store.cc:276] 0709 13:18:17.655194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.655361 (+   167us) service_pool.cc:224] Handling call
0709 13:18:17.665912 (+ 10551us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:17.671706 (+  5794us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:17.671739 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:17.749052 (+ 77313us) log.cc:844] Serialized 26 byte log entry
0709 13:18:17.899343 (+150291us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:18:17.918371 (+ 19028us) log.cc:844] Serialized 14 byte log entry
0709 13:18:17.978043 (+ 59672us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.978148 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:18.811386 (+833238us) raft_consensus.cc:1685] finished
0709 13:18:18.811422 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:18.811694 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":70490,"prepare.queue_time_us":81454,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":69669,"raft.run_cpu_time_us":620,"raft.run_wall_time_us":17835,"thread_start_us":73577,"threads_started":1,"wal-append.queue_time_us":316410}
I20260709 13:18:18.844061 16214 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:18.845081 16214 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:18.847478 16203 log.cc:927] Time spent T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.264s	user 0.001s	sys 0.000s
W20260709 13:18:18.849663 16202 log.cc:927] Time spent T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
W20260709 13:18:18.851866 16196 log.cc:927] Time spent T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.184s	user 0.000s	sys 0.000s
I20260709 13:18:18.805168 16014 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:18.804118 16182 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:18.751968 15798 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:18.821008 16129 raft_consensus.cc:3055] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:18.730485 16200 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:18.812366 16036 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:18.946246 16036 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:18.888199 16214 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:18.952026 16200 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:18.889243 16067 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:18.953202 16187 consensus_queue.cc:237] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:18.787873 16163 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:18.956833 16163 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:18.824697 16206 raft_consensus.cc:2804] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:18.957818 16206 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:18.958287 16206 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:18.961090 10695 catalog_manager.cc:5697] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 11 to 14, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 14 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:18:18.880496 15859 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:18.875046 15853 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:18.964118 15853 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:18.967695 16184 log.cc:927] Time spent T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.699s	user 0.000s	sys 0.004s
I20260709 13:18:18.950441 11512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56410 (request call id 832) took 1074 ms. Trace:
I20260709 13:18:18.972635 11512 rpcz_store.cc:276] 0709 13:18:17.876004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.876159 (+   155us) service_pool.cc:224] Handling call
0709 13:18:17.876482 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:17.912223 (+ 35741us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:18.950023 (+1037800us) spinlock_profiling.cc:243] Waited 33.3 ms on lock 0x7b54001d06d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:18.950308 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":76591360}
I20260709 13:18:18.891278 16161 raft_consensus.cc:3055] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:18.974673 16161 raft_consensus.cc:740] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:18.976166 16161 consensus_queue.cc:260] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:18.978096 10874 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 2085) took 1331 ms (client timeout 1515 ms). Trace:
W20260709 13:18:18.892889 16208 log.cc:927] Time spent T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 13:18:18.980437 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:18.984091 10792 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:18:18.978690 10874 rpcz_store.cc:269] 0709 13:18:17.646823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.647465 (+   642us) service_pool.cc:224] Handling call
0709 13:18:18.977792 (+1330327us) spinlock_profiling.cc:243] Waited 767 ms on lock 0x7b54000f69d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:18.977985 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1763063552}
I20260709 13:18:18.789687 16039 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:18.907596 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:18.910444 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:18.911561 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:18.912472 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:18.913159 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2199) took 1695 ms. Trace:
I20260709 13:18:18.913451 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:18.916688 16156 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:18.990273 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:18:18.990715 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 50426us
W20260709 13:18:18.920554 16178 log.cc:927] Time spent T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.714s	user 0.000s	sys 0.002s
I20260709 13:18:18.921891 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:18.989240 10867 rpcz_store.cc:276] 0709 13:18:17.217768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.217900 (+   132us) service_pool.cc:224] Handling call
0709 13:18:17.238451 (+ 20551us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:17.540822 (+302371us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:18.912820 (+1371998us) spinlock_profiling.cc:243] Waited 274 ms on lock 0x7b540009a1d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:18.913083 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":630759168}
I20260709 13:18:19.012264 10797 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:18.922753 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:18.937552 16133 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:18.938867 16179 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:18.978477 16161 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:18.941205 16173 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:18.870479 16026 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:18.944586 16123 consensus_queue.cc:237] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:18.869369 15928 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:18.993222 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:19.003485 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:18.992766 16039 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:19.004916 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:19.066027 10794 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
I20260709 13:18:18.963411 10695 catalog_manager.cc:5697] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:18:19.006850 16188 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:19.013411 10793 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
I20260709 13:18:18.939628 16172 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:19.043526 11503 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 13.
I20260709 13:18:18.901108 16047 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:19.043802 16200 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:19.044693 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:18.902623 16129 raft_consensus.cc:740] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:19.044960 15907 raft_consensus.cc:2749] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:19.044329 15859 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:19.071514 10766 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:19.058040 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 402590us
I20260709 13:18:19.063086 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:18.825657 16193 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:19.134199 11508 raft_consensus.cc:1275] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:18:19.135360 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:19.136468 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:19.137497 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:19.139461 11506 raft_consensus.cc:1240] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:19.064163 16197 consensus_queue.cc:1059] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.092s
I20260709 13:18:19.027558 16156 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:19.068738 16039 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:19.013278 15853 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:19.158327 10801 raft_consensus.cc:2393] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 13: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:19.163666 10916 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:18:19.047295 10767 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.054s	sys 0.036s
I20260709 13:18:19.070865 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:19.202996 10767 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.055s	sys 0.036s
I20260709 13:18:19.057500 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:19.182224 11505 raft_consensus.cc:1275] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:18:19.047675 16001 consensus_meta.cc:287] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.525s	user 0.008s	sys 0.003s
I20260709 13:18:19.207358 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:19.201376 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:19.212991 10794 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 17.
I20260709 13:18:19.204579 11512 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
W20260709 13:18:19.214382 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 50787.8us
W20260709 13:18:19.214526 10699 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42956 (request call id 260) took 33568 ms (client timeout 15000 ms). Trace:
W20260709 13:18:19.224435 16199 log.cc:927] Time spent T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 13:18:19.224315 10695 catalog_manager.cc:5697] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:19.225504 16147 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:19.226099 16147 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:19.227180 16194 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:19.227687 16194 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:19.228348 16147 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:19.229892 16194 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:19.231380 10872 raft_consensus.cc:1240] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:19.201169 16188 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:19.232270 16188 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:19.232918 16031 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:19.233408 16031 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:19.234376 16188 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:19.159307 16064 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:19.235368 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:19.235606 10874 pending_rounds.cc:85] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 11
I20260709 13:18:19.235951 10874 pending_rounds.cc:107] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:18:19.236171 10874 raft_consensus.cc:2889] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:19.237149 10916 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:19.181211 16047 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:19.186882 16133 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:19.253505 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 182128us
I20260709 13:18:19.254534 10766 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:19.224117 16052 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:19.258809 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:19.249908 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:19.260491 16031 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:19.253062 10870 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.688s	user 0.015s	sys 0.020s
I20260709 13:18:19.197515 16210 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:19.256131 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:19.257939 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:19.261714 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:19.259483 10872 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 17.
W20260709 13:18:19.250923 10914 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.014s	sys 0.023s
W20260709 13:18:19.268280 10870 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.704s	user 0.015s	sys 0.020s
W20260709 13:18:19.248037 10948 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.701s	user 0.004s	sys 0.029s
W20260709 13:18:19.275041 10948 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.728s	user 0.005s	sys 0.029s
I20260709 13:18:19.185577 16200 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:19.270015 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:19.203938 15859 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:19.275440 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:19.274803 10914 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.014s	sys 0.023s
I20260709 13:18:19.276077 10870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2214) took 1026 ms. Trace:
I20260709 13:18:19.288532 10870 rpcz_store.cc:276] 0709 13:18:18.249673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:18.285420 (+ 35747us) service_pool.cc:224] Handling call
0709 13:18:18.304172 (+ 18752us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:18.304438 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:18:18.564410 (+259972us) log.cc:844] Serialized 14 byte log entry
0709 13:18:19.275717 (+711307us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:19.275770 (+    53us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:19.275829 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:19.275901 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:19.276018 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":193549,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":30,"raft.queue_time_us":771,"thread_start_us":969811,"threads_started":2}
I20260709 13:18:19.272387 10766 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:19.299513 10800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2644) took 1632 ms. Trace:
I20260709 13:18:19.299994 10800 rpcz_store.cc:276] 0709 13:18:17.667112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.667702 (+   590us) service_pool.cc:224] Handling call
0709 13:18:17.668112 (+   410us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:18:17.864754 (+196642us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:17.864874 (+   120us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:18:17.903781 (+ 38907us) log.cc:844] Serialized 78 byte log entry
0709 13:18:18.119319 (+215538us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:18.119371 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:18.119444 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:18.919647 (+800203us) spinlock_profiling.cc:243] Waited 26.9 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:18:19.284941 (+365294us) raft_consensus.cc:1685] finished
0709 13:18:19.284966 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:19.299425 (+ 14459us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10575,"raft.queue_time_us":160245,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":12641,"spinlock_wait_cycles":61825280,"thread_start_us":342315,"threads_started":2,"wal-append.queue_time_us":236110}
I20260709 13:18:19.301986 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:19.302489 10948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 2273) took 1029 ms. Trace:
I20260709 13:18:19.303143 16211 raft_consensus.cc:2749] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:19.286381 10871 raft_consensus.cc:1240] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:19.306951 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:19.302891 10948 rpcz_store.cc:276] 0709 13:18:18.273140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:18.273285 (+   145us) service_pool.cc:224] Handling call
0709 13:18:18.294946 (+ 21661us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:18.295142 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:18:18.530034 (+234892us) log.cc:844] Serialized 14 byte log entry
0709 13:18:19.302133 (+772099us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:19.302192 (+    59us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:19.302249 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:19.302315 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:19.302436 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1977,"prepare.queue_time_us":78709,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":4375,"thread_start_us":960346,"threads_started":2}
I20260709 13:18:19.307324 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:18:19.309127 10873 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.500s	user 0.015s	sys 0.006s
I20260709 13:18:19.309377 10800 raft_consensus.cc:2376] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 14. Re-sending same reply.
I20260709 13:18:19.309653 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:19.310446 10794 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 16.
I20260709 13:18:19.310811 10800 raft_consensus.cc:1275] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:18:19.236356 16001 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:19.270351 16040 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:19.313719 10872 raft_consensus.cc:1240] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:19.315613 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:19.267093 16052 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:19.277009 16037 raft_consensus.cc:2749] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:19.275372 16067 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:19.289670 16150 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:19.288398 16197 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:18:19.307869 16104 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.103s	user 0.007s	sys 0.024s
I20260709 13:18:19.326962 16185 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:19.327454 16219 raft_consensus.cc:2804] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:18:19.328534 16215 log.cc:927] Time spent T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:18:19.330144 16200 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:19.330874 16040 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:18:19.224211 10699 rpcz_store.cc:269] 0709 13:17:45.456378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:45.459316 (+  2938us) service_pool.cc:224] Handling call
0709 13:17:47.301367 (+1842051us) spinlock_profiling.cc:243] Waited 37.6 ms on lock 0x7b54002b0110. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd910b2 00007f9c86d0aa89 00007f9c86ca2da6 00007f9c86c74088 00007f9c86d7237e 00007f9c852332a4 00007f9c85233232 00007f9c85233161 00007f9c852330dc
0709 13:18:19.016189 (+31714822us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:11.181872 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:11.408034 (+226162us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:11.421466 (+ 13432us) write_op.cc:435] Acquired schema lock
0709 13:18:11.421485 (+    19us) tablet.cc:662] Decoding operations
0709 13:18:11.512915 (+ 91430us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:11.513027 (+   112us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:11.513065 (+    38us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:18:11.514024 (+   959us) tablet.cc:701] Row locks acquired
0709 13:18:11.514075 (+    51us) write_op.cc:260] PREPARE: finished
0709 13:18:11.536319 (+ 22244us) write_op.cc:270] Start()
0709 13:18:11.593341 (+ 57022us) write_op.cc:276] Timestamp: P: 1783603091520712 usec, L: 0
0709 13:18:11.593383 (+    42us) op_driver.cc:348] REPLICATION: starting
0709 13:18:11.640198 (+ 46815us) log.cc:844] Serialized 4014 byte log entry
0709 13:18:11.989111 (+348913us) op_driver.cc:464] REPLICATION: finished
0709 13:18:12.397440 (+408329us) write_op.cc:301] APPLY: starting
0709 13:18:12.427647 (+ 30207us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:12.456436 (+ 28789us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:12.456536 (+   100us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:12.936787 (+480251us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:12.937065 (+   278us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=20,mrs_lookups=0
0709 13:18:12.937118 (+    53us) write_op.cc:312] APPLY: finished
0709 13:18:12.937426 (+   308us) log.cc:844] Serialized 97 byte log entry
0709 13:18:12.937643 (+   217us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:12.937794 (+   151us) write_op.cc:454] Released schema lock
0709 13:18:12.937842 (+    48us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:13.047932 (+110090us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:13.051839 (+  3907us) write_op.cc:454] Released schema lock
0709 13:18:13.653723 (+601884us) spinlock_profiling.cc:243] Waited 8.23 ms on lock 0x7b180028dfe0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc8522 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
Metrics: {"mutex_wait_us":3428956,"reported_tablets":48,"spinlock_wait_cycles":86443904,"child_traces":[["op",{"apply.queue_time_us":352301,"apply.run_cpu_time_us":16958,"apply.run_wall_time_us":653569,"cfile_cache_hit":42,"cfile_cache_hit_bytes":45947,"num_ops":10,"prepare.queue_time_us":767326,"prepare.run_cpu_time_us":17205,"prepare.run_wall_time_us":542312,"raft.queue_time_us":168328,"raft.run_cpu_time_us":53831,"raft.run_wall_time_us":313796,"replication_time_us":371947,"spinlock_wait_cycles":18930048,"thread_start_us":1067248,"threads_started":4,"wal-append.queue_time_us":823907,"wal-append.run_cpu_time_us":41268,"wal-append.run_wall_time_us":2148545}]]}
I20260709 13:18:19.332582 16052 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:19.285557 11477 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.017s	sys 0.030s
W20260709 13:18:19.388787 11477 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.018s	sys 0.030s
W20260709 13:18:19.389238 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 639248us
W20260709 13:18:19.352761 15883 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.080s	user 0.004s	sys 0.022s
W20260709 13:18:19.399039 15883 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.152s	user 0.004s	sys 0.022s
I20260709 13:18:19.400329 15859 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:19.263967 16133 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:19.402863 16206 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:19.403426 16133 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:19.268826 16047 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:19.404932 16206 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:19.405407 16036 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:19.406052 16047 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:19.338686 11478 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:19.341022 16067 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:19.407583 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 68965.4us
I20260709 13:18:19.408780 16067 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:19.409603 11478 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:19.410136 10792 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 14.
I20260709 13:18:19.284578 16210 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:19.340380 16214 raft_consensus.cc:2749] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:19.306351 16025 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:18:19.355458 10913 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.018s	sys 0.031s
W20260709 13:18:19.411394 10913 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.018s	sys 0.031s
I20260709 13:18:19.385581 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:19.363240 16032 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:19.386641 10870 raft_consensus.cc:1240] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:19.390302 11504 raft_consensus.cc:1240] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 16 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:18:19.337450 11509 raft_consensus.cc:1240] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:19.322764 16209 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:19.226555 10695 catalog_manager.cc:5697] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 7 to 11, leader changed from <none> to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 11 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:18:19.392975 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:19.344990 16064 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:19.356365 16219 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:19.324409 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:19.385660 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:19.345927 16150 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:19.363617 16104 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.144s	user 0.007s	sys 0.024s
I20260709 13:18:19.263804 16129 consensus_queue.cc:260] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:19.418349 10871 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:19.393729 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:19.419637 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:19.445046 10792 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 13.
I20260709 13:18:19.420560 16209 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:19.400024 16040 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:19.423619 10797 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 16.
I20260709 13:18:19.423679 11478 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:19.429373 10794 raft_consensus.cc:2376] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 13. Re-sending same reply.
I20260709 13:18:19.429388 10800 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 17.
I20260709 13:18:19.433864 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:19.411978 16210 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:19.421017 10695 catalog_manager.cc:5697] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 12 to 13, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 13 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:19.457698 10872 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 14.
I20260709 13:18:19.418632 16141 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:19.467481 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:19.479384 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:19.434206 16219 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:19.442950 16145 raft_consensus.cc:2749] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:19.505748 11511 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:19.506345 10797 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 14.
I20260709 13:18:19.350567 16197 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:19.538924 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:19.462590 16031 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:19.360037 16185 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:19.469244 15888 raft_consensus.cc:3055] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:19.546761 10915 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:19.543396 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:19.543190 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:19.447213 16162 consensus_queue.cc:1059] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.101s
I20260709 13:18:19.433590 16064 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:19.560108 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:19.540141 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:19.568275 10872 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 15.
I20260709 13:18:19.563817 10871 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
I20260709 13:18:19.343600 16189 consensus_queue.cc:260] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:19.587077 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:19.578254 10797 raft_consensus.cc:1240] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:18:19.428362 16032 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:19.559864 16147 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:19.592106 10869 raft_consensus.cc:1240] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:19.558871 15888 raft_consensus.cc:740] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:19.591176 16210 raft_consensus.cc:2749] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:18:19.631851 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 394760us
I20260709 13:18:19.592006 16036 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:19.592338 16156 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:19.611289 16223 log.cc:927] Time spent T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
I20260709 13:18:19.595755 16147 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:19.605634 10915 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:19.648638 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:18:19.651568 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 227962us
I20260709 13:18:19.645951 10916 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:19.652526 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:19.658035 10946 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
I20260709 13:18:19.661164 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:19.661839 10941 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:19.634121 16162 raft_consensus.cc:3055] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:19.666716 10946 raft_consensus.cc:1275] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
W20260709 13:18:19.642438 16154 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.839s	user 0.017s	sys 0.019s
I20260709 13:18:19.662797 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:19.669060 10916 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:19.634778 16014 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:19.609562 15859 consensus_queue.cc:1059] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 13:18:19.665211 11478 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:19.607543 16047 consensus_queue.cc:1048] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 13:18:19.647955 16147 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:18:19.641542 15854 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.018s	sys 0.011s
I20260709 13:18:19.670415 10916 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:19.672466 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 65908.6us
I20260709 13:18:19.685151 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:18:19.668481 16154 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.089s	user 0.018s	sys 0.020s
I20260709 13:18:19.686533 10944 raft_consensus.cc:1275] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:18:19.661378 16001 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:19.747455 10876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2215) took 1081 ms. Trace:
I20260709 13:18:19.767159 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:19.746141 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:18:19.688687 15854 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.679s	user 0.018s	sys 0.012s
I20260709 13:18:19.672173 16036 raft_consensus.cc:2749] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:19.768535 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:19.727221 16123 consensus_queue.cc:1059] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 13:18:19.851334 10947 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:19.671568 16210 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:19.666922 16162 raft_consensus.cc:740] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:19.725967 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:19.772070 10695 catalog_manager.cc:5697] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
W20260709 13:18:19.848064 16238 log.cc:927] Time spent T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.191s	user 0.005s	sys 0.000s
I20260709 13:18:19.771580 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:18:19.751740 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 94998.7us
I20260709 13:18:19.734467 16197 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:19.717381 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:19.747596 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:18:19.743360 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 73006.2us
I20260709 13:18:19.746063 16206 raft_consensus.cc:2749] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:19.765959 16150 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:19.764146 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:19.684242 16014 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:19.870635 11478 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:19.745958 16224 raft_consensus.cc:3055] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:19.700752 16141 raft_consensus.cc:697] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 14 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:19.851046 11512 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 13.
I20260709 13:18:19.717212 10946 raft_consensus.cc:1240] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:19.853878 11508 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 16.
I20260709 13:18:19.859128 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:19.864177 11511 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
W20260709 13:18:19.846338 16235 log.cc:927] Time spent T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.199s	user 0.000s	sys 0.007s
I20260709 13:18:19.876533 11508 raft_consensus.cc:1275] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 13:18:19.877025 11508 pending_rounds.cc:85] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586: Aborting all ops after (but not including) 10
I20260709 13:18:19.877305 11508 pending_rounds.cc:107] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:18:19.867048 10943 raft_consensus.cc:2410] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:18:19.877592 11508 raft_consensus.cc:2889] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:19.870311 10948 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:19.850390 10876 rpcz_store.cc:276] 0709 13:18:18.665869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:18.666474 (+   605us) service_pool.cc:224] Handling call
0709 13:18:18.666856 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:18.708814 (+ 41958us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:18.708846 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:18.729034 (+ 20188us) log.cc:844] Serialized 26 byte log entry
0709 13:18:19.251779 (+522745us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:19.251835 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:19.251927 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:19.747215 (+495288us) raft_consensus.cc:1685] finished
0709 13:18:19.747237 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:19.747391 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4157,"raft.queue_time_us":6456,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":6787,"thread_start_us":457904,"threads_started":1,"wal-append.queue_time_us":667873}
I20260709 13:18:19.873081 16224 raft_consensus.cc:740] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:19.880236 16224 consensus_queue.cc:260] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:19.874258 16141 consensus_queue.cc:237] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:19.875612 16014 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:19.850135 16247 log.cc:927] Time spent T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.063s	user 0.001s	sys 0.003s
I20260709 13:18:19.874284 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:19.872177 10916 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:19.885546 10948 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 14.
I20260709 13:18:19.681953 16236 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:19.847223 16040 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:19.895937 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:19.900954 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:19.903132 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:19.904831 10950 raft_consensus.cc:2410] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:18:19.849298 16209 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:19.910094 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:19.913327 10947 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 16.
I20260709 13:18:19.913625 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:19.860018 16200 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:19.914826 10798 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
I20260709 13:18:19.937222 10944 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
I20260709 13:18:19.937397 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:19.937707 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:19.938237 10943 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:19.893378 16047 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:19.897583 16206 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:19.907692 16210 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:19.955524 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:19.685052 15888 consensus_queue.cc:260] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:19.907302 16064 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:20.013592 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:20.005041 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:20.003398 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:19.997960 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:19.965734 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:20.047966 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:20.060735 10798 raft_consensus.cc:2393] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 14: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:19.916280 16150 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:18:20.071054 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 174454us
I20260709 13:18:20.073920 10800 raft_consensus.cc:1275] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:18:20.010051 15870 consensus_meta.cc:287] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 1.283s	user 0.011s	sys 0.001s
W20260709 13:18:20.010483 16219 consensus_meta.cc:287] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.508s	user 0.010s	sys 0.000s
W20260709 13:18:19.997609 16243 log.cc:927] Time spent T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.177s	user 0.005s	sys 0.000s
W20260709 13:18:19.998370 16249 log.cc:927] Time spent T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
I20260709 13:18:20.092047 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:18:19.997812 16226 log.cc:927] Time spent T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.337s	user 0.002s	sys 0.003s
W20260709 13:18:19.999724 16233 log.cc:927] Time spent T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.260s	user 0.000s	sys 0.006s
W20260709 13:18:19.999215 16234 log.cc:927] Time spent T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.261s	user 0.000s	sys 0.005s
W20260709 13:18:20.003913 16246 log.cc:927] Time spent T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
W20260709 13:18:19.998049 16245 log.cc:927] Time spent T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.103s	user 0.000s	sys 0.006s
I20260709 13:18:20.092073 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:19.996928 16232 log.cc:927] Time spent T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.254s	user 0.003s	sys 0.003s
W20260709 13:18:19.995589 16230 log.cc:927] Time spent T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.228s	user 0.000s	sys 0.005s
I20260709 13:18:20.145972 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 2088) took 1409 ms. Trace:
I20260709 13:18:20.146420 10873 rpcz_store.cc:276] 0709 13:18:18.736417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:18.736896 (+   479us) service_pool.cc:224] Handling call
0709 13:18:18.737234 (+   338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:18.737446 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:18.737482 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:18.743991 (+  6509us) log.cc:844] Serialized 26 byte log entry
0709 13:18:19.309579 (+565588us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:19.309625 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:19.309690 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:20.145725 (+836035us) raft_consensus.cc:1685] finished
0709 13:18:20.145747 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:20.145904 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":500813,"threads_started":1,"wal-append.queue_time_us":681033}
W20260709 13:18:19.993863 16251 log.cc:927] Time spent T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
I20260709 13:18:19.963939 16210 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:20.148520 16210 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:19.995147 16170 raft_consensus.cc:2804] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:20.102366 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:20.153002 16170 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:20.153478 16170 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:18:20.046325 16248 consensus_queue.cc:1175] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" responder_term: 14 status { last_received { term: 13 index: 10 } last_committed_idx: 9 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 13. Current term is 14. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:18:20.159087 16220 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:20.159909 16220 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:20.106413 15798 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:20.100912 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:20.181566 10798 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:18:20.117380 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 174051us
I20260709 13:18:20.118800 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:20.069377 16150 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:19.952891 16206 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:20.106187 16238 log.cc:927] Time spent T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 13:18:19.952986 16047 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:20.154666 10795 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 12.
W20260709 13:18:20.126509 16039 consensus_meta.cc:287] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.884s	user 0.013s	sys 0.000s
I20260709 13:18:20.184440 10842 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:20.210573 10842 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:19.995695 16032 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:20.212587 16224 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:20.213440 16179 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:20.213903 16179 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:20.214299 16179 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:20.180503 16198 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:20.215191 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:20.215461 16198 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:20.215883 10873 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
I20260709 13:18:20.217720 16198 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:20.218187 16216 log.cc:927] Time spent T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.983s	user 0.004s	sys 0.000s
I20260709 13:18:20.218329 16198 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:20.189404 16104 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:20.218339 16039 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:20.187296 16014 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:20.117481 16219 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:20.219209 16104 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:20.219592 16014 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:20.221171 16219 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:20.226634 16104 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:20.209923 16259 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:20.227578 16259 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:20.229990 16259 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:20.231801 15986 raft_consensus.cc:3055] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:20.232230 15986 raft_consensus.cc:740] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:20.233019 15986 consensus_queue.cc:260] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:20.233986 15986 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:20.235719 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 814) took 1183 ms. Trace:
I20260709 13:18:20.190209 16150 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:20.219188 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:20.186233 10941 consensus_meta.cc:287] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.524s	user 0.007s	sys 0.001s
I20260709 13:18:20.256397 10941 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
I20260709 13:18:20.257529 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:20.236114 10867 rpcz_store.cc:276] 0709 13:18:19.052054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:19.063078 (+ 11024us) service_pool.cc:224] Handling call
0709 13:18:19.160119 (+ 97041us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:19.195593 (+ 35474us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:19.195640 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:19.196407 (+   767us) log.cc:844] Serialized 52 byte log entry
0709 13:18:19.468042 (+271635us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:18:19.587884 (+119842us) log.cc:844] Serialized 14 byte log entry
0709 13:18:19.588305 (+   421us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:19.588442 (+   137us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:19.770052 (+181610us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b180043d1a0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7dd962a6 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc83f3 00007f9c7ddca7c1
0709 13:18:20.235410 (+465358us) raft_consensus.cc:1685] finished
0709 13:18:20.235503 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:20.235662 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75821,"prepare.queue_time_us":119630,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":1619,"raft.queue_time_us":14267,"raft.run_cpu_time_us":240,"raft.run_wall_time_us":13440,"spinlock_wait_cycles":27726592,"thread_start_us":208108,"threads_started":1,"wal-append.queue_time_us":351128}
I20260709 13:18:19.958037 16200 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:19.962568 16162 consensus_queue.cc:260] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:20.007712 10695 catalog_manager.cc:5697] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:18:20.246650 10943 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
I20260709 13:18:20.263689 10793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2655) took 1325 ms. Trace:
I20260709 13:18:20.264299 10945 raft_consensus.cc:1275] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:18:20.209743 16161 consensus_meta.cc:287] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 1.151s	user 0.009s	sys 0.008s
I20260709 13:18:20.200621 16239 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:20.270535 10875 raft_consensus.cc:1240] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 15. Current term is 16. Ops: [15.7-15.7]
I20260709 13:18:20.218727 16198 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:20.272764 10874 raft_consensus.cc:1240] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:18:20.268576 10793 rpcz_store.cc:276] 0709 13:18:18.938525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:19.284563 (+346038us) service_pool.cc:224] Handling call
0709 13:18:19.312673 (+ 28110us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:19.388850 (+ 76177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:19.388881 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:19.389340 (+   459us) log.cc:844] Serialized 26 byte log entry
0709 13:18:19.651190 (+261850us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:19.651253 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:19.789066 (+137813us) spinlock_profiling.cc:243] Waited 35.3 ms on lock 0x7b54000d7354. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:19.789132 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:20.235872 (+446740us) raft_consensus.cc:1685] finished
0709 13:18:20.235892 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:20.263600 (+ 27708us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40479,"raft.queue_time_us":40610,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":10448,"spinlock_wait_cycles":81117440,"thread_start_us":184908,"threads_started":1,"wal-append.queue_time_us":355406}
I20260709 13:18:20.273586 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:20.187976 16220 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:20.274883 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:20.273737 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:20.280372 10868 raft_consensus.cc:2393] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 10: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:20.316927 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:20.281067 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:20.236157 15870 raft_consensus.cc:3055] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:20.283157 11507 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 13.
I20260709 13:18:20.294930 16198 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:20.295205 16014 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:20.297425 16039 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:20.299767 16245 log.cc:927] Time spent T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:18:20.300190 11505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56416 (request call id 856) took 1507 ms. Trace:
W20260709 13:18:20.301493 16217 log.cc:927] Time spent T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 1.070s	user 0.003s	sys 0.000s
I20260709 13:18:20.302279 16206 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:20.283849 16239 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:20.365571 16239 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:20.304229 10695 catalog_manager.cc:5697] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 7 to 9, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 9 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:18:20.366197 16228 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:20.367101 16228 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:20.367589 10695 catalog_manager.cc:5697] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 7 to 12, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 12 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:18:20.369601 10695 catalog_manager.cc:5697] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 13 to 16, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 16 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:20.308576 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:20.308992 16266 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:20.371217 10944 raft_consensus.cc:2393] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 17: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:20.371140 16266 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:20.372265 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:20.372889 10944 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 16.
I20260709 13:18:20.373692 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:20.373864 16266 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:20.317121 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:20.315973 10875 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
I20260709 13:18:20.325011 15870 raft_consensus.cc:740] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:20.329659 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:20.375550 10867 raft_consensus.cc:2393] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 14: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:20.330323 15798 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:20.375739 10796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2653) took 1456 ms. Trace:
I20260709 13:18:20.375674 15870 consensus_queue.cc:260] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:20.376328 15798 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:20.376739 15870 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:20.378477 15798 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:20.333648 10950 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 17.
I20260709 13:18:20.260584 16032 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:20.398103 16032 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:20.340855 11505 rpcz_store.cc:276] 0709 13:18:18.792717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:18.796606 (+  3889us) service_pool.cc:224] Handling call
0709 13:18:19.124762 (+328156us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:19.203624 (+ 78862us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:20.299965 (+1096341us) spinlock_profiling.cc:243] Waited 301 ms on lock 0x7b54005014d0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:20.300124 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":693069568}
I20260709 13:18:20.343178 15928 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:20.355505 16047 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:20.362948 16242 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:20.308190 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:20.380386 16200 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:20.381675 10695 catalog_manager.cc:5697] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:18:20.366163 16239 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:20.384963 11509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56410 (request call id 842) took 1674 ms. Trace:
I20260709 13:18:20.337636 16064 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:20.385950 10942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 2284) took 1195 ms. Trace:
I20260709 13:18:20.393199 16179 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:20.405277 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:20.407995 10798 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 15.
I20260709 13:18:20.324687 16104 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:20.376359 10796 rpcz_store.cc:276] 0709 13:18:18.919499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:18.925853 (+  6354us) service_pool.cc:224] Handling call
0709 13:18:18.937587 (+ 11734us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:19.241972 (+304385us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:18:19.262348 (+ 20376us) log.cc:844] Serialized 14 byte log entry
0709 13:18:19.609763 (+347415us) log.cc:844] Serialized 14 byte log entry
0709 13:18:19.909353 (+299590us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:19.911054 (+  1701us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:19.911123 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:20.375433 (+464310us) spinlock_profiling.cc:243] Waited 304 ms on lock 0x7b54001aa4d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:20.375503 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:20.375664 (+   161us) inbound_call.cc:177] Queueing success response
0709 13:18:20.392352 (+ 16688us) spinlock_profiling.cc:243] Waited 2.02 ms on lock 0x7b18003a4ec0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc83f3 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
Metrics: {"mutex_wait_us":40753,"prepare.queue_time_us":25523,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":5750,"raft.queue_time_us":52033,"raft.run_cpu_time_us":232,"raft.run_wall_time_us":7599,"spinlock_wait_cycles":716019968,"thread_start_us":579792,"threads_started":2,"wal-append.queue_time_us":406205}
I20260709 13:18:20.416786 10766 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:20.400699 16032 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:20.332615 10841 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:20.452596 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 120142us
I20260709 13:18:20.453522 10841 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:20.421833 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:20.420531 11509 rpcz_store.cc:276] 0709 13:18:18.710179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:18.710357 (+   178us) service_pool.cc:224] Handling call
0709 13:18:18.735192 (+ 24835us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:19.418007 (+682815us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:20.384710 (+966703us) spinlock_profiling.cc:243] Waited 602 ms on lock 0x7b54006dd154. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:20.384900 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1384617856}
I20260709 13:18:20.439982 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:20.443945 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:20.456459 10795 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:20.433604 10916 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:20.425735 10942 rpcz_store.cc:276] 0709 13:18:19.190116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:19.190935 (+   819us) service_pool.cc:224] Handling call
0709 13:18:19.191309 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:19.315625 (+124316us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:18:19.322695 (+  7070us) log.cc:844] Serialized 14 byte log entry
0709 13:18:19.493886 (+171191us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:19.493966 (+    80us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:19.494040 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:20.385735 (+891695us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b540004db54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:20.385777 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:20.385894 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":7386,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":37431,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":35888,"spinlock_wait_cycles":285302656,"thread_start_us":140835,"threads_started":1,"wal-append.queue_time_us":319898}
I20260709 13:18:20.424566 10915 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:20.457703 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:20.456022 10793 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:20.458495 10942 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
I20260709 13:18:20.459656 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:20.448243 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:18:20.449164 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:20.460580 11505 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:20.456787 10943 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 13.
I20260709 13:18:20.463565 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:20.464219 10945 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 14.
I20260709 13:18:20.466180 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:20.468120 10841 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:20.469750 10841 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:20.413875 16158 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:20.495975 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:20.495595 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:18:20.496101 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 79407.5us
I20260709 13:18:20.499666 10766 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:20.496793 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:18:20.471593 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 47100us
I20260709 13:18:20.415624 15928 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:20.505739 16237 log.cc:927] Time spent T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.398s	user 0.001s	sys 0.001s
I20260709 13:18:20.495136 16014 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:20.428318 16228 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:20.498759 10944 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 12.
I20260709 13:18:20.507702 10942 raft_consensus.cc:2393] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 14: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
W20260709 13:18:20.508476 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:20.507917 10916 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:20.513878 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:20.516589 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:20.511327 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:20.516476 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:20.523237 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:20.531021 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:20.532047 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:20.445763 16239 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:20.423473 15765 consensus_queue.cc:1048] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.092s
I20260709 13:18:20.538239 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:20.495297 15986 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:20.536463 10799 raft_consensus.cc:2410] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:18:20.451527 16200 consensus_queue.cc:1048] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.086s
I20260709 13:18:20.537300 15870 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:20.547441 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:20.551672 10800 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 17.
I20260709 13:18:20.552441 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:20.562852 10798 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 9.
I20260709 13:18:20.416042 16242 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:20.466984 15798 raft_consensus.cc:2749] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:18:20.557548 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:20.570237 10796 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
I20260709 13:18:20.507011 16206 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:20.506629 16194 raft_consensus.cc:2749] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:18:20.506870 16209 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:20.568323 10766 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:20.510632 15859 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:20.495448 15854 consensus_queue.cc:1048] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.065s
I20260709 13:18:20.577666 11504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56416 (request call id 850) took 2034 ms. Trace:
I20260709 13:18:20.580401 16129 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:20.471007 16104 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:20.581925 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:20.583959 10797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 719) took 1250 ms. Trace:
I20260709 13:18:20.584507 10941 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
W20260709 13:18:20.549563 16267 log.cc:927] Time spent T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:18:20.574918 10792 raft_consensus.cc:1240] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 LEADER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:18:20.511350 16273 raft_consensus.cc:2804] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:20.513070 16158 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:20.586354 16265 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:20.596628 16265 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:20.598105 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:20.600596 11478 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:20.601135 16252 raft_consensus.cc:3055] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:20.601642 16252 raft_consensus.cc:740] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:20.531487 16117 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:20.550634 16224 raft_consensus.cc:2804] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:20.602734 11478 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:20.587674 11504 rpcz_store.cc:276] 0709 13:18:18.549940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:18.550080 (+   140us) service_pool.cc:224] Handling call
0709 13:18:18.550511 (+   431us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:19.417127 (+866616us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:20.577396 (+1160269us) spinlock_profiling.cc:243] Waited 839 ms on lock 0x7b54004e5554. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:20.577600 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1930638080}
I20260709 13:18:20.589721 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:20.590770 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:20.591043 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:20.615171 10941 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
I20260709 13:18:20.593132 11511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56410 (request call id 843) took 1199 ms. Trace:
I20260709 13:18:20.617779 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:20.618394 11504 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
I20260709 13:18:20.619076 16239 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:20.619599 11504 raft_consensus.cc:1240] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:18:20.566435 15986 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:20.531165 16220 consensus_queue.cc:1048] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 13:18:20.593497 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:20.616451 11511 rpcz_store.cc:276] 0709 13:18:19.393582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:19.415840 (+ 22258us) service_pool.cc:224] Handling call
0709 13:18:20.592943 (+1177103us) spinlock_profiling.cc:243] Waited 9.76 ms on lock 0x7b54001d1354. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:20.593050 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22439680}
I20260709 13:18:20.625097 10943 raft_consensus.cc:2393] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 14: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:20.588064 10795 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
I20260709 13:18:20.613351 11478 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:20.603838 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:20.633095 16001 raft_consensus.cc:2749] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:20.633641 10915 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:20.628904 10941 raft_consensus.cc:1240] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 15. Current term is 16. Ops: [12.9-15.10]
I20260709 13:18:20.635155 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:20.606632 10766 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:20.583877 16209 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:20.589857 10797 rpcz_store.cc:276] 0709 13:18:19.333803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:19.560139 (+226336us) service_pool.cc:224] Handling call
0709 13:18:19.560473 (+   334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:19.588446 (+ 27973us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:20.583714 (+995268us) spinlock_profiling.cc:243] Waited 17.4 ms on lock 0x7b54003ecbd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:20.583900 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":40005376}
I20260709 13:18:20.636389 11505 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 14.
I20260709 13:18:20.556293 10793 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 14.
I20260709 13:18:20.465695 16064 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:20.639762 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:20.589385 15928 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:20.641023 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:20.592515 16264 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:20.642235 10875 raft_consensus.cc:2393] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 17: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:18:20.640664 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 434370us
I20260709 13:18:20.643373 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:20.644188 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:20.641903 10871 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 11.
I20260709 13:18:20.602563 16252 consensus_queue.cc:260] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:20.593343 16273 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:20.570981 15870 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:20.646696 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:20.692178 10868 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 13.
I20260709 13:18:20.694276 10915 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:18:20.694569 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:20.639081 16209 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:20.638850 16192 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:20.627923 16258 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:18:20.655371 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 322874us
I20260709 13:18:20.629630 15986 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:20.680516 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:20.606702 15859 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:20.683105 10870 raft_consensus.cc:1275] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:18:20.605470 16185 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:20.590233 16104 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:20.595857 16158 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:20.686895 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:20.636363 15798 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:20.635147 16239 raft_consensus.cc:2804] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:20.610496 16224 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:18:20.695264 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:20.691118 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:20.759186 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:20.639036 16174 raft_consensus.cc:2764] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
W20260709 13:18:20.679205 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:20.759335 10875 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 15.
I20260709 13:18:20.705669 16258 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:20.586745 16039 consensus_queue.cc:1048] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.066s
I20260709 13:18:20.701390 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:20.645624 16265 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:20.733423 15859 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:20.773181 16258 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:20.604553 16117 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:20.712785 10874 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 11.
I20260709 13:18:20.698837 16270 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:20.718174 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:20.635548 16170 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:20.649298 15875 consensus_queue.cc:1048] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.068s
I20260709 13:18:20.735502 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:20.740327 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:20.783993 10872 raft_consensus.cc:2393] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 14: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:20.691296 16273 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:20.680613 16141 consensus_queue.cc:1059] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 6, Time since last communication: 0.076s
I20260709 13:18:20.760378 11477 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:20.697788 16264 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:20.790872 15859 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:20.791410 16264 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:20.791956 16264 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:20.791677 15859 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:20.769896 10870 raft_consensus.cc:2393] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 16: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
W20260709 13:18:20.794945 16147 consensus_meta.cc:287] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 1.073s	user 0.008s	sys 0.005s
I20260709 13:18:20.795696 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:20.795540 16147 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:20.796996 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:20.719151 15765 raft_consensus.cc:2804] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:20.798238 15765 raft_consensus.cc:697] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 14 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:20.798302 16147 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:18:20.707916 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:20.589736 10695 catalog_manager.cc:5697] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 6 to 11. New cstate: current_term: 11 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:18:20.775513 10867 raft_consensus.cc:2393] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 14: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:20.737566 16194 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:20.761580 16064 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:20.760351 16239 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:20.760171 16224 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:18:20.771620 16210 consensus_meta.cc:287] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.614s	user 0.007s	sys 0.005s
I20260709 13:18:20.837745 16210 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:20.838816 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:20.840126 16210 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:20.827451 15854 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:20.841923 10872 raft_consensus.cc:1275] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 13:18:20.842149 16277 log.cc:927] Time spent T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:18:20.777591 16179 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:20.826228 16150 consensus_meta.cc:287] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.556s	user 0.010s	sys 0.000s
W20260709 13:18:20.787225 16225 log.cc:927] Time spent T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 1.066s	user 0.002s	sys 0.003s
I20260709 13:18:20.799289 15765 consensus_queue.cc:237] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:20.773958 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:20.761178 16158 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:20.803762 11503 raft_consensus.cc:1240] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:20.760609 15798 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:20.805050 16170 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:20.808092 15888 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:20.711145 16192 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:20.843372 15986 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:20.815109 16270 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:20.781283 16117 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:20.836038 16239 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:20.816416 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:20.817094 16141 raft_consensus.cc:3055] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:20.818161 16001 raft_consensus.cc:2804] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:18:20.818274 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 57976.3us
W20260709 13:18:20.824090 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 351956us
I20260709 13:18:20.825583 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:20.874697 10841 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:20.827787 11505 raft_consensus.cc:1240] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:20.794492 15859 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:20.828934 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:20.829300 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:20.836675 11478 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:20.841565 15854 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:20.781276 10868 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
I20260709 13:18:20.843082 16104 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:20.840796 16014 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:20.840060 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:20.846977 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 339103us
I20260709 13:18:20.840058 10948 raft_consensus.cc:2393] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 18: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
W20260709 13:18:20.851570 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 244994us
I20260709 13:18:20.851670 10842 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:20.812566 16198 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:20.865255 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:20.816079 16242 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:18:20.868413 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:20.878628 10841 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:20.880270 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 43700.6us
I20260709 13:18:20.901227 11478 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:20.901401 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:20.882201 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:20.901348 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:20.876011 10870 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 15.
I20260709 13:18:20.903620 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:20.902313 10875 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 14.
I20260709 13:18:20.889526 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:20.887387 10766 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:20.884954 11506 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 10.
I20260709 13:18:20.884168 16026 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:20.894387 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:20.896829 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:20.896992 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:20.905747 11478 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:20.897490 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:20.897595 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:20.899163 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:20.829239 16194 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:20.909924 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:20.907424 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:20.910813 10798 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 15.
I20260709 13:18:20.877152 16001 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:20.872694 16141 raft_consensus.cc:740] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:20.885393 16170 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:18:20.919309 16141 consensus_queue.cc:260] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:20.920858 16141 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:20.921211 11507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56416 (request call id 859) took 1892 ms. Trace:
I20260709 13:18:20.921589 11507 rpcz_store.cc:276] 0709 13:18:19.029217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:19.044216 (+ 14999us) service_pool.cc:224] Handling call
0709 13:18:20.921039 (+1876823us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54005f0354. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:20.921145 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2580732672}
I20260709 13:18:20.914620 16001 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:20.924259 16147 raft_consensus.cc:2804] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:20.924706 16147 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:20.925055 16147 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:20.927680 10766 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:20.928306 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:20.929039 10944 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 11.
I20260709 13:18:20.930016 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:20.931121 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:20.883692 16179 consensus_queue.cc:1048] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 13:18:20.933945 16179 consensus_queue.cc:1243] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer cb0428cc55984daba1ba63744a5102a2 log is divergent from this leader: its last log entry 12.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:18:20.936708 16292 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:20.937247 16292 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:20.937618 16292 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:20.938546 16252 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:20.940665 16193 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:20.896009 16014 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:20.945391 10874 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 14.
I20260709 13:18:20.924026 16162 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:20.859527 16150 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:20.909631 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:20.953912 10868 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 14.
I20260709 13:18:20.859220 16158 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:20.913878 10869 raft_consensus.cc:2393] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 14: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:20.906962 10797 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
I20260709 13:18:20.913566 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:20.908631 10872 raft_consensus.cc:2393] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 18: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:20.917162 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:20.957932 10792 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:20.958011 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:20.945528 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:20.924062 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:20.904213 10870 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 15.
I20260709 13:18:20.959903 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:20.960526 10870 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
I20260709 13:18:20.908809 10876 raft_consensus.cc:2393] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 14: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:20.961422 10867 raft_consensus.cc:2393] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 14: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:20.913461 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:20.947115 16266 raft_consensus.cc:2749] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:20.922678 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:20.964897 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:20.973459 10798 raft_consensus.cc:2410] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 17 because replica has last-logged OpId of term: 16 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 13:18:20.926142 16145 consensus_queue.cc:1048] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:18:20.949343 10944 raft_consensus.cc:2410] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:18:20.900411 16104 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:20.968820 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:20.974951 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:20.986671 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:20.982053 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:18:20.983587 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 131989us
I20260709 13:18:20.980507 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:20.988583 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:20.984690 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:20.989290 10795 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:20.989346 11511 raft_consensus.cc:2393] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 15: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:20.992352 10876 raft_consensus.cc:2410] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:18:20.992615 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:20.995748 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:20.990808 10916 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:21.002841 10949 raft_consensus.cc:1240] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:18:20.996384 10944 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 16.
I20260709 13:18:20.952649 15859 consensus_queue.cc:1059] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 13:18:21.007349 10793 raft_consensus.cc:1240] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:21.008383 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:20.895210 10695 catalog_manager.cc:5697] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:18:20.966696 16192 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:20.976881 16179 raft_consensus.cc:2804] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:20.969429 16211 raft_consensus.cc:2749] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:18:21.000941 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 122434us
I20260709 13:18:21.006415 10766 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:21.010146 10916 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:21.011991 16274 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:21.012621 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:21.013356 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:21.013617 16150 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:21.014358 16104 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:21.017602 10793 raft_consensus.cc:1275] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 11. (term mismatch)
I20260709 13:18:21.045274 10793 pending_rounds.cc:85] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 10
I20260709 13:18:21.045584 10793 pending_rounds.cc:107] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 13:18:21.045873 10793 raft_consensus.cc:2889] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:21.022117 10792 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
I20260709 13:18:21.022465 10795 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 18.
I20260709 13:18:20.985162 15798 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:21.026708 10841 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:21.027472 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:21.085239 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:21.014693 16014 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:21.035490 10947 raft_consensus.cc:1240] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:21.039597 10766 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:21.036007 11507 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 13.
W20260709 13:18:21.098218 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 58691.8us
I20260709 13:18:21.098837 16248 consensus_queue.cc:1048] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:21.099298 15853 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:21.099779 15853 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:21.100162 15853 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:21.046962 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:21.047310 10796 raft_consensus.cc:1240] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:21.047595 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:21.015107 16242 raft_consensus.cc:2749] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:21.103019 16179 raft_consensus.cc:697] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:21.081362 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:21.104046 10799 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 13.
I20260709 13:18:21.104146 16179 consensus_queue.cc:237] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:21.014612 16158 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:21.031349 16026 raft_consensus.cc:697] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:18:21.096599 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 86572.4us
I20260709 13:18:21.105304 10795 raft_consensus.cc:2393] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 18: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:18:21.118693 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 92071.1us
I20260709 13:18:21.036331 16192 raft_consensus.cc:2749] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:21.110092 11478 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:21.119128 10916 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:21.120334 10916 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:21.120478 10841 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:21.054059 16188 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:21.124878 10800 raft_consensus.cc:1240] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 14. Current term is 15. Ops: []
W20260709 13:18:21.123515 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 134407us
I20260709 13:18:21.087200 16109 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:21.136873 10841 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:21.138070 10916 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:21.106451 15859 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:21.143714 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:21.035257 16154 consensus_queue.cc:1059] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.086s
I20260709 13:18:21.154011 10916 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:21.154469 10947 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 13.
I20260709 13:18:21.161180 10947 raft_consensus.cc:1240] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:21.155474 16198 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:21.146726 16295 log.cc:927] Time spent T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 13:18:21.096591 16167 consensus_queue.cc:1059] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:18:21.084077 16129 consensus_queue.cc:1059] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.066s
I20260709 13:18:21.123947 16242 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:21.145979 16104 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:21.125689 16150 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:21.048051 16274 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:21.130374 16188 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:21.128733 16064 raft_consensus.cc:2749] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:18:21.174253 11506 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56416 (request call id 862) took 1518 ms (client timeout 1602 ms). Trace:
I20260709 13:18:21.175575 10943 raft_consensus.cc:1240] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:21.146718 15853 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:21.156883 16109 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:21.177572 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 46229.7us
I20260709 13:18:21.178300 10841 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:21.154649 15854 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:21.153478 15859 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:18:21.182103 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 82903.5us
I20260709 13:18:21.406337 10944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 897) took 1218 ms. Trace:
I20260709 13:18:21.425686 10944 rpcz_store.cc:276] 0709 13:18:20.187959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:21.017146 (+829187us) service_pool.cc:224] Handling call
0709 13:18:21.023689 (+  6543us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:21.023932 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:18:21.045882 (+ 21950us) log.cc:844] Serialized 14 byte log entry
0709 13:18:21.405887 (+360005us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:21.405956 (+    69us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:18:21.406023 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:21.406106 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:21.406262 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":22150,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":146819,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":93,"thread_start_us":359607,"threads_started":2,"wal-append.queue_time_us":283755}
I20260709 13:18:21.427542 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:18:21.418246 16264 consensus_meta.cc:287] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.626s	user 0.012s	sys 0.000s
W20260709 13:18:21.431136 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:21.271402 16026 consensus_queue.cc:237] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:21.431270 16264 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:21.236016 11510 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 14.
I20260709 13:18:21.327853 10945 raft_consensus.cc:1240] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:21.296327 10947 raft_consensus.cc:1240] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 14. Current term is 15. Ops: []
W20260709 13:18:21.296254 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:21.423426 11503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56404 (request call id 924) took 1115 ms. Trace:
I20260709 13:18:21.169512 16206 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:18:21.310534 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:21.437522 11506 rpcz_store.cc:269] 0709 13:18:19.655912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:19.689339 (+ 33427us) service_pool.cc:224] Handling call
0709 13:18:21.174070 (+1484731us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400691c54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:21.174184 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2333404544}
W20260709 13:18:21.265291 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:21.351850 10948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41014 (request call id 2285) took 1434 ms. Trace:
W20260709 13:18:21.410547 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 256596us
I20260709 13:18:21.475980 10946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 2286) took 1567 ms. Trace:
W20260709 13:18:21.421351 16273 consensus_meta.cc:287] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.634s	user 0.015s	sys 0.000s
I20260709 13:18:21.430617 10766 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:21.433977 16266 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:21.436709 11503 rpcz_store.cc:276] 0709 13:18:20.307790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:20.362012 (+ 54222us) service_pool.cc:224] Handling call
0709 13:18:20.362486 (+   474us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:20.845278 (+482792us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:21.423105 (+577827us) spinlock_profiling.cc:243] Waited 441 ms on lock 0x7b54004290d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:21.423347 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1014280064}
I20260709 13:18:21.339458 10949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 2298) took 1132 ms. Trace:
W20260709 13:18:21.444518 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 266466us
I20260709 13:18:21.415056 16315 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:18:21.357111 16209 consensus_meta.cc:287] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.643s	user 0.009s	sys 0.002s
I20260709 13:18:21.477172 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 2224) took 1218 ms. Trace:
I20260709 13:18:21.470397 16303 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:21.445513 16206 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:21.490550 10948 rpcz_store.cc:276] 0709 13:18:19.902610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:19.902881 (+   271us) service_pool.cc:224] Handling call
0709 13:18:21.336412 (+1433531us) spinlock_profiling.cc:243] Waited 935 ms on lock 0x7b540018e7d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:21.336578 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2150455168}
I20260709 13:18:21.489421 10916 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:21.434231 16001 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:21.564000 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 74726.7us
W20260709 13:18:21.319989 16241 log.cc:927] Time spent T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 1.573s	user 0.011s	sys 0.002s
I20260709 13:18:21.481446 10842 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:21.565311 10916 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:21.565389 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 83998.6us
I20260709 13:18:21.211138 15907 raft_consensus.cc:2804] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:21.419534 16292 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:21.488616 10946 rpcz_store.cc:276] 0709 13:18:19.908362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:19.908514 (+   152us) service_pool.cc:224] Handling call
0709 13:18:20.314799 (+406285us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:20.315179 (+   380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:20.315218 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:20.337357 (+ 22139us) log.cc:844] Serialized 26 byte log entry
0709 13:18:20.778112 (+440755us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:20.778154 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:20.778222 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:20.989941 (+211719us) raft_consensus.cc:1685] finished
0709 13:18:20.989966 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:21.475659 (+485693us) spinlock_profiling.cc:243] Waited 376 ms on lock 0x7b54003f0050. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:21.475899 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9412,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":5138,"spinlock_wait_cycles":864298752,"thread_start_us":356824,"threads_started":1,"wal-append.queue_time_us":464132}
I20260709 13:18:21.297180 16185 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:21.541404 10841 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:21.472455 16239 consensus_meta.cc:287] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.591s	user 0.003s	sys 0.007s
I20260709 13:18:21.577942 11507 raft_consensus.cc:1240] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:18:21.579092 10841 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:21.216111 16104 raft_consensus.cc:697] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:18:21.453728 16287 log.cc:927] Time spent T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.518s	user 0.000s	sys 0.002s
I20260709 13:18:21.539743 10949 rpcz_store.cc:276] 0709 13:18:20.207471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:21.048297 (+840826us) service_pool.cc:224] Handling call
0709 13:18:21.105919 (+ 57622us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:21.106141 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:18:21.106373 (+   232us) log.cc:844] Serialized 14 byte log entry
0709 13:18:21.338971 (+232598us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:21.339048 (+    77us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:21.339117 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:21.339207 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:21.339379 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":10977,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":9398,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":34467,"thread_start_us":232182,"threads_started":1,"wal-append.queue_time_us":311512}
I20260709 13:18:21.158631 16192 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:21.590750 16192 raft_consensus.cc:697] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:21.591491 16274 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:21.543568 16273 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:21.592033 16192 consensus_queue.cc:237] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:21.593787 16273 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:21.569595 16303 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:21.595441 16290 log.cc:927] Time spent T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.650s	user 0.003s	sys 0.000s
I20260709 13:18:21.596519 16303 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:21.597285 11504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56404 (request call id 929) took 1022 ms. Trace:
I20260709 13:18:21.597877 10949 raft_consensus.cc:1275] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:18:21.597641 11504 rpcz_store.cc:276] 0709 13:18:20.574661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:20.620341 (+ 45680us) service_pool.cc:224] Handling call
0709 13:18:20.620621 (+   280us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:20.620961 (+   340us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:20.620989 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:20.621418 (+   429us) log.cc:844] Serialized 26 byte log entry
0709 13:18:20.846485 (+225067us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:20.846513 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:20.846557 (+    44us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:21.594862 (+748305us) spinlock_profiling.cc:243] Waited 9.03 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:18:21.597020 (+  2158us) raft_consensus.cc:1685] finished
0709 13:18:21.597040 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:21.597231 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":113,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":84,"spinlock_wait_cycles":20771200,"thread_start_us":224720,"threads_started":1,"wal-append.queue_time_us":273080}
I20260709 13:18:21.600471 16286 raft_consensus.cc:3055] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:21.600921 16286 raft_consensus.cc:740] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:21.601758 16286 consensus_queue.cc:260] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:21.604157 16145 raft_consensus.cc:3055] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:21.603605 16104 consensus_queue.cc:237] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:21.604588 16145 raft_consensus.cc:740] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:21.605490 16145 consensus_queue.cc:260] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:21.606508 16145 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:18:21.481518 16162 consensus_meta.cc:287] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.526s	user 0.006s	sys 0.003s
W20260709 13:18:21.470552 16252 consensus_meta.cc:287] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.532s	user 0.011s	sys 0.000s
I20260709 13:18:21.445346 16188 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:21.623071 16286 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:21.629000 15986 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:21.636127 15888 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:21.636902 15888 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:21.639532 16185 raft_consensus.cc:2749] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:21.443948 16150 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:21.647367 16150 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:18:21.466981 16141 consensus_meta.cc:287] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.546s	user 0.011s	sys 0.000s
I20260709 13:18:21.541013 16175 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:21.439896 10943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 902) took 1227 ms. Trace:
I20260709 13:18:21.219820 16270 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:21.404353 16242 raft_consensus.cc:697] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 13 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:21.445360 15859 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:21.470958 16264 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:21.548193 10869 rpcz_store.cc:276] 0709 13:18:20.258947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:20.259622 (+   675us) service_pool.cc:224] Handling call
0709 13:18:21.476920 (+1217298us) spinlock_profiling.cc:243] Waited 592 ms on lock 0x7b540006ccd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:21.477098 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1360923904}
I20260709 13:18:21.491283 11477 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:21.451969 16111 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:21.279199 16258 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:21.680401 16111 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:21.680895 16111 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:21.681969 16258 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:18:21.561074 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:21.683415 16175 raft_consensus.cc:2749] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:21.566460 10916 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:21.686766 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 120368us
I20260709 13:18:21.687523 10916 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:21.688855 10916 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:21.690296 11510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56416 (request call id 864) took 2033 ms. Trace:
I20260709 13:18:21.690698 11510 rpcz_store.cc:276] 0709 13:18:19.656509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:19.771287 (+114778us) service_pool.cc:224] Handling call
0709 13:18:21.690132 (+1918845us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b54001f90d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:21.690239 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3144011520}
I20260709 13:18:21.692409 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:21.209230 15826 consensus_queue.cc:1059] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.340s
I20260709 13:18:21.695152 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:21.695864 10870 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:21.696722 16111 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:21.566655 10842 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:21.351982 16224 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:21.712109 15888 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:21.712852 15986 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:21.713274 15986 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:21.713485 16224 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:21.713653 15986 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:21.417568 16147 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:21.715566 16147 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:21.716265 16258 raft_consensus.cc:2749] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:18:21.716789 16270 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:21.717299 16270 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:21.719514 16270 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:21.722148 16129 consensus_queue.cc:1059] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:21.723227 16129 consensus_queue.cc:1059] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:21.725049 16129 raft_consensus.cc:3055] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:21.725400 16129 raft_consensus.cc:740] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:21.726528 16145 raft_consensus.cc:3055] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:18:21.726963 16145 raft_consensus.cc:740] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:21.570709 15798 raft_consensus.cc:2804] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:21.572872 16064 raft_consensus.cc:2749] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:21.569293 15907 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:21.575534 15853 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:21.580747 10841 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:21.735437 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 154813us
W20260709 13:18:21.537415 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 106885us
I20260709 13:18:21.576298 16001 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:21.586548 16316 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:21.550804 16315 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:21.587435 16239 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:21.545490 16266 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:21.588510 16297 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:21.589993 16292 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:21.608125 16220 raft_consensus.cc:3055] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:21.609411 10949 raft_consensus.cc:1240] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:21.551999 16209 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:21.580713 16039 raft_consensus.cc:2749] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:21.613336 15870 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:21.202126 16265 raft_consensus.cc:3055] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:21.568524 16206 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:21.661365 16189 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:21.672194 10943 rpcz_store.cc:276] 0709 13:18:20.212508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:21.153741 (+941233us) service_pool.cc:224] Handling call
0709 13:18:21.175389 (+ 21648us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:21.438725 (+263336us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:21.439806 (+  1081us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:21.273003 16117 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:21.674547 16242 consensus_queue.cc:237] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:21.698031 10916 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:21.700299 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:21.700811 10797 raft_consensus.cc:1240] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:18:21.823670 10950 raft_consensus.cc:2393] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 14: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:21.790026 16194 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:21.701429 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:21.707402 15765 raft_consensus.cc:2749] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:21.707535 16311 raft_consensus.cc:2749] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:18:21.707768 10872 raft_consensus.cc:1240] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 18. Current term is 19. Ops: []
I20260709 13:18:21.708072 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:21.726181 16129 consensus_queue.cc:260] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:21.709329 16219 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:18:21.710753 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 144184us
W20260709 13:18:21.711020 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 219824us
I20260709 13:18:21.842314 11477 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:21.709789 16314 consensus_queue.cc:1059] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:18:21.729447 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:21.737360 10766 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:21.771857 11508 raft_consensus.cc:1275] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:18:21.772384 10841 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:21.778437 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:21.782119 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:21.846365 10876 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:21.759568 15907 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:21.735200 15798 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:21.851116 15798 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:21.852036 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:21.794476 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:21.795660 11478 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:21.856230 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 60641.8us
I20260709 13:18:21.857275 16314 raft_consensus.cc:3055] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:21.857687 16314 raft_consensus.cc:740] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:21.772984 15883 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:21.858508 16314 consensus_queue.cc:260] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:21.859248 15883 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:21.859733 15883 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:21.859882 16314 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:21.772089 16064 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:21.863165 16064 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:21.863587 16111 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:21.219036 10695 catalog_manager.cc:5697] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 9 to 11, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 11 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:18:21.865348 16064 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:21.808534 10945 raft_consensus.cc:1240] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:18:21.773298 16316 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:21.866257 16316 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:18:21.867125 16322 log.cc:927] Time spent T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:18:21.867286 16209 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:21.869062 16315 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:18:21.791384 16321 log.cc:927] Time spent T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:18:21.784174 16289 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:21.727784 16145 consensus_queue.cc:260] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:21.537952 16182 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.007s	sys 0.006s
W20260709 13:18:21.825769 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 127808us
I20260709 13:18:21.792707 16297 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:21.826924 10796 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:21.799091 16220 raft_consensus.cc:740] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:21.834287 10794 raft_consensus.cc:2410] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:18:21.708631 16210 consensus_queue.cc:1048] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:18:21.929581 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 157282us
I20260709 13:18:21.813771 16265 raft_consensus.cc:740] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:21.836553 10870 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 15.
I20260709 13:18:21.864732 10695 catalog_manager.cc:5697] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 9 to 12, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 12 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:18:21.695218 15859 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:21.872903 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:21.877082 10876 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 14.
W20260709 13:18:21.880055 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 142795us
I20260709 13:18:21.944070 10766 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:21.898190 16316 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:21.892526 10916 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:18:21.944929 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 52501.8us
I20260709 13:18:21.882891 11477 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:21.899587 16129 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:21.889365 16117 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:21.907085 15798 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:21.912012 10842 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:21.911948 15883 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:21.926681 16297 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:21.919448 16252 raft_consensus.cc:2804] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:21.885514 16258 raft_consensus.cc:2749] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:18:21.920220 15888 consensus_queue.cc:1059] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:18:21.932418 10841 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:21.890379 16182 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.947s	user 0.008s	sys 0.007s
I20260709 13:18:21.859052 16150 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:21.833855 16224 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:21.870002 16219 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:18:21.945962 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 63205.6us
I20260709 13:18:21.991948 11477 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:21.948465 16315 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:21.918834 16162 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:21.941953 16014 consensus_queue.cc:1048] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:18:21.953761 10842 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:18:21.964303 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:21.962878 10801 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 12.
I20260709 13:18:21.889094 16188 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:21.939215 15854 raft_consensus.cc:2764] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election decision vote started in defunct term 14: won
W20260709 13:18:22.018723 16323 log.cc:927] Time spent T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:18:22.049304 16285 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:18:22.057493 16285 raft_consensus.cc:697] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:22.058554 16285 consensus_queue.cc:237] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:22.063027 16239 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:22.067883 16316 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:22.024080 11478 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:22.070533 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 46545.1us
I20260709 13:18:22.071360 16147 raft_consensus.cc:2749] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:22.007212 16174 raft_consensus.cc:2749] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:22.072525 11478 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:22.028570 16150 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:22.074067 16258 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:22.074560 16258 raft_consensus.cc:697] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 15 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:22.074968 16141 raft_consensus.cc:3055] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:18:22.075387 16141 raft_consensus.cc:740] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:22.075426 16258 consensus_queue.cc:237] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:22.077801 16220 consensus_queue.cc:260] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:22.035326 16173 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:22.079167 16173 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:22.079339 11478 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:22.080305 16150 raft_consensus.cc:2804] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:22.080802 16039 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:22.023017 15986 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:22.081355 16039 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:21.981014 10942 raft_consensus.cc:1240] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:22.003007 16104 raft_consensus.cc:2764] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
I20260709 13:18:22.005419 16315 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:22.086671 10942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 2310) took 1116 ms. Trace:
I20260709 13:18:22.087096 16315 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:22.087101 10942 rpcz_store.cc:276] 0709 13:18:20.970748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:21.443511 (+472763us) service_pool.cc:224] Handling call
0709 13:18:21.443799 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:22.084823 (+641024us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:22.086443 (+  1620us) spinlock_profiling.cc:243] Waited 537 ms on lock 0x7b54000aafd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:22.086615 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1235027712}
I20260709 13:18:21.943666 16145 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:21.890519 16289 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:21.990876 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 58547.1us
I20260709 13:18:21.993501 10950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41014 (request call id 2291) took 1981 ms. Trace:
I20260709 13:18:21.946880 10766 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:18:21.996464 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:22.097839 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:22.097998 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 151218us
I20260709 13:18:21.888039 16311 consensus_queue.cc:1048] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 13:18:21.963555 16211 consensus_queue.cc:1059] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 13:18:21.967250 16252 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:22.017342 11510 raft_consensus.cc:1240] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:21.963400 16111 raft_consensus.cc:2804] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:22.034277 11477 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:22.091269 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:18:22.107227 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 73029us
I20260709 13:18:22.107966 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:22.108574 10801 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:21.952059 10916 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:22.035401 10945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 2313) took 1055 ms. Trace:
I20260709 13:18:22.039635 10944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 2308) took 1069 ms. Trace:
I20260709 13:18:21.996726 16219 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:22.110775 10945 rpcz_store.cc:276] 0709 13:18:20.979947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:21.545976 (+566029us) service_pool.cc:224] Handling call
0709 13:18:21.546281 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:21.865946 (+319665us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:22.035126 (+169180us) spinlock_profiling.cc:243] Waited 262 ms on lock 0x7b54000ab9d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:22.035326 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":602570496}
I20260709 13:18:22.083320 15986 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:22.031930 16162 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:22.032954 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 79259.2us
I20260709 13:18:22.018910 15859 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:22.005947 16224 raft_consensus.cc:697] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 15 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:22.092008 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:22.009204 10796 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 15.
I20260709 13:18:22.132606 10767 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:21.954213 16001 raft_consensus.cc:2749] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:18:21.953235 15870 raft_consensus.cc:2804] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:22.103811 10841 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:22.096472 10950 rpcz_store.cc:276] 0709 13:18:20.011741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:20.626371 (+614630us) service_pool.cc:224] Handling call
0709 13:18:21.993255 (+1366884us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54001f2f54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:21.993432 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2626056576}
I20260709 13:18:22.106719 16147 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:22.111840 10944 rpcz_store.cc:276] 0709 13:18:20.970213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:21.429585 (+459372us) service_pool.cc:224] Handling call
0709 13:18:21.429839 (+   254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:21.800032 (+370193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:21.800064 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:21.800101 (+    37us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:21.800136 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:22.039278 (+239142us) spinlock_profiling.cc:243] Waited 370 ms on lock 0x7b54001e1254. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:22.039341 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:22.039553 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":850917760}
I20260709 13:18:22.114374 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:22.150499 10946 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:22.082041 16173 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:22.029738 16109 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:22.067942 15883 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:22.117405 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 165445us
I20260709 13:18:22.169544 10916 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:22.169900 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:22.138451 16239 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:22.117724 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:22.199679 16239 raft_consensus.cc:697] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:22.200892 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:22.201614 10948 raft_consensus.cc:2393] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 14: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:22.023195 16206 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:22.080822 16150 raft_consensus.cc:697] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 14 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:22.203713 16206 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:22.203428 16150 consensus_queue.cc:237] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:22.200446 10801 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 14.
I20260709 13:18:22.059391 16188 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:22.097699 11478 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:22.125006 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:22.125413 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:22.125835 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:22.213500 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:18:22.213642 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 116030us
I20260709 13:18:22.130033 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:22.214570 10947 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:22.127460 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:22.215675 10795 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:22.104717 16220 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:22.086431 15907 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:22.136595 11512 raft_consensus.cc:2393] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 14: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:22.106842 16197 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:22.130002 10766 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:22.110402 16252 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:21.941618 16292 raft_consensus.cc:2804] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:22.034919 16265 consensus_queue.cc:260] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:22.144670 10841 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:22.239656 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:18:22.241786 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 97198.2us
I20260709 13:18:22.146297 10767 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:22.148947 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:22.244230 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:22.244913 10876 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 14.
I20260709 13:18:22.114283 16174 raft_consensus.cc:2749] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:18:22.110716 16111 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:22.152284 10873 raft_consensus.cc:1240] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:22.116192 16211 raft_consensus.cc:3055] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:22.153663 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:22.259416 10794 raft_consensus.cc:2376] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 16. Re-sending same reply.
I20260709 13:18:22.154820 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:22.260193 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:22.160494 10799 raft_consensus.cc:1240] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:22.166373 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:22.263864 15907 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:22.264223 10870 raft_consensus.cc:1240] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:22.264441 16039 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:22.115432 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:22.166978 16026 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:22.140534 15870 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:22.168381 15859 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:22.156352 16117 raft_consensus.cc:2749] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:22.121156 10695 catalog_manager.cc:5697] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 9 to 12, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 12 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:18:22.177201 10941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 2314) took 1184 ms. Trace:
I20260709 13:18:22.149132 16316 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:22.178817 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:22.114012 15888 consensus_queue.cc:1059] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 13:18:22.179116 16266 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:22.274668 11478 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:22.230171 16270 raft_consensus.cc:2804] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:22.180938 15798 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:22.159531 16147 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:22.181632 16326 log.cc:927] Time spent T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:18:22.114439 16104 consensus_queue.cc:1059] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
W20260709 13:18:22.182538 16324 log.cc:927] Time spent T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 13:18:22.110872 16014 consensus_queue.cc:1048] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 13:18:22.184181 16141 consensus_queue.cc:260] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:22.188871 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:22.341535 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:22.342650 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:22.191196 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:22.192090 16289 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:22.192927 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:22.192621 16224 consensus_queue.cc:237] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:22.349884 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:22.195062 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:22.196681 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:22.355365 10867 raft_consensus.cc:2393] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 16: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:22.200563 16239 consensus_queue.cc:237] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:22.356563 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:22.358117 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:22.120720 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:22.345592 16315 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:22.359809 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:22.360772 10800 raft_consensus.cc:2393] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 16: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:22.361078 16315 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:22.362102 11505 raft_consensus.cc:2393] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 14: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:22.204983 10949 raft_consensus.cc:1240] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:22.205775 11477 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:22.364244 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 158574us
I20260709 13:18:22.288146 16270 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:22.367570 16162 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:22.367934 16270 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:22.368151 16162 raft_consensus.cc:2749] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:18:22.369001 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:22.205981 10916 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:22.329742 16039 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:22.291549 16001 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:18:22.371435 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 165502us
I20260709 13:18:22.371205 10695 catalog_manager.cc:5697] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:18:22.372008 16001 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:22.372491 16001 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:22.373167 10695 catalog_manager.cc:5697] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 9 to 12. New cstate: current_term: 12 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:22.375044 16210 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:18:22.375680 16210 raft_consensus.cc:697] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 18 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:22.375841 16292 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:22.376340 16292 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:22.376646 16210 consensus_queue.cc:237] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:22.378607 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:22.379712 11512 raft_consensus.cc:2393] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 18: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:22.205083 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:22.202497 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:22.226024 10793 raft_consensus.cc:1240] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 17. Current term is 18. Ops: []
W20260709 13:18:22.231843 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 101925us
I20260709 13:18:22.382560 10842 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:22.386649 10942 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 14.
I20260709 13:18:22.381913 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:22.384196 10766 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:22.389230 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:22.239287 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:22.393013 11504 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 14.
I20260709 13:18:22.239503 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:22.256253 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:22.371520 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:18:22.259946 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 113761us
I20260709 13:18:22.271744 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:22.271899 10871 raft_consensus.cc:1240] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:22.394951 11511 raft_consensus.cc:2393] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 17: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:22.273953 10876 raft_consensus.cc:1240] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:18:22.395421 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:22.396026 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:22.396457 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:22.397148 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:22.397845 10876 raft_consensus.cc:2393] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 14: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:22.274674 10873 raft_consensus.cc:1240] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:18:22.274911 10795 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 14.
I20260709 13:18:22.280375 10941 rpcz_store.cc:276] 0709 13:18:20.992941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:21.580172 (+587231us) service_pool.cc:224] Handling call
0709 13:18:21.580460 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:21.640537 (+ 60077us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:18:21.640838 (+   301us) log.cc:844] Serialized 14 byte log entry
0709 13:18:22.118734 (+477896us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:22.119558 (+   824us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:22.119630 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:22.176429 (+ 56799us) spinlock_profiling.cc:243] Waited 6.25 ms on lock 0x7b18000fc3a0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7dd962a6 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419
0709 13:18:22.176967 (+   538us) spinlock_profiling.cc:243] Waited 59.7 ms on lock 0x7b5400376754. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:22.177013 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:22.177137 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":567,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":11813,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":12748,"spinlock_wait_cycles":151717888,"thread_start_us":465372,"threads_started":1}
I20260709 13:18:22.399319 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:22.229202 16197 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:22.273247 16296 raft_consensus.cc:2749] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:22.295388 15888 raft_consensus.cc:3055] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:22.232192 16219 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:22.272819 16211 raft_consensus.cc:740] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:22.239830 16206 consensus_queue.cc:1059] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.060s
I20260709 13:18:22.277364 16167 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:22.335063 10947 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 17.
I20260709 13:18:22.275619 15870 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:22.276351 16111 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:22.337594 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:22.343557 10944 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:22.349342 10874 raft_consensus.cc:1240] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:22.281493 15859 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:22.228217 10946 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 16.
I20260709 13:18:22.394295 11503 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:22.272081 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:22.425211 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:22.401219 10793 raft_consensus.cc:1240] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:22.404994 10842 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:18:22.392737 16259 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.022s	sys 0.031s
I20260709 13:18:22.402855 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:22.403098 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:22.426026 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:22.407246 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:22.402405 15986 consensus_queue.cc:1059] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 17.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.007s
I20260709 13:18:22.413724 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:22.289137 16316 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:22.416026 10916 connection.cc:256] RPC call timeout handler was delayed by -1.36486152700013s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:18:22.433980 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:22.434409 10916 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:22.416371 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:22.419401 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:22.399036 16192 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:22.439291 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:22.440018 10950 raft_consensus.cc:2393] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 14: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:22.436818 10942 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 16.
I20260709 13:18:22.440523 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:22.345180 16297 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:22.413296 16179 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:22.416699 16197 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:22.455628 10941 raft_consensus.cc:1240] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 16. Current term is 17. Ops: [16.8-16.8]
I20260709 13:18:22.467104 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:22.441604 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:22.416036 16173 raft_consensus.cc:2749] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:22.462059 10916 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:22.472398 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:22.371224 16039 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:22.443931 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:22.511291 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:22.511762 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:22.463475 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:22.441587 16315 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:22.512631 10948 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 17.
I20260709 13:18:22.512992 16315 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:22.514334 16265 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:22.478826 16192 raft_consensus.cc:697] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:22.515683 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:22.515676 16192 consensus_queue.cc:237] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:22.440505 16209 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:22.518013 10841 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:22.480188 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:22.519099 11511 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:22.519399 10841 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:22.519670 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:22.520753 15854 raft_consensus.cc:2749] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:22.521453 15854 raft_consensus.cc:2749] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:22.483093 10872 raft_consensus.cc:1275] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 13:18:22.400239 16014 consensus_queue.cc:1059] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:18:22.503574 15798 raft_consensus.cc:3055] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:22.504594 16242 raft_consensus.cc:3055] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:22.505970 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:22.522334 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:22.520710 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:22.506654 16219 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:22.507453 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:22.528613 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:22.510053 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:22.511439 16297 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:22.535648 10871 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:22.518234 16174 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:22.482122 16316 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:22.529493 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:22.417014 15888 raft_consensus.cc:740] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:22.522564 10841 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:22.531898 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:22.522406 16147 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:22.536999 11511 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 11.
I20260709 13:18:22.579461 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:22.554332 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:22.534631 10795 raft_consensus.cc:2393] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 18: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:22.535068 10799 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 11.
I20260709 13:18:22.430383 16314 raft_consensus.cc:2764] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:18:22.554605 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:22.544745 10942 raft_consensus.cc:1240] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:22.530555 10944 raft_consensus.cc:2410] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:18:22.526382 16141 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:18:22.556706 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 94726.5us
I20260709 13:18:22.557619 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:22.562057 10950 raft_consensus.cc:1217] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Deduplicated request from leader. Original: 9.4->[10.5-13.7]   Dedup: 10.5->[11.6-13.7]
I20260709 13:18:22.589900 10950 pending_rounds.cc:85] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 5
I20260709 13:18:22.590415 10950 pending_rounds.cc:107] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 12.6
I20260709 13:18:22.590858 10950 raft_consensus.cc:2889] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:22.432754 16104 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:22.578823 16026 raft_consensus.cc:2749] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:22.572602 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:22.535710 10948 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
I20260709 13:18:22.573812 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:18:22.441675 16259 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.023s	sys 0.031s
I20260709 13:18:22.537063 10872 pending_rounds.cc:85] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 7
I20260709 13:18:22.583375 10793 raft_consensus.cc:1240] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:22.526757 16228 raft_consensus.cc:3055] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:18:22.584748 10795 raft_consensus.cc:1275] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:18:22.551224 11503 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 14.
I20260709 13:18:22.589170 16219 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:22.604293 10944 raft_consensus.cc:1275] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:18:22.535230 15798 raft_consensus.cc:740] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:22.544103 16242 raft_consensus.cc:740] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:22.555866 16206 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:22.609315 10794 raft_consensus.cc:1275] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:18:22.609606 10796 raft_consensus.cc:1240] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:18:22.610291 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:22.611059 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:22.618068 10941 raft_consensus.cc:1275] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:18:22.571707 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:22.518116 16209 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:22.619874 10945 raft_consensus.cc:1240] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:22.623399 10872 pending_rounds.cc:107] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:18:22.699287 10948 raft_consensus.cc:1275] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:18:22.701128 10872 raft_consensus.cc:2889] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:22.509517 10695 catalog_manager.cc:5697] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:18:22.594727 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 72226.3us
I20260709 13:18:22.704530 10841 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:22.625726 10875 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:22.705431 10942 raft_consensus.cc:2393] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 16: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:22.628535 10793 raft_consensus.cc:1275] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:18:22.629115 16001 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:22.632795 16292 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:22.580252 16174 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:22.634718 16145 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:22.612784 16297 raft_consensus.cc:2749] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:22.652695 16311 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:22.528903 16179 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:22.661384 11477 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:22.661729 10916 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:22.670253 10871 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
I20260709 13:18:22.753033 11504 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
I20260709 13:18:22.706307 10792 raft_consensus.cc:2393] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 14: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:22.617624 16104 raft_consensus.cc:697] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:22.528389 16211 consensus_queue.cc:260] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:22.624300 16228 raft_consensus.cc:740] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:22.721254 16001 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:22.510600 16252 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:22.728070 10695 catalog_manager.cc:5697] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 8 to 9, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 9 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:18:22.729857 10946 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:22.707705 15986 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:22.733896 15859 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:22.734742 11512 raft_consensus.cc:2393] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 15: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
W20260709 13:18:22.740005 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 66115.9us
I20260709 13:18:22.747700 15883 raft_consensus.cc:3055] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:22.749672 15888 consensus_queue.cc:260] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:22.731247 16209 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:22.823257 16270 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:22.823982 16209 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:22.824525 16209 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:22.825127 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:22.825558 16270 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:22.826977 11509 raft_consensus.cc:1240] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:22.827725 10793 raft_consensus.cc:1275] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:18:22.829257 16014 consensus_queue.cc:1048] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:22.830741 16014 consensus_queue.cc:1048] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:22.832520 16252 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:22.833285 11507 raft_consensus.cc:1240] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:22.821569 15986 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:22.836970 15986 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:22.837599 16039 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:22.838172 16039 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:22.759385 16311 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:22.838651 16039 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:22.840032 16292 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:22.840790 16311 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:22.826556 16209 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:22.841406 16039 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:22.841977 16179 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:22.842741 16104 consensus_queue.cc:237] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:22.841434 16311 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:22.843775 16311 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:22.844139 16311 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:22.798935 16242 consensus_queue.cc:260] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:22.846357 16242 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:22.848554 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:22.849599 10792 raft_consensus.cc:2410] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:18:22.848625 10695 catalog_manager.cc:5697] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: config changed from index 4 to 8, term changed from 10 to 13, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to <none>, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New cstate: current_term: 13 committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:18:22.752007 16145 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:22.852653 16145 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:22.855063 16228 consensus_queue.cc:260] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:22.856392 10792 raft_consensus.cc:1240] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:22.859217 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:22.860399 10792 raft_consensus.cc:2410] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 17 because replica has last-logged OpId of term: 15 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 6.
W20260709 13:18:22.774685 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 113371us
I20260709 13:18:22.658526 16219 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:18:22.798501 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 94091.7us
I20260709 13:18:22.662691 16206 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:22.719082 15798 consensus_queue.cc:260] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:22.758234 16331 log.cc:927] Time spent T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.349s	user 0.000s	sys 0.005s
I20260709 13:18:22.812770 15883 raft_consensus.cc:740] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:22.874915  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:22.856138 16228 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:22.800859 16129 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:18:22.889957 10798 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.022s	sys 0.030s
I20260709 13:18:22.804457 16297 raft_consensus.cc:2749] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:22.866470 10841 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:22.811195 16211 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:22.854367 16197 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:22.916560 16111 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:22.918036 10869 raft_consensus.cc:2393] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 14: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:22.918313 16197 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:22.876237 10946 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 14.
I20260709 13:18:22.919121 10797 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 14.
I20260709 13:18:22.822721 16303 consensus_queue.cc:1048] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:18:22.920269 10867 raft_consensus.cc:1240] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 13. Current term is 14. Ops: [11.6-13.7]
I20260709 13:18:22.794677 15870 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:22.850713 10695 catalog_manager.cc:5697] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 9 to 10, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 10 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:18:22.924607 10800 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:22.924463 10695 catalog_manager.cc:5697] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:18:22.857471 15859 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:22.927055 15859 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:22.929725 15859 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:22.784593 10875 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 14.
I20260709 13:18:22.933183 16206 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:22.937611 15883 consensus_queue.cc:260] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:22.939106 15883 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:22.944967 10800 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 11.
W20260709 13:18:22.945649 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 79273.9us
I20260709 13:18:22.946740 10841 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:22.949530 10841 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:18:22.892444 10798 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.022s	sys 0.030s
I20260709 13:18:22.870749 16219 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:22.892478 15888 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:18:22.905435 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:22.906497 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:22.974360 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:22.910197 16315 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:22.976920 10842 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:18:22.978946 10767 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.000s	sys 0.019s
W20260709 13:18:22.979374 10767 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.000s	sys 0.020s
W20260709 13:18:22.980887 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 586045us
I20260709 13:18:22.942600 16111 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:22.956991 10801 raft_consensus.cc:2410] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:18:22.985368 10799 raft_consensus.cc:2410] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:18:22.924484 15870 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:22.954260 16026 raft_consensus.cc:2749] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:22.889196 16129 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:22.916283 16174 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:22.976828 16299 raft_consensus.cc:2804] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:18:22.956902 16224 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.009s	sys 0.014s
I20260709 13:18:22.954478 16303 raft_consensus.cc:2749] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:22.978292 10797 raft_consensus.cc:1275] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:18:22.978226 16286 raft_consensus.cc:2749] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:18:22.963133 10841 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:23.029574 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 66549us
I20260709 13:18:23.024739 16147 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.031144 16147 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:23.034099 16147 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:23.034767 16026 consensus_queue.cc:1048] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:23.036427 16026 raft_consensus.cc:2749] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:23.037425 10842 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:23.038297 16285 raft_consensus.cc:2749] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:18:22.957036 10766 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.014s	sys 0.039s
I20260709 13:18:22.981144 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:23.040055 10794 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:18:23.040539 10766 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.014s	sys 0.039s
W20260709 13:18:23.041004 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 648979us
I20260709 13:18:22.989388 16179 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:18:23.028004 16224 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.009s	sys 0.014s
I20260709 13:18:23.050251 11477 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:23.021296 16129 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:23.027827 16299 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:23.047071 16179 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:23.114735 16299 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:23.212229 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:23.245504 10794 raft_consensus.cc:2410] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
W20260709 13:18:23.159034 10947 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.006s	sys 0.030s
W20260709 13:18:23.247664 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:23.264366 11508 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 17.
W20260709 13:18:23.264397 10947 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.011s	sys 0.031s
W20260709 13:18:23.248137 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 197988us
W20260709 13:18:23.048151 16210 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.006s	sys 0.016s
W20260709 13:18:23.269449 16210 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.742s	user 0.013s	sys 0.023s
W20260709 13:18:23.250490 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 588795us
W20260709 13:18:23.252244 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 210610us
W20260709 13:18:23.253401 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 200812us
I20260709 13:18:23.267398 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:23.271672 10916 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:23.275843 10916 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:23.254886 10950 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.007s	sys 0.023s
W20260709 13:18:23.226168 11511 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.014s	sys 0.014s
I20260709 13:18:23.267241 16258 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.267161 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:18:23.253044 16192 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.001s	sys 0.018s
I20260709 13:18:23.269615 16179 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:18:23.278584 11511 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.015s	sys 0.014s
I20260709 13:18:23.313266 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:23.314170 10946 raft_consensus.cc:2410] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 17 because replica has last-logged OpId of term: 15 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 6.
I20260709 13:18:23.315104 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:23.315676 10946 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 16.
I20260709 13:18:23.316435 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:23.318238 10941 raft_consensus.cc:1275] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:18:23.319291 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:23.321347 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:23.321973 10944 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 17.
W20260709 13:18:23.323189 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 47427.3us
I20260709 13:18:23.324417 10916 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:23.325479 10916 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:23.278366 10950 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.007s	sys 0.023s
I20260709 13:18:23.302850 11507 raft_consensus.cc:2393] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 14: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
W20260709 13:18:23.303476 10915 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.000s	sys 0.030s
W20260709 13:18:23.327657 10915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.000s	sys 0.031s
I20260709 13:18:23.331009 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:23.331777 11504 raft_consensus.cc:2393] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 19: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:23.332221 11477 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:23.334630 10916 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
W20260709 13:18:23.301473 16192 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.001s	sys 0.019s
I20260709 13:18:23.335489 11503 raft_consensus.cc:1275] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 13:18:23.335953 11503 pending_rounds.cc:85] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586: Aborting all ops after (but not including) 10
I20260709 13:18:23.336172 11503 pending_rounds.cc:107] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 13:18:23.336426 11503 raft_consensus.cc:2889] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:23.333413 10915 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:23.338485 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:18:23.339807 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 69158.3us
I20260709 13:18:23.346567 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:23.346715 10946 raft_consensus.cc:1275] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 13:18:23.347293 10946 pending_rounds.cc:85] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 10
I20260709 13:18:23.347503 11509 raft_consensus.cc:2410] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:18:23.347668 10946 pending_rounds.cc:107] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:18:23.347941 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:18:23.348763 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 76426.9us
I20260709 13:18:23.348802 10841 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:23.350926 10916 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:23.351125 10841 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:23.348659 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:23.352341 16014 raft_consensus.cc:2749] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:18:23.353029 16014 raft_consensus.cc:2804] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:23.353461 16014 raft_consensus.cc:697] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:23.346372 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:23.309057 16242 raft_consensus.cc:2804] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:23.360402 10946 raft_consensus.cc:2889] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:23.360324 10916 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:23.360811 10766 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:23.262336 16174 raft_consensus.cc:2749] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:23.342118 16270 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:23.308470 16206 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:23.338308 16210 raft_consensus.cc:2749] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:23.338140 16314 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:23.269980 16311 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:23.364192 10916 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:23.331733 15859 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:23.337790 16239 raft_consensus.cc:2749] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:23.355717 16064 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:23.354321 16014 consensus_queue.cc:237] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:23.359937 16242 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:23.365908 16209 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:23.308786 16258 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:23.366474 16270 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:23.377497 16147 consensus_queue.cc:1048] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 13:18:23.393190 10916 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:18:23.412828 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:23.412930 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:23.386523 16064 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:23.372157 16292 consensus_queue.cc:1048] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:18:23.395268 16242 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:23.425889 10798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2686) took 1286 ms. Trace:
I20260709 13:18:23.372290 16316 consensus_queue.cc:1059] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 13:18:23.411052 16239 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.431391 10798 rpcz_store.cc:276] 0709 13:18:22.141998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:22.224093 (+ 82095us) service_pool.cc:224] Handling call
0709 13:18:22.224472 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:22.224730 (+   258us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:18:22.276070 (+ 51340us) log.cc:844] Serialized 14 byte log entry
0709 13:18:23.410478 (+1134408us) log.cc:844] Serialized 14 byte log entry
0709 13:18:23.415180 (+  4702us) log.cc:844] Serialized 14 byte log entry
0709 13:18:23.424286 (+  9106us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:23.424810 (+   524us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:23.425102 (+   292us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.425469 (+   367us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:23.425822 (+   353us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11438,"prepare.queue_time_us":54588,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":44,"raft.queue_time_us":22349,"raft.run_cpu_time_us":310,"raft.run_wall_time_us":13513,"spinlock_wait_cycles":8146688,"thread_start_us":1127491,"threads_started":2,"wal-append.queue_time_us":1063806}
I20260709 13:18:23.439702 10871 raft_consensus.cc:1240] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:23.397363 16210 raft_consensus.cc:2749] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:18:23.394335 16224 raft_consensus.cc:2749] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:23.393356 16209 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:23.401602 16270 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:23.453372 15798 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:18:23.454068 15883 consensus_meta.cc:287] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.514s	user 0.008s	sys 0.000s
I20260709 13:18:23.458189 11503 raft_consensus.cc:1275] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:18:23.399212 16341 consensus_queue.cc:1059] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:18:23.398536 16356 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:23.459514 16356 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:23.459972 16356 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:23.400625 16220 raft_consensus.cc:2749] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:23.437690 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:23.374396 16206 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:23.429975 16275 raft_consensus.cc:2804] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:18:23.441164 16339 log.cc:927] Time spent T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
I20260709 13:18:23.441066 16357 consensus_queue.cc:1059] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:18:23.442289 16340 log.cc:927] Time spent T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
I20260709 13:18:23.442981 16239 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:23.444152 16336 log.cc:927] Time spent T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:18:23.447086 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:23.447732 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:23.430619 16354 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:23.468518 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:23.472530 16314 raft_consensus.cc:2749] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:23.475148 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:23.529469 10943 raft_consensus.cc:2410] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:18:23.429613 16064 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:23.386287 15859 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:23.531605 10948 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:23.501138 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:23.503582 10876 raft_consensus.cc:1275] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 13:18:23.505486 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:23.540983 11503 raft_consensus.cc:1240] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:23.467568 10871 raft_consensus.cc:1275] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:18:23.482867 16224 consensus_queue.cc:1048] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:18:23.453389 16209 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:23.474604 16147 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:23.534559 16344 log.cc:927] Time spent T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.158s	user 0.004s	sys 0.000s
W20260709 13:18:23.560163 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 209946us
W20260709 13:18:23.520367 16228 consensus_meta.cc:287] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.630s	user 0.006s	sys 0.005s
I20260709 13:18:23.568768 11505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56410 (request call id 855) took 1141 ms. Trace:
W20260709 13:18:23.572108 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 215709us
W20260709 13:18:23.573503 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:23.582474 10916 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:23.572093 11505 rpcz_store.cc:276] 0709 13:18:22.427199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:22.574334 (+147135us) service_pool.cc:224] Handling call
0709 13:18:22.588760 (+ 14426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:22.865432 (+276672us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:22.865607 (+   175us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:22.866068 (+   461us) log.cc:844] Serialized 26 byte log entry
0709 13:18:23.342758 (+476690us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:23.563453 (+220695us) log.cc:844] Serialized 14 byte log entry
0709 13:18:23.568315 (+  4862us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.568423 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:23.568485 (+    62us) raft_consensus.cc:1685] finished
0709 13:18:23.568504 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:23.568678 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9162,"raft.queue_time_us":39471,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":105,"thread_start_us":944182,"threads_started":3,"wal-append.queue_time_us":660826}
I20260709 13:18:23.583630 10948 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 12.
I20260709 13:18:23.573132 10841 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:23.481777 16210 consensus_queue.cc:1048] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.080s
I20260709 13:18:23.596391 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:23.598556 10869 raft_consensus.cc:2393] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 14: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:23.501298 16299 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:23.600481 10767 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:23.587834 16340 log.cc:927] Time spent T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:18:23.565613 16348 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.604643 10767 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:23.512468 16174 consensus_queue.cc:1059] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.063s
I20260709 13:18:23.606242 11505 raft_consensus.cc:1275] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:18:23.574319 16354 raft_consensus.cc:697] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:23.609128 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:23.586760 16001 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:23.612720 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 251985us
I20260709 13:18:23.613193 10947 raft_consensus.cc:1240] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:23.611824 11511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56410 (request call id 857) took 1135 ms. Trace:
I20260709 13:18:23.617267 11510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56404 (request call id 938) took 1401 ms. Trace:
I20260709 13:18:23.617658 11510 rpcz_store.cc:276] 0709 13:18:22.216279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:22.216402 (+   123us) service_pool.cc:224] Handling call
0709 13:18:22.256751 (+ 40349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:22.257173 (+   422us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:22.257200 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:22.261666 (+  4466us) log.cc:844] Serialized 26 byte log entry
0709 13:18:22.508515 (+246849us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b18004593a0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a550d1 00007f9c80e70a0f
0709 13:18:22.980243 (+471728us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:22.980280 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:22.980349 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:23.533871 (+553522us) spinlock_profiling.cc:243] Waited 1.68 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:18:23.617059 (+ 83188us) raft_consensus.cc:1685] finished
0709 13:18:23.617085 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:23.617205 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1305,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":7656,"spinlock_wait_cycles":29105792,"thread_start_us":471553,"threads_started":1,"wal-append.queue_time_us":1087521}
I20260709 13:18:23.602497 16210 consensus_queue.cc:1243] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer 12682eb70bc84c76879e04a8628ee586 log is divergent from this leader: its last log entry 17.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:18:23.618705 10841 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:23.620244 10949 raft_consensus.cc:1275] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:18:23.616775 11511 rpcz_store.cc:276] 0709 13:18:22.476364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:22.587502 (+111138us) service_pool.cc:224] Handling call
0709 13:18:22.587863 (+   361us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:22.596812 (+  8949us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:22.596865 (+    53us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:22.622428 (+ 25563us) log.cc:844] Serialized 52 byte log entry
0709 13:18:23.309391 (+686963us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:18:23.593460 (+284069us) log.cc:844] Serialized 14 byte log entry
0709 13:18:23.594136 (+   676us) log.cc:844] Serialized 14 byte log entry
0709 13:18:23.599010 (+  4874us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.599124 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:23.599182 (+    58us) raft_consensus.cc:1685] finished
0709 13:18:23.599204 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:23.611756 (+ 12552us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21882,"raft.queue_time_us":10297,"raft.run_cpu_time_us":450,"raft.run_wall_time_us":31198,"thread_start_us":848154,"threads_started":2,"wal-append.queue_time_us":714134}
I20260709 13:18:23.605799 16188 raft_consensus.cc:2749] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:23.604436 16311 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:23.625144 10949 pending_rounds.cc:85] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 7
I20260709 13:18:23.624711 10870 raft_consensus.cc:1240] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:23.638634 11509 raft_consensus.cc:1275] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:18:23.646080 10949 pending_rounds.cc:107] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:18:23.646608 10949 raft_consensus.cc:2889] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:23.639943 11478 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:23.575707 16342 consensus_queue.cc:1059] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 13:18:23.605568 16292 consensus_queue.cc:1059] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 13:18:23.626991 10841 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:23.566433 16147 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:23.592974 16220 consensus_queue.cc:1059] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:18:23.605928 16228 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:18:23.684015 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:23.631093 10766 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:23.600664 16129 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:23.701483 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 74600.6us
I20260709 13:18:23.625739 16357 raft_consensus.cc:2749] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:18:23.716279 16220 raft_consensus.cc:3055] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:18:23.716794 16220 raft_consensus.cc:740] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:23.717890 16220 consensus_queue.cc:260] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:23.702724 10842 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.9.116.190:36007, user_credentials={real_user=slave}} blocked reactor thread for 50371.6us
I20260709 13:18:23.614384 16192 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:23.720885 16192 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:23.606575 16182 consensus_queue.cc:1059] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 13:18:23.639879 16167 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.646715 16104 raft_consensus.cc:2804] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:23.649718 16252 consensus_queue.cc:1048] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 13:18:23.712220 10841 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:23.707551 16026 raft_consensus.cc:2749] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:18:23.738523 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:23.739387 10797 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:23.740674 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:18:23.713163 16363 log.cc:927] Time spent T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 13:18:23.714951 16292 raft_consensus.cc:3055] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:23.721401 16299 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:23.616250 15986 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:23.623687 16001 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:23.777946 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:23.703332 16174 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:18:23.725361 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 94384.6us
I20260709 13:18:23.779872 10766 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:23.640182 16368 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:23.649637 16349 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.649328 16315 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:23.613791 16343 consensus_queue.cc:1048] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.106s
I20260709 13:18:23.729830 16314 consensus_queue.cc:1059] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.006s
I20260709 13:18:23.733503 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:23.734048 16206 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:23.735140 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:23.737041 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:18:23.721467 16347 log.cc:927] Time spent T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.260s	user 0.000s	sys 0.001s
I20260709 13:18:23.744186 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:23.745769 11512 raft_consensus.cc:2393] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 18: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:23.743088 16303 raft_consensus.cc:2749] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:23.747474 16258 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:23.748337 16239 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:23.785605 16174 raft_consensus.cc:697] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 14 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:23.802749 16001 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:23.803303 16174 consensus_queue.cc:237] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:23.786108 16349 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:23.798650 16182 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.782600 15986 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:23.807269 16182 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:23.809304 15986 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:23.779069 16354 consensus_queue.cc:237] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:23.779397 15883 consensus_queue.cc:1048] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 13:18:23.786324 16299 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:23.811156 16182 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:23.811522 16299 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:23.812098 16182 raft_consensus.cc:2804] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:23.752676 16147 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:23.754053 16219 consensus_meta.cc:287] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.768s	user 0.012s	sys 0.007s
I20260709 13:18:23.738226 16167 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:23.728713 16104 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:23.829464 16104 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:23.764209 16129 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:23.765077 15798 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:23.833424 15798 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:23.835964 16209 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:23.836612 15798 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:23.765069 10873 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 14.
I20260709 13:18:23.726732 16370 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.769886 16311 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.641301 16350 consensus_queue.cc:1048] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.099s
I20260709 13:18:23.728497 16228 raft_consensus.cc:697] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:23.613791 16348 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:23.777508 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 195097us
I20260709 13:18:23.795717 16368 raft_consensus.cc:697] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:23.722522 16220 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:23.727471 16224 raft_consensus.cc:3055] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:23.813310 16192 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:23.782387 10797 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 15.
W20260709 13:18:23.781518 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 141695us
W20260709 13:18:23.793351 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:23.892807 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:23.893879 10794 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 18.
I20260709 13:18:23.894394 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:23.895077 10798 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 15.
I20260709 13:18:23.775672 16292 raft_consensus.cc:740] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:23.785468 16315 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:23.795089 15888 consensus_meta.cc:287] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.812s	user 0.014s	sys 0.000s
W20260709 13:18:23.814554 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 77064.9us
I20260709 13:18:23.819036 16219 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:23.838267 10874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 2126) took 1446 ms. Trace:
I20260709 13:18:23.905066 10874 rpcz_store.cc:276] 0709 13:18:22.392249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:22.511003 (+118754us) service_pool.cc:224] Handling call
0709 13:18:23.820373 (+1309370us) spinlock_profiling.cc:243] Waited 874 ms on lock 0x7b540006ccd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:23.838208 (+ 17835us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2009404672}
I20260709 13:18:23.812813 16182 raft_consensus.cc:697] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:23.830241 16179 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:23.799319 16375 raft_consensus.cc:3055] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:23.913480 11478 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:23.913807 16375 raft_consensus.cc:740] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:23.842464 10916 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:23.916112 11507 raft_consensus.cc:1217] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Deduplicated request from leader. Original: 13.10->[14.11-18.13]   Dedup: 14.11->[16.12-18.13]
W20260709 13:18:23.916247 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 73863.8us
I20260709 13:18:23.865008 16224 raft_consensus.cc:740] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:23.867180 16167 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:23.730177 16188 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:18:23.867254 16209 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:23.842209 16311 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:23.837846 16370 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:23.873009 16228 consensus_queue.cc:237] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:23.875777 16348 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:23.879558 16242 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:23.879731 16368 consensus_queue.cc:237] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:23.882669 16351 log.cc:927] Time spent T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.432s	user 0.002s	sys 0.002s
I20260709 13:18:23.834254 16270 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:23.884532 16358 log.cc:927] Time spent T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.433s	user 0.002s	sys 0.000s
I20260709 13:18:23.883903 16356 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:23.863198 16064 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:23.930482 16242 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:23.931721 16064 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:23.932876 16370 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:23.933622 16356 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:23.933972 16224 consensus_queue.cc:260] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:23.935635 16224 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:23.938314 16348 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:23.886857 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 107055us
I20260709 13:18:23.895545 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:23.896299 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:23.939754 10797 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 16.
I20260709 13:18:23.903851 11508 raft_consensus.cc:1275] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:18:23.907783 10841 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:23.887627 16286 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.898758 16343 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.911304 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:23.911440 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:23.914805 16375 consensus_queue.cc:260] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:23.916308 11478 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:23.916756 11507 pending_rounds.cc:85] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586: Aborting all ops after (but not including) 11
I20260709 13:18:23.917781 10916 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:23.710729 16364 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:23.918285 10868 raft_consensus.cc:2393] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 15: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:23.907493 16377 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.929914 10795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2702) took 1085 ms. Trace:
I20260709 13:18:23.913311 16179 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:23.917701 16192 consensus_queue.cc:1048] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 13:18:23.920984 16129 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:24.053139 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:18:24.311592 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 393873us
I20260709 13:18:24.312317 11507 pending_rounds.cc:107] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586: Aborting uncommitted NO_OP operation due to leader change: 17.12
I20260709 13:18:24.057857 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:24.312604 10916 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:24.312870 11507 raft_consensus.cc:2889] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:18:24.021294 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:23.969841 11503 raft_consensus.cc:1240] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:23.959596 10766 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:24.285611 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:24.316123 10873 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 14.
I20260709 13:18:24.256667 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:24.317150 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:24.317413 10876 raft_consensus.cc:2393] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 17: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:24.317839 10873 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 16.
I20260709 13:18:24.310482 11506 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 15.
I20260709 13:18:24.062951 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:24.061729 10842 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.116.129:39685 timed out after 1.550s (SENT) [suppressed 9 similar messages]
I20260709 13:18:24.060542 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:24.319205 10871 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 15.
W20260709 13:18:24.314167 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:24.320552 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:24.060009 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:24.320572 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:24.322100 10949 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:24.322742 10873 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:24.020799 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:24.060989 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:24.325075 10944 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 15.
I20260709 13:18:24.102413 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:24.327721 10945 raft_consensus.cc:2393] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 18: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:23.971273 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:24.057310 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:24.313876 10916 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:24.054329 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:24.326494 11477 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:24.328222 10795 rpcz_store.cc:276] 0709 13:18:22.844472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:22.850903 (+  6431us) service_pool.cc:224] Handling call
0709 13:18:22.851299 (+   396us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:22.851700 (+   401us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:22.851739 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:22.852306 (+   567us) log.cc:844] Serialized 26 byte log entry
0709 13:18:23.340878 (+488572us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:23.341021 (+   143us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.341290 (+   269us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:23.883972 (+542682us) spinlock_profiling.cc:243] Waited 22.3 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:18:23.929631 (+ 45659us) raft_consensus.cc:1685] finished
0709 13:18:23.929652 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:23.929852 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1261,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":109,"spinlock_wait_cycles":51250944,"thread_start_us":488030,"threads_started":1,"wal-append.queue_time_us":598516}
I20260709 13:18:24.332387 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:24.339401 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 379856us
I20260709 13:18:24.355856 10870 raft_consensus.cc:2393] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 16: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:24.283411 10950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2609) took 2030 ms. Trace:
I20260709 13:18:24.387346 10766 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:23.944998 10841 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:24.392802 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 79000.7us
I20260709 13:18:24.395298 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:24.395397 10876 raft_consensus.cc:1240] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 15. Current term is 16. Ops: []
W20260709 13:18:24.282580 16366 log.cc:927] Time spent T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.535s	user 0.001s	sys 0.002s
W20260709 13:18:24.100237 16289 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.723s	user 0.007s	sys 0.001s
W20260709 13:18:24.034329 16352 log.cc:927] Time spent T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.586s	user 0.000s	sys 0.001s
I20260709 13:18:24.398350 10876 raft_consensus.cc:1240] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:24.294553 16355 raft_consensus.cc:3055] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:24.051002 16373 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:24.145335 16197 raft_consensus.cc:3055] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:24.438100 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:24.387976 10950 rpcz_store.cc:276] 0709 13:18:22.252822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:22.452964 (+200142us) service_pool.cc:224] Handling call
0709 13:18:22.561758 (+108794us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:18:23.327329 (+765571us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:18:23.342828 (+ 15499us) log.cc:844] Serialized 14 byte log entry
0709 13:18:23.566670 (+223842us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:23.586769 (+ 20099us) log.cc:844] Serialized 52 byte log entry
0709 13:18:23.597854 (+ 11085us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:23.598103 (+   249us) log.cc:844] Serialized 14 byte log entry
0709 13:18:23.607227 (+  9124us) log.cc:844] Serialized 14 byte log entry
0709 13:18:23.648651 (+ 41424us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.648954 (+   303us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:23.723421 (+ 74467us) spinlock_profiling.cc:243] Waited 3.51 ms on lock 0x7b18001bc220. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc8522 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:18:24.277663 (+554242us) spinlock_profiling.cc:243] Waited 3.27 ms on lock 0x7b18001bc220. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc8557 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:18:24.278168 (+   505us) spinlock_profiling.cc:243] Waited 4.89 ms on lock 0x7b18001bc220. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc83f3 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:18:24.283163 (+  4995us) raft_consensus.cc:1685] finished
0709 13:18:24.283185 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:24.283349 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5416,"prepare.queue_time_us":34944,"prepare.run_cpu_time_us":136,"prepare.run_wall_time_us":3768,"raft.queue_time_us":411051,"raft.run_cpu_time_us":357,"raft.run_wall_time_us":15289,"spinlock_wait_cycles":26838912,"thread_start_us":953802,"threads_started":3,"wal-append.queue_time_us":252697}
I20260709 13:18:24.294571 15859 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:23.925482 16354 raft_consensus.cc:2749] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:18:24.211472 16345 log.cc:927] Time spent T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.770s	user 0.005s	sys 0.000s
I20260709 13:18:23.926574 15907 raft_consensus.cc:3055] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:18:24.419832 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 474910us
W20260709 13:18:24.215518 16359 log.cc:927] Time spent T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.764s	user 0.003s	sys 0.000s
W20260709 13:18:24.043813 16338 log.cc:927] Time spent T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.594s	user 0.002s	sys 0.000s
I20260709 13:18:23.938869 16348 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:24.202219 16111 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:24.048946 16145 raft_consensus.cc:3055] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:24.448794 16297 raft_consensus.cc:2804] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:18:24.453217 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 65954us
I20260709 13:18:24.311101 16224 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:24.439833 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:24.487599 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:24.459533 10795 raft_consensus.cc:1240] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:24.490993 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:24.493221 10795 raft_consensus.cc:1275] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:18:24.106086 16270 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:24.047421 16219 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:24.049189 16311 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:24.110404 16350 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:24.484691 16174 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:24.202831 16299 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:24.332697 16346 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:24.033387 16275 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:24.463511 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:24.259670 16315 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:24.015810 16356 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:18:24.468834 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 142422us
I20260709 13:18:24.469892 10797 raft_consensus.cc:1275] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:18:24.549712 10797 pending_rounds.cc:85] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 6
I20260709 13:18:24.550052 10797 pending_rounds.cc:107] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:18:24.550297 10797 raft_consensus.cc:2889] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:24.391431 16314 raft_consensus.cc:2804] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:24.470327 10841 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:24.552241 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 82015.7us
I20260709 13:18:24.474277 10916 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:24.476190 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:24.398618 16109 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:18:24.556700 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 82520.7us
W20260709 13:18:24.449888 16289 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.073s	user 0.011s	sys 0.014s
I20260709 13:18:24.463779 16188 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:24.414876 16242 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:18:24.480800 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 564546us
I20260709 13:18:24.333475 16377 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:24.143019 16104 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:24.562417 11478 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:24.468320 16373 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:24.447153 16355 raft_consensus.cc:740] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:24.394326 16343 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:24.496419 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 434704us
I20260709 13:18:23.926549 16292 consensus_queue.cc:260] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:24.488400 10792 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
I20260709 13:18:24.463922 16197 raft_consensus.cc:740] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:24.476215 15907 raft_consensus.cc:740] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:24.577414 10793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2701) took 1733 ms. Trace:
I20260709 13:18:24.580878 10793 rpcz_store.cc:276] 0709 13:18:22.844151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:22.853181 (+  9030us) service_pool.cc:224] Handling call
0709 13:18:22.853620 (+   439us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:22.854100 (+   480us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:22.854142 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:22.854911 (+   769us) log.cc:844] Serialized 52 byte log entry
0709 13:18:23.315410 (+460499us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:23.315530 (+   120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.315671 (+   141us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:24.214581 (+898910us) spinlock_profiling.cc:243] Waited 22.2 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:18:24.561057 (+346476us) raft_consensus.cc:1685] finished
0709 13:18:24.561078 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:24.577350 (+ 16272us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1545,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":101,"spinlock_wait_cycles":51002240,"thread_start_us":459858,"threads_started":1,"wal-append.queue_time_us":578422}
I20260709 13:18:24.583536 10799 raft_consensus.cc:1240] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:24.585139 10799 raft_consensus.cc:1275] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:18:24.556627 16299 raft_consensus.cc:2749] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:24.554669 16314 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:24.560487 16346 raft_consensus.cc:2749] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:18:24.395933 16286 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:24.484442 16111 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:24.498409 16228 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.028s	sys 0.087s
I20260709 13:18:23.945307 16316 raft_consensus.cc:2749] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:18:24.393916 16364 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:23.969193 16014 consensus_queue.cc:1048] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.499s
I20260709 13:18:24.484927 16145 raft_consensus.cc:740] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:24.482234 16129 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:24.495661 16147 raft_consensus.cc:2804] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:24.520676 11505 raft_consensus.cc:1240] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:24.516350 11512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56404 (request call id 941) took 1324 ms. Trace:
I20260709 13:18:24.676626 11512 rpcz_store.cc:276] 0709 13:18:23.191587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.267119 (+ 75532us) service_pool.cc:224] Handling call
0709 13:18:24.516011 (+1248892us) spinlock_profiling.cc:243] Waited 473 ms on lock 0x7b5400527954. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:24.516250 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1087535488}
I20260709 13:18:24.529021 10876 raft_consensus.cc:1240] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:24.678968 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:24.481112 15859 consensus_queue.cc:1048] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 13:18:24.544224 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:24.546418 10798 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
I20260709 13:18:24.545415 10801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2711) took 1505 ms. Trace:
I20260709 13:18:24.682304 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:24.502435 16297 raft_consensus.cc:697] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:24.560007 10841 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:24.514729 16220 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:24.500321 16224 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:24.684883 16398 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:24.547717 16275 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:24.571451 10797 raft_consensus.cc:1275] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:18:24.686631 16224 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:24.560770 10868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49290 (request call id 849) took 1146 ms. Trace:
I20260709 13:18:24.687906 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:24.572516 11478 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:24.571960 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:24.690121 10872 raft_consensus.cc:2393] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 18: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:24.576288 10867 raft_consensus.cc:1275] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:18:24.691344 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:24.691512 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:24.692348 10872 raft_consensus.cc:2393] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 15: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:24.693037 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:24.693441 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:24.580246 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:24.503388 10795 raft_consensus.cc:1275] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:18:24.599555 10796 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 15.
I20260709 13:18:24.601466 16396 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:24.601601 10842 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:24.605156 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:24.607340 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:24.569917 16109 raft_consensus.cc:697] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:24.575780 16174 raft_consensus.cc:2749] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:18:24.670620 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 181284us
I20260709 13:18:24.554883 16356 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:24.578110 16188 raft_consensus.cc:697] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:24.694550 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:24.702967 16174 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:24.703567 16174 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:24.703800 16392 consensus_queue.cc:1059] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:24.705725 16174 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:24.705850 16392 raft_consensus.cc:3055] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:24.706332 16174 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:24.703588 16391 consensus_queue.cc:1059] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 13:18:24.708103 16356 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:24.708305 16391 raft_consensus.cc:3055] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:24.708878 16391 raft_consensus.cc:740] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:18:24.710201 10766 connection.cc:256] RPC call timeout handler was delayed by -1.2370418289999634s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15200000000000002s for a response
W20260709 13:18:24.710595 10766 connection.cc:256] RPC call timeout handler was delayed by -1.1398418289998062s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1628s for a response
I20260709 13:18:24.711073 10948 raft_consensus.cc:1240] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:24.711284 10766 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:24.711565 10943 raft_consensus.cc:1240] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:24.712370 10766 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:24.713308 10944 raft_consensus.cc:1240] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:24.714459 10942 raft_consensus.cc:1240] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:24.715021 10945 raft_consensus.cc:1240] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:24.716101 16379 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:24.716796 16379 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:24.687505 10868 rpcz_store.cc:276] 0709 13:18:23.413990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.414142 (+   152us) service_pool.cc:224] Handling call
0709 13:18:24.560509 (+1146367us) spinlock_profiling.cc:243] Waited 454 ms on lock 0x7b540006b8d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:24.560694 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1044193152}
I20260709 13:18:24.719314 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:24.719581 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:24.721599 10868 raft_consensus.cc:1275] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:18:24.578632 15883 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:24.722435 15883 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:24.723840 16379 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:24.724677 16379 raft_consensus.cc:2804] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:24.725369 15883 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:24.569432 16285 consensus_queue.cc:1048] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:18:24.024065 16182 consensus_queue.cc:237] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:24.728039 16396 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:24.684650 16220 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:24.731914 16220 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:24.731513 16188 consensus_queue.cc:237] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:24.508684 10949 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 18.
I20260709 13:18:24.725466 16379 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:24.721510 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:24.735121 10944 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 14.
I20260709 13:18:24.736240 16379 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:24.735677 16109 consensus_queue.cc:237] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:24.675631 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:24.706473 16392 raft_consensus.cc:740] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:24.717413 16289 raft_consensus.cc:2749] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:24.739823 16392 consensus_queue.cc:260] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:24.740921 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:24.741823 10947 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
I20260709 13:18:24.741988 16392 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:24.744851 16398 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:24.747480 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:24.710011 16391 consensus_queue.cc:260] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:24.749616 16391 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:24.753526 16396 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:24.754305 16398 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:24.694329 10872 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 11.
I20260709 13:18:24.783658 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:24.784507 10876 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:18:24.788235 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 228257us
I20260709 13:18:24.681675 10801 rpcz_store.cc:276] 0709 13:18:23.040372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.040977 (+   605us) service_pool.cc:224] Handling call
0709 13:18:23.041322 (+   345us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:23.067181 (+ 25859us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:18:23.246098 (+178917us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:23.256190 (+ 10092us) log.cc:844] Serialized 52 byte log entry
0709 13:18:23.583333 (+327143us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:23.583375 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.583441 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:24.281969 (+698528us) spinlock_profiling.cc:243] Waited 29.7 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:18:24.539257 (+257288us) raft_consensus.cc:1685] finished
0709 13:18:24.539289 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:24.545327 (+  6038us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":123714,"raft.queue_time_us":25506,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":23999,"spinlock_wait_cycles":68424576,"thread_start_us":233777,"threads_started":1,"wal-append.queue_time_us":398141}
W20260709 13:18:24.790457 10916 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.9.116.190:36007, user_credentials={real_user=slave}} blocked reactor thread for 109382us
W20260709 13:18:24.669653 16228 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.724s	user 0.028s	sys 0.088s
I20260709 13:18:24.596129 16314 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:24.594957 16242 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:24.595225 16315 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:24.800705 10916 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:24.802292 10841 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:24.803591 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 202061us
I20260709 13:18:24.804371 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:24.808768 16197 consensus_queue.cc:260] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:24.810949 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:24.848712 10793 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 18.
I20260709 13:18:24.811947 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:24.850274 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 49643.1us
I20260709 13:18:24.812803 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:24.812870 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:24.851572 10916 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:24.813916 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:24.852402 10798 raft_consensus.cc:2393] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 15: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:24.814167 10873 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 17.
I20260709 13:18:24.815212 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:24.853063 10916 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:24.854570 10916 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:24.858567 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 2134) took 1323 ms. Trace:
I20260709 13:18:24.859390 10873 rpcz_store.cc:276] 0709 13:18:23.535130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.318675 (+783545us) service_pool.cc:224] Handling call
0709 13:18:24.858500 (+539825us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7296}
I20260709 13:18:24.815618 10876 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 15.
I20260709 13:18:24.817643 16286 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:24.882194 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 80040.6us
I20260709 13:18:24.819535 16111 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:24.822274 16297 consensus_queue.cc:237] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:24.793363 15907 consensus_queue.cc:260] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:24.824301 16145 consensus_queue.cc:260] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:24.798488 16285 consensus_queue.cc:1059] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:18:24.795476 16316 raft_consensus.cc:2749] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:18:24.681663 16147 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:24.543659 16350 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:24.827858 11505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56410 (request call id 860) took 1217 ms. Trace:
I20260709 13:18:24.928054 11505 rpcz_store.cc:276] 0709 13:18:23.610257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.610396 (+   139us) service_pool.cc:224] Handling call
0709 13:18:24.520404 (+910008us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:24.675374 (+154970us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:24.827608 (+152234us) spinlock_profiling.cc:243] Waited 910 ms on lock 0x7b5400527950. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:24.827788 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2092088064}
I20260709 13:18:24.834933 16375 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:24.841238 16377 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:24.842279 16348 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:24.786716 16129 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:24.940305 16364 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:24.940747 16129 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:24.900223 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:24.942173 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:24.919528 16197 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:24.917512 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:24.858634 16224 raft_consensus.cc:2749] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:18:24.880834 16228 raft_consensus.cc:2749] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:24.856837 16370 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:24.882121 16299 raft_consensus.cc:2749] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:18:24.890842 15859 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:24.891070 16356 raft_consensus.cc:2804] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:24.900349 16219 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:24.849565 15883 consensus_queue.cc:1048] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.070s
I20260709 13:18:24.849334 16242 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:24.935357 16373 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:24.937386 10796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39442 (request call id 754) took 1044 ms. Trace:
I20260709 13:18:24.969849 10796 rpcz_store.cc:276] 0709 13:18:23.892763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.895169 (+  2406us) service_pool.cc:224] Handling call
0709 13:18:24.937198 (+1042029us) spinlock_profiling.cc:243] Waited 660 ms on lock 0x7b54000c1f54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:24.937323 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1517987072}
I20260709 13:18:24.900815 10916 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:24.856678 16315 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:24.948199 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 375754us
I20260709 13:18:24.782099 16275 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:24.902737 16064 consensus_queue.cc:1048] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
W20260709 13:18:24.950042 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 237720us
I20260709 13:18:24.959581 16346 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:24.968124 16337 log.cc:927] Time spent T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 1.712s	user 0.015s	sys 0.006s
I20260709 13:18:24.848541 16355 consensus_queue.cc:260] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:24.904441 16311 consensus_queue.cc:1048] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:18:24.931149 16014 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:24.945510 16145 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:24.963960 16356 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:18:24.971432 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 70678.1us
I20260709 13:18:25.078495 10916 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:24.972883 11478 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:18:25.079456 16395 log.cc:927] Time spent T 00000000000000000000000000000000 P ae957c8fc66c4b6ba063d0c74cbf5484: Append to log took a long time: real 0.123s	user 0.003s	sys 0.000s
I20260709 13:18:24.976974 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:25.080083 10948 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 18.
I20260709 13:18:24.980701 10766 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:18:25.081547 16388 log.cc:927] Time spent T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.296s	user 0.004s	sys 0.000s
W20260709 13:18:25.081780 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 101192us
I20260709 13:18:24.927754 16147 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:24.983774 10942 raft_consensus.cc:1240] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:24.976625 11477 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:18:25.085332 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:25.087188 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 110641us
I20260709 13:18:25.090320 11477 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:18:24.994378 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 111492us
I20260709 13:18:24.977758 16349 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:24.957231 16370 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:25.096680 10941 raft_consensus.cc:1240] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:24.995682 10793 raft_consensus.cc:2393] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 14: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:24.987780 15859 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:24.980604 16286 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:25.084712 16356 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:24.978180 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:25.156268 10944 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 14.
I20260709 13:18:25.157500 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:18:25.083292 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 110488us
I20260709 13:18:25.083556 10766 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:25.159226 11478 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:25.159525 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 76852us
I20260709 13:18:25.158720 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:25.161482 10944 raft_consensus.cc:2393] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 16: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:25.162225 16375 raft_consensus.cc:2749] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:18:25.162715 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:25.162945 16375 consensus_queue.cc:1059] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:25.163410 10944 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
I20260709 13:18:25.164049 16375 raft_consensus.cc:3055] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:25.164431 16375 raft_consensus.cc:740] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:25.094910 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:25.165850 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:25.166431 10944 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
I20260709 13:18:25.166831 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:25.167217 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:25.168269 10944 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:25.095476 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:25.100412 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:25.097803 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:25.169955 10941 raft_consensus.cc:2393] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 12: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:25.101114 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:25.104074 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:25.171996 16311 consensus_queue.cc:1048] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:25.103958 16162 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:18:25.172712 16162 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:25.173092 16162 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:25.089340 16129 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:25.087190 16403 raft_consensus.cc:2749] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:25.083829 16402 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.075093 16350 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:25.121028 16242 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:25.122510 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:18:25.124137 10800 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39442 (request call id 744) took 1389 ms (client timeout 1705 ms). Trace:
I20260709 13:18:25.187042 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:25.188002 10800 rpcz_store.cc:269] 0709 13:18:23.734699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.734854 (+   155us) service_pool.cc:224] Handling call
0709 13:18:25.123921 (+1389067us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b5400387a54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:25.124051 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3010256256}
W20260709 13:18:25.193696 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:25.195703 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:25.125476 16286 raft_consensus.cc:697] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 17 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:25.075892 16104 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:25.077260 16343 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:25.128029 16219 raft_consensus.cc:2749] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:25.165266 16375 consensus_queue.cc:260] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:25.203379 10944 raft_consensus.cc:1240] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:18:25.183982 16179 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:25.128546 16370 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:25.216754 10947 raft_consensus.cc:1240] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:24.993438 16346 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:25.129499 16379 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:24.995173 16014 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:25.130271 15859 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:25.130342 15888 consensus_queue.cc:1059] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:18:25.132201 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:25.195516 16393 consensus_queue.cc:1048] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:18:25.222427 15888 raft_consensus.cc:3055] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:25.222813 15888 raft_consensus.cc:740] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:25.223598 15888 consensus_queue.cc:260] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:25.224578 11477 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:25.224851 15888 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:25.226001 11503 raft_consensus.cc:1240] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 13:18:25.227133 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:25.227286 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:25.228333 16349 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:25.228914 16349 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:25.230973 16350 raft_consensus.cc:2804] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:25.231424 16350 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:25.231772 16350 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:25.189002 16377 consensus_queue.cc:1048] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:18:25.234544 11507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56404 (request call id 951) took 1334 ms. Trace:
I20260709 13:18:25.211190 16409 consensus_queue.cc:1059] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 13:18:25.132812 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:25.133395 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:25.134969 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:25.236965 10801 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
I20260709 13:18:25.237793 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:25.238621 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:25.136230 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:25.136873 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:25.239814 10876 raft_consensus.cc:2393] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 13: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:25.140087 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:25.075862 16373 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:25.077528 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 176343us
W20260709 13:18:25.145031 10869 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 2139) took 1558 ms (client timeout 1699 ms). Trace:
I20260709 13:18:25.172302 11508 raft_consensus.cc:2393] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 12: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
W20260709 13:18:25.244283 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:25.098230 16348 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:18:25.097616 16355 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:18:25.177398 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:25.178413 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:25.252765 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:25.253239 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:25.184677 11478 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:25.102067 16064 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:25.196583 10800 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 14.
I20260709 13:18:25.186877 16129 raft_consensus.cc:697] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 15 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:25.127810 16315 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:25.194852 16174 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:25.193382 16402 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:25.243289 10801 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154"
dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
mode: GRACEFUL
new_leader_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:40148
W20260709 13:18:25.244318 10869 rpcz_store.cc:269] 0709 13:18:23.587108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.599147 (+ 12039us) service_pool.cc:224] Handling call
0709 13:18:25.144785 (+1545638us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b5400109ad4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:25.144957 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2840839424}
W20260709 13:18:25.245045 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 80104.4us
I20260709 13:18:25.234973 11507 rpcz_store.cc:276] 0709 13:18:23.900159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.904728 (+  4569us) service_pool.cc:224] Handling call
0709 13:18:23.915876 (+ 11148us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:18:24.346806 (+430930us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:18:24.611018 (+264212us) spinlock_profiling.cc:243] Waited 16.2 ms on lock 0x7b1800081520. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc5455 00007f9c848ad3c7 00007f9c80ebf9a7 00007f9c80ecf0bf 00007f9c80ecf038
0709 13:18:24.914910 (+303892us) log.cc:844] Serialized 14 byte log entry
0709 13:18:25.129294 (+214384us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:25.129864 (+   570us) log.cc:844] Serialized 52 byte log entry
0709 13:18:25.130033 (+   169us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:25.130069 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:25.130140 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:25.234232 (+104092us) raft_consensus.cc:1685] finished
0709 13:18:25.234336 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:25.234479 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75473,"prepare.queue_time_us":644271,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":59619,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":22521,"spinlock_wait_cycles":37330176,"thread_start_us":483297,"threads_started":2,"wal-append.queue_time_us":306062}
W20260709 13:18:25.250985 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 77700us
I20260709 13:18:25.264406 10949 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
W20260709 13:18:25.264911 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 80340us
W20260709 13:18:25.266554 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 57842.7us
I20260709 13:18:25.274695 10916 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:25.276027 16315 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:25.276465 16315 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:25.276434 10916 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:25.277241 16315 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:25.278491 16242 consensus_queue.cc:1048] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:18:25.279217 16219 consensus_queue.cc:1048] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:25.279623 10948 raft_consensus.cc:1240] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 12. Current term is 13. Ops: []
W20260709 13:18:25.253783 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:25.241729 10766 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:25.281262 10916 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:25.282382 10766 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:25.277768 10841 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:18:25.299638 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 45382us
I20260709 13:18:25.218636 16343 consensus_queue.cc:1059] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 13:18:25.242995 16275 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:25.217952 16104 consensus_queue.cc:1059] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.054s
W20260709 13:18:25.243541 16289 consensus_queue.cc:1175] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "cb0428cc55984daba1ba63744a5102a2" responder_term: 12 status { last_received { term: 11 index: 7 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 11. Current term is 12. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:18:25.269156 16228 raft_consensus.cc:3055] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:18:25.348516 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 72835.2us
I20260709 13:18:25.219933 16179 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:25.334709 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:25.267642 16174 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:25.355240 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:25.268322 15883 consensus_queue.cc:1059] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.076s
I20260709 13:18:25.210721 16414 consensus_queue.cc:1048] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.079s
I20260709 13:18:25.247998 16368 consensus_queue.cc:1059] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.075s
I20260709 13:18:25.267891 16349 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:25.349562 10948 raft_consensus.cc:2393] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 13: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:25.351253 11478 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:25.334931 16220 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:25.342482 16219 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:25.317595 16109 raft_consensus.cc:2804] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:25.317734 16211 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:25.314679 16064 raft_consensus.cc:697] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 15 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:25.272032 16393 consensus_queue.cc:1048] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.085s
I20260709 13:18:25.250439 16391 consensus_queue.cc:1059] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.073s
I20260709 13:18:25.378762 10916 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:25.379958 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:25.421393 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:25.383270 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:25.423116 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:25.423297 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:25.384163 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:25.424712 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:25.425473 10871 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 14.
I20260709 13:18:25.343560 16014 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:25.253633 16373 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:25.390106 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:25.391006 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:25.348277 16228 raft_consensus.cc:740] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:25.361112 16174 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:18:25.376612 16314 consensus_meta.cc:287] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.533s	user 0.009s	sys 0.005s
W20260709 13:18:25.394639 11478 connection.cc:256] RPC call timeout handler was delayed by -1.3128368769998815s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1705s for a response
I20260709 13:18:25.324244 16286 consensus_queue.cc:237] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:25.395550 10915 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:25.430223 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:25.440610 15859 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:25.441053 16428 consensus_queue.cc:1059] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.064s
I20260709 13:18:25.395208 16109 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:25.391680 16219 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:25.457042 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:25.430647 16316 raft_consensus.cc:3055] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:25.456081 16109 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:18:25.468858 16416 log.cc:927] Time spent T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:18:25.468739 16162 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:25.451822 16242 raft_consensus.cc:2749] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:25.471091 11505 raft_consensus.cc:1240] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 19. Current term is 20. Ops: []
I20260709 13:18:25.446924 16420 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:25.472069 16162 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:25.472404 16420 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:25.473031 16420 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:25.473222 16162 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.473997 16162 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:25.476219 16162 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:25.446954 16430 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:25.479406 16430 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:25.479856 16430 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:25.483460 16316 raft_consensus.cc:740] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:25.484722 16316 consensus_queue.cc:260] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:25.394913 16349 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:25.398767 16211 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:25.434553 16408 log.cc:927] Time spent T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.156s	user 0.000s	sys 0.000s
I20260709 13:18:25.446905 16425 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.417832 16147 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:25.430837 16419 consensus_queue.cc:1048] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.080s
I20260709 13:18:25.492095 16372 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.395742 16220 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:25.495347 15859 raft_consensus.cc:2804] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:25.486284 16316 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:25.454027 16403 raft_consensus.cc:3055] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:25.403553 16145 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.394515 16129 consensus_queue.cc:237] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:25.446939 16401 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:25.486493 16219 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:25.452836 16314 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:25.457798 15907 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:18:25.456676 16418 log.cc:927] Time spent T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 13:18:25.464906 16179 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:25.503775 16349 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:25.505003 16211 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:25.522608 16425 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:25.540251 16346 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:25.539322 16403 raft_consensus.cc:740] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:25.541791 16379 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:25.519891 16348 raft_consensus.cc:697] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 18 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
W20260709 13:18:25.545082 16197 consensus_meta.cc:287] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.598s	user 0.012s	sys 0.000s
I20260709 13:18:25.528900 16419 consensus_queue.cc:1243] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer 12682eb70bc84c76879e04a8628ee586 log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:18:25.560604 16220 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:25.561054 16427 log.cc:927] Time spent T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:18:25.551239 16349 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:25.566037 16350 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:25.567289 16403 consensus_queue.cc:260] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:25.545898 16401 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:25.568202 16314 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:25.570240 16373 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:25.539286 16372 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:25.571184 16402 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:25.522033 16228 consensus_queue.cc:260] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:25.576148 16424 log.cc:927] Time spent T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:18:25.577235 16377 raft_consensus.cc:3055] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:18:25.578748 16405 log.cc:927] Time spent T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.465s	user 0.000s	sys 0.004s
I20260709 13:18:25.580237 16425 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:25.581065 16348 consensus_queue.cc:237] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:25.574221 16356 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:25.602175 16412 log.cc:927] Time spent T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.486s	user 0.005s	sys 0.001s
I20260709 13:18:25.584602 16401 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:25.606405 16403 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:25.601666 16377 raft_consensus.cc:740] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:25.611816 16377 consensus_queue.cc:260] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:25.613369 16377 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:25.613865 16356 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:25.617776 16372 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:25.625870 16430 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:25.628378 16350 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:25.629138 16430 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:25.636909 16420 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:25.652892 16420 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:25.695155 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:25.318194 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:25.703733 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:25.715247 10798 raft_consensus.cc:1275] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:18:25.720796 10801 raft_consensus.cc:606] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Received request to transfer leadership to TS cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:25.387048 10867 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
I20260709 13:18:25.390286 10869 raft_consensus.cc:1240] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:18:25.428802 10871 raft_consensus.cc:1240] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:25.782897 10870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 866) took 1247 ms. Trace:
W20260709 13:18:25.448395 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 52945.5us
I20260709 13:18:25.721535 10801 raft_consensus.cc:614] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:18:25.754966 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:25.539201 16064 consensus_queue.cc:237] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:25.785892 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:25.786402 10915 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:25.756798 16197 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.759248 16228 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:25.759531 16220 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:25.541932 15859 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:25.768725 16315 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:25.545406 16145 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:25.784638 16242 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:25.784298 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 405431us
W20260709 13:18:25.790665 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:25.811506 16242 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:25.817484 16147 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:25.800200 10876 raft_consensus.cc:1240] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 14. Current term is 15. Ops: []
W20260709 13:18:25.818341 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 540429us
W20260709 13:18:25.799377 10941 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.014s	sys 0.007s
W20260709 13:18:25.819931 10941 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.015s	sys 0.008s
I20260709 13:18:25.820293 10841 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:25.807191 15859 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:25.805343 16439 raft_consensus.cc:2749] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:18:25.817745 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:25.822504 16425 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:25.872643 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:25.875392 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:25.876536 10869 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 18.
I20260709 13:18:25.878518 10869 raft_consensus.cc:1275] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:18:25.832933 16350 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:25.836210 16403 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.929579 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:25.839222 10841 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:25.839890 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:25.843539 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:25.843133 16197 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:25.844365 10875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 862) took 1320 ms. Trace:
I20260709 13:18:25.844527 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:25.823320 10944 raft_consensus.cc:1240] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:25.783558 10870 rpcz_store.cc:276] 0709 13:18:24.535625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.535762 (+   137us) service_pool.cc:224] Handling call
0709 13:18:24.536159 (+   397us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:24.592758 (+ 56599us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:18:24.681234 (+ 88476us) log.cc:844] Serialized 14 byte log entry
0709 13:18:24.984179 (+302945us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:24.991096 (+  6917us) log.cc:844] Serialized 26 byte log entry
0709 13:18:24.991306 (+   210us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:24.991362 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:25.400378 (+409016us) spinlock_profiling.cc:243] Waited 56.2 ms on lock 0x7b5400078854. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:25.400455 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:25.776606 (+376151us) raft_consensus.cc:1685] finished
0709 13:18:25.782260 (+  5654us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:25.782648 (+   388us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15346,"prepare.queue_time_us":244685,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":29746,"raft.run_cpu_time_us":260,"raft.run_wall_time_us":6410,"spinlock_wait_cycles":129272704,"thread_start_us":147817,"threads_started":2,"wal-append.queue_time_us":285103}
I20260709 13:18:25.822569 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:18:25.855744 10946 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.009s	sys 0.013s
I20260709 13:18:25.863025 16220 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:25.941468 10798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2150) took 1127 ms. Trace:
I20260709 13:18:25.943257 16220 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:25.887204 15888 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.943576 16403 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:25.944239 15888 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:25.945962 10949 raft_consensus.cc:1240] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:25.946318 16403 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:25.891721 11507 raft_consensus.cc:1240] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 13. Current term is 15. Ops: [13.11-13.11]
I20260709 13:18:25.892328 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:25.946751 15888 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:25.902884 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:25.956106 10947 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:25.846887 16315 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:25.915483 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:25.942004 10798 rpcz_store.cc:276] 0709 13:18:24.814322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.945306 (+130984us) service_pool.cc:224] Handling call
0709 13:18:24.945586 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:25.716132 (+770546us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:25.941057 (+224925us) spinlock_profiling.cc:243] Waited 769 ms on lock 0x7b54000bd1d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:25.941373 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1767713024}
I20260709 13:18:25.895154 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:25.936177 16350 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:25.937812 16197 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:25.923164 16425 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:25.938715 16348 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.966102 16348 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:25.966998 15883 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:25.967357 16350 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:25.968106 16425 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:25.967780 15883 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:25.938773 10868 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 13.
I20260709 13:18:25.975231 16197 consensus_queue.cc:1059] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:25.976964 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:25.977341 16220 raft_consensus.cc:3055] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:25.977944 16220 raft_consensus.cc:740] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:25.979475 16220 consensus_queue.cc:260] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:25.970444 16392 raft_consensus.cc:3055] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:25.939538 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:25.940563 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:18:25.941066 10946 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.706s	user 0.009s	sys 0.013s
I20260709 13:18:25.947166 16145 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:25.947476 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:25.951287 16167 raft_consensus.cc:3055] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:25.948153 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:25.948844 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:25.933979 10875 rpcz_store.cc:276] 0709 13:18:24.523890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.525176 (+  1286us) service_pool.cc:224] Handling call
0709 13:18:24.525615 (+   439us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:24.856381 (+330766us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:24.856411 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:24.881121 (+ 24710us) log.cc:844] Serialized 26 byte log entry
0709 13:18:24.979749 (+ 98628us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:24.979802 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:25.401166 (+421364us) spinlock_profiling.cc:243] Waited 272 ms on lock 0x7b54001095d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:25.401231 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:25.582366 (+181135us) spinlock_profiling.cc:243] Waited 9.8 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:18:25.781253 (+198887us) raft_consensus.cc:1685] finished
0709 13:18:25.793839 (+ 12586us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:25.844181 (+ 50342us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58184,"raft.queue_time_us":58750,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":39639,"spinlock_wait_cycles":648386304,"thread_start_us":51389,"threads_started":1,"wal-append.queue_time_us":190577}
I20260709 13:18:25.823930 16391 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:25.954401 11505 raft_consensus.cc:1275] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:18:25.938516 10793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2151) took 1123 ms. Trace:
I20260709 13:18:25.922176 16219 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:25.963035 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:25.977981 10949 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 17.
I20260709 13:18:25.981051 16220 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:26.024794 16145 consensus_queue.cc:1059] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:18:26.026738 16350 raft_consensus.cc:3055] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:26.027472 16350 raft_consensus.cc:740] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:26.028936 16350 consensus_queue.cc:260] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:25.982124 16392 raft_consensus.cc:740] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:26.010744 11511 raft_consensus.cc:2410] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
W20260709 13:18:26.010998 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 188202us
I20260709 13:18:26.031929 16392 consensus_queue.cc:260] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:26.003603 10793 rpcz_store.cc:276] 0709 13:18:24.814535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.899713 (+ 85178us) service_pool.cc:224] Handling call
0709 13:18:25.938019 (+1038306us) spinlock_profiling.cc:243] Waited 49 ms on lock 0x7b54003ccb54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:25.938313 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":112770432}
I20260709 13:18:26.010485 10942 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 17.
I20260709 13:18:26.015143 16316 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:26.019335 11503 raft_consensus.cc:2368] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 10. Current term is 11.
W20260709 13:18:26.019922 16406 log.cc:927] Time spent T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.917s	user 0.001s	sys 0.025s
I20260709 13:18:26.021888 10947 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 15.
I20260709 13:18:26.026943 11507 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:18:26.032205 16109 consensus_meta.cc:287] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.563s	user 0.015s	sys 0.002s
I20260709 13:18:26.011613 16391 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:25.982352 16192 raft_consensus.cc:3055] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:26.014395 16219 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:26.005218 16167 raft_consensus.cc:740] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:26.033161 16392 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:26.039764 11510 raft_consensus.cc:2393] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 16: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:25.975224 16348 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:26.026361 16211 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:25.970463 15883 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:26.047206 10841 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:26.041097 16192 raft_consensus.cc:740] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:26.043414 16425 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:26.074064 16425 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:26.075268 16228 raft_consensus.cc:2749] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:26.076049 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:26.084789 16192 consensus_queue.cc:260] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:26.086385 16192 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:26.092412 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:26.093540 10946 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 18.
I20260709 13:18:26.099222 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:26.099558 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:26.099999 10793 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
I20260709 13:18:26.100167 10945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41020 (request call id 2640) took 1004 ms. Trace:
I20260709 13:18:26.100353 10797 raft_consensus.cc:2376] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 11. Re-sending same reply.
I20260709 13:18:26.100622 10945 rpcz_store.cc:276] 0709 13:18:25.095465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:25.103595 (+  8130us) service_pool.cc:224] Handling call
0709 13:18:26.099849 (+996254us) spinlock_profiling.cc:243] Waited 619 ms on lock 0x7b54000a96d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:26.100001 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1424593152}
I20260709 13:18:26.101290 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:26.101961 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:26.103022 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:26.103174 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:26.103901 10792 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 16.
I20260709 13:18:26.103586 16167 consensus_queue.cc:260] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:26.105285 16167 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:26.105384 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:26.105628 11512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56410 (request call id 863) took 1005 ms. Trace:
I20260709 13:18:26.106806 10947 raft_consensus.cc:1275] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:18:26.104063 10798 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
I20260709 13:18:26.082213 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:26.080093 10841 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:26.108155 10793 raft_consensus.cc:1240] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:26.106019 11512 rpcz_store.cc:276] 0709 13:18:25.099769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:25.100872 (+  1103us) service_pool.cc:224] Handling call
0709 13:18:26.094589 (+993717us) spinlock_profiling.cc:243] Waited 786 ms on lock 0x7b54001f90d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:26.105446 (+ 10857us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1806973184}
I20260709 13:18:26.109527 10942 raft_consensus.cc:2393] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 15: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:26.106187 10945 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 14.
I20260709 13:18:26.115551 10792 raft_consensus.cc:1275] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:18:26.116111 10943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 2340) took 1399 ms. Trace:
I20260709 13:18:26.116529 10943 rpcz_store.cc:276] 0709 13:18:24.716839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.722289 (+  5450us) service_pool.cc:224] Handling call
0709 13:18:24.722718 (+   429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:24.992670 (+269952us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:24.992705 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:24.992750 (+    45us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:24.992785 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:26.115676 (+1122891us) spinlock_profiling.cc:243] Waited 270 ms on lock 0x7b5400399c54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:26.115742 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:26.115937 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":620311296}
I20260709 13:18:26.124697 16375 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:26.110021 15883 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:26.126451 15883 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:26.129141 15883 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:26.129900 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:26.130354 16349 consensus_meta.cc:287] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.567s	user 0.010s	sys 0.007s
I20260709 13:18:26.130939 16349 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:26.132848 16391 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:26.134718 10949 raft_consensus.cc:1275] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:18:26.135339 11509 raft_consensus.cc:2393] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 17: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:26.042089 16174 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:26.135921 16428 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.136602 16428 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:26.137943 16174 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:26.135221 16211 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:26.042459 16109 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:26.152009 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:26.155560 11477 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:26.161057 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:26.162387 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:26.164001 10799 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
I20260709 13:18:26.167791 11477 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:26.168166 15888 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:26.168677 15888 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:26.168895 11477 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:26.169085 15888 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:26.169971 16162 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:26.170362 16162 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:26.170720 16162 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:26.171173 10942 raft_consensus.cc:1240] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:26.175004 16348 raft_consensus.cc:2804] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:26.175673 16348 raft_consensus.cc:697] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 15 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
W20260709 13:18:26.175791 16427 log.cc:927] Time spent T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.368s	user 0.003s	sys 0.003s
I20260709 13:18:26.176765 16348 consensus_queue.cc:237] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:26.180590 10946 raft_consensus.cc:1275] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:18:26.182001 15883 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:18:26.182490 15883 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:26.182910 15883 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:26.186002 16292 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:26.141515 16349 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:26.190923 10947 raft_consensus.cc:1240] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:26.194051 16174 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.195664 16174 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:26.197937 16174 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:26.199515 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 119626us
I20260709 13:18:26.199888 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:26.200450 10841 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:26.200496 11477 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:26.201891 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:26.202708 10946 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 17.
I20260709 13:18:26.203989 10766 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:26.204875 16428 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:26.205499 16428 raft_consensus.cc:2749] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:18:26.205933 16220 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:26.206427 16220 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:26.206856 16167 consensus_queue.cc:1048] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:18:26.207255 16401 consensus_meta.cc:287] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.567s	user 0.013s	sys 0.006s
I20260709 13:18:26.207818 16401 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:26.209434 16270 raft_consensus.cc:2804] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:26.209903 16270 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:26.210259 16401 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:26.210312 16270 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:26.211171 16242 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:26.211489 10949 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 12.
I20260709 13:18:26.211207 10795 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8"
dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
mode: GRACEFUL
new_leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
 from {username='slave'} at 127.0.0.1:40148
I20260709 13:18:26.199527 16350 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:26.212041 16220 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:26.215963 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:26.215698 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:26.221001 11477 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:26.224413 10795 raft_consensus.cc:606] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Received request to transfer leadership to TS 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:26.202715 10841 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:26.224395 11507 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
I20260709 13:18:26.228055 10795 raft_consensus.cc:614] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:26.224396 11478 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:26.254056 16428 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.254719 16428 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:26.261220 16428 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:26.262444 15859 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:26.263164 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:26.258694 10842 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:26.268369 10948 raft_consensus.cc:2393] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 16: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:26.229244 16432 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.241323 16316 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:26.240579 16109 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:26.243186 16129 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:26.222903 16174 consensus_queue.cc:1048] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
W20260709 13:18:26.275672 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:26.291088 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 99368.7us
I20260709 13:18:26.293203 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:26.277460 16211 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.293880 10947 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:26.277963 16129 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:26.297670 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 95038.2us
I20260709 13:18:26.299283 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:18:26.303655 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:26.304071 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:26.319990 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:26.310729 10945 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:26.300338 16316 raft_consensus.cc:2749] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:18:26.301720 16401 raft_consensus.cc:2749] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:18:26.330711 16275 consensus_queue.cc:1059] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 13:18:26.340078 15907 raft_consensus.cc:2749] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:26.342530 16275 raft_consensus.cc:3055] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:26.343031 16275 raft_consensus.cc:740] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:26.344187 16275 consensus_queue.cc:260] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:26.345784 16275 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:26.352746 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:26.359074 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:26.361492 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 167932us
W20260709 13:18:26.361675 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 103181us
I20260709 13:18:26.360025 10875 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
I20260709 13:18:26.355363 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:26.310549 16167 consensus_queue.cc:1048] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:18:26.302474 16129 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:26.373755 10842 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:26.378685 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:26.381137 10801 raft_consensus.cc:1240] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:26.308842 16432 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:26.353542 16211 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:26.390654 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:26.391209 10945 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
I20260709 13:18:26.400411 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:26.402566 10793 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 16.
I20260709 13:18:26.392796 16316 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:26.420408 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:26.400619 15859 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:26.425511 10947 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 17.
I20260709 13:18:26.414922 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:26.428824 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 48890.9us
I20260709 13:18:26.413851 16316 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:26.461109 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:26.425266 16392 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:26.464666 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:26.466918 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:18:26.469305 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 105408us
I20260709 13:18:26.462494 16316 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:26.468645 10801 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 12.
I20260709 13:18:26.479554 10842 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:26.477872 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:26.480786 11507 raft_consensus.cc:2393] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 18: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:26.481424 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:26.482533 10867 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
I20260709 13:18:26.487392 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:26.486148 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:26.462317 15888 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:26.495433 10801 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
I20260709 13:18:26.496582 15888 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:26.496912 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:26.497743 10869 raft_consensus.cc:2393] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 18: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:26.498878 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:26.499579 10948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41014 (request call id 2345) took 1213 ms. Trace:
I20260709 13:18:26.496548 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:26.499991 10948 rpcz_store.cc:276] 0709 13:18:25.286503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:25.442950 (+156447us) service_pool.cc:224] Handling call
0709 13:18:26.499311 (+1056361us) spinlock_profiling.cc:243] Waited 773 ms on lock 0x7b54002a53d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:26.499508 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1778979072}
I20260709 13:18:26.501437 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:26.502235 11510 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 14.
I20260709 13:18:26.491811 10867 raft_consensus.cc:1275] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:18:26.503049 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:26.503474 16428 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:26.503911 16428 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:26.504354 16428 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:26.488314 16431 consensus_queue.cc:1059] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.019s
I20260709 13:18:26.507402 16431 raft_consensus.cc:3055] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:18:26.507915 16431 raft_consensus.cc:740] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:26.508240 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:26.508833 16431 consensus_queue.cc:260] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:26.509528 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:26.510202 16431 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:26.511271 10796 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:26.502930 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:26.485798 16368 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.491591 16162 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:26.507320 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:26.496522 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:26.515106 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:26.520440 10793 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 17.
I20260709 13:18:26.524852 16368 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:26.525890 16270 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:26.527387 16368 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:26.528168 16270 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:26.528461 16162 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:26.473515 15907 consensus_queue.cc:1048] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 13:18:26.532411 10875 raft_consensus.cc:2393] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 16: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:26.533469 16432 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:26.533867 11504 raft_consensus.cc:1240] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 16. Current term is 17. Ops: [14.12-16.13]
I20260709 13:18:26.535145 10801 raft_consensus.cc:2368] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 16. Current term is 17.
I20260709 13:18:26.536137 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:26.535174 11505 raft_consensus.cc:2393] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 18: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:26.537220 10801 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca"
dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
mode: GRACEFUL
new_leader_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:40148
I20260709 13:18:26.539395 11508 raft_consensus.cc:2393] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 15: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:26.540160 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:26.540421 16426 consensus_queue.cc:1059] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:26.541010 11511 raft_consensus.cc:2393] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 18: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:26.541774 16426 raft_consensus.cc:3055] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:26.542197 16426 raft_consensus.cc:740] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:18:26.544252 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:26.544790 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:26.545131 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:26.525374 11512 raft_consensus.cc:2393] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 12: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
W20260709 13:18:26.528064 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 120631us
I20260709 13:18:26.547437 10872 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 14.
W20260709 13:18:26.516721 16470 log.cc:927] Time spent T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.072s	user 0.001s	sys 0.001s
I20260709 13:18:26.548110 10842 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:26.550966 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:26.546689 10801 raft_consensus.cc:606] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Received request to transfer leadership to TS cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:26.555419 10801 raft_consensus.cc:614] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:26.557148 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:18:26.557843 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:26.560117 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:18:26.562134 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:26.562762 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:26.563447 11512 raft_consensus.cc:2393] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 14: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:26.486521 16392 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:26.567451 10841 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:26.573166 10796 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 14.
I20260709 13:18:26.580927 10796 raft_consensus.cc:1275] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:18:26.586930 11511 raft_consensus.cc:1240] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:26.584282 15907 raft_consensus.cc:2749] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:18:26.577916 16393 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:26.556653 16428 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:26.601140 11512 raft_consensus.cc:1240] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:18:26.613862 10799 raft_consensus.cc:1275] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 13:18:26.615356 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 61696us
I20260709 13:18:26.597111 16393 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:26.624145 10842 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:26.583238 16426 consensus_queue.cc:260] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:26.620293 16393 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:26.602000 16220 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:26.630358 10800 raft_consensus.cc:2393] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 19: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:26.626935 15907 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:26.633446 15907 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:26.659147 16242 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.659934 10915 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:26.659835 16242 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:26.661228 15859 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:26.661688 15859 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:26.662133 15859 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:26.662443 16242 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:26.661649 10915 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:26.640645 16129 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:26.651713 15907 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:26.675242 16242 raft_consensus.cc:2749] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:26.675493 16426 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:26.679476 16129 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:26.685624 10915 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:26.686976 16242 raft_consensus.cc:2749] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:18:26.687295 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 46693.5us
I20260709 13:18:26.690668 10767 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:26.700908 10797 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 18.
W20260709 13:18:26.705742 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 73438.4us
I20260709 13:18:26.706679 10842 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:26.670949 15883 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:26.703830 16316 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:26.690984 16275 consensus_queue.cc:1059] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 13:18:26.707231 16197 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:26.717670 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:26.709707 16192 raft_consensus.cc:2804] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:26.720960 16192 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:26.721375 16192 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:26.723188 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:18:26.716795 16292 consensus_meta.cc:287] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.530s	user 0.015s	sys 0.000s
I20260709 13:18:26.718565 16197 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:26.731925 16197 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:26.733121 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:26.733994 10872 raft_consensus.cc:2393] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 19: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:26.716367 16428 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:26.736909 10871 raft_consensus.cc:2393] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 20: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:26.735118 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:26.739619 16129 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.740173 16129 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:26.740617 16349 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:26.741302 16316 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:26.741236 16349 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:26.742322 16129 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:26.749109 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:26.749336 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:26.749903 11510 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
I20260709 13:18:26.750125 11512 raft_consensus.cc:2393] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 18: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:26.751164 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:26.751698 16242 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:26.752278 16242 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:26.754285 16242 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:26.755836 15883 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:26.755170 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:26.757511 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:26.758179 10798 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:26.759027 16220 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:26.760641 10876 raft_consensus.cc:2393] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 16: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:26.763425 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:26.763993 10796 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:26.777638 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:26.778961 11507 raft_consensus.cc:1275] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:18:26.780153 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:26.780845 11505 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
I20260709 13:18:26.781467 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:26.780900 11504 raft_consensus.cc:1240] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:26.793578 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:26.797199 15907 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:26.763301 16403 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.805758 10868 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 14.
I20260709 13:18:26.810354 10793 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
I20260709 13:18:26.807538 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:26.812595 11504 raft_consensus.cc:2393] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 16: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:26.824090 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:26.827287 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:26.838832 10797 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
I20260709 13:18:26.842304 10793 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 16.
I20260709 13:18:26.848932 10842 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:26.854012 11503 raft_consensus.cc:1275] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:18:26.858983 10796 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 16.
I20260709 13:18:26.816231 16403 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:26.890215 16474 log.cc:927] Time spent T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.178s	user 0.000s	sys 0.001s
I20260709 13:18:26.892596 16197 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:26.901901 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:26.887305 10767 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:26.907318 16292 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.908001 16242 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.907994 16292 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:26.908473 10916 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:26.847548 16220 consensus_queue.cc:1048] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:18:26.884202 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:26.884255 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:26.885221 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:26.917088 10947 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
I20260709 13:18:26.886106 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:26.888075 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:26.908716 16242 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:26.919523 10946 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:26.918422 10945 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:26.880482 15907 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:26.921594 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:26.923640 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:26.925740 10868 raft_consensus.cc:2410] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 13:18:26.924198 10867 raft_consensus.cc:1275] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 15. (term mismatch)
W20260709 13:18:26.926407 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 77605us
I20260709 13:18:26.924521 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:26.926584 10867 pending_rounds.cc:85] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 14
I20260709 13:18:26.924111 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:26.928947 10876 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 16.
I20260709 13:18:26.928233 10867 pending_rounds.cc:107] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 17.15
I20260709 13:18:26.936412 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:26.936893 10867 raft_consensus.cc:2889] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:26.918198 16375 raft_consensus.cc:2749] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:26.916325 16484 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:26.942698 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:26.938382 10766 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:26.945677 10873 raft_consensus.cc:1275] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:18:26.946204 10794 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 14.
I20260709 13:18:26.946990 16485 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:18:26.947609 16476 log.cc:927] Time spent T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 13:18:26.947731 16485 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:26.951035 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:26.951766 10797 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:26.944077 10870 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 17.
I20260709 13:18:26.929512 16349 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:26.956377 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:26.969308 10948 raft_consensus.cc:2393] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 15: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:26.969813 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:26.985878 10873 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:18:26.972121 16167 raft_consensus.cc:2804] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:26.973573 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:26.976722 16403 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:26.991216 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 147239us
I20260709 13:18:26.993355 10876 raft_consensus.cc:1240] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:26.997730 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:26.952529 16484 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:26.998811 10867 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 14.
I20260709 13:18:26.997869 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:27.001127 11508 raft_consensus.cc:2393] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 12: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:27.007578 10767 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:27.008929 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:27.012367 10949 raft_consensus.cc:1275] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:18:27.013583 16491 raft_consensus.cc:2749] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:18:27.014454 16487 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:27.014739 15859 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:27.015173 16487 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:27.017742 10944 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 12.
I20260709 13:18:27.018193 10945 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
I20260709 13:18:27.019533 10945 raft_consensus.cc:1240] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 17. Current term is 18. Ops: [16.8-17.9]
I20260709 13:18:27.022050 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:27.022635 10867 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:27.027581 10873 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 18.
I20260709 13:18:27.004563 16167 raft_consensus.cc:697] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:27.026619 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:27.031425 16393 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:27.032569 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:27.033541 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:27.035097 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:27.039496 16368 raft_consensus.cc:3055] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:27.043298 10950 raft_consensus.cc:1240] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:27.049307 10949 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:27.002887 16484 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:27.052021 10944 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:27.050107 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:27.017253 16487 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:27.017264 15859 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:27.053421 16368 raft_consensus.cc:740] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:27.055063 10798 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
I20260709 13:18:27.050920 16220 raft_consensus.cc:2804] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:27.057265 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:27.030561 15883 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:27.068202 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:27.069197 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 69595.2us
I20260709 13:18:27.073890 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:27.114828 10796 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 16.
I20260709 13:18:27.115144 10871 raft_consensus.cc:1240] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:27.075591 10799 raft_consensus.cc:1240] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:27.076036 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:27.115262 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 206907us
I20260709 13:18:27.079741 10792 raft_consensus.cc:1275] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:18:27.082999 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:27.116709 10842 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:27.089994 10944 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 18.
I20260709 13:18:27.065083 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:27.117892 10950 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 14.
I20260709 13:18:27.118741 11478 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:27.079277 10794 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2136) took 2642 ms (client timeout 1566 ms). Trace:
I20260709 13:18:27.110203 10945 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:27.055343 10797 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
I20260709 13:18:27.062412 16211 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:27.084038 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:27.119163 11477 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:27.116024 10801 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 16.
I20260709 13:18:27.131524 10949 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 15.
I20260709 13:18:27.132652 10946 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
I20260709 13:18:27.134146 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:27.130961 10842 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:27.134768 10944 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:18:27.120111 10794 rpcz_store.cc:269] 0709 13:18:24.437035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.437767 (+   732us) service_pool.cc:224] Handling call
0709 13:18:27.078995 (+2641228us) spinlock_profiling.cc:243] Waited 2.46 s on lock 0x7b54000bccd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:27.079149 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5663550848}
I20260709 13:18:27.145418 10941 raft_consensus.cc:1275] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:18:27.145723 10767 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:27.136076 11477 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:27.148082 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:27.081888 16392 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:27.063064 16197 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:27.176954 10916 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:27.148721 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:27.056715 16242 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:27.136018 10842 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:27.158149 10941 raft_consensus.cc:1275] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 13:18:27.163810 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:27.077508 16292 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:27.148406 16479 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:27.217242 16479 raft_consensus.cc:697] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:27.218519 16479 consensus_queue.cc:237] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:27.112783 16220 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:27.180980 10949 raft_consensus.cc:2393] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 19: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:18:27.168674 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 230314us
I20260709 13:18:27.222870 16220 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:27.223315 10766 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:27.181722 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:27.130391 16487 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:27.224236 10949 raft_consensus.cc:1275] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:18:27.224169 16487 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:27.226265 16487 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:27.146382 16350 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:18:27.056994 16393 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:27.089545 16491 consensus_queue.cc:1059] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.094s
I20260709 13:18:27.191797 16486 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:27.196071 10945 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 18.
W20260709 13:18:27.200048 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 81415.8us
I20260709 13:18:27.200323 15907 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:27.200625 10841 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:27.199975 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 64003.7us
I20260709 13:18:27.205555 16368 consensus_queue.cc:260] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:27.180344 15883 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:27.185477 16316 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:27.186874 16129 raft_consensus.cc:2749] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:18:27.148725 16426 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:27.094405 15859 consensus_queue.cc:1059] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.086s
W20260709 13:18:27.232751 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 118246us
I20260709 13:18:27.237025 10946 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 14.
I20260709 13:18:27.184506 16211 raft_consensus.cc:2749] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:18:27.253719 10842 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:27.063678 16167 consensus_queue.cc:237] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:27.148207 16192 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:27.293192 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 158843us
I20260709 13:18:27.294116 10916 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:27.295158 10766 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:27.295688 10870 raft_consensus.cc:2393] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 14: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:27.296227 16493 consensus_queue.cc:1048] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:18:27.298844 16316 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:27.299840 16484 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:27.300277 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 99783us
I20260709 13:18:27.301107 16316 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:27.301726 16316 consensus_queue.cc:1059] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:18:27.301951 16484 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:27.301775 10841 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:27.302853 16487 raft_consensus.cc:3055] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:27.303256 16487 raft_consensus.cc:740] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:27.304057 15883 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:27.304219 10767 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:27.304092 16487 consensus_queue.cc:260] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:27.305364 16368 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:27.305460 16375 raft_consensus.cc:2749] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:27.305663 10767 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:27.305905 16487 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:27.307596 10794 raft_consensus.cc:1275] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:18:27.307845 16392 raft_consensus.cc:2804] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:27.308430 16392 raft_consensus.cc:697] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:27.309120 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:27.309306 16392 consensus_queue.cc:237] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:27.292755 16486 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:27.310943 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:27.311775 10794 raft_consensus.cc:2410] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:18:27.312312 16486 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:27.304782 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 62032.2us
I20260709 13:18:27.312981 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:27.313220 11478 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:27.289284 10842 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:18:27.332860 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 251859us
I20260709 13:18:27.333920 10944 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 18.
I20260709 13:18:27.255477 16242 raft_consensus.cc:2749] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:18:27.256815 16350 raft_consensus.cc:697] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 18 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:27.263639 16393 raft_consensus.cc:2749] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:27.256579 16292 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:27.252826 15907 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:27.257095 16349 raft_consensus.cc:2804] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:18:27.341353 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 52193.3us
I20260709 13:18:27.315819 16431 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:27.295205 16192 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:27.344805 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 211932us
I20260709 13:18:27.260783 16197 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.348703 10841 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:27.263680 16491 consensus_queue.cc:1243] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer 12682eb70bc84c76879e04a8628ee586 log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:18:27.344295 15907 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:27.340989 16426 raft_consensus.cc:697] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:27.360846 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:27.349659 16484 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:27.361590 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:27.362368 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 67339.1us
I20260709 13:18:27.345144 16292 raft_consensus.cc:697] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:27.360985 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:27.385146 10797 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:18:27.386404 16292 consensus_queue.cc:237] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:27.353703 16349 raft_consensus.cc:697] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 15 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:27.358381 16129 raft_consensus.cc:2749] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:27.398314 10801 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 14.
I20260709 13:18:27.410610 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:27.411139 10792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2737) took 1106 ms. Trace:
I20260709 13:18:27.415009 10792 rpcz_store.cc:276] 0709 13:18:26.295243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:26.301731 (+  6488us) service_pool.cc:224] Handling call
0709 13:18:27.079377 (+777646us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:27.116423 (+ 37046us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:27.393054 (+276631us) spinlock_profiling.cc:243] Waited 768 ms on lock 0x7b54000bccd0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:27.401655 (+  8601us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1765272448}
I20260709 13:18:27.413389 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:27.419071 10766 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:27.421109 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 72542.1us
I20260709 13:18:27.425879 16299 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.426208 16504 consensus_queue.cc:1059] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:18:27.426503 16299 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:27.376495 16484 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:27.430119 10867 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 18.
I20260709 13:18:27.393003 16486 raft_consensus.cc:2749] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:27.427588 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:27.432210 10841 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:27.434466 10797 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 14.
I20260709 13:18:27.359416 16475 consensus_queue.cc:1059] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.059s
I20260709 13:18:27.440811 10796 raft_consensus.cc:2393] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 15: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:27.442014 10794 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:27.418918 16431 raft_consensus.cc:697] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:27.445148 16289 raft_consensus.cc:2749] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:27.442809 16211 raft_consensus.cc:2749] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:27.445473 16431 consensus_queue.cc:237] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:27.447446 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 153436us
I20260709 13:18:27.433100 16479 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:27.448772 16491 raft_consensus.cc:3055] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:18:27.449265 16497 log.cc:927] Time spent T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:18:27.448801 16479 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:27.449426 16491 raft_consensus.cc:740] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:27.449635 10916 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:27.451407 16479 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:27.359474 16197 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:27.456805 16495 raft_consensus.cc:3055] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:18:27.457091 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 45555.6us
I20260709 13:18:27.457274 16495 raft_consensus.cc:740] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:27.458042 10766 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:27.458508 10916 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:27.458891 16500 consensus_queue.cc:1048] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:18:27.460116 10916 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:27.462891 10916 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:27.457489 16349 consensus_queue.cc:237] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:27.463984 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:27.453233 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:27.469467 10793 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:27.458927 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:27.441427 16484 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:27.436093 16316 raft_consensus.cc:2764] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:18:27.427500 16498 raft_consensus.cc:2749] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:18:27.475371 15883 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:18:27.471096 16509 raft_consensus.cc:2749] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:27.475450 10801 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 14.
I20260709 13:18:27.505383 10794 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 14.
I20260709 13:18:27.468840 16350 consensus_queue.cc:237] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:27.469092 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:27.476850 16299 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:27.476989 10766 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:27.477842 16485 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:27.496322 16504 raft_consensus.cc:2749] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:27.508406 10799 raft_consensus.cc:2393] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 16: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
W20260709 13:18:27.472124 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 134473us
W20260709 13:18:27.495007 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 72320.4us
I20260709 13:18:27.446460 16426 consensus_queue.cc:237] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:27.474290 10797 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
I20260709 13:18:27.450434 16491 consensus_queue.cc:260] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:27.506356 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:18:27.470083 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 104773us
I20260709 13:18:27.511621 10794 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:27.512614 10793 raft_consensus.cc:2410] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
W20260709 13:18:27.474699 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 161559us
I20260709 13:18:27.457100 16129 consensus_queue.cc:1048] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 13:18:27.509956 10841 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:27.514686 11478 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:27.514856 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:27.513188 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:27.471457 16493 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:27.516791 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:27.517117 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:27.515738 10799 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 17.
I20260709 13:18:27.519752 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:27.520351 10801 raft_consensus.cc:2393] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 16: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:27.520960 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:27.521384 10801 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:27.514315 16485 consensus_queue.cc:1059] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 13:18:27.523355 16507 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:27.523862 16507 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:27.523917 16506 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:27.524326 16507 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:27.524468 16506 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:27.524991 16506 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:27.526859 16494 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:18:27.527006 10797 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:27.528053 16511 raft_consensus.cc:2749] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:27.526883 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:27.530988 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:27.531570 10799 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 14.
I20260709 13:18:27.532712 10799 raft_consensus.cc:1240] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
W20260709 13:18:27.533169 16488 log.cc:927] Time spent T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.567s	user 0.002s	sys 0.003s
I20260709 13:18:27.533970 10799 raft_consensus.cc:1275] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:18:27.534945 16475 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.528059 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:27.535488 16475 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:27.535884 10793 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 12.
I20260709 13:18:27.536947 10793 raft_consensus.cc:1240] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:27.537761 16475 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:27.538468 16494 raft_consensus.cc:697] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:27.538481 10797 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 16.
I20260709 13:18:27.539600 16494 consensus_queue.cc:237] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:27.543164 16485 raft_consensus.cc:3055] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:27.543709 16485 raft_consensus.cc:740] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:27.544823 16485 consensus_queue.cc:260] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:27.546360 16485 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:27.548753 16242 raft_consensus.cc:3055] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:27.549240 16242 raft_consensus.cc:740] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:27.554359 10842 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:27.511365 15883 raft_consensus.cc:697] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:27.568625 10799 raft_consensus.cc:1275] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 13:18:27.566407 10841 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:27.514325 11477 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:18:27.577953 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 101037us
I20260709 13:18:27.577777 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:27.578992 10945 raft_consensus.cc:2393] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 16: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:27.578276 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:27.579531 10766 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:27.581429 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:18:27.582626 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 68378.7us
I20260709 13:18:27.560582 10794 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 19.
I20260709 13:18:27.569031 10799 pending_rounds.cc:85] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 10
I20260709 13:18:27.589152 10799 pending_rounds.cc:107] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:18:27.589401 10799 raft_consensus.cc:2889] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:27.589887 10766 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:27.592042 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:27.592288 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:18:27.592406 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 77836.3us
I20260709 13:18:27.593392 10792 raft_consensus.cc:2393] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 14: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:18:27.587930 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 73921.2us
I20260709 13:18:27.592760 10950 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
I20260709 13:18:27.557327 16211 raft_consensus.cc:2804] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:27.556883 16511 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.596164 10948 raft_consensus.cc:2393] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 16: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:27.603868 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:27.550272 16242 consensus_queue.cc:260] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:27.578032 10842 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:27.597725 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:27.588738 11477 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:27.598551 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:27.604732 10944 raft_consensus.cc:2393] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 14: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:27.595803 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:27.580895 16475 raft_consensus.cc:2804] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:27.609400 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:27.552625 16479 consensus_queue.cc:1048] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 13:18:27.613600 10842 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:27.618567 10842 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:27.619071 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:27.619573 16428 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:27.619812 10945 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
W20260709 13:18:27.615813 16481 log.cc:927] Time spent T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:18:27.616061 16475 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:27.621001 16475 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:27.622038 16220 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:27.603255 16368 consensus_queue.cc:1048] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 13:18:27.627264 16479 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:27.627460 16197 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:27.630302 10801 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 18.
I20260709 13:18:27.597581 16289 raft_consensus.cc:2749] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:27.612959 16511 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:27.631605 16499 log.cc:927] Time spent T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.470s	user 0.001s	sys 0.002s
I20260709 13:18:27.612764 16211 raft_consensus.cc:697] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:27.632273 16431 raft_consensus.cc:2804] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:27.658756 16375 raft_consensus.cc:2749] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:18:27.659008 16503 log.cc:927] Time spent T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:18:27.659943 15883 consensus_queue.cc:237] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:27.660495 16431 raft_consensus.cc:697] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:27.661650 16431 consensus_queue.cc:237] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:27.643707 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:27.627282 16299 consensus_queue.cc:1048] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.012s
I20260709 13:18:27.639969 16475 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:27.671382 10948 raft_consensus.cc:2393] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 16: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:27.671911 10915 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:27.675062 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 81836.7us
I20260709 13:18:27.645071 16220 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:27.642716 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:27.595489 16393 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:27.614214 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:27.646232 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:27.637279 16479 raft_consensus.cc:2749] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:27.655745 16508 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:18:27.656615 16511 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:27.658087 16211 consensus_queue.cc:237] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:27.679538 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 216725us
I20260709 13:18:27.680540 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:18:27.691776 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 103118us
I20260709 13:18:27.683898 10799 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d"
dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
mode: GRACEFUL
new_leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
 from {username='slave'} at 127.0.0.1:40148
I20260709 13:18:27.692698 10799 raft_consensus.cc:606] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Received request to transfer leadership to TS 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:27.693147 10799 raft_consensus.cc:614] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:27.685077 10942 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
I20260709 13:18:27.556978 16495 consensus_queue.cc:260] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:27.613168 16486 consensus_queue.cc:1048] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 13:18:27.641589 16507 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:27.684495 16393 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:27.699013 16511 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:18:27.699569 16508 raft_consensus.cc:697] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 18 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:27.699728 16495 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:18:27.700805 16426 consensus_queue.cc:1175] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "12682eb70bc84c76879e04a8628ee586" responder_term: 15 status { last_received { term: 14 index: 12 } last_committed_idx: 12 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 14. Current term is 15. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:18:27.692044 10943 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
I20260709 13:18:27.693922 16491 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:27.687363 16275 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.684347 16498 raft_consensus.cc:2749] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:18:27.730921 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 68025us
I20260709 13:18:27.728893 16393 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:18:27.734344 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 115857us
W20260709 13:18:27.735458 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 59262.2us
W20260709 13:18:27.737946 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:27.776559 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:27.778613 10915 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:27.688402 16242 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:18:27.779371 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 46732.1us
I20260709 13:18:27.786767 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:27.732966 16428 raft_consensus.cc:697] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:27.781030 10842 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:27.777876 16511 raft_consensus.cc:697] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 18 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:27.785620 10943 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 16.
I20260709 13:18:27.791494 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:27.801252  9682 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:18:27.928622 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:27.786859 16275 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:27.815552 16508 consensus_queue.cc:237] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:27.953670 16518 log.cc:927] Time spent T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
I20260709 13:18:27.949828 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:27.810997 16516 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:27.728466 16349 consensus_queue.cc:1048] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.057s
I20260709 13:18:27.794278 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:27.710227 16521 consensus_queue.cc:1048] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.097s
I20260709 13:18:27.955726 11508 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:27.955735 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:27.936787 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:27.825361 16523 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:27.957086 10873 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 14.
W20260709 13:18:27.937917 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 72663us
I20260709 13:18:27.941946 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:18:27.929364 15907 consensus_meta.cc:287] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.525s	user 0.009s	sys 0.004s
I20260709 13:18:27.946815 10942 raft_consensus.cc:1275] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:18:27.955272 10875 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 17.
I20260709 13:18:27.945185 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:27.946378 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:27.950560 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:27.951169 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:27.951730 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:27.934880 16316 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:27.952929 11512 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 14.
I20260709 13:18:27.966662 10870 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:27.967159 11504 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
I20260709 13:18:27.909809 16350 consensus_queue.cc:1059] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 13:18:27.929165 16484 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:27.979921 10876 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:27.971441 10871 raft_consensus.cc:2368] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 14. Current term is 15.
I20260709 13:18:27.987910 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:27.947083 16479 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.958946 16523 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:27.939056 16506 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:27.989426 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 210881us
I20260709 13:18:27.987031 16475 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:27.987190 16428 consensus_queue.cc:237] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:27.996021 10916 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:27.997769 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:28.007052 10874 raft_consensus.cc:1240] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:28.008862 10867 raft_consensus.cc:1240] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 19. Current term is 20. Ops: []
I20260709 13:18:28.011552 11508 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 14.
I20260709 13:18:28.026736 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:27.982527 16275 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:28.028731 10946 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:27.968647 15907 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:28.037941 15907 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:28.040109 16493 raft_consensus.cc:2749] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:28.041019 16289 consensus_queue.cc:1048] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:18:28.008101 16479 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:27.980458 16316 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:28.044946 16479 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:28.045217 16316 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:28.033934 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:27.997509 16484 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:28.046376 10798 raft_consensus.cc:2410] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:18:28.047058 10867 raft_consensus.cc:2393] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 17: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:27.993883 16500 raft_consensus.cc:3055] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:28.068710 10766 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:27.993785 16523 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:28.076980 10870 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 17.
I20260709 13:18:28.014263 16498 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:28.077343 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:28.083500 10947 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:28.098631 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:28.102926 10942 raft_consensus.cc:2393] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 13: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:28.103971 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:28.104602 10871 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:18:28.109946 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 114023us
I20260709 13:18:28.111800 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:28.078514 16500 raft_consensus.cc:740] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:28.112622 10942 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
I20260709 13:18:28.094378 16197 raft_consensus.cc:2804] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:28.150192 16197 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:28.150862 16197 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:28.001999 16516 consensus_queue.cc:1048] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.104s
I20260709 13:18:28.156481 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:28.157465 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:28.160027 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:28.163568 16506 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:18:28.163978 16527 log.cc:927] Time spent T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.213s	user 0.002s	sys 0.000s
I20260709 13:18:28.165314 11508 raft_consensus.cc:3055] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:28.165848 11508 raft_consensus.cc:740] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:28.166859 11508 consensus_queue.cc:260] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:28.168001 11508 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:28.111537 16507 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:28.115049 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:28.118451 16524 log.cc:927] Time spent T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 13:18:28.123862 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:28.173344 10941 raft_consensus.cc:2393] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 14: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
W20260709 13:18:28.115053 16517 log.cc:927] Time spent T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.259s	user 0.000s	sys 0.003s
I20260709 13:18:28.126650 15859 raft_consensus.cc:3055] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:18:28.175853 15859 raft_consensus.cc:740] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:28.177052 15859 consensus_queue.cc:260] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:28.178411 15859 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:18:28.131381 16529 log.cc:927] Time spent T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.189s	user 0.001s	sys 0.000s
I20260709 13:18:28.131861 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:28.132311 16530 log.cc:927] Time spent T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
W20260709 13:18:28.204309 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:28.207719 10944 raft_consensus.cc:2393] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 16: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:28.208851 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:28.022797 16511 consensus_queue.cc:237] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:28.144019 10916 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:28.147491 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:18:28.114930 16515 log.cc:927] Time spent T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.215s	user 0.000s	sys 0.011s
I20260709 13:18:28.226421 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:28.174463 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:28.229799 10876 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
I20260709 13:18:28.180676 16393 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:28.182116 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:28.231377 10870 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
I20260709 13:18:28.078137 16493 consensus_queue.cc:1059] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:18:28.219274 10871 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 19.
I20260709 13:18:28.221760 10946 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 14.
I20260709 13:18:28.136667 10841 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:28.221168 10868 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 13.
I20260709 13:18:28.236032 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:28.247342 10872 raft_consensus.cc:1275] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:18:28.247449 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 110864us
I20260709 13:18:28.248620 10841 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:28.249281 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:28.251026 10876 raft_consensus.cc:1275] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 13:18:28.252882 10947 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 19.
I20260709 13:18:28.236930 10874 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
I20260709 13:18:28.258147 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:28.259013 10767 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:28.263818 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:28.264444 10792 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:28.276191 10801 raft_consensus.cc:1275] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:18:28.279657 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 135766us
W20260709 13:18:28.247692 16485 consensus_meta.cc:287] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.690s	user 0.012s	sys 0.003s
I20260709 13:18:28.289536 10947 raft_consensus.cc:1240] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:28.289414 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:28.290457 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:28.293184 10876 raft_consensus.cc:1275] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:18:28.296041 10941 raft_consensus.cc:1275] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 13:18:28.298585 10948 raft_consensus.cc:1275] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:18:28.299572 10767 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:28.300019 10796 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 14.
I20260709 13:18:28.302270 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:28.307817 10841 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:28.309178 11508 raft_consensus.cc:1275] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 13:18:28.311749 11478 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.046s	sys 0.065s
W20260709 13:18:28.312183 11478 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.046s	sys 0.065s
W20260709 13:18:28.312507 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 528639us
I20260709 13:18:28.313238 11478 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:28.271737 16507 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:28.303151 10799 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
I20260709 13:18:28.308154 11477 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:28.335865 10767 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:28.332535 11478 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:28.339587 10795 raft_consensus.cc:1275] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:18:28.342335 10950 raft_consensus.cc:1275] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:18:28.348330 10941 raft_consensus.cc:1240] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:18:28.292163 16486 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:28.389461 10916 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:28.393247 10945 raft_consensus.cc:1275] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:18:28.394106 10916 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:28.350936 10947 raft_consensus.cc:1275] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:18:28.300927 16487 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:28.344985 16528 consensus_queue.cc:1059] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 13:18:28.346544 16479 raft_consensus.cc:2749] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:18:28.327764 16493 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:28.406081 10798 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 16.
I20260709 13:18:28.341958 16375 raft_consensus.cc:2749] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:28.391922 16486 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:28.392477 16242 consensus_meta.cc:287] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.605s	user 0.007s	sys 0.004s
I20260709 13:18:28.397779 16506 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:28.370711 10766 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:18:28.423404 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 52784.4us
I20260709 13:18:28.399398 10767 leader_election.cc:400] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 election: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 14. Current term is 15.
I20260709 13:18:28.425915 10767 leader_election.cc:403] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:18:28.374269 10842 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:28.338727 16485 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:28.390741 16392 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:18:28.349130 16349 raft_consensus.cc:2804] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:28.394578 16129 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:18:28.438571 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 44532.7us
I20260709 13:18:28.439782 10916 leader_election.cc:400] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 election: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 16. Current term is 17.
I20260709 13:18:28.440254 10916 leader_election.cc:403] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:18:28.352488 10841 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:28.376287 16522 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:28.404531 16493 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:28.389191 16220 raft_consensus.cc:3055] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:18:28.403795 16516 raft_consensus.cc:2764] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:18:28.377152 16484 consensus_queue.cc:1048] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 13:18:28.451695 10793 raft_consensus.cc:1275] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 13:18:28.396701 16521 consensus_queue.cc:1048] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 13:18:28.423099 16486 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:28.449085  9682 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:28.296708 16500 consensus_queue.cc:260] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:28.454355 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 80176.6us
W20260709 13:18:28.453678 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 109472us
W20260709 13:18:28.462682 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 130237us
I20260709 13:18:28.431468 16506 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:28.437989 16349 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:28.437721 16289 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:18:28.438390 16129 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:28.415307 16535 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:28.484556 10842 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:28.404983 16487 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:28.452242 16220 raft_consensus.cc:740] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:28.480322 16375 consensus_queue.cc:1048] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:18:28.457203 16508 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:28.451817 16242 raft_consensus.cc:2749] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 16. Current term is 17.
I20260709 13:18:28.483661 16431 raft_consensus.cc:2804] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:18:28.512667 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 59164.7us
I20260709 13:18:28.452958 16493 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:28.487043 10767 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:28.530165 10842 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:18:28.482553 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 130180us
I20260709 13:18:28.513854 16349 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:28.495846 16506 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:28.541846 10841 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:28.515793 16129 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:28.514587 16392 raft_consensus.cc:2749] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 14. Current term is 15.
I20260709 13:18:28.553802 10916 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:28.461633 16485 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:28.556496 10841 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:28.481340 16522 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:28.527333 15883 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:28.531281 15907 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:28.564613 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 140061us
I20260709 13:18:28.536041 16479 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:18:28.569775 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 82821.9us
I20260709 13:18:28.553808 16500 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:18:28.563256 11478 connection.cc:256] RPC call timeout handler was delayed by -1.428648001000056s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19410000000000002s for a response
W20260709 13:18:28.606173 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:28.531761 16289 raft_consensus.cc:697] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:28.603544 10916 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:28.608901 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:28.560985 16428 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:28.535511 16426 consensus_queue.cc:1048] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 13:18:28.561453 16299 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:28.613526 10876 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 17.
W20260709 13:18:28.614815 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 54045.3us
I20260709 13:18:28.556793 16431 raft_consensus.cc:697] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:28.574411 16242 raft_consensus.cc:2764] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader election decision vote started in defunct term 18: won
I20260709 13:18:28.611538 15907 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:28.621600 16533 consensus_queue.cc:1048] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.179s
I20260709 13:18:28.622627 16491 raft_consensus.cc:2804] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:28.625986 10916 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:28.579074 16316 raft_consensus.cc:2804] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:28.574754 16393 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:28.578992 16479 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:28.536389 16508 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:28.560626 16516 consensus_queue.cc:1048] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.070s
W20260709 13:18:28.647274 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 73695.5us
I20260709 13:18:28.626871 16537 raft_consensus.cc:2749] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:18:28.561857 16498 consensus_queue.cc:1048] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.069s
I20260709 13:18:28.619102 16299 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:28.605279 16484 consensus_queue.cc:1048] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:18:28.612392 16392 consensus_queue.cc:1048] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 13:18:28.642190 16316 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:28.660008 16479 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:28.605468 15883 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:28.659737 16350 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:28.690398 16289 consensus_queue.cc:237] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:28.691404 16548 consensus_queue.cc:1059] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.124s
W20260709 13:18:28.692247 16536 log.cc:927] Time spent T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.175s	user 0.000s	sys 0.000s
W20260709 13:18:28.692757 16546 log.cc:927] Time spent T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:18:28.666656 16475 consensus_queue.cc:1048] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
W20260709 13:18:28.693594 16539 log.cc:927] Time spent T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 13:18:28.670935 16299 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:28.613188 16220 consensus_queue.cc:260] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:28.708513 10792 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
W20260709 13:18:28.709352 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 61839.1us
W20260709 13:18:28.693204 16545 log.cc:927] Time spent T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:18:28.689035 16316 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:18:28.721390 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 152471us
W20260709 13:18:28.729310 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:28.733484 10841 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:28.741782 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 74897.3us
W20260709 13:18:28.750099 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 130084us
W20260709 13:18:28.750623 11478 connection.cc:256] RPC call timeout handler was delayed by -1.3908480010002222s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19830000000000003s for a response
W20260709 13:18:28.751034 11478 connection.cc:256] RPC call timeout handler was delayed by -1.3611480010001742s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2016s for a response
W20260709 13:18:28.751416 11478 connection.cc:256] RPC call timeout handler was delayed by -1.2531480010002269s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2136s for a response
I20260709 13:18:28.741901 16498 raft_consensus.cc:2804] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:28.742913 16487 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:28.737782 16428 raft_consensus.cc:697] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:28.745615 16392 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:28.764497 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:28.768010 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:28.750346 16426 raft_consensus.cc:2749] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:28.769973 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:28.710511 16431 consensus_queue.cc:237] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:28.793073 11504 raft_consensus.cc:1275] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
W20260709 13:18:28.796574 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 170655us
I20260709 13:18:28.798377 11507 raft_consensus.cc:1240] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:18:28.797900 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:28.802531 11512 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
I20260709 13:18:28.828620 10793 raft_consensus.cc:1275] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:18:28.800841 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:28.829910 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:28.830067 11511 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:28.832006 10943 raft_consensus.cc:2393] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 15: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:28.766739 16485 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:28.755462 16537 consensus_queue.cc:1048] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:18:28.842167 11506 raft_consensus.cc:1240] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 15 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:18:28.846315 11508 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
I20260709 13:18:28.853089 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:28.853798 11507 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
I20260709 13:18:28.855079 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:18:28.843711 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 235467us
I20260709 13:18:28.859006 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:28.859071 10767 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:28.860067 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 102532us
I20260709 13:18:28.794265 16563 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:28.855852 10941 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
I20260709 13:18:28.801323 16522 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:28.863749 11478 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:28.864473 10945 raft_consensus.cc:1275] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:18:28.828042 16487 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:28.826180 16292 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:28.876238 10949 raft_consensus.cc:1275] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 13:18:28.876587 10767 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:28.873765 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:28.772233 16550 consensus_queue.cc:1059] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.084s
I20260709 13:18:28.880445 10916 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:28.883944 10950 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 16.
I20260709 13:18:28.773837 16392 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:28.859493 16508 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:28.895334 11508 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:18:28.889846 15859 consensus_meta.cc:287] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.687s	user 0.009s	sys 0.002s
I20260709 13:18:28.881918 15907 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:28.884737 16547 raft_consensus.cc:2749] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:28.908941 10767 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:28.840938 16511 consensus_queue.cc:1048] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 13:18:28.916175 11478 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:28.916559 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:18:28.919200 16553 log.cc:927] Time spent T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 13:18:28.919636 16506 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:28.940727 16506 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:18:28.920770 16507 consensus_meta.cc:287] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.574s	user 0.009s	sys 0.002s
I20260709 13:18:28.941597 16507 raft_consensus.cc:2749] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:28.944746 16535 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:28.945259 16535 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:28.937309 11511 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 15.
I20260709 13:18:28.886229 16487 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:28.883996 16563 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:28.977959 16576 raft_consensus.cc:2804] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:28.978556 16576 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:28.978968 16576 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:18:28.960439 16197 consensus_meta.cc:287] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.809s	user 0.008s	sys 0.004s
I20260709 13:18:28.903029 16428 consensus_queue.cc:237] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:28.998137 16316 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:28.959712 16486 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:28.989192 16299 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:29.031661 16299 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:29.032454 16479 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:29.033905 16486 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:29.035081 10874 raft_consensus.cc:1240] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 18. Current term is 19. Ops: [18.15-18.15]
W20260709 13:18:29.027058 16523 consensus_meta.cc:287] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.925s	user 0.013s	sys 0.000s
I20260709 13:18:29.035887 16523 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:29.001878 16349 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:29.003445 16562 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:29.021059 16498 raft_consensus.cc:697] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:18:29.007527 16564 log.cc:927] Time spent T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 13:18:29.007891 16578 log.cc:927] Time spent T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
W20260709 13:18:29.008569 16553 log.cc:927] Time spent T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:18:28.954187 16548 consensus_queue.cc:1059] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 13:18:28.956086 16580 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:18:29.026177 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 110075us
I20260709 13:18:29.029724 16577 raft_consensus.cc:2749] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:18:29.045742 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 165367us
W20260709 13:18:29.046598 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 137741us
I20260709 13:18:29.024933 16197 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:29.101130 10916 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:29.088116 16548 consensus_queue.cc:1243] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Peer 468a8fd46e824f62b9c2c0d3532a6d26 log is divergent from this leader: its last log entry 18.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:18:28.989287 16493 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:29.003957 16392 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:29.038345 10696 catalog_manager.cc:5697] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 12 to 13, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 13 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:29.088833 10767 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:29.139747 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 51006us
I20260709 13:18:29.141099 10767 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:29.141142 11478 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:29.084599 16129 consensus_meta.cc:287] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.514s	user 0.011s	sys 0.000s
I20260709 13:18:29.091197 16498 consensus_queue.cc:237] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:29.093909 16544 log.cc:927] Time spent T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.555s	user 0.003s	sys 0.000s
I20260709 13:18:29.085884 16580 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:29.112807 11509 raft_consensus.cc:1275] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:18:29.054878 16479 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:29.153887 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 52867.9us
I20260709 13:18:29.155033 10916 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:29.144884 16392 raft_consensus.cc:2804] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:28.934404 11508 raft_consensus.cc:1275] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:18:29.148590 16562 consensus_queue.cc:1048] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 13:18:29.153390 16560 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.178995 16560 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:29.151733 16577 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:29.180276 16577 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:29.180759 16577 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:29.181573 16560 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:29.153946 16580 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:29.182976 16583 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:29.183799 16583 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:29.184357 16591 consensus_queue.cc:1048] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:18:29.186620 16547 raft_consensus.cc:2749] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:29.186944 16595 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:29.187511 16595 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:29.187942 16595 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:29.149176 16516 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.191208 16516 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:29.193665 16516 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:29.147957 16289 consensus_queue.cc:1048] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:18:29.195770 16561 raft_consensus.cc:2804] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:29.196462 16561 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:29.196954 16561 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:29.158510 16475 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:29.199628 16316 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:29.199245 16475 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:29.201131 16563 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:29.201339 16535 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:29.223497 16535 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:29.237807 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:29.241883 16560 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.242328 16487 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:29.242815 16560 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:29.245425 16560 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:29.201920 16475 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:29.202759 16493 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:29.152397 16299 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.213797 16375 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.195868 16197 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:29.215098 16562 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:29.215898 16582 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.217760 15859 raft_consensus.cc:2749] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:18:29.218031 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 63104.1us
I20260709 13:18:29.218992 16591 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:29.219655 16548 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.219741 16392 raft_consensus.cc:697] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:29.311429 16392 consensus_queue.cc:237] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:29.289539 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 188468us
I20260709 13:18:29.297541 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:29.314565 10944 raft_consensus.cc:2410] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:18:29.302515 10916 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:29.305020 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:29.308245 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:29.315519 10947 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 14.
I20260709 13:18:29.315659 10941 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:29.308939 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:29.316372 10946 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:29.314314 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:29.150328 16129 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:29.326213 16577 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:29.330883 16577 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:29.298341 16576 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:29.291482 16487 consensus_queue.cc:1048] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 13:18:29.331862 16129 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:29.299407 16299 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:29.333922 16576 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:29.334242 16299 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:29.299602 16375 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:29.313304 16582 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:29.336418 16375 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:29.318706 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 455496us
I20260709 13:18:29.328415 16580 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:29.328500 16583 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:29.304591 16491 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.635s	user 0.002s	sys 0.011s
I20260709 13:18:29.327348 16523 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:29.327499 16562 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:29.338641 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:29.341389 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:29.347072 11478 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:29.361703 10946 raft_consensus.cc:2410] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 19 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 13:18:29.363616 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:29.365350 10944 raft_consensus.cc:2393] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 15: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:29.366125 10947 raft_consensus.cc:1275] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:18:29.367870 10946 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 12.
I20260709 13:18:29.319796 16548 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:29.369537 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:29.370730 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:29.371237 10948 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:29.371431 10794 raft_consensus.cc:1275] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:18:29.372570 15859 raft_consensus.cc:2764] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:18:29.375976 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:29.377115 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:29.377643 10947 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
I20260709 13:18:29.378041 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:29.319917 16591 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:29.380311 16349 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:29.384914 16562 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:29.378427 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:29.387061 10947 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:29.387212 16548 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:29.336759 16582 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:29.391341 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 52890.1us
I20260709 13:18:29.392328 11478 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:29.396188 16129 raft_consensus.cc:2749] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:18:29.396503 16592 log.cc:927] Time spent T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 13:18:29.396909 16129 raft_consensus.cc:2749] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:18:29.397591 16571 log.cc:927] Time spent T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.444s	user 0.000s	sys 0.012s
I20260709 13:18:29.398564 15883 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:29.401160 16275 raft_consensus.cc:3055] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:18:29.401602 16275 raft_consensus.cc:740] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:29.402498 16275 consensus_queue.cc:260] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:29.404018 16275 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:29.404507 16580 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:29.409157 16220 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:29.397874 10941 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
W20260709 13:18:29.354966 16491 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.003s	sys 0.011s
I20260709 13:18:29.425822 16491 raft_consensus.cc:697] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:29.426862 16491 consensus_queue.cc:237] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:29.429768 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 81993us
I20260709 13:18:29.429993 10947 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
I20260709 13:18:29.434501 10841 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:29.429970 11478 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:18:29.437218 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:29.421190 16350 consensus_queue.cc:1048] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 13:18:29.439378 11478 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:29.436488 16554 raft_consensus.cc:2804] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:29.445564 10948 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 13.
I20260709 13:18:29.444713 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:29.455283 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:29.456570 11478 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:29.456806 10916 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:29.437973 16375 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.467671 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:29.471907 10941 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:29.468446 10947 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 14.
I20260709 13:18:29.436429 16493 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.473724 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:29.438053 16192 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:18:29.472322 11478 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:29.438730 16349 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.474630 10949 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
I20260709 13:18:29.459213 10766 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:29.440502 16498 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.443297 16577 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.494015 16493 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:29.499748 16585 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:29.500944 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 44257.5us
I20260709 13:18:29.501730 16506 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:29.511783 16349 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:29.512969 16350 consensus_queue.cc:1048] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:29.455812 16576 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:29.425498 16561 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:29.493784 16500 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:18:29.564307 16500 raft_consensus.cc:697] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 19 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:29.565343 16500 consensus_queue.cc:237] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:29.520889 16552 log.cc:927] Time spent T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.783s	user 0.002s	sys 0.000s
W20260709 13:18:29.536334 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 77189.8us
W20260709 13:18:29.460273 16550 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.038s	sys 0.074s
I20260709 13:18:29.458721 16554 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:29.574960 16554 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:29.578195 16211 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.578522 16600 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:29.569114 10766 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:29.579140 16600 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:29.581302 16600 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:29.581722 16596 log.cc:927] Time spent T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.276s	user 0.006s	sys 0.000s
I20260709 13:18:29.582319 16548 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.582423 16600 raft_consensus.cc:2804] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:29.582928 16548 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:29.583254 16600 raft_consensus.cc:697] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:29.584717 16591 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:29.584432 16600 consensus_queue.cc:237] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:29.585341 16548 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:29.577407 10941 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 16.
I20260709 13:18:29.464886 16129 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:29.436694 16516 consensus_queue.cc:1048] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:18:29.555744 10916 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:29.494288 16511 raft_consensus.cc:2749] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:29.560879 16506 raft_consensus.cc:697] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 14 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:29.438068 16595 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:29.563131 16576 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:29.601532 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 45875.3us
I20260709 13:18:29.603154 16576 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:29.491981 16192 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:29.603593 10874 raft_consensus.cc:1240] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:29.603832 16192 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:29.548847 16275 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:29.607735 16275 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:29.608275 16275 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:29.608585 16493 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:29.572351 10792 raft_consensus.cc:1275] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 9. (term mismatch)
W20260709 13:18:29.572710 16550 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.038s	sys 0.076s
W20260709 13:18:29.567111 10915 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.116.132:40555 timed out after 1.544s (SENT) [suppressed 14 similar messages]
I20260709 13:18:29.612649 10792 pending_rounds.cc:85] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 8
I20260709 13:18:29.613042 10792 pending_rounds.cc:107] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 17.9
I20260709 13:18:29.613327 10792 raft_consensus.cc:2889] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:29.574105 11503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56410 (request call id 890) took 1293 ms. Trace:
I20260709 13:18:29.614169 11503 rpcz_store.cc:276] 0709 13:18:28.280932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.281058 (+   126us) service_pool.cc:224] Handling call
0709 13:18:28.281413 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:28.327077 (+ 45664us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:28.327128 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:28.340937 (+ 13809us) log.cc:844] Serialized 26 byte log entry
0709 13:18:28.552952 (+212015us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:28.572147 (+ 19195us) log.cc:844] Serialized 14 byte log entry
0709 13:18:28.612565 (+ 40418us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:28.612665 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:29.520154 (+907489us) spinlock_profiling.cc:243] Waited 5.55 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:18:29.573733 (+ 53579us) raft_consensus.cc:1685] finished
0709 13:18:29.573868 (+   135us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:29.574017 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39482,"prepare.queue_time_us":46782,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":96924,"raft.run_cpu_time_us":330,"raft.run_wall_time_us":22701,"spinlock_wait_cycles":12773120,"thread_start_us":129352,"threads_started":1,"wal-append.queue_time_us":290165}
I20260709 13:18:29.618342 16595 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:29.620189 16349 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:29.569222 16585 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:29.625844 16585 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:29.632694 10868 raft_consensus.cc:1240] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:29.574411 16561 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:29.477025 16375 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:29.550491 16498 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:29.552011 16577 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:29.634703 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:29.635660 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:29.647550 10801 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:29.637174 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:29.647958 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:29.637957 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:29.638530 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:29.639128 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:29.599538 15859 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:29.639709 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:29.650676 10874 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
I20260709 13:18:29.640268 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:29.640704 16275 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:29.641898 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:29.642664 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:29.643057 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:29.656890 10873 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 14.
I20260709 13:18:29.643872 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:29.637360 10871 raft_consensus.cc:1240] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:29.653247 10797 raft_consensus.cc:1240] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:18:29.599802 16129 raft_consensus.cc:697] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 16 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:29.588389 16211 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:29.660564 10796 raft_consensus.cc:1240] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:18:29.663625 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:29.639148 16375 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:29.667078 10799 raft_consensus.cc:2368] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 13. Current term is 14.
I20260709 13:18:29.655011 16585 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.683588 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:29.684242 10871 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:29.689853 10801 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 15.
W20260709 13:18:29.691812 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 198494us
I20260709 13:18:29.692893 11478 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:29.693666 10800 raft_consensus.cc:1240] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:18:29.702567 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:29.668291 10916 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:29.706832 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:18:29.711174 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 129381us
I20260709 13:18:29.674242 16583 consensus_queue.cc:1059] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 13:18:29.719532 16583 raft_consensus.cc:3055] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:29.720131 16583 raft_consensus.cc:740] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:29.719504 16129 consensus_queue.cc:237] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:29.720938 16583 consensus_queue.cc:260] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:29.722131 16583 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:29.723381 16604 raft_consensus.cc:2749] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:29.683301 16506 consensus_queue.cc:237] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:29.728061 16211 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:29.728361 11478 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:29.712700 10766 leader_election.cc:336] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Timed out: RequestConsensusVote RPC to 127.9.116.132:40555 timed out after 1.692s (SENT)
I20260709 13:18:29.733656 10869 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 15.
I20260709 13:18:29.713271 10916 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:29.735431 16580 raft_consensus.cc:2764] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:18:29.735525 16604 raft_consensus.cc:2749] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:18:29.732311 10766 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:29.738560 10871 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 17.
I20260709 13:18:29.740685 16604 consensus_queue.cc:1048] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:18:29.742066 16580 consensus_queue.cc:1059] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:18:29.743455 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:29.728936 16516 raft_consensus.cc:2804] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:29.745908 16516 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:29.746300 16516 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:29.672628 16595 consensus_queue.cc:1059] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:18:29.754362 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:29.755019 10868 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:29.738569 10916 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:18:29.747054 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 179961us
I20260709 13:18:29.765280 10801 raft_consensus.cc:1240] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 14. Current term is 15. Ops: [13.11-14.12]
I20260709 13:18:29.766819 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:29.767953 10876 raft_consensus.cc:1275] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:18:29.775200 10841 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:29.775671 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:29.777251 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:29.777362 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:29.788797 11503 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
I20260709 13:18:29.778369 11510 raft_consensus.cc:1275] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:18:29.778281 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:29.779210 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:29.779582 11505 raft_consensus.cc:1275] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:18:29.779301 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:29.780277 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:29.782235 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:29.791267 10794 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
I20260709 13:18:29.791177 10869 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 15.
I20260709 13:18:29.792480 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:29.793172 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:29.793880 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:29.797430 10875 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:29.792727 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:29.795482 11478 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:29.798722 10799 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:29.801498 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:29.696730 16585 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:29.805148 16491 raft_consensus.cc:3055] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:29.774899 16375 raft_consensus.cc:2749] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:18:29.822090 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:29.823272 10941 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 15.
I20260709 13:18:29.814517 10801 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:29.824400 10943 raft_consensus.cc:1240] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 18. Current term is 19. Ops: []
I20260709 13:18:29.805963 16486 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:29.792681 16587 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:29.829612 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:29.827917 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:29.809188 16523 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:29.812808 16491 raft_consensus.cc:740] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:29.813620 16576 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.813403 16220 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.832509 11503 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 18.
I20260709 13:18:29.809442 16493 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.785915 10868 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 15.
I20260709 13:18:29.825713 10766 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:29.802315 16561 consensus_queue.cc:1059] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 13:18:29.819391 16349 raft_consensus.cc:2749] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:29.819500 16548 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.832458 10800 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 19.
I20260709 13:18:29.792963 16494 consensus_queue.cc:1048] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.032s
W20260709 13:18:29.832968 16603 log.cc:927] Time spent T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:18:29.848650 16486 raft_consensus.cc:697] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:29.852886 16576 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:29.858105 16242 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:29.880631 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:29.880735 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:29.881073 16242 raft_consensus.cc:697] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 15 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:29.881445 10949 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:18:29.882046 16242 consensus_queue.cc:237] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:29.882925 16587 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:29.883620 16584 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.884189 16584 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:29.863457 16275 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:29.885099 16587 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:29.882416 16576 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:29.879861 16500 consensus_queue.cc:1048] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.012s
I20260709 13:18:29.886670 16584 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:29.815393 16600 consensus_queue.cc:1059] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 13:18:29.893041 16192 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:29.869439 16585 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:29.822264 16511 consensus_queue.cc:1059] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
W20260709 13:18:29.872116 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 46484.5us
I20260709 13:18:29.873528 16591 raft_consensus.cc:2749] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:29.849594 16523 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:29.857108 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:29.871102 10801 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 15.
I20260709 13:18:29.877612 16486 consensus_queue.cc:237] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:29.826305 16292 consensus_queue.cc:1048] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.049s
I20260709 13:18:29.879945 10943 raft_consensus.cc:1275] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:18:29.880019 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:29.850569 16521 consensus_queue.cc:1048] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 13:18:29.897784 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:29.900777 10875 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 18.
I20260709 13:18:29.902697 16533 raft_consensus.cc:3055] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:29.939256 16533 raft_consensus.cc:740] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:29.940207 16533 consensus_queue.cc:260] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:29.940886 16349 consensus_queue.cc:1059] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:18:29.905346 15859 raft_consensus.cc:2749] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:18:29.942883 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:29.943070 16523 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:29.921130 10841 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:29.944401 16554 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:29.931214 16600 consensus_queue.cc:1243] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer 12682eb70bc84c76879e04a8628ee586 log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:18:29.947124 11507 raft_consensus.cc:2393] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 13: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:29.947044 15883 consensus_queue.cc:1048] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:18:29.948218 16577 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:29.946486 10798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2783) took 1095 ms. Trace:
I20260709 13:18:29.905956 16548 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:29.925105 10946 raft_consensus.cc:1240] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:29.949321 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:29.819578 16197 consensus_queue.cc:1048] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 13:18:29.950652 16548 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:29.946619 10841 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:29.950557 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:29.951323 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:29.951756 11507 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:29.949013 10798 rpcz_store.cc:276] 0709 13:18:28.851150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.851369 (+   219us) service_pool.cc:224] Handling call
0709 13:18:28.851684 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:28.875497 (+ 23813us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:28.875530 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:28.890927 (+ 15397us) log.cc:844] Serialized 26 byte log entry
0709 13:18:29.279144 (+388217us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:29.279196 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:29.279280 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:29.945933 (+666653us) spinlock_profiling.cc:243] Waited 98.7 ms on lock 0x7b54003064a0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80e2adbe 00007f9c80e35930 00007f9c80e358d8 00007f9c80e35858 00007f9c80e35813 00007f9c80e34a9c 00007f9c86d20626 00007f9c86d1add8 00007f9c80e717cb 00007f9c80e742c0
0709 13:18:29.946178 (+   245us) raft_consensus.cc:1685] finished
0709 13:18:29.946201 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:29.946423 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11025,"raft.queue_time_us":62778,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":24138,"spinlock_wait_cycles":227029248,"thread_start_us":227545,"threads_started":1,"wal-append.queue_time_us":269651}
I20260709 13:18:29.956874 10949 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 18.
I20260709 13:18:29.927278 10799 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 15.
I20260709 13:18:29.931592 11503 raft_consensus.cc:2393] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 15: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:29.915179 16498 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:29.953006 10841 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:29.959517 10841 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:18:29.961886 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 144619us
I20260709 13:18:29.855155 16220 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:29.966209 10799 raft_consensus.cc:3055] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:29.855471 16493 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:29.972417 10868 raft_consensus.cc:1275] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
W20260709 13:18:29.975580 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 237068us
I20260709 13:18:29.941505 16533 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:29.980728 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:29.981456 10875 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 19.
I20260709 13:18:29.984338 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:29.985281 11503 raft_consensus.cc:2393] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 15: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:29.986629 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:29.987270 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:29.990155 11507 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
I20260709 13:18:29.961694 16431 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:18:29.935396 16511 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.967085 16523 raft_consensus.cc:2804] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:30.000551 10875 raft_consensus.cc:1275] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:18:29.949697 15883 consensus_queue.cc:1048] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:18:29.997540 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:29.998811 10794 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 17.
I20260709 13:18:29.999096 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:30.005389 11507 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 14.
I20260709 13:18:29.980883 16393 raft_consensus.cc:2764] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:18:30.002373 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:30.006335 10916 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:30.007143 11503 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
I20260709 13:18:30.010003 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:18:30.012805 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 252791us
I20260709 13:18:30.016016 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:30.016654 11507 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 16.
I20260709 13:18:30.016650 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:30.018949 10916 leader_election.cc:400] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 election: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 13. Current term is 14.
I20260709 13:18:30.019474 10916 leader_election.cc:403] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:18:30.002791 16523 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:30.004371 16494 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:30.004806 16431 raft_consensus.cc:697] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:30.027765 10916 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:29.952188 16491 consensus_queue.cc:260] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:30.031776 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:30.015656 16617 raft_consensus.cc:2749] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:30.035491 11503 raft_consensus.cc:2393] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 18: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:30.029774 16523 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:30.030287 16494 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:30.002934 16511 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:30.040566 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:30.035881 10916 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:30.040750 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:18:30.040954 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 81519.4us
I20260709 13:18:30.039963 10947 raft_consensus.cc:1240] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:30.038103 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:30.042272 10944 raft_consensus.cc:1240] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:30.062392 10841 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:30.046238 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:30.035202 16506 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:29.967406 10799 raft_consensus.cc:740] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:30.065374 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:30.065672 10942 raft_consensus.cc:2393] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 12: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:30.065838 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:30.067163 10916 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:30.068317 11503 raft_consensus.cc:2393] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 13: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:30.028262 15859 consensus_queue.cc:1048] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 13:18:30.035409 16620 raft_consensus.cc:2749] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 13. Current term is 14.
I20260709 13:18:30.068708 10799 consensus_queue.cc:260] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:30.076578 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:30.072402 10916 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:30.077229 10876 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:30.078393 10916 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:30.079010 16619 raft_consensus.cc:2749] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:18:30.079900 16619 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:30.080286 16619 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:30.080618 16619 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:30.083333 16616 raft_consensus.cc:2749] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:18:30.086903 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:30.089128 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:30.089480 16220 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:30.060356 16494 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:30.064416 16491 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:18:30.039520 16580 consensus_queue.cc:1175] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" responder_term: 16 status { last_received { term: 14 index: 10 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 15. Current term is 16. Ops: [15.11-15.11]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:18:30.108400 10948 raft_consensus.cc:1275] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:18:30.100528 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:30.110320 10798 raft_consensus.cc:2393] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 13: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:30.102582 10876 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 13.
I20260709 13:18:30.106339 10799 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:30.105509 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:30.038249 16242 consensus_queue.cc:1059] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
W20260709 13:18:30.114351 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:30.118041 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:30.115840 10875 raft_consensus.cc:1240] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 13:18:30.070665 16192 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:30.075989 16506 raft_consensus.cc:697] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 15 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:30.071921 16625 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:18:30.132458 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 54147.2us
I20260709 13:18:30.133922 10916 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:30.135280 10916 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:30.112298 16554 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:30.134831 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:30.148187 10948 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 13.
I20260709 13:18:30.107234 16516 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:30.151121 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:30.152715 10947 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 14.
I20260709 13:18:30.138828 16617 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:18:30.150560 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:30.153577 10799 raft_consensus.cc:1275] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 13:18:30.155791 10799 pending_rounds.cc:85] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 11
I20260709 13:18:30.156165 10799 pending_rounds.cc:107] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 13:18:30.156492 10799 raft_consensus.cc:2889] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:18:30.156461 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 94186.1us
I20260709 13:18:30.146867 16242 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:30.149955 16192 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:30.187764 16506 consensus_queue.cc:237] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:30.158545 16617 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:30.155514 16493 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:30.121628 16431 consensus_queue.cc:237] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:30.186460 16619 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:30.219448 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:30.209893 10841 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:18:30.179566 16574 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 13:18:30.177896 16242 raft_consensus.cc:697] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 17 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:30.227123 16242 consensus_queue.cc:237] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:30.231266 16631 log.cc:927] Time spent T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:18:30.225031 10947 raft_consensus.cc:2393] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 16: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:30.202119 16192 raft_consensus.cc:697] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:30.205156 16617 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:18:30.259919 16633 log.cc:927] Time spent T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:18:30.261960 16192 consensus_queue.cc:237] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:30.290076 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 80274.5us
I20260709 13:18:30.304910 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:30.306002 10947 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 14.
I20260709 13:18:30.323438 10841 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:30.300405 16431 raft_consensus.cc:2804] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:30.342064 10876 raft_consensus.cc:1240] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:30.343245 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:30.343837 10876 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:30.318378 16511 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:30.354878 10875 raft_consensus.cc:1240] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 19. Current term is 20. Ops: []
I20260709 13:18:30.364094 10876 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
I20260709 13:18:30.340059 16431 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:30.371682 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:30.376873 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:30.321430 16554 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:30.377761 10875 raft_consensus.cc:2393] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 16: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:30.377876 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:30.376495 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:30.367062 16292 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:30.380491 16292 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:30.383535 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:30.384298 10950 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 16.
I20260709 13:18:30.378813 10947 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 14.
I20260709 13:18:30.373818 16431 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:30.394385 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:30.395306 10875 raft_consensus.cc:2393] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 14: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:30.387822 16648 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.411420 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:30.392268 16645 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.410904 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:30.413849 10875 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:30.415683 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:30.392997 16516 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:30.418098 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:30.420176 10842 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:30.421965 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:30.423183 10869 raft_consensus.cc:2393] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 15: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:30.388015 16554 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:30.411321 16644 raft_consensus.cc:2749] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:30.426932 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:18:30.430145 16626 log.cc:927] Time spent T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 13:18:30.430924 16554 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:30.431143 16650 log.cc:927] Time spent T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:18:30.433277 16644 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:30.433966 16644 raft_consensus.cc:697] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:30.435145 16644 consensus_queue.cc:237] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:30.437507 16643 log.cc:927] Time spent T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 13:18:30.420852 16645 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:30.438903 16647 log.cc:927] Time spent T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
W20260709 13:18:30.440270 16642 log.cc:927] Time spent T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 13:18:30.440734 16645 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:30.441313 16645 consensus_queue.cc:1048] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:30.443320 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:30.446007 16292 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:30.414028 16648 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:30.449167 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:30.419384 16556 raft_consensus.cc:3055] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:18:30.450500 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:30.456130 10870 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 16.
I20260709 13:18:30.456315 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:30.465502 10947 raft_consensus.cc:2410] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:18:30.420819 10876 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 14.
I20260709 13:18:30.474454 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:30.454619 16648 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:30.485162 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:30.485275 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:30.478061 16641 log.cc:927] Time spent T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
I20260709 13:18:30.487278 10869 raft_consensus.cc:2393] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 13: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:30.463830 16556 raft_consensus.cc:740] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:30.499163 10869 raft_consensus.cc:1275] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:18:30.504956 16292 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:30.508189 16292 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:30.510816 16292 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:30.514249 10875 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 14.
I20260709 13:18:30.512866 16657 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.518149 16657 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:30.530476 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:30.523011 16556 consensus_queue.cc:260] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:30.534224 16556 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:18:30.537521 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:30.534180 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:18:30.539039 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:30.540681 10944 raft_consensus.cc:1240] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:18:30.538501 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:30.545802 10942 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:18:30.545336 10947 raft_consensus.cc:2393] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 17: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:30.504513 16242 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.547133 16635 consensus_queue.cc:1059] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 13:18:30.547219 16242 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:30.562194 16635 raft_consensus.cc:3055] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:30.563864 16635 raft_consensus.cc:740] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:30.564760 16635 consensus_queue.cc:260] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:30.566156 16635 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:30.567140 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:30.568550 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:30.569178 10945 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
W20260709 13:18:30.572413 16608 log.cc:927] Time spent T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.671s	user 0.000s	sys 0.003s
I20260709 13:18:30.575560 16657 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:30.577201 16599 log.cc:927] Time spent T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 1.273s	user 0.006s	sys 0.000s
I20260709 13:18:30.577785 10916 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:30.579612 10916 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:30.570341 10945 raft_consensus.cc:1240] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:30.580436 16616 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:18:30.580511 15859 consensus_queue.cc:1048] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:30.580925 16616 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:30.581298 16523 consensus_meta.cc:287] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.515s	user 0.010s	sys 0.000s
I20260709 13:18:30.581403 16616 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:30.582123 15859 raft_consensus.cc:2804] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:30.582672 15859 raft_consensus.cc:697] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 14 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:18:30.583223 16612 log.cc:927] Time spent T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.681s	user 0.000s	sys 0.003s
I20260709 13:18:30.583582 15859 consensus_queue.cc:237] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:30.546020 16392 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:30.587051 16392 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:30.589361 16392 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:30.590881 16616 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:30.592182 16431 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:30.594477 16431 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:30.594820 10868 raft_consensus.cc:2368] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 12. Current term is 13.
I20260709 13:18:30.594410 16616 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:30.608191 10841 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:30.611542 10841 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:30.609804 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 186617us
I20260709 13:18:30.561520 16583 raft_consensus.cc:3055] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:30.631698 11477 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:30.632231 10841 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:30.633100 11477 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:30.637718 10767 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:30.612912 16575 raft_consensus.cc:2749] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:30.638900 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:30.639233 16575 raft_consensus.cc:2749] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:18:30.639551 11504 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:30.644107 16292 consensus_queue.cc:1059] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 17.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:18:30.645256 16292 raft_consensus.cc:3055] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:18:30.645613 16292 raft_consensus.cc:740] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:18:30.645946 16494 consensus_meta.cc:287] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.533s	user 0.008s	sys 0.002s
I20260709 13:18:30.648335 10841 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:30.649323 10868 raft_consensus.cc:1240] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:30.656517 10841 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:30.649757 10767 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:30.650096 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:30.650347 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:30.652247 10873 raft_consensus.cc:1240] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:30.659212 11508 raft_consensus.cc:1275] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:18:30.581758 16523 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:30.686569 10767 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:30.689563 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:30.647356 16511 raft_consensus.cc:2804] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:18:30.647075 16129 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:30.687111 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:30.650950 16392 raft_consensus.cc:2749] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:18:30.687728 10869 raft_consensus.cc:1275] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:18:30.631718 16583 raft_consensus.cc:740] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:30.652380 16657 raft_consensus.cc:2804] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:29.733634  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:30.703830 11504 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 21.
I20260709 13:18:30.699601 16220 raft_consensus.cc:2749] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:30.685338 16663 raft_consensus.cc:2804] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:30.693589 10767 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:30.710675 10767 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:30.707690 16663 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:30.686764 16574 raft_consensus.cc:2749] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:18:30.715310 10876 raft_consensus.cc:2393] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 15: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:30.647197 16533 consensus_queue.cc:1048] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.048s
I20260709 13:18:30.691901 16636 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:18:30.695037 16129 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:30.745965 16242 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:30.691534 16292 consensus_queue.cc:260] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:30.708591 16657 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:30.685474 16494 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:30.715339 16220 raft_consensus.cc:2804] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:30.718550 16392 raft_consensus.cc:2804] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:30.705281 16484 consensus_queue.cc:1059] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:18:30.719091 16636 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:30.715263 16663 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:30.747570 16292 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:30.742493 16129 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:30.773330 11509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56410 (request call id 891) took 2023 ms. Trace:
I20260709 13:18:30.776436 16494 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:30.781796 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:30.766800 16617 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:30.785030 16617 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:30.787716 10941 raft_consensus.cc:2393] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 20: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:30.789477 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:30.755291 16657 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:30.793237 16375 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.793339 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:30.793821 16375 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:30.794458 10946 raft_consensus.cc:2393] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 13: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:30.751176 16242 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.802698 16242 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:30.767275 16583 consensus_queue.cc:260] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:30.803813 16583 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:30.746727 16523 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:30.804667 16523 raft_consensus.cc:3055] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:30.805040 16523 raft_consensus.cc:740] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:30.762342 16665 consensus_queue.cc:1048] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 13:18:30.764887 16392 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:30.805927 16523 consensus_queue.cc:260] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:30.807418 16523 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:30.807747 16375 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:30.767997 16636 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:18:30.772027 16638 log.cc:927] Time spent T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.468s	user 0.001s	sys 0.000s
W20260709 13:18:30.770323 16588 log.cc:927] Time spent T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 1.464s	user 0.001s	sys 0.004s
I20260709 13:18:30.774323 11509 rpcz_store.cc:276] 0709 13:18:28.750217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.750386 (+   169us) service_pool.cc:224] Handling call
0709 13:18:28.760267 (+  9881us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:29.153060 (+392793us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:30.773069 (+1620009us) spinlock_profiling.cc:243] Waited 335 ms on lock 0x7b54005ff5d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:30.773270 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":770611584}
I20260709 13:18:30.762691 16220 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:30.810997 11512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56416 (request call id 895) took 2039 ms. Trace:
I20260709 13:18:30.811162 16220 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:30.812263 16242 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:30.811415 11512 rpcz_store.cc:276] 0709 13:18:28.771624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.857606 (+ 85982us) service_pool.cc:224] Handling call
0709 13:18:28.879865 (+ 22259us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:28.908779 (+ 28914us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:28.908812 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:28.912664 (+  3852us) log.cc:844] Serialized 26 byte log entry
0709 13:18:29.315305 (+402641us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:29.315347 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:29.315409 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:30.769730 (+1454321us) spinlock_profiling.cc:243] Waited 32.4 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:18:30.810622 (+ 40892us) raft_consensus.cc:1685] finished
0709 13:18:30.810661 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:30.810930 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2012,"raft.queue_time_us":14988,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":9470,"spinlock_wait_cycles":74431360,"thread_start_us":368212,"threads_started":1,"wal-append.queue_time_us":379902}
I20260709 13:18:30.812660 16619 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:30.813735 10792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2778) took 1987 ms. Trace:
I20260709 13:18:30.814097 10792 rpcz_store.cc:276] 0709 13:18:28.825876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.827520 (+  1644us) service_pool.cc:224] Handling call
0709 13:18:28.835499 (+  7979us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:29.613871 (+778372us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:30.813494 (+1199623us) spinlock_profiling.cc:243] Waited 736 ms on lock 0x7b54002125d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:30.813678 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1693773696}
I20260709 13:18:30.764597 16491 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.815253 16491 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:30.817322 16491 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:30.782495 10942 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 21.
I20260709 13:18:30.765882 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:30.800259 10946 raft_consensus.cc:1240] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 18. Current term is 20. Ops: []
I20260709 13:18:30.820470 10948 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:30.830121 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:30.830958 10868 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:30.831578 10942 raft_consensus.cc:1275] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:18:30.832716 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:30.834420 10946 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 14.
I20260709 13:18:30.837883 10873 raft_consensus.cc:1240] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:30.805960 16392 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:30.840317 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:30.845959 10942 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 20.
I20260709 13:18:30.847043 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:30.847765 10942 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 17.
I20260709 13:18:30.841131 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:18:30.837759 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 127195us
I20260709 13:18:30.844041 10870 raft_consensus.cc:1240] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:30.849514 10946 raft_consensus.cc:1240] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 14 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:30.855623 10948 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
W20260709 13:18:30.856163 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 162272us
I20260709 13:18:30.869608 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:30.874279 10942 raft_consensus.cc:1275] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:18:30.875988 10942 raft_consensus.cc:1275] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:18:30.889792 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:30.891139 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:30.896893 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:30.897768 10948 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:30.896592 10841 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:30.837328 16557 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.907886 10868 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 20.
I20260709 13:18:30.910705 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:30.843401 16574 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.910347 10916 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:30.914754 10841 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:30.908705 16292 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.917382 16292 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:30.894603 16657 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:30.919549 16292 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:30.903148 16665 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.915724 16557 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:30.920094 16292 consensus_queue.cc:1059] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 13:18:30.922987 16575 raft_consensus.cc:2804] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:30.923367 16557 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:30.923561 16575 raft_consensus.cc:697] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:30.867871 16491 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.924444 16655 consensus_queue.cc:1048] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:18:30.920894 16645 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:18:30.925014 16491 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:30.925475 16645 raft_consensus.cc:697] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:30.927258 16491 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:30.921281 16665 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:30.928609 16574 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:30.904153 16619 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:30.922456 15859 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:18:30.931308 16619 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:30.931711 15859 raft_consensus.cc:697] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 20 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:30.933581 16619 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:30.933166 15859 consensus_queue.cc:237] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:30.920753 16657 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:30.924479 16575 consensus_queue.cc:237] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:30.930738 16129 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:30.926718 16645 consensus_queue.cc:237] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:30.943290 16624 log.cc:927] Time spent T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.774s	user 0.000s	sys 0.003s
I20260709 13:18:30.930747 16574 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:30.944569 16129 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:30.842388 16511 raft_consensus.cc:697] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 18 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:30.949878 10948 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
I20260709 13:18:30.949239 10946 raft_consensus.cc:1275] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 13:18:30.963666 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:30.956442 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:30.930518 16665 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:30.924639 16486 consensus_queue.cc:1048] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:18:30.968954 10841 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:30.971799 16375 raft_consensus.cc:3055] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:30.972067 10873 raft_consensus.cc:2410] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:18:30.970517 10868 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 19.
I20260709 13:18:30.973569 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:30.980147 16659 consensus_queue.cc:1048] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:18:31.016871 10916 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:30.982616 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:30.984792 16636 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:30.991673 16220 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:30.975440 16375 raft_consensus.cc:740] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:31.002365 10941 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 14.
I20260709 13:18:31.002948 16523 raft_consensus.cc:2804] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:31.003257 10869 raft_consensus.cc:1275] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:18:30.972784 16511 consensus_queue.cc:237] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:31.006080 10868 raft_consensus.cc:1275] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:18:31.009302 10948 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
W20260709 13:18:31.010437 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:31.045188 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 284937us
I20260709 13:18:31.069073 10950 raft_consensus.cc:1275] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:18:31.111343 10841 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:18:31.114670 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 264619us
I20260709 13:18:31.116402 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:31.116495 10767 leader_election.cc:400] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 12. Current term is 13.
I20260709 13:18:31.118767 10767 leader_election.cc:403] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:31.120103 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:31.121610 10942 raft_consensus.cc:1240] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 19. Current term is 20. Ops: []
W20260709 13:18:31.123044 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 294243us
I20260709 13:18:31.123407 16493 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:18:31.123870 10766 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:31.129669 11505 raft_consensus.cc:1240] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:30.915899 16498 consensus_queue.cc:1059] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.193s
I20260709 13:18:31.133780 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:30.973507 16663 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:31.132493 16523 raft_consensus.cc:697] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:31.143826 16636 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:31.143599 16523 consensus_queue.cc:237] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:31.144435 16636 consensus_queue.cc:1048] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:18:31.145815 16636 consensus_queue.cc:1048] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:18:31.118717 16580 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:18:31.131008 16617 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:31.115993 16600 consensus_queue.cc:1048] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 13:18:31.148398 16486 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:18:31.148980 16500 consensus_queue.cc:1048] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.020s
I20260709 13:18:31.153743 16556 consensus_queue.cc:1059] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.053s
I20260709 13:18:31.168960 16486 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:31.158064 16617 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:31.249754 16612 log.cc:927] Time spent T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 13:18:31.247189 16491 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:31.251943 16486 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:18:31.271265 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 133936us
I20260709 13:18:31.274416 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:31.275276 10941 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:31.297082 16673 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:31.272823 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 147830us
I20260709 13:18:31.205936 16375 consensus_queue.cc:260] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:31.261732 16392 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:31.273348 16491 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:31.371204 10941 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 21.
I20260709 13:18:31.393535 10841 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:31.297857 16673 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:31.411172 16648 raft_consensus.cc:2749] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:18:31.351193 16220 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:31.423262 16699 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:31.423930 16699 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:31.424010 16220 consensus_queue.cc:1048] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:31.430910 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:31.433878 10876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 2334) took 1061 ms. Trace:
I20260709 13:18:31.426541 10841 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:31.401326 16375 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:31.388052 16617 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:31.392264 16392 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:31.430212 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:31.457576 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:31.434489 10876 rpcz_store.cc:276] 0709 13:18:30.372192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:30.474188 (+101996us) service_pool.cc:224] Handling call
0709 13:18:31.433578 (+959390us) spinlock_profiling.cc:243] Waited 235 ms on lock 0x7b540037ad54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:31.433791 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":539708160}
I20260709 13:18:31.455068 16556 consensus_queue.cc:1048] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:18:31.458216 11512 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 17.
I20260709 13:18:31.387383 16687 consensus_queue.cc:1059] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 13:18:31.348498 16600 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:18:31.479491 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 53060.5us
I20260709 13:18:31.456454 16663 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:31.480271 16655 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:31.405174 16644 consensus_queue.cc:1059] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.058s
W20260709 13:18:31.485002 16666 log.cc:927] Time spent T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.714s	user 0.003s	sys 0.000s
I20260709 13:18:31.462455 16620 raft_consensus.cc:3055] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:18:31.480091 16604 raft_consensus.cc:3055] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:18:31.510938 10841 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:31.518644 10941 raft_consensus.cc:1240] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:18:31.494692 16678 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:31.523104 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 397915us
I20260709 13:18:31.509053 16676 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:31.522712 16604 raft_consensus.cc:740] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:31.515574 16655 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:31.513998 16673 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:31.536720 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:31.540094 10941 raft_consensus.cc:1275] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:18:31.517688 16620 raft_consensus.cc:740] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:31.525983 16644 raft_consensus.cc:3055] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:31.543711 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:31.542168 10766 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:31.536307 16655 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:31.337750 16684 consensus_queue.cc:1059] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.237s
I20260709 13:18:31.561942 10766 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:31.546657 16645 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:18:31.547067 16644 raft_consensus.cc:740] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:31.582556 10766 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:31.584631 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 281846us
I20260709 13:18:31.526633 16678 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:31.573061 16632 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.266s	user 0.004s	sys 0.017s
I20260709 13:18:31.589906 10766 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:31.533631 16676 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:31.597777 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:31.625802 10796 raft_consensus.cc:2393] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 15: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:31.625965 10792 raft_consensus.cc:1275] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:18:31.598086 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:31.599174 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:31.628238 10799 raft_consensus.cc:2393] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 16: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:31.599267 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:31.600098 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:31.600193 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:31.568372 16683 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:31.546882 16675 consensus_queue.cc:1048] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
W20260709 13:18:31.642593 16704 log.cc:927] Time spent T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
W20260709 13:18:31.647596 16702 log.cc:927] Time spent T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:18:31.583691 16684 raft_consensus.cc:2804] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:31.585686 16645 raft_consensus.cc:697] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:31.709287 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:18:31.712311 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 190694us
W20260709 13:18:31.632786 16689 log.cc:927] Time spent T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
I20260709 13:18:31.715086 10800 raft_consensus.cc:2393] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 15: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
W20260709 13:18:31.623337 16632 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.322s	user 0.004s	sys 0.017s
I20260709 13:18:31.712102 10795 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 14.
I20260709 13:18:31.626210 16687 raft_consensus.cc:2749] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:18:31.727592 10941 raft_consensus.cc:1275] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:18:31.639385 16511 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:31.586977 16703 consensus_queue.cc:1059] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.147s
I20260709 13:18:31.680778 16498 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:31.579468 16604 consensus_queue.cc:260] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:31.726723 16491 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:31.767302 16511 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:31.729271 16684 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:31.747741 16710 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:31.835090 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 376664us
I20260709 13:18:31.712574 16712 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:31.765578 16699 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:31.632539 16620 consensus_queue.cc:260] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:31.835875 11478 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:31.839564 16604 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:31.844543 16710 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:31.840592 16684 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:31.841419 16709 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:31.726310 16644 consensus_queue.cc:260] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:31.783707 16498 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:31.834342 16500 consensus_queue.cc:1175] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "cb0428cc55984daba1ba63744a5102a2" responder_term: 20 status { last_received { term: 18 index: 13 } last_committed_idx: 13 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 19. Current term is 20. Ops: [19.14-19.14]" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:18:31.886682 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 296837us
I20260709 13:18:31.888125 10766 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:31.890240 10766 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:31.894330 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:31.895704 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:31.863307 16583 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:31.896502 11512 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
W20260709 13:18:31.899065 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:31.879015 16709 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:31.877429 16620 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:18:31.889166 16493 consensus_meta.cc:287] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.765s	user 0.014s	sys 0.001s
I20260709 13:18:31.921808 10766 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:31.876273 16718 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:18:31.926518 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 90733.6us
I20260709 13:18:31.849407 16712 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:31.915638 16493 raft_consensus.cc:2749] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 12. Current term is 13.
I20260709 13:18:31.931161 11478 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:31.927721 10766 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:31.912442 16644 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:31.909482 16491 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:31.905673 16706 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:31.933046 10766 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:18:31.939163 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:31.915776 16709 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:31.915846 16723 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:31.943167 10798 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
W20260709 13:18:31.948936 10915 leader_election.cc:336] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Timed out: RequestConsensusVote RPC to 127.9.116.132:40555 timed out after 1.544s (SENT)
I20260709 13:18:31.922947 16683 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:31.879215 16645 consensus_queue.cc:237] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:32.042984 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:31.907768 16583 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:31.929141 16728 raft_consensus.cc:2804] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:18:32.056128 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 206806us
I20260709 13:18:31.937155 16687 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:18:32.076719 10876 raft_consensus.cc:1240] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:18:31.941519 16706 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:32.076521 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:32.082684 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 149690us
I20260709 13:18:32.091374 10766 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:32.085626 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:32.085796 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:32.086743 10874 raft_consensus.cc:1240] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:32.107125 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:32.108700 10875 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:32.094596 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:32.093429 16493 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:32.107072 10870 raft_consensus.cc:1240] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:32.106536 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:32.113878 10876 raft_consensus.cc:1240] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 14. Current term is 15. Ops: []
W20260709 13:18:32.115512 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 184430us
I20260709 13:18:32.052493 16676 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:32.111446 16493 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:32.051978 16699 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.120193 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:32.043202 16683 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:32.120748 11478 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:32.106204 16671 log.cc:927] Time spent T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
I20260709 13:18:31.931934 16718 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:31.944070 16491 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:32.078140 16687 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:32.065585 16728 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:32.107709 16654 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:32.078477 16523 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:32.088909 16648 consensus_queue.cc:1048] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:18:32.094343 16735 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:32.089630 16656 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:18:32.093555 16715 log.cc:927] Time spent T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.325s	user 0.000s	sys 0.001s
I20260709 13:18:32.096508 16706 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:32.136821 16493 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:32.163066 10875 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
I20260709 13:18:32.142359 16683 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:18:32.167229 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 75269.2us
I20260709 13:18:32.043857 16723 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:32.123962 16665 consensus_queue.cc:1048] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 13:18:32.150821 16687 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:32.176317 10842 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:32.180511 16708 log.cc:927] Time spent T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.260s	user 0.000s	sys 0.001s
W20260709 13:18:32.181062 16486 consensus_meta.cc:287] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.879s	user 0.009s	sys 0.004s
I20260709 13:18:32.181222 16738 raft_consensus.cc:2804] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:32.181528 16486 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:32.181919 16738 raft_consensus.cc:697] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 15 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:32.182950 16738 consensus_queue.cc:237] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:32.183379 16729 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:32.184023 16729 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:32.186307 16732 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:32.187017 16676 consensus_queue.cc:1048] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:18:32.171912 16717 log.cc:927] Time spent T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.315s	user 0.000s	sys 0.001s
I20260709 13:18:32.188990 16732 raft_consensus.cc:697] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 15 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:32.189519 16663 raft_consensus.cc:2749] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:18:32.190347 16711 log.cc:927] Time spent T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.215s	user 0.001s	sys 0.000s
I20260709 13:18:32.180392 16740 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:32.192675 10867 raft_consensus.cc:1240] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:18:32.154757 16728 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:18:32.194599 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 103366us
I20260709 13:18:32.167868 16511 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:32.144797 16699 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:32.172122 16734 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.190090 16557 raft_consensus.cc:3055] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:32.229225 16683 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:32.233484 16716 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.234323 16716 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:32.164767 16654 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:32.190076 16732 consensus_queue.cc:237] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:32.164703 16523 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:32.242611 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:32.223426 16375 consensus_meta.cc:287] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.762s	user 0.005s	sys 0.007s
I20260709 13:18:32.248548 16745 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:18:32.248903 16649 log.cc:927] Time spent T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 1.942s	user 0.005s	sys 0.000s
I20260709 13:18:32.170305 16735 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:32.250227 16736 log.cc:927] Time spent T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
W20260709 13:18:32.250988 16726 log.cc:927] Time spent T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 13:18:32.222442 16557 raft_consensus.cc:740] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:18:32.251348 16707 log.cc:927] Time spent T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.295s	user 0.001s	sys 0.000s
W20260709 13:18:32.251652 16713 log.cc:927] Time spent T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.317s	user 0.001s	sys 0.000s
W20260709 13:18:32.252351 16722 log.cc:927] Time spent T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 13:18:32.331511 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:32.253919 16682 log.cc:927] Time spent T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 1.048s	user 0.005s	sys 0.003s
I20260709 13:18:32.171206 16656 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:32.256601 16654 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:32.212162 16734 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:32.259490 16720 log.cc:927] Time spent T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:18:32.204116 16729 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:32.301266 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 100343us
I20260709 13:18:32.346119 10766 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:32.346487 10871 raft_consensus.cc:1275] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:18:32.301218 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:32.232684 16683 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:32.247998 16716 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:32.305284 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 152866us
I20260709 13:18:32.264604 16498 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:32.264266 16710 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:32.334970 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:32.303968 16491 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:18:32.344722 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 138304us
I20260709 13:18:32.329432 16712 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:32.353425 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 161622us
I20260709 13:18:32.351641 11478 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:32.359666 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:32.369007 10945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2739) took 1015 ms. Trace:
I20260709 13:18:32.370391 10945 rpcz_store.cc:276] 0709 13:18:31.353542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:31.353735 (+   193us) service_pool.cc:224] Handling call
0709 13:18:31.354185 (+   450us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:31.574157 (+219972us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:31.574185 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:31.574644 (+   459us) log.cc:844] Serialized 26 byte log entry
0709 13:18:31.852903 (+278259us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:31.853426 (+   523us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:31.863285 (+  9859us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:32.368801 (+505516us) raft_consensus.cc:1685] finished
0709 13:18:32.368825 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:32.368948 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":123832,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":4449,"thread_start_us":450059,"threads_started":2,"wal-append.queue_time_us":518132}
I20260709 13:18:32.298306 16745 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:32.376008 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:32.388162 10871 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
I20260709 13:18:32.380167 16718 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:32.358954 16699 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:32.395511 10943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2738) took 1044 ms. Trace:
I20260709 13:18:32.364776 16498 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.366559 16716 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.398126 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:18:32.398996 16742 log.cc:927] Time spent T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 13:18:32.397846 10943 rpcz_store.cc:276] 0709 13:18:31.351580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:31.351801 (+   221us) service_pool.cc:224] Handling call
0709 13:18:31.352129 (+   328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:31.371913 (+ 19784us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:31.371962 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:31.372626 (+   664us) log.cc:844] Serialized 26 byte log entry
0709 13:18:31.585072 (+212446us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:31.585111 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:31.586126 (+  1015us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:32.395301 (+809175us) raft_consensus.cc:1685] finished
0709 13:18:32.395322 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:32.395455 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":90912,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":22331,"thread_start_us":211982,"threads_started":1,"wal-append.queue_time_us":461820}
I20260709 13:18:32.403936 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:32.372087 16486 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:32.372296 16654 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:32.402925 11505 raft_consensus.cc:2376] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 15. Re-sending same reply.
I20260709 13:18:32.405099 10950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 1001) took 1290 ms. Trace:
I20260709 13:18:32.396750 16743 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:32.394053 16727 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:18:32.405079 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 59073.1us
I20260709 13:18:32.407712 11510 raft_consensus.cc:2393] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 16: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:32.370391 16710 consensus_queue.cc:1048] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 13:18:32.395735 16712 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:32.370613 16683 consensus_queue.cc:1059] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:18:32.409472 10950 rpcz_store.cc:276] 0709 13:18:31.115105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:31.115420 (+   315us) service_pool.cc:224] Handling call
0709 13:18:31.115805 (+   385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:31.339841 (+224036us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:31.340053 (+   212us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:31.399552 (+ 59499us) log.cc:844] Serialized 26 byte log entry
0709 13:18:31.719486 (+319934us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:18:31.719534 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:31.719601 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:32.404817 (+685216us) raft_consensus.cc:1685] finished
0709 13:18:32.404846 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:32.405025 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11288,"raft.queue_time_us":32322,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":2196,"thread_start_us":445016,"threads_started":2,"wal-append.queue_time_us":683023}
I20260709 13:18:32.406612 16675 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:32.413921 10766 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:32.418989 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:32.420481 11505 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:32.426990 11508 raft_consensus.cc:1275] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 13:18:32.428128 16620 consensus_meta.cc:287] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.507s	user 0.010s	sys 0.000s
I20260709 13:18:32.428782 11506 raft_consensus.cc:1275] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:18:32.406857 16699 consensus_queue.cc:1059] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.015s
W20260709 13:18:32.434617 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 79052.6us
I20260709 13:18:32.376632 16557 consensus_queue.cc:260] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:32.466150 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:18:32.435276 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 138655us
I20260709 13:18:32.422596 16675 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:32.435170 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:32.406941 16716 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:32.472676 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:32.413285 16654 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:32.403913 16498 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:32.472729 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 58947.5us
I20260709 13:18:32.473500 11511 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 17.
I20260709 13:18:32.479830 10766 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:32.436434 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:32.426981 16723 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:32.416486 16712 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:32.434711 16523 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:32.495620 16746 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:32.502555 16746 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:32.472069 16683 consensus_queue.cc:1243] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer cb0428cc55984daba1ba63744a5102a2 log is divergent from this leader: its last log entry 14.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 13:18:32.478392 16733 log.cc:927] Time spent T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.180s	user 0.001s	sys 0.000s
I20260709 13:18:32.503434 16393 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.519466 16393 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:32.520761 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:32.522009 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:32.479121 16574 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:32.527248 16753 raft_consensus.cc:3055] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:32.527892 16753 raft_consensus.cc:740] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:32.528825 16753 consensus_queue.cc:260] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:32.529551 16721 log.cc:927] Time spent T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.189s	user 0.001s	sys 0.000s
I20260709 13:18:32.529536 16393 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:32.414342 16743 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:32.536221 16393 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:32.530123 16753 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:32.530809 11503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56416 (request call id 911) took 1103 ms. Trace:
I20260709 13:18:32.543181 11503 rpcz_store.cc:276] 0709 13:18:31.427720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:31.427908 (+   188us) service_pool.cc:224] Handling call
0709 13:18:31.455347 (+ 27439us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:31.455707 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:31.455738 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:31.456259 (+   521us) log.cc:844] Serialized 26 byte log entry
0709 13:18:31.747248 (+290989us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:18:31.747311 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:31.749301 (+  1990us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:32.530539 (+781238us) raft_consensus.cc:1685] finished
0709 13:18:32.530566 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:32.530704 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1484,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":27945,"thread_start_us":290004,"threads_started":1,"wal-append.queue_time_us":809267}
W20260709 13:18:32.544739 16655 consensus_meta.cc:287] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.971s	user 0.011s	sys 0.000s
I20260709 13:18:32.545187 16655 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:32.545075 16393 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:32.527244 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:32.535085 16728 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:32.537578 16743 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:32.508667 16759 log.cc:927] Time spent T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:18:32.538192 16486 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.538936 16687 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:32.540136 16498 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:32.540334 11505 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 20.
I20260709 13:18:32.437252 16718 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.540797 16656 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:32.542266 16727 raft_consensus.cc:3055] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:32.542104 16706 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:32.499181 16644 consensus_meta.cc:287] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.562s	user 0.007s	sys 0.005s
I20260709 13:18:32.507302 11478 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:32.436755 16744 consensus_queue.cc:1059] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.308s
W20260709 13:18:32.640146 10764 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:46758 dropped due to backpressure: service queue is full with 50 items
I20260709 13:18:32.475483 16751 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.560894 10794 raft_consensus.cc:2393] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 17: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:32.684412 11508 raft_consensus.cc:1275] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
W20260709 13:18:32.554864 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:32.503064 16746 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:32.643141 16750 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:18:32.691708 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 187342us
I20260709 13:18:32.689913 16729 raft_consensus.cc:2749] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:18:32.684676 16709 consensus_meta.cc:287] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.630s	user 0.007s	sys 0.004s
W20260709 13:18:32.691825 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 184605us
W20260709 13:18:32.550436 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 48708.8us
I20260709 13:18:32.696364 10874 raft_consensus.cc:2410] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
W20260709 13:18:32.695295 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 215502us
I20260709 13:18:32.724577 10766 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:32.616371 16716 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:32.683768 16727 raft_consensus.cc:740] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:18:32.603003 16684 consensus_meta.cc:287] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.722s	user 0.008s	sys 0.004s
I20260709 13:18:32.482690 16557 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:32.761148 16684 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:32.757651 16744 raft_consensus.cc:2749] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:32.762888 16696 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.476174 16675 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:32.688565 10794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2804) took 1086 ms. Trace:
I20260709 13:18:32.767668 10794 rpcz_store.cc:276] 0709 13:18:31.602295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:32.526961 (+924666us) service_pool.cc:224] Handling call
0709 13:18:32.688471 (+161510us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:32.723789 10767 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:32.606207 16393 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:32.728091 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:32.758733 10766 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:32.730840 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:32.769721 11511 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
I20260709 13:18:32.606178 16735 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:32.601806 16493 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:32.770740 16574 raft_consensus.cc:2749] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:32.771113 16728 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:32.605468 16755 raft_consensus.cc:2804] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:32.561347 16723 consensus_queue.cc:1048] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.195s
I20260709 13:18:32.771098 10766 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:32.772356 10798 raft_consensus.cc:2393] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 18: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:32.606868 16654 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:32.598758 16757 consensus_queue.cc:1059] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.198s
I20260709 13:18:32.772811 16728 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:32.771862 16493 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:32.772986 16755 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:32.774058 16755 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:32.602272 16734 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:32.690876 16718 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:32.680205 16760 consensus_queue.cc:1048] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.189s
I20260709 13:18:32.760673 16687 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:32.769109 11505 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 17.
I20260709 13:18:32.560858 16745 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:32.769766 16744 raft_consensus.cc:2804] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:32.776372 16744 raft_consensus.cc:697] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:32.778153 16718 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:32.783607 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:32.784456 16728 raft_consensus.cc:2749] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:18:32.784724 11503 raft_consensus.cc:2393] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 16: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
W20260709 13:18:32.770181 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 46469.8us
I20260709 13:18:32.757615 16748 consensus_queue.cc:1048] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
W20260709 13:18:32.787744 16604 consensus_meta.cc:287] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.911s	user 0.013s	sys 0.000s
I20260709 13:18:32.788228 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:32.788648 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:32.790040 11509 raft_consensus.cc:2410] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 13:18:32.791734 10766 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:32.792932 10766 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:32.793619 16687 raft_consensus.cc:2749] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:32.794296 16687 raft_consensus.cc:2749] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:18:32.796123 16694 log.cc:927] Time spent T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 1.490s	user 0.008s	sys 0.000s
I20260709 13:18:32.798547 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:32.799327 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:32.800506 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:32.729446 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:32.649891 16431 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:32.695762 16600 consensus_meta.cc:287] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 1.182s	user 0.014s	sys 0.000s
I20260709 13:18:32.618162 16712 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:32.810881 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:32.807305 16759 log.cc:927] Time spent T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:18:32.817183 10869 raft_consensus.cc:1240] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:32.722960 16498 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.665369 16486 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:32.806491 16375 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:32.824867 10914 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.018s	sys 0.025s
I20260709 13:18:32.806335 16493 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.805413 16716 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:32.777246 16744 consensus_queue.cc:237] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:32.722609 16750 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:32.695921 16655 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:32.815143 16728 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.763608 16696 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:32.811476 16600 raft_consensus.cc:2749] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:32.831467 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:32.823741 16349 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:18:32.827863 10914 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.019s	sys 0.025s
I20260709 13:18:32.833765 10868 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 17.
I20260709 13:18:32.836052 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:18:32.840718 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:32.840975 11511 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:32.722882 16709 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:32.837191 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:32.695307 16751 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:32.839046 11504 raft_consensus.cc:2410] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
W20260709 13:18:32.836311 16652 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.005s	sys 0.042s
I20260709 13:18:32.843426 11503 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 11.
I20260709 13:18:32.813359 16431 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:32.848609 11512 raft_consensus.cc:1240] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 15 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:32.689700 10764 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:46758 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2796)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2800)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2197)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2203)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2812)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2797)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39442 (request call id 791)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2199)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2809)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2795)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2201)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2806)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2815)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2794)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2810)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2196)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2805)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2821)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2825)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2824)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2194)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2827)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2828)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2829)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2830)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2802)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2826)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2823)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2798)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2801)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2803)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2807)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2811)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2191)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2192)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2195)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2200)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2202)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2204)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2205)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 792)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 793)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2813)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2814)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2816)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2817)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2818)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2819)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2820)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2822)
I20260709 13:18:32.825702 16498 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:32.829727 16375 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:32.829298 16493 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:32.889361 11511 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 18.
I20260709 13:18:32.882536 10766 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:18:32.849491 16652 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.702s	user 0.006s	sys 0.042s
I20260709 13:18:32.843894 16748 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:32.832249 16716 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:32.898617 10766 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:32.844563 16349 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:32.847028 16718 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:32.903167 16600 raft_consensus.cc:2804] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:32.839872 16728 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:32.906170 16557 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:18:32.855405 16620 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:18:32.907351 16738 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.721s	user 0.013s	sys 0.016s
I20260709 13:18:32.909564 16600 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:32.926573 16349 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:32.928808 16706 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:32.922118 16766 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:32.931526 16766 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:32.933672 16766 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:32.935444 16750 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:32.938712 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:32.934763 10764 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:39442 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2800)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2197)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2203)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2812)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2797)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39442 (request call id 791)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2199)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2809)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2795)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2201)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2806)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2815)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2794)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2810)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2196)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2805)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2821)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2825)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2824)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2194)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2827)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2828)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2829)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2830)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2802)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2826)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2823)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2831)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2798)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2801)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2803)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2807)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2811)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2191)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2192)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2195)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2200)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2202)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2204)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2205)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 792)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 793)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2813)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2814)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2816)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2817)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2818)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2819)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2820)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2822)
I20260709 13:18:32.938383 10916 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:32.923038 16557 raft_consensus.cc:697] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:32.911298 16763 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:32.945330 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:32.945683 16751 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:32.945950 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:32.954288 10869 raft_consensus.cc:2393] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 15: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:32.903640 16748 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:32.949928 16716 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:32.951556 16696 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:32.958753 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:32.923249 16738 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.737s	user 0.014s	sys 0.016s
I20260709 13:18:32.947412 16766 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:32.937496 16600 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:32.910760 16718 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:32.965642 10869 raft_consensus.cc:1240] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:18:32.965761 16728 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:32.966491 16431 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:32.966881 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:32.968514 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:32.952567 16375 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:32.961180 10871 raft_consensus.cc:2393] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 20: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:32.970772 10871 raft_consensus.cc:1275] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:18:32.956868 16763 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:32.968868 16486 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:32.960979 16766 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:32.949044 16645 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.971014 16580 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:32.982199 16580 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:32.985682 16745 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:32.972201 10869 raft_consensus.cc:2393] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 15: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:32.954938 10764 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:46758 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2800)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2197)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2203)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2812)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2797)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39442 (request call id 791)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2199)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2809)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2795)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2201)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2806)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2815)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2794)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2810)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2196)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2805)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2821)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2825)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2824)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2194)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2827)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2828)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2829)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2830)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2802)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2826)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2823)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2831)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2798)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2801)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2803)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2807)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2811)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2191)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2192)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2195)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2200)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2202)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2204)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2205)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 792)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 793)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2813)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2814)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2816)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2817)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2818)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2819)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2820)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2822)
I20260709 13:18:32.986325 16745 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:32.988713 16745 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:32.975358 11510 raft_consensus.cc:1240] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 15 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:32.975081 16349 consensus_queue.cc:1059] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:18:32.980175 16696 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.987723 10764 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:46758 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2800)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2197)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2203)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2812)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2797)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39442 (request call id 791)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2199)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2809)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2795)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2201)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2806)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2815)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2794)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2810)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2196)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2805)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2821)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2825)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2824)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2194)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2827)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2828)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2829)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2830)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2802)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2826)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2823)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2831)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2798)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2801)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2803)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2807)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2811)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2191)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2192)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2195)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2200)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2202)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2204)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2205)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 792)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 793)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2813)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2814)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2816)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2817)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2818)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2819)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2820)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2822)
I20260709 13:18:32.991091 16349 raft_consensus.cc:3055] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:18:32.980623 16716 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:32.990758 16696 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:32.991458 10798 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2198) took 1384 ms (client timeout 1557 ms). Trace:
W20260709 13:18:32.991947 10798 rpcz_store.cc:269] 0709 13:18:31.606556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:32.346375 (+739819us) service_pool.cc:224] Handling call
0709 13:18:32.991267 (+644892us) spinlock_profiling.cc:243] Waited 412 ms on lock 0x7b54002120d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:32.991401 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":946794880}
I20260709 13:18:32.993729 16696 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:32.994371 16493 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:32.994457 16716 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:32.991966 10764 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:46758 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2800)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2197)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2203)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2812)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2797)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39442 (request call id 791)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2199)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2809)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2795)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2201)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2806)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2815)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2794)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2810)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2196)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2805)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2821)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2825)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2824)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2194)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2827)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2828)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2829)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2830)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2802)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2826)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2823)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2831)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2798)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2801)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2803)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2807)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2811)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2191)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2192)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2195)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2200)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2202)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2204)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2205)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 792)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 793)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2813)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2814)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2816)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2817)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2818)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2819)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2820)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2822)
W20260709 13:18:32.996155 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:32.996629 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:32.997677 16716 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:32.999822 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:32.996881 10764 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:46758 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2800)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2197)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2203)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2812)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2797)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39442 (request call id 791)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2199)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2809)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2795)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2201)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2806)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2815)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2794)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2810)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2196)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2805)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2821)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2825)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2824)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2194)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2827)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2828)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2829)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2830)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2802)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2826)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2823)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2831)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2798)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2801)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2803)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2807)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2811)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2191)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2192)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2195)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2200)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2202)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2204)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2205)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 792)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 793)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2813)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2814)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2816)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2817)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2818)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2819)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2820)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2822)
I20260709 13:18:33.001253 16557 consensus_queue.cc:237] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:33.002650 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:33.003314 10945 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 19.
I20260709 13:18:33.003821 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:33.004451 10947 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 16.
I20260709 13:18:33.001344 10764 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:46758 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2800)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2197)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2203)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2812)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2797)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39442 (request call id 791)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2199)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2809)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2795)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2201)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2806)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2815)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2794)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2810)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2196)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2805)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2821)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2825)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2824)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2194)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2827)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2828)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2829)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2830)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2802)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2826)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2823)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2831)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2798)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2801)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2803)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2807)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2811)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2191)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2192)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2195)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2200)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2202)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2204)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2205)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 792)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 793)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2813)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2814)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2816)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2817)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2818)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2819)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2820)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2822)
I20260709 13:18:33.005645 10942 raft_consensus.cc:1275] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 13:18:32.992053 10800 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2790) took 1393 ms (client timeout 1817 ms). Trace:
I20260709 13:18:32.991519 16349 raft_consensus.cc:740] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:33.006649 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:33.010267 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:33.011126 10950 raft_consensus.cc:2410] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:18:33.011704 10945 raft_consensus.cc:1275] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:18:33.011981 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:33.012755 10943 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:18:33.017011 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:33.017339 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:33.017474 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:33.017951 10945 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:33.018060 11478 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:33.018282 10950 raft_consensus.cc:2393] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 21: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:33.018168 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:33.019348 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:18:33.006294 10800 rpcz_store.cc:269] 0709 13:18:31.598456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:31.599010 (+   554us) service_pool.cc:224] Handling call
0709 13:18:32.991855 (+1392845us) spinlock_profiling.cc:243] Waited 79.3 ms on lock 0x7b54000bc2d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:32.991976 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":182358912}
I20260709 13:18:33.006090 10764 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:46758 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2800)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2197)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2203)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2812)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2797)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39442 (request call id 791)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2199)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2809)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2795)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2201)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2806)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2815)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2794)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2810)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2196)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2805)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2821)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2825)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2824)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2194)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2827)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2828)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2829)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2830)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2802)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2826)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2837)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2823)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2798)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2801)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2803)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2807)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2811)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2191)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2192)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2195)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2200)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2202)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2204)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2205)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 792)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 793)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2813)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2814)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2816)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2817)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2818)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2819)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2820)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2822)
I20260709 13:18:32.922508 16620 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:33.021692 10944 raft_consensus.cc:2393] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 17: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:33.021561 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:33.022275 10950 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:33.028314 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:33.031705 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:33.032328 11503 raft_consensus.cc:2376] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 20. Re-sending same reply.
I20260709 13:18:33.034252 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:33.035921 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:33.036564 11503 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
I20260709 13:18:33.037180 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:33.038506 10869 raft_consensus.cc:1275] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:18:33.039544 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:33.040361 10947 raft_consensus.cc:2410] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 20 because replica has last-logged OpId of term: 19 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 8.
I20260709 13:18:32.965806 16718 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:33.046759 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:33.048309 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:33.050426 11477 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:33.050964 10949 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:32.968575 16748 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:33.048372 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:33.049981 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:33.057947 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:33.058616 10874 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 18.
I20260709 13:18:32.976962 16766 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:33.048542 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:33.055176 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:33.007925 16349 consensus_queue.cc:260] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:33.055812 16617 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:33.075325 16617 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:33.049114 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:33.073156 10800 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
W20260709 13:18:33.080751 10800 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2800) took 1479 ms (client timeout 1922 ms). Trace:
W20260709 13:18:33.081238 10800 rpcz_store.cc:269] 0709 13:18:31.601231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.029138 (+1427907us) service_pool.cc:224] Handling call
0709 13:18:33.080669 (+ 51531us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:33.078675 10942 raft_consensus.cc:2410] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 19 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 13:18:33.017582 16732 consensus_queue.cc:1048] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.103s
I20260709 13:18:33.026537 16645 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:33.041486 16738 raft_consensus.cc:2749] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:18:33.147316 10947 raft_consensus.cc:1240] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 14 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:18:33.145906 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:33.135881 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:33.150097 10946 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:33.137223 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:33.151340 10868 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 15.
I20260709 13:18:33.153299 10949 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 18.
I20260709 13:18:33.138367 10915 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:33.154466 10869 raft_consensus.cc:1240] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:18:33.154460 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:33.160578 10945 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
I20260709 13:18:33.157665 10874 raft_consensus.cc:2393] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 18: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:33.172235 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:18:33.175983 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:33.179527 10945 raft_consensus.cc:2393] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 17: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:33.172740 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:33.180473 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:33.180903 10800 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
W20260709 13:18:33.181370 10800 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2203) took 1573 ms (client timeout 1984 ms). Trace:
I20260709 13:18:33.172381 10915 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:18:33.181790 10800 rpcz_store.cc:269] 0709 13:18:31.607880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.123596 (+1515716us) service_pool.cc:224] Handling call
0709 13:18:33.181319 (+ 57723us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:33.182442 10915 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:33.079911 16724 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:18:33.183488 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:33.184103 10947 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 17.
W20260709 13:18:33.078033 16561 consensus_queue.cc:1175] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "cb0428cc55984daba1ba63744a5102a2" responder_term: 18 status { last_received { term: 18 index: 14 } last_committed_idx: 14 last_received_current_leader { term: 18 index: 14 } }
I20260709 13:18:33.137560 16678 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:33.163789 16349 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:33.187995 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:33.189262 10766 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:33.198868 10871 raft_consensus.cc:2410] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:18:33.191788 16557 consensus_queue.cc:1048] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 13:18:33.190063 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:33.201733 10942 raft_consensus.cc:2410] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:18:33.183566 16696 raft_consensus.cc:2764] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:18:33.203454 16220 raft_consensus.cc:2749] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:33.204126 16220 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:33.204566 16220 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:33.172602 16718 raft_consensus.cc:2749] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:33.180326 16580 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:33.207252 10795 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2792) took 1608 ms (client timeout 1902 ms). Trace:
W20260709 13:18:33.207662 10795 rpcz_store.cc:269] 0709 13:18:31.599049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:31.599856 (+   807us) service_pool.cc:224] Handling call
0709 13:18:33.207083 (+1607227us) spinlock_profiling.cc:243] Waited 82.7 ms on lock 0x7b54003ccb54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:33.207181 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":190160768}
I20260709 13:18:33.208825 16580 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:33.190663 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:33.210346 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:33.211000 10799 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 16.
I20260709 13:18:33.211218 16220 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:33.213644 10944 raft_consensus.cc:1240] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:18:33.215277 16494 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.215384 10766 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:33.215911 16494 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:33.217020 16220 raft_consensus.cc:2749] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:18:33.218226 16494 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:33.214638 16700 log.cc:927] Time spent T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 1.679s	user 0.010s	sys 0.001s
I20260709 13:18:33.219077 10766 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:33.220245 16220 raft_consensus.cc:2749] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:33.221820 10943 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
I20260709 13:18:33.224200 16620 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:33.225378 16684 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:33.211483 10799 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39442 (request call id 791) took 1362 ms (client timeout 1759 ms). Trace:
I20260709 13:18:33.137259 16645 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:33.208752 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:33.227267 10799 rpcz_store.cc:269] 0709 13:18:31.848652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.210074 (+1361422us) service_pool.cc:224] Handling call
0709 13:18:33.211426 (+  1352us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:33.083292 16675 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:33.230661 10946 raft_consensus.cc:2410] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:18:33.220745 10948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 1010) took 1504 ms. Trace:
I20260709 13:18:33.132020 16763 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:33.233327 10766 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:33.138535 16748 consensus_queue.cc:1048] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 13:18:33.238391 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:33.194418 16755 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:33.249373 10944 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 16.
I20260709 13:18:33.210379 10800 raft_consensus.cc:2393] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 21: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:33.185550 16745 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:33.241331 10948 rpcz_store.cc:276] 0709 13:18:31.716408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:31.716594 (+   186us) service_pool.cc:224] Handling call
0709 13:18:33.219995 (+1503401us) spinlock_profiling.cc:243] Waited 2.12 ms on lock 0x7b5800112598. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd90a22 00007f9c86ca9c09 00007f9c85e03ee0 00007f9c85ecd0ab 00007f9c85ecd131 00007f9c85e8efc0 00007f9c85e79929 00007f9c85e793d2 00007f9c7f3f0bd4
0709 13:18:33.220195 (+   200us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:33.220412 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:33.220436 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:33.220482 (+    46us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:33.220515 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:33.220572 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:33.220688 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4886784}
W20260709 13:18:33.244092 10795 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2797) took 1643 ms (client timeout 2003 ms). Trace:
I20260709 13:18:33.126677 16746 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:33.248021 10766 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:33.187139 16724 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:18:33.252924 10795 rpcz_store.cc:269] 0709 13:18:31.600392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.208490 (+1608098us) service_pool.cc:224] Handling call
0709 13:18:33.244026 (+ 35536us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:33.254299 10766 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:33.250367 10800 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2812) took 1645 ms (client timeout 1995 ms). Trace:
I20260709 13:18:33.255189 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:18:33.255204 10800 rpcz_store.cc:269] 0709 13:18:31.604534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.185557 (+1581023us) service_pool.cc:224] Handling call
0709 13:18:33.250289 (+ 64732us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:33.256569 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:33.181316 16738 consensus_queue.cc:1048] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.060s
I20260709 13:18:33.259572 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:33.262370 10942 raft_consensus.cc:2393] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 15: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:33.266364 10766 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:33.267561 10766 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:33.269042 10766 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:33.270628 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:18:33.272921 10799 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2199) took 1666 ms (client timeout 2035 ms). Trace:
W20260709 13:18:33.273389 10799 rpcz_store.cc:269] 0709 13:18:31.606833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.240920 (+1634087us) service_pool.cc:224] Handling call
0709 13:18:33.272857 (+ 31937us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:33.242611 16498 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:33.252177 16220 raft_consensus.cc:2749] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:33.255227 16724 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:33.268841 16753 raft_consensus.cc:2749] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:33.308518 10950 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
I20260709 13:18:33.310503 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:33.252447 16766 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:33.317988 10799 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 18.
W20260709 13:18:33.318658 10799 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2201) took 1711 ms (client timeout 2091 ms). Trace:
W20260709 13:18:33.318933 16771 log.cc:927] Time spent T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
W20260709 13:18:33.319058 10799 rpcz_store.cc:269] 0709 13:18:31.607369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.274040 (+1666671us) service_pool.cc:224] Handling call
0709 13:18:33.318597 (+ 44557us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:33.320158 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:33.320683 10841 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:33.283535 16728 raft_consensus.cc:2804] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:33.270603 16703 raft_consensus.cc:2804] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:33.275710 16375 raft_consensus.cc:2749] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:18:33.334131 10949 raft_consensus.cc:2393] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 17: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:33.271006 16493 consensus_queue.cc:1059] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 13:18:33.333031 16575 consensus_queue.cc:1048] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.008s
I20260709 13:18:33.340123 16764 consensus_queue.cc:1048] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:18:33.340993 16675 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:33.342700 16766 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:33.346276 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:33.297820 16574 consensus_queue.cc:1059] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:18:33.323029 16635 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:33.350639 11507 raft_consensus.cc:1240] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:33.325203 16728 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:33.359090 10945 raft_consensus.cc:1275] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:18:33.360697 11507 raft_consensus.cc:1275] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:18:33.362902 10945 raft_consensus.cc:1240] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:33.369901 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:33.374953 11507 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
I20260709 13:18:33.377833 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:33.377569 10799 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
I20260709 13:18:33.330322 16703 raft_consensus.cc:697] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:33.354498 16645 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.382654 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:33.378839 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:33.364635 16493 consensus_queue.cc:1243] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer cb0428cc55984daba1ba63744a5102a2 log is divergent from this leader: its last log entry 14.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:18:33.355036 16635 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:33.385779 10945 raft_consensus.cc:2393] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 20: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:33.387380 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:33.366513 16728 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:33.395411 11477 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:33.380131 10799 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2806) took 1777 ms (client timeout 2120 ms). Trace:
W20260709 13:18:33.399045 10799 rpcz_store.cc:269] 0709 13:18:31.602936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.319884 (+1716948us) service_pool.cc:224] Handling call
0709 13:18:33.380052 (+ 60168us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:33.400321 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:33.400539 10946 raft_consensus.cc:1240] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 14. Current term is 15. Ops: []
W20260709 13:18:33.401764 10799 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2815) took 1465 ms (client timeout 1842 ms). Trace:
W20260709 13:18:33.402172 10799 rpcz_store.cc:269] 0709 13:18:31.936871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.399924 (+1463053us) service_pool.cc:224] Handling call
0709 13:18:33.401689 (+  1765us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:33.408843 10870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2355) took 1322 ms. Trace:
I20260709 13:18:33.410143 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:33.412045 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:33.410509 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:33.393082 16635 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:33.433975 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:33.436425 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:33.437736 16706 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:33.447702 16703 consensus_queue.cc:237] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:33.415230 16684 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:33.392689 16645 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:33.451903 16699 raft_consensus.cc:3055] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:33.586827 16699 raft_consensus.cc:740] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:33.587872 16699 consensus_queue.cc:260] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:33.588960 16645 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:33.589600 16699 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:33.433449 16750 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:33.401351 16732 raft_consensus.cc:2749] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:18:33.495569 10800 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
W20260709 13:18:33.597230 10800 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2795) took 1997 ms (client timeout 2082 ms). Trace:
W20260709 13:18:33.597759 10800 rpcz_store.cc:269] 0709 13:18:31.599890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.256338 (+1656448us) service_pool.cc:224] Handling call
0709 13:18:33.597161 (+340823us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:33.524674 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:33.538580 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:33.541499 10795 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
W20260709 13:18:33.600121 10795 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2809) took 1996 ms (client timeout 2041 ms). Trace:
W20260709 13:18:33.600484 10795 rpcz_store.cc:269] 0709 13:18:31.603819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.254933 (+1651114us) service_pool.cc:224] Handling call
0709 13:18:33.600059 (+345126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:33.597254 10798 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 20.
I20260709 13:18:33.542366 10870 rpcz_store.cc:276] 0709 13:18:32.086828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:32.112999 (+ 26171us) service_pool.cc:224] Handling call
0709 13:18:32.140949 (+ 27950us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:32.298846 (+157897us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:18:32.410079 (+111233us) log.cc:844] Serialized 14 byte log entry
0709 13:18:32.575731 (+165652us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:32.575817 (+    86us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:32.575879 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:33.408590 (+832711us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b540009a1d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:33.408645 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:33.408788 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":117242,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":1638,"raft.queue_time_us":122,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":98,"spinlock_wait_cycles":362328320,"thread_start_us":274737,"threads_started":2,"wal-append.queue_time_us":246333}
W20260709 13:18:33.601840 10798 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2197) took 1995 ms (client timeout 1964 ms). Trace:
I20260709 13:18:33.413327 16392 raft_consensus.cc:3055] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:33.556278 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:33.602617 10950 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
W20260709 13:18:33.562764 10873 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 2201) took 2722 ms (client timeout 2694 ms). Trace:
I20260709 13:18:33.450034 16583 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:33.583190 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:33.548502 10946 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
W20260709 13:18:33.604257 10798 rpcz_store.cc:269] 0709 13:18:31.606287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.411849 (+1805562us) service_pool.cc:224] Handling call
0709 13:18:33.601456 (+189607us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:33.603461 10873 rpcz_store.cc:269] 0709 13:18:30.823715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:30.890898 (+ 67183us) service_pool.cc:224] Handling call
0709 13:18:33.537982 (+2647084us) spinlock_profiling.cc:243] Waited 80.1 ms on lock 0x7b5400078d54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:33.545641 (+  7659us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":184124160}
W20260709 13:18:33.606942 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:33.607486 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:33.609128 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:33.610059 10795 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 14.
W20260709 13:18:33.610733 10795 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2805) took 2008 ms (client timeout 2396 ms). Trace:
I20260709 13:18:33.610772 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:33.611380 10795 rpcz_store.cc:269] 0709 13:18:31.602634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.608859 (+2006225us) service_pool.cc:224] Handling call
0709 13:18:33.610660 (+  1801us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:33.612998 10800 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2196) took 2007 ms (client timeout 2379 ms). Trace:
W20260709 13:18:33.613514 10800 rpcz_store.cc:269] 0709 13:18:31.606044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.607208 (+2001164us) service_pool.cc:224] Handling call
0709 13:18:33.612922 (+  5714us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:33.614353 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:33.614912 10800 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:33.618988 10696 catalog_manager.cc:5697] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:18:33.619926 16776 log.cc:927] Time spent T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.216s	user 0.000s	sys 0.000s
I20260709 13:18:33.620865 10696 catalog_manager.cc:5697] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:33.605943 16392 raft_consensus.cc:740] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:33.623678 16392 consensus_queue.cc:260] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:33.624733 16392 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:33.626117 16755 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:33.605347 16684 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:33.628734 16684 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:33.618983 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:33.632781 10945 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:33.633428 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:33.634413 10798 raft_consensus.cc:2376] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13. Re-sending same reply.
I20260709 13:18:33.609206 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:33.635761 10799 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2810) took 2031 ms (client timeout 2218 ms). Trace:
W20260709 13:18:33.636124 10799 rpcz_store.cc:269] 0709 13:18:31.604071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.577002 (+1972931us) service_pool.cc:224] Handling call
0709 13:18:33.635691 (+ 58689us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2432}
I20260709 13:18:33.639462 10796 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
I20260709 13:18:33.639771 10798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2825) took 1106 ms. Trace:
I20260709 13:18:33.640348 10798 rpcz_store.cc:276] 0709 13:18:32.533178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.633045 (+1099867us) service_pool.cc:224] Handling call
0709 13:18:33.639691 (+  6646us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:33.641827 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:33.643533 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:33.639467 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:33.646813 10798 raft_consensus.cc:2393] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 16: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:33.656122 10799 raft_consensus.cc:2393] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 15: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:33.656765 10799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2827) took 1123 ms. Trace:
I20260709 13:18:33.657192 10799 rpcz_store.cc:276] 0709 13:18:32.533696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.636839 (+1103143us) service_pool.cc:224] Handling call
0709 13:18:33.656680 (+ 19841us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:33.659061 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:33.659761 10946 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
I20260709 13:18:33.660842 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:33.661264 16600 consensus_meta.cc:287] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.685s	user 0.007s	sys 0.003s
I20260709 13:18:33.661652 10795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2824) took 1128 ms. Trace:
I20260709 13:18:33.662035 10795 rpcz_store.cc:276] 0709 13:18:32.532797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.637389 (+1104592us) service_pool.cc:224] Handling call
0709 13:18:33.661592 (+ 24203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:33.663062 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:33.633914 16746 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:33.663655 10795 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
I20260709 13:18:33.664177 10795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2830) took 1129 ms. Trace:
I20260709 13:18:33.664469 10795 rpcz_store.cc:276] 0709 13:18:32.534600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.662768 (+1128168us) service_pool.cc:224] Handling call
0709 13:18:33.664123 (+  1355us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:33.665390 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:33.666152 10795 raft_consensus.cc:2393] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 21: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:33.658492 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:33.667814 10799 raft_consensus.cc:2393] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 15: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:33.668375 10799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2829) took 1134 ms. Trace:
I20260709 13:18:33.668771 10799 rpcz_store.cc:276] 0709 13:18:32.534176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.658091 (+1123915us) service_pool.cc:224] Handling call
0709 13:18:33.668289 (+ 10198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:33.669978 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:33.671074 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:33.671765 10798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2828) took 1137 ms. Trace:
I20260709 13:18:33.672258 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:33.661691 16600 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:33.675295 10948 raft_consensus.cc:2393] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 18: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
W20260709 13:18:33.675482 10796 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2821) took 1303 ms (client timeout 1636 ms). Trace:
W20260709 13:18:33.675918 10796 rpcz_store.cc:269] 0709 13:18:32.371763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.610540 (+1238777us) service_pool.cc:224] Handling call
0709 13:18:33.675324 (+ 64784us) spinlock_profiling.cc:243] Waited 27.7 ms on lock 0x7b540024ead4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:33.675426 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":63636352}
I20260709 13:18:33.672957 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:18:33.674108 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:33.677227 10950 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
I20260709 13:18:33.666548 10795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2802) took 2064 ms. Trace:
I20260709 13:18:33.679270 10798 rpcz_store.cc:276] 0709 13:18:32.533939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.641494 (+1107555us) service_pool.cc:224] Handling call
0709 13:18:33.671694 (+ 30200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:33.679378 10795 rpcz_store.cc:276] 0709 13:18:31.601769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.665112 (+2063343us) service_pool.cc:224] Handling call
0709 13:18:33.666495 (+  1383us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:33.673946 10799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2826) took 1140 ms. Trace:
I20260709 13:18:33.680681 10799 rpcz_store.cc:276] 0709 13:18:32.533453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.669602 (+1136149us) service_pool.cc:224] Handling call
0709 13:18:33.673858 (+  4256us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:33.688041 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2354) took 1601 ms. Trace:
I20260709 13:18:33.693095 10948 raft_consensus.cc:3055] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:33.693595 10948 raft_consensus.cc:740] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:33.584025 16706 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:33.694883 11511 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 19.
I20260709 13:18:33.688622 10867 rpcz_store.cc:276] 0709 13:18:32.086567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:32.111906 (+ 25339us) service_pool.cc:224] Handling call
0709 13:18:32.112298 (+   392us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:32.194693 (+ 82395us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:33.687632 (+1492939us) spinlock_profiling.cc:243] Waited 80.1 ms on lock 0x7b540006bdd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:33.687954 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":184175488}
I20260709 13:18:33.697132 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:33.697994 10798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2823) took 1298 ms. Trace:
I20260709 13:18:33.698326 10798 rpcz_store.cc:276] 0709 13:18:32.399737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.680470 (+1280733us) service_pool.cc:224] Handling call
0709 13:18:33.697934 (+ 17464us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:33.700378 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:33.701417 10796 raft_consensus.cc:2393] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 17: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:33.704862 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:33.705750 10799 raft_consensus.cc:2393] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 18: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:33.712679 10946 raft_consensus.cc:3055] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:33.713358 10946 raft_consensus.cc:740] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:33.694419 10948 consensus_queue.cc:260] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:33.714635 10948 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:33.715951 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:33.717957 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:33.717875 10945 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
I20260709 13:18:33.601190 16750 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:33.719097 10795 raft_consensus.cc:2393] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 15: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:33.719646 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:33.720369 10798 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:33.727908 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:33.729521 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:33.736064 10798 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
I20260709 13:18:33.731953 10946 consensus_queue.cc:260] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:33.777755 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:33.777990 10800 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 17.
W20260709 13:18:33.778918 10800 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2194) took 2173 ms (client timeout 2694 ms). Trace:
W20260709 13:18:33.792730 10800 rpcz_store.cc:269] 0709 13:18:31.605549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.614139 (+2008590us) service_pool.cc:224] Handling call
0709 13:18:33.778834 (+164695us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:33.794276 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:33.742257 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:33.796028 10798 raft_consensus.cc:2393] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 21: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:33.804594 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:33.805459 10795 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 16.
I20260709 13:18:33.806573 10946 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:33.817940 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:33.795140 10800 raft_consensus.cc:2376] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15. Re-sending same reply.
I20260709 13:18:33.820699 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:33.821367 10798 raft_consensus.cc:2376] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13. Re-sending same reply.
I20260709 13:18:33.738137 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:33.821990 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:33.823669 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:33.825282 10800 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 17.
I20260709 13:18:33.828755 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:33.829705 10798 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
I20260709 13:18:33.830912 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:33.836514 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:33.837471 10798 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
I20260709 13:18:33.840502 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:33.835014 10948 raft_consensus.cc:1275] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 13:18:33.839047 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:33.842419 10948 pending_rounds.cc:85] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 10
I20260709 13:18:33.842876 10948 pending_rounds.cc:107] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 13:18:33.843250 10948 raft_consensus.cc:2889] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:33.844795 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:33.847090 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:33.847802 10796 raft_consensus.cc:2376] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15. Re-sending same reply.
I20260709 13:18:33.848841 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:33.849617 10796 raft_consensus.cc:2376] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 20. Re-sending same reply.
I20260709 13:18:33.849720 10946 raft_consensus.cc:1275] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:18:33.738582 16635 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:33.852789 10798 raft_consensus.cc:1240] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:33.853569 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:33.855996 10798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2798) took 2255 ms. Trace:
I20260709 13:18:33.855480 10800 raft_consensus.cc:1240] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 18. Current term is 19. Ops: []
I20260709 13:18:33.857096 10800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2801) took 2255 ms. Trace:
I20260709 13:18:33.857476 10800 rpcz_store.cc:276] 0709 13:18:31.601506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.853228 (+2251722us) service_pool.cc:224] Handling call
0709 13:18:33.855312 (+  2084us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:33.856813 (+  1501us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:33.857030 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:33.858878 10798 rpcz_store.cc:276] 0709 13:18:31.600689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.852221 (+2251532us) service_pool.cc:224] Handling call
0709 13:18:33.852546 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:33.855771 (+  3225us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:33.855944 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:33.861704 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:33.864686 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:33.865721 10795 raft_consensus.cc:2410] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:18:33.866657 10797 raft_consensus.cc:1240] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:33.866704 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:33.868952 10798 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
I20260709 13:18:33.871654 10797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2807) took 2268 ms. Trace:
I20260709 13:18:33.872104 10797 rpcz_store.cc:276] 0709 13:18:31.603168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.853333 (+2250165us) service_pool.cc:224] Handling call
0709 13:18:33.866497 (+ 13164us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:33.871363 (+  4866us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:33.871594 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:33.904947 11512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56404 (request call id 987) took 1482 ms. Trace:
I20260709 13:18:33.905544 11512 rpcz_store.cc:276] 0709 13:18:32.422505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:32.422630 (+   125us) service_pool.cc:224] Handling call
0709 13:18:32.422968 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:32.881162 (+458194us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:33.904622 (+1023460us) spinlock_profiling.cc:243] Waited 425 ms on lock 0x7b54006dd154. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:33.904866 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":978391296}
I20260709 13:18:33.909233 10945 raft_consensus.cc:1240] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:33.855479 16738 consensus_queue.cc:1048] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 13:18:33.984961 16349 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:33.985572 16349 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:33.990985 16575 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.991578 16575 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:33.996574 16789 log.cc:927] Time spent T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 13:18:33.854091 16732 consensus_queue.cc:1048] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
W20260709 13:18:34.054387 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:34.061264 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:34.062290 10798 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 15.
I20260709 13:18:34.082079 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:34.066777 10841 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:18:34.096800 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:34.115284 10798 raft_consensus.cc:1240] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:34.116662 10798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2192) took 2511 ms. Trace:
I20260709 13:18:34.117146 10798 rpcz_store.cc:276] 0709 13:18:31.605034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.114176 (+2509142us) service_pool.cc:224] Handling call
0709 13:18:34.114885 (+   709us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:34.116303 (+  1418us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:34.116582 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:34.119017 10798 raft_consensus.cc:1240] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:18:34.119755 10798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2200) took 2512 ms. Trace:
I20260709 13:18:34.119968 10841 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:34.120131 10798 rpcz_store.cc:276] 0709 13:18:31.607097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.118229 (+2511132us) service_pool.cc:224] Handling call
0709 13:18:34.118713 (+   484us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:34.119509 (+   796us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:34.119703 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:34.056780 16619 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 13:18:34.151592 10949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41014 (request call id 2442) took 1017 ms. Trace:
I20260709 13:18:34.009249 16645 consensus_queue.cc:1059] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.118s
I20260709 13:18:34.133934 16575 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:34.129768 16760 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.020s	sys 0.024s
W20260709 13:18:34.141407 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:34.141634 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:34.024256 16349 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:34.129526 16655 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:34.152264 10949 rpcz_store.cc:276] 0709 13:18:33.133950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.154235 (+ 20285us) service_pool.cc:224] Handling call
0709 13:18:34.151298 (+997063us) spinlock_profiling.cc:243] Waited 162 ms on lock 0x7b54002a58d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:34.151509 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":373375232}
I20260709 13:18:34.163436 10798 raft_consensus.cc:1275] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:18:34.150014 16600 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:34.043087 16784 consensus_queue.cc:1048] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.219s
I20260709 13:18:34.121729 16764 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:34.173458 10868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 2218) took 1093 ms. Trace:
W20260709 13:18:34.159662 16745 consensus_meta.cc:287] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.900s	user 0.005s	sys 0.006s
W20260709 13:18:34.172857 16760 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.021s	sys 0.024s
I20260709 13:18:34.179337 16655 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:34.124817 16787 consensus_queue.cc:1059] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 13:18:34.192471 10868 rpcz_store.cc:276] 0709 13:18:33.079825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.141337 (+1061512us) service_pool.cc:224] Handling call
0709 13:18:34.173373 (+ 32036us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:34.203368 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 345200us
I20260709 13:18:34.177696 10798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2202) took 2570 ms. Trace:
I20260709 13:18:34.207657 10798 rpcz_store.cc:276] 0709 13:18:31.607634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.137311 (+2529677us) service_pool.cc:224] Handling call
0709 13:18:34.140717 (+  3406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:34.177234 (+ 36517us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:34.177624 (+   390us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17280}
W20260709 13:18:34.210348 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:34.217262 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:34.219257 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:34.190413 16750 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:18:34.205042 16724 consensus_meta.cc:287] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.889s	user 0.011s	sys 0.002s
I20260709 13:18:34.224247 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:34.220891 11507 raft_consensus.cc:1275] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:18:34.196976 16764 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:34.249541 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:34.260699 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:34.247486 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:34.201332 16745 raft_consensus.cc:2749] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:34.208494 16655 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:34.220453 16738 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:34.262722 11512 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:18:34.270047 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:34.270694 11509 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56404 (request call id 988) took 1846 ms (client timeout 1964 ms). Trace:
W20260709 13:18:34.272061 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:34.272049 10792 raft_consensus.cc:1240] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:34.277045 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:34.259275 10947 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.028s	sys 0.071s
I20260709 13:18:34.221115 16349 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:34.297941 10792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2191) took 2693 ms. Trace:
I20260709 13:18:34.260138 16764 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:34.281517 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:18:34.282554 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:34.288307 16727 consensus_queue.cc:260] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:34.295816 11509 rpcz_store.cc:269] 0709 13:18:32.424174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:32.424370 (+   196us) service_pool.cc:224] Handling call
0709 13:18:34.270379 (+1846009us) spinlock_profiling.cc:243] Waited 309 ms on lock 0x7b540046d2d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:34.270574 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":709894016}
W20260709 13:18:34.296417 10947 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.028s	sys 0.072s
I20260709 13:18:34.296991 10795 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 15.
W20260709 13:18:34.278745 16781 log.cc:927] Time spent T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
I20260709 13:18:34.298331 10792 rpcz_store.cc:276] 0709 13:18:31.604797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.271464 (+2666667us) service_pool.cc:224] Handling call
0709 13:18:34.271851 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:34.297610 (+ 25759us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:34.297857 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:34.280148 16709 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:34.262212 16635 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:34.315625 10792 raft_consensus.cc:1240] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:34.316331 10792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 792) took 2467 ms. Trace:
I20260709 13:18:34.316653 10792 rpcz_store.cc:276] 0709 13:18:31.848977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.310393 (+2461416us) service_pool.cc:224] Handling call
0709 13:18:34.315459 (+  5066us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:34.316085 (+   626us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:34.316275 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:34.317937 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:34.259501 16724 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:34.326777 10795 raft_consensus.cc:1275] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:18:34.334853 10795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2205) took 2726 ms. Trace:
I20260709 13:18:34.335354 10795 rpcz_store.cc:276] 0709 13:18:31.608388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.307793 (+2699405us) service_pool.cc:224] Handling call
0709 13:18:34.326433 (+ 18640us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:34.334431 (+  7998us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:34.334743 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:34.324115 16728 consensus_meta.cc:287] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.920s	user 0.011s	sys 0.000s
I20260709 13:18:34.324400 16727 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:34.297355 16738 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:34.424209 16500 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:34.307042 16349 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:34.388372 16699 consensus_meta.cc:287] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.798s	user 0.011s	sys 0.003s
W20260709 13:18:34.388160 16781 log.cc:927] Time spent T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
W20260709 13:18:34.436592 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 317200us
I20260709 13:18:34.434720 16557 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:18:34.440428 16795 log.cc:927] Time spent T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:18:34.433924 16500 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:34.447156 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:34.448267 10868 raft_consensus.cc:2368] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 14. Current term is 15.
I20260709 13:18:34.449983 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:34.451268 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:34.452144 10870 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
I20260709 13:18:34.453944 10873 raft_consensus.cc:1240] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:18:34.457260 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:34.460354 16617 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:18:34.463173 16774 log.cc:927] Time spent T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 1.135s	user 0.011s	sys 0.013s
I20260709 13:18:34.439327 10841 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:34.425532 16728 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:34.483422 10876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2359) took 2388 ms. Trace:
I20260709 13:18:34.484576 10876 rpcz_store.cc:276] 0709 13:18:32.087764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:32.119847 (+ 32083us) service_pool.cc:224] Handling call
0709 13:18:32.120371 (+   524us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:18:32.138158 (+ 17787us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:32.138635 (+   477us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:18:32.145075 (+  6440us) log.cc:844] Serialized 78 byte log entry
0709 13:18:33.008183 (+863108us) spinlock_profiling.cc:243] Waited 2.52 ms on lock 0x7b1800361420. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a550d1 00007f9c80e70a0f
0709 13:18:33.171535 (+163352us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:33.171575 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:33.171640 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:34.462540 (+1290900us) spinlock_profiling.cc:243] Waited 1.35 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:18:34.475493 (+ 12953us) raft_consensus.cc:1685] finished
0709 13:18:34.475520 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:34.475701 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":734,"raft.queue_time_us":23753,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":6586,"spinlock_wait_cycles":8907904,"thread_start_us":163135,"threads_started":1,"wal-append.queue_time_us":1111023}
W20260709 13:18:34.486443 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 47211us
W20260709 13:18:34.487916 10800 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.014s	sys 0.078s
W20260709 13:18:34.488343 10800 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.015s	sys 0.078s
I20260709 13:18:34.488214 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:34.490607 10867 raft_consensus.cc:2393] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 19: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:34.491827 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:34.492954 10876 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
I20260709 13:18:34.490429 10841 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:34.503629 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:34.487689 16799 log.cc:927] Time spent T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:18:34.504606 10872 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 14.
I20260709 13:18:34.514753 10801 raft_consensus.cc:2368] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 18. Current term is 19.
I20260709 13:18:34.496331 16632 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:34.512358 16500 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:34.512257 16725 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:18:34.526002 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:34.527112 16632 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:34.529430 16632 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:34.529543 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:34.530319 16792 log.cc:927] Time spent T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.250s	user 0.000s	sys 0.001s
W20260709 13:18:34.531540 16786 log.cc:927] Time spent T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.263s	user 0.001s	sys 0.000s
I20260709 13:18:34.556770 11508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56404 (request call id 993) took 2106 ms. Trace:
I20260709 13:18:34.520421 16349 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:34.557262 16500 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:34.558699 10767 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:34.582619 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:34.594669 10841 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:34.587057 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:34.588608 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:34.586215 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:34.568975 10794 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2796) took 2968 ms (client timeout 1865 ms). Trace:
W20260709 13:18:34.597591 10794 rpcz_store.cc:269] 0709 13:18:31.600129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.568745 (+2968616us) service_pool.cc:211] Skipping call since client already timed out
0709 13:18:34.568896 (+   151us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:18:34.598703 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:34.599555 10794 raft_consensus.cc:2393] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 15: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:34.590142 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:34.587852 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:34.600521 10870 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:34.601008 10875 raft_consensus.cc:2393] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 20: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:34.581526 11512 raft_consensus.cc:2410] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 17 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
W20260709 13:18:34.606701 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 48093.4us
I20260709 13:18:34.607508 16699 raft_consensus.cc:2749] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:18:34.610775 10796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2803) took 3008 ms. Trace:
I20260709 13:18:34.611308 10796 rpcz_store.cc:276] 0709 13:18:31.602038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.853435 (+2251397us) service_pool.cc:224] Handling call
0709 13:18:33.853725 (+   290us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:33.854175 (+   450us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:33.854210 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:33.854914 (+   704us) log.cc:844] Serialized 52 byte log entry
0709 13:18:34.295792 (+440878us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:34.295833 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:34.295907 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:34.610479 (+314572us) raft_consensus.cc:1685] finished
0709 13:18:34.610506 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:34.610691 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":92,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":8097,"thread_start_us":395798,"threads_started":1,"wal-append.queue_time_us":657062}
I20260709 13:18:34.619269 10767 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:34.613361 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:34.622146 10796 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 15.
I20260709 13:18:34.626168 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:34.622758 16699 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:34.631613 16699 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:34.632117 16699 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:34.627022 16744 consensus_queue.cc:1059] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:34.642894 10796 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:34.568032 11508 rpcz_store.cc:276] 0709 13:18:32.427308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:32.427796 (+   488us) service_pool.cc:224] Handling call
0709 13:18:32.684081 (+256285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:32.684843 (+   762us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:34.533299 (+1848456us) spinlock_profiling.cc:243] Waited 256 ms on lock 0x7b5400527950. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:34.533447 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":588745984}
W20260709 13:18:34.649084 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:34.649169 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:34.651427 11507 raft_consensus.cc:1240] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 16. Current term is 17. Ops: []
W20260709 13:18:34.652801 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:34.648600 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:34.653752 10794 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 16.
I20260709 13:18:34.654911 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:34.655982 10794 raft_consensus.cc:2393] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 17: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:34.595875 10841 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:34.630409 16632 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:34.591413 16800 log.cc:927] Time spent T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
W20260709 13:18:34.658994 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 353420us
I20260709 13:18:34.559638 16709 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:34.661367 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 65556us
I20260709 13:18:34.659303 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:34.690358 10794 raft_consensus.cc:2393] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 19: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
W20260709 13:18:34.665076 10842 leader_election.cc:336] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Timed out: RequestConsensusVote RPC to 127.9.116.129:39685 timed out after 1.525s (SENT)
W20260709 13:18:34.691605 10842 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.116.129:39685 timed out after 1.636s (SENT) [suppressed 36 similar messages]
I20260709 13:18:34.675299 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:34.565869 16725 raft_consensus.cc:697] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:34.696914 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:34.699338 10794 raft_consensus.cc:2393] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 15: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:34.690367 10841 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:34.692461 10842 leader_election.cc:336] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Timed out: RequestConsensusVote RPC to 127.9.116.129:39685 timed out after 1.650s (SENT)
I20260709 13:18:34.565503 16724 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:34.700773 10842 leader_election.cc:336] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Timed out: RequestConsensusVote RPC to 127.9.116.129:39685 timed out after 1.795s (SENT)
I20260709 13:18:34.703207 10842 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:34.704419 10842 leader_election.cc:336] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Timed out: RequestConsensusVote RPC to 127.9.116.129:39685 timed out after 1.817s (SENT)
I20260709 13:18:34.593736 16500 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:34.701880 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:34.656718 16659 raft_consensus.cc:2804] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:18:34.709492 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:34.704941 10842 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:34.662576 16636 raft_consensus.cc:2749] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:34.589330 16719 consensus_queue.cc:1059] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.111s
I20260709 13:18:34.698284 16676 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:34.734140 16676 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:34.736254 16706 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:34.736785 16676 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:34.710665 16659 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:34.738514 16803 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:34.738979 16659 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:34.739190 16725 consensus_queue.cc:237] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:34.721535 16500 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:34.722604 11510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56404 (request call id 985) took 2308 ms. Trace:
I20260709 13:18:34.707136 16784 raft_consensus.cc:2749] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:34.647877 16654 consensus_queue.cc:1048] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.055s
W20260709 13:18:34.727437 10842 leader_election.cc:336] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Timed out: RequestConsensusVote RPC to 127.9.116.129:39685 timed out after 1.865s (SENT)
W20260709 13:18:34.756814 10842 leader_election.cc:336] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Timed out: RequestConsensusVote RPC to 127.9.116.129:39685 timed out after 1.902s (SENT)
W20260709 13:18:34.757889 10842 leader_election.cc:336] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Timed out: RequestConsensusVote RPC to 127.9.116.129:39685 timed out after 1.922s (SENT)
I20260709 13:18:34.758391 10842 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:34.718855 10841 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:34.732671 10870 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 17.
I20260709 13:18:34.741539 10796 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 18.
W20260709 13:18:34.741575 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:34.741770 10766 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:18:34.743199 10945 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.031s	sys 0.061s
I20260709 13:18:34.743911 10868 raft_consensus.cc:2368] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 17. Current term is 18.
W20260709 13:18:34.761152 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:34.710656 16750 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:34.744628 11510 rpcz_store.cc:276] 0709 13:18:32.414441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:32.414832 (+   391us) service_pool.cc:224] Handling call
0709 13:18:32.421617 (+  6785us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:32.990209 (+568592us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:34.722368 (+1732159us) spinlock_profiling.cc:243] Waited 552 ms on lock 0x7b54005802d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:34.722538 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1269969664}
I20260709 13:18:34.723979 10794 raft_consensus.cc:1275] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:18:34.768570 10794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2814) took 2834 ms. Trace:
I20260709 13:18:34.747324 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:18:34.769001 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 50277.1us
I20260709 13:18:34.768908 10794 rpcz_store.cc:276] 0709 13:18:31.934355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.723179 (+2788824us) service_pool.cc:224] Handling call
0709 13:18:34.723544 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:34.768269 (+ 44725us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:34.768512 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:34.750432 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:34.761569 10945 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.031s	sys 0.061s
I20260709 13:18:34.775523 10941 raft_consensus.cc:1240] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 17 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:34.776770 10796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2816) took 2718 ms. Trace:
I20260709 13:18:34.779850 10949 raft_consensus.cc:1275] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:18:34.783114 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:34.738555 16719 raft_consensus.cc:3055] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:18:34.746074 16575 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:34.789319 10794 raft_consensus.cc:1240] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:34.790261 10794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2817) took 2683 ms. Trace:
I20260709 13:18:34.790663 10794 rpcz_store.cc:276] 0709 13:18:32.106817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.770708 (+2663891us) service_pool.cc:224] Handling call
0709 13:18:34.789059 (+ 18351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:34.789919 (+   860us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:34.790174 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:34.703421 16709 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:34.797753 10948 raft_consensus.cc:1240] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 14 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:18:34.797596 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:34.798492 10942 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 18.
I20260709 13:18:34.795410 10947 raft_consensus.cc:1275] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:18:34.767762 16728 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:34.802436 11504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56404 (request call id 997) took 2073 ms. Trace:
I20260709 13:18:34.803722 11506 raft_consensus.cc:1240] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 19. Current term is 20. Ops: []
I20260709 13:18:34.804944 10794 raft_consensus.cc:1275] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:18:34.802922 11504 rpcz_store.cc:276] 0709 13:18:32.729095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:32.729212 (+   117us) service_pool.cc:224] Handling call
0709 13:18:34.802224 (+2073012us) spinlock_profiling.cc:243] Waited 33.3 ms on lock 0x7b54001d1354. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:34.802373 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":76596992}
I20260709 13:18:34.805820 10794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2818) took 2682 ms. Trace:
I20260709 13:18:34.784889 16652 raft_consensus.cc:2749] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:18:34.808534 11511 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56404 (request call id 1003) took 1773 ms (client timeout 1814 ms). Trace:
I20260709 13:18:34.806280 10794 rpcz_store.cc:276] 0709 13:18:32.123943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.791675 (+2667732us) service_pool.cc:224] Handling call
0709 13:18:34.804545 (+ 12870us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:34.805517 (+   972us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:34.805753 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:34.808918 11511 rpcz_store.cc:269] 0709 13:18:33.035307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.035433 (+   126us) service_pool.cc:224] Handling call
0709 13:18:34.808348 (+1772915us) spinlock_profiling.cc:243] Waited 657 ms on lock 0x7b54005fee54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:34.808474 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1510404480}
I20260709 13:18:34.811640 10867 raft_consensus.cc:1240] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:18:34.813392 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:34.787868 16719 raft_consensus.cc:740] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:18:34.802467 16793 log.cc:927] Time spent T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.533s	user 0.004s	sys 0.000s
W20260709 13:18:34.803445 16655 consensus_meta.cc:287] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.504s	user 0.009s	sys 0.001s
I20260709 13:18:34.801107 16706 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:34.821606 16699 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:34.824532 16706 raft_consensus.cc:697] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:34.825842 10800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2811) took 3221 ms. Trace:
I20260709 13:18:34.823421 16655 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:34.808311 16803 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:34.830622 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:34.825815 16706 consensus_queue.cc:237] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:34.827864 10794 raft_consensus.cc:1275] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:18:34.839231 10794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2819) took 2600 ms. Trace:
I20260709 13:18:34.839618 10794 rpcz_store.cc:276] 0709 13:18:32.238557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.810552 (+2571995us) service_pool.cc:224] Handling call
0709 13:18:34.827577 (+ 17025us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:34.838952 (+ 11375us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:34.839175 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:34.828447 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:34.828531 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:34.829154 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:34.841528 10867 raft_consensus.cc:2393] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 17: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:34.783190 16750 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:34.800318 16575 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:34.842305 10870 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 18.
I20260709 13:18:34.846448 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:34.852322 16375 raft_consensus.cc:3055] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:18:34.852772 16375 raft_consensus.cc:740] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:34.853639 16375 consensus_queue.cc:260] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:34.854913 16632 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:34.849869 16805 log.cc:927] Time spent T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 13:18:34.855468 16645 raft_consensus.cc:3055] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:34.855911 16645 raft_consensus.cc:740] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:34.848724 16719 consensus_queue.cc:260] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:34.856736 16645 consensus_queue.cc:260] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:34.857662 16645 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:34.859061 16575 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:34.860997 10872 raft_consensus.cc:1240] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:18:34.861581 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:34.865274 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:34.869967 10793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2789) took 3271 ms. Trace:
I20260709 13:18:34.870411 10793 rpcz_store.cc:276] 0709 13:18:31.598019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:31.598126 (+   107us) service_pool.cc:224] Handling call
0709 13:18:31.598430 (+   304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:34.113994 (+2515564us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b1800330a60. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c80a5d4bc 00007f9c7ddab48b 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc5455 00007f9c80ead060 00007f9c80eadbfb 00007f9c80eb864b
0709 13:18:34.114174 (+   180us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:34.114203 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:34.128395 (+ 14192us) log.cc:844] Serialized 26 byte log entry
0709 13:18:34.433589 (+305194us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:34.434090 (+   501us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:34.434179 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:34.860472 (+426293us) raft_consensus.cc:1685] finished
0709 13:18:34.860494 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:34.869875 (+  9381us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15984,"raft.queue_time_us":170542,"raft.run_cpu_time_us":271,"raft.run_wall_time_us":43310,"spinlock_wait_cycles":24819584,"thread_start_us":479044,"threads_started":2,"wal-append.queue_time_us":533381}
I20260709 13:18:34.872494 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:34.873265 10793 raft_consensus.cc:2393] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 15: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:18:34.847604 16790 log.cc:927] Time spent T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.568s	user 0.003s	sys 0.000s
I20260709 13:18:34.886413 10792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 793) took 3036 ms. Trace:
I20260709 13:18:34.877406 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:34.867563 10916 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:34.887045 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:34.889675 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:34.878340 10915 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:34.890273 10916 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:34.879902 16655 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:34.880429 16785 log.cc:927] Time spent T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.600s	user 0.001s	sys 0.001s
I20260709 13:18:34.880712 16709 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:34.888631 10793 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 16.
I20260709 13:18:34.878361 10795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2813) took 2947 ms. Trace:
I20260709 13:18:34.883682 10799 raft_consensus.cc:2410] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 13:18:34.884294 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:34.877779 16620 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:34.885336 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:34.883358 16750 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:34.900172 10876 raft_consensus.cc:2393] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 16: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:34.886821 10792 rpcz_store.cc:276] 0709 13:18:31.849667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.317391 (+2467724us) service_pool.cc:224] Handling call
0709 13:18:34.330206 (+ 12815us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:34.330645 (+   439us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:34.330674 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:34.331277 (+   603us) log.cc:844] Serialized 26 byte log entry
0709 13:18:34.800336 (+469059us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:34.822928 (+ 22592us) log.cc:844] Serialized 14 byte log entry
0709 13:18:34.823820 (+   892us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:34.823892 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:34.886216 (+ 62324us) raft_consensus.cc:1685] finished
0709 13:18:34.886248 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:34.886363 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":489,"prepare.queue_time_us":16075,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":1369,"raft.queue_time_us":48943,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":8079,"thread_start_us":372926,"threads_started":1,"wal-append.queue_time_us":420019}
I20260709 13:18:34.890280 10874 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 14.
I20260709 13:18:34.893092 10950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 1024) took 1223 ms. Trace:
I20260709 13:18:34.857662 16719 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:34.896121 10795 rpcz_store.cc:276] 0709 13:18:31.930565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.336354 (+2405789us) service_pool.cc:224] Handling call
0709 13:18:34.380219 (+ 43865us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:34.380494 (+   275us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:18:34.380519 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:34.381010 (+   491us) log.cc:844] Serialized 26 byte log entry
0709 13:18:34.833117 (+452107us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:34.833175 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:34.834585 (+  1410us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:34.878046 (+ 43461us) raft_consensus.cc:1685] finished
0709 13:18:34.878074 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:34.878280 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":397969,"threads_started":1,"wal-append.queue_time_us":442101}
I20260709 13:18:34.896646 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:34.905823 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:34.905380 10950 rpcz_store.cc:276] 0709 13:18:33.669187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.678271 (+  9084us) service_pool.cc:224] Handling call
0709 13:18:33.695200 (+ 16929us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:33.695564 (+   364us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:33.695594 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:33.696077 (+   483us) log.cc:844] Serialized 26 byte log entry
0709 13:18:33.949654 (+253577us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:18:33.949715 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:33.949814 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:34.657853 (+708039us) spinlock_profiling.cc:243] Waited 19.9 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:18:34.885862 (+228009us) raft_consensus.cc:1685] finished
0709 13:18:34.885886 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:34.893020 (+  7134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7907,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":6089,"spinlock_wait_cycles":45763456,"thread_start_us":253054,"threads_started":1,"wal-append.queue_time_us":549633}
W20260709 13:18:34.911633 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:34.894433 16617 raft_consensus.cc:2804] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:34.914301 10792 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 20.
I20260709 13:18:34.933040 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:34.933176 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:34.883605 16714 consensus_queue.cc:1048] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 13:18:34.935742 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:34.938618 16500 consensus_queue.cc:1059] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:18:34.939958 16500 raft_consensus.cc:3055] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:34.940402 16500 raft_consensus.cc:740] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:34.941247 11478 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:34.941473 16500 consensus_queue.cc:260] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:34.942961 16500 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:34.949354 16511 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:34.829195 10800 rpcz_store.cc:276] 0709 13:18:31.604296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.858553 (+2254257us) service_pool.cc:224] Handling call
0709 13:18:33.858909 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:33.859106 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:18:34.488760 (+629654us) log.cc:844] Serialized 14 byte log entry
0709 13:18:34.822944 (+334184us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:34.823021 (+    77us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:34.823088 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:34.825646 (+  2558us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:34.825779 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":862022,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":40,"raft.queue_time_us":779,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":87,"thread_start_us":921864,"threads_started":2,"wal-append.queue_time_us":382551}
I20260709 13:18:34.900267 16375 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:34.969916 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:34.970755 10876 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:34.898033 16600 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:34.961465 11478 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:34.974042 10795 raft_consensus.cc:1240] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:18:34.975638 16676 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:34.979120 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:18:34.981904 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 91699.5us
W20260709 13:18:34.982116 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 205177us
W20260709 13:18:34.982755 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:34.987344 10795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2822) took 2599 ms. Trace:
I20260709 13:18:34.987221 11478 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:34.987807 10795 rpcz_store.cc:276] 0709 13:18:32.387584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.913819 (+2526235us) service_pool.cc:224] Handling call
0709 13:18:34.971795 (+ 57976us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:34.987069 (+ 15274us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:34.987290 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:34.988785 11478 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:34.989463 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:34.990157 10869 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
I20260709 13:18:34.992951 10876 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 19.
W20260709 13:18:34.994616 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 116339us
I20260709 13:18:34.995662 11478 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:35.005887 16292 consensus_queue.cc:1048] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:18:34.935292 16617 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:35.052976 16724 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:18:34.986193 16600 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:34.914186 16620 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:35.015445 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:35.021605 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 210649us
I20260709 13:18:34.985170 16635 raft_consensus.cc:2749] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:34.999763 16349 raft_consensus.cc:2749] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:35.044035 10916 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:35.076370 10867 raft_consensus.cc:2393] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 16: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:35.045042 10766 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:35.004137 10876 raft_consensus.cc:1275] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:18:35.055125 16764 consensus_meta.cc:287] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.740s	user 0.016s	sys 0.000s
W20260709 13:18:35.078541 10842 leader_election.cc:336] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Timed out: RequestConsensusVote RPC to 127.9.116.129:39685 timed out after 2.003s (SENT)
W20260709 13:18:35.082175 10842 leader_election.cc:336] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Timed out: RequestConsensusVote RPC to 127.9.116.129:39685 timed out after 2.082s (SENT)
W20260709 13:18:35.056069 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 60503.8us
I20260709 13:18:34.997618 10869 raft_consensus.cc:1275] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:18:35.082741 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:35.083173 11478 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:35.083467 10792 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 18.
I20260709 13:18:35.047040 10767 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:35.083582 11506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56410 (request call id 931) took 1692 ms. Trace:
I20260709 13:18:34.992340 16242 consensus_queue.cc:1048] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.059s
I20260709 13:18:35.067868 10944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2749) took 1719 ms. Trace:
W20260709 13:18:35.069088 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:35.084321 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:35.082562 10842 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:35.067461 16818 log.cc:927] Time spent T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
W20260709 13:18:35.071614 16806 log.cc:927] Time spent T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.194s	user 0.000s	sys 0.001s
I20260709 13:18:35.071931 16617 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:35.078872 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:35.063659 16732 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:35.069458 16738 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:35.086268 11477 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:35.103941 16738 raft_consensus.cc:2749] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:35.087751 11478 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:35.105491 11478 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:18:35.086828 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:35.106314 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:35.091518 10792 raft_consensus.cc:2393] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 15: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
W20260709 13:18:35.093089 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 46138.6us
I20260709 13:18:35.084000 11506 rpcz_store.cc:276] 0709 13:18:33.391021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.391237 (+   216us) service_pool.cc:224] Handling call
0709 13:18:34.803554 (+1412317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:34.804101 (+   547us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:35.083394 (+279293us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b54001d1350. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:35.083502 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3247321472}
I20260709 13:18:35.084209 10944 rpcz_store.cc:276] 0709 13:18:33.348229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.348345 (+   116us) service_pool.cc:224] Handling call
0709 13:18:33.348677 (+   332us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:33.349067 (+   390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:33.349092 (+    25us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:33.349664 (+   572us) log.cc:844] Serialized 52 byte log entry
0709 13:18:33.864894 (+515230us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:33.864948 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:33.865040 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:34.879775 (+1014735us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:18:35.061271 (+181496us) raft_consensus.cc:1685] finished
0709 13:18:35.061294 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:35.067796 (+  6502us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5630,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":3333,"spinlock_wait_cycles":26285312,"thread_start_us":473134,"threads_started":1,"wal-append.queue_time_us":825050}
I20260709 13:18:35.107441 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:18:35.093106 10842 leader_election.cc:336] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Timed out: RequestConsensusVote RPC to 127.9.116.129:39685 timed out after 2.128s (SENT)
W20260709 13:18:35.096789 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:35.107774 10767 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:35.108060 10950 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
W20260709 13:18:35.108525 10842 leader_election.cc:336] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Timed out: RequestConsensusVote RPC to 127.9.116.129:39685 timed out after 2.221s (SENT)
I20260709 13:18:35.108965 10842 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:35.109946 10842 leader_election.cc:336] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Timed out: RequestConsensusVote RPC to 127.9.116.129:39685 timed out after 2.283s (SENT)
I20260709 13:18:35.110347 10842 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:35.069391 16724 raft_consensus.cc:697] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 18 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
W20260709 13:18:35.089663 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:35.106956 10947 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:18:35.103996 10797 raft_consensus.cc:1240] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:35.112982 10797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2195) took 3507 ms. Trace:
I20260709 13:18:35.113332 10797 rpcz_store.cc:276] 0709 13:18:31.605795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.090764 (+3484969us) service_pool.cc:224] Handling call
0709 13:18:35.103832 (+ 13068us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:35.112664 (+  8832us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:35.112925 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:35.114488 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:35.118413 10767 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:35.120803 10767 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:35.094516 16706 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:18:35.129926 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:35.093807 16486 raft_consensus.cc:2749] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:18:35.093811 16561 raft_consensus.cc:2804] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:35.144694 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:35.145455 10946 raft_consensus.cc:1240] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:35.146062 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:35.145607 10944 raft_consensus.cc:2410] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 16 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:18:35.148295 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:35.148931 11507 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 16.
I20260709 13:18:35.112468 16349 raft_consensus.cc:2749] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:35.152591 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:35.112358 16787 raft_consensus.cc:2749] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:18:35.159904 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:35.094836 16764 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:35.127648 16709 raft_consensus.cc:2749] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:18:35.094432 16728 consensus_queue.cc:1048] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 13:18:35.128034 16655 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.121341 16494 raft_consensus.cc:3055] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:35.172811 10950 raft_consensus.cc:1275] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:18:35.182778 10945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2771) took 1356 ms. Trace:
I20260709 13:18:35.185602 10945 rpcz_store.cc:276] 0709 13:18:33.826477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.083616 (+257139us) service_pool.cc:224] Handling call
0709 13:18:34.127035 (+ 43419us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:34.127418 (+   383us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:34.127446 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:34.131883 (+  4437us) log.cc:844] Serialized 26 byte log entry
0709 13:18:34.773345 (+641462us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:34.773387 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:34.773460 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:35.182480 (+409020us) raft_consensus.cc:1685] finished
0709 13:18:35.182508 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:35.182695 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":136,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":3126,"thread_start_us":583399,"threads_started":1,"wal-append.queue_time_us":657049}
I20260709 13:18:35.150115 16561 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:34.777184 10796 rpcz_store.cc:276] 0709 13:18:32.058655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.760082 (+2701427us) service_pool.cc:224] Handling call
0709 13:18:34.760509 (+   427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:34.776414 (+ 15905us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:34.776442 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:34.776484 (+    42us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:34.776522 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:34.776584 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:34.776710 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":16381,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":3878}
I20260709 13:18:35.165660 16656 raft_consensus.cc:2749] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:18:35.305972 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:18:35.231179 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 187235us
I20260709 13:18:35.233348 10873 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 18.
I20260709 13:18:35.139812 16816 raft_consensus.cc:2749] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:35.365027 10696 catalog_manager.cc:5697] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:18:35.304582 10842 leader_election.cc:336] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Timed out: RequestConsensusVote RPC to 127.9.116.129:39685 timed out after 1.995s (SENT)
I20260709 13:18:35.155689 16815 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:35.285317 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:18:35.525550 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 420111us
I20260709 13:18:35.537705 10944 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:35.348929 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:35.181077 10947 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 19.
I20260709 13:18:35.284940 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:18:35.527323 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 222749us
W20260709 13:18:35.543227 10842 leader_election.cc:336] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Timed out: RequestConsensusVote RPC to 127.9.116.129:39685 timed out after 2.041s (SENT)
W20260709 13:18:35.275117 16819 log.cc:927] Time spent T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.219s	user 0.000s	sys 0.001s
W20260709 13:18:35.300829 16797 log.cc:927] Time spent T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.876s	user 0.007s	sys 0.000s
I20260709 13:18:35.160704 16500 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:18:35.321926 10801 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2794) took 3722 ms (client timeout 2221 ms). Trace:
I20260709 13:18:35.284533 16242 raft_consensus.cc:2804] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:18:35.264158 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 158275us
I20260709 13:18:35.287720 16600 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:35.296977 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 142892us
W20260709 13:18:35.206995 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 114689us
I20260709 13:18:35.287194 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:35.592795 10949 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 16.
I20260709 13:18:35.594061 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:35.282168 16764 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:35.296733 10798 raft_consensus.cc:1275] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 13:18:35.596740 10798 pending_rounds.cc:85] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 6
I20260709 13:18:35.596665 10766 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:35.302047 16511 raft_consensus.cc:2749] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:35.597168 10798 pending_rounds.cc:107] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 13:18:35.597527 10798 raft_consensus.cc:2889] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:35.337746 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:35.525547 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:35.599320 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:35.282177 10799 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2210) took 1643 ms (client timeout 1510 ms). Trace:
W20260709 13:18:35.600301 10799 rpcz_store.cc:269] 0709 13:18:33.638817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.706544 (+ 67727us) service_pool.cc:224] Handling call
0709 13:18:35.281922 (+1575378us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54000bd6d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:35.282092 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2651574144}
I20260709 13:18:35.352170 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:35.602207 10942 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 20.
W20260709 13:18:35.543695 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 422983us
I20260709 13:18:35.541641 10946 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:35.240090 16731 raft_consensus.cc:2749] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:35.574496 10800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 800) took 1042 ms. Trace:
I20260709 13:18:35.315649 16676 raft_consensus.cc:2749] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:35.541064 10795 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
I20260709 13:18:35.579383 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:35.631254 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:35.314574 10941 raft_consensus.cc:1240] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 14. Current term is 15. Ops: []
W20260709 13:18:35.334347 16803 consensus_meta.cc:287] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.502s	user 0.010s	sys 0.000s
W20260709 13:18:35.583303 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:35.295258 16831 log.cc:927] Time spent T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
W20260709 13:18:35.320637 16659 consensus_meta.cc:287] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.581s	user 0.011s	sys 0.000s
W20260709 13:18:35.296042 16832 log.cc:927] Time spent T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:18:35.293548 16714 raft_consensus.cc:2749] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:18:35.580567 10801 rpcz_store.cc:269] 0709 13:18:31.599641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.410280 (+1810639us) service_pool.cc:224] Handling call
0709 13:18:35.321660 (+1911380us) spinlock_profiling.cc:243] Waited 967 ms on lock 0x7b5400385c54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:35.321828 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2224422656}
I20260709 13:18:35.654042 10944 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
I20260709 13:18:35.658399 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:35.659334 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:35.659936 10801 raft_consensus.cc:2410] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:18:35.490078 10916 leader_election.cc:400] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 14. Current term is 15.
I20260709 13:18:35.660351 10916 leader_election.cc:403] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:35.455281 16727 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:35.661484 16727 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:35.321066 16706 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:35.663995 16706 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:35.519651 16561 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:18:35.586162 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:35.306452 16732 raft_consensus.cc:2749] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:35.584782 10696 catalog_manager.cc:5697] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
W20260709 13:18:35.380262 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:35.321835 16620 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:35.673678 16620 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.674177 16620 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:35.676294 16620 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:35.677211 16620 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:35.677675 16620 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:35.679947 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:35.680831 10872 raft_consensus.cc:2410] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:18:35.595575 10949 raft_consensus.cc:2410] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 22 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 13:18:35.604111 10767 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:35.612501 10943 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.020s	sys 0.032s
W20260709 13:18:35.685407 10943 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.020s	sys 0.033s
W20260709 13:18:35.685734 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 81699.9us
I20260709 13:18:35.526410 16494 raft_consensus.cc:740] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:35.230077 16827 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:35.632324 10796 raft_consensus.cc:2393] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 18: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:35.686036 10943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 1020) took 2018 ms. Trace:
I20260709 13:18:35.686522 16827 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:35.687165 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:35.688064 10796 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:35.634948 10841 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:35.200816 16655 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:35.687070 10943 rpcz_store.cc:276] 0709 13:18:33.667995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.670881 (+  2886us) service_pool.cc:224] Handling call
0709 13:18:34.852723 (+1181842us) spinlock_profiling.cc:243] Waited 3.37 ms on lock 0x7b5800112598. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90e9c 00007f9c7dd90a5b 00007f9c86ca9c09 00007f9c85e03ee0 00007f9c85ecd0ab 00007f9c85ecd131 00007f9c85e8efc0 00007f9c85e79929 00007f9c85e793d2 00007f9c7f3f0bd4
0709 13:18:34.852901 (+   178us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:34.853095 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:18:34.931828 (+ 78733us) log.cc:844] Serialized 14 byte log entry
0709 13:18:35.685735 (+753907us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:35.685787 (+    52us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:18:35.685833 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:35.685893 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:35.685988 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":122644,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":235263,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":105,"spinlock_wait_cycles":7748352,"thread_start_us":826884,"threads_started":3,"wal-append.queue_time_us":178321}
I20260709 13:18:35.581072 16500 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:35.651124 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:35.651607 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:35.584472 16699 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:35.200501 16292 consensus_queue.cc:1048] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.425s
I20260709 13:18:35.239893 16724 consensus_queue.cc:237] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:35.543839 10842 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:35.686518 10767 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:18:35.689549 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 54699.9us
I20260709 13:18:35.642879 10946 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
I20260709 13:18:35.296828 16575 consensus_queue.cc:1059] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.305s
I20260709 13:18:35.651860 16484 consensus_queue.cc:1059] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:18:35.712973 10792 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:35.713743 11477 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:35.713763 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:18:35.705796 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:35.585227 16242 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:18:35.714775 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:35.655682 16803 raft_consensus.cc:2749] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:35.734463 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:35.716799 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 2235) took 1114 ms. Trace:
I20260709 13:18:35.721482 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:35.786357 10795 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 17.
I20260709 13:18:35.787081 10801 raft_consensus.cc:2410] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 16 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:18:35.787459 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:35.788336 10795 raft_consensus.cc:2410] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:18:35.786226 10873 rpcz_store.cc:276] 0709 13:18:34.588194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.588326 (+   132us) service_pool.cc:224] Handling call
0709 13:18:35.702291 (+1113965us) spinlock_profiling.cc:243] Waited 636 ms on lock 0x7b54000f64d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:35.702412 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1463816448}
I20260709 13:18:35.789280 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:35.761195 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 217973us
W20260709 13:18:35.790360 10842 leader_election.cc:336] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Timed out: RequestConsensusVote RPC to 127.9.116.129:39685 timed out after 2.120s (SENT)
W20260709 13:18:35.762303 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 75874.7us
I20260709 13:18:35.792245 10767 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:35.756156 10942 raft_consensus.cc:2410] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:18:35.688674 16600 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.793401 10796 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
I20260709 13:18:35.795145 10792 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
I20260709 13:18:35.700431 16656 raft_consensus.cc:2749] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:18:35.687207 16827 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:35.695267 16676 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:35.800222 10767 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:35.802719 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:35.807741 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:35.808180 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:35.815868 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 325946us
I20260709 13:18:35.817698 10796 raft_consensus.cc:2376] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15. Re-sending same reply.
I20260709 13:18:35.818960 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:35.717563 16732 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:35.836704 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 59761.3us
I20260709 13:18:35.837110 16732 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:35.733155 16500 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:35.733309 16292 consensus_queue.cc:1243] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer 12682eb70bc84c76879e04a8628ee586 log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:18:35.783052 16242 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:35.819859 10795 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 14.
I20260709 13:18:35.733340 16843 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:35.808923 10766 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:35.736766 16709 raft_consensus.cc:2749] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:18:35.736935 16731 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.778227 16750 raft_consensus.cc:2749] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:18:35.857235 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:35.780936 16714 raft_consensus.cc:2749] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 14. Current term is 15.
I20260709 13:18:35.788228 16632 raft_consensus.cc:2749] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:18:35.874951 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 94733.1us
I20260709 13:18:35.717459 16841 raft_consensus.cc:2749] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:18:35.835700 10916 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:35.788635 16699 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:35.882401 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:35.695847 16659 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:35.920701 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:35.921674 10792 raft_consensus.cc:2368] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 17. Current term is 18.
I20260709 13:18:35.922643 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:35.910254 10801 raft_consensus.cc:2393] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 18: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:35.925156 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:35.883713 16832 log.cc:927] Time spent T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:18:35.798122 16828 consensus_queue.cc:1059] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.128s
W20260709 13:18:35.931262 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 95653us
W20260709 13:18:35.932610 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 94992.6us
I20260709 13:18:35.933271 11504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56404 (request call id 1012) took 1026 ms. Trace:
I20260709 13:18:35.934250 10801 raft_consensus.cc:1275] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:18:35.933662 11504 rpcz_store.cc:276] 0709 13:18:34.907080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.907219 (+   139us) service_pool.cc:224] Handling call
0709 13:18:34.907572 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:34.907800 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:18:35.062161 (+154361us) log.cc:844] Serialized 14 byte log entry
0709 13:18:35.924332 (+862171us) spinlock_profiling.cc:243] Waited 5.61 ms on lock 0x7b1800136900. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c80a5d4bc 00007f9c7ddab48b 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a5523a
0709 13:18:35.932910 (+  8578us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:35.932970 (+    60us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:18:35.933022 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:35.933092 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:35.933219 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":358710,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":2483,"spinlock_wait_cycles":12898688,"thread_start_us":590436,"threads_started":2}
I20260709 13:18:35.818749 16600 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:35.833981 16676 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:35.940083 16839 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.897349 16843 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:35.939785 16841 raft_consensus.cc:2749] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:35.889591 16844 raft_consensus.cc:2749] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:18:35.966107 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:35.888018 16803 consensus_queue.cc:1048] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
W20260709 13:18:35.983227 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 50868.1us
I20260709 13:18:36.004892 10916 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:36.011831 16835 raft_consensus.cc:3055] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:36.012403 16835 raft_consensus.cc:740] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:36.013115 16845 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:36.013540 16845 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:36.014122 16845 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:36.013437 16835 consensus_queue.cc:260] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:36.020668 16676 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:36.014427 16500 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:36.026568 16500 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:36.003212 16839 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:35.967463 16842 log.cc:927] Time spent T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:18:36.032578 16620 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:36.035084 16843 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:36.037448 16600 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:35.912760 16731 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:36.045176 16731 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:36.046998 16727 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:35.999145 16617 consensus_meta.cc:287] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.900s	user 0.009s	sys 0.008s
I20260709 13:18:36.051280 16617 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:36.054209 16617 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:35.964104 16575 consensus_queue.cc:1059] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.092s
I20260709 13:18:36.057838 16714 consensus_queue.cc:1059] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:18:36.059196 16714 raft_consensus.cc:3055] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:36.059710 16714 raft_consensus.cc:740] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:36.060900 16714 consensus_queue.cc:260] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:36.063658 10942 raft_consensus.cc:1240] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 18. Current term is 19. Ops: []
I20260709 13:18:36.062642 16714 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:36.102941 16494 consensus_queue.cc:260] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:36.104416 16494 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:36.104230 16845 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:36.108109 16645 raft_consensus.cc:3055] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:35.968360 16655 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:36.074728 16656 raft_consensus.cc:3055] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:18:35.973557 16561 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:35.962743 16706 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:36.118393 16659 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:36.116708 16839 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:36.128921 16738 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.003s	sys 0.007s
I20260709 13:18:36.152395 16845 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:35.960074 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:35.942716 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:35.953756 10841 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:18:36.114893 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:35.626641 10800 rpcz_store.cc:276] 0709 13:18:34.532137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.953201 (+421064us) service_pool.cc:224] Handling call
0709 13:18:34.953527 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:34.953731 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:18:35.160777 (+207046us) log.cc:844] Serialized 14 byte log entry
0709 13:18:35.574008 (+413231us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:35.574083 (+    75us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:35.574146 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:35.574242 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:35.574417 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39878,"prepare.queue_time_us":109917,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":45550,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":5103,"thread_start_us":539786,"threads_started":2}
I20260709 13:18:36.134810 16645 raft_consensus.cc:740] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:36.137835 16656 raft_consensus.cc:740] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:18:36.146329 16738 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.003s	sys 0.007s
I20260709 13:18:36.242825 10792 raft_consensus.cc:2393] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 21: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:36.245970 10799 raft_consensus.cc:2410] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 20 because replica has last-logged OpId of term: 19 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 8.
I20260709 13:18:36.271428 10795 raft_consensus.cc:1240] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:36.244578 10796 raft_consensus.cc:2410] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
W20260709 13:18:36.251149 11478 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.669s	user 0.001s	sys 0.010s
W20260709 13:18:36.324766 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:36.253813 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:36.311148 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:36.362733 10795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2238) took 1257 ms. Trace:
I20260709 13:18:36.268872 10798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2204) took 4660 ms. Trace:
W20260709 13:18:36.367405 11478 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.786s	user 0.002s	sys 0.010s
I20260709 13:18:36.243805 10794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2820) took 3879 ms. Trace:
W20260709 13:18:36.209120 10695 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42968 (request call id 247) took 28054 ms (client timeout 15000 ms). Trace:
I20260709 13:18:36.329159 16714 raft_consensus.cc:2804] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:36.303400 16375 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:36.267187 10793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 798) took 1711 ms. Trace:
I20260709 13:18:36.319921 16655 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:36.324321 16706 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:36.321110 16656 consensus_queue.cc:260] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:36.312594 16561 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:36.315892 16632 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:36.326491 16728 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.967625 10801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 803) took 1402 ms. Trace:
I20260709 13:18:36.384589 10799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2233) took 1271 ms. Trace:
W20260709 13:18:36.325979 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 321005us
I20260709 13:18:36.401430 10916 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
W20260709 13:18:36.254886 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 301075us
W20260709 13:18:35.893365 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 93241.4us
I20260709 13:18:36.305701 16557 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:18:36.241256 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 432257us
I20260709 13:18:36.376784 16714 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:36.366822 10795 rpcz_store.cc:276] 0709 13:18:35.101402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:36.266139 (+1164737us) service_pool.cc:224] Handling call
0709 13:18:36.270968 (+  4829us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:36.357650 (+ 86682us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:36.359004 (+  1354us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:36.383749 10796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2862) took 1066 ms. Trace:
I20260709 13:18:36.392791 10799 rpcz_store.cc:276] 0709 13:18:35.100121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.909657 (+809536us) service_pool.cc:224] Handling call
0709 13:18:36.367753 (+458096us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:36.386368 10792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2850) took 1222 ms. Trace:
I20260709 13:18:36.381747 16655 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:36.407655 10796 rpcz_store.cc:276] 0709 13:18:35.311940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.914179 (+602239us) service_pool.cc:224] Handling call
0709 13:18:36.374183 (+460004us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:36.408066 10792 rpcz_store.cc:276] 0709 13:18:35.163592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.922346 (+758754us) service_pool.cc:224] Handling call
0709 13:18:36.382903 (+460557us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:36.413120 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:36.370378 10798 rpcz_store.cc:276] 0709 13:18:31.608129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.209617 (+2601488us) service_pool.cc:224] Handling call
0709 13:18:35.296298 (+1086681us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:35.597945 (+301647us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:36.268318 (+670373us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54000bd6d0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:36.268611 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2382627584}
I20260709 13:18:36.414981 16816 raft_consensus.cc:2804] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:18:36.415071 16375 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:36.415889 16816 raft_consensus.cc:697] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:36.271487 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:36.300379 16349 consensus_queue.cc:1059] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.110s
I20260709 13:18:36.263206 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:36.317298 16645 consensus_queue.cc:260] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:36.422914 10800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2852) took 1169 ms. Trace:
I20260709 13:18:36.424351 10800 rpcz_store.cc:276] 0709 13:18:35.163964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:36.285173 (+1121209us) service_pool.cc:224] Handling call
0709 13:18:36.285876 (+   703us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:36.286378 (+   502us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:36.286430 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:36.286515 (+    85us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:36.286572 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:36.286675 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:36.333650 (+ 46975us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:36.409060 16714 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:36.410809 16655 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:36.155333 16827 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:36.429256 10842 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.012s	sys 0.009s
W20260709 13:18:36.433228 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:36.438012 16617 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:36.437834 16728 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:36.439112 16617 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:36.442376 16617 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:18:36.443073 10842 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.012s	sys 0.009s
W20260709 13:18:36.443899 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 608338us
I20260709 13:18:36.207624 10696 catalog_manager.cc:5697] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 12 to 13, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 13 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:36.451998 16375 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:36.453835 10767 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:36.372246 10794 rpcz_store.cc:276] 0709 13:18:32.363846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.840338 (+2476492us) service_pool.cc:224] Handling call
0709 13:18:34.847469 (+  7131us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:34.847941 (+   472us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:18:34.848942 (+  1001us) log.cc:844] Serialized 14 byte log entry
0709 13:18:34.986230 (+137288us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:35.323942 (+337712us) spinlock_profiling.cc:243] Waited 8.02 ms on lock 0x7b2c000014c8. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7f7aea11 00007f9c7f7be372 00007f9c80ef0a01 00007f9c80eb543a 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2
0709 13:18:35.324474 (+   532us) log.cc:844] Serialized 26 byte log entry
0709 13:18:35.324697 (+   223us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:35.324993 (+   296us) log.cc:844] Serialized 14 byte log entry
0709 13:18:35.337374 (+ 12381us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:35.337457 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:36.243227 (+905770us) raft_consensus.cc:1685] finished
0709 13:18:36.243273 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:36.243597 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12097,"prepare.queue_time_us":18737,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":78,"raft.queue_time_us":13035,"raft.run_cpu_time_us":302,"raft.run_wall_time_us":14016,"spinlock_wait_cycles":18439552,"thread_start_us":112442,"threads_started":1,"wal-append.queue_time_us":965243}
I20260709 13:18:36.455619 16375 raft_consensus.cc:2749] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:36.456219 10767 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:18:36.456616 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 55384.3us
I20260709 13:18:36.435251 10801 rpcz_store.cc:276] 0709 13:18:34.532907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.933215 (+1400308us) service_pool.cc:224] Handling call
0709 13:18:35.933509 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:35.934938 (+  1429us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:35.935211 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:36.459405 16816 consensus_queue.cc:237] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:36.460052 10948 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:36.463086 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:36.463943 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:36.464884 16491 raft_consensus.cc:2804] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:36.441783 16728 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:36.467154 16732 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:36.471171 16656 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:36.471851 16557 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:36.428951 16242 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:36.472388 16815 consensus_meta.cc:287] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.892s	user 0.014s	sys 0.004s
I20260709 13:18:36.473819 10841 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:18:36.433456 10945 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.138s	user 0.021s	sys 0.038s
I20260709 13:18:36.480257 16645 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:36.450160 10696 catalog_manager.cc:5697] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:18:36.485936 16867 log.cc:927] Time spent T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.095s	user 0.001s	sys 0.002s
I20260709 13:18:36.480279 16375 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:36.462982 11478 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:36.414981 16632 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:36.492304 10696 catalog_manager.cc:5697] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 10 to 11, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 11 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
W20260709 13:18:36.494102 16870 log.cc:927] Time spent T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 13:18:36.487718 10942 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 19.
W20260709 13:18:36.499696 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:36.497696 16375 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:36.465395 16491 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:36.504830 16709 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:36.506662 16827 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:36.506968 16738 raft_consensus.cc:3055] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:18:36.379055 10793 rpcz_store.cc:276] 0709 13:18:34.531628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.916916 (+385288us) service_pool.cc:224] Handling call
0709 13:18:34.919199 (+  2283us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:34.919672 (+   473us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:34.919714 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:34.920307 (+   593us) log.cc:844] Serialized 26 byte log entry
0709 13:18:35.081359 (+161052us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:18:35.122316 (+ 40957us) log.cc:844] Serialized 14 byte log entry
0709 13:18:35.314491 (+192175us) spinlock_profiling.cc:243] Waited 3.28 ms on lock 0x7b180009bd40. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc83f3 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:18:35.497517 (+183026us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:35.497676 (+   159us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:36.241837 (+744161us) raft_consensus.cc:1685] finished
0709 13:18:36.242104 (+   267us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:36.242549 (+   445us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8790,"prepare.queue_time_us":42907,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":137029,"raft.run_cpu_time_us":260,"raft.run_wall_time_us":22319,"spinlock_wait_cycles":7553280,"thread_start_us":453653,"threads_started":2,"wal-append.queue_time_us":183772}
I20260709 13:18:36.508462 16242 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:36.510572 16632 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:36.510594 16557 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:36.496294 10945 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.233s	user 0.022s	sys 0.039s
I20260709 13:18:36.513111 10795 raft_consensus.cc:1275] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:18:36.519289 16738 raft_consensus.cc:740] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
W20260709 13:18:36.372689 10695 rpcz_store.cc:269] 0709 13:18:08.154597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:08.341244 (+186647us) service_pool.cc:224] Handling call
0709 13:18:36.208825 (+27867581us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:23.943315 (+     0us) spinlock_profiling.cc:243] Waited 5.82 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c7ddab0ac 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc5455 00007f9c848ea16c
0709 13:18:24.147306 (+203991us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:24.148256 (+   950us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:24.148333 (+    77us) write_op.cc:435] Acquired schema lock
0709 13:18:24.148371 (+    38us) tablet.cc:662] Decoding operations
0709 13:18:24.212965 (+ 64594us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:24.213086 (+   121us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:24.213139 (+    53us) tablet.cc:685] Acquiring locks for 23 operations
0709 13:18:24.296086 (+ 82947us) tablet.cc:701] Row locks acquired
0709 13:18:24.414326 (+118240us) write_op.cc:260] PREPARE: finished
0709 13:18:24.480138 (+ 65812us) write_op.cc:270] Start()
0709 13:18:24.538050 (+ 57912us) write_op.cc:276] Timestamp: P: 1783603104464079 usec, L: 0
0709 13:18:24.543952 (+  5902us) op_driver.cc:348] REPLICATION: starting
0709 13:18:24.544909 (+   957us) log.cc:844] Serialized 9100 byte log entry
0709 13:18:25.246834 (+701925us) op_driver.cc:464] REPLICATION: finished
0709 13:18:25.614869 (+368035us) write_op.cc:301] APPLY: starting
0709 13:18:25.615019 (+   150us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:25.755975 (+140956us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:25.756054 (+    79us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:26.018298 (+262244us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:26.080781 (+ 62483us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=46,key_file_lookups=46,delta_file_lookups=46,mrs_lookups=0
0709 13:18:26.081166 (+   385us) write_op.cc:312] APPLY: finished
0709 13:18:26.081905 (+   739us) log.cc:844] Serialized 204 byte log entry
0709 13:18:26.083683 (+  1778us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:26.084162 (+   479us) write_op.cc:454] Released schema lock
0709 13:18:26.084284 (+   122us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:26.084934 (+   650us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:26.084967 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1542724,"reported_tablets":48,"spinlock_wait_cycles":100480,"child_traces":[["op",{"apply.queue_time_us":356657,"apply.run_cpu_time_us":44131,"apply.run_wall_time_us":476846,"cfile_cache_hit":94,"cfile_cache_hit_bytes":100313,"num_ops":23,"prepare.queue_time_us":385322,"prepare.run_cpu_time_us":29768,"prepare.run_wall_time_us":691549,"raft.queue_time_us":132157,"raft.run_cpu_time_us":18513,"raft.run_wall_time_us":308194,"replication_time_us":702690,"spinlock_wait_cycles":13376896,"thread_start_us":1022750,"threads_started":4,"wal-append.queue_time_us":411091,"wal-append.run_cpu_time_us":23546,"wal-append.run_wall_time_us":2127358}]]}
W20260709 13:18:36.520283 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 46678.4us
W20260709 13:18:36.516328 10915 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.787s	user 0.008s	sys 0.016s
I20260709 13:18:36.523799 10795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2853) took 1359 ms. Trace:
I20260709 13:18:36.524919 10795 rpcz_store.cc:276] 0709 13:18:35.164173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:36.423784 (+1259611us) service_pool.cc:224] Handling call
0709 13:18:36.486715 (+ 62931us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:36.522081 (+ 35366us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:36.523735 (+  1654us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:36.525992 16732 raft_consensus.cc:2804] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:36.515327 16375 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:18:36.543013 10915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.908s	user 0.008s	sys 0.017s
W20260709 13:18:36.521747 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 44484.5us
I20260709 13:18:36.517361 16491 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:18:36.545172 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 99939.4us
I20260709 13:18:36.548198 10841 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:36.553634 16835 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:36.554352 16645 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:18:36.555083 16709 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:36.546964 10696 catalog_manager.cc:5697] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 11 to 14, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 14 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:36.555231 16738 consensus_queue.cc:260] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:36.514911 16815 raft_consensus.cc:2749] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:36.559566 16738 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:36.543567 16732 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:36.569453 16732 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:18:36.565044 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 930056us
W20260709 13:18:36.569672 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 106865us
I20260709 13:18:36.569514 16645 raft_consensus.cc:697] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:36.543352 16728 consensus_queue.cc:1048] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:18:36.558521 16709 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:36.587020 11505 raft_consensus.cc:1240] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 18. Current term is 19. Ops: []
W20260709 13:18:36.588658 16865 log.cc:927] Time spent T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.199s	user 0.001s	sys 0.001s
I20260709 13:18:36.589318 16709 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:36.591037 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:36.591400 16709 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:36.558562 10696 catalog_manager.cc:5697] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:36.600466 10696 catalog_manager.cc:5697] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 13 to 12, leader changed from <none> to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 12 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:36.602038 11478 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:36.603677 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:36.602537 11477 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:36.621368 10797 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 17.
I20260709 13:18:36.622442 10916 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:18:36.628948 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 45091.9us
I20260709 13:18:36.623013 16575 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:18:36.634704 10696 catalog_manager.cc:5697] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:36.640255 16864 raft_consensus.cc:2749] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:18:36.641000 10797 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2230) took 1541 ms (client timeout 1654 ms). Trace:
W20260709 13:18:36.641480 10797 rpcz_store.cc:269] 0709 13:18:35.099346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.127579 (+ 28233us) service_pool.cc:224] Handling call
0709 13:18:36.640751 (+1513172us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b5400386154. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:36.640917 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3298700544}
I20260709 13:18:36.642864 16828 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:36.646773 16861 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:18:36.647331 16861 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:36.647677 16861 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:36.637068 16575 raft_consensus.cc:697] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:36.653000 16828 raft_consensus.cc:697] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 17 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:36.665699 10916 leader_election.cc:400] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 election: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 17. Current term is 18.
I20260709 13:18:36.666126 10916 leader_election.cc:403] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:18:36.668365 16869 raft_consensus.cc:2749] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 17. Current term is 18.
I20260709 13:18:36.661085 16645 consensus_queue.cc:237] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:36.677847 16575 consensus_queue.cc:237] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:36.699257 10943 raft_consensus.cc:1275] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:18:36.673806 16491 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:36.664619 16828 consensus_queue.cc:237] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:36.709023 16732 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:36.713032 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:36.714116 11504 raft_consensus.cc:2410] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 21 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 13:18:36.715081 10916 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:36.715322 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:36.681910 16375 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:36.688273 16709 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:36.688295 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:36.691927 16523 consensus_queue.cc:1048] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:18:36.692504 16821 consensus_queue.cc:1059] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:18:36.693524 16871 consensus_queue.cc:1048] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:18:36.696271 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:36.694350 16803 consensus_queue.cc:1048] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
W20260709 13:18:36.722724 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 126026us
I20260709 13:18:36.731190 10949 raft_consensus.cc:1240] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:36.732447 10945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2777) took 1500 ms. Trace:
W20260709 13:18:36.739701 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:36.752857 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:36.750476 10946 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
I20260709 13:18:36.742404 16491 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:36.749282 10945 rpcz_store.cc:276] 0709 13:18:35.231681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.231862 (+   181us) service_pool.cc:224] Handling call
0709 13:18:35.263043 (+ 31181us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:36.548042 (+1284999us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:36.548094 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:36.548872 (+   778us) log.cc:844] Serialized 26 byte log entry
0709 13:18:36.637986 (+ 89114us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:36.638032 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:36.644705 (+  6673us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:36.732150 (+ 87445us) raft_consensus.cc:1685] finished
0709 13:18:36.732174 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:36.732367 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":413336,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":82,"thread_start_us":1343816,"threads_started":2,"wal-append.queue_time_us":175061}
I20260709 13:18:36.772549 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:36.773409 10797 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 21.
I20260709 13:18:36.776449 16632 raft_consensus.cc:3055] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:36.776996 16632 raft_consensus.cc:740] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:36.778685 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:36.779337 11504 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:36.783174 16655 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:36.785181 11478 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:36.785638 16655 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:36.787242 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:36.787438 10948 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 21.
I20260709 13:18:36.790503 11504 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 14.
I20260709 13:18:36.788481 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:36.759703 16242 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:36.788540 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:36.794595 10944 raft_consensus.cc:2368] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 17. Current term is 18.
I20260709 13:18:36.788481 16714 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:36.778136 16632 consensus_queue.cc:260] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:36.772598 16375 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:36.817101 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:36.820788 16242 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:36.821332 16242 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:36.831140 10950 raft_consensus.cc:2393] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 14: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:36.833261 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:36.834161 10950 raft_consensus.cc:2410] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 18 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:18:36.835309 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:36.837639 16714 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:36.798673 16349 consensus_queue.cc:1048] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:18:36.794297 16864 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:36.850909 10949 raft_consensus.cc:2393] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 18: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:36.851089 10842 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:36.852663 10842 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:36.853608 16656 raft_consensus.cc:2749] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:36.854067 11478 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:36.854417 16656 raft_consensus.cc:2749] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:36.857435 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:36.858422 10944 raft_consensus.cc:2410] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:18:36.858939 10941 raft_consensus.cc:1275] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:18:36.861316 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:36.862154 16839 raft_consensus.cc:2749] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:36.864948 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:36.865216 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:36.866135 10795 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:36.866585 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:36.871315 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:36.872309 10793 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 15.
I20260709 13:18:36.873661 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:36.868161 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:36.874537 10797 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
I20260709 13:18:36.875026 10869 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:36.878975 16655 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:36.879479 11477 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:36.879726 16655 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:36.880228 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 80260.1us
W20260709 13:18:36.855322 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 55647.4us
I20260709 13:18:36.856449 10842 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:36.838040 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:18:36.891880 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:36.896049 10948 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
I20260709 13:18:36.802561 16732 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:36.900105 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:36.900255 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:36.905726 10795 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 18.
W20260709 13:18:36.895875 10916 connection.cc:256] RPC call timeout handler was delayed by -1.857076220999943s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1776s for a response
W20260709 13:18:36.913874 10916 connection.cc:256] RPC call timeout handler was delayed by -1.8398119160001443s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20470000000000002s for a response
W20260709 13:18:36.914235 10916 connection.cc:256] RPC call timeout handler was delayed by -1.8381762210001398s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1797s for a response
W20260709 13:18:36.914537 10916 connection.cc:256] RPC call timeout handler was delayed by -1.832776221000131s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18030000000000002s for a response
W20260709 13:18:36.914877 10916 connection.cc:256] RPC call timeout handler was delayed by -1.7624119160002465s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21330000000000002s for a response
W20260709 13:18:36.915462 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 59088.7us
I20260709 13:18:36.916453 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:36.917517 10870 raft_consensus.cc:2393] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 19: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:36.918196 10842 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:36.915545 10916 leader_election.cc:336] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Timed out: RequestConsensusVote RPC to 127.9.116.129:39685 timed out after 1.510s (SENT)
I20260709 13:18:36.919515 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:18:36.919777 10916 leader_election.cc:336] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Timed out: RequestConsensusVote RPC to 127.9.116.129:39685 timed out after 1.557s (SENT)
I20260709 13:18:36.921890 10869 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 18.
W20260709 13:18:36.924163 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:36.891119 16375 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:36.881979 16655 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:36.925468 10950 raft_consensus.cc:1240] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:36.906608 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:36.928954 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:36.904115 16839 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:18:36.932199 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:36.932898 10945 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 20.
I20260709 13:18:36.941442 10948 raft_consensus.cc:1240] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:36.929059 16864 raft_consensus.cc:2749] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:36.901722 16709 consensus_queue.cc:1048] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
W20260709 13:18:36.968251 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:37.019800 10872 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:36.927034 16728 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:36.933301 16803 raft_consensus.cc:2749] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:18:36.930939 16375 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:37.031674 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:37.021265 16839 raft_consensus.cc:697] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 21 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:37.018677 16645 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:37.041280 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:37.026052 16575 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:37.057670 16575 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:37.060894 16575 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:37.089224 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:37.044703 16841 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:37.035933 16728 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:37.047205 10869 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:37.039707 16656 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:37.115283 16656 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:37.043617 16557 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:18:37.131649 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 213550us
I20260709 13:18:37.112013 16816 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:37.139812 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:37.143409 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:37.148273 16375 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:37.149418 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:37.149272 16839 consensus_queue.cc:237] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:37.154619 16349 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:37.155298 16349 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:37.156176 10766 leader_election.cc:400] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 election: Vote denied by peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553) with higher term. Message: Invalid argument: T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 17. Current term is 18.
I20260709 13:18:37.156495 10766 leader_election.cc:403] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:18:37.157524 16349 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:37.159138 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:37.160086 10867 raft_consensus.cc:2393] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 16: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:37.160910 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:37.163312 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:37.163724 16656 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:37.163990 10876 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 19.
I20260709 13:18:37.164767 16815 raft_consensus.cc:2749] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553) with higher term. Message: Invalid argument: T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 17. Current term is 18.
I20260709 13:18:37.166872 16815 consensus_queue.cc:1048] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:37.167927 16732 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:37.168498 16732 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:37.172608 10872 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
I20260709 13:18:37.186580 10869 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 15.
W20260709 13:18:37.171640 16861 consensus_meta.cc:287] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.524s	user 0.012s	sys 0.000s
I20260709 13:18:37.112849 16523 consensus_queue.cc:1059] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 13:18:37.113180 16841 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:37.198933 10870 raft_consensus.cc:1240] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:37.199023 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:37.175796 16655 raft_consensus.cc:2749] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:37.201879 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:37.204625 10948 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 17.
I20260709 13:18:37.199097 16861 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:37.205847 16523 raft_consensus.cc:3055] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:18:37.206251 16523 raft_consensus.cc:740] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:37.207688 16861 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:37.119153 16645 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:37.210599 16645 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:37.207288 16523 consensus_queue.cc:260] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:37.211156 16375 consensus_queue.cc:1059] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:37.137159 16557 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:37.212245 16523 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:37.213069 16728 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:37.213896 16557 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:37.216367 16841 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:37.218114 16732 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:37.158600 16349 consensus_queue.cc:1059] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 13:18:37.219616 16349 raft_consensus.cc:3055] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:37.220001 16349 raft_consensus.cc:740] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:37.220921 16349 consensus_queue.cc:260] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:37.222653 16349 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:37.225888 16242 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:37.176733 16816 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:37.229555 10766 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:37.187208 16864 consensus_queue.cc:1048] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
W20260709 13:18:37.270011 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 348917us
W20260709 13:18:37.283643 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 97799.7us
I20260709 13:18:37.272163 16557 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:18:37.321810 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 46076.1us
I20260709 13:18:37.325011 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:37.325943 11504 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:37.274144 16871 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
W20260709 13:18:37.332553 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 163560us
W20260709 13:18:37.331441 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:37.339193 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:37.325342 16557 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:37.352666 10841 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:37.344532 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:37.360762 11504 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
I20260709 13:18:37.355161 16557 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:18:37.365756 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 124303us
I20260709 13:18:37.357931 16523 raft_consensus.cc:2749] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:18:37.375645 10841 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:37.378357 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:37.397848 10842 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:37.399665 11504 raft_consensus.cc:1240] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 14. Current term is 15. Ops: []
W20260709 13:18:37.396121 16894 log.cc:927] Time spent T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.058s	user 0.001s	sys 0.001s
W20260709 13:18:37.406579 16897 log.cc:927] Time spent T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:18:37.413609 16816 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:37.414099 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:37.415395 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:37.416226 11503 raft_consensus.cc:2393] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 14: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:37.417001 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:37.417794 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:37.418622 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:18:37.422333 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:37.417266 11509 pending_rounds.cc:85] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586: Aborting all ops after (but not including) 11
I20260709 13:18:37.414258 16816 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:37.418273 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:37.418638 11508 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
I20260709 13:18:37.398487 16375 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:37.430939 16709 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:37.429951 11503 raft_consensus.cc:2410] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 18 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:18:37.431700 16709 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:37.382822 16871 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:37.406708 16821 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:18:37.416880 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:37.423557 11509 pending_rounds.cc:107] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:18:37.439914 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:37.441802 11504 raft_consensus.cc:2368] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 19. Current term is 20.
I20260709 13:18:37.441301 11509 raft_consensus.cc:2889] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:37.425490 16908 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:37.457214 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:37.457664 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:37.421394 16841 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:37.459205 10947 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:37.460513 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:37.466596 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:37.440760 16866 raft_consensus.cc:2749] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:37.472352 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:37.473332 10797 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 15.
I20260709 13:18:37.434620 16709 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:37.470741 10841 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:37.478497 10944 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 16.
I20260709 13:18:37.448941 16821 raft_consensus.cc:697] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:37.435115 16816 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:37.479926 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 95095.2us
I20260709 13:18:37.474887 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:37.481830 16494 raft_consensus.cc:2804] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:37.482084 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:37.482904 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:18:37.483273 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:37.486608 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:37.443634 16375 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:37.487152 10945 raft_consensus.cc:1275] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:18:37.464689 16892 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:37.488498 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:37.489856 10941 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:18:37.493376 10916 connection.cc:256] RPC call timeout handler was delayed by -1.6832119160001184s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22210000000000002s for a response
W20260709 13:18:37.494560 10916 leader_election.cc:336] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Timed out: RequestConsensusVote RPC to 127.9.116.129:39685 timed out after 1.964s (SENT)
I20260709 13:18:37.495016 10916 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:37.470881 16908 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:37.497977 16908 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:37.498906 16242 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:37.489563 16494 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:37.498560 16892 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:37.492075 16871 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:38.124506 16375 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:38.141206 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.141564 16375 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:18:38.124608 16893 log.cc:927] Time spent T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.786s	user 0.017s	sys 0.030s
I20260709 13:18:38.142122 16375 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:38.142917 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:38.143582 16892 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:38.144162 16892 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:38.144634 16892 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:38.145087 16375 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:37.465008 16841 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:38.127823 16816 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:38.128834 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 634267us
I20260709 13:18:38.131145 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:18:38.131556 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 628378us
I20260709 13:18:37.668804 16821 consensus_queue.cc:237] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:38.132833 10945 raft_consensus.cc:1275] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:18:38.134192 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:18:38.134466 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:37.484889 16709 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:38.138366 10797 raft_consensus.cc:1240] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 19. Current term is 20. Ops: []
W20260709 13:18:38.139778 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.139577 16645 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.147378 16841 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:38.156252 16523 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.156893 16523 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:38.157918 16709 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:38.154597 16655 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.160167 16655 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:38.163164 16919 raft_consensus.cc:2749] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:18:38.163954 16919 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.165410 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:38.165180 16919 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:38.164058 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:38.164245 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:18:38.169996 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.166009 11503 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 16.
I20260709 13:18:38.166908 11512 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 14.
W20260709 13:18:38.173100 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.175953 10842 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:38.179032 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:38.179258 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:38.179960 10948 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 18.
W20260709 13:18:38.180732 10941 consensus_meta.cc:287] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.688s	user 0.002s	sys 0.020s
I20260709 13:18:38.180863 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:38.184571 10941 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
W20260709 13:18:38.152283 16349 consensus_meta.cc:287] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.929s	user 0.015s	sys 0.006s
I20260709 13:18:38.181700 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:38.134665 16728 raft_consensus.cc:3055] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:38.174665 11511 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
I20260709 13:18:38.181623 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:38.149526 16494 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:38.147305 16861 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.146741 16892 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:38.202932 10944 raft_consensus.cc:2393] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 14: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:38.200711 10945 raft_consensus.cc:2393] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 21: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
W20260709 13:18:38.220539 10916 connection.cc:256] RPC call timeout handler was delayed by -1.5397541410000031s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20350000000000001s for a response
W20260709 13:18:38.221163 10916 connection.cc:256] RPC call timeout handler was delayed by -1.5202645419999499s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1932s for a response
W20260709 13:18:38.221575 10916 connection.cc:256] RPC call timeout handler was delayed by -1.4893541409999216s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20910000000000004s for a response
W20260709 13:18:38.222105 10916 connection.cc:256] RPC call timeout handler was delayed by -1.4628762209999877s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2214s for a response
W20260709 13:18:38.222548 10916 connection.cc:256] RPC call timeout handler was delayed by -1.4494321619999937s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1786s for a response
W20260709 13:18:38.223860 10916 connection.cc:256] RPC call timeout handler was delayed by -1.415911916000141s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.25179999999999997s for a response
W20260709 13:18:38.224279 10916 connection.cc:256] RPC call timeout handler was delayed by -1.2712321620001603s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19840000000000002s for a response
W20260709 13:18:38.224686 10916 connection.cc:256] RPC call timeout handler was delayed by -1.2301541409999572s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2379s for a response
I20260709 13:18:38.222457 11505 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 19.
W20260709 13:18:38.225996 10947 consensus_meta.cc:287] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.765s	user 0.013s	sys 0.009s
I20260709 13:18:38.227937 10947 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 20.
I20260709 13:18:38.179028 16523 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:38.154565 16645 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:38.229596 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:38.230621 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:38.233156 10797 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:38.200484 16738 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:38.238481 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:38.191643 16841 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.212314 16375 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.240527 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:38.244292 10793 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 14.
I20260709 13:18:38.191910 16728 raft_consensus.cc:740] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:38.245826 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:18:38.249382 16890 log.cc:927] Time spent T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 1.134s	user 0.000s	sys 0.019s
I20260709 13:18:38.249874 16841 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:38.250631 10943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2790) took 1532 ms. Trace:
I20260709 13:18:38.212575 16861 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:38.251003 16655 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:38.252681 16861 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:38.252575 16866 raft_consensus.cc:2749] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:18:38.251046 10943 rpcz_store.cc:276] 0709 13:18:36.718088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:36.731607 (+ 13519us) service_pool.cc:224] Handling call
0709 13:18:36.732024 (+   417us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:36.732738 (+   714us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:36.732768 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:36.733426 (+   658us) log.cc:844] Serialized 26 byte log entry
0709 13:18:37.039211 (+305785us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:37.039259 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:37.039336 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:37.780859 (+741523us) spinlock_profiling.cc:243] Waited 2.82 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:18:38.250327 (+469468us) raft_consensus.cc:1685] finished
0709 13:18:38.250348 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:38.250565 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":121,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":97,"spinlock_wait_cycles":6491392,"thread_start_us":305356,"threads_started":1,"wal-append.queue_time_us":381787}
I20260709 13:18:38.255118 10947 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 19.
I20260709 13:18:38.227952 16561 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.258289 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:38.258566 10941 raft_consensus.cc:2410] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:18:38.259093 11511 raft_consensus.cc:2393] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 14: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:38.242328 16632 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:18:38.261272 16916 log.cc:927] Time spent T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.054s	user 0.011s	sys 0.001s
I20260709 13:18:38.261758  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:18:38.265062 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.272089 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:38.274087 11477 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:38.276475 10795 raft_consensus.cc:2393] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 12: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:38.252938 16375 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:38.281014 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:38.282881 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:38.293753 11477 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:38.295233 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:38.283156 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:38.300480 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:38.295544 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:38.306989 11477 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:38.306989 10766 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:38.324054 10915 leader_election.cc:400] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 19. Current term is 20.
I20260709 13:18:38.327375 10915 leader_election.cc:403] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:18:38.281095 16349 raft_consensus.cc:2804] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:38.331631 16839 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:38.332468 10801 raft_consensus.cc:1275] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:18:38.332634 11511 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 16.
I20260709 13:18:38.246418 16738 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:38.326311 16523 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.336041 11510 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 15.
I20260709 13:18:38.327469 10766 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:38.345747 10797 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 21.
W20260709 13:18:38.352686 11478 leader_election.cc:336] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Timed out: RequestConsensusVote RPC to 127.9.116.129:39685 timed out after 1.639s (SENT)
I20260709 13:18:38.353329 11478 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:38.357220 10946 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 16.
I20260709 13:18:38.270365 16561 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:38.350242 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 45768us
I20260709 13:18:38.351027 10916 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:38.332353 16839 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:38.366611 11478 connection.cc:256] RPC call timeout handler was delayed by -1.0638959030000024s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18630000000000002s for a response
I20260709 13:18:38.371088 10950 raft_consensus.cc:1275] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:18:38.334033 16349 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:38.371402 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:38.375241 10941 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 19.
I20260709 13:18:38.348261 16709 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:38.380129 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:38.360090 16645 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:38.358606 16866 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.361758 16714 raft_consensus.cc:2749] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 19. Current term is 20.
I20260709 13:18:38.387100 10870 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 20.
I20260709 13:18:38.376940 16900 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:18:38.374667 16557 consensus_meta.cc:287] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.994s	user 0.020s	sys 0.002s
I20260709 13:18:38.371497 16861 raft_consensus.cc:2749] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:38.326701 16242 consensus_queue.cc:1059] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 13:18:38.378082 16839 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:38.396268 16375 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:38.396941 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 111839us
I20260709 13:18:38.399163 16917 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:18:38.406888 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:38.413830 10793 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 16.
I20260709 13:18:38.425900 10873 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:38.379541 16349 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:38.349965 16523 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:38.428596 16917 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:38.434077 10916 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:38.362336 16728 consensus_queue.cc:260] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:38.451609 16925 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:38.439791 16815 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:38.453203 10916 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:38.453626 16242 raft_consensus.cc:2749] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:38.395700 16866 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:38.419328 16557 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:38.432124 16375 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:38.455585 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 307510us
I20260709 13:18:38.442292 16917 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:38.460175 16841 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:38.462908 16913 raft_consensus.cc:2804] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:38.463480 16912 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:38.437358 16709 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.460227 16929 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.485399 11477 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:38.477166 16728 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:38.474841 16375 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:38.479774 16911 raft_consensus.cc:2749] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:38.483814 16913 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:38.485275 16912 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:38.473181 16815 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:38.494973 16924 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:18:38.510980 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 183582us
I20260709 13:18:38.521477 11510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56404 (request call id 1018) took 1102 ms. Trace:
I20260709 13:18:38.522038 11510 rpcz_store.cc:276] 0709 13:18:37.413413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.413682 (+   269us) service_pool.cc:224] Handling call
0709 13:18:38.515339 (+1101657us) spinlock_profiling.cc:243] Waited 788 ms on lock 0x7b54005f96d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:38.515506 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1811893504}
I20260709 13:18:38.489003 16709 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:38.479151 16925 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:38.509644 16912 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:38.509799 16913 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:38.510675 16242 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.517561 16924 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:38.515749 16738 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:18:38.541460 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 91352.9us
I20260709 13:18:38.546193 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:38.549131 10841 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:38.551846 10793 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 17.
I20260709 13:18:38.505345 16929 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:38.542225 16930 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.557523 10795 raft_consensus.cc:1240] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:18:38.559857 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:38.560571 10801 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 17.
I20260709 13:18:38.560998 16925 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:38.562412 16375 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:38.544821 16924 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:38.558396 10841 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:38.546243 16900 raft_consensus.cc:697] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 20 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
W20260709 13:18:38.603902 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 45563.6us
I20260709 13:18:38.569193 10916 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:38.592535 16930 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:38.607426 16900 consensus_queue.cc:237] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:38.608042 16645 consensus_queue.cc:1048] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:38.608690 10916 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:38.609647 16930 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:38.575091 16523 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:38.568245 16911 consensus_queue.cc:1048] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:38.590454 10944 raft_consensus.cc:1240] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:38.611263 16921 raft_consensus.cc:2749] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:18:38.569880 16617 consensus_queue.cc:1048] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:18:38.610247 10916 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:38.612900 16617 raft_consensus.cc:2749] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:38.612675 16523 consensus_queue.cc:1048] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:38.545047 16242 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:38.613459 10916 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:38.613780 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.569505 10801 raft_consensus.cc:1240] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 18. Current term is 19. Ops: []
I20260709 13:18:38.615089 16921 raft_consensus.cc:2749] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:38.610832 16491 raft_consensus.cc:2749] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:38.616214 16242 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:38.586939 16927 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:38.616987 10916 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:38.617281 16927 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:38.618297 10873 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 20.
I20260709 13:18:38.571432 16866 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:38.619798 16927 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:38.619987 10916 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:38.572350 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:38.580441 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:38.621565 10950 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 19.
I20260709 13:18:38.621672 10794 raft_consensus.cc:2376] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18. Re-sending same reply.
I20260709 13:18:38.562426 16709 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:38.623047 11477 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:38.592280 16929 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:38.624147 16732 raft_consensus.cc:2804] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:18:38.624441 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 59209.3us
I20260709 13:18:38.624643 16732 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:38.625083 16732 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:38.573612 16845 raft_consensus.cc:3055] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:38.627718 16845 raft_consensus.cc:740] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:38.629050 16845 consensus_queue.cc:260] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:38.580535 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:38.630513 10800 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 14.
I20260709 13:18:38.578567 16919 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:38.574220 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.589337 10941 raft_consensus.cc:1275] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 13:18:38.555768 16821 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.579991 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:38.605178 16815 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:38.569933 16714 consensus_queue.cc:1059] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:18:38.630635 16845 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:18:38.588608 16923 log.cc:927] Time spent T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:18:38.587996 16557 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:38.580962 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:38.641829 10795 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
W20260709 13:18:38.642138 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.620456 16617 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:38.570642 16561 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:38.559257 16935 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:38.646775 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:38.631724 10793 raft_consensus.cc:2376] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18. Re-sending same reply.
I20260709 13:18:38.603047 16871 raft_consensus.cc:2749] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:38.577062 16925 consensus_queue.cc:1048] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 13:18:38.638172 16927 consensus_queue.cc:1048] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:18:38.589293 16928 log.cc:927] Time spent T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 13:18:38.641130 11509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56416 (request call id 930) took 1226 ms. Trace:
W20260709 13:18:38.611012 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.664520 11510 raft_consensus.cc:2410] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:18:38.667340 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:38.687052 11511 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
I20260709 13:18:38.669052 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:38.672530 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:38.688190 10950 raft_consensus.cc:2410] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:18:38.675073 10766 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:38.662725 16919 raft_consensus.cc:2749] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:38.696437 11506 raft_consensus.cc:2393] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 15: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:38.684265 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:38.705092 10801 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
I20260709 13:18:38.644930 16617 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:38.700028 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:38.707665 11510 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
I20260709 13:18:38.685736 11509 rpcz_store.cc:276] 0709 13:18:37.415240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.416534 (+  1294us) service_pool.cc:224] Handling call
0709 13:18:37.416905 (+   371us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:18:37.484920 (+ 68015us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:37.484985 (+    65us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:18:37.523052 (+ 38067us) log.cc:844] Serialized 78 byte log entry
0709 13:18:38.323806 (+800754us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:38.323847 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:38.323907 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:38.640847 (+316940us) raft_consensus.cc:1685] finished
0709 13:18:38.640865 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:38.641049 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":19677,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":117,"thread_start_us":180364,"threads_started":1,"wal-append.queue_time_us":838645}
I20260709 13:18:38.590754 16494 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:38.709411 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 89506.5us
I20260709 13:18:38.700927 10766 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:38.643965 16929 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:38.671041 16935 raft_consensus.cc:697] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:38.714578 11478 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:38.714363 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:38.664707 16523 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:38.644640 16917 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:38.700018 16714 raft_consensus.cc:3055] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:38.699885 16815 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.710098 16617 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:38.667124 16349 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:38.729820 10766 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:38.674638 16821 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:38.728829 10841 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:38.726717 11478 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:38.744565 10916 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:38.716403 16709 raft_consensus.cc:2749] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:38.715850 16937 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.724738 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:38.756238 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:38.747047 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:38.756894 10950 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
I20260709 13:18:38.748989 10766 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:38.722764 16714 raft_consensus.cc:740] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:38.766889 10841 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:38.770637 16375 raft_consensus.cc:2804] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:38.771236 16375 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:38.771665 16375 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:38.773272 16935 consensus_queue.cc:237] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:38.774868 16937 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:38.723737 16929 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:38.777143 16937 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:38.747354 16523 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:38.723608 16933 raft_consensus.cc:2749] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:18:38.781322 16523 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:38.781710 16912 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:38.782083 16821 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:38.756490 11509 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
I20260709 13:18:38.766305 16242 raft_consensus.cc:2749] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:38.783926 11507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56410 (request call id 947) took 1347 ms. Trace:
I20260709 13:18:38.784071 16913 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:38.770332 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 50987.2us
I20260709 13:18:38.785753 10942 raft_consensus.cc:1240] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 13. Current term is 14. Ops: []
W20260709 13:18:38.786518 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 59870.9us
I20260709 13:18:38.748287 16728 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:18:38.751924 16841 raft_consensus.cc:2804] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:18:38.798282 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.778509 16929 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:38.804003 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:38.758183 16866 raft_consensus.cc:2804] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:38.784334 11507 rpcz_store.cc:276] 0709 13:18:37.342179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.374477 (+ 32298us) service_pool.cc:224] Handling call
0709 13:18:37.408041 (+ 33564us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:37.408283 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:18:38.140417 (+732134us) log.cc:844] Serialized 14 byte log entry
0709 13:18:38.378968 (+238551us) log.cc:844] Serialized 14 byte log entry
0709 13:18:38.589940 (+210972us) spinlock_profiling.cc:243] Waited 2.87 ms on lock 0x7b18002c1e80. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc83f3 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:18:38.684130 (+ 94190us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:38.684181 (+    51us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:18:38.684223 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:38.684281 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:38.689438 (+  5157us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":748146,"prepare.run_cpu_time_us":114,"prepare.run_wall_time_us":1921,"raft.queue_time_us":107976,"raft.run_cpu_time_us":319,"raft.run_wall_time_us":15070,"spinlock_wait_cycles":6606080,"thread_start_us":962899,"threads_started":3,"wal-append.queue_time_us":324188}
I20260709 13:18:38.801892 16841 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:38.814898 16841 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:38.815182 10916 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:38.817173 10916 leader_election.cc:400] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 election: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 17. Current term is 18.
I20260709 13:18:38.748123 16815 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:38.828873 10916 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:38.795902 16523 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:18:38.831593 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.837682 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:38.838670 11510 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
I20260709 13:18:38.813076 16866 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:38.831676 16919 raft_consensus.cc:2804] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:38.844468 10800 raft_consensus.cc:1240] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 19. Current term is 20. Ops: []
I20260709 13:18:38.847079 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:38.853538 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:38.854147 10797 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:38.851303 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:38.835634 16523 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:38.856817 10915 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:38.811340 16732 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:38.861191 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:38.847847 16866 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:38.809312 16924 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:38.846901 16919 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:38.879432 10873 raft_consensus.cc:1275] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:18:38.849364 16491 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:18:38.880484 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:38.878165 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:38.881367 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:38.883252 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:38.893335 10800 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
I20260709 13:18:38.890220 10797 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 14.
I20260709 13:18:38.899395 11506 raft_consensus.cc:2368] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 19. Current term is 20.
I20260709 13:18:38.862826 16523 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:38.875618 16921 raft_consensus.cc:2804] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:38.912405 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:38.917716 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 168810us
I20260709 13:18:38.918299 10797 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
I20260709 13:18:38.918493 10766 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:38.917469 10871 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 18.
I20260709 13:18:38.920723 10766 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:38.892088 16919 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:38.925415 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:38.936050 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 107258us
I20260709 13:18:38.936538 10766 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:38.912161 16913 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:38.939963 10766 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:38.948727 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:38.940183 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:38.958874 16892 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:38.959429 16892 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:38.967000 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:38.967447 16632 consensus_meta.cc:287] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.689s	user 0.010s	sys 0.002s
I20260709 13:18:38.971719 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:38.974499 10797 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 19.
I20260709 13:18:38.921981 16921 raft_consensus.cc:697] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 15 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:18:38.972718 11478 leader_election.cc:336] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Timed out: RequestConsensusVote RPC to 127.9.116.129:39685 timed out after 1.863s (SENT)
I20260709 13:18:38.975898 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:38.975971 11478 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:38.975284 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:38.976773 10949 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:38.976868 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:38.977660 10947 raft_consensus.cc:2410] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:18:38.935424 16934 raft_consensus.cc:2749] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:18:38.979557 10916 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:38.983920 10945 raft_consensus.cc:2393] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 19: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:38.974583 10874 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 12.
I20260709 13:18:38.950668 16917 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:38.898538 16714 consensus_queue.cc:260] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:38.993626 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:38.994307 10800 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
I20260709 13:18:38.966934 16728 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:38.970598 16892 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:38.988327 16242 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:18:38.997745 16841 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:38.998598 16932 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:38.999145 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:38.999418 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:38.999688 16941 log.cc:927] Time spent T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.237s	user 0.001s	sys 0.000s
I20260709 13:18:38.999887 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:38.998622 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:39.000818 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:39.001672 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:39.003417 10872 raft_consensus.cc:1240] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:39.056416 10792 raft_consensus.cc:2393] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 17: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:39.004047 16950 raft_consensus.cc:2749] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:18:39.004395 16952 log.cc:927] Time spent T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:18:39.011350 10916 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:18:39.054057 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 74047.8us
I20260709 13:18:38.988478 16491 consensus_queue.cc:1048] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:18:39.019233 16242 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:39.020088 16815 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:39.022886 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:39.023448 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 56732.5us
W20260709 13:18:39.024070 10874 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49290 (request call id 931) took 1909 ms (client timeout 2189 ms). Trace:
I20260709 13:18:39.028422 10867 raft_consensus.cc:2410] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 15 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:18:38.978420 16349 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:38.986604 16655 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.987668 16738 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:39.032902 10876 raft_consensus.cc:2393] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 18: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:39.086552 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:39.065599 10949 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 21.
I20260709 13:18:39.071640 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:39.090566 10872 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 16.
I20260709 13:18:39.072098 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:39.072919 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:39.027010 16932 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:39.120334 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:18:39.074707 10874 rpcz_store.cc:269] 0709 13:18:37.114387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.123686 (+  9299us) service_pool.cc:224] Handling call
0709 13:18:39.023870 (+1900184us) spinlock_profiling.cc:243] Waited 1.83 s on lock 0x7b540009bad4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:39.023984 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4217787264}
I20260709 13:18:39.080022 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:39.081228 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:39.082211 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:39.052383 16861 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:39.085518 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:39.094393 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:39.094686 16841 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:39.095343 16732 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:39.095208 16738 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:39.086621 16913 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:18:39.109052 16839 consensus_meta.cc:287] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.677s	user 0.010s	sys 0.000s
I20260709 13:18:39.075759 16242 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:18:39.111395 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 55603.8us
I20260709 13:18:39.115037 16921 consensus_queue.cc:237] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:39.116941 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:39.134332 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:39.134846 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:39.135224 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:39.134855 10950 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:39.135448 10872 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
I20260709 13:18:39.134989 11512 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 16.
I20260709 13:18:39.047736 16375 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:39.127216 11508 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
I20260709 13:18:39.126585 11510 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 19.
I20260709 13:18:39.128566 10876 raft_consensus.cc:2410] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:18:39.128798 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:39.129918 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:39.035743 16816 consensus_queue.cc:1048] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:18:39.133893 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:39.117589 16655 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:39.129622 16924 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:39.147585 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:39.144538 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:39.166417 10797 raft_consensus.cc:2393] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 16: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:39.124549 16932 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:39.138183 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:39.141875 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:39.141152 16738 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:39.149356 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 138082us
I20260709 13:18:39.134079 16913 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:39.149446 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:39.138447 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:39.192014 10874 raft_consensus.cc:2410] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 15 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:18:39.137750 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:39.192428 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:39.192778 10867 raft_consensus.cc:2410] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:18:39.143442 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:39.145504 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:39.193557 10793 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
I20260709 13:18:39.193667 10795 raft_consensus.cc:1240] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:39.154847 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:39.154981 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:39.158803 16951 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.160395 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:39.168736 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:39.171948 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:39.290385 10872 raft_consensus.cc:2410] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
W20260709 13:18:39.290755 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:39.292748 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 43848.8us
I20260709 13:18:39.289340 10876 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
I20260709 13:18:39.296039 10766 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:39.299124 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:39.297925 10841 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:39.303077 16821 raft_consensus.cc:2749] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:39.303555 16919 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:39.297097 16951 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:39.307134 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:39.308404 16951 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:39.153396 10873 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:39.308840 16815 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:18:39.309391 16815 raft_consensus.cc:697] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:39.310263 16815 consensus_queue.cc:237] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:39.312583 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:39.245381 16375 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:39.313251 10872 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 18.
I20260709 13:18:39.245980 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:39.201930 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:39.314128 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:39.314772 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:39.316272 10867 raft_consensus.cc:1240] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:39.317401 10950 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 20.
I20260709 13:18:39.200752 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:39.250342 16494 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:39.318178 10945 raft_consensus.cc:1240] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 19. Current term is 20. Ops: []
I20260709 13:18:39.243155 10916 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:39.194299 16913 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:18:39.319638 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 128372us
I20260709 13:18:39.255098 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:39.255416 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:39.320432 10916 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:39.322330 10869 raft_consensus.cc:1275] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 13:18:39.322983 16911 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:39.323449 16911 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:39.323853 16911 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:39.319780 10874 raft_consensus.cc:1275] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:18:39.260707 10941 raft_consensus.cc:1240] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 18. Current term is 19. Ops: []
I20260709 13:18:39.268697 16632 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.269093 16709 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.274044 10794 raft_consensus.cc:1240] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 18. Current term is 19. Ops: []
I20260709 13:18:39.149461 16839 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:39.276192 16714 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:39.204239 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:39.330592 10873 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 21.
I20260709 13:18:39.211228 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:39.203009 16645 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.283838 10793 raft_consensus.cc:1275] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
W20260709 13:18:39.240415 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 80925.5us
I20260709 13:18:39.239585 11478 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:39.166242 10792 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
I20260709 13:18:39.308352 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:39.327423 10801 raft_consensus.cc:1275] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:18:39.328954 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:39.340296 10795 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:39.149091 10842 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:39.334718 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:39.352051 10799 raft_consensus.cc:1275] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:18:39.352913 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:39.353514 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:39.353773 10873 raft_consensus.cc:2393] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 14: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:39.353363 10766 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:39.356223 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:39.358088 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:39.390954 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:39.357101 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:39.390591 10799 raft_consensus.cc:2376] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 17. Re-sending same reply.
I20260709 13:18:39.405901 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:39.405699 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:39.297932 16866 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:39.413350 10869 raft_consensus.cc:1275] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:18:39.413525 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:39.354589 16907 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.351161 16912 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:39.415066 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 113706us
I20260709 13:18:39.416904 10800 raft_consensus.cc:2393] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 14: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:39.424036 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:39.424542 10795 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 19.
I20260709 13:18:39.390190 16375 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:18:39.354812 16494 raft_consensus.cc:2804] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:39.424705 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:18:39.427004 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 187555us
I20260709 13:18:39.427116 10948 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:39.399111 16933 raft_consensus.cc:2804] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:39.427402 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:39.428818 10841 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:18:39.434432 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:39.436117 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:39.437209 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 303915us
I20260709 13:18:39.355506 16632 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:39.389575 16969 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:18:39.445731 16969 raft_consensus.cc:697] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 18 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:39.446692 16969 consensus_queue.cc:237] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:39.417305 16919 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:39.438918 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:39.450738 10943 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:18:39.441083 10842 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:39.443393 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:39.439582 16375 raft_consensus.cc:697] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:39.354339 16645 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:39.445232 16845 consensus_meta.cc:287] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.800s	user 0.011s	sys 0.000s
I20260709 13:18:39.462373 10944 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
I20260709 13:18:39.450668 10868 raft_consensus.cc:2393] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 16: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:39.440433 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:39.444185 16866 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:39.455206 16732 consensus_queue.cc:1048] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.006s
I20260709 13:18:39.463600 16645 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:39.455947 16375 consensus_queue.cc:237] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:39.464529 16632 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:39.465698 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:39.467324 10950 raft_consensus.cc:1275] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:18:39.467782 10795 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
I20260709 13:18:39.468683 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:39.469372 10795 raft_consensus.cc:2376] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 19. Re-sending same reply.
I20260709 13:18:39.470546 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:39.470945 10876 raft_consensus.cc:2393] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 14: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:39.471143 10795 raft_consensus.cc:2376] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 18. Re-sending same reply.
I20260709 13:18:39.472107 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:39.475328 16732 consensus_queue.cc:1048] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:18:39.477265 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:39.457721 16617 consensus_meta.cc:287] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.690s	user 0.009s	sys 0.002s
I20260709 13:18:39.459122 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:39.460270 16242 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:39.431771 16907 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:39.479429 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 50683.2us
I20260709 13:18:39.351199 16709 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:39.487329 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:39.489468 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 50236.1us
I20260709 13:18:39.490342 11478 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:39.433331 16494 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:39.496260 16839 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:39.439481 16933 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:39.500766 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:39.500754 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 62563.7us
I20260709 13:18:39.499398 10943 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 21.
W20260709 13:18:39.507400 10916 connection.cc:256] RPC call timeout handler was delayed by -1.0475321499998245s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17310000000000003s for a response
I20260709 13:18:39.511575 10947 raft_consensus.cc:1240] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 21. Current term is 22. Ops: []
I20260709 13:18:39.515158 10949 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:18:39.497656 16523 consensus_meta.cc:287] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.577s	user 0.007s	sys 0.001s
I20260709 13:18:39.521147 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:39.516732 11478 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:39.490047 16821 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:39.523671 10915 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:39.545070 10915 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:39.525169 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:39.546041 10946 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:18:39.525620 16632 consensus_queue.cc:1059] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:39.526168 16242 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:39.526198 16709 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:39.527428 16907 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:39.519838 16655 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:39.477842 16892 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:39.500123 16494 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:39.498910 16866 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:39.532737 10916 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:39.490072 16617 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:39.522897 16738 consensus_queue.cc:1059] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.012s
I20260709 13:18:39.514976 16732 raft_consensus.cc:2804] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:39.536118 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:39.536648 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:39.514708 16839 consensus_queue.cc:1059] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 13:18:39.537392 16937 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:18:39.538371 10874 raft_consensus.cc:1275] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:18:39.536127 10948 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 20.
I20260709 13:18:39.541579 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:39.556319 16921 raft_consensus.cc:2749] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:39.577769 10867 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
I20260709 13:18:39.505635 16933 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:39.552114 11478 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:39.583624 10915 leader_election.cc:400] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 19. Current term is 20.
I20260709 13:18:39.583990 10915 leader_election.cc:403] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:18:39.561435 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:39.584769 11512 raft_consensus.cc:2393] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 18: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:39.585421 16242 raft_consensus.cc:2749] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 19. Current term is 20.
I20260709 13:18:39.586933 16839 raft_consensus.cc:3055] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:39.587071 16924 raft_consensus.cc:2749] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:39.587404 16839 raft_consensus.cc:740] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:39.587704 16732 raft_consensus.cc:697] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 16 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:39.588445 11478 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:39.514768 16929 consensus_queue.cc:1059] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 13:18:39.561024 16959 consensus_queue.cc:1048] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.013s
I20260709 13:18:39.569041 16892 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:39.543596 16821 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:39.498008 16913 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:39.562327 10841 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:39.551514 16491 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:39.625799 10945 raft_consensus.cc:1275] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 13:18:39.626554 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 64320.1us
I20260709 13:18:39.627295 10799 raft_consensus.cc:1275] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:18:39.632928 10946 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 18.
I20260709 13:18:39.627583 10841 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:39.573274 10800 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 17: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:39.573331 10949 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 18.
I20260709 13:18:39.548039 16861 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:39.575052 10872 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:18:39.536880 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:39.571434 16866 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:39.541782 16523 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:39.576135 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 43487.8us
I20260709 13:18:39.589512 10842 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:39.589958 16645 consensus_queue.cc:1059] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 19.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:18:39.666652 10916 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:39.668324 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:39.589169 16911 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:39.627112 16821 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:39.567711 16932 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:39.578656 16937 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:39.636782 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 48430.9us
I20260709 13:18:39.676293 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:39.677902 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:39.599344 16969 raft_consensus.cc:2804] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:39.656327 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:39.680316 10794 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:39.683066 10870 raft_consensus.cc:2393] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 20: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:39.622990 16655 raft_consensus.cc:3055] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:39.590605 16732 consensus_queue.cc:237] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:39.668972 10799 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:39.644964 11477 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:39.628046 16950 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.596663 16617 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:39.701592 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:39.682566 16937 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:39.671509 16714 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:18:39.708112 10767 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.116.130:37895 timed out after 3.075s (SENT) [suppressed 38 similar messages]
I20260709 13:18:39.625020 16561 raft_consensus.cc:2804] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:39.724454 10916 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:39.724521 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 135060us
I20260709 13:18:39.672542 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:39.633494 16491 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:39.728305 16491 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:39.729008 16949 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:39.730145 16930 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:39.730530 16930 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:39.730893 16930 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:39.673563 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:39.623967 16494 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:39.733486 16494 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:39.656401 10873 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:39.724324 16561 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:39.735013 16561 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:39.639616 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:39.739872 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:39.741057 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:39.741277 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:39.742169 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:39.660581 16866 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:39.743500 10794 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 18.
I20260709 13:18:39.695482 16961 consensus_queue.cc:1048] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:18:39.747880 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 56434.4us
I20260709 13:18:39.695853 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:39.696676 16523 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:39.697547 16971 log.cc:927] Time spent T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:18:39.685077 16655 raft_consensus.cc:740] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:39.698302 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:39.667424 16861 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:39.762501 16861 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:39.764075 16913 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:39.764382 10916 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:39.715900 16907 consensus_queue.cc:1048] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 13:18:39.765882 16907 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:18:39.765467 16561 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:39.766264 10916 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:39.725464 10842 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:39.725941 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:39.725489 10795 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 18.
I20260709 13:18:39.724915 10798 raft_consensus.cc:2393] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 16: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:39.669899 16839 consensus_queue.cc:260] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:39.715960 16969 raft_consensus.cc:697] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:18:39.747773 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 102905us
I20260709 13:18:39.734114 16937 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:39.751089 10872 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 20.
W20260709 13:18:39.751328 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 71462.1us
I20260709 13:18:39.754300 11478 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:39.638685 16929 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:39.713204 16933 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:39.758563 16924 consensus_queue.cc:1048] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:18:39.759995 16655 consensus_queue.cc:260] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:39.839927 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:39.766402 16907 raft_consensus.cc:697] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 19 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:39.756942 10873 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 21.
I20260709 13:18:39.766963 11508 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 12.
I20260709 13:18:39.753583 16921 raft_consensus.cc:2749] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:18:39.812939 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 58748us
W20260709 13:18:39.834003 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 206477us
I20260709 13:18:39.750512 16738 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:18:39.844254 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:39.847549 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:39.842149 11477 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:39.812422 10766 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:39.768491 16911 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:39.863801 11510 raft_consensus.cc:1240] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:39.813396 16929 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:39.877650 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:39.725209 16950 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:39.866809 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 54432us
I20260709 13:18:39.906245 10766 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:39.906152 10942 raft_consensus.cc:1240] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:39.778486 16932 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:39.775728 16617 consensus_queue.cc:1048] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.099s
I20260709 13:18:39.814972 16349 raft_consensus.cc:2749] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:18:39.869226 16951 raft_consensus.cc:2749] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:39.835260 16913 raft_consensus.cc:2749] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:39.872814 10949 raft_consensus.cc:1240] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:39.879007 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:39.950179 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:39.881522 10946 raft_consensus.cc:1240] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:39.900319 16924 raft_consensus.cc:2804] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:18:39.901161 16969 consensus_queue.cc:237] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:39.938934 16839 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:39.878222 10841 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
W20260709 13:18:39.910919 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 185496us
W20260709 13:18:39.869746 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 161649us
I20260709 13:18:39.922919 16974 raft_consensus.cc:3055] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:18:39.923972 16949 raft_consensus.cc:3055] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:18:39.936067 10943 raft_consensus.cc:1240] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 16. Current term is 17. Ops: []
W20260709 13:18:39.869990 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 103776us
W20260709 13:18:39.997321 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 155247us
I20260709 13:18:40.001989 10916 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:39.880645 16655 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:39.890395 16929 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:39.940016 10766 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:39.937428 16738 raft_consensus.cc:697] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 21 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:39.903304 16937 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:39.948269 16557 raft_consensus.cc:2749] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:18:40.027546 10696 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42506 (request call id 156) took 39477 ms (client timeout 15000 ms). Trace:
I20260709 13:18:39.946368 16987 raft_consensus.cc:2804] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:40.003697 10870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49290 (request call id 944) took 1080 ms. Trace:
I20260709 13:18:39.981359 16900 raft_consensus.cc:3055] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:18:40.039669 16984 log.cc:927] Time spent T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:18:39.997085 16974 raft_consensus.cc:740] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:18:40.009936 16965 log.cc:927] Time spent T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 13:18:40.010306 16990 log.cc:927] Time spent T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:18:40.014889 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:40.016052 11477 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:39.998726 16924 raft_consensus.cc:697] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 20 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
W20260709 13:18:40.064327 16991 log.cc:927] Time spent T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:18:40.064524 16924 consensus_queue.cc:237] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:40.041026 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 45565.4us
I20260709 13:18:40.011957 16349 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:40.008188 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 68268.1us
I20260709 13:18:40.035912 10870 rpcz_store.cc:276] 0709 13:18:38.923155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:38.923297 (+   142us) service_pool.cc:224] Handling call
0709 13:18:40.003514 (+1080217us) spinlock_profiling.cc:243] Waited 712 ms on lock 0x7b5400316fd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:40.003636 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1638728064}
W20260709 13:18:40.097165 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 81218.5us
W20260709 13:18:40.053386 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 51467.7us
I20260709 13:18:40.035940 16961 consensus_queue.cc:1048] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.027s
I20260709 13:18:40.038731 16900 raft_consensus.cc:740] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:40.071323 10842 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:40.041481 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 163367us
I20260709 13:18:40.015924 16892 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:40.070528 16987 raft_consensus.cc:697] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:18:40.097966 16984 log.cc:927] Time spent T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.058s	user 0.002s	sys 0.000s
W20260709 13:18:40.028215 10696 rpcz_store.cc:269] 0709 13:18:00.535528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:00.535881 (+   353us) service_pool.cc:224] Handling call
0709 13:18:40.012666 (+39476785us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:37.103501 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:37.164959 (+ 61458us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:37.165011 (+    52us) write_op.cc:435] Acquired schema lock
0709 13:18:37.165033 (+    22us) tablet.cc:662] Decoding operations
0709 13:18:37.166209 (+  1176us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:37.166313 (+   104us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:37.166341 (+    28us) tablet.cc:685] Acquiring locks for 12 operations
0709 13:18:37.191141 (+ 24800us) tablet.cc:701] Row locks acquired
0709 13:18:37.191191 (+    50us) write_op.cc:260] PREPARE: finished
0709 13:18:37.191369 (+   178us) write_op.cc:270] Start()
0709 13:18:37.191520 (+   151us) write_op.cc:276] Timestamp: P: 1783603117191304 usec, L: 0
0709 13:18:37.191558 (+    38us) op_driver.cc:348] REPLICATION: starting
0709 13:18:37.192426 (+   868us) log.cc:844] Serialized 4868 byte log entry
0709 13:18:38.534533 (+1342107us) op_driver.cc:464] REPLICATION: finished
0709 13:18:38.732069 (+197536us) write_op.cc:301] APPLY: starting
0709 13:18:38.732169 (+   100us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:39.097174 (+365005us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:39.097323 (+   149us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:39.107966 (+ 10643us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:39.108320 (+   354us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=24,key_file_lookups=24,delta_file_lookups=24,mrs_lookups=0
0709 13:18:39.108391 (+    71us) write_op.cc:312] APPLY: finished
0709 13:18:39.108807 (+   416us) log.cc:844] Serialized 113 byte log entry
0709 13:18:39.163856 (+ 55049us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:39.336051 (+172195us) write_op.cc:454] Released schema lock
0709 13:18:39.390210 (+ 54159us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:39.451925 (+ 61715us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:39.478939 (+ 27014us) write_op.cc:454] Released schema lock
Metrics: {"dns_us":223,"mutex_wait_us":8034512,"reported_tablets":31,"spinlock_wait_cycles":487168,"child_traces":[["op",{"apply.queue_time_us":197208,"apply.run_cpu_time_us":28650,"apply.run_wall_time_us":698439,"cfile_cache_hit":50,"cfile_cache_hit_bytes":54311,"num_ops":12,"prepare.queue_time_us":172625,"prepare.run_cpu_time_us":40372,"prepare.run_wall_time_us":355547,"raft.queue_time_us":963571,"raft.run_cpu_time_us":38702,"raft.run_wall_time_us":175566,"replication_time_us":1342732,"spinlock_wait_cycles":1408,"thread_start_us":689942,"threads_started":4,"wal-append.queue_time_us":208597}]]}
I20260709 13:18:40.058917 16645 raft_consensus.cc:2804] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:40.100343 10766 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:40.000310 16907 consensus_queue.cc:237] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:40.071189 16242 raft_consensus.cc:2749] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:40.000321 16949 raft_consensus.cc:740] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:18:40.120131 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 48925.1us
I20260709 13:18:40.114706 16892 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:40.133905 10842 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:40.138429 10766 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:40.139761 10766 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:40.107328 16937 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:18:40.146860 10766 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:40.152027 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 295955us
I20260709 13:18:40.120048 16645 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
W20260709 13:18:40.156971 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 55739.6us
I20260709 13:18:40.157015 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:40.159312 10875 raft_consensus.cc:2393] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 20: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:40.131851 16632 raft_consensus.cc:2749] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:40.113437 16349 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:40.137151 16933 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:40.142391 16892 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:40.163160 10766 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:40.187150 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:40.187245 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:40.148267 16709 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:18:40.144109 16932 raft_consensus.cc:2804] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:40.148658 16557 raft_consensus.cc:2749] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:40.131295 16494 consensus_queue.cc:1059] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:18:40.209403 16494 raft_consensus.cc:3055] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:40.209820 16494 raft_consensus.cc:740] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:40.205451 10941 raft_consensus.cc:1275] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:18:40.172463 16815 raft_consensus.cc:2749] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:40.216308 10872 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 12.
I20260709 13:18:40.153811 16937 raft_consensus.cc:697] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 18 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:40.167940 16645 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:40.168952 10799 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 18.
I20260709 13:18:40.131114 16985 consensus_queue.cc:1059] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.060s
W20260709 13:18:40.221750 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 120608us
I20260709 13:18:40.243242 11477 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:18:40.245661 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 57832.8us
I20260709 13:18:40.246893 10916 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:40.188920 16950 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:40.250316 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:40.191586 16969 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.217561 16709 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:40.220396 16821 consensus_meta.cc:287] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.536s	user 0.009s	sys 0.001s
I20260709 13:18:40.217247 16932 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:40.136286 16738 consensus_queue.cc:237] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:40.204232 16523 raft_consensus.cc:2749] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:40.218266 16917 raft_consensus.cc:2804] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:40.214541 16866 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:40.242702 16961 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.218582 16491 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:40.320915 17011 raft_consensus.cc:3055] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 13:18:40.321238 17000 log.cc:927] Time spent T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 13:18:40.321424 17011 raft_consensus.cc:740] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:40.310608 16709 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:40.306481 16242 raft_consensus.cc:2749] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:40.258046 11508 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650"
dest_uuid: "12682eb70bc84c76879e04a8628ee586"
mode: GRACEFUL
new_leader_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:57818
I20260709 13:18:40.309671 16932 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:40.309979 16950 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.307138 16996 raft_consensus.cc:2804] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:18:40.316545 16917 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:40.203722 16900 consensus_queue.cc:260] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:40.332717 16937 consensus_queue.cc:237] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:40.334579 16349 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:40.336431 16930 consensus_meta.cc:287] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.605s	user 0.004s	sys 0.005s
I20260709 13:18:40.337812 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:40.311410 16987 consensus_queue.cc:237] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:40.311522 16821 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:40.314460 16992 consensus_queue.cc:1059] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 19.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.221s
I20260709 13:18:40.345333 11507 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:18:40.351296 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 104520us
I20260709 13:18:40.346382 16900 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:40.150606 16974 consensus_queue.cc:260] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:40.352437 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:40.324123 16961 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:40.322387 17011 consensus_queue.cc:260] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:40.355428 17011 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:18:40.356293 16961 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:40.358057 16491 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:40.362670 16645 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:40.364197 16821 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:40.364486 16815 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:40.364902 16645 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:40.365080 16815 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:40.367105 16491 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.367723 16491 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:40.369783 16491 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:40.340739 16917 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:40.371699 10766 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:40.312372 16969 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:40.384109 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:40.384776 11506 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:40.384850 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:18:40.385321 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 251539us
I20260709 13:18:40.386197 10842 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:40.386896 11504 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:40.395358 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:40.397459 10842 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:40.399418 10842 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:40.403379 11509 raft_consensus.cc:2393] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 20: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:40.402824 10842 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:40.405133 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:40.405922 11503 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 12.
I20260709 13:18:40.406863 10842 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:40.346861 16930 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:40.380003 16996 raft_consensus.cc:697] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 18 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:40.416345 16645 raft_consensus.cc:2749] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:18:40.401899 16821 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:40.421834 11506 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 16.
I20260709 13:18:40.421761 11504 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 17.
I20260709 13:18:40.393824 16992 raft_consensus.cc:2749] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:40.391801 16557 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:18:40.394356 16974 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:40.341473 16950 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:40.433329 10915 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:40.427551 10841 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:40.406755 16632 raft_consensus.cc:2749] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:18:40.440860 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 88796.2us
I20260709 13:18:40.408538 16523 raft_consensus.cc:2749] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:40.444964 11503 raft_consensus.cc:1275] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:18:40.440730 16815 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:40.462139 10916 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:40.464648 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:40.451924 16866 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:40.471047 16993 log.cc:927] Time spent T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.234s	user 0.000s	sys 0.001s
I20260709 13:18:40.456374 16892 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:40.472559 16930 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:40.473377 16892 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:40.453881 16950 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:40.476382 10916 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:40.477253 16491 raft_consensus.cc:2749] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:40.477874 16491 raft_consensus.cc:2749] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:18:40.477905 16917 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:40.439004 10841 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:40.439813 16998 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:18:40.444870 17004 log.cc:927] Time spent T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
W20260709 13:18:40.445310 17003 log.cc:927] Time spent T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
I20260709 13:18:40.446372 16996 consensus_queue.cc:237] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:40.447431 17017 raft_consensus.cc:2804] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:40.479535 10916 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:40.496236 11510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56410 (request call id 957) took 1137 ms. Trace:
I20260709 13:18:40.443037 10873 raft_consensus.cc:1240] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
W20260709 13:18:40.492899 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 54025.9us
W20260709 13:18:40.491762 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 84999.5us
I20260709 13:18:40.401108 16929 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:40.495450 11507 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 16.
I20260709 13:18:40.496693 11510 rpcz_store.cc:276] 0709 13:18:39.358663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:39.358764 (+   101us) service_pool.cc:224] Handling call
0709 13:18:39.359065 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:39.868236 (+509171us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:40.495989 (+627753us) spinlock_profiling.cc:243] Waited 504 ms on lock 0x7b54005f8a54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:40.496171 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1160344320}
I20260709 13:18:40.499380 11508 raft_consensus.cc:606] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 18 LEADER]: Received request to transfer leadership to TS cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:40.497491 10842 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:40.508699 16930 consensus_queue.cc:1048] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:18:40.511715 17019 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:40.512428 17019 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:40.512884 17019 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:40.516671 10916 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:40.498064 17017 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:40.519712 16491 raft_consensus.cc:2749] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:40.520222 17017 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:40.520653 16491 raft_consensus.cc:2749] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:40.500681 16998 raft_consensus.cc:697] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 17 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:40.524041 16998 consensus_queue.cc:237] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:40.450762 16932 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:40.528945 16932 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:40.496424 16866 raft_consensus.cc:2804] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:40.518368 10916 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:40.533876 16655 consensus_queue.cc:1059] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:18:40.535799 16907 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:40.536355 16929 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:40.536419 16907 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:40.537197 16907 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:18:40.537786 17002 log.cc:927] Time spent T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.228s	user 0.000s	sys 0.002s
W20260709 13:18:40.538233 17006 log.cc:927] Time spent T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.185s	user 0.001s	sys 0.000s
I20260709 13:18:40.540014 16655 raft_consensus.cc:3055] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:18:40.534595 16978 log.cc:927] Time spent T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.655s	user 0.006s	sys 0.007s
I20260709 13:18:40.547775 10916 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:40.548933 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 47229.4us
I20260709 13:18:40.550853 10842 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:40.552503 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 168287us
I20260709 13:18:40.544826 16561 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:40.576078 16821 raft_consensus.cc:2749] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:40.575845 16930 consensus_queue.cc:1048] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:40.588196 16917 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:40.588924 16709 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:40.590530 16709 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:40.591326 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:40.591779 10792 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:40.563262 16929 consensus_queue.cc:1059] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 13:18:40.596151 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:40.596725 10800 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:40.597679 10792 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 17.
I20260709 13:18:40.598919 17009 raft_consensus.cc:2749] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:18:40.604540 17019 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:40.605156 10800 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
I20260709 13:18:40.566565 16907 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:40.609215 16907 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:18:40.609906 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 57548.2us
I20260709 13:18:40.567144 16617 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:40.612167 17009 consensus_queue.cc:1059] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:40.613559 17009 consensus_queue.cc:1048] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:40.541786 16866 raft_consensus.cc:697] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:40.615680 17009 raft_consensus.cc:3055] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:18:40.616209 17009 raft_consensus.cc:740] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:40.616353 16866 consensus_queue.cc:237] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:40.617540 17009 consensus_queue.cc:260] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:40.616127 10798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2889) took 1607 ms. Trace:
I20260709 13:18:40.621903 10798 rpcz_store.cc:276] 0709 13:18:39.008757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:39.154706 (+145949us) service_pool.cc:224] Handling call
0709 13:18:40.615922 (+1461216us) spinlock_profiling.cc:243] Waited 472 ms on lock 0x7b5400387a54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:40.616061 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1085690112}
I20260709 13:18:40.540405 16655 raft_consensus.cc:740] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:40.624970 10792 raft_consensus.cc:1240] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:40.627076 10799 raft_consensus.cc:1275] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:18:40.630031 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:40.630445 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:40.631042 10943 raft_consensus.cc:2393] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 18: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:40.635835 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:40.640856 17015 consensus_queue.cc:1048] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.076s
I20260709 13:18:40.651396 10800 raft_consensus.cc:1240] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 18. Current term is 19. Ops: []
I20260709 13:18:40.638995 10796 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 17.
I20260709 13:18:40.669910 10798 raft_consensus.cc:1275] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:18:40.671542 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:40.672498 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:40.672744 10797 raft_consensus.cc:2393] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 16: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:40.673153 10794 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
I20260709 13:18:40.671663 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:40.675588 10795 raft_consensus.cc:2393] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 13: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:40.677719 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:40.678505 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:40.693832 10795 raft_consensus.cc:2393] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 20: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:40.665146 16911 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:40.695072 10800 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2778f463396643809ca1f6bef2755afc"
dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
mode: GRACEFUL
new_leader_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:40148
I20260709 13:18:40.699092 10800 raft_consensus.cc:606] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Received request to transfer leadership to TS cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:40.699704 10800 raft_consensus.cc:614] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:40.701207 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:40.712601 10945 raft_consensus.cc:1275] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:18:40.678983 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:40.721369 10798 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 16.
I20260709 13:18:40.819298 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:40.820004 10795 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:40.697818 16974 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.820917 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:40.822396 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:40.825572 10870 raft_consensus.cc:2393] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 18: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:40.826298 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:40.836596 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:40.837581 10875 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:40.836678 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:40.840190 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:40.841442 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:40.842228 10870 raft_consensus.cc:2393] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 16: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:40.842819 10916 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:40.719298 17019 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:40.843583 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:40.844365 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:40.844247 10916 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:40.845348 10798 raft_consensus.cc:2393] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 16: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:40.845911 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:40.846745 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:40.852155 10916 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:40.820240 16900 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.853874 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:40.845333 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:40.862499 10876 raft_consensus.cc:1275] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:18:40.863098 10842 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:40.714587 16911 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:40.694465 16655 consensus_queue.cc:260] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:40.875064 10795 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 16.
I20260709 13:18:40.848158 16617 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:40.854288 16907 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:18:40.854712 16491 raft_consensus.cc:2749] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:18:40.881963 16998 raft_consensus.cc:2749] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:18:40.886209 17011 consensus_queue.cc:1048] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:40.837685 16974 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:40.698585 17017 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:40.869053 16738 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.903252 16871 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:40.872474 16900 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:40.880286 17019 consensus_queue.cc:1059] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 13:18:40.925725 10867 raft_consensus.cc:2393] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 13: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:40.912781 16996 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:40.935555 11508 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4"
dest_uuid: "12682eb70bc84c76879e04a8628ee586"
mode: GRACEFUL
new_leader_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:57818
I20260709 13:18:40.935431 16996 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:40.940622 16900 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:40.942198 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:40.945420 11508 raft_consensus.cc:606] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Received request to transfer leadership to TS cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:40.946027 11508 raft_consensus.cc:614] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:18:40.948144 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 92058.5us
I20260709 13:18:40.924779 16871 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:40.951107 10916 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:40.952662 10916 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:18:40.957944 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 96999.2us
I20260709 13:18:40.916841 16738 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:40.962848 10916 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:40.974869 16845 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:40.979606 16845 raft_consensus.cc:697] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:40.980502 16845 consensus_queue.cc:237] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:40.980974 17015 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:40.981765 17015 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:40.957944 16871 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:40.959772 16998 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:40.982944 16907 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:40.987072 16996 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:40.975157 16491 consensus_queue.cc:1048] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
W20260709 13:18:40.997948 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 119537us
I20260709 13:18:40.976655 17026 raft_consensus.cc:2804] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:41.003222 10842 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:40.978615 16912 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:41.008330 10842 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:41.013801 10767 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:41.014103 10842 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:41.015772 10767 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:40.999687 16998 raft_consensus.cc:697] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 17 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:41.003954 17026 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:41.019760 16821 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:41.022541 17026 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:41.023370 16821 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:41.024806 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:41.025568 16821 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:41.026042 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:41.026628 10801 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:40.999769 16907 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:41.030928 10685 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 171abdebfc4142cda86de477a5c89a7e with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER failed with leader 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 12682eb70bc84c76879e04a8628ee586 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 16 committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:18:41.031334 17026 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:41.032724 16907 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:41.033740 17026 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:41.017817 16523 raft_consensus.cc:2749] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:18:41.038762 10801 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 19.
I20260709 13:18:41.039837 16974 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:41.042840 10799 raft_consensus.cc:1240] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 13:18:41.044329 10799 raft_consensus.cc:1240] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 18. Current term is 19. Ops: [16.10-18.11]
I20260709 13:18:41.045502 10799 raft_consensus.cc:1275] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:18:41.017614 17019 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:41.047454 17019 raft_consensus.cc:697] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:41.048429 17019 consensus_queue.cc:237] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:41.051108 16911 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:41.046670 10799 raft_consensus.cc:1275] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:18:41.052691 16738 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:41.063675 16949 consensus_queue.cc:260] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:41.068564 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:41.069371 11510 raft_consensus.cc:2393] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 15: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:41.069167 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:41.014820 16845 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:41.073074 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:41.072002 10916 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:41.074007 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:41.017258 17033 raft_consensus.cc:2804] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:41.076750 10799 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:41.015584 16912 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:41.065124 16949 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:41.081598 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:41.083647 11503 raft_consensus.cc:2393] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 16: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:41.083791 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:41.085114 11509 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
I20260709 13:18:41.086676 17017 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:41.078217 17033 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:41.079660 16491 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:18:41.085316 17026 consensus_queue.cc:1048] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 13:18:41.100709 10841 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:41.080449 16907 consensus_queue.cc:1048] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:18:41.106199 10875 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 17.
I20260709 13:18:41.105549 17015 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:41.113153 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:41.114912 10941 raft_consensus.cc:2393] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 22: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:41.114943 10799 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 18.
I20260709 13:18:41.116494 10916 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:41.078445 16845 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:41.118409 10841 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:41.124708 10841 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:41.106909 17033 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:41.113703 17017 raft_consensus.cc:2749] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:18:41.125764 16985 raft_consensus.cc:2749] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:18:41.129030 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:41.127350 17030 raft_consensus.cc:2804] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:18:41.116606 16491 raft_consensus.cc:697] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 19 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:41.131697 11509 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:41.134375 17030 raft_consensus.cc:697] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 18 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:41.135964 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:41.143831 10941 raft_consensus.cc:2393] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 15: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:41.145654 17030 consensus_queue.cc:237] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:41.146912 16987 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.789s	user 0.012s	sys 0.016s
W20260709 13:18:41.147290 16987 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.790s	user 0.012s	sys 0.016s
I20260709 13:18:41.136758 16845 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:41.129506 16974 raft_consensus.cc:2804] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:41.142508 17026 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:41.144582 16912 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:41.155186 17026 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:41.119580 16998 consensus_queue.cc:237] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:41.151010 16974 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:41.168812 16871 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:41.170208 10869 raft_consensus.cc:1275] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:18:41.171532 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:41.171343 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:41.173503 16974 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:41.173841 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:41.181516 10841 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:41.181934 10948 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 14.
I20260709 13:18:41.179618 10942 raft_consensus.cc:1275] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:18:41.185967 11509 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
I20260709 13:18:41.189618 10946 raft_consensus.cc:1275] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:18:41.194279 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:41.195323 10946 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 15.
I20260709 13:18:41.169559 16845 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:41.224355 10841 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:41.225354 17032 consensus_queue.cc:1059] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:41.227119 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:41.228012 10945 raft_consensus.cc:2410] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 18 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:18:41.227937 17032 raft_consensus.cc:2804] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:41.228578 17032 raft_consensus.cc:697] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:41.229000 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:41.229393 17032 consensus_queue.cc:237] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:41.230976 16985 raft_consensus.cc:3055] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:41.231359 16985 raft_consensus.cc:740] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:41.232213 16985 consensus_queue.cc:260] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:41.233234 10942 raft_consensus.cc:1240] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:41.234247 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:41.227137 16491 consensus_queue.cc:237] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:41.192090 17017 consensus_queue.cc:1059] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 13:18:41.235319 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:41.235680 17017 raft_consensus.cc:3055] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:18:41.236095 17017 raft_consensus.cc:740] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:41.238284 10946 raft_consensus.cc:1275] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:18:41.239440 10946 raft_consensus.cc:1240] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:41.260804 10943 raft_consensus.cc:1240] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 19. Current term is 20. Ops: []
I20260709 13:18:41.191282 16949 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:18:41.264297 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:41.267130 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:41.267962 10948 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:41.265202 10947 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 17.
I20260709 13:18:41.274138 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:41.278662 16871 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:41.236950 17017 consensus_queue.cc:260] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:41.287621 17017 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:41.290731 10841 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:41.292593 10841 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:41.192842 16987 consensus_queue.cc:1059] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.102s
I20260709 13:18:41.296485 17026 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:41.258891 16985 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:41.307265 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:41.309296 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:41.313988 16934 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.320001 11478 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:41.322299 10945 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
I20260709 13:18:41.306318 17019 raft_consensus.cc:2749] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:18:41.325984 11477 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:41.258769 16845 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:41.305827 17026 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.309468 16645 raft_consensus.cc:2749] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:18:41.329857 10948 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 16.
I20260709 13:18:41.305984 16523 consensus_queue.cc:1059] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:18:41.323961 16900 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.339597 16992 raft_consensus.cc:3055] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:41.340108 16992 raft_consensus.cc:740] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:41.343217 10872 raft_consensus.cc:1240] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 17. Current term is 18. Ops: [15.11-17.12]
I20260709 13:18:41.328222 17026 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:41.343705 10870 raft_consensus.cc:1240] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:41.345180 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:41.348335 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:41.349859 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:41.326102 16821 consensus_queue.cc:1048] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.016s
I20260709 13:18:41.329056 16839 consensus_queue.cc:1048] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.018s
I20260709 13:18:41.351351 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:41.344080 16523 consensus_queue.cc:1243] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer cb0428cc55984daba1ba63744a5102a2 log is divergent from this leader: its last log entry 16.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 13:18:41.347467 17027 log.cc:927] Time spent T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.392s	user 0.004s	sys 0.001s
I20260709 13:18:41.351794 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:41.352602 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:41.343858 16738 raft_consensus.cc:2749] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:41.314568 16934 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:41.366930 10875 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 17.
W20260709 13:18:41.365995 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 350294us
I20260709 13:18:41.368068 10767 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:41.347488 16996 raft_consensus.cc:2804] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:41.366030 10872 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:18:41.369093 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:41.341043 16992 consensus_queue.cc:260] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:41.325790 16997 consensus_queue.cc:1048] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 13:18:41.372354 10767 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:41.359181 10868 raft_consensus.cc:1240] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:41.373606 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:41.438897 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:41.438490 10766 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:18:41.440340 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:41.440805 10799 raft_consensus.cc:1240] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:41.375658 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:41.468475 10800 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:41.470331 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:41.471161 10794 raft_consensus.cc:2393] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 13: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:41.399762 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:41.378686 10944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 2514) took 1180 ms. Trace:
I20260709 13:18:41.375283 10801 raft_consensus.cc:1240] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:41.346966 16998 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.344182 16900 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:41.473623 10944 rpcz_store.cc:276] 0709 13:18:40.198128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:40.198270 (+   142us) service_pool.cc:224] Handling call
0709 13:18:40.212488 (+ 14218us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:41.041750 (+829262us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:18:41.107200 (+ 65450us) log.cc:844] Serialized 14 byte log entry
0709 13:18:41.221315 (+114115us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:41.221401 (+    86us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:41.221463 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:41.378477 (+157014us) spinlock_profiling.cc:243] Waited 829 ms on lock 0x7b54001e1254. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:41.378523 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:41.378632 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":53079,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":507,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":111,"spinlock_wait_cycles":1906554752,"thread_start_us":178816,"threads_started":2,"wal-append.queue_time_us":114152}
I20260709 13:18:41.485050 10800 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
I20260709 13:18:41.485930 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:41.489768 10869 raft_consensus.cc:2376] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 17. Re-sending same reply.
I20260709 13:18:41.370945 16951 raft_consensus.cc:2749] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:41.368185 16709 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:41.486927 11510 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884"
dest_uuid: "12682eb70bc84c76879e04a8628ee586"
mode: GRACEFUL
new_leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
 from {username='slave'} at 127.0.0.1:57818
I20260709 13:18:41.491118 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:41.509451 11510 raft_consensus.cc:606] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 18 LEADER]: Received request to transfer leadership to TS 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:41.510179 10868 raft_consensus.cc:2410] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:18:41.491550 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:41.491624 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:41.371106 16996 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:41.511350 10801 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 15.
I20260709 13:18:41.449218 17023 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:41.448027 16821 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.519940 10872 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 17.
I20260709 13:18:41.492321 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:41.521157 10794 raft_consensus.cc:1240] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 18. Current term is 19. Ops: [18.9-18.9]
I20260709 13:18:41.509428 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:41.526232 10796 raft_consensus.cc:2393] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 19: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
W20260709 13:18:41.488162 17027 log.cc:927] Time spent T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:18:41.521586 10870 raft_consensus.cc:2393] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 20: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:41.529317 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:41.449477 16375 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:41.531157 16375 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:41.530735 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:41.536792 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:41.537847 16375 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:41.540876 16375 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:41.541945 16375 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:41.546876 16375 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:41.550706 16242 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:41.551358 16242 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:41.553580 16242 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:41.554910 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:41.555565 10869 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:41.558291 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:18:41.491047 17048 log.cc:927] Time spent T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 13:18:41.560931 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:41.492367 16917 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:41.515906 16932 raft_consensus.cc:2749] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:41.491806 16523 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:41.572750 11512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56416 (request call id 954) took 1896 ms. Trace:
I20260709 13:18:41.571264 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:41.521870 16996 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:41.576088 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:41.525429 17023 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:18:41.581993 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 143586us
I20260709 13:18:41.584064 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:41.584889 10944 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 14.
I20260709 13:18:41.575021 11512 rpcz_store.cc:276] 0709 13:18:39.676188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:39.676302 (+   114us) service_pool.cc:224] Handling call
0709 13:18:39.696520 (+ 20218us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:39.763270 (+ 66750us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:18:39.910588 (+147318us) log.cc:844] Serialized 14 byte log entry
0709 13:18:40.100072 (+189484us) log.cc:844] Serialized 14 byte log entry
0709 13:18:40.254598 (+154526us) log.cc:844] Serialized 14 byte log entry
0709 13:18:40.308450 (+ 53852us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:40.308507 (+    57us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:18:40.308566 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:40.537097 (+228531us) spinlock_profiling.cc:243] Waited 5.34 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:18:41.080568 (+543471us) spinlock_profiling.cc:243] Waited 7.41 ms on lock 0x7b180033e440. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc8522 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:18:41.492657 (+412089us) spinlock_profiling.cc:243] Waited 3.21 ms on lock 0x7b180033e440. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc8522 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:18:41.572446 (+ 79789us) spinlock_profiling.cc:243] Waited 66.5 ms on lock 0x7b5400467654. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:41.572513 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:41.572674 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":72878,"prepare.queue_time_us":251840,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":5392,"raft.queue_time_us":113352,"raft.run_cpu_time_us":454,"raft.run_wall_time_us":16400,"spinlock_wait_cycles":189634432,"thread_start_us":476083,"threads_started":3,"wal-append.queue_time_us":329022,"wal-append.run_cpu_time_us":4982,"wal-append.run_wall_time_us":1294673}
I20260709 13:18:41.588955 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:41.501982 16998 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:41.576989 16992 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:41.591572 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:41.595011 17057 consensus_queue.cc:1059] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:18:41.596364 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 224047us
I20260709 13:18:41.575591 16917 raft_consensus.cc:697] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:41.604609 10767 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:41.561774 16821 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:41.611364 10868 raft_consensus.cc:1275] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
I20260709 13:18:41.611918 10868 pending_rounds.cc:85] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 10
I20260709 13:18:41.612183 10868 pending_rounds.cc:107] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 13:18:41.612468 10868 raft_consensus.cc:2889] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:41.581856 17023 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:41.523911 16494 consensus_queue.cc:260] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:41.605633 10872 raft_consensus.cc:1275] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:18:41.623937 10767 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:41.579849 16523 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:41.628327 11512 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd"
dest_uuid: "12682eb70bc84c76879e04a8628ee586"
mode: GRACEFUL
new_leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
 from {username='slave'} at 127.0.0.1:57818
W20260709 13:18:41.632187 17050 log.cc:927] Time spent T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 13:18:41.632488 16929 raft_consensus.cc:3055] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:18:41.633020 16929 raft_consensus.cc:740] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:41.633934 16929 consensus_queue.cc:260] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:41.639420 17049 log.cc:927] Time spent T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:18:41.641175 17055 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:41.644793 17026 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:41.648991 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 70923.1us
I20260709 13:18:41.646382 17055 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:18:41.681033 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 360898us
I20260709 13:18:41.626941 16987 consensus_queue.cc:1059] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
W20260709 13:18:41.687929 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 363878us
I20260709 13:18:41.688002 16913 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.748644 16913 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:41.749127 10841 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
W20260709 13:18:41.716898 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 383592us
W20260709 13:18:41.729586 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:41.743656 17060 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.651217 16917 consensus_queue.cc:237] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:41.649725 10872 raft_consensus.cc:1275] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
W20260709 13:18:41.746155 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 122319us
W20260709 13:18:41.741818 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 91962us
I20260709 13:18:41.761394 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:41.745046 10869 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 18.
I20260709 13:18:41.754695 10950 raft_consensus.cc:1275] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:18:41.762735 11510 raft_consensus.cc:2393] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 16: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:41.764638 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:41.763828 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:41.761564 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:18:41.746387 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 129443us
I20260709 13:18:41.767638 11506 raft_consensus.cc:2410] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:18:41.762538 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:41.769754 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:41.773294 11504 raft_consensus.cc:2393] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 20: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:41.778434 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:41.786834 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:41.787535 10942 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 15.
I20260709 13:18:41.788365 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:41.788972 10942 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 18.
I20260709 13:18:41.756940 17063 raft_consensus.cc:2804] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:41.789603 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:41.790385 11507 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 14.
I20260709 13:18:41.791359 10950 raft_consensus.cc:1275] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:18:41.791558 11508 raft_consensus.cc:2393] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 13: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:41.791735 10767 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:41.777058 11478 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:41.795130 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:41.747046 17055 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:41.802697 11509 raft_consensus.cc:1275] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:18:41.806854 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:18:41.808614 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 46071.1us
I20260709 13:18:41.808990 17066 consensus_queue.cc:1059] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:18:41.809636 10842 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:41.814942 16997 raft_consensus.cc:2764] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader election decision vote started in defunct term 12: won
I20260709 13:18:41.758100 16998 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:41.805424 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:41.756623 16900 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:41.819039 10950 raft_consensus.cc:1240] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:41.757853 16821 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:41.773401 16845 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:41.790269 16934 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:41.822907 11477 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:41.752139 17056 raft_consensus.cc:2749] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:18:41.846827 17012 log.cc:927] Time spent T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 1.484s	user 0.000s	sys 0.003s
I20260709 13:18:41.817127 17063 raft_consensus.cc:697] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:41.777940 16932 consensus_queue.cc:1059] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 13:18:41.756901 17060 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:41.850239 10869 raft_consensus.cc:1275] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:18:41.825474 16951 raft_consensus.cc:2749] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:41.792851 16816 consensus_queue.cc:1048] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 13:18:41.802377 16912 consensus_queue.cc:1048] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:18:41.837427 16998 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.842110 16821 raft_consensus.cc:2749] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:41.863022 17063 consensus_queue.cc:237] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:41.864337 17056 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:41.864939 17056 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:41.865339 16985 consensus_meta.cc:287] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.558s	user 0.007s	sys 0.003s
I20260709 13:18:41.865476 16996 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:41.866854 11512 raft_consensus.cc:606] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Received request to transfer leadership to TS 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:41.866950 16523 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:41.867302 11512 raft_consensus.cc:614] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:41.867076 16913 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:41.871958 17025 log.cc:927] Time spent T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.913s	user 0.002s	sys 0.001s
I20260709 13:18:41.872772 17056 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:41.875032 10874 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 17.
I20260709 13:18:41.877091 17023 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:41.880029 17023 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:41.851981 16997 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:18:41.912459 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 114933us
W20260709 13:18:41.914186 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 136804us
W20260709 13:18:41.906423 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 88166.6us
W20260709 13:18:41.921455 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 117042us
I20260709 13:18:41.925751 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:41.926505 10869 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 15.
I20260709 13:18:41.930419 10767 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:41.883827 16969 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:41.908064 16997 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:41.933334 17005 log.cc:927] Time spent T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 1.675s	user 0.005s	sys 0.007s
I20260709 13:18:41.934612 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:41.853984 16738 consensus_queue.cc:1048] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.064s
W20260709 13:18:41.946648 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 137096us
I20260709 13:18:41.947609 11478 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:41.951125 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:41.904891 16998 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:41.964973 10842 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:41.963406 10766 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:41.991976 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 61657.8us
I20260709 13:18:41.974450 11478 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:41.996380 10869 raft_consensus.cc:1240] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:41.952581 16655 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:41.957051 16997 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:42.010571 10767 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:41.941537 16951 consensus_queue.cc:1048] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
W20260709 13:18:41.994227 17017 consensus_meta.cc:287] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.706s	user 0.015s	sys 0.006s
I20260709 13:18:41.973006 16998 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:42.020661 16996 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:42.021163 16871 raft_consensus.cc:3055] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:42.021700 16871 raft_consensus.cc:740] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:41.956454 16821 consensus_queue.cc:1059] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 13:18:41.949553 17019 consensus_queue.cc:1048] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 13:18:42.013351 10799 raft_consensus.cc:1275] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 13:18:42.001096 16974 consensus_meta.cc:287] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.801s	user 0.013s	sys 0.006s
I20260709 13:18:41.998958 16934 raft_consensus.cc:2749] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:42.017625 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:42.019337 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:42.034967 10794 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 13.
W20260709 13:18:42.007921 17033 consensus_meta.cc:287] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.870s	user 0.011s	sys 0.009s
I20260709 13:18:42.013336 10798 raft_consensus.cc:1275] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:18:42.028085 10767 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:42.024916 10870 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 18.
I20260709 13:18:42.008596 17023 raft_consensus.cc:2749] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:42.040602 11478 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:18:42.013191 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 168051us
I20260709 13:18:42.017012 16937 raft_consensus.cc:2749] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:41.957304 17079 consensus_queue.cc:1059] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.083s
I20260709 13:18:42.045059 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:42.012981 17075 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:41.998652 16839 consensus_queue.cc:1048] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 13:18:42.032099 16985 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:18:42.141572 17005 log.cc:927] Time spent T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 13:18:42.149904 11507 raft_consensus.cc:1275] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 13:18:42.150492 10685 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 171abdebfc4142cda86de477a5c89a7e with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER failed with leader 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 12682eb70bc84c76879e04a8628ee586 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 16 committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:18:42.143615 17023 raft_consensus.cc:2749] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:42.042690 17017 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:42.036715 16917 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.212005 16917 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:42.213052 16929 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:42.033267 17060 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:42.158156 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:42.215476 11509 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
I20260709 13:18:42.215806 16985 raft_consensus.cc:697] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:18:42.149716 16992 consensus_meta.cc:287] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.540s	user 0.007s	sys 0.001s
I20260709 13:18:42.216063 10793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2264) took 2536 ms. Trace:
W20260709 13:18:42.157301 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 129362us
I20260709 13:18:42.216770 16985 consensus_queue.cc:237] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:42.218585 16917 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:42.042176 17033 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:42.219161 10767 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:42.216548 10793 rpcz_store.cc:276] 0709 13:18:39.679858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:39.742481 (+ 62623us) service_pool.cc:224] Handling call
0709 13:18:39.747362 (+  4881us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:39.794158 (+ 46796us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:39.794192 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:39.794698 (+   506us) log.cc:844] Serialized 26 byte log entry
0709 13:18:40.127127 (+332429us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:40.150147 (+ 23020us) log.cc:844] Serialized 14 byte log entry
0709 13:18:40.158429 (+  8282us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:40.158531 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:41.116414 (+957883us) spinlock_profiling.cc:243] Waited 30.8 ms on lock 0x7b18001aa040. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc8522 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:18:41.921913 (+805499us) spinlock_profiling.cc:243] Waited 41.9 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:18:42.215799 (+293886us) raft_consensus.cc:1685] finished
0709 13:18:42.215823 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:42.215968 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":249787,"prepare.queue_time_us":22716,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":63986,"raft.run_cpu_time_us":293,"raft.run_wall_time_us":37694,"spinlock_wait_cycles":167103104,"thread_start_us":331912,"threads_started":1,"wal-append.queue_time_us":362194}
I20260709 13:18:42.220939 17033 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:42.034297 16974 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:42.223053 10868 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 17.
I20260709 13:18:42.223366 16974 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:42.224032 10876 raft_consensus.cc:1240] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:42.224797 10869 raft_consensus.cc:1240] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:42.225879 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:42.157359 10941 raft_consensus.cc:1275] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:18:42.227614 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:42.228389 10941 raft_consensus.cc:2410] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 19 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 13:18:42.229234 17055 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:42.160735 17082 raft_consensus.cc:2749] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:42.161375 17066 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.231290 17066 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:42.170640 16997 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:42.233601 17066 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:42.161451 17069 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.234092 16997 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:42.234300 17052 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:42.234396 17069 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:42.234961 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:42.235184 17052 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:42.235644 10870 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 21.
I20260709 13:18:42.236534 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:42.162361 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:42.163249 17073 consensus_queue.cc:1048] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:42.237113 10870 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 21.
I20260709 13:18:42.164378 10842 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:18:42.240259 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 75973.6us
I20260709 13:18:42.240532 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:42.241195 10870 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 15.
I20260709 13:18:42.039827 17032 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.242218 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:42.242144 17032 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:42.243393 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:42.244377 17032 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:42.244942 17032 raft_consensus.cc:2804] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:42.245268 17032 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:42.245591 17032 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:18:42.175997 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 135523us
I20260709 13:18:42.182976 16821 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:42.248131 16821 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:42.249080 17085 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.249627 17085 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:42.250273 16821 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:42.023218 17070 consensus_queue.cc:1048] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.135s
I20260709 13:18:42.251864 16714 consensus_queue.cc:1059] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:42.252755 16714 raft_consensus.cc:3055] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:18:42.253068 16714 raft_consensus.cc:740] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:42.253123 17085 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:42.253997 16714 consensus_queue.cc:260] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:42.188612 16934 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:42.255682 16934 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:42.256063 16934 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:42.208866 17076 raft_consensus.cc:3055] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:18:42.259459 17076 raft_consensus.cc:740] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:42.260454 17076 consensus_queue.cc:260] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:42.261770 17076 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:42.237187 17052 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:18:42.163293 17086 log.cc:927] Time spent T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 13:18:42.236776 17069 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:42.238826 10947 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:42.238595 16917 consensus_queue.cc:1048] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:42.238240 10767 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:42.285790 11512 raft_consensus.cc:1275] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 13:18:42.286154 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 47991.8us
I20260709 13:18:42.287977 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:42.238278 10869 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 20.
I20260709 13:18:42.262627  9682 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:18:42.208667 17017 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:42.294368 11508 raft_consensus.cc:2368] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 19. Current term is 20.
I20260709 13:18:42.268457 17055 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:42.272151 10916 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:42.282528 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:42.291829  9682 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:42.298753 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 2425) took 1483 ms. Trace:
I20260709 13:18:42.299247 10867 rpcz_store.cc:276] 0709 13:18:40.823242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:40.823698 (+   456us) service_pool.cc:224] Handling call
0709 13:18:42.298580 (+1474882us) spinlock_profiling.cc:243] Waited 73.9 ms on lock 0x7b540009bad4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:42.298695 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":169915264}
I20260709 13:18:42.303158 11507 raft_consensus.cc:1275] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:18:42.321522 10916 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:42.323766 10767 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:42.303807 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:42.325456 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:18:42.306749 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:42.335852 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:42.295077 16714 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:42.304421 17082 consensus_queue.cc:1048] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 13:18:42.339864 17065 consensus_queue.cc:1048] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:18:42.340788 17026 consensus_queue.cc:1059] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:42.341053 10947 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 16.
I20260709 13:18:42.341914 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:42.345562 10767 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:42.276034 16845 consensus_queue.cc:1059] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 13:18:42.346465 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:42.321334 17052 raft_consensus.cc:2749] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:18:42.348958 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:42.366398 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:42.367041 10792 raft_consensus.cc:2393] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 19: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:42.277956 17070 consensus_queue.cc:1059] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.067s
I20260709 13:18:42.356181 10766 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:42.367198 10801 raft_consensus.cc:2393] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 20: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:42.365779 10867 raft_consensus.cc:1275] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:18:42.337419 16617 raft_consensus.cc:2749] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:42.350243 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:42.370060 10793 raft_consensus.cc:2393] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 16: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
W20260709 13:18:42.374083 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:42.386395 10870 raft_consensus.cc:1240] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:18:42.388104 10766 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:42.388266 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:42.334134 16911 consensus_queue.cc:1059] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:18:42.391402 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:42.395371 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:42.395454 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:42.396301 10792 raft_consensus.cc:2368] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for earlier term 12. Current term is 13.
I20260709 13:18:42.343101 17085 consensus_queue.cc:1048] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:18:42.362159 17069 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:42.400609 10792 raft_consensus.cc:1240] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 15. Current term is 16. Ops: []
W20260709 13:18:42.403074 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 64037.9us
I20260709 13:18:42.403560 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:42.403811 10916 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:42.404534 10793 raft_consensus.cc:2393] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 15: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:42.404706 10916 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:42.407454 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:42.390893 17075 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:42.412047 10801 raft_consensus.cc:1275] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:18:42.412683 16871 consensus_queue.cc:260] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:42.419806 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:42.415159 10916 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:42.420102 10793 raft_consensus.cc:1275] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 13:18:42.422803 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 63877.5us
I20260709 13:18:42.397396 16839 raft_consensus.cc:3055] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:42.425631 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:42.418257 16998 raft_consensus.cc:3055] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:42.427845 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:42.435575 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 46935.8us
I20260709 13:18:42.436842 10792 raft_consensus.cc:1275] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:18:42.438509 10766 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:42.441319 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:42.421666 17075 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:42.447386 10766 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:42.447768 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:42.448752 10793 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 17: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:42.430910 17054 raft_consensus.cc:2749] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:18:42.426873 16839 raft_consensus.cc:740] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:42.457477 17017 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:42.455404 16917 raft_consensus.cc:2749] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:18:42.458657 16917 raft_consensus.cc:2749] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:18:42.451455 10766 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:42.460089 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:42.460778 10766 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:42.461548 16917 raft_consensus.cc:2804] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:42.461934 16917 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:42.462283 16917 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:42.454226 10793 raft_consensus.cc:1275] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:18:42.406901 17069 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:42.429194 16998 raft_consensus.cc:740] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:42.491966 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:42.496414 10799 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 19.
I20260709 13:18:42.450877 17092 raft_consensus.cc:2804] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:42.468217 10871 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
I20260709 13:18:42.451941 17065 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:18:42.499282 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:42.500118 10799 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 16.
I20260709 13:18:42.493248 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:42.501263 10795 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 18.
I20260709 13:18:42.450754 17075 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:42.455789 17089 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:42.510282 10795 raft_consensus.cc:1275] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:18:42.458292 16839 consensus_queue.cc:260] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:42.512333 10799 raft_consensus.cc:1240] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 18. Current term is 19. Ops: []
I20260709 13:18:42.518332 17093 raft_consensus.cc:2749] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:18:42.519495 17097 log.cc:927] Time spent T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
I20260709 13:18:42.520555 17106 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:18:42.521031 17106 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:42.521405 17106 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:18:42.531196 10767 connection.cc:256] RPC call timeout handler was delayed by -1.6570304849999502s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1789s for a response
W20260709 13:18:42.531847 10767 connection.cc:256] RPC call timeout handler was delayed by -1.6411658550000539s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18580000000000002s for a response
I20260709 13:18:42.535311 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:42.495998 16998 consensus_queue.cc:260] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:42.498601 16557 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.497581 17085 consensus_queue.cc:1059] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 13:18:42.503785 17065 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:18:42.561867 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:42.561151 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 136734us
I20260709 13:18:42.514567 17082 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.510546 17091 raft_consensus.cc:2749] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:42.573079 10915 leader_election.cc:400] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 19. Current term is 20.
I20260709 13:18:42.517037 17092 raft_consensus.cc:697] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:18:42.575953 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 214166us
W20260709 13:18:42.579703 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:42.576486 10915 leader_election.cc:403] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:42.582558 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:42.585377 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:42.507504 16934 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:42.573560 17065 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:18:42.572424 16839 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:42.585377 10794 raft_consensus.cc:1275] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:18:42.587826 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:42.580644 16911 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:42.591223 16561 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:42.591688 11504 raft_consensus.cc:1275] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 13:18:42.592320 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 92091.2us
I20260709 13:18:42.591666 11478 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:42.580305 17082 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:42.596206 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:42.579066 16816 raft_consensus.cc:2749] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:18:42.600380 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:42.517534 17089 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:42.659812 17032 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:42.661118 17089 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:42.661980 17032 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:42.603812 16561 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:42.662849 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 71256.4us
I20260709 13:18:42.663436 17069 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:42.664274 16561 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:42.664474 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:42.616540 11508 raft_consensus.cc:2368] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 16. Current term is 17.
I20260709 13:18:42.572233 16557 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:42.632296 17098 log.cc:927] Time spent T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.185s	user 0.000s	sys 0.000s
W20260709 13:18:42.633349 17095 log.cc:927] Time spent T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
I20260709 13:18:42.666999 17109 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.633719 16911 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:42.667749 17109 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:42.669718 16911 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:42.670276 16911 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:42.640976 17100 log.cc:927] Time spent T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 13:18:42.584195 17085 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:42.671674 17085 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:42.673853 17085 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:42.673957 16934 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:42.675240 10799 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:42.676025 16917 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:42.666677 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:42.670730 16911 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:42.593256 17104 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:42.644398 10797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2922) took 1270 ms. Trace:
W20260709 13:18:42.689088 10874 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 2265) took 1349 ms (client timeout 1500 ms). Trace:
I20260709 13:18:42.646046 17107 raft_consensus.cc:3055] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:18:42.647650 16494 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:42.649501 10876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 2266) took 1309 ms. Trace:
I20260709 13:18:42.625543 17101 raft_consensus.cc:2804] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:42.694444 17101 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:42.694928 17101 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:42.697037 17026 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:42.697633 17026 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:42.692181 10876 rpcz_store.cc:276] 0709 13:18:41.339685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:41.342237 (+  2552us) service_pool.cc:224] Handling call
0709 13:18:41.342485 (+   248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:42.224440 (+881955us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:42.649205 (+424765us) spinlock_profiling.cc:243] Waited 881 ms on lock 0x7b54001095d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:42.649393 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2026966784}
I20260709 13:18:42.652582 10800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2287) took 1143 ms. Trace:
I20260709 13:18:42.699901 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:42.700740 10876 raft_consensus.cc:2393] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 19: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:42.702116 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:42.688684 10797 rpcz_store.cc:276] 0709 13:18:41.373403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:41.374724 (+  1321us) service_pool.cc:224] Handling call
0709 13:18:41.476203 (+101479us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:41.487693 (+ 11490us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:18:42.269687 (+781994us) log.cc:844] Serialized 14 byte log entry
0709 13:18:42.359239 (+ 89552us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:42.396360 (+ 37121us) log.cc:844] Serialized 26 byte log entry
0709 13:18:42.402051 (+  5691us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:42.402658 (+   607us) log.cc:844] Serialized 14 byte log entry
0709 13:18:42.577815 (+175157us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:42.577895 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:42.643925 (+ 66030us) raft_consensus.cc:1685] finished
0709 13:18:42.643957 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:42.644099 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":796153,"prepare.queue_time_us":797508,"prepare.run_cpu_time_us":120,"prepare.run_wall_time_us":16437,"raft.queue_time_us":85491,"raft.run_cpu_time_us":333,"raft.run_wall_time_us":12812,"thread_start_us":259504,"threads_started":2,"wal-append.queue_time_us":309727}
I20260709 13:18:42.703852 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:42.704154 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:42.604681 17108 raft_consensus.cc:2749] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:42.704986 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:42.558109 16845 consensus_queue.cc:1048] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 13:18:42.658268 17118 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 20
W20260709 13:18:42.707929 10871 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 2428) took 1883 ms (client timeout 1789 ms). Trace:
W20260709 13:18:42.708372 10871 rpcz_store.cc:269] 0709 13:18:40.825465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:40.825609 (+   144us) service_pool.cc:224] Handling call
0709 13:18:42.707745 (+1882136us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b5400077954. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:42.707863 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3258076928}
I20260709 13:18:42.700176 10800 rpcz_store.cc:276] 0709 13:18:41.509493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:41.521278 (+ 11785us) service_pool.cc:224] Handling call
0709 13:18:41.521711 (+   433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:41.521902 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:18:42.281453 (+759551us) log.cc:844] Serialized 14 byte log entry
0709 13:18:42.642446 (+360993us) log.cc:844] Serialized 14 byte log entry
0709 13:18:42.652224 (+  9778us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:42.652275 (+    51us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:18:42.652335 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:42.652408 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:42.652524 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":831762,"prepare.queue_time_us":827823,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":1512,"raft.queue_time_us":24166,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":4175,"thread_start_us":294123,"threads_started":1,"wal-append.queue_time_us":389423}
W20260709 13:18:42.689759 10874 rpcz_store.cc:269] 0709 13:18:41.339404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:41.342154 (+  2750us) service_pool.cc:224] Handling call
0709 13:18:42.688875 (+1346721us) spinlock_profiling.cc:243] Waited 517 ms on lock 0x7b540009a6d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:42.689014 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1189231872}
I20260709 13:18:42.713864 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:42.714329 16617 raft_consensus.cc:3055] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:18:42.714848 16617 raft_consensus.cc:740] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:42.715153 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:42.715943 10874 raft_consensus.cc:2393] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 18: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:42.717437 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:42.717936 10874 raft_consensus.cc:2376] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 21. Re-sending same reply.
I20260709 13:18:42.693236 16934 consensus_queue.cc:1059] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:18:42.715852 16617 consensus_queue.cc:260] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:42.686254 10799 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 20.
I20260709 13:18:42.724104 16917 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:42.724457 16617 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:18:42.724931 17112 log.cc:927] Time spent T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.145s	user 0.002s	sys 0.000s
I20260709 13:18:42.729223 17009 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:42.687608 17109 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:42.704035 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:42.653208 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 80310.1us
I20260709 13:18:42.623799 17092 consensus_queue.cc:237] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:42.686101 17082 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:42.709594 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:42.731938 10876 raft_consensus.cc:2393] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 16: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:42.732111 10915 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:42.740888 10871 raft_consensus.cc:2393] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 18: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:42.706564 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:42.741603 11512 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 19.
I20260709 13:18:42.736977 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:42.701938 17106 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:42.742527 10874 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:42.688458 17070 raft_consensus.cc:2749] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:18:42.742075 10797 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 13.
I20260709 13:18:42.745370 10915 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:42.745236 11478 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:42.747982 17118 raft_consensus.cc:2749] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 19. Current term is 20.
I20260709 13:18:42.744920 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:42.757791 10792 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:18:42.750595 16491 consensus_queue.cc:1048] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:42.751077 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:42.747792 17075 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:42.699925 17026 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:42.692998 17107 raft_consensus.cc:740] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:42.745975 17109 raft_consensus.cc:2749] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:18:42.762982 16934 consensus_queue.cc:1048] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:42.758060 16845 raft_consensus.cc:2749] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:42.765671 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:42.770731 10874 raft_consensus.cc:2410] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:18:42.769719 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:42.772888 10800 raft_consensus.cc:1275] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:18:42.776510 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:42.777437 10941 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 17.
I20260709 13:18:42.778986 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:42.779328 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:42.779984 10871 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 22.
I20260709 13:18:42.780357 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:42.780928 10941 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:42.759739 17076 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:42.732069 17104 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:42.786545 10792 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
I20260709 13:18:42.787914 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:42.791980 11504 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 16.
I20260709 13:18:42.794700 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:18:42.781896 17096 log.cc:927] Time spent T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.384s	user 0.001s	sys 0.001s
I20260709 13:18:42.803903 11510 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:42.801390 10766 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:42.775378 16934 consensus_queue.cc:1048] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 13:18:42.796432 16845 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:42.820309 10871 raft_consensus.cc:1240] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:18:42.821799 10766 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:42.819612 16714 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.838621 17107 consensus_queue.cc:260] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:42.865998 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 44329.2us
I20260709 13:18:42.866501 16714 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:42.866973 10766 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:42.868104 10766 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:42.864257 10841 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:42.873828 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:42.874536 10945 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 14.
I20260709 13:18:42.875299 10942 raft_consensus.cc:1240] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 19 LEADER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 18. Current term is 19. Ops: []
I20260709 13:18:42.859942 17091 raft_consensus.cc:2749] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:42.876830 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:42.877466 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:42.880580 10873 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 19.
I20260709 13:18:42.874744 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:42.858994 17069 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:42.882380 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 192631us
I20260709 13:18:42.903084 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:42.864985 17072 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.904206 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:42.906569 10792 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 18.
I20260709 13:18:42.908134 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:42.908705 10800 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 21.
I20260709 13:18:42.881351 10915 leader_election.cc:400] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 16. Current term is 17.
I20260709 13:18:42.910229 10915 leader_election.cc:403] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:42.911609 10941 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
I20260709 13:18:42.927961 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:42.932245 10941 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:42.953927 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:42.873385 17056 raft_consensus.cc:2749] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:42.859707 17105 consensus_queue.cc:1048] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.034s
W20260709 13:18:42.954989 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:42.955992 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:42.864943 16935 consensus_queue.cc:1048] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:18:42.958920 16911 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:42.903294 17026 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.882300 16929 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:42.860088 16845 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:42.881851 17093 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:42.963104 10947 raft_consensus.cc:1240] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:18:42.963779 11510 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
I20260709 13:18:42.966732 10941 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
I20260709 13:18:42.873852 17055 raft_consensus.cc:2749] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:42.970150 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:42.955242 16969 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:18:42.973136 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:42.882500 17065 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:42.977794 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:42.965972 17011 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.982311 17011 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:42.984457 17011 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:42.984745 17032 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:42.985291 17032 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:42.953459 17072 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:42.910393 17069 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:42.961705 17091 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:42.987073 17091 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:42.987449 17091 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:42.987709 10875 raft_consensus.cc:2410] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:18:42.992311 17065 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:42.992672 10946 raft_consensus.cc:1275] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:18:42.993129 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:42.993623 17055 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:42.994132 17055 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:42.980043 17093 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:42.996615 10868 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49290 (request call id 962) took 1348 ms (client timeout 1790 ms). Trace:
W20260709 13:18:42.997022 10868 rpcz_store.cc:269] 0709 13:18:41.648738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:41.649220 (+   482us) service_pool.cc:224] Handling call
0709 13:18:41.934399 (+285179us) spinlock_profiling.cc:243] Waited 3.72 ms on lock 0x7b5800177098. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd90a22 00007f9c86ca9c09 00007f9c85e03ee0 00007f9c85ecd0ab 00007f9c85ecd131 00007f9c85e7b0a0 00007f9c85e7a984 00007f9c7f3f56d4 00007f9c7f3f5662
0709 13:18:42.996454 (+1062055us) spinlock_profiling.cc:243] Waited 267 ms on lock 0x7b5400078354. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:42.996559 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":623252480}
I20260709 13:18:42.997491 17093 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:42.974100 17026 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:42.993227 17075 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:43.001601 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:43.003022 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:43.004372 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:42.994693 10946 raft_consensus.cc:1240] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 21. Current term is 22. Ops: []
I20260709 13:18:42.966536 17041 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.011157 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:43.026316 10799 raft_consensus.cc:1275] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 13:18:43.028792 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:43.036208 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:43.036067 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:43.041052 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:43.042905 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:42.984481 16845 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:18:43.088045 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:42.974572 16929 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:43.091387 11477 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:18:43.071655 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 178598us
I20260709 13:18:43.038439 17041 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:43.111843 10766 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:43.071609 17075 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:43.119388 17093 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:18:43.121078 17093 raft_consensus.cc:697] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:43.121757 17026 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:43.122087 17093 consensus_queue.cc:237] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:43.119671 17105 consensus_queue.cc:1059] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 13:18:43.124965 17075 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:43.127295 17075 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:43.129112 17041 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:43.105159 17062 raft_consensus.cc:2804] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:43.043769 16494 consensus_queue.cc:1048] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.070s
I20260709 13:18:43.131186 17101 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:43.131295 10698 catalog_manager.cc:5697] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 11 to 14, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 14 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:43.133476 17101 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:43.092715 16738 consensus_queue.cc:1048] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.043s
I20260709 13:18:43.139113 16932 raft_consensus.cc:3055] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:43.139629 16932 raft_consensus.cc:740] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:43.140528 16932 consensus_queue.cc:260] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:43.141381 17071 raft_consensus.cc:3055] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:43.141753 16932 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:43.141932 17071 raft_consensus.cc:740] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:43.143178 17071 consensus_queue.cc:260] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:43.145694 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:43.146384 17070 raft_consensus.cc:3055] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:43.146812 17070 raft_consensus.cc:740] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:43.094249 16935 consensus_queue.cc:1048] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 13:18:43.147430 10916 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:43.147739 17070 consensus_queue.cc:260] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:43.148980 17070 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:43.149819 10698 catalog_manager.cc:5697] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 11 to 15, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 15 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:18:43.154176 10873 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 2423) took 2339 ms (client timeout 1858 ms). Trace:
W20260709 13:18:43.154758 10873 rpcz_store.cc:269] 0709 13:18:40.814501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:40.814645 (+   144us) service_pool.cc:224] Handling call
0709 13:18:43.153905 (+2339260us) spinlock_profiling.cc:243] Waited 2.05 s on lock 0x7b54002d4954. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:43.154052 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4705732480}
I20260709 13:18:43.156013 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:43.158064 10949 raft_consensus.cc:2393] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 18: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:43.159005 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:43.159613 17032 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:43.160084 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:43.161784 17106 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:43.163316 16714 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:43.165685 17083 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:43.170246 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:43.170169 16242 consensus_queue.cc:1048] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:43.173982 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:18:43.180100 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:43.180246 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:43.170331 10916 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:43.199999 10916 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:43.194219 17009 raft_consensus.cc:2749] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:18:43.194248 17062 raft_consensus.cc:697] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 16 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:43.205129 17062 consensus_queue.cc:237] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:43.207666 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:43.210343 10766 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:43.212739 10916 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:43.184984 16998 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:43.217820 10916 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:43.206951 17057 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.216905 10841 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:43.239231 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:43.240296 10799 raft_consensus.cc:2393] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 15: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:43.245319 10792 raft_consensus.cc:1240] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:43.219324 10698 catalog_manager.cc:5697] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:43.214380 16935 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.253922 10767 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:43.212431 17032 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.212399 16714 raft_consensus.cc:2764] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:18:43.217455 17026 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:18:43.257974 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 155524us
W20260709 13:18:43.257296 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 47062us
I20260709 13:18:43.259205 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:43.260702 10794 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 15.
I20260709 13:18:43.262043 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:43.262778 10798 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:43.263844 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:18:43.264282 10685 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 171abdebfc4142cda86de477a5c89a7e with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER failed with leader 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 12682eb70bc84c76879e04a8628ee586 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 17 committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:18:43.212491 17106 consensus_queue.cc:1048] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 13:18:43.269019 10796 raft_consensus.cc:2368] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for earlier term 12. Current term is 13.
I20260709 13:18:43.268056 10698 catalog_manager.cc:5697] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 13 to 14, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 14 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:43.216035 17041 consensus_queue.cc:1048] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:18:43.256088 16845 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:18:43.266009 17105 raft_consensus.cc:2804] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:43.284878 10698 catalog_manager.cc:5697] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 16 to 17, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 17 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:43.262065 17026 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:43.289687 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:43.291307 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:43.260993 16935 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:43.248019 17057 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:43.305227 16561 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:43.265239 17032 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:43.280177 16845 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:43.301772 17104 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:43.340082 10945 raft_consensus.cc:1240] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:43.325743 17026 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:18:43.356034 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 311093us
I20260709 13:18:43.352465 10698 catalog_manager.cc:5697] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 6 to 11. New cstate: current_term: 11 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
W20260709 13:18:43.344684 10875 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 2437) took 1987 ms (client timeout 1707 ms). Trace:
I20260709 13:18:43.276031 17015 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:43.349156 10942 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 20.
I20260709 13:18:43.324712 17069 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:43.342262 10943 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 18.
I20260709 13:18:43.351116 10798 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 17.
I20260709 13:18:43.266458 10767 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:43.357849 10875 rpcz_store.cc:269] 0709 13:18:41.356866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:41.487565 (+130699us) service_pool.cc:224] Handling call
0709 13:18:43.307921 (+1820356us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b54002f0654. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:43.344592 (+ 36671us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3362006400}
I20260709 13:18:43.374511 17054 raft_consensus.cc:2749] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:43.374652 17132 raft_consensus.cc:2749] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:18:43.375708 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:43.373654 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 107277us
W20260709 13:18:43.353180 17123 log.cc:927] Time spent T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.071s	user 0.000s	sys 0.002s
I20260709 13:18:43.382261 16738 consensus_queue.cc:1059] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:18:43.383410 16738 raft_consensus.cc:3055] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:43.383797 16738 raft_consensus.cc:740] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:43.384337 10874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 971) took 1025 ms. Trace:
I20260709 13:18:43.384680 16738 consensus_queue.cc:260] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:43.385984 16738 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:43.384716 10874 rpcz_store.cc:276] 0709 13:18:42.358930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:42.781797 (+422867us) service_pool.cc:224] Handling call
0709 13:18:42.782103 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:42.782417 (+   314us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:42.782438 (+    21us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:42.792024 (+  9586us) log.cc:844] Serialized 26 byte log entry
0709 13:18:43.186907 (+394883us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:43.186966 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:43.187080 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:43.384097 (+197017us) raft_consensus.cc:1685] finished
0709 13:18:43.384123 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:43.384276 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":418,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":5681,"thread_start_us":372821,"threads_started":1,"wal-append.queue_time_us":416653}
I20260709 13:18:43.379055 10698 catalog_manager.cc:5697] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 12 to 13, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 13 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:18:43.379606 11477 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:43.352093 16845 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:18:43.391181 10767 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:43.348452 17105 raft_consensus.cc:697] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:43.393347 17129 raft_consensus.cc:2749] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:18:43.393771  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:43.373593 17083 raft_consensus.cc:2749] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:43.377954 16929 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:43.370163 17104 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:43.389783 17030 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.351750 16561 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:43.420257 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:43.395174 10698 catalog_manager.cc:5697] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 10 to 15, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 15 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:18:43.395349 10767 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:43.394059 16935 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:43.427212 16242 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:43.428386 10698 catalog_manager.cc:5697] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:18:43.435631 10698 catalog_manager.cc:5697] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:43.439988 10698 catalog_manager.cc:5697] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 13 to 16, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 16 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:43.425377 17135 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:18:43.453919 17127 log.cc:927] Time spent T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:18:43.456837 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 2283) took 1231 ms. Trace:
I20260709 13:18:43.457266 10869 rpcz_store.cc:276] 0709 13:18:42.225217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:42.225624 (+   407us) service_pool.cc:224] Handling call
0709 13:18:43.456619 (+1230995us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b54000997d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:43.456778 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":27177856}
I20260709 13:18:43.452426 10698 catalog_manager.cc:5697] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 13 to 15, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 15 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:43.461958 17107 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:18:43.464354 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 106921us
I20260709 13:18:43.465158 10842 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:43.395783 17069 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:43.467842 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 250993us
I20260709 13:18:43.470825 11478 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:43.462059 10698 catalog_manager.cc:5697] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 7 to 12, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 12 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:43.404772 17132 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.481623 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:43.443589 17129 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:18:43.479631 10943 raft_consensus.cc:1240] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 21. Current term is 22. Ops: []
I20260709 13:18:43.471362 10841 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:43.402174 17065 consensus_queue.cc:1048] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.074s
I20260709 13:18:43.416636 17104 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:43.473915 17130 log.cc:927] Time spent T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:18:43.409263 16911 consensus_queue.cc:1059] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.067s
I20260709 13:18:43.423485 17030 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:43.472494 16929 raft_consensus.cc:2804] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:43.500998 11478 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:43.473068 17057 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:43.494202 17131 log.cc:927] Time spent T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:18:43.486420 17129 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:43.529321 11478 leader_election.cc:400] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 12 election: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for earlier term 12. Current term is 13.
I20260709 13:18:43.507526 16839 raft_consensus.cc:2749] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:18:43.511475 17052 raft_consensus.cc:3055] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:43.481679 17069 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:43.515309 16935 raft_consensus.cc:2749] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:43.548364 17150 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.548473 17161 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:43.549325 17161 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:43.549180 17150 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:43.549873 17161 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:43.551854 17150 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:43.552706 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:43.553069 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:43.553831 10874 raft_consensus.cc:2393] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 17: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:43.558130 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:43.559056 10871 raft_consensus.cc:2410] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:18:43.561671 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:43.562285 10871 raft_consensus.cc:2376] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15. Re-sending same reply.
I20260709 13:18:43.562012 10869 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 18.
I20260709 13:18:43.568688 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:43.577594 10767 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:43.579800 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:43.580629 10874 raft_consensus.cc:2393] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 18: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:43.571326 10871 raft_consensus.cc:2376] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 19. Re-sending same reply.
I20260709 13:18:43.530160 16929 raft_consensus.cc:697] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:43.535902 17129 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:18:43.474299 17105 consensus_queue.cc:237] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:43.485492 17132 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:43.595202 10767 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:43.527179 17070 consensus_queue.cc:1059] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 13:18:43.542382 17052 raft_consensus.cc:740] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:43.608223 17149 raft_consensus.cc:2749] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:43.607909 17052 consensus_queue.cc:260] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:43.594975 17135 raft_consensus.cc:697] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 20 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:43.606055 10767 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:43.609040 17052 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:43.610148 17030 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:43.513820 17142 consensus_queue.cc:1048] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.074s
I20260709 13:18:43.571761 17032 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:43.601976 17023 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:43.635594 10950 raft_consensus.cc:2410] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:18:43.617177 10767 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:43.643713 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:43.646351 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:43.583912 16911 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:18:43.627746 16969 consensus_meta.cc:287] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.646s	user 0.007s	sys 0.004s
W20260709 13:18:43.638247 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 106294us
W20260709 13:18:43.629949 17091 consensus_meta.cc:287] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.642s	user 0.010s	sys 0.000s
I20260709 13:18:43.658058 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:43.664528 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:43.671541 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:43.672210 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:43.673352 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:43.672431 10795 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
I20260709 13:18:43.552253 17158 consensus_queue.cc:1048] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.086s
I20260709 13:18:43.674945 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:43.618930 17054 raft_consensus.cc:2804] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:18:43.682578 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 175006us
I20260709 13:18:43.675882 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:43.684005 10841 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:43.616861 17015 raft_consensus.cc:2749] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:43.619953 17104 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:43.620558 17070 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:43.694227 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:43.662650 17108 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:18:43.697106 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:43.701900 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:43.701807 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:43.702836 10947 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
I20260709 13:18:43.661732 16912 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:43.704563 10947 raft_consensus.cc:1240] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 15. Current term is 16. Ops: []
W20260709 13:18:43.706422 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:43.709628 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:43.710518 10841 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:43.710853 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:43.713940 17163 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.714509 10947 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
I20260709 13:18:43.650653 17065 consensus_queue.cc:1048] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 13:18:43.685791 16969 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 16. Current term is 17.
I20260709 13:18:43.683197 17083 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:43.725049 10916 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:43.728248 17163 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:43.719913 16912 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:43.729815 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:18:43.704676 17144 log.cc:927] Time spent T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.149s	user 0.001s	sys 0.001s
W20260709 13:18:43.755345 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:43.631502 16929 consensus_queue.cc:237] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:43.677661 16911 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:43.731042 17147 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.620021 17135 consensus_queue.cc:237] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:43.701907 17054 raft_consensus.cc:697] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:18:43.716710 17053 log.cc:927] Time spent T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 2.127s	user 0.006s	sys 0.000s
W20260709 13:18:43.775364 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 50433.4us
I20260709 13:18:43.803655 10916 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:43.782267 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 91815.6us
I20260709 13:18:43.720373 17108 raft_consensus.cc:697] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:43.705258 17032 consensus_queue.cc:1059] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 13:18:43.797683 17083 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:43.694686 17091 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:43.703799 17070 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:43.807185 10792 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 14.
I20260709 13:18:43.807507 11506 raft_consensus.cc:1275] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:18:43.807185 10949 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
W20260709 13:18:43.814442 10872 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49290 (request call id 963) took 2100 ms (client timeout 2563 ms). Trace:
I20260709 13:18:43.792464 16998 raft_consensus.cc:2749] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:43.832604 10766 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:43.842343 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:43.843686 11507 raft_consensus.cc:2410] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 18 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
W20260709 13:18:43.838667 10872 rpcz_store.cc:269] 0709 13:18:41.713719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:41.763531 (+ 49812us) service_pool.cc:224] Handling call
0709 13:18:43.814222 (+2050691us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b54002d4454. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:43.814375 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3370477824}
I20260709 13:18:43.721571 16845 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:43.811156 17153 log.cc:927] Time spent T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 13:18:43.851416 10946 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 17.
W20260709 13:18:43.838233 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:43.808844 17158 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:18:43.853345 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:43.821556 16969 raft_consensus.cc:2749] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:18:43.820927 17083 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:43.878472 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:43.857583 11510 raft_consensus.cc:2410] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:18:43.847142 16932 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:43.799146 17147 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:43.816483 17163 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:18:43.902304 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 191834us
I20260709 13:18:43.849505 17052 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:43.903381 10841 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:43.904066 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:43.904906 10872 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:43.909089 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:43.856720 17057 raft_consensus.cc:2749] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:43.910085 10871 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 17.
I20260709 13:18:43.818321 17026 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:43.877811 17065 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.857075 17158 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:43.936853 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:18:43.909751 17144 log.cc:927] Time spent T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 13:18:43.933244 17153 log.cc:927] Time spent T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:18:43.945806 10841 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:43.956590 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:43.952723 10801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 844) took 1523 ms. Trace:
I20260709 13:18:43.850178 16494 consensus_queue.cc:1048] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 13:18:43.958233 10801 rpcz_store.cc:276] 0709 13:18:42.429733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:42.429948 (+   215us) service_pool.cc:224] Handling call
0709 13:18:42.430324 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:42.659917 (+229593us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:42.659949 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:42.660000 (+    51us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:42.660041 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:43.952562 (+1292521us) spinlock_profiling.cc:243] Waited 229 ms on lock 0x7b5400294d54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:43.952593 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:43.952678 (+    85us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":527490304}
I20260709 13:18:43.959806 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:43.899148 16932 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:43.908885 17023 raft_consensus.cc:2749] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:43.960386 10801 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
I20260709 13:18:43.965713 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:43.943501 17158 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:43.968539 10841 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:43.815359 17108 consensus_queue.cc:237] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:43.968912 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:43.955708 17092 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.160s	user 0.009s	sys 0.028s
I20260709 13:18:43.971058 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:43.981673 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:43.984141 10872 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 17.
W20260709 13:18:43.983911 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 52672.6us
I20260709 13:18:43.987735 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:43.988193 10766 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:43.989439 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:43.935278 17052 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:43.996475 10841 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:43.964599 16932 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:43.943142 17065 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:44.021974 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:43.982864 17092 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.196s	user 0.010s	sys 0.028s
I20260709 13:18:44.022702 17023 raft_consensus.cc:2749] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:44.054337 17083 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.026968 17091 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:44.028909 17070 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:44.071655 17083 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:44.079476 17178 log.cc:927] Time spent T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:18:44.080025 16845 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:43.910717 17054 consensus_queue.cc:237] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:44.084573 17065 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:43.974359 17094 raft_consensus.cc:2749] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:44.047281 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:44.053359 16932 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:44.058271 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 54783.2us
I20260709 13:18:43.957865 17179 raft_consensus.cc:2749] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:18:43.979878 17169 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:44.062013 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 60496.6us
I20260709 13:18:44.000682 17170 raft_consensus.cc:2749] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:18:44.000831 17101 raft_consensus.cc:3055] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:44.079428 17133 raft_consensus.cc:2749] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:18:44.038865 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:44.104064 10872 raft_consensus.cc:1275] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:18:44.111302 11507 raft_consensus.cc:1275] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:18:44.127969 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:44.128854 10872 raft_consensus.cc:2376] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 17. Re-sending same reply.
I20260709 13:18:44.131112 17128 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:44.131914 17128 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:44.134918 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:44.135733 11510 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:18:44.139693 17147 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:44.148684 16932 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:44.118615 17169 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:44.152313 17169 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:44.115249 17101 raft_consensus.cc:740] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:44.165645 10869 raft_consensus.cc:1275] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:18:44.103164 16912 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:44.173897 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:44.127099 17107 consensus_meta.cc:287] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.664s	user 0.016s	sys 0.000s
I20260709 13:18:44.174803 10869 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 18.
I20260709 13:18:44.175375 17128 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:44.183454 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:44.123034 16929 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:44.190847 10941 raft_consensus.cc:1275] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 9. (term mismatch)
I20260709 13:18:44.196223 10941 pending_rounds.cc:85] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 8
I20260709 13:18:44.196461 10941 pending_rounds.cc:107] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 18.9
I20260709 13:18:44.196766 10941 raft_consensus.cc:2889] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:44.194633 11510 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 18.
I20260709 13:18:44.186969 17065 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.158301 17101 consensus_queue.cc:260] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:44.207854 17161 consensus_meta.cc:287] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.657s	user 0.011s	sys 0.004s
I20260709 13:18:44.188791 17184 raft_consensus.cc:3055] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:18:44.221793 17072 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:44.215227 17069 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:44.264367 11478 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:44.286106 17188 raft_consensus.cc:2764] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:18:44.288532 17188 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:44.289129 17188 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:44.291095 17164 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:44.291750 17164 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:44.294051 17164 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:44.296312 11478 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:44.297950 17015 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.298135 11478 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:44.244936 17184 raft_consensus.cc:740] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:44.261565 17108 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:44.199472 16929 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:44.296957 17026 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:44.245296 17065 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:44.294916 17188 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:44.303632 17135 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:44.275883 17161 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:44.321060 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:44.314498 17139 raft_consensus.cc:2749] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:18:44.325219 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:44.334960 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:44.376528 10945 raft_consensus.cc:1275] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:18:44.305198 17015 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:44.374538 17135 raft_consensus.cc:697] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 17 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:44.313959 17108 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:44.328807 17052 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:44.407275 17135 consensus_queue.cc:237] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:44.410030 17196 log.cc:927] Time spent T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:18:44.411129 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:44.411345 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:44.417582 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:44.419003 17128 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.426051 10793 raft_consensus.cc:1240] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 LEADER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 16. Current term is 17. Ops: []
W20260709 13:18:44.427912 17137 log.cc:927] Time spent T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.953s	user 0.004s	sys 0.000s
W20260709 13:18:44.428519 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:44.429405 17092 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.431140 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:44.436335 10941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2876) took 1479 ms. Trace:
I20260709 13:18:44.437803 10950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41020 (request call id 2864) took 1563 ms. Trace:
I20260709 13:18:44.440642 16242 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:44.444437 17026 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:44.478034 10950 rpcz_store.cc:276] 0709 13:18:42.874412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:42.874520 (+   108us) service_pool.cc:224] Handling call
0709 13:18:44.437519 (+1562999us) spinlock_profiling.cc:243] Waited 732 ms on lock 0x7b5400399c54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:44.437693 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1684632960}
I20260709 13:18:44.449096 16561 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:44.452632 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:44.477967 10941 rpcz_store.cc:276] 0709 13:18:42.956782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:42.999504 (+ 42722us) service_pool.cc:224] Handling call
0709 13:18:42.999813 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:44.197390 (+1197577us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:44.436008 (+238618us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b5400376754. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:44.436257 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2737840512}
I20260709 13:18:44.459893 10868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2460) took 1930 ms. Trace:
W20260709 13:18:44.402627 17190 log.cc:927] Time spent T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 13:18:44.461813 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:44.462512 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:44.485213 10800 raft_consensus.cc:2393] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 17: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:44.465359 17161 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:44.466838 17132 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:44.465421 17065 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:44.473399 10869 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 22.
I20260709 13:18:44.489679 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:44.474723 11506 raft_consensus.cc:1240] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:44.476718 17139 consensus_queue.cc:1048] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.014s
I20260709 13:18:44.478211 16242 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:44.491735 17055 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:44.493108 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:44.493281 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:44.484455 17063 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.494040 17063 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:44.494380 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:44.480983 10795 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 14.
W20260709 13:18:44.495221 10946 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36714 (request call id 1118) took 1471 ms (client timeout 1554 ms). Trace:
I20260709 13:18:44.455101 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:44.495659 10947 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
I20260709 13:18:44.480703 17128 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:44.497088 16561 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.497642 16561 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:44.498222 17128 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:44.484493 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:18:44.495769 10946 rpcz_store.cc:269] 0709 13:18:43.023591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:43.035738 (+ 12147us) service_pool.cc:224] Handling call
0709 13:18:44.495065 (+1459327us) spinlock_profiling.cc:243] Waited 755 ms on lock 0x7b54001e1754. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:44.495167 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1736878464}
I20260709 13:18:44.484504 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:44.500394 10872 raft_consensus.cc:2393] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 16: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:44.501298 17015 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:44.501338 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:44.505071 11478 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:44.484784 16986 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:44.516119 16242 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:44.522203 10948 raft_consensus.cc:1240] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 18. Current term is 19. Ops: []
I20260709 13:18:44.481990 10868 rpcz_store.cc:276] 0709 13:18:42.529301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:42.997912 (+468611us) service_pool.cc:224] Handling call
0709 13:18:42.998273 (+   361us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:43.044219 (+ 45946us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:43.044270 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:43.071395 (+ 27125us) log.cc:844] Serialized 26 byte log entry
0709 13:18:43.731439 (+660044us) spinlock_profiling.cc:243] Waited 1.72 ms on lock 0x7b1800450b20. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a550d1 00007f9c80e70a0f
0709 13:18:43.953705 (+222266us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:43.969741 (+ 16036us) log.cc:844] Serialized 14 byte log entry
0709 13:18:43.999217 (+ 29476us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:44.459436 (+460219us) spinlock_profiling.cc:243] Waited 40.7 ms on lock 0x7b540006d1d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:44.459500 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:44.459564 (+    64us) raft_consensus.cc:1685] finished
0709 13:18:44.459585 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:44.459799 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29069,"prepare.queue_time_us":15908,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":81525,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":33080,"spinlock_wait_cycles":97449088,"thread_start_us":213355,"threads_started":1,"wal-append.queue_time_us":977309}
I20260709 13:18:44.488060 17108 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:44.490741 17032 raft_consensus.cc:3055] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:44.492830 17026 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:44.442673 16929 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:44.478022 10874 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:18:44.527099 17161 raft_consensus.cc:2749] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:44.526993 16986 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:44.538370 10869 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 15.
I20260709 13:18:44.540344 10876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2459) took 2014 ms. Trace:
I20260709 13:18:44.540769 10876 rpcz_store.cc:276] 0709 13:18:42.526214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:42.994203 (+467989us) service_pool.cc:224] Handling call
0709 13:18:42.994515 (+   312us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:42.994904 (+   389us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:42.994949 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:42.995657 (+   708us) log.cc:844] Serialized 52 byte log entry
0709 13:18:43.351016 (+355359us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:43.351087 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:43.351184 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:44.539042 (+1187858us) spinlock_profiling.cc:243] Waited 5.05 ms on lock 0x7b2400124028. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dc4e9c4 00007f9c7dc4cacb 00007f9c7de07dc9 00007f9c7de07a55 00007f9c7de06fb0 00007f9c80a7d488 00007f9c80a7dbe8 00007f9c80a55398 00007f9c80a5076a 00007f9c80a4fbb6
0709 13:18:44.539950 (+   908us) raft_consensus.cc:1685] finished
0709 13:18:44.539974 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:44.540272 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8479,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":17436,"spinlock_wait_cycles":11615488,"thread_start_us":354826,"threads_started":1,"wal-append.queue_time_us":414610}
I20260709 13:18:44.542771 16561 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:18:44.543198 17158 consensus_meta.cc:287] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.559s	user 0.011s	sys 0.000s
I20260709 13:18:44.543821 17158 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:44.547125 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:44.507091 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:44.404129 17184 consensus_queue.cc:260] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:44.518340 17162 log.cc:927] Time spent T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.698s	user 0.010s	sys 0.000s
I20260709 13:18:44.548434 10947 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:44.495282 17092 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:44.521234 17129 consensus_meta.cc:287] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.885s	user 0.006s	sys 0.007s
I20260709 13:18:44.542672 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:44.551386 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:44.551769 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:44.552155 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:44.554122 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:44.555032 10797 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 18.
I20260709 13:18:44.556166 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:44.546510 17158 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:44.567796 16242 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:44.569022 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:44.574911 10876 raft_consensus.cc:1240] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:44.542438 17026 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:44.571832 10916 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:44.576867 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:44.577704 10916 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:44.577812 10798 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 17.
I20260709 13:18:44.578598 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:44.574654 10874 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 22.
I20260709 13:18:44.573105 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:44.527608 17032 raft_consensus.cc:740] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:44.580593 16242 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.585870 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:44.586267 10876 raft_consensus.cc:1275] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:18:44.586026 16242 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:44.580824 10874 raft_consensus.cc:1240] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:18:44.582320  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:44.584486 10916 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:44.588950 10916 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:44.590837 10916 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:44.559680 16714 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:44.561087 16561 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:44.619285 10872 raft_consensus.cc:1240] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:18:44.619027 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:44.600543 10947 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 18.
I20260709 13:18:44.576433 16913 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:44.623556 16242 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:44.621727 10874 raft_consensus.cc:3055] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:44.567412 17129 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:44.586033 17083 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:44.659286 10874 raft_consensus.cc:740] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:44.586186 17052 raft_consensus.cc:2749] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:44.662367 10874 consensus_queue.cc:260] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:44.586587 17065 raft_consensus.cc:2804] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:18:44.679388 10875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2466) took 1126 ms. Trace:
I20260709 13:18:44.645347 17164 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.622991 17026 raft_consensus.cc:2749] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:44.592660 16617 raft_consensus.cc:2749] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:18:44.692169 10801 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:18:44.690865 10874 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:44.593173 17109 raft_consensus.cc:2749] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:44.690759 10875 rpcz_store.cc:276] 0709 13:18:43.552955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:43.553075 (+   120us) service_pool.cc:224] Handling call
0709 13:18:43.553427 (+   352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:43.553804 (+   377us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:43.553834 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:43.554332 (+   498us) log.cc:844] Serialized 26 byte log entry
0709 13:18:43.650460 (+ 96128us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:43.680733 (+ 30273us) log.cc:844] Serialized 14 byte log entry
0709 13:18:43.681043 (+   310us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:43.681143 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:44.517585 (+836442us) spinlock_profiling.cc:243] Waited 7.77 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:18:44.671586 (+154001us) raft_consensus.cc:1685] finished
0709 13:18:44.679035 (+  7449us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:44.679272 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":30668,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":2894,"raft.queue_time_us":53425,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":12931,"spinlock_wait_cycles":17868160,"thread_start_us":95724,"threads_started":1,"wal-append.queue_time_us":250744}
I20260709 13:18:44.623983 16714 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:44.596834 16907 consensus_queue.cc:1048] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.146s
I20260709 13:18:44.690151 17065 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:44.726171 10949 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 18.
I20260709 13:18:44.761512 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:44.627410 16561 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:44.762805 10874 raft_consensus.cc:1275] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 13:18:44.762408 10949 raft_consensus.cc:2376] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 17. Re-sending same reply.
I20260709 13:18:44.709971 17092 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:44.763744 10801 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 23.
I20260709 13:18:44.628021 17104 consensus_queue.cc:1048] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.133s
I20260709 13:18:44.765412 10874 pending_rounds.cc:85] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 11
I20260709 13:18:44.778497 10874 pending_rounds.cc:107] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:18:44.778842 10874 raft_consensus.cc:2889] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:44.785537 10949 raft_consensus.cc:1240] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 21. Current term is 22. Ops: []
I20260709 13:18:44.709436 17164 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:44.762665 17065 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:44.792841 10949 raft_consensus.cc:1275] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:18:44.832432 16714 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:44.834581 16561 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:44.837291 17197 log.cc:927] Time spent T 00000000000000000000000000000000 P ae957c8fc66c4b6ba063d0c74cbf5484: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 13:18:44.860236 10793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 848) took 2395 ms. Trace:
I20260709 13:18:44.860829 10793 rpcz_store.cc:276] 0709 13:18:42.464875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:42.465487 (+   612us) service_pool.cc:224] Handling call
0709 13:18:42.494963 (+ 29476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:44.476139 (+1981176us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:44.859900 (+383761us) spinlock_profiling.cc:243] Waited 1.93 s on lock 0x7b54000d7354. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:44.860156 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4440571648}
W20260709 13:18:44.894939 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:44.909552 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:44.783098 17063 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:44.709642 17032 consensus_queue.cc:260] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:45.017186 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 394285us
I20260709 13:18:44.966998 16907 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.019734 10916 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:18:45.021852 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 386028us
I20260709 13:18:45.008939 16557 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:45.027331 10916 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:44.762889 16913 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:45.028370 17092 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:45.028218 17075 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:45.034298 17104 raft_consensus.cc:2804] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:45.045372 10916 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:45.035970 16911 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:45.033759 17107 raft_consensus.cc:2749] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:45.023129 16907 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:45.043181 17032 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:45.059090 17089 raft_consensus.cc:3055] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:18:45.059566 17089 raft_consensus.cc:740] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:45.060725 16907 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:45.060483 17089 consensus_queue.cc:260] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:45.061538 17089 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:18:45.057376 10916 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:45.051524 17104 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:45.067137 16491 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:45.070420 10916 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:45.063140 16911 raft_consensus.cc:2804] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:45.059156 17164 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:45.054742 17075 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:45.076460 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:45.070904 17104 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:45.074659 17109 raft_consensus.cc:2804] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:45.054488 17092 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:45.073663 16911 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:45.071511 16491 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:45.075915 17198 log.cc:927] Time spent T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.285s	user 0.000s	sys 0.001s
I20260709 13:18:45.090893 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:45.095065 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:45.130211 10793 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 19.
I20260709 13:18:45.134394 10916 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:45.077316 17109 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:45.137620 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:45.128962 17199 log.cc:927] Time spent T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.262s	user 0.001s	sys 0.000s
I20260709 13:18:45.127344 16911 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:45.130055 16491 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:45.078867 17129 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:45.158874 10916 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:45.156881 16242 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:18:45.144731 10798 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 19.
I20260709 13:18:45.160441 16242 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:45.147547 17109 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:45.161187 16242 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:45.169914 17092 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:45.175251 11503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56416 (request call id 973) took 2066 ms. Trace:
I20260709 13:18:45.177986 10916 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:45.170625 17092 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:18:45.185246 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:45.176903 11503 rpcz_store.cc:276] 0709 13:18:43.108475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:43.108684 (+   209us) service_pool.cc:224] Handling call
0709 13:18:43.120936 (+ 12252us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:43.370247 (+249311us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:18:43.521969 (+151722us) log.cc:844] Serialized 14 byte log entry
0709 13:18:44.163903 (+641934us) spinlock_profiling.cc:243] Waited 7.7 ms on lock 0x7b1800157c00. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c80a5d4bc 00007f9c7ddab48b 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a5523a
0709 13:18:44.164185 (+   282us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:44.172099 (+  7914us) log.cc:844] Serialized 26 byte log entry
0709 13:18:44.172294 (+   195us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:18:44.172353 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:44.172450 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:44.677906 (+505456us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b1800157c00. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7dd962a6 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc83f3 00007f9c7ddca7c1
0709 13:18:45.170126 (+492220us) spinlock_profiling.cc:243] Waited 17.7 ms on lock 0x7b1800157c00. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc83f3 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:18:45.174932 (+  4806us) raft_consensus.cc:1685] finished
0709 13:18:45.174965 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:45.175174 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61247,"prepare.queue_time_us":143136,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":80461,"raft.run_cpu_time_us":337,"raft.run_wall_time_us":24937,"spinlock_wait_cycles":83406720,"thread_start_us":350246,"threads_started":3,"wal-append.queue_time_us":30714}
I20260709 13:18:45.175776 17164 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.177453 16907 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:18:45.181384 17129 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:45.183058 17204 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:18:45.206542 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:45.200724 16913 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:45.217813 17204 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:45.189502 17065 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:45.208283 16907 raft_consensus.cc:697] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 23 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:45.202529 17092 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:45.205189 17164 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:45.218556 17129 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:45.236946 16871 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:45.240109 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:45.242501 10950 raft_consensus.cc:1240] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 20. Current term is 21. Ops: []
I20260709 13:18:45.243216 10792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 2953) took 1596 ms. Trace:
I20260709 13:18:45.245807 10792 rpcz_store.cc:276] 0709 13:18:43.646465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:43.673656 (+ 27191us) service_pool.cc:224] Handling call
0709 13:18:45.243025 (+1569369us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b54000bd6d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:45.243152 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":301592832}
I20260709 13:18:45.226528 17204 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:45.248346 16242 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:45.252822 10916 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.116.130:37895 timed out after 1.771s (SENT) [suppressed 18 similar messages]
I20260709 13:18:45.257712 16242 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:45.246591 17075 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:45.277271 17093 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:45.278329 16907 consensus_queue.cc:237] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:45.279682 16375 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.282924 16986 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:45.303871 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 51089.7us
I20260709 13:18:45.307164 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:45.308030 10792 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:45.313886 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:45.314980 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:45.315810 10799 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
I20260709 13:18:45.316839 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:45.317727 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:45.319738 10842 connection.cc:256] RPC call timeout handler was delayed by -1.715535583000019s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
W20260709 13:18:45.321014 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:45.323740 10800 raft_consensus.cc:2410] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:18:45.325717 10842 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:45.324960 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:45.334203 10842 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:45.348091 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:45.294091 17075 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:45.360790 10795 raft_consensus.cc:2393] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 16: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:45.355304 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:45.367321 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:45.374508 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:45.376580 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:45.349490 17135 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.301590 17093 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:45.383424 17135 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:45.354269 17133 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:18:45.385021 10870 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
I20260709 13:18:45.306489 16375 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:45.392568 10792 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 20.
I20260709 13:18:45.394338 10793 raft_consensus.cc:1240] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 19. Current term is 20. Ops: [19.9-19.9]
I20260709 13:18:45.365769 17101 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:45.418747 10767 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:45.386634 17092 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:45.427271 10801 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
I20260709 13:18:45.393352 17133 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:45.370548 17075 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:45.393368 17009 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.387173 17032 raft_consensus.cc:2804] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:18:45.435118 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:45.443786 17222 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:45.444442 17222 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:45.446631 17222 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:45.448100 17093 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:45.450183 17135 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:45.451603 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:45.456672 17129 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:45.457361 17129 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:18:45.457826 17129 raft_consensus.cc:697] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:45.460028 16375 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:45.394398 17210 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:45.435250 17133 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:45.462997 17032 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:45.439780 17065 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:45.484722 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 970) took 3122 ms. Trace:
I20260709 13:18:45.417766 16491 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:45.495818 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 161699us
I20260709 13:18:45.496405 10796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2314) took 1825 ms. Trace:
I20260709 13:18:45.500094 10876 raft_consensus.cc:1275] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 13:18:45.495426 17138 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.702s	user 0.013s	sys 0.018s
I20260709 13:18:45.490595 10867 rpcz_store.cc:276] 0709 13:18:42.358599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:42.787025 (+428426us) service_pool.cc:224] Handling call
0709 13:18:43.456763 (+669738us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:43.457163 (+   400us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:18:43.457472 (+   309us) log.cc:844] Serialized 14 byte log entry
0709 13:18:43.533593 (+ 76121us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:43.543575 (+  9982us) log.cc:844] Serialized 26 byte log entry
0709 13:18:43.543769 (+   194us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:18:43.543821 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:43.543919 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:44.068375 (+524456us) raft_consensus.cc:1685] finished
0709 13:18:44.068517 (+   142us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:45.481082 (+1412565us) spinlock_profiling.cc:243] Waited 669 ms on lock 0x7b54000997d0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:45.481272 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12235,"prepare.queue_time_us":671,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":26150,"raft.run_cpu_time_us":289,"raft.run_wall_time_us":13586,"spinlock_wait_cycles":1539331456,"thread_start_us":75696,"threads_started":1,"wal-append.queue_time_us":193428,"wal-append.run_cpu_time_us":8918,"wal-append.run_wall_time_us":1468253}
I20260709 13:18:45.506703 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:45.461342 17009 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:45.492983 17032 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:45.505806 17218 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:45.515003 17218 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:45.493917 17226 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:45.510275 10869 raft_consensus.cc:1275] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:18:45.517652 17065 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.517470 17226 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:45.512566 10867 raft_consensus.cc:1275] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:18:45.518229 17065 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:45.520833 17065 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:45.522186 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:45.523761 10841 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:45.503999 17138 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.724s	user 0.013s	sys 0.018s
I20260709 13:18:45.525573 10841 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:45.526866 10947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 2559) took 1081 ms. Trace:
I20260709 13:18:45.527575 17134 raft_consensus.cc:2804] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:45.528048 17134 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:45.528419 17134 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:45.500154 10796 rpcz_store.cc:276] 0709 13:18:43.676125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:43.986806 (+310681us) service_pool.cc:224] Handling call
0709 13:18:43.987175 (+   369us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:18:44.023540 (+ 36365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:44.031272 (+  7732us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:18:44.078813 (+ 47541us) log.cc:844] Serialized 78 byte log entry
0709 13:18:44.689634 (+610821us) spinlock_profiling.cc:243] Waited 6.14 ms on lock 0x7b1800021b20. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a550d1 00007f9c80e70a0f
0709 13:18:45.076413 (+386779us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:45.331936 (+255523us) log.cc:844] Serialized 14 byte log entry
0709 13:18:45.332346 (+   410us) log.cc:844] Serialized 14 byte log entry
0709 13:18:45.489421 (+157075us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:45.489507 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:45.495580 (+  6073us) raft_consensus.cc:1685] finished
0709 13:18:45.495635 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:45.496345 (+   710us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32549,"prepare.queue_time_us":199252,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":84,"raft.queue_time_us":218281,"raft.run_cpu_time_us":375,"raft.run_wall_time_us":70615,"spinlock_wait_cycles":46126720,"thread_start_us":712373,"threads_started":3,"wal-append.queue_time_us":1069725}
I20260709 13:18:45.529389 17011 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:18:45.527351 10947 rpcz_store.cc:276] 0709 13:18:44.445187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:44.625313 (+180126us) service_pool.cc:224] Handling call
0709 13:18:44.625659 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:44.754573 (+128914us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:44.754606 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:44.754660 (+    54us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:44.754708 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:45.526549 (+771841us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b54003589d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:45.526601 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:45.526766 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":295911808}
I20260709 13:18:45.530174 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:45.532691 10876 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:45.534200 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:45.534963 10875 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 17.
I20260709 13:18:45.535017 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:45.534112 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:45.536531 10867 raft_consensus.cc:2393] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 18: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:45.537881 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:45.541888 10869 raft_consensus.cc:1275] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:18:45.458714 17129 consensus_queue.cc:237] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:45.547259 10842 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:45.542932 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:45.548519 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:45.549187 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:45.480751 17210 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:45.574009 17210 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:45.550727 17032 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:45.575081 10874 raft_consensus.cc:2368] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 15. Current term is 16.
I20260709 13:18:45.576066 17032 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:45.577257 16907 consensus_queue.cc:1059] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:18:45.577289 10915 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:45.578378 17030 raft_consensus.cc:3055] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:45.578720 17030 raft_consensus.cc:740] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:45.578764 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:45.579679 17030 consensus_queue.cc:260] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:45.580431 10867 raft_consensus.cc:1275] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:18:45.504657 17104 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:45.552338 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:45.556453 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:18:45.560024 10685 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 171abdebfc4142cda86de477a5c89a7e with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER failed with leader 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:18:45.562935 17164 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:45.591842 10916 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:45.581035 10915 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:45.583521 10842 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:45.564301 10869 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 22.
I20260709 13:18:45.525573 17218 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:45.568344 10947 raft_consensus.cc:1275] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:18:45.568989 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:45.578954 16907 raft_consensus.cc:2749] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:18:45.551971 16491 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:45.595134 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:18:45.588508 17167 log.cc:927] Time spent T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 1.768s	user 0.018s	sys 0.010s
I20260709 13:18:45.624847 10941 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 16.
W20260709 13:18:45.626147 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 45172us
I20260709 13:18:45.627240 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:45.631129 10793 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:45.621810 10876 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 18.
I20260709 13:18:45.631517 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:45.629107 17065 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:18:45.624971 17052 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:45.644683 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:45.642750 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:45.642339 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:45.581820 17011 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:45.644260 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:45.641858 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:45.581025 17030 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:45.642251 10943 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 16.
I20260709 13:18:45.585037 17228 consensus_queue.cc:1059] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 13:18:45.634384 17218 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:45.646741 17065 raft_consensus.cc:697] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 20 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:45.657150 17169 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.657612 17065 consensus_queue.cc:237] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:45.646981 10801 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 20.
I20260709 13:18:45.647374 17052 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:45.634267 11507 pending_rounds.cc:85] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586: Aborting all ops after (but not including) 13
I20260709 13:18:45.661860 11507 pending_rounds.cc:107] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586: Aborting uncommitted NO_OP operation due to leader change: 16.14
I20260709 13:18:45.662142 11507 raft_consensus.cc:2889] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:45.662599 17229 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.663168 10916 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:45.664475 17210 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:45.665030 17210 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:45.665558 17089 consensus_queue.cc:1048] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:18:45.667440 16969 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:18:45.667858 16969 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:45.668179 16969 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:45.670068 17009 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:45.631028 16617 raft_consensus.cc:2749] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:45.643738 10945 raft_consensus.cc:1275] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:18:45.677078 10948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 1132) took 1393 ms. Trace:
I20260709 13:18:45.677507 10948 rpcz_store.cc:276] 0709 13:18:44.283881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:44.519934 (+236053us) service_pool.cc:224] Handling call
0709 13:18:44.520349 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:44.522712 (+  2363us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:45.676812 (+1154100us) spinlock_profiling.cc:243] Waited 1.6 ms on lock 0x7b54001e3554. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:45.677010 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3689728}
I20260709 13:18:45.653658 10793 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 19.
I20260709 13:18:45.653887 10947 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 18.
I20260709 13:18:45.655616 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2472) took 2085 ms. Trace:
I20260709 13:18:45.657712 17169 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:45.660544 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:45.649541 10950 raft_consensus.cc:2393] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 16: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
W20260709 13:18:45.698264 10870 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 2450) took 3174 ms (client timeout 1787 ms). Trace:
W20260709 13:18:45.700286 10870 rpcz_store.cc:269] 0709 13:18:42.523960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:42.703938 (+179978us) service_pool.cc:224] Handling call
0709 13:18:45.698069 (+2994131us) spinlock_profiling.cc:243] Waited 2.68 s on lock 0x7b5400076f54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:45.698194 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6163752832}
W20260709 13:18:45.701364 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 175855us
I20260709 13:18:45.683230 10873 rpcz_store.cc:276] 0709 13:18:43.569888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:43.569999 (+   111us) service_pool.cc:224] Handling call
0709 13:18:43.579906 (+  9907us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:43.580267 (+   361us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:43.580295 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:43.581471 (+  1176us) log.cc:844] Serialized 26 byte log entry
0709 13:18:43.720973 (+139502us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:43.729701 (+  8728us) log.cc:844] Serialized 14 byte log entry
0709 13:18:43.855313 (+125612us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:43.881095 (+ 25782us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:45.487040 (+1605945us) spinlock_profiling.cc:243] Waited 3.64 ms on lock 0x7b18003599e0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7dd962a6 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db
0709 13:18:45.587893 (+100853us) spinlock_profiling.cc:243] Waited 7.09 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:18:45.655425 (+ 67532us) raft_consensus.cc:1685] finished
0709 13:18:45.655447 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:45.655566 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":46135,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":46,"raft.queue_time_us":141060,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":3888,"spinlock_wait_cycles":24676480,"thread_start_us":258378,"threads_started":2,"wal-append.queue_time_us":232621}
W20260709 13:18:45.708204 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:45.654738 10794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2312) took 2006 ms. Trace:
I20260709 13:18:45.710852 10794 rpcz_store.cc:276] 0709 13:18:43.648509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:43.717685 (+ 69176us) service_pool.cc:224] Handling call
0709 13:18:45.244617 (+1526932us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:45.244797 (+   180us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:45.244821 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:45.244858 (+    37us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:45.244888 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:45.244943 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:45.654559 (+409616us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b54000bd6d0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:45.654668 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3386944768}
W20260709 13:18:45.712378 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:45.709055 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 43715us
I20260709 13:18:45.728628 10947 raft_consensus.cc:1240] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 18. Current term is 19. Ops: []
I20260709 13:18:45.732532 10797 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154"
dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
mode: GRACEFUL
new_leader_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:40148
I20260709 13:18:45.733392 10797 raft_consensus.cc:606] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Received request to transfer leadership to TS cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:45.733798 10797 raft_consensus.cc:614] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:45.733767 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:45.734490 10950 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 14.
I20260709 13:18:45.732453 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:45.737385 10948 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 18.
I20260709 13:18:45.741487 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:45.743072 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:45.639416 17226 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:45.743964 10950 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 16.
I20260709 13:18:45.746726 10766 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:45.744231 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 326168us
I20260709 13:18:45.749184 17009 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:45.752027 17009 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:45.754992 17009 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:45.652323 17011 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:45.763087 10943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36714 (request call id 1134) took 1462 ms. Trace:
I20260709 13:18:45.655035 17063 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:18:45.655478 17228 raft_consensus.cc:3055] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:18:45.651968 17109 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:45.763535 10943 rpcz_store.cc:276] 0709 13:18:44.300556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:44.314138 (+ 13582us) service_pool.cc:224] Handling call
0709 13:18:45.762898 (+1448760us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b540018e2d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:45.763028 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3034205952}
I20260709 13:18:45.764735 10766 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:45.663234 17229 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:45.736325 17075 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:45.648053 17134 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:45.812592 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 110190us
I20260709 13:18:45.645772 16911 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:45.771763 17129 raft_consensus.cc:2749] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:18:45.815490 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 50820.5us
W20260709 13:18:45.815908 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 84233.6us
I20260709 13:18:45.766561 17063 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:45.649381 17133 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:45.655674 17218 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:45.762557 16907 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:45.765167 17011 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:45.744573 17101 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:18:45.810621 17221 log.cc:927] Time spent T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:18:45.645754 17204 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:45.765183 17228 raft_consensus.cc:740] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:45.832583 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:45.856537 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:45.865247 10873 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 16.
I20260709 13:18:45.850507 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:45.866947 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:45.867554 10867 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 15.
I20260709 13:18:45.867585 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:45.868446 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:45.851996 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:45.868983 17104 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:45.869410 10875 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 17.
I20260709 13:18:45.870256 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:45.870774 10875 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 17.
I20260709 13:18:45.872706 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:45.873353 11512 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 16.
I20260709 13:18:45.873554 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:45.874392 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:45.871498 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:45.876775 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:45.877365 10873 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 17.
I20260709 13:18:45.878438 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:45.842730 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:45.828501 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:45.810993 17210 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:45.881884 10868 raft_consensus.cc:1240] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:45.860204 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:45.882663 10874 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 15.
I20260709 13:18:45.883551 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:45.884105 10874 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
I20260709 13:18:45.884886 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:45.880259 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:45.879241 10870 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 19.
I20260709 13:18:45.885417 10874 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 19.
I20260709 13:18:45.886169 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:45.886236 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:45.868611 10872 raft_consensus.cc:2393] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 16: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:45.826881 10766 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:45.886823 10870 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:45.880301 10869 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 17.
I20260709 13:18:45.889298 10874 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:18:45.893162 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:45.893302 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:45.893873 11509 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 13.
I20260709 13:18:45.893946 11504 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 16.
I20260709 13:18:45.895392 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:45.896277 10869 raft_consensus.cc:1275] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:18:45.899068 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:45.899206 17226 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:18:45.899718 10873 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 20.
I20260709 13:18:45.899928 17226 raft_consensus.cc:697] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:45.901239 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:45.902061 11503 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 16.
W20260709 13:18:45.902721 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 75988us
I20260709 13:18:45.821557 17075 raft_consensus.cc:2804] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:45.904807 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:45.912807 10870 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 19.
I20260709 13:18:45.857652 17063 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:45.916811 10766 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:18:45.918453 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 153072us
I20260709 13:18:45.924332 10874 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 23.
I20260709 13:18:45.876477 17101 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:45.894761 17139 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.958284 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:45.901188 17226 consensus_queue.cc:237] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:45.957854 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:45.913954 17075 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
W20260709 13:18:45.915861 17105 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.513s	user 0.022s	sys 0.023s
I20260709 13:18:45.887815 16907 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:46.033043 17139 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:46.036923 17105 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.657s	user 0.023s	sys 0.034s
I20260709 13:18:45.966603 17052 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:46.076018 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 219338us
W20260709 13:18:46.059556 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 185221us
I20260709 13:18:46.036293 17075 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:18:46.046105 17238 log.cc:927] Time spent T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:18:45.942515 16871 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:46.045331 17233 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:18:46.082624 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:46.092329 10799 raft_consensus.cc:1240] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:46.079419 10841 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:46.057464 17241 log.cc:927] Time spent T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 13:18:46.057330 17235 log.cc:927] Time spent T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.216s	user 0.000s	sys 0.000s
I20260709 13:18:46.120985 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:46.121884 10875 raft_consensus.cc:2376] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 22. Re-sending same reply.
I20260709 13:18:46.122982 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:46.130360 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:46.076103 17092 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:46.143450 17240 log.cc:927] Time spent T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:18:46.144452 17071 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:46.149211 17229 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:46.150396 17218 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:46.151530 17244 log.cc:927] Time spent T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:18:46.152549 17133 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:46.153559 17204 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:18:46.156194 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:46.156497 17246 log.cc:927] Time spent T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:18:46.157649 10871 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 17.
I20260709 13:18:45.967984 17228 consensus_queue.cc:260] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.9, Last appended by leader: 9, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:46.158381 16935 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:46.158936 17228 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:46.159049 16935 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:46.160756 17220 consensus_queue.cc:1048] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:18:46.165818 10798 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.023s	sys 0.064s
W20260709 13:18:46.166354 10798 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.024s	sys 0.064s
W20260709 13:18:46.166865 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 213808us
I20260709 13:18:46.105719 17169 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:18:46.157912 17245 log.cc:927] Time spent T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:18:46.154474 17256 raft_consensus.cc:2749] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:18:46.172142 10766 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:46.170606 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:18:46.143502 17239 log.cc:927] Time spent T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 13:18:46.110330 17233 raft_consensus.cc:697] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:46.102210 16871 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:46.178287 10766 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:46.138255 17248 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:46.188947 10766 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:46.193888 17164 consensus_queue.cc:1048] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:18:46.191679 10766 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:46.197103 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:46.176064 10875 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:18:46.176872 17105 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:46.147440 17092 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:46.208976 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 126932us
I20260709 13:18:46.177175 17134 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:46.219933 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 140685us
I20260709 13:18:46.220381 10870 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:46.188916 17109 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:46.238938 10841 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:46.189560 16911 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:46.187563 17169 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:46.241680 10870 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 21.
I20260709 13:18:46.176717 16375 consensus_queue.cc:1048] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:18:46.195508 17229 raft_consensus.cc:2804] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:46.201697 17237 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:46.206467 17093 raft_consensus.cc:2804] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:46.253834 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:46.255518 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:46.256162 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:46.263880 10946 raft_consensus.cc:2410] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:18:46.264003 10875 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 22.
I20260709 13:18:46.266940 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:46.267727 10949 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:46.268662 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:46.268824 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:46.269572 10947 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
I20260709 13:18:46.269903 10942 raft_consensus.cc:2393] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 16: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:46.271019 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:46.272629 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:46.274035 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:46.274716 10946 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:46.281267 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:46.280983 10950 raft_consensus.cc:2393] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 19: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:46.219663 17105 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:46.234956 17052 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:46.298610 10948 raft_consensus.cc:1240] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 14. Current term is 16. Ops: []
I20260709 13:18:46.293533 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:46.296241 10945 raft_consensus.cc:1275] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:18:46.303386 10949 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 16.
W20260709 13:18:46.303412 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 381922us
I20260709 13:18:46.195608 17248 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:46.248533 17218 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:18:46.305655 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:46.245590 16935 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:46.219295 17251 consensus_queue.cc:1059] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.113s
I20260709 13:18:46.261601 17093 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:46.280910 17257 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:46.289841 17134 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:46.249464 17229 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:46.317312 10841 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:46.318986 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:46.290833 17101 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:46.300351 17105 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:18:46.337602 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:46.339308 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:46.351312 11503 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4"
dest_uuid: "12682eb70bc84c76879e04a8628ee586"
mode: GRACEFUL
new_leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
 from {username='slave'} at 127.0.0.1:57818
I20260709 13:18:46.301713 17232 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:46.249111 17169 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:46.311844 17218 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:46.344424 17257 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:46.351927 11503 raft_consensus.cc:606] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Received request to transfer leadership to TS 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:46.357692 10841 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:46.359861 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 128861us
I20260709 13:18:46.325533 17093 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:46.364493 11503 raft_consensus.cc:614] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:46.338729 17134 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:46.348658 17229 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:46.290537 17237 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:46.340257 16907 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:46.369936 10946 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 16.
W20260709 13:18:46.362505 17030 consensus_meta.cc:287] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.705s	user 0.008s	sys 0.003s
I20260709 13:18:46.372457 17266 raft_consensus.cc:2749] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:46.373840 17257 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:46.376103 17169 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:46.379238 17243 log.cc:927] Time spent T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
W20260709 13:18:46.380092 17258 log.cc:927] Time spent T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:18:46.303006 17233 consensus_queue.cc:237] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:46.382395 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:46.374675 17249 consensus_queue.cc:1048] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 13:18:46.367484 17232 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:46.311599 17052 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:46.388214 16871 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:46.390111 17093 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:46.390836 10841 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:46.378134 17075 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:46.352259 16935 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:46.393895 11503 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665"
dest_uuid: "12682eb70bc84c76879e04a8628ee586"
mode: GRACEFUL
new_leader_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:57818
I20260709 13:18:46.363972 17218 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:46.362368 17250 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:46.401039 11477 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:46.371543 17134 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:18:46.403761 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 173127us
I20260709 13:18:46.421388 11478 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:46.405586 11508 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 20.
I20260709 13:18:46.391633 17264 raft_consensus.cc:2749] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:18:46.409332 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 94901.9us
I20260709 13:18:46.424929 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:46.416563 11503 raft_consensus.cc:606] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Received request to transfer leadership to TS cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:46.394845 17063 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:46.403227 17076 consensus_queue.cc:1059] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.015s
I20260709 13:18:46.407817 17139 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:46.423633 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:46.434422 10949 raft_consensus.cc:2393] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 21: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:46.434341 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:46.426172 11503 raft_consensus.cc:614] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:46.428012 10767 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:46.428239 11478 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:46.429600 11477 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:46.435683 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:46.402958 17160 consensus_queue.cc:1048] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 13:18:46.437718 10841 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:46.421326 16871 raft_consensus.cc:2749] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:46.448004 10767 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:46.442317 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:46.420729 16935 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:46.427273 17259 consensus_queue.cc:1048] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 13:18:46.443161 17270 consensus_queue.cc:1048] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.005s
I20260709 13:18:46.439466 10873 raft_consensus.cc:2393] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 16: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:46.448455 10841 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:46.448858 10875 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:46.452102 10916 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:46.477813 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:46.478433 10873 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:46.473040 10841 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:46.430974 17182 raft_consensus.cc:2804] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:18:46.481706 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 52174.6us
W20260709 13:18:46.482225 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 54071.4us
I20260709 13:18:46.483659 10916 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:46.500269 10841 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:46.447233 16557 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:46.449460 17264 raft_consensus.cc:2749] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:46.425940 17250 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:46.453481 17101 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:18:46.529045 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 45468.6us
I20260709 13:18:46.436539 17263 consensus_queue.cc:1059] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.053s
I20260709 13:18:46.489537 17204 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:18:46.535329 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:46.484202 17182 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:46.536814 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:46.583312 11503 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 16.
I20260709 13:18:46.586983 11503 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34"
dest_uuid: "12682eb70bc84c76879e04a8628ee586"
mode: GRACEFUL
new_leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
 from {username='slave'} at 127.0.0.1:57818
I20260709 13:18:46.532814 16557 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:46.556104 16911 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:46.488552 10875 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 20.
I20260709 13:18:46.477396 17139 raft_consensus.cc:3055] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:18:46.483371 17133 raft_consensus.cc:2749] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:46.513967 11478 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
W20260709 13:18:46.612150 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 98310.1us
I20260709 13:18:46.587591 17182 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:46.473728 17271 raft_consensus.cc:2749] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:46.557806 10946 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 18.
I20260709 13:18:46.536073 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:46.559592 16935 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:46.559738 11506 raft_consensus.cc:1240] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:18:46.560508 10949 raft_consensus.cc:1240] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 21. Current term is 22. Ops: []
I20260709 13:18:46.561124 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:46.561337 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:46.564646 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:46.618846 11509 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 16.
I20260709 13:18:46.530614 17076 raft_consensus.cc:2804] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:46.567183 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:46.527992 16871 raft_consensus.cc:2804] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:46.517505 17266 raft_consensus.cc:2749] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:46.453212 17108 consensus_queue.cc:1048] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.082s
I20260709 13:18:46.540489 10873 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 18.
I20260709 13:18:46.530109 17272 raft_consensus.cc:2804] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:46.544859 17109 raft_consensus.cc:2749] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:46.599905 17089 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:46.600456 17118 consensus_queue.cc:1059] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:18:46.601300 17263 raft_consensus.cc:3055] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:18:46.596814 16845 consensus_queue.cc:1048] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 13:18:46.606351 17248 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:46.607607 17075 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:46.608130 16969 consensus_meta.cc:287] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.940s	user 0.012s	sys 0.003s
I20260709 13:18:46.434262 16375 consensus_queue.cc:1048] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 13:18:46.613070 11478 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:46.627126 17272 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:46.615571 10947 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 18.
W20260709 13:18:46.569183 17247 log.cc:927] Time spent T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.251s	user 0.001s	sys 0.000s
I20260709 13:18:46.557700 17101 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:46.611486 17139 raft_consensus.cc:740] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:46.611804 16557 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:46.624125 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:46.614697 17212 raft_consensus.cc:2804] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:46.637902 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:46.679484 10946 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
I20260709 13:18:46.679587 17212 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:46.636857 17263 raft_consensus.cc:740] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:46.680099 17212 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:46.622318 16871 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:46.680600 16871 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:46.680913 17263 consensus_queue.cc:260] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:46.682219 17263 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:46.613615 16911 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:46.684154 10949 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
W20260709 13:18:46.669185 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 56189.2us
I20260709 13:18:46.685114 11478 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:46.685894 17280 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:18:46.686056 11478 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:46.686326 17280 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:46.686697 17280 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:46.686998 17271 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:46.687388 17271 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:46.687772 17271 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:46.689553 17092 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:46.556165 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:18:46.673401 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:46.691116 11508 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
I20260709 13:18:46.664101 17093 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:46.621196 17076 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:46.673401 17169 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:46.673862 17063 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:46.719892 17139 consensus_queue.cc:260] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:46.713053 17169 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:46.678400 17282 raft_consensus.cc:3055] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:18:46.671619 16969 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:46.673750 17272 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:46.728780 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:46.728121 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:46.735404 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:18:46.722755 17011 consensus_meta.cc:287] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.825s	user 0.009s	sys 0.002s
I20260709 13:18:46.723768 17169 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:46.713440 17076 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:46.724169 17101 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:46.748045 17052 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:46.726488 17164 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:46.749317 17164 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:46.752635 17218 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:46.752983 17212 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:46.753806 17184 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:18:46.723925 17282 raft_consensus.cc:740] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:46.755000 17212 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:46.755789 17282 consensus_queue.cc:260] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:46.756978 17262 log.cc:927] Time spent T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.364s	user 0.001s	sys 0.000s
I20260709 13:18:46.757071 17282 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:46.757771 17105 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:46.728873 17188 consensus_queue.cc:1048] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
W20260709 13:18:46.758922 17277 log.cc:927] Time spent T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.159s	user 0.002s	sys 0.000s
I20260709 13:18:46.760051 17105 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:46.760743 17164 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:46.760887 17134 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:46.762157 17218 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:46.719904 17237 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:46.722503 17182 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:46.762861 17250 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:46.763876 17253 log.cc:927] Time spent T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.411s	user 0.001s	sys 0.001s
I20260709 13:18:46.768088 17139 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:46.768576 17182 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:46.769073 17210 raft_consensus.cc:3055] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:18:46.769578 17210 raft_consensus.cc:740] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:46.770569 17210 consensus_queue.cc:260] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:46.772214 17210 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:46.765822 17232 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:46.774920 17228 consensus_meta.cc:287] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.615s	user 0.012s	sys 0.000s
I20260709 13:18:46.775875 17134 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:46.776681 17229 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:46.778880 17229 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:46.780519 17272 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:46.782738 17272 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:46.787495 17280 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:46.789628 17280 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:46.725311 17271 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:46.820454 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:46.821211 11504 raft_consensus.cc:1240] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
W20260709 13:18:46.627571 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:46.827771 10874 raft_consensus.cc:2410] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 20 because replica has last-logged OpId of term: 19 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 13:18:46.833024 11510 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 19.
W20260709 13:18:46.824220 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:46.618966 10870 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:46.841547 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:46.845651 10766 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:46.846320 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:46.848969 17204 raft_consensus.cc:697] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 19 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:46.727070 16969 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:46.824297 11507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56416 (request call id 980) took 1220 ms. Trace:
I20260709 13:18:46.852311 17011 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:46.825771 11508 raft_consensus.cc:1240] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 15. Current term is 16. Ops: []
W20260709 13:18:46.609943 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 162104us
I20260709 13:18:46.855904 10867 raft_consensus.cc:2393] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 19: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:46.856647 17204 consensus_queue.cc:237] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:46.858011 10767 leader_election.cc:400] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 15. Current term is 16.
W20260709 13:18:46.667797 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 73967.8us
I20260709 13:18:46.859697 10946 raft_consensus.cc:2410] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:18:46.860189 10916 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:46.852042 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:46.861183 17011 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:46.862406 10872 raft_consensus.cc:2393] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 21: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:46.863436 16969 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:46.864209 16969 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:46.864878 16969 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:46.827745 10941 raft_consensus.cc:2368] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 20. Current term is 21.
I20260709 13:18:46.867198 16911 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:46.871706 17271 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:46.874706 11505 raft_consensus.cc:1275] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:18:46.878504 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:46.852663 11507 rpcz_store.cc:276] 0709 13:18:45.603368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:45.603530 (+   162us) service_pool.cc:224] Handling call
0709 13:18:45.634042 (+ 30512us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:45.680478 (+ 46436us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:45.680533 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:45.681021 (+   488us) log.cc:844] Serialized 26 byte log entry
0709 13:18:45.945374 (+264353us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:18:45.945426 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:45.945512 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:46.568449 (+622937us) spinlock_profiling.cc:243] Waited 70 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:18:46.823078 (+254629us) raft_consensus.cc:1685] finished
0709 13:18:46.823142 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:46.823934 (+   792us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":160981248,"thread_start_us":264004,"threads_started":1,"wal-append.queue_time_us":577915}
I20260709 13:18:46.884630 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:46.886835 17011 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:46.887614 17011 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:46.889544 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:46.889801 10869 raft_consensus.cc:1240] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 21. Current term is 22. Ops: []
I20260709 13:18:46.890583 10873 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 18.
I20260709 13:18:46.892796 10875 raft_consensus.cc:1240] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 20. Current term is 21. Ops: []
I20260709 13:18:46.766466 17089 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:46.927117 17229 raft_consensus.cc:2804] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:46.927759 17229 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:46.928287 17229 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:46.928900 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:46.930047 10947 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 22.
I20260709 13:18:46.931563 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:46.933725 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:18:46.938565 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 93516.1us
I20260709 13:18:46.940322 10766 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:46.941913 17231 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:46.943775 17233 raft_consensus.cc:2804] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:46.944293 17233 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:46.944801 17233 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:46.945536 10799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2962) took 1595 ms. Trace:
I20260709 13:18:46.946949 17231 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:46.947782 10871 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49290 (request call id 979) took 2673 ms (client timeout 1691 ms). Trace:
W20260709 13:18:46.948419 10871 rpcz_store.cc:269] 0709 13:18:44.274014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:44.417272 (+143258us) service_pool.cc:224] Handling call
0709 13:18:46.947333 (+2530061us) spinlock_profiling.cc:243] Waited 1.68 s on lock 0x7b5400078854. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:46.947584 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3872975744}
W20260709 13:18:46.949589 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 89819.7us
I20260709 13:18:46.950713 17231 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:46.950940 10868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2480) took 1419 ms. Trace:
I20260709 13:18:46.957211 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:46.957894 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 100530us
I20260709 13:18:46.958427 10944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41014 (request call id 2571) took 1320 ms. Trace:
I20260709 13:18:46.987283 10868 rpcz_store.cc:276] 0709 13:18:45.531082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:45.531624 (+   542us) service_pool.cc:224] Handling call
0709 13:18:45.548796 (+ 17172us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:45.894952 (+346156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:46.950263 (+1055311us) spinlock_profiling.cc:243] Waited 333 ms on lock 0x7b540009a1d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:46.950689 (+   426us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":765556480}
I20260709 13:18:46.988384 10944 rpcz_store.cc:276] 0709 13:18:45.637485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:45.642495 (+  5010us) service_pool.cc:224] Handling call
0709 13:18:46.958046 (+1315551us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b5400376754. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:46.958221 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2685283584}
I20260709 13:18:46.989738 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:46.961966 10916 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:46.992753 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:46.995536 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:46.996326 10793 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 21.
I20260709 13:18:46.963440 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:46.997946 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:46.964633 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 2313) took 1112 ms. Trace:
I20260709 13:18:46.965987 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:46.999121 10867 rpcz_store.cc:276] 0709 13:18:45.851735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:45.868204 (+ 16469us) service_pool.cc:224] Handling call
0709 13:18:46.964351 (+1096147us) spinlock_profiling.cc:243] Waited 986 ms on lock 0x7b54002d3f54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:46.964552 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2268100992}
I20260709 13:18:46.999439 10793 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 15.
I20260709 13:18:46.965999 10941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41014 (request call id 2568) took 1329 ms. Trace:
I20260709 13:18:46.967250 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:46.968693 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:47.002085 10873 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:47.002278 10794 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:46.972128 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:47.001753 10941 rpcz_store.cc:276] 0709 13:18:45.636439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:45.660281 (+ 23842us) service_pool.cc:224] Handling call
0709 13:18:46.965765 (+1305484us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54003aff54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:46.965937 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2591924608}
I20260709 13:18:46.974879 16557 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:46.987700 10869 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 16.
I20260709 13:18:46.997948 10876 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:47.006259 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:47.007198 10871 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:47.008073 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:46.945950 10799 rpcz_store.cc:276] 0709 13:18:45.350466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:45.350738 (+   272us) service_pool.cc:224] Handling call
0709 13:18:45.351393 (+   655us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:46.103201 (+751808us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:46.944934 (+841733us) spinlock_profiling.cc:243] Waited 728 ms on lock 0x7b540024e5d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:46.945282 (+   348us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1673618816}
I20260709 13:18:47.009461 16557 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:47.010370 10793 raft_consensus.cc:1240] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:47.011152 17228 consensus_queue.cc:1048] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:47.013408 10841 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:47.005102 10796 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:46.980062 10874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 2327) took 1022 ms. Trace:
I20260709 13:18:47.042239 10874 rpcz_store.cc:276] 0709 13:18:45.957446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:45.957588 (+   142us) service_pool.cc:224] Handling call
0709 13:18:46.979703 (+1022115us) spinlock_profiling.cc:243] Waited 792 ms on lock 0x7b5400077454. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:46.979946 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1822287232}
I20260709 13:18:47.044075 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:47.046540 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:47.047906 10941 raft_consensus.cc:2393] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 19: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:47.024231 16911 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:47.048758 16911 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:47.024950 17065 raft_consensus.cc:2749] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:47.050973 10795 raft_consensus.cc:1240] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:47.052423 16911 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:47.053881 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:47.056109 17228 raft_consensus.cc:2804] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:47.025579 16907 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:47.071074 16907 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:47.074329 16907 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:47.028128 10793 raft_consensus.cc:1275] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:18:47.031098 10792 raft_consensus.cc:1240] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:47.076143 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:47.078050 10792 raft_consensus.cc:1240] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 18. Current term is 19. Ops: []
I20260709 13:18:47.039558 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:47.079084 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:47.047780 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:47.079895 10941 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
W20260709 13:18:46.964728 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:47.085075 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:47.004325 11478 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:47.086378 17280 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:46.941936 10766 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:47.070108 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:47.026201 17030 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:47.076225 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:47.059918 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:47.063797 10841 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:47.091966 10795 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
W20260709 13:18:47.109071 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 167348us
I20260709 13:18:47.068642 17011 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:47.129014 17076 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:47.121196 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 66657.6us
W20260709 13:18:47.130432 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 141749us
W20260709 13:18:47.131896 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 181145us
I20260709 13:18:47.142838 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:47.144920 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:47.143112 17273 raft_consensus.cc:2804] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:47.123306 17228 raft_consensus.cc:697] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:47.159814 17076 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:47.165494 16871 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:47.147800 17273 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:47.167655 16986 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:47.167737 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:47.114076 17030 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:47.175137 17269 raft_consensus.cc:2804] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:47.178620 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:47.175138 10841 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:47.191490 10794 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 18.
I20260709 13:18:47.187206 10798 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
W20260709 13:18:47.200327 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:47.193516 17273 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:47.223214 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:47.215180 10767 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:47.226085 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:18:47.226239 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 51741us
W20260709 13:18:47.235101 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:47.236647 10945 raft_consensus.cc:2368] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 15. Current term is 16.
W20260709 13:18:47.238547 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 234549us
I20260709 13:18:47.248041 16986 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:47.251832 16871 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:47.259414 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 134437us
I20260709 13:18:47.263892 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:47.253437 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:47.267148 17158 raft_consensus.cc:2804] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:18:47.267652 17158 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:47.268071 17158 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:47.270208 10792 raft_consensus.cc:1240] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:47.272329 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:47.276706 10793 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:47.279685 10876 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 16.
I20260709 13:18:47.281416 10946 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 16.
W20260709 13:18:47.285890 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 107566us
W20260709 13:18:47.287526 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:47.291703 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:47.292657 10948 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
W20260709 13:18:47.294970 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:47.295249 10950 raft_consensus.cc:1240] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 20. Current term is 21. Ops: []
I20260709 13:18:47.296296 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:47.297174 10795 raft_consensus.cc:2393] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 19: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:47.297484 10841 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:47.298475 10949 raft_consensus.cc:1240] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 20. Current term is 21. Ops: [20.14-20.14]
W20260709 13:18:47.298708 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 44599.9us
I20260709 13:18:47.299909 17232 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:47.300241 10841 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:47.300509 17232 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:47.301069 17232 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:47.301333 17032 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:47.301384 10841 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:47.301906 17032 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:47.302312 17032 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:47.303208 17105 raft_consensus.cc:2764] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader election decision vote started in defunct term 22: won
I20260709 13:18:47.306335 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:47.307047 10798 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 22.
I20260709 13:18:47.312276 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:47.265856 17269 raft_consensus.cc:697] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:47.317534 10793 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 21.
I20260709 13:18:47.318670 16632 raft_consensus.cc:2749] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:18:47.322522 10873 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 20.
I20260709 13:18:47.336040 17011 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:47.336982 17011 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:47.339330 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:18:47.350149 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 48834.1us
I20260709 13:18:47.350607 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:47.351138 10841 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:47.354827 10766 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:18:47.359484 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 165421us
W20260709 13:18:47.360746 17169 consensus_meta.cc:287] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.594s	user 0.007s	sys 0.007s
I20260709 13:18:47.361426 17169 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:47.364423 17169 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:47.365811 11506 raft_consensus.cc:2393] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 17: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
W20260709 13:18:47.367027 17210 consensus_meta.cc:287] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.533s	user 0.010s	sys 0.014s
I20260709 13:18:47.368482 10870 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
I20260709 13:18:47.352537 16632 raft_consensus.cc:2749] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:18:47.382236 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 56833.7us
I20260709 13:18:47.379981 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:47.386439 10871 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 18.
I20260709 13:18:47.379508 17231 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:47.393473 10916 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:47.410964 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:47.399992 17231 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:47.429590 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:47.396737 16986 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:47.430047 10842 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:47.435480 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:47.372053 17228 consensus_queue.cc:237] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:47.437672 17282 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:47.437885 17139 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:47.438414 10699 catalog_manager.cc:5697] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 14 to 18. New cstate: current_term: 18 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:47.430974 17231 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:47.460196 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:47.452857 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:47.465839 10945 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:47.470104 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:18:47.475279 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 144328us
W20260709 13:18:47.476646 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 190064us
W20260709 13:18:47.487476 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 93231.4us
I20260709 13:18:47.491050 17089 consensus_queue.cc:1059] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:47.492753 17089 raft_consensus.cc:3055] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:47.493237 17089 raft_consensus.cc:740] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:47.494470 17089 consensus_queue.cc:260] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:47.496560 17089 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:18:47.508862 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 152686us
W20260709 13:18:47.541455 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:47.543593 11477 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:47.545178 17169 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:47.545727 17169 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:47.546206 17169 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:47.506196 17011 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:47.573705 10766 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:47.468328 17139 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:47.580199 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 44499.9us
W20260709 13:18:47.582916 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 66694.7us
W20260709 13:18:47.554124 17229 consensus_meta.cc:287] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.625s	user 0.009s	sys 0.005s
I20260709 13:18:47.586206 17134 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:47.524210 17282 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:47.590310 17293 log.cc:927] Time spent T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.291s	user 0.002s	sys 0.003s
I20260709 13:18:47.590998 17282 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:47.474182 17269 consensus_queue.cc:237] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:47.596591 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:47.585191 17233 consensus_meta.cc:287] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.640s	user 0.005s	sys 0.010s
W20260709 13:18:47.612986 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 79096.1us
I20260709 13:18:47.614977 17089 raft_consensus.cc:2764] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:18:47.615931 17139 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:47.616217 10948 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 16.
I20260709 13:18:47.646706 11477 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:18:47.649255 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 96094.4us
I20260709 13:18:47.650539 10916 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:47.651744 17282 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:18:47.652256 17282 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:47.652732 17282 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:47.659343 10916 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:18:47.662509 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:47.667070 11478 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
W20260709 13:18:47.679276 10796 consensus_meta.cc:287] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.639s	user 0.012s	sys 0.000s
I20260709 13:18:47.679888 10796 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
I20260709 13:18:47.674086 10841 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:47.674839 10842 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:47.622004 17032 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:47.624213 17233 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:47.695196 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 49246.3us
I20260709 13:18:47.666569 16911 raft_consensus.cc:2749] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:18:47.644030 17030 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:47.702600 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:47.698123 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 54242.4us
I20260709 13:18:47.587797 17229 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:47.708261 16986 raft_consensus.cc:2749] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:18:47.719237 17229 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:47.705334 16969 consensus_meta.cc:287] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.840s	user 0.012s	sys 0.004s
I20260709 13:18:47.721347 16969 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:47.722874 16632 raft_consensus.cc:2804] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:47.723377 16632 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:47.723795 16632 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:47.723842 16969 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:47.700851 17076 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:47.724572 10699 catalog_manager.cc:5697] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 13 to 16. New cstate: current_term: 16 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:47.729597 10699 catalog_manager.cc:5697] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:18:47.731050 10800 raft_consensus.cc:1240] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 13. Current term is 14. Ops: []
W20260709 13:18:47.732095 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 77246.8us
W20260709 13:18:47.742620 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 77361us
I20260709 13:18:47.732460 17271 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:47.731230 16986 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:47.738821 11478 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:47.747195 10841 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:47.726912 17076 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:47.748145 17271 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:47.752540 17076 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:47.749315 17134 consensus_queue.cc:1059] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 13:18:47.772086 17032 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:47.751268 16871 consensus_queue.cc:1048] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:18:47.785012 17232 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:47.788376 10947 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.037s	sys 0.112s
I20260709 13:18:47.752344 17229 consensus_queue.cc:1048] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 13:18:47.764139 16986 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:47.767663 11478 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:47.740675 17273 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:47.798377 10947 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.037s	sys 0.113s
I20260709 13:18:47.804827 10945 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 17.
I20260709 13:18:47.769544 17271 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:47.759102 10699 catalog_manager.cc:5697] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 10 to 14, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 14 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
W20260709 13:18:47.806334 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 57624.8us
I20260709 13:18:47.754736 17158 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:47.816324 10842 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:47.821239 10798 raft_consensus.cc:1240] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:47.822566 10699 catalog_manager.cc:5697] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 11 to 14. New cstate: current_term: 14 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:47.813310 17032 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:18:47.830165 17032 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:47.840653 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 134226us
I20260709 13:18:47.819255 17231 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:47.848893 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 110962us
I20260709 13:18:47.843086 10699 catalog_manager.cc:5697] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 13 to 14, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 14 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:18:47.856119 10841 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:18:47.859133 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 88726.8us
I20260709 13:18:47.848089 17032 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:18:47.850435 16907 raft_consensus.cc:2804] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:18:47.864102 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 100839us
I20260709 13:18:47.862651 10766 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:47.853514 17233 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:47.869171 11478 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:47.874851 16986 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:47.875211 16557 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:47.859974 17210 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:47.860615 16871 raft_consensus.cc:2749] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:18:47.867867 17184 consensus_meta.cc:287] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 1.049s	user 0.010s	sys 0.016s
I20260709 13:18:47.878346 17232 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:47.921140 11503 raft_consensus.cc:606] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Received request to transfer leadership to TS 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:47.921553 11503 raft_consensus.cc:614] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:47.920548 10916 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:47.880594 16632 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:47.885320 16907 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:18:47.870597 10698 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42984 (request call id 247) took 40124 ms (client timeout 15000 ms). Trace:
I20260709 13:18:47.894060 17282 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:47.895291 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:47.897320 17231 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:47.903142 17303 raft_consensus.cc:2804] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:47.903774 17158 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:47.906352 17089 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:47.862342 17229 consensus_queue.cc:1059] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.050s
I20260709 13:18:47.915109 17139 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:18:47.916437 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 47580.4us
I20260709 13:18:47.861317 17228 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:18:47.933497 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 122683us
I20260709 13:18:47.935907 17089 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:47.936210 17210 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:47.947815 16911 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:18:47.942984 17089 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:47.948487 16911 raft_consensus.cc:697] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 19 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:47.949517 17210 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:47.949422 16911 consensus_queue.cc:237] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:47.938612 17165 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:47.951591 17269 raft_consensus.cc:3055] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:47.951407 17165 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:47.952010 17269 raft_consensus.cc:740] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:47.953065 17269 consensus_queue.cc:260] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:47.941581 17228 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:47.933190 16907 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:47.954610 17269 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:48.017938 17089 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:48.014256 17139 raft_consensus.cc:697] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 20 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:48.021777 17139 consensus_queue.cc:237] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:48.023932 17282 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:47.954892 17228 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:48.031010 17273 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:48.031176 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 110780us
I20260709 13:18:47.960095 17134 consensus_queue.cc:1059] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.072s
I20260709 13:18:47.933028 17076 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:48.032639 17306 raft_consensus.cc:3055] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:18:48.033075 17306 raft_consensus.cc:740] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:47.935696 16557 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:48.034178 17306 consensus_queue.cc:260] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:48.035995 17306 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:48.036185 17165 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:48.036337 16557 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:48.037496 17089 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:47.968537 17158 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:48.037839 17308 consensus_queue.cc:1059] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:18:48.020476 17310 raft_consensus.cc:2804] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:18:48.019928 17271 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:48.034230 10916 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
W20260709 13:18:48.045933 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 102011us
I20260709 13:18:48.045799 17158 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:48.046947 10842 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:48.047595 11503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:57818 (request call id 17) took 1461 ms. Trace:
I20260709 13:18:48.048014 11503 rpcz_store.cc:276] 0709 13:18:46.586607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:46.586823 (+   216us) service_pool.cc:224] Handling call
0709 13:18:48.047311 (+1460488us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b54001d1354. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eaf853 00007f9c85e7d60a 00007f9c7f4037d4 00007f9c7f403762 00007f9c7f403691 00007f9c7f40360c 00007f9c7f4028c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:48.047472 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3066032384}
I20260709 13:18:48.049103 10842 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:48.050143 10916 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:48.050218 17232 raft_consensus.cc:2749] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:48.050976 17134 consensus_queue.cc:1059] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:18:48.051692 17165 raft_consensus.cc:3055] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:18:47.965526 17280 consensus_meta.cc:287] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.879s	user 0.012s	sys 0.000s
I20260709 13:18:47.960240 16632 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:48.048017 17011 raft_consensus.cc:2749] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:48.034379 17076 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:47.968389 16871 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:48.052863 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 157156us
I20260709 13:18:48.053485 10842 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:47.962121 17233 consensus_queue.cc:1059] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.079s
I20260709 13:18:48.058349 17280 raft_consensus.cc:2749] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:48.011508 10699 catalog_manager.cc:5697] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 9 to 11, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 11 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:48.066720 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:48.069949 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:48.050354 17271 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:48.067934 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:48.052179 17232 raft_consensus.cc:3055] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:18:48.072063 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 206157us
I20260709 13:18:48.052160 17165 raft_consensus.cc:740] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:48.078265 10947 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:48.073318 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:48.060119 17282 raft_consensus.cc:2749] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:48.082671 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:48.081821 10842 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:48.056221 17303 raft_consensus.cc:697] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 16 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:48.083659 10948 raft_consensus.cc:2410] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:18:48.069504 16969 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:18:48.083894 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:48.059607 17310 raft_consensus.cc:697] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 18 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:48.052464 17032 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:48.086495 10946 raft_consensus.cc:1240] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:48.077276 17232 raft_consensus.cc:740] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:48.064685 16871 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:48.096863 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:48.081007 16986 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:48.099504 10949 raft_consensus.cc:1240] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 19. Current term is 20. Ops: []
W20260709 13:18:47.929410 10698 rpcz_store.cc:269] 0709 13:18:07.746065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.746333 (+   268us) service_pool.cc:224] Handling call
0709 13:18:47.865791 (+40119458us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:44.023052 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:44.160396 (+137344us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:44.160450 (+    54us) write_op.cc:435] Acquired schema lock
0709 13:18:44.160478 (+    28us) tablet.cc:662] Decoding operations
0709 13:18:44.162055 (+  1577us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:44.162151 (+    96us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:44.162192 (+    41us) tablet.cc:685] Acquiring locks for 13 operations
0709 13:18:44.360796 (+198604us) tablet.cc:701] Row locks acquired
0709 13:18:44.376137 (+ 15341us) write_op.cc:260] PREPARE: finished
0709 13:18:44.405897 (+ 29760us) write_op.cc:270] Start()
0709 13:18:44.411864 (+  5967us) write_op.cc:276] Timestamp: P: 1783603124404489 usec, L: 0
0709 13:18:44.411894 (+    30us) op_driver.cc:348] REPLICATION: starting
0709 13:18:44.412618 (+   724us) log.cc:844] Serialized 5173 byte log entry
0709 13:18:45.236204 (+823586us) op_driver.cc:464] REPLICATION: finished
0709 13:18:45.375026 (+138822us) write_op.cc:301] APPLY: starting
0709 13:18:45.381910 (+  6884us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:45.457954 (+ 76044us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:45.458103 (+   149us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:45.652074 (+193971us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:45.652344 (+   270us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=26,key_file_lookups=26,delta_file_lookups=26,mrs_lookups=0
0709 13:18:45.652393 (+    49us) write_op.cc:312] APPLY: finished
0709 13:18:45.652700 (+   307us) log.cc:844] Serialized 121 byte log entry
0709 13:18:45.655839 (+  3139us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:45.656034 (+   195us) write_op.cc:454] Released schema lock
0709 13:18:45.656084 (+    50us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:45.656486 (+   402us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:45.656517 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":102901,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":112765,"apply.run_cpu_time_us":21496,"apply.run_wall_time_us":287893,"cfile_cache_hit":54,"cfile_cache_hit_bytes":58493,"num_ops":13,"prepare.queue_time_us":343118,"prepare.run_cpu_time_us":69116,"prepare.run_wall_time_us":576573,"raft.queue_time_us":390733,"raft.run_cpu_time_us":4748,"raft.run_wall_time_us":80426,"replication_time_us":824062,"thread_start_us":810414,"threads_started":4,"wal-append.queue_time_us":229538,"wal-append.run_cpu_time_us":59235,"wal-append.run_wall_time_us":1996771}]]}
I20260709 13:18:48.099581 16986 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:48.101948 16986 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:48.103219 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:48.103022 17310 consensus_queue.cc:237] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:48.103824 10949 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
I20260709 13:18:48.103876 17165 consensus_queue.cc:260] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:48.106323 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:48.102547 16986 raft_consensus.cc:2749] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:18:48.081727 16632 raft_consensus.cc:2749] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:18:48.109784 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 59769.6us
I20260709 13:18:48.107009 10943 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 22.
I20260709 13:18:48.115056 17032 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:48.117857 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:18:48.118223 17169 consensus_meta.cc:287] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.571s	user 0.009s	sys 0.005s
I20260709 13:18:48.118744 17169 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:48.119608 10699 catalog_manager.cc:5697] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 13 to 16, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 16 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:18:48.120980 10916 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:48.121703 10699 catalog_manager.cc:5697] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 12 to 14, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 14 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:48.123651 17282 raft_consensus.cc:2749] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:48.095485 17158 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:48.118656 11506 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 17.
I20260709 13:18:48.125221 11509 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb"
dest_uuid: "12682eb70bc84c76879e04a8628ee586"
mode: GRACEFUL
new_leader_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:57818
I20260709 13:18:48.126775 11509 raft_consensus.cc:606] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Received request to transfer leadership to TS cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:48.127408 11503 raft_consensus.cc:1240] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:48.127492 11509 raft_consensus.cc:614] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:48.129292 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:48.129269 17303 consensus_queue.cc:237] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:48.130167 10950 raft_consensus.cc:2368] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 16. Current term is 17.
I20260709 13:18:48.106384 17210 raft_consensus.cc:3055] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:18:48.130812 17210 raft_consensus.cc:740] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:48.131808 17210 consensus_queue.cc:260] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:48.133270 17210 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:48.133330 11509 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14"
dest_uuid: "12682eb70bc84c76879e04a8628ee586"
mode: GRACEFUL
new_leader_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:57818
I20260709 13:18:48.133913 11509 raft_consensus.cc:606] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Received request to transfer leadership to TS cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:48.134261 11509 raft_consensus.cc:614] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:48.135475  9682 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:18:48.137346 10841 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:48.138985 16632 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:18:48.139364 16632 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:48.139711 16632 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:48.141774 16871 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:48.141639 17232 consensus_queue.cc:260] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:48.143416 10949 raft_consensus.cc:1275] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:18:48.097950 16969 raft_consensus.cc:697] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:48.143440 17232 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:48.145349 16969 consensus_queue.cc:237] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:48.146266 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:48.147018 10796 raft_consensus.cc:1275] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:18:48.146266 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:48.147338 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:48.147578 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:48.148195 10797 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 16.
I20260709 13:18:48.148228 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:48.148860 10794 raft_consensus.cc:1240] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 18. Current term is 19. Ops: []
I20260709 13:18:48.148980 10948 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 16.
I20260709 13:18:48.148988 17169 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:48.149657 10798 raft_consensus.cc:1240] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 18. Current term is 19. Ops: []
I20260709 13:18:48.122431 10916 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:48.151288 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:48.154491 10949 raft_consensus.cc:2393] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 19: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:48.154843 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:48.148103 10795 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:48.141669 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:48.156594 10943 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:18:48.151569 10798 raft_consensus.cc:1275] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:18:48.151624 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:48.152011 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:48.158331 10796 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 16.
I20260709 13:18:48.152328 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:48.153347 10916 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:48.162572 10947 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
I20260709 13:18:48.164096 10841 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:18:48.164446 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 67142.5us
I20260709 13:18:48.167075 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:48.167383 10842 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:48.167866 10801 raft_consensus.cc:2393] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 21: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:48.168196 17011 raft_consensus.cc:2804] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:18:48.139645 17089 raft_consensus.cc:3055] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:48.168711 17011 raft_consensus.cc:697] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:48.168830 17089 raft_consensus.cc:740] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:48.169848 17011 consensus_queue.cc:237] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:48.169984 17089 consensus_queue.cc:260] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:48.170977 17158 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:18:48.171620 10842 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:48.174911 10800 raft_consensus.cc:2393] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 14: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:48.177495 10795 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 16.
I20260709 13:18:48.138615  9682 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:48.185334 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:48.191949 17273 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:48.192679 17273 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:48.159485 17316 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:48.195210 17316 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:48.199636 10842 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:48.199088 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:48.202338 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 338882us
I20260709 13:18:48.195240 17089 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:48.205068 10766 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:48.208423 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:48.201761 17158 raft_consensus.cc:697] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:48.211226 10699 catalog_manager.cc:5697] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 11 to 14. New cstate: current_term: 14 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:18:48.201480 17282 raft_consensus.cc:2804] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:18:48.247620 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:48.203866 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:48.206703 10766 leader_election.cc:400] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Vote denied by peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553) with higher term. Message: Invalid argument: T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 20. Current term is 21.
I20260709 13:18:48.250873 10766 leader_election.cc:403] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:48.201635 16986 raft_consensus.cc:2749] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:18:48.251683 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:48.253554 11477 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:48.198746 17316 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:48.257314 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:48.247605 17273 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:48.259464 17312 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:48.266661 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:48.252780 17032 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:18:48.264716 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 111475us
I20260709 13:18:48.251108 17105 raft_consensus.cc:2749] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:18:48.265092 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:48.274678 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:48.275800 11509 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 19: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
W20260709 13:18:48.278510 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 75557.5us
W20260709 13:18:48.277594 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 71053.2us
I20260709 13:18:48.277582 10699 catalog_manager.cc:5697] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 14 to 18, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 18 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:48.287988 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:48.276482 16871 consensus_queue.cc:1059] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:18:48.289492 11477 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:48.296721 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:48.308485 10699 catalog_manager.cc:5697] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 9 to 16. New cstate: current_term: 16 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:48.314476 10943 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 23.
I20260709 13:18:48.314196 17158 consensus_queue.cc:237] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:48.316318 10841 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:48.318274 17165 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:48.274143 16986 consensus_queue.cc:1059] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.037s
I20260709 13:18:48.295543 17282 raft_consensus.cc:697] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 21 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:48.327139 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:48.329262 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:48.290838 16907 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:48.330207 11477 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:48.334414 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:48.335054 10793 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:18:48.335539 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 54886.3us
I20260709 13:18:48.331678 11509 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 15.
I20260709 13:18:48.338438 10766 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:48.340190 17105 raft_consensus.cc:2804] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:48.340274 16907 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:48.340655 17105 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:48.341161 17105 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:48.342908 10797 raft_consensus.cc:2410] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 13:18:48.311098 17310 raft_consensus.cc:2804] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:48.308038 17231 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:48.328300 10699 catalog_manager.cc:5697] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 12 to 17, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 17 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:48.346272 10916 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:48.319061 16911 raft_consensus.cc:2764] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader election decision vote started in defunct term 22: won
W20260709 13:18:48.349287 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 93200us
W20260709 13:18:48.383844 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:48.331816 17032 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:18:48.383982 11503 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
I20260709 13:18:48.387022 10793 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 17.
I20260709 13:18:48.387588 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:48.393183 10792 raft_consensus.cc:2393] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 16: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:48.340844 17312 raft_consensus.cc:2749] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553) with higher term. Message: Invalid argument: T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 20. Current term is 21.
I20260709 13:18:48.400942 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:48.401175 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:18:48.392884 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 46692.4us
W20260709 13:18:48.402175 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:48.393961 10943 raft_consensus.cc:1275] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:18:48.399000 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:48.403765 10800 raft_consensus.cc:2393] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 18: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:48.403895 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:48.406819 10875 raft_consensus.cc:2393] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 16: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:48.324774 17228 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:48.409816 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:48.385959 17310 raft_consensus.cc:697] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 16 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:48.410944 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:48.411458 10799 raft_consensus.cc:1275] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:18:48.412637 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:48.407689 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:48.413071 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:48.414119 10870 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:48.404716 17308 raft_consensus.cc:2804] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:48.418684 10801 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 23.
I20260709 13:18:48.418051 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:48.376506 16632 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:48.407526 17032 raft_consensus.cc:697] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:48.463529 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:48.464268 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:48.483186 10798 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:18:48.483439 10801 raft_consensus.cc:2410] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 13:18:48.415442 17322 raft_consensus.cc:2749] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:18:48.453361 17304 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:48.473373 10792 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:48.473737 17323 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:48.455235 17326 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:18:48.458428 17320 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:48.476591 10796 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
I20260709 13:18:48.490250 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:48.484258 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 65572.1us
I20260709 13:18:48.391709 17231 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:48.475234 10916 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:48.386149 17282 consensus_queue.cc:237] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:48.489507 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:48.477008 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:48.490931 10876 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 22.
I20260709 13:18:48.489674 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:48.496119 10875 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:48.478125 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:18:48.479064 11478 connection.cc:256] RPC call timeout handler was delayed by -1.2719644910000625s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1647s for a response
W20260709 13:18:48.493839 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 68628.2us
I20260709 13:18:48.494509 10766 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:48.496701 10872 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:18:48.496726 11478 connection.cc:256] RPC call timeout handler was delayed by -1.2638644910000494s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1656s for a response
I20260709 13:18:48.494015 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:48.498101 17030 raft_consensus.cc:2804] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:18:48.498291 10867 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 17.
I20260709 13:18:48.481988 17308 raft_consensus.cc:697] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 17 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:48.498669 17030 raft_consensus.cc:697] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 18 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:48.499258 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:48.499926 11512 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:48.501744 10916 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:48.509059 10798 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 23.
I20260709 13:18:48.535578 10870 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 17.
I20260709 13:18:48.488392 17158 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:48.539853 10699 catalog_manager.cc:5697] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 10 to 13, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 13 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:18:48.525764 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:48.545006 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:48.572397 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 81294.2us
I20260709 13:18:48.546213 11508 raft_consensus.cc:2393] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 18: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:48.580677 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:48.576748 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:48.584364 11504 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
I20260709 13:18:48.544729 16969 raft_consensus.cc:2804] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:48.591152 10876 raft_consensus.cc:2393] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 20: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:48.581970 10699 catalog_manager.cc:5697] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 8 to 14, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 14 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:18:48.543141 17326 raft_consensus.cc:697] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 20 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:48.527403 17312 raft_consensus.cc:2749] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:18:48.594995 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:48.598150 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:48.602243 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:48.573724 17269 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:48.499647 17030 consensus_queue.cc:237] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:48.605223 10766 leader_election.cc:400] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Vote denied by peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553) with higher term. Message: Invalid argument: T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 15. Current term is 16.
I20260709 13:18:48.609418 10766 leader_election.cc:403] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:18:48.600836 11508 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 22.
I20260709 13:18:48.612808 10915 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:48.613009 17329 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:48.613415 17314 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:48.630585 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 139609us
I20260709 13:18:48.630849 16632 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:48.631932 11512 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 21.
W20260709 13:18:48.615937 17324 log.cc:927] Time spent T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.227s	user 0.001s	sys 0.000s
I20260709 13:18:48.616488 17307 raft_consensus.cc:3055] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:18:48.618206 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:48.621929 17231 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:48.592067 17158 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:48.604751 17265 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:48.623912 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:48.661552 11504 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:48.593297 16907 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:48.627190 11506 raft_consensus.cc:2410] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 21 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 13:18:48.627847 10793 raft_consensus.cc:2410] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 16 because replica has last-logged OpId of term: 15 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 13:18:48.636750 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:48.649443 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:48.653955 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 62793.6us
I20260709 13:18:48.662953 11512 raft_consensus.cc:2368] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 16. Current term is 17.
I20260709 13:18:48.660749 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:48.605870 16969 raft_consensus.cc:697] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:48.663559 10798 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:48.664525 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:48.664954 10699 catalog_manager.cc:5697] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 14 to 15, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 15 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:18:48.608163 17228 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:48.651650 17307 raft_consensus.cc:740] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:48.667906 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:48.667827 16907 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:48.667662 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:48.671911 10867 raft_consensus.cc:2393] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 17: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:48.673147 10873 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 18.
I20260709 13:18:48.674315 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:48.682507 10766 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:48.688266 10699 catalog_manager.cc:5697] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 12 to 14, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 14 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:48.665130 17269 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:48.692354 17269 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:48.667665 17265 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:48.607613 17032 consensus_queue.cc:237] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:48.698680 17158 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:48.699199 16907 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:48.667596 17326 consensus_queue.cc:237] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:48.701289 11512 consensus_queue.cc:237] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:48.703352 17089 raft_consensus.cc:2804] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:48.703802 17089 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:48.704145 17089 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:48.705209 17210 consensus_queue.cc:1059] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 17.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:18:48.705745 10766 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:48.706967 17273 raft_consensus.cc:3055] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:18:48.707361 17273 raft_consensus.cc:740] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:48.707484 17316 raft_consensus.cc:2764] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 13:18:48.708325 17273 consensus_queue.cc:260] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:48.709564 17273 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:48.711767 10699 catalog_manager.cc:5697] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 12 to 16, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 16 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:48.706646 17210 consensus_queue.cc:1059] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:48.712822 17307 consensus_queue.cc:260] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:48.713666 17316 raft_consensus.cc:3055] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:18:48.713903 17307 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:48.714076 17316 raft_consensus.cc:740] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:48.686995 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:48.715245 17316 consensus_queue.cc:260] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:48.716941 17316 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:48.583039 17323 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:48.720686 17323 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:48.628762 17329 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:48.722174 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:48.722815 17329 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:48.629572 17314 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:48.546368 17320 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:48.732388 10792 raft_consensus.cc:1275] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:18:48.546125 17304 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:48.736802 17304 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:48.737560 17231 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553) with higher term. Message: Invalid argument: T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 15. Current term is 16.
I20260709 13:18:48.592612 17310 consensus_queue.cc:237] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:48.664710 16911 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:48.748931 16911 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:48.749352 16911 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:48.624338 17308 consensus_queue.cc:237] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:48.756108 17306 raft_consensus.cc:3055] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:48.756565 17306 raft_consensus.cc:740] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:48.757591 17306 consensus_queue.cc:260] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:48.759716 17320 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:48.760579 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:48.706705 10766 leader_election.cc:400] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 election: Vote denied by peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553) with higher term. Message: Invalid argument: T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 16. Current term is 17.
I20260709 13:18:48.763319 10766 leader_election.cc:403] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:18:48.740454 10872 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 19.
I20260709 13:18:48.745885 17327 consensus_queue.cc:1059] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:18:48.777717 10868 raft_consensus.cc:1240] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 19. Current term is 20. Ops: [18.13-19.14]
I20260709 13:18:48.747305 16969 consensus_queue.cc:237] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:48.748268 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:48.770587 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:18:48.779088 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 72511.6us
W20260709 13:18:48.793560 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:48.796967 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:48.780031 11504 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
I20260709 13:18:48.785112 10946 raft_consensus.cc:1240] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 16. Current term is 17. Ops: []
W20260709 13:18:48.803117 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:48.806131 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:48.809446 10766 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:48.809854 10871 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 16.
I20260709 13:18:48.809489 10876 raft_consensus.cc:1240] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 19. Current term is 20. Ops: []
I20260709 13:18:48.810328 10798 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 17.
I20260709 13:18:48.813006 10872 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 17.
I20260709 13:18:48.814240 10867 raft_consensus.cc:1240] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 19. Current term is 21. Ops: []
I20260709 13:18:48.822237 10870 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 19.
I20260709 13:18:48.823812 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:48.824857 10869 raft_consensus.cc:1240] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 17. Current term is 18. Ops: [15.13-17.14]
W20260709 13:18:48.826262 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 324600us
I20260709 13:18:48.900306 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:48.911602 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:48.778627 17105 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:48.830526 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:48.833125 10868 raft_consensus.cc:1240] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 15. Current term is 16. Ops: [15.14-15.14]
I20260709 13:18:48.814512 17210 consensus_queue.cc:1048] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.088s
I20260709 13:18:48.837946 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:48.838268 10946 raft_consensus.cc:1275] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:18:48.926703 10945 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:48.926946 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:48.805605 17306 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:48.783659 17316 raft_consensus.cc:2749] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553) with higher term. Message: Invalid argument: T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 16. Current term is 17.
I20260709 13:18:48.846014 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:48.846354 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:18:48.877364 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 275127us
I20260709 13:18:48.879944 17327 raft_consensus.cc:3055] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:18:48.880586 16907 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:18:48.892912 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 83525.8us
I20260709 13:18:48.834009 17089 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:48.957034 10949 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:48.922850 10870 raft_consensus.cc:1240] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:48.923528 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:48.838624 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:48.930029 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:48.932276 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:48.937170 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:48.958761 17089 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:48.959050 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:48.968328 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:48.969022 10876 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 19.
I20260709 13:18:48.969985 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:48.959663 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:48.971567 10870 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:48.963498 10871 raft_consensus.cc:1240] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 18. Current term is 19. Ops: []
I20260709 13:18:48.976294 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:48.977144 10871 raft_consensus.cc:2393] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 18: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:48.978047 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:48.965189 17350 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:48.979559 17350 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:48.982218 17350 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:48.964692 10949 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 19.
I20260709 13:18:48.965396 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:48.936322 11478 connection.cc:256] RPC call timeout handler was delayed by -1.1432644909998544s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17900000000000002s for a response
I20260709 13:18:48.984390 10869 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 16.
W20260709 13:18:48.984431 11478 connection.cc:256] RPC call timeout handler was delayed by -1.0991644910000105s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1839s for a response
I20260709 13:18:48.985498 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:48.965780 10943 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
I20260709 13:18:48.986616 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:48.987351 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:48.987463 17089 raft_consensus.cc:2764] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:18:48.988641 17308 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:48.986680 11510 raft_consensus.cc:1240] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 18. Current term is 19. Ops: []
I20260709 13:18:48.989311 17308 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:48.991395 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:48.992141 11511 raft_consensus.cc:2393] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 18: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:48.988772 11507 raft_consensus.cc:1240] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 18 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:48.965063 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:48.997715 10793 raft_consensus.cc:1240] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:49.001191 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:18:49.005213 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:49.006909 10870 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 18.
I20260709 13:18:49.012080 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:49.011893 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:49.008311 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:18:49.008852 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:49.017810 10870 raft_consensus.cc:1240] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:49.018627 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:49.020737 11508 raft_consensus.cc:2393] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 16: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:49.021180 10793 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:49.023612 10944 raft_consensus.cc:2393] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 17: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:49.024099 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:49.025674 17308 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:49.026667 10870 raft_consensus.cc:1275] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 13:18:49.028157 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:49.032667 16911 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:49.034963 10767 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:49.008235 16907 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:49.037096 11510 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 19.
I20260709 13:18:49.039067 11510 raft_consensus.cc:1240] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 16 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 15. Current term is 16. Ops: [15.14-15.14]
I20260709 13:18:49.039139 10870 raft_consensus.cc:1240] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:49.039273 10945 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 19.
I20260709 13:18:48.944960 17327 raft_consensus.cc:740] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:49.044822 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:49.046093 10871 raft_consensus.cc:1240] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:49.046825 17349 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:49.048012 16907 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:49.048038 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:49.075125 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:49.080590 10870 raft_consensus.cc:2393] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 16: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:49.067538 10767 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:49.007347 17299 log.cc:927] Time spent T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:18:49.066367 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:49.081365 10798 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 16.
I20260709 13:18:49.069404 10797 raft_consensus.cc:1275] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 13:18:49.076176 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:49.057823 17105 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:49.083918 10871 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 18.
I20260709 13:18:49.086480 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:49.088024 10794 raft_consensus.cc:1275] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 20 index: 9. (index mismatch)
I20260709 13:18:49.088534 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:49.089038 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:49.089849 10792 raft_consensus.cc:2410] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:18:49.091931 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:49.088455 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:49.094264 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:49.095510 10870 raft_consensus.cc:1240] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:18:49.096693 10870 raft_consensus.cc:1275] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 13:18:49.097813 10870 raft_consensus.cc:1275] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
W20260709 13:18:49.099958 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:49.101164 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:49.103376 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:49.105188 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:49.105867 10870 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 21.
I20260709 13:18:48.929060 10841 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:49.112267 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:49.113219 10792 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 17.
I20260709 13:18:49.122773 16907 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:49.127287 11510 raft_consensus.cc:1240] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 18. Current term is 19. Ops: []
I20260709 13:18:49.127622 17320 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:18:49.128767 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 199793us
I20260709 13:18:49.127365 16907 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:49.136736 17320 raft_consensus.cc:2749] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:49.138604 16907 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:49.123437 17338 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:49.123824 16969 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:49.140523 17338 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:49.141278 16969 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:49.139863 10841 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:49.142381 17320 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:18:49.142767 17320 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:49.142997 17338 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:49.143108 17320 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:18:49.143219 10841 leader_election.cc:400] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 16. Current term is 17.
I20260709 13:18:49.143419 16969 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:49.143826 10841 leader_election.cc:403] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:18:49.144372 17158 raft_consensus.cc:2749] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 16. Current term is 17.
I20260709 13:18:49.145695 17228 consensus_queue.cc:1048] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:18:49.122447 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 54987us
I20260709 13:18:49.146940 10767 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:49.148088 17354 raft_consensus.cc:2804] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:49.148591 17354 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:49.149083 17354 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:49.149684 17105 consensus_queue.cc:1059] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 19.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:49.150507 17105 consensus_queue.cc:1243] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer 468a8fd46e824f62b9c2c0d3532a6d26 log is divergent from this leader: its last log entry 20.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:18:49.151161 17089 raft_consensus.cc:3055] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:18:49.151096 17320 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:49.151753 17089 raft_consensus.cc:740] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:49.152689 17089 consensus_queue.cc:260] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:49.153390 17320 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:49.153925 17089 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:49.123378 10915 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:49.155380 17228 consensus_queue.cc:1048] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:49.156366 17338 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:49.156925 17338 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:49.157361 17338 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:49.161124 10944 raft_consensus.cc:1240] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 20. Current term is 21. Ops: []
I20260709 13:18:49.161453 10943 raft_consensus.cc:1240] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 19. Current term is 20. Ops: []
I20260709 13:18:49.161352 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:49.162592 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:49.154589 10767 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:49.163817 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:49.163972 17273 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:49.164539 17273 raft_consensus.cc:697] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:49.165222 10941 raft_consensus.cc:1240] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 18. Current term is 19. Ops: []
I20260709 13:18:49.165401 10767 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:49.165518 17273 consensus_queue.cc:237] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:49.166339 17316 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:18:49.125396 10842 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:49.175390 17320 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:49.176647 17320 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:49.179486 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 54568.6us
I20260709 13:18:49.115010 17352 raft_consensus.cc:2749] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:49.180836 17341 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:49.191177 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:49.191111 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:49.188886 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:49.195483 10867 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
I20260709 13:18:49.182723 10916 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
W20260709 13:18:49.201413 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 46224.2us
I20260709 13:18:49.183256 17339 raft_consensus.cc:2749] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:18:49.204663 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:49.204980 10916 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:49.205897 10871 raft_consensus.cc:2393] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 18: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:49.183959 17352 consensus_queue.cc:1059] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.020s
I20260709 13:18:49.206756 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:49.206827 10916 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:49.208432 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:49.209215 10871 raft_consensus.cc:2393] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 15: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:49.207123 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:49.210229 10793 raft_consensus.cc:2410] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 21 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 9.
I20260709 13:18:49.212774 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:49.213552 10792 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
I20260709 13:18:49.209388 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:49.214928 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:49.219509 10800 raft_consensus.cc:2393] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 15: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:49.220611 10796 raft_consensus.cc:1275] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:18:49.223367 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:49.189517 17320 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:49.225868 10841 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:18:49.247592 10875 consensus_meta.cc:287] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.723s	user 0.010s	sys 0.000s
I20260709 13:18:49.248067 10875 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 17.
I20260709 13:18:49.248853 10873 raft_consensus.cc:1240] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:18:49.231061 17366 consensus_queue.cc:1059] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:49.232111 17352 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:49.251627 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:49.252063 17352 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:49.255421 10950 raft_consensus.cc:1275] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:18:49.255667 17352 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:49.232846 10872 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 17.
I20260709 13:18:49.225171 17329 raft_consensus.cc:2804] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:49.233655 10867 raft_consensus.cc:2393] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 21: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:18:49.234028 17332 log.cc:927] Time spent T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.653s	user 0.002s	sys 0.002s
I20260709 13:18:49.239131 10801 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 21.
I20260709 13:18:49.240213 10870 raft_consensus.cc:1240] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:49.258363 17366 raft_consensus.cc:3055] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:49.258872 17366 raft_consensus.cc:740] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:49.256868 17352 raft_consensus.cc:3055] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:18:49.261183 17352 raft_consensus.cc:740] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:49.262274 17352 consensus_queue.cc:260] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:49.263968 17352 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:18:49.268023 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 259124us
I20260709 13:18:49.221688 16907 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:18:49.206986 17353 raft_consensus.cc:2804] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:49.271245 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:49.274366 10875 raft_consensus.cc:1275] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:18:49.218271 16911 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:49.276129 10870 raft_consensus.cc:1240] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 22. Current term is 23. Ops: []
I20260709 13:18:49.227002 17316 raft_consensus.cc:697] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:49.186939 17341 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:49.285988 17316 consensus_queue.cc:237] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:49.265650 17032 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:18:49.286597  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:18:49.290287 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:49.289947 10868 raft_consensus.cc:1240] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:49.207122 17340 consensus_queue.cc:1059] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.070s
W20260709 13:18:49.322537 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 197257us
I20260709 13:18:49.271242 16907 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:49.276866 17353 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:49.326606 10766 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:49.330694 17353 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:49.287005 17269 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:49.331748 17269 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:49.332902 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:49.334005 17269 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:49.335312 17269 consensus_queue.cc:1048] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:49.335839 10950 raft_consensus.cc:1240] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 18. Current term is 19. Ops: []
I20260709 13:18:49.260005 17366 consensus_queue.cc:260] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:49.293186 17032 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:18:49.342002 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:49.349977 10841 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:49.351740 10841 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:49.355736 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:49.324908 17306 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:49.324146 17341 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:49.330353 16907 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:18:49.329819 17339 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:49.217762 17327 consensus_queue.cc:260] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:49.357169 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:49.330504 17329 raft_consensus.cc:697] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:49.347214 17032 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:49.329911 17089 consensus_queue.cc:1059] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:18:49.381460 17366 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:49.382972 17329 consensus_queue.cc:237] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:49.383601 17363 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:18:49.384151 17363 raft_consensus.cc:697] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:49.385046 17363 consensus_queue.cc:237] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:49.380139 17339 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:49.399938 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 129956us
I20260709 13:18:49.401473 10916 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:49.405149 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 81202.2us
I20260709 13:18:49.408377 17282 consensus_queue.cc:1048] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:18:49.409615 17341 consensus_queue.cc:1048] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.002s
I20260709 13:18:49.410230 17380 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:49.408974 10842 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:49.372402 17306 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:49.423053 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:49.423926 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:49.390749 17327 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:49.424942 17269 raft_consensus.cc:2749] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:49.447949 10875 raft_consensus.cc:2393] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 17: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:49.444366 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:49.422152 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:49.450655 10875 raft_consensus.cc:1275] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 20 index: 9. (term mismatch)
I20260709 13:18:49.451455 10875 pending_rounds.cc:85] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 8
I20260709 13:18:49.452123 10875 pending_rounds.cc:107] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 19.9
I20260709 13:18:49.451948 17386 consensus_queue.cc:1059] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:18:49.453121 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:49.454505 10875 raft_consensus.cc:2889] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:49.455075 10867 raft_consensus.cc:2393] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 19: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
W20260709 13:18:49.453533 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:49.453424 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 98737.9us
I20260709 13:18:49.424777 17373 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:49.472076 10916 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:49.475976 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:49.477942 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:49.478876 17362 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:49.475307 17089 raft_consensus.cc:3055] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:18:49.479686 17089 raft_consensus.cc:740] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:49.480661 17089 consensus_queue.cc:260] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:49.482151 17089 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:18:49.485615 17348 log.cc:927] Time spent T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.456s	user 0.000s	sys 0.001s
I20260709 13:18:49.488566 11503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56410 (request call id 997) took 1081 ms. Trace:
I20260709 13:18:49.488993 11503 rpcz_store.cc:276] 0709 13:18:48.407138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.411826 (+  4688us) service_pool.cc:224] Handling call
0709 13:18:48.476009 (+ 64183us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:48.591527 (+115518us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:48.591560 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:48.600411 (+  8851us) log.cc:844] Serialized 26 byte log entry
0709 13:18:48.865326 (+264915us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:49.082461 (+217135us) log.cc:844] Serialized 14 byte log entry
0709 13:18:49.082708 (+   247us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:49.082778 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:49.484820 (+402042us) spinlock_profiling.cc:243] Waited 18.1 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:18:49.488243 (+  3423us) raft_consensus.cc:1685] finished
0709 13:18:49.488348 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:49.488499 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":76729,"prepare.queue_time_us":357953,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":18956,"raft.queue_time_us":94976,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":20643,"spinlock_wait_cycles":41588608,"thread_start_us":366283,"threads_started":2,"wal-append.queue_time_us":377103}
I20260709 13:18:49.492828 17369 raft_consensus.cc:3055] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:18:49.493384 17369 raft_consensus.cc:740] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:49.494402 17369 consensus_queue.cc:260] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:49.495857 17369 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:18:49.503151 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 74697.1us
I20260709 13:18:49.504050 10842 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:49.507885 10797 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 18.
W20260709 13:18:49.509594 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 44686.4us
I20260709 13:18:49.509618 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:49.510562 10916 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:49.475876 17386 raft_consensus.cc:3055] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:18:49.507846 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:49.513928 10796 raft_consensus.cc:2393] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 17: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:49.518674 10916 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:49.521379 10916 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:49.475643 17320 consensus_queue.cc:1048] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:18:49.524598 17364 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:18:49.479696 17362 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:49.513890 17386 raft_consensus.cc:740] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:49.531376 17349 raft_consensus.cc:2749] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:18:49.532271 17339 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:49.529089 17354 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:49.547391 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:49.551065 10842 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:49.476164 17373 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:49.553408 17381 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:18:49.530918 17383 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:49.537868 17282 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:49.530825 16911 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:49.529446 17376 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:18:49.566638 10842 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:49.546741 17364 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:49.577028 10798 raft_consensus.cc:2410] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 13:18:49.577632 17304 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:49.580035 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:49.598713 10842 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:49.599207 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:49.591977 17329 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:18:49.593886 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 55781.2us
I20260709 13:18:49.578572 17349 consensus_queue.cc:1059] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.022s
I20260709 13:18:49.549093 17323 consensus_queue.cc:1048] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:18:49.563895 17306 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:49.567338 17381 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:49.578452 17339 raft_consensus.cc:2749] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:18:49.611742 10916 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:49.612068 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:49.614830 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:49.615484 11504 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:18:49.578794 16911 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:49.601327 17158 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:49.625195 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:18:49.621135 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:49.599572 17364 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:49.628731 17387 consensus_queue.cc:1059] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.109s
I20260709 13:18:49.631088 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:49.633603 17032 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:49.635790 17032 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:49.643667 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:49.606731 17304 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:49.625306 16911 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:49.625571 17158 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:49.627974 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:49.626806 11504 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 23.
W20260709 13:18:49.647013 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:49.613389 17381 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:18:49.655608 11510 raft_consensus.cc:1275] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:18:49.659960 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:49.638551 17306 raft_consensus.cc:2749] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:18:49.660225 17158 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:49.666553 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:49.612088 17326 consensus_queue.cc:1059] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 13:18:49.693114 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:49.694180 10869 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:49.648232 17329 raft_consensus.cc:2749] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:49.637033 17364 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:49.697163 17307 consensus_meta.cc:287] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.983s	user 0.007s	sys 0.006s
I20260709 13:18:49.648715 17282 consensus_queue.cc:1048] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 13:18:49.708249 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:49.638168 17395 consensus_queue.cc:1048] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.064s
I20260709 13:18:49.705454 17032 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:49.694840 17376 raft_consensus.cc:697] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:18:49.726420 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 70196.7us
I20260709 13:18:49.727803 11477 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:18:49.737550 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:49.707800 17354 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:49.742589 10876 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 14.
W20260709 13:18:49.737814 17338 consensus_meta.cc:287] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.580s	user 0.010s	sys 0.006s
I20260709 13:18:49.735412 17388 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:49.705746 17308 consensus_queue.cc:1059] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
W20260709 13:18:49.751605 17335 log.cc:927] Time spent T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 1.084s	user 0.004s	sys 0.000s
I20260709 13:18:49.751766 17304 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:49.730758 17032 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:49.755218 10795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 2993) took 1368 ms. Trace:
I20260709 13:18:49.755609 10795 rpcz_store.cc:276] 0709 13:18:48.386599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.386755 (+   156us) service_pool.cc:224] Handling call
0709 13:18:48.387115 (+   360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:48.392708 (+  5593us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:48.392750 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:48.414323 (+ 21573us) log.cc:844] Serialized 26 byte log entry
0709 13:18:48.578557 (+164234us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:48.578595 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:48.578656 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:49.727556 (+1148900us) spinlock_profiling.cc:243] Waited 15.7 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:18:49.754856 (+ 27300us) raft_consensus.cc:1685] finished
0709 13:18:49.754892 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:49.755150 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6923,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":4954,"spinlock_wait_cycles":36204928,"thread_start_us":89085,"threads_started":1,"wal-append.queue_time_us":202452}
I20260709 13:18:49.757301 17388 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:49.760766 17282 consensus_queue.cc:1048] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:18:49.708730 17353 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:49.773065 17304 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:18:49.779462 10874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 1014) took 1301 ms. Trace:
I20260709 13:18:49.779706 17304 raft_consensus.cc:697] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 19 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:49.779913 10874 rpcz_store.cc:276] 0709 13:18:48.478035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.486350 (+  8315us) service_pool.cc:224] Handling call
0709 13:18:48.497934 (+ 11584us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:48.659317 (+161383us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:18:48.659692 (+   375us) log.cc:844] Serialized 14 byte log entry
0709 13:18:48.876797 (+217105us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:48.876877 (+    80us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:18:48.876943 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:49.779193 (+902250us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b54002d4454. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:49.779245 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:49.779402 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":4151,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":131,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":87,"spinlock_wait_cycles":404093568,"thread_start_us":216345,"threads_started":1,"wal-append.queue_time_us":369713}
I20260709 13:18:49.780853 17304 consensus_queue.cc:237] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:49.781046 17353 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:49.773940 17308 raft_consensus.cc:3055] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:49.743933 17398 consensus_queue.cc:1048] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
W20260709 13:18:49.763302 17390 log.cc:927] Time spent T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:18:49.763521 17338 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:49.778374 16907 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:49.782796 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2502) took 1024 ms. Trace:
I20260709 13:18:49.787268 17308 raft_consensus.cc:740] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:49.770717 17369 raft_consensus.cc:3055] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:18:49.787719 17354 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:49.792117 10767 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:49.799075 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:49.773267 17265 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:49.763060 17314 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:49.775629 17388 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:49.774591 17373 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:49.737900 17329 consensus_queue.cc:1059] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 13:18:49.800060 10873 rpcz_store.cc:276] 0709 13:18:48.758840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.761043 (+  2203us) service_pool.cc:224] Handling call
0709 13:18:49.248627 (+487584us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:49.249261 (+   634us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:49.782544 (+533283us) spinlock_profiling.cc:243] Waited 440 ms on lock 0x7b54000f78d0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:49.782712 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1011028352}
I20260709 13:18:49.815732 17383 raft_consensus.cc:2749] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:49.817842 10767 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:49.783054 17366 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:49.880668 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 62952.3us
I20260709 13:18:49.898515 10767 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:49.803799 17389 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:49.893221 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:49.804105 17354 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:49.923269 10767 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:49.931951 10945 raft_consensus.cc:1240] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:49.804919 17316 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:49.804714 17369 raft_consensus.cc:740] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:49.884927 17352 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:18:49.884979 17378 raft_consensus.cc:2749] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:18:49.921705 17373 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:49.956759 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:49.958679 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:49.959583 10946 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:49.959925 10950 raft_consensus.cc:1275] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:18:49.960871 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:49.962453 10950 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
I20260709 13:18:49.938606 17362 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:49.972836 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:49.885140 17273 consensus_queue.cc:1059] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.068s
I20260709 13:18:49.951566 17352 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:49.981516 11478 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:49.985327 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 62204.4us
I20260709 13:18:49.991202 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:49.991513 10767 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:49.930675 17366 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:50.000393 17366 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:49.934442 17389 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:50.001955 17327 consensus_meta.cc:287] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.548s	user 0.011s	sys 0.000s
I20260709 13:18:50.002377 17389 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:49.949018 17316 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:49.989359 17352 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:50.003732 16907 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:50.004503 17316 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:49.976776 17378 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:49.983181 17308 consensus_queue.cc:260] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:50.005264 17378 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:50.005722 17378 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:50.005769 17308 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:49.996807 17273 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:49.938832 17376 consensus_queue.cc:237] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:49.964725 17373 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:50.011760 17373 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:50.014636 17375 raft_consensus.cc:2749] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:50.015004 10942 raft_consensus.cc:1240] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:50.015811 17373 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:50.016781 17354 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:50.016469 17373 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:50.018895 17373 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:50.020752 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:50.021438 10948 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 21.
I20260709 13:18:50.022981 10841 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:50.024565 10869 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 17.
I20260709 13:18:50.025409 17354 consensus_queue.cc:1059] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:50.025741 17378 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:50.026685 17354 raft_consensus.cc:3055] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:50.027108 17354 raft_consensus.cc:740] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:50.027974 17378 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:50.028632 17397 consensus_queue.cc:1059] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.080s
I20260709 13:18:50.029304 17406 raft_consensus.cc:3055] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:18:50.029770 17406 raft_consensus.cc:740] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:18:50.030211 17344 log.cc:927] Time spent T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.928s	user 0.000s	sys 0.002s
I20260709 13:18:50.030674 17406 consensus_queue.cc:260] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:50.031776 17406 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:18:50.031876 17269 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:50.032366 17269 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:50.032717 17269 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:50.033499 17373 consensus_queue.cc:1048] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:50.034487 17362 consensus_queue.cc:1059] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:18:50.035349 17393 log.cc:927] Time spent T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.297s	user 0.001s	sys 0.000s
I20260709 13:18:50.035657 17362 raft_consensus.cc:3055] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:18:50.036084 17362 raft_consensus.cc:740] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:50.035980 17273 consensus_queue.cc:1048] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:50.037051 17362 consensus_queue.cc:260] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:50.038337 17362 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:50.039146 10767 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:50.041051 17366 consensus_queue.cc:1059] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:18:50.042318 17316 raft_consensus.cc:2749] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:18:50.043116 17316 raft_consensus.cc:3055] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:50.043571 17316 raft_consensus.cc:740] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:50.044627 17316 consensus_queue.cc:260] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:50.045941 17316 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:50.047066 10767 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:50.024016 10946 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 21.
I20260709 13:18:50.056540 10950 raft_consensus.cc:1240] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 20. Current term is 21. Ops: []
I20260709 13:18:50.056517 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:50.055047 10841 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:50.058050 10941 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 19.
I20260709 13:18:50.059003 10942 raft_consensus.cc:1240] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 21. Current term is 22. Ops: []
W20260709 13:18:50.065133 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 83735.6us
I20260709 13:18:50.071764 10767 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:50.073011 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 345269us
I20260709 13:18:50.060278 17387 raft_consensus.cc:2804] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:50.078704 17354 consensus_queue.cc:260] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:50.079334 17032 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:50.083155 10841 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:50.080715 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 343135us
I20260709 13:18:50.086391 10767 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:50.064707 17364 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:50.064431 17373 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:18:50.095671 17373 raft_consensus.cc:697] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:50.096653 17373 consensus_queue.cc:237] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:50.088126 17375 raft_consensus.cc:2749] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:18:50.098932 17399 raft_consensus.cc:2749] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:18:50.092752 17411 consensus_queue.cc:1059] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 13:18:50.094424 17158 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:50.105894 11478 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:50.088893 10841 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:50.103291 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:50.090723 17032 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:50.108409 10869 raft_consensus.cc:1275] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:18:50.111191 10872 raft_consensus.cc:1240] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 20. Current term is 21. Ops: [20.9-20.9]
I20260709 13:18:50.106859 11477 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:50.113130 10872 raft_consensus.cc:1240] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:18:50.082978 17387 raft_consensus.cc:697] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:50.113168 10871 raft_consensus.cc:1240] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 22. Current term is 23. Ops: [22.15-22.15]
I20260709 13:18:50.114431 10841 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:50.023739 17369 consensus_queue.cc:260] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:50.119128 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:50.121956 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:18:50.108582 17312 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.178s	user 0.013s	sys 0.029s
I20260709 13:18:50.116348 17388 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:50.117563 17228 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:18:50.154579 17228 raft_consensus.cc:697] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:50.154618 17231 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:50.155304 17231 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:50.155691 17228 consensus_queue.cc:237] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:50.157485 17231 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:50.124451 17402 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:18:50.133122 17398 consensus_queue.cc:1048] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:50.124421 17364 consensus_queue.cc:1048] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.011s
I20260709 13:18:50.136453 17314 raft_consensus.cc:3055] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:18:50.139715 16911 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:50.166410 17314 raft_consensus.cc:740] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:50.167937 16911 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:50.127358 17383 consensus_queue.cc:1059] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 13:18:50.167852 17314 consensus_queue.cc:260] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:50.168990 17383 raft_consensus.cc:3055] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:18:50.169344 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:50.169988 17314 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:50.170567 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:50.130275 16907 consensus_queue.cc:1059] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:18:50.171834 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:50.172350 10870 raft_consensus.cc:2376] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15. Re-sending same reply.
I20260709 13:18:50.173166 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:50.174504 10870 raft_consensus.cc:1275] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:18:50.128194 17381 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:50.138280 17352 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:50.175385 17369 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:18:50.139726 17342 log.cc:927] Time spent T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 1.110s	user 0.008s	sys 0.003s
W20260709 13:18:50.127915 17312 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.204s	user 0.013s	sys 0.030s
I20260709 13:18:50.120497 17350 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:50.120710 17030 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.178042 17350 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:50.145238 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:50.178534 17030 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:50.178972 17370 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.148120 17386 consensus_queue.cc:260] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:50.179554 17370 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:50.149535 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:50.180552 10874 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 20.
I20260709 13:18:50.180874 17030 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:50.182214 17370 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:50.117722 17304 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:50.198858 17304 raft_consensus.cc:697] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 17 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:50.199399 17375 raft_consensus.cc:3055] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:50.199837 17375 raft_consensus.cc:740] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:50.199923 17304 consensus_queue.cc:237] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:50.200831 17375 consensus_queue.cc:260] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:50.202255 17375 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:50.203634 17388 raft_consensus.cc:2749] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:18:50.204378 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:50.126909 11478 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:50.158428 17330 log.cc:927] Time spent T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 1.660s	user 0.004s	sys 0.007s
I20260709 13:18:50.127779 17411 raft_consensus.cc:2804] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:50.132073 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:50.163112 17402 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:50.169420 17383 raft_consensus.cc:740] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:50.170667 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:50.177335 17352 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:50.144215 11509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56410 (request call id 998) took 1726 ms. Trace:
I20260709 13:18:50.185387 10949 raft_consensus.cc:1240] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 18. Current term is 19. Ops: []
I20260709 13:18:50.186087 10867 raft_consensus.cc:1240] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 23. Current term is 24. Ops: [23.12-23.12]
I20260709 13:18:50.188565 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:50.172358 16907 raft_consensus.cc:3055] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:18:50.190253 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:50.184892 10868 raft_consensus.cc:1240] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 13:18:50.191077 17387 consensus_queue.cc:237] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:50.192804 10767 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:50.193248 10870 raft_consensus.cc:1275] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:18:50.193480 10801 raft_consensus.cc:1275] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:18:50.194862 17386 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:50.196362 17158 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:50.196671 17350 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:50.120869 17354 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:50.197635 17381 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:50.198143 17363 raft_consensus.cc:3055] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:18:50.213050 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:50.227399 17184 raft_consensus.cc:2749] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:18:50.222249 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:50.252758 10873 raft_consensus.cc:2393] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 16: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:50.228821 10869 raft_consensus.cc:2376] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 18. Re-sending same reply.
I20260709 13:18:50.240281 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:50.229960 11509 rpcz_store.cc:276] 0709 13:18:48.418913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.419019 (+   106us) service_pool.cc:224] Handling call
0709 13:18:48.474222 (+ 55203us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:48.735653 (+261431us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:18:49.127066 (+391413us) log.cc:844] Serialized 14 byte log entry
0709 13:18:49.572100 (+445034us) log.cc:844] Serialized 14 byte log entry
0709 13:18:49.639573 (+ 67473us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:49.639621 (+    48us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:18:49.639669 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:50.143956 (+504287us) spinlock_profiling.cc:243] Waited 261 ms on lock 0x7b540068d8d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:50.144010 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:50.144151 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":430134,"prepare.queue_time_us":473780,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":2269,"raft.queue_time_us":433942,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":229,"spinlock_wait_cycles":600560512,"thread_start_us":472491,"threads_started":3,"wal-append.queue_time_us":13161}
W20260709 13:18:50.216006 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 89195.6us
I20260709 13:18:50.254732 11478 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:50.227984 17402 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:18:50.251621 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 58877us
I20260709 13:18:50.255759 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:50.256431 10767 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:18:50.258478 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:50.188598 17269 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:50.256381 10868 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
I20260709 13:18:50.261171 17312 consensus_queue.cc:1048] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:18:50.248957 17363 raft_consensus.cc:740] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:50.262528 17352 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:18:50.232014 16907 raft_consensus.cc:740] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:50.232384 17411 raft_consensus.cc:697] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:50.268616 11478 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:50.258317 17089 raft_consensus.cc:2804] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:50.272534 11478 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:50.271217 10916 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:50.274019 11478 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:50.275494 11478 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:50.248934 17340 consensus_queue.cc:1048] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.022s
I20260709 13:18:50.269167 17399 raft_consensus.cc:2764] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 13:18:50.269366 17352 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:50.281890 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:50.282128 11508 raft_consensus.cc:1240] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:50.288657 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:50.276222 17304 raft_consensus.cc:2804] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:18:50.311143 17304 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:50.311651 17304 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:18:50.274981 17184 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:18:50.313906 17184 raft_consensus.cc:697] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 19 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:50.315140 17184 consensus_queue.cc:237] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:50.316385 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:50.317040 10944 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
I20260709 13:18:50.317924 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:50.318171 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:50.318881 10941 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
I20260709 13:18:50.292639 17415 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:50.295594 16907 consensus_queue.cc:260] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:50.322050 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:50.322448 17415 raft_consensus.cc:697] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 17 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:50.322597 16907 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:50.323475 17415 consensus_queue.cc:237] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:50.297627 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:18:50.298075 17347 log.cc:927] Time spent T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 1.269s	user 0.003s	sys 0.003s
I20260709 13:18:50.299067 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:50.299808 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:50.326962 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:50.300537 17363 consensus_queue.cc:260] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:50.276486 17265 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:50.302148 17411 consensus_queue.cc:237] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:50.303347 17231 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:50.290859 10767 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.116.130:37895 timed out after 1.679s (SENT) [suppressed 17 similar messages]
W20260709 13:18:50.343567 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 52729.6us
I20260709 13:18:50.344316 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:50.286518 17089 raft_consensus.cc:697] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:50.275182 17383 consensus_queue.cc:260] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 20.9, Last appended by leader: 9, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:50.306685 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:50.307294 11508 raft_consensus.cc:1240] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 22. Current term is 23. Ops: []
I20260709 13:18:50.290655 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:50.350589 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:50.345117 10943 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 21.
I20260709 13:18:50.319211 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:50.321396 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:50.321691 10875 raft_consensus.cc:1275] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:18:50.296723 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:50.321691 10872 raft_consensus.cc:1275] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:18:50.327684 10916 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:50.329262 10841 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:50.329461 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:50.330336 10798 raft_consensus.cc:1275] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:18:50.358470 10841 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:50.358717 10797 raft_consensus.cc:2410] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 19 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:18:50.359215 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:50.332924 10915 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:50.284642 17370 raft_consensus.cc:2804] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:50.360185 17370 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:50.360303 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:50.360438 17158 raft_consensus.cc:2804] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:50.287598 17352 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:50.361195 17158 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:50.361469 17381 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:50.361665 17158 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:50.362146 17381 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:50.362627 17381 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:50.319999 11477 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:50.360590 17370 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:50.326249 11505 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 19.
I20260709 13:18:50.370734 17352 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:50.351275 11511 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 19.
I20260709 13:18:50.377720 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:50.378221 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 50603.2us
I20260709 13:18:50.378888 10794 raft_consensus.cc:2410] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:18:50.379611 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:50.383257 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:50.353606 10942 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 21.
I20260709 13:18:50.385551 10841 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:50.385739 10799 raft_consensus.cc:1275] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:18:50.386826 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:50.387872 10796 raft_consensus.cc:2410] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 17 because replica has last-logged OpId of term: 15 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 13:18:50.380321 10800 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
I20260709 13:18:50.392820 10916 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:50.399911 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:18:50.403015 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 70173us
I20260709 13:18:50.403740 11509 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 17.
I20260709 13:18:50.403818 10915 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:50.404712 10942 raft_consensus.cc:1275] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:18:50.398434 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:50.343333 17312 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.413324 10916 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:50.417717 10798 raft_consensus.cc:2393] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 21: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:50.419420 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:50.423947 10799 raft_consensus.cc:2393] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 14: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:50.425776 10915 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:50.427331 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:50.428875 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:50.429963 10794 raft_consensus.cc:2410] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 16 because replica has last-logged OpId of term: 15 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 13:18:50.432583 10875 raft_consensus.cc:1275] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:18:50.399420 17363 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:50.441496 11478 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:50.459125 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:50.409789 17030 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:50.407662 17406 raft_consensus.cc:2749] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:50.409094 17388 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:18:50.406137 16911 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:18:50.428184 10800 raft_consensus.cc:2376] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 17. Re-sending same reply.
I20260709 13:18:50.441246 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:50.382963 17231 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:50.412045 17282 consensus_queue.cc:1048] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:18:50.533478 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:50.472066 10875 raft_consensus.cc:1275] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:18:50.464109 10915 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:50.531286 10796 raft_consensus.cc:2376] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 17. Re-sending same reply.
I20260709 13:18:50.437521 17383 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:18:50.464365 17418 log.cc:927] Time spent T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:18:50.540717 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:50.543205 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:18:50.546021 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 226096us
I20260709 13:18:50.544191 10799 raft_consensus.cc:2376] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 16. Re-sending same reply.
I20260709 13:18:50.554003 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:50.555428 10799 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 21.
I20260709 13:18:50.557678 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:50.459537 17398 raft_consensus.cc:2749] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:50.561357 10795 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 16.
I20260709 13:18:50.563354 10796 raft_consensus.cc:2393] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 19: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:50.565784 10945 raft_consensus.cc:2410] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:18:50.570456 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:50.524247 17269 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:50.535027 17388 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:50.645558 17388 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:18:50.478096 17399 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:18:50.651149 17399 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:50.651648 17399 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:50.465875 17340 raft_consensus.cc:2749] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:18:50.575636 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:50.655249 10796 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 16.
I20260709 13:18:50.656374 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:50.656952 10796 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:50.559201 17381 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:50.660744 17381 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:50.581485 17310 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:50.581914 17387 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.583581 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:50.585808 17397 consensus_queue.cc:1059] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:18:50.661844 17387 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:50.586532 17402 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:50.664435 17402 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:50.665102 11512 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 21.
I20260709 13:18:50.590489 10801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 893) took 1113 ms. Trace:
I20260709 13:18:50.666992 17387 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:50.666965 10801 rpcz_store.cc:276] 0709 13:18:49.476646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:49.476798 (+   152us) service_pool.cc:224] Handling call
0709 13:18:50.190722 (+713924us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:50.235412 (+ 44690us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:50.590260 (+354848us) spinlock_profiling.cc:243] Waited 687 ms on lock 0x7b54000d7350. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:50.590420 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1580933888}
I20260709 13:18:50.562198 17406 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:18:50.668481 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:50.669111 17406 raft_consensus.cc:697] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 21 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:50.669883 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:50.592658 10947 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41014 (request call id 2611) took 1260 ms (client timeout 1637 ms). Trace:
W20260709 13:18:50.670966 10947 rpcz_store.cc:269] 0709 13:18:49.332444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:49.332609 (+   165us) service_pool.cc:224] Handling call
0709 13:18:50.592463 (+1259854us) spinlock_profiling.cc:243] Waited 258 ms on lock 0x7b5400016dd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:50.592599 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":594201984}
I20260709 13:18:50.596517 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:50.673139 10798 raft_consensus.cc:1275] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:18:50.674424 10798 raft_consensus.cc:1240] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 20. Current term is 21. Ops: []
I20260709 13:18:50.678864 10798 raft_consensus.cc:1275] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 14. (term mismatch)
I20260709 13:18:50.687165 10798 pending_rounds.cc:85] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 13
W20260709 13:18:50.631803 17345 log.cc:927] Time spent T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 1.603s	user 0.007s	sys 0.000s
W20260709 13:18:50.643456 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:50.515007 17089 consensus_queue.cc:237] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:50.520294 17158 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:50.537904 17314 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:50.444402 17312 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:50.670538 10801 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
I20260709 13:18:50.679147 17388 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:50.702482 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:50.703118 10801 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:18:50.704218 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:50.704037 17399 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:50.683259 17369 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:18:50.706094 17369 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:50.706306 17399 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:50.706486 17369 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:50.706707 17381 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.706990 17158 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:50.682296 17310 raft_consensus.cc:2749] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:50.612054 17352 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:50.618343 17425 log.cc:927] Time spent T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.076s	user 0.001s	sys 0.001s
I20260709 13:18:50.687768 10798 pending_rounds.cc:107] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 15.14
I20260709 13:18:50.726698 10798 raft_consensus.cc:2889] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:50.670210 17406 consensus_queue.cc:237] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:50.728545 10800 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:18:50.707298 17381 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:50.716507 17388 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:50.718112 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 276735us
I20260709 13:18:50.626595 17338 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:50.721798 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:50.721524 17314 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:50.745388 17432 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.746731 17314 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:50.748883 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:50.750099 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:50.763489 10767 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:50.770808 17352 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:18:50.771392 17352 raft_consensus.cc:697] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:50.772239 17352 consensus_queue.cc:237] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:50.774354 10948 raft_consensus.cc:2410] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 21 because replica has last-logged OpId of term: 20 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 13:18:50.775087 17381 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:50.776609 17158 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:50.777129 17312 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:50.778887 10842 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:50.787546 10800 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 24.
I20260709 13:18:50.745985 17432 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:50.789499 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:50.790341 17432 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:50.728974 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:50.793339 10798 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 23.
I20260709 13:18:50.795020 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:50.797438 10798 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:50.799597 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:50.800288 10792 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:50.709487 10796 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 21.
I20260709 13:18:50.790721 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:50.805042 10795 raft_consensus.cc:1275] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:18:50.777333 17158 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:50.795696 10867 raft_consensus.cc:1275] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:18:50.813382 10798 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
I20260709 13:18:50.744001 10799 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
I20260709 13:18:50.816190 10799 raft_consensus.cc:1240] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:18:50.819497 10796 raft_consensus.cc:1240] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:50.817901 10842 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:50.821547 10799 raft_consensus.cc:1240] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 20. Current term is 21. Ops: [20.9-20.9]
I20260709 13:18:50.725281 10801 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 22.
I20260709 13:18:50.828008 10792 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
I20260709 13:18:50.829661 10799 raft_consensus.cc:1240] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:18:50.807950 17432 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.837422 10842 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:50.838429 10792 raft_consensus.cc:1240] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 16. Current term is 17. Ops: []
W20260709 13:18:50.838454 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 425229us
I20260709 13:18:50.839627 10916 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:50.812547 17269 raft_consensus.cc:2749] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:18:50.841185 10916 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:50.812644 17388 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.842541 10801 raft_consensus.cc:1275] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:18:50.844890 10792 raft_consensus.cc:1240] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:18:50.850366 10798 raft_consensus.cc:1240] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:18:50.850291 10916 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:50.849675 16969 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:50.851729 10916 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:50.852494 16969 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:50.853173 10916 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:50.810911 17304 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:50.859826 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 295469us
I20260709 13:18:50.854352 10916 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:18:50.857003 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 393051us
I20260709 13:18:50.844386 17362 raft_consensus.cc:2749] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:50.865347 10915 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:50.866987 10792 raft_consensus.cc:1240] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 21. Current term is 22. Ops: []
I20260709 13:18:50.830459 17372 consensus_queue.cc:1048] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:18:50.847870 17383 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:18:50.870348 11477 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:50.870442 10916 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:50.873485 10916 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:50.873885 10915 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:50.854110 17380 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:50.854970 17364 raft_consensus.cc:2749] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:50.862985 17407 raft_consensus.cc:2749] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:18:50.880744 10792 raft_consensus.cc:1275] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:18:50.859251 17329 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:50.860440 17404 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:18:50.869438 17370 consensus_meta.cc:287] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.503s	user 0.010s	sys 0.000s
I20260709 13:18:50.848726 17432 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:50.894505 10916 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:50.851436 17388 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:50.881917 17338 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:18:50.881783 16907 raft_consensus.cc:2804] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:50.894538 17437 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:18:50.894124 17398 raft_consensus.cc:2804] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:18:50.932557 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 95203.3us
W20260709 13:18:50.929608 17265 consensus_meta.cc:287] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.586s	user 0.005s	sys 0.005s
I20260709 13:18:50.898180 17383 raft_consensus.cc:697] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 18 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:50.937809 10842 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:50.922590 17387 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:50.938071 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 96117.8us
I20260709 13:18:50.880071 17314 consensus_queue.cc:1059] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.051s
W20260709 13:18:50.942028 10876 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 2517) took 1982 ms (client timeout 1679 ms). Trace:
I20260709 13:18:50.945597 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:18:50.944391 10876 rpcz_store.cc:269] 0709 13:18:48.959602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.969709 (+ 10107us) service_pool.cc:224] Handling call
0709 13:18:50.941791 (+1972082us) spinlock_profiling.cc:243] Waited 749 ms on lock 0x7b540009bad4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:50.941941 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1721821056}
I20260709 13:18:50.946364 10842 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:50.931804 17338 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:50.930400 17440 raft_consensus.cc:2804] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:50.935058 17437 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:50.925247 17370 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:50.936842 17398 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:50.941596 17265 raft_consensus.cc:2749] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:50.940537 17407 raft_consensus.cc:2804] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:50.950053 17364 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:18:50.963189 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:50.947499 17269 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.965237 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:50.944223 16907 raft_consensus.cc:697] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:50.980950 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:50.984191 10947 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:18:50.981659 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:50.947832 16969 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:50.953308 10842 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:50.983039 11507 raft_consensus.cc:1240] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 17 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:50.943295 17323 consensus_queue.cc:1048] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 13:18:50.953366 17338 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:18:50.989005 10943 raft_consensus.cc:1240] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:50.965934 17437 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:18:50.995394 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:50.995601 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:50.992529 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:50.955099 17440 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:50.967433 17398 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:50.946902 17387 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:50.994500 11506 raft_consensus.cc:1275] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:18:50.938102 17439 consensus_queue.cc:1048] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.041s
I20260709 13:18:51.012236 10799 raft_consensus.cc:1240] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:51.015908 10792 raft_consensus.cc:1275] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:18:50.996022 17312 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.989264 17407 raft_consensus.cc:697] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 17 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:51.030997 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:50.991040 17269 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:51.031951 17407 consensus_queue.cc:237] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:51.032413 17369 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:51.033603 17030 raft_consensus.cc:2749] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:51.019130 17383 consensus_queue.cc:237] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:51.020357 17446 consensus_queue.cc:1048] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:18:51.035842 16907 consensus_queue.cc:237] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:51.037124 17446 raft_consensus.cc:2804] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:51.037556 17446 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:51.037949 17446 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:18:51.020953 17441 log.cc:927] Time spent T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:18:51.017462 17447 raft_consensus.cc:2749] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:18:51.021726 17442 log.cc:927] Time spent T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
W20260709 13:18:51.022269 17424 log.cc:927] Time spent T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.554s	user 0.000s	sys 0.004s
I20260709 13:18:51.023489 17387 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:51.023797 17388 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:51.015486 17448 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:50.999019 17440 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:51.034917 17312 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:51.020237 17438 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:18:51.041666 10944 raft_consensus.cc:2393] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 20: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:51.043079 10792 raft_consensus.cc:1240] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 15. Current term is 16. Ops: []
W20260709 13:18:51.043449 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 149023us
I20260709 13:18:51.065907 17438 raft_consensus.cc:697] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:51.066812 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:51.066905 17438 consensus_queue.cc:237] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:51.067519 10792 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 18.
I20260709 13:18:51.034525 17369 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:51.068526 17369 consensus_queue.cc:1059] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:18:51.069525 17369 consensus_queue.cc:1059] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:51.070552 17369 raft_consensus.cc:3055] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:18:51.070909 17369 raft_consensus.cc:740] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:51.070966 16969 raft_consensus.cc:3055] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:51.071489 16969 raft_consensus.cc:740] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:51.071887 17369 consensus_queue.cc:260] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:51.072348 16969 consensus_queue.cc:260] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:51.073292 17369 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:51.042891 10947 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 22.
I20260709 13:18:51.073843 16969 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:51.074635 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:18:51.075276 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:51.075323 10947 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 15.
I20260709 13:18:51.078150 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:51.078832 10947 raft_consensus.cc:2368] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for earlier term 21. Current term is 22.
I20260709 13:18:51.079506 17432 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:51.080112 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:51.080727 10947 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 18.
I20260709 13:18:51.080876 17370 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:51.082114 17386 consensus_meta.cc:287] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.838s	user 0.009s	sys 0.004s
I20260709 13:18:51.083379 10797 raft_consensus.cc:2410] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 19 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
W20260709 13:18:50.992544 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 118743us
I20260709 13:18:51.085031 17370 consensus_queue.cc:1059] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:18:51.085453 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:51.086030 17341 raft_consensus.cc:3055] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:18:51.086221 10797 raft_consensus.cc:2393] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 16: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:51.086431 17341 raft_consensus.cc:740] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:51.086958 17304 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:51.087446 17341 consensus_queue.cc:260] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:51.083231 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:51.088472 17329 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:51.089023 17329 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:51.065244 17312 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:51.091600 10870 raft_consensus.cc:1275] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:18:51.091919 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:51.093470 10876 raft_consensus.cc:1240] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:51.093966 17323 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.094245 17404 raft_consensus.cc:2749] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:51.094513 17323 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:51.096863 17323 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:51.038839 17158 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:51.098484 17158 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.101056 10876 raft_consensus.cc:1240] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 LEADER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 20. Current term is 21. Ops: []
I20260709 13:18:51.102008 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:51.088616 10792 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:51.103829 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:51.106925 10876 raft_consensus.cc:1275] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:18:51.107213 11477 leader_election.cc:400] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 election: Vote denied by peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553) with higher term. Message: Invalid argument: T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for earlier term 21. Current term is 22.
I20260709 13:18:51.107525 11477 leader_election.cc:403] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 13:18:51.108289 10950 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
I20260709 13:18:51.107932 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:51.109949 10876 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 15.
I20260709 13:18:51.111042 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:51.111672 10876 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 18.
I20260709 13:18:51.112509 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:51.112792 10867 raft_consensus.cc:1275] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:18:51.116138 10875 raft_consensus.cc:2368] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for earlier term 21. Current term is 22.
I20260709 13:18:51.113219 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:51.110249 11477 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:51.117034 11509 raft_consensus.cc:2393] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 20: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:51.117640 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:51.118263 11506 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:51.119455 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:51.120057 10869 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:51.034525 17269 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:51.135691 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:51.136456 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:51.137705 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 192294us
I20260709 13:18:51.148926 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:51.144511 17446 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:51.154078 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:51.154739 10794 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
I20260709 13:18:51.121345 17432 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:18:51.166513 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:51.162695 10841 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:51.106842 17158 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:51.156464 17386 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.174816 17455 consensus_queue.cc:1048] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 13:18:51.196108 11506 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 17.
I20260709 13:18:51.179177 17380 raft_consensus.cc:2749] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:51.165441 16969 consensus_queue.cc:1048] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 13:18:51.165268 17265 consensus_queue.cc:1048] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.021s
I20260709 13:18:51.185422 17338 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:51.107055 17304 consensus_queue.cc:1048] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 13:18:51.246510 11509 raft_consensus.cc:1275] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:18:51.148638 17411 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.148355 17312 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.193913 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:51.153846 17415 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:51.189338 10766 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:51.200052 10792 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 17.
I20260709 13:18:51.203044 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:51.175489 17329 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:51.203971 10942 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 17.
I20260709 13:18:51.252943 11510 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
I20260709 13:18:51.253681 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:18:51.211392 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 258167us
I20260709 13:18:51.103947 17448 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:51.215324 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:18:51.227944 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:51.223481 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 72247.8us
I20260709 13:18:51.256611 11478 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:51.253229 10841 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:51.258850 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:51.222740 10869 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 17.
W20260709 13:18:51.219080 17184 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.016s	sys 0.002s
I20260709 13:18:51.259059 10842 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:51.265064 10842 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:51.167553 17370 consensus_queue.cc:1059] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.082s
I20260709 13:18:51.214283 17269 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:51.225306 17364 raft_consensus.cc:2749] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:18:51.219308 17458 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:18:51.275933 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 86698.1us
I20260709 13:18:51.279529 10766 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:51.281626 17352 raft_consensus.cc:2749] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:18:51.282462 17341 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:18:51.276607 10942 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
I20260709 13:18:51.285840 10766 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:51.252082 17415 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:51.301800 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:51.267160 17184 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.017s	sys 0.002s
I20260709 13:18:51.213163 17386 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:51.312469 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 164060us
W20260709 13:18:51.324856 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:51.254017 17310 consensus_queue.cc:1059] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.025s
I20260709 13:18:51.245296 17398 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:51.343772 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 58163us
I20260709 13:18:51.275300 17387 raft_consensus.cc:2749] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:51.353938 10867 raft_consensus.cc:1240] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 18. Current term is 19. Ops: []
I20260709 13:18:51.278460 17404 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.257656 17312 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:51.306515 17352 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.254586 17411 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:51.330843 17380 raft_consensus.cc:3055] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:18:51.383987 17457 log.cc:927] Time spent T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
W20260709 13:18:51.385061 17453 log.cc:927] Time spent T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
I20260709 13:18:51.362609 17364 raft_consensus.cc:2749] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:18:51.387738 17472 raft_consensus.cc:2804] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:51.388245 17472 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:51.388619 17472 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:51.388837 17478 consensus_queue.cc:1048] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:18:51.394387 10948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41020 (request call id 2964) took 1077 ms. Trace:
I20260709 13:18:51.395713 10948 rpcz_store.cc:276] 0709 13:18:50.316911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:50.320697 (+  3786us) service_pool.cc:224] Handling call
0709 13:18:51.394169 (+1073472us) spinlock_profiling.cc:243] Waited 451 ms on lock 0x7b5400376754. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:51.394309 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1038301440}
I20260709 13:18:51.396173 10943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 1179) took 1068 ms. Trace:
I20260709 13:18:51.396879 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:51.398038 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:51.398765 10948 raft_consensus.cc:2410] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 22 because replica has last-logged OpId of term: 20 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
I20260709 13:18:51.402992 17308 raft_consensus.cc:3055] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:18:51.396634 10943 rpcz_store.cc:276] 0709 13:18:50.327882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:50.352771 (+ 24889us) service_pool.cc:224] Handling call
0709 13:18:50.353086 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:50.993338 (+640252us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:51.395919 (+402581us) spinlock_profiling.cc:243] Waited 634 ms on lock 0x7b540018e2d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:51.396111 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1457119104}
I20260709 13:18:51.403687 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:51.404389 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:51.404779 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:51.405161 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:51.405400 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:51.405781 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:51.406255 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:51.406472 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:51.331064 17456 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:18:51.287978 17269 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:51.414757 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:18:51.420807 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:51.508940 10943 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
I20260709 13:18:51.403604 17308 raft_consensus.cc:740] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:51.381383 17380 raft_consensus.cc:740] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:18:51.442303 17461 log.cc:927] Time spent T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:18:51.370383 17450 consensus_queue.cc:1048] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.311s
I20260709 13:18:51.374842 17228 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:18:51.446588 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:51.447557 10949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 2619) took 1679 ms. Trace:
I20260709 13:18:51.448330 17269 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:51.447710 17437 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:51.414515 17364 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.360327 17184 consensus_queue.cc:1048] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.081s
I20260709 13:18:51.449002 11508 raft_consensus.cc:3055] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:18:51.449811 17448 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:51.449934 17477 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.451704 17158 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:51.453212 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:51.716854 10944 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 16.
I20260709 13:18:51.456261 17411 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:51.458827 17456 raft_consensus.cc:697] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:51.366647 17310 consensus_queue.cc:1243] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer 468a8fd46e824f62b9c2c0d3532a6d26 log is divergent from this leader: its last log entry 21.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 13:18:51.459316 10945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41020 (request call id 2959) took 1143 ms. Trace:
I20260709 13:18:51.728747 10945 rpcz_store.cc:276] 0709 13:18:50.315472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:50.319624 (+  4152us) service_pool.cc:224] Handling call
0709 13:18:51.459104 (+1139480us) spinlock_profiling.cc:243] Waited 211 ms on lock 0x7b54001e2b54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:51.459252 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":485504896}
I20260709 13:18:51.371902 17089 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:18:51.380393 17449 log.cc:927] Time spent T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.322s	user 0.001s	sys 0.000s
W20260709 13:18:51.466514 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 213364us
I20260709 13:18:51.729151 17437 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:51.527597 17369 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.730684 17089 raft_consensus.cc:697] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:51.731866 17369 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:51.732395 17089 consensus_queue.cc:237] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:51.732671 17362 consensus_queue.cc:1048] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:18:51.382663 17352 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:51.479979 17462 log.cc:927] Time spent T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 13:18:51.734902 17362 consensus_queue.cc:1048] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:18:51.493439 17312 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:51.494436 17398 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:51.369401 17404 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:51.736076 17352 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:51.737226 17404 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:51.408404 10941 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
I20260709 13:18:51.504006 10694 catalog_manager.cc:5697] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:51.574498 10766 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:51.737740 17380 consensus_queue.cc:260] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:51.738685 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 164315us
W20260709 13:18:51.700048 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 435038us
I20260709 13:18:51.739089 17380 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:51.739682 10766 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:51.741024 10766 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:51.741465 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:51.741614 17312 raft_consensus.cc:2749] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:18:51.742290 17312 raft_consensus.cc:2749] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:18:51.742152 17308 consensus_queue.cc:260] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:51.743003 17312 raft_consensus.cc:2749] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:51.743165 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:51.745143 17419 log.cc:927] Time spent T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 1.286s	user 0.000s	sys 0.002s
I20260709 13:18:51.745193 10799 raft_consensus.cc:1275] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:18:51.694097 17327 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.660046 17350 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:51.746621 17327 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:51.747195 17350 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:51.695441 17381 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.704501 11508 raft_consensus.cc:740] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:51.748281 17381 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:51.749423 11508 consensus_queue.cc:260] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:51.719305 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 420725us
I20260709 13:18:51.722865 10948 raft_consensus.cc:2410] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 24 because replica has last-logged OpId of term: 23 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 11.
I20260709 13:18:51.514238 16911 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:51.752220 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:51.752526 16911 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:51.753898 17308 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:18:51.754087 17265 consensus_queue.cc:1059] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:18:51.755255 16911 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:51.756189 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:51.756709 17402 raft_consensus.cc:3055] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:51.757095 17402 raft_consensus.cc:740] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:51.757519 10945 raft_consensus.cc:1240] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 13:18:51.757943 17402 consensus_queue.cc:260] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:51.758939 17407 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.759317 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:51.759466 17407 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:51.760011 10945 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 19.
I20260709 13:18:51.761037 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:51.696914 10949 rpcz_store.cc:276] 0709 13:18:49.767859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:49.767996 (+   137us) service_pool.cc:224] Handling call
0709 13:18:49.769821 (+  1825us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:50.230578 (+460757us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:51.447274 (+1216696us) spinlock_profiling.cc:243] Waited 415 ms on lock 0x7b54002a58d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:51.447468 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":955265408}
I20260709 13:18:51.713158 10950 raft_consensus.cc:1240] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 19. Current term is 20. Ops: []
I20260709 13:18:51.763545 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:51.752240 10948 raft_consensus.cc:1275] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:18:51.724790 17477 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:51.330188 17415 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:51.727396 17456 consensus_queue.cc:237] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 9, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:51.734186 17369 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:51.460153 17446 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:51.717376 17364 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:51.749666 17313 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:51.764971 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:51.784049 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:51.784977 10945 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
W20260709 13:18:51.774320 17438 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.704s	user 0.013s	sys 0.017s
I20260709 13:18:51.779829 17372 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.765834 10948 raft_consensus.cc:1275] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 15. (term mismatch)
I20260709 13:18:51.792296 10948 pending_rounds.cc:85] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 14
I20260709 13:18:51.792570 10948 pending_rounds.cc:107] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 22.15
I20260709 13:18:51.792883 10948 raft_consensus.cc:2889] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:51.794333 17350 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:51.766913 11508 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:51.768774 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:51.804796 10795 raft_consensus.cc:2410] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 20 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:18:51.770303 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:51.770800 10766 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:51.771186 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:51.806118 10793 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 21.
I20260709 13:18:51.806284 10797 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 16.
I20260709 13:18:51.805636 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:51.807126 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:51.807791 10797 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:51.809625 11507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56416 (request call id 1011) took 1461 ms. Trace:
I20260709 13:18:51.810050 11507 rpcz_store.cc:276] 0709 13:18:50.348582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:50.349956 (+  1374us) service_pool.cc:224] Handling call
0709 13:18:50.406296 (+ 56340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:50.985069 (+578773us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:51.809357 (+824288us) spinlock_profiling.cc:243] Waited 576 ms on lock 0x7b54005f0354. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:51.809563 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1325941120}
I20260709 13:18:51.727520 10944 raft_consensus.cc:1240] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 18. Current term is 19. Ops: []
I20260709 13:18:51.811291 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:51.811726 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:51.812070 10793 raft_consensus.cc:2393] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 19: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:51.813174 10944 raft_consensus.cc:1275] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:18:51.815477 17269 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:51.816043 17269 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:51.816572 10841 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:51.817509 10841 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:51.818109 17407 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:51.818295 10841 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:51.774221 10949 raft_consensus.cc:2393] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 21: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:51.709656 17458 raft_consensus.cc:697] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:18:51.794498 17438 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.722s	user 0.013s	sys 0.017s
I20260709 13:18:51.796330 17381 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:51.763552 17228 raft_consensus.cc:697] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:51.774763 17362 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.773628 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:51.783378 17402 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:51.842561 10950 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 21.
I20260709 13:18:51.843039 10841 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:51.843852 10950 raft_consensus.cc:1240] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 23. Current term is 24. Ops: []
I20260709 13:18:51.819289 17448 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:18:51.819777 17475 raft_consensus.cc:2804] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:51.819965 17364 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:51.816301 17477 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:51.818151 17269 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:51.826349 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:51.828289 10944 raft_consensus.cc:1275] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:18:51.807425 10945 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
I20260709 13:18:51.835232 17312 raft_consensus.cc:2804] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:18:51.726152 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:51.835893 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:51.891057 10948 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 20.
I20260709 13:18:51.892611 10948 raft_consensus.cc:1240] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:51.893275 10800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 3025) took 1493 ms. Trace:
I20260709 13:18:51.893656 10800 rpcz_store.cc:276] 0709 13:18:50.386673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:50.583214 (+196541us) service_pool.cc:224] Handling call
0709 13:18:51.880089 (+1296875us) spinlock_profiling.cc:243] Waited 66.6 ms on lock 0x7b54000d9654. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:51.880220 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":153111936}
W20260709 13:18:51.873845 17440 consensus_meta.cc:287] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.766s	user 0.010s	sys 0.001s
I20260709 13:18:51.896675 10948 raft_consensus.cc:1275] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 11. (term mismatch)
I20260709 13:18:51.897086 10948 pending_rounds.cc:85] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 10
I20260709 13:18:51.897353 10948 pending_rounds.cc:107] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260709 13:18:51.897624 10948 raft_consensus.cc:2889] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:51.847314 17448 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:51.818406 17446 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.846542 11478 leader_election.cc:400] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 election: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for earlier term 21. Current term is 22.
I20260709 13:18:51.863281 17475 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:51.877285 10766 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:51.816589 17386 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:51.880919 10796 raft_consensus.cc:1240] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 23. Current term is 24. Ops: [23.12-23.12]
I20260709 13:18:51.882006 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:51.846463 17338 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:51.832053 11508 raft_consensus.cc:1275] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:18:51.912812 10766 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:51.837085 10949 raft_consensus.cc:1275] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:18:51.890077 17312 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:51.874056 17381 raft_consensus.cc:2749] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:18:51.825522 17327 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:51.901319 10948 raft_consensus.cc:1275] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:18:51.873558 17480 raft_consensus.cc:2804] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:51.840797 17273 consensus_queue.cc:1059] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.052s
I20260709 13:18:51.893350 17479 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:51.843755 10797 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
I20260709 13:18:51.880016 17352 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.896253 10944 raft_consensus.cc:1275] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:18:51.905933 17364 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.891010 17339 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.892139 17265 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.793217 17313 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:51.926699 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 82529.2us
I20260709 13:18:51.835866 17372 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:51.862648 17362 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:51.907095 17448 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:51.842130 17411 consensus_queue.cc:1059] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.054s
W20260709 13:18:51.917765 17314 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 21 current term: 21 is higher.
I20260709 13:18:51.912925 17475 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:51.917230 17386 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:18:51.880216 17387 consensus_queue.cc:1048] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 13:18:51.966409 10874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2539) took 1842 ms. Trace:
I20260709 13:18:51.962664 11478 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:51.923604 17312 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:51.921767 17338 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:51.928743 17480 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:51.900265 17440 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:51.917500 17269 consensus_queue.cc:1048] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.050s
I20260709 13:18:51.915527 17446 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:51.997617 17474 raft_consensus.cc:2749] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:18:51.934106 17478 consensus_queue.cc:1048] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.051s
W20260709 13:18:52.006589 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 44023.9us
I20260709 13:18:51.979097 17273 raft_consensus.cc:3055] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:18:51.994825 10874 rpcz_store.cc:276] 0709 13:18:50.124139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:50.181210 (+ 57071us) service_pool.cc:224] Handling call
0709 13:18:50.181588 (+   378us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:18:50.181984 (+   396us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:50.182034 (+    50us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:18:50.182697 (+   663us) log.cc:844] Serialized 78 byte log entry
0709 13:18:51.770984 (+1588287us) spinlock_profiling.cc:243] Waited 30.7 ms on lock 0x7b54003e8620. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1f090 00007f9c80e2c6b8 00007f9c80eb592b 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c
0709 13:18:51.771040 (+    56us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:18:51.782716 (+ 11676us) log.cc:844] Serialized 14 byte log entry
0709 13:18:51.902267 (+119551us) log.cc:844] Serialized 14 byte log entry
0709 13:18:51.934640 (+ 32373us) log.cc:844] Serialized 14 byte log entry
0709 13:18:51.959147 (+ 24507us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:51.966015 (+  6868us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:51.966088 (+    73us) raft_consensus.cc:1685] finished
0709 13:18:51.966107 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:51.966334 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":81228,"prepare.queue_time_us":15817,"prepare.run_cpu_time_us":150,"prepare.run_wall_time_us":3532,"raft.queue_time_us":75473,"raft.run_cpu_time_us":416,"raft.run_wall_time_us":26986,"spinlock_wait_cycles":91567232,"thread_start_us":113138,"threads_started":2,"wal-append.queue_time_us":165736,"wal-append.run_cpu_time_us":2568,"wal-append.run_wall_time_us":1048925}
I20260709 13:18:51.960448 17158 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:51.957031 17352 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:51.964148 17364 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:51.953647 17450 consensus_queue.cc:1048] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:18:51.978991 17373 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:51.953140 17437 consensus_queue.cc:1059] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 13:18:51.980419 17327 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:51.936563 17458 consensus_queue.cc:237] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:51.964967 17479 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:52.013854 17476 consensus_queue.cc:1048] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.119s
I20260709 13:18:52.012369 17338 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:52.012535 17480 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:52.016285 10946 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:52.017503 17440 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:51.986455 17386 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:52.028851 17483 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:18:51.966518 17339 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:52.036053 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:52.037473 11512 raft_consensus.cc:1240] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 19. Current term is 20. Ops: []
I20260709 13:18:52.039011 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:18:51.983011 17388 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.919s	user 0.018s	sys 0.064s
I20260709 13:18:51.978061 17265 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:52.065224 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:52.077972 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:52.023408 17231 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:52.080204 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:52.017938 17089 raft_consensus.cc:3055] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:18:52.081736 11504 raft_consensus.cc:1275] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:18:52.082078 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:52.082677 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:51.958124 17228 consensus_queue.cc:237] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:52.027486 17273 raft_consensus.cc:740] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:18:52.084631 17482 log.cc:927] Time spent T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 13:18:52.114435 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:52.116735 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:52.163319 10876 raft_consensus.cc:2393] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 24: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:18:52.162415 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 314727us
I20260709 13:18:52.118505 10869 raft_consensus.cc:1275] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:18:52.122254 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:52.123266 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:52.076323 17439 consensus_queue.cc:1059] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.068s
I20260709 13:18:52.147716 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:52.119446 17386 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:18:52.034488 17158 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:52.153775 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:51.570811  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:52.155258 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:52.158288 10867 raft_consensus.cc:1275] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:18:52.158416 10842 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:52.200822 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:52.172792 10946 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
I20260709 13:18:52.162815 10874 raft_consensus.cc:2376] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 14. Re-sending same reply.
I20260709 13:18:52.175453 10868 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 21.
I20260709 13:18:52.106442 17437 consensus_queue.cc:1243] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer cb0428cc55984daba1ba63744a5102a2 log is divergent from this leader: its last log entry 16.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
W20260709 13:18:52.123584 17388 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.060s	user 0.018s	sys 0.064s
I20260709 13:18:52.119149 17483 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:52.164501 10841 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:52.187008 10871 raft_consensus.cc:2393] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 17: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:52.188694 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:52.204588 10876 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 16.
I20260709 13:18:52.101114 17327 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:52.148746 17231 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:52.171454 17210 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.711s	user 0.009s	sys 0.028s
W20260709 13:18:52.162611 17432 consensus_meta.cc:287] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.991s	user 0.000s	sys 0.021s
I20260709 13:18:52.222772 11507 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 22.
W20260709 13:18:52.210273 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 297523us
I20260709 13:18:52.079821 17373 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:52.210953 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:52.146993 17089 raft_consensus.cc:740] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:18:52.214087 17490 log.cc:927] Time spent T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
I20260709 13:18:52.214622 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:52.258458 10869 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 20.
I20260709 13:18:52.258667 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:52.208629 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:52.259683 11506 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 19.
I20260709 13:18:52.215981 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:52.260989 10797 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:18:52.216565 10792 raft_consensus.cc:1275] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 12. (term mismatch)
I20260709 13:18:52.264024 10792 pending_rounds.cc:85] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 11
I20260709 13:18:52.264307 10792 pending_rounds.cc:107] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 18.12
I20260709 13:18:52.264539 10792 raft_consensus.cc:2889] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:52.191213 17411 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:52.213269 10915 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:52.218575 17327 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:18:52.187611 17472 consensus_meta.cc:287] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.798s	user 0.018s	sys 0.005s
I20260709 13:18:52.225363 10873 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 21.
W20260709 13:18:52.232333 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 67910.8us
W20260709 13:18:52.236166 17210 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.764s	user 0.009s	sys 0.028s
I20260709 13:18:52.212514 10874 raft_consensus.cc:1275] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:18:52.251513 10868 raft_consensus.cc:1275] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:18:52.208602 17483 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:52.254391 17502 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:18:52.255707 17373 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:52.256470 17089 consensus_queue.cc:260] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:52.256726 17448 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:52.231825 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:52.273062 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 59853.7us
I20260709 13:18:52.295574 10875 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 20.
I20260709 13:18:52.297791 10945 raft_consensus.cc:1240] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 18. Current term is 19. Ops: []
I20260709 13:18:52.300096 10874 raft_consensus.cc:1240] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 16. Current term is 17. Ops: [15.12-16.13]
W20260709 13:18:52.275766 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 117426us
I20260709 13:18:52.175740 17381 consensus_queue.cc:1048] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.081s
I20260709 13:18:52.278482 10873 raft_consensus.cc:1275] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:18:52.279781 17447 consensus_queue.cc:1048] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:18:52.279495 17472 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:52.280328 17411 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:52.283638 10867 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 19.
W20260709 13:18:52.276044 17488 log.cc:927] Time spent T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.290s	user 0.001s	sys 0.000s
I20260709 13:18:52.250038 17432 raft_consensus.cc:2749] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553) with higher term. Message: Invalid argument: T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for earlier term 21. Current term is 22.
I20260709 13:18:52.306529 10869 raft_consensus.cc:1240] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 20. Current term is 21. Ops: [18.13-20.14]
I20260709 13:18:52.286249 10797 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 22.
I20260709 13:18:52.302855 10842 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:52.317642 10942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41020 (request call id 2978) took 1125 ms. Trace:
I20260709 13:18:52.321519 10942 rpcz_store.cc:276] 0709 13:18:51.191924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:51.253428 (+ 61504us) service_pool.cc:224] Handling call
0709 13:18:52.317391 (+1063963us) spinlock_profiling.cc:243] Waited 22.2 ms on lock 0x7b54003d8e54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:52.317561 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":51093888}
I20260709 13:18:52.323348 10842 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:52.297310 17388 raft_consensus.cc:2804] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:52.296316 17327 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:52.330214 10943 raft_consensus.cc:1275] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:18:52.297860 17362 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:52.235368 17231 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:52.341696 17491 log.cc:927] Time spent T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
W20260709 13:18:52.342121 17486 log.cc:927] Time spent T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.293s	user 0.000s	sys 0.001s
I20260709 13:18:52.305431 17499 consensus_queue.cc:1048] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 13:18:52.343034 17479 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:52.343918 17487 log.cc:927] Time spent T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.180s	user 0.000s	sys 0.000s
W20260709 13:18:52.345033 17493 log.cc:927] Time spent T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.261s	user 0.001s	sys 0.000s
I20260709 13:18:52.344933 17474 raft_consensus.cc:3055] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:52.346426 17476 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:52.346859 17269 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:18:52.347311 17313 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:52.351068 17338 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:52.329185 17323 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:18:52.354914 11511 raft_consensus.cc:1240] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:52.356468 10801 raft_consensus.cc:1275] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:18:52.269927 17497 consensus_queue.cc:1048] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.084s
I20260709 13:18:52.336155 17089 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:52.339881 17372 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:52.374099 17474 raft_consensus.cc:740] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:52.362577 17327 raft_consensus.cc:697] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 17 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:52.358613 17438 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:52.367264 17364 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:52.371312 17352 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:52.367846 17446 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:52.385344 17494 log.cc:927] Time spent T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.278s	user 0.000s	sys 0.001s
I20260709 13:18:52.310582 17273 consensus_queue.cc:260] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:52.374202 17451 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:52.414324 17439 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:52.389173 17323 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:18:52.407068 17415 consensus_meta.cc:287] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.619s	user 0.012s	sys 0.000s
I20260709 13:18:52.401695 17269 raft_consensus.cc:697] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:52.384601 17476 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:52.430142 17339 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:52.437037 17273 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:52.405941 17438 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:52.438095 17265 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:52.437913 17323 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:18:52.516680 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:52.439955 17231 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:52.558209 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:52.595176 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:52.626484 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:52.668430 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:52.760586 17416 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:52.761488 17416 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:52.761790 17327 consensus_queue.cc:237] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:52.760980 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:52.429672 17451 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:52.809023 10799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3031) took 1058 ms. Trace:
I20260709 13:18:52.454138 17499 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
W20260709 13:18:52.801555 10767 leader_election.cc:336] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Timed out: RequestConsensusVote RPC to 127.9.116.130:37895 timed out after 1.679s (SENT)
I20260709 13:18:52.815764 10767 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:52.809478 10799 rpcz_store.cc:276] 0709 13:18:51.751090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:51.761757 (+ 10667us) service_pool.cc:224] Handling call
0709 13:18:51.762134 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:51.762570 (+   436us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:51.762605 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:51.763079 (+   474us) log.cc:844] Serialized 26 byte log entry
0709 13:18:51.957570 (+194491us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:52.003530 (+ 45960us) log.cc:844] Serialized 14 byte log entry
0709 13:18:52.129450 (+125920us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:52.129561 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:52.808723 (+679162us) raft_consensus.cc:1685] finished
0709 13:18:52.808756 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:52.808952 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":46101,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":55,"raft.queue_time_us":123006,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":9635,"thread_start_us":297357,"threads_started":2,"wal-append.queue_time_us":319107}
I20260709 13:18:52.434217 17439 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:52.445081 17415 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:52.789095 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 465852us
I20260709 13:18:52.763221 17338 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:52.813197 17339 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:52.755280 17500 consensus_queue.cc:1048] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.512s
I20260709 13:18:52.821787 17352 raft_consensus.cc:2749] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:18:52.829278 17402 consensus_meta.cc:287] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.966s	user 0.013s	sys 0.001s
I20260709 13:18:52.828914 17448 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:52.827374 10842 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:52.561662 17474 consensus_queue.cc:260] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:52.823751 17499 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:52.844822 17411 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:52.870872 17210 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.018s	sys 0.076s
I20260709 13:18:52.839156 17338 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:52.904348 10841 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.037s	sys 0.025s
I20260709 13:18:52.868564 17446 raft_consensus.cc:2804] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:18:52.904843 10841 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.037s	sys 0.025s
W20260709 13:18:52.907990 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 635400us
I20260709 13:18:52.851074 17339 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:52.901057 17474 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:52.926273 17476 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:52.860821 17269 consensus_queue.cc:237] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:52.939491 17511 log.cc:927] Time spent T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 13:18:52.941216 10873 raft_consensus.cc:1240] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 19. Current term is 20. Ops: []
I20260709 13:18:52.942624 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:52.943336 10873 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 17.
I20260709 13:18:52.944315 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:52.905740 17210 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.019s	sys 0.076s
W20260709 13:18:52.863001 10868 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.031s	sys 0.030s
W20260709 13:18:52.958585 10868 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.032s	sys 0.030s
I20260709 13:18:52.960466 10868 raft_consensus.cc:1275] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:18:52.899812 17399 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:52.875845 10875 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.016s	sys 0.034s
W20260709 13:18:52.963639 10875 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.016s	sys 0.035s
I20260709 13:18:52.912220 17446 raft_consensus.cc:697] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:18:52.972458 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:52.975831 17505 log.cc:927] Time spent T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:18:52.949612 17472 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:52.976583 17514 consensus_queue.cc:1059] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:18:52.984930 17517 raft_consensus.cc:3055] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:18:52.985479 17517 raft_consensus.cc:740] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:52.986514 17517 consensus_queue.cc:260] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:52.973477 17399 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:52.970589 17507 log.cc:927] Time spent T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.152s	user 0.000s	sys 0.002s
I20260709 13:18:53.005187 17406 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:53.025398 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:52.990541 17399 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:52.973587 17438 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:53.025008 17439 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:53.029449 17416 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:53.050066 17415 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:53.015162 17406 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:53.073410 17386 consensus_meta.cc:287] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.862s	user 0.011s	sys 0.000s
I20260709 13:18:53.073910 17386 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:53.075646 17339 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:53.085536 17489 log.cc:927] Time spent T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.802s	user 0.014s	sys 0.007s
W20260709 13:18:53.087695 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:53.093704 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:53.055907 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 238605us
I20260709 13:18:53.098642 10767 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:53.099707 10794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3037) took 1332 ms. Trace:
I20260709 13:18:53.100236 10794 rpcz_store.cc:276] 0709 13:18:51.767613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:51.770732 (+  3119us) service_pool.cc:224] Handling call
0709 13:18:51.778980 (+  8248us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:51.779351 (+   371us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:51.779407 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:51.787659 (+  8252us) log.cc:844] Serialized 26 byte log entry
0709 13:18:52.021112 (+233453us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:52.021161 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:52.021245 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:53.084649 (+1063404us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:18:53.093329 (+  8680us) raft_consensus.cc:1685] finished
0709 13:18:53.093364 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:53.099631 (+  6267us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19205,"raft.queue_time_us":3994,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":15477,"spinlock_wait_cycles":27279616,"thread_start_us":226709,"threads_started":1,"wal-append.queue_time_us":386226}
I20260709 13:18:53.066146 17446 consensus_queue.cc:237] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:53.111581 17516 log.cc:927] Time spent T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 13:18:53.112748 17506 log.cc:927] Time spent T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.205s	user 0.001s	sys 0.000s
I20260709 13:18:53.115381 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:53.119267 10947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 1186) took 2079 ms. Trace:
I20260709 13:18:53.119949 10947 rpcz_store.cc:276] 0709 13:18:51.039699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:51.082430 (+ 42731us) service_pool.cc:224] Handling call
0709 13:18:51.082727 (+   297us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:51.102313 (+ 19586us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:51.102342 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:51.105054 (+  2712us) log.cc:844] Serialized 26 byte log entry
0709 13:18:52.217424 (+1112370us) spinlock_profiling.cc:243] Waited 4.82 ms on lock 0x7b1800386b60. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a550d1 00007f9c80e70a0f
0709 13:18:52.373768 (+156344us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:18:52.373805 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:52.373869 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:53.118808 (+744939us) raft_consensus.cc:1685] finished
0709 13:18:53.118838 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:53.119164 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33510,"raft.queue_time_us":148,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":4836,"spinlock_wait_cycles":11075456,"thread_start_us":156176,"threads_started":1,"wal-append.queue_time_us":1745644}
I20260709 13:18:53.123055 17228 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:53.123765 17228 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:53.125156 10796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2371) took 2599 ms. Trace:
I20260709 13:18:53.125535 10796 rpcz_store.cc:276] 0709 13:18:50.525939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:50.820763 (+294824us) service_pool.cc:224] Handling call
0709 13:18:51.880616 (+1059853us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:51.907876 (+ 27260us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:53.124913 (+1217037us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54000d9650. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:53.125066 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2376796928}
I20260709 13:18:53.126183 17228 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:53.127866 17483 consensus_meta.cc:287] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.848s	user 0.012s	sys 0.000s
I20260709 13:18:53.128504 17483 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:53.134020 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:53.134758 10942 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 23.
I20260709 13:18:53.136734 17387 raft_consensus.cc:2749] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:18:53.137255 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:53.137768 17510 consensus_queue.cc:1048] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:18:53.139009 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:53.139669 10948 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 17.
I20260709 13:18:53.139848 17510 consensus_queue.cc:1048] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:53.135412 10767 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:18:53.144855 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.144641 10767 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:53.149058 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:53.149215 10767 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:53.146371 10947 raft_consensus.cc:1240] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:53.149959 10942 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:18:53.155682 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 208489us
I20260709 13:18:53.156459 10841 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:53.157618 17339 raft_consensus.cc:2804] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:53.158151 17339 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:53.158468 17499 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:53.158656 17339 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:53.163810 10767 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:53.143213 17497 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:53.148113 17387 raft_consensus.cc:2804] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:53.177765 10841 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:53.156843 17314 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:53.155596 17500 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:53.180985 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:53.163273 17510 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:18:53.165004 17338 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:53.183983 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:53.172412 17352 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:53.174727 11477 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:53.184417 10947 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
I20260709 13:18:53.189036 10942 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 19.
I20260709 13:18:53.195029 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:53.179858 17387 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:53.195736 10942 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:53.197109 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:53.197834 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:18:53.203044 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.163131 17476 consensus_queue.cc:1048] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:18:53.206633 10947 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
W20260709 13:18:53.175417 17456 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.001s	user 0.007s	sys 0.015s
I20260709 13:18:53.175380 17509 consensus_queue.cc:1048] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:18:53.209854 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:53.217965 10947 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
I20260709 13:18:53.218825 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:53.219244 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:53.219784 10949 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
I20260709 13:18:53.172222 17411 consensus_queue.cc:1059] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 13:18:53.214215 10841 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:53.222340 11477 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:53.191114 17352 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:53.225903 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:53.221065 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:53.176908 17497 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:53.234431 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.238722 10942 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
I20260709 13:18:53.180756 17512 consensus_queue.cc:1048] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:18:53.194662 17510 raft_consensus.cc:697] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:53.206828 17387 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:53.231819 10841 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:18:53.243979 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.233023 10949 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
W20260709 13:18:53.217566 17456 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.059s	user 0.007s	sys 0.015s
I20260709 13:18:53.247015 11477 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:53.187739 17314 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:53.227475 17472 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:18:53.364920 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 133195us
W20260709 13:18:53.361378 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 114511us
I20260709 13:18:53.243881 17352 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:53.246186 17500 raft_consensus.cc:3055] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:18:53.367910 10841 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:53.242844 17304 raft_consensus.cc:2804] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:18:53.371044 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.247994 17364 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:53.361061 17416 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:18:53.378175 17323 consensus_meta.cc:287] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.572s	user 0.008s	sys 0.003s
W20260709 13:18:53.359679 17273 consensus_meta.cc:287] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.800s	user 0.012s	sys 0.001s
I20260709 13:18:53.379132 17304 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:53.383140 17364 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:53.390870 10841 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:53.407689 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:53.407686 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:53.409623 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:53.369663 17411 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:53.370939 17472 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:53.248895 17228 consensus_queue.cc:1048] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.135s
I20260709 13:18:53.371981 17313 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:18:53.438944 17406 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:53.418241 17304 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:53.373821 17500 raft_consensus.cc:740] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:53.431566 11505 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
I20260709 13:18:53.432497 11509 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
I20260709 13:18:53.391803 17269 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:53.386840 17456 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:18:53.393113 17480 consensus_meta.cc:287] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 1.273s	user 0.009s	sys 0.006s
I20260709 13:18:53.420585 17364 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:53.360854 17338 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:53.401459 17386 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:53.438637 17472 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:18:53.410588 17323 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:53.446938 17452 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 2.005s	user 0.014s	sys 0.011s
I20260709 13:18:53.404220 17416 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:53.523934 17416 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:53.479487 17383 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:53.524981 17383 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:53.484077 17452 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.043s	user 0.015s	sys 0.011s
W20260709 13:18:53.530181 17089 consensus_meta.cc:287] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 1.138s	user 0.011s	sys 0.001s
I20260709 13:18:53.439275 17510 consensus_queue.cc:237] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:53.484759 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.480562 17338 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:53.495473 17472 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:53.499184 17520 log.cc:927] Time spent T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:18:53.463588 17210 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:53.464246 17446 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:53.467150 17438 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:53.513057 17517 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:18:53.514233 17530 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:18:53.515226 17523 log.cc:927] Time spent T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:18:53.520715 17323 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:53.521608 17312 consensus_meta.cc:287] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 1.500s	user 0.011s	sys 0.004s
I20260709 13:18:53.448441 17313 raft_consensus.cc:697] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 19 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:53.468215 17456 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:53.481181 17406 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:18:53.559437 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.562286 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:53.564383 10941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 2976) took 2375 ms. Trace:
I20260709 13:18:53.566769 11504 raft_consensus.cc:1240] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 23. Current term is 24. Ops: [23.15-23.15]
I20260709 13:18:53.438365 17411 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:53.570405 10941 rpcz_store.cc:276] 0709 13:18:51.189383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:51.738935 (+549552us) service_pool.cc:224] Handling call
0709 13:18:51.739208 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:51.763093 (+ 23885us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:51.763126 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:51.763175 (+    49us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:18:51.763219 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:53.564029 (+1800810us) spinlock_profiling.cc:243] Waited 23.7 ms on lock 0x7b54001707d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:53.564102 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:53.564303 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":54544640}
W20260709 13:18:53.548604 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 157720us
I20260709 13:18:53.471580 17480 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:53.585881 11505 raft_consensus.cc:2393] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 22: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:53.589237 10841 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:53.548550 17446 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:53.591780 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:53.610281 11505 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 14.
W20260709 13:18:53.592346 17531 log.cc:927] Time spent T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 13:18:53.594997 17383 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:53.595546 17323 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:53.587040 17530 raft_consensus.cc:697] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:53.569218 17456 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:53.587568 17406 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:53.599063 17497 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:53.599401 17467 log.cc:927] Time spent T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 1.783s	user 0.009s	sys 0.009s
I20260709 13:18:53.601428 17411 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:53.602283 17314 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:53.570627 17438 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:53.603688 17475 consensus_meta.cc:287] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 1.599s	user 0.011s	sys 0.004s
I20260709 13:18:53.546772 17339 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:53.611822 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 192588us
I20260709 13:18:53.614564 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:53.616707 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:53.562904 17210 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:53.608933 17446 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:18:53.634325 17518 log.cc:927] Time spent T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.653s	user 0.000s	sys 0.001s
I20260709 13:18:53.616740 10916 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:53.625681 11508 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
W20260709 13:18:53.640419 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.572649 17312 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:53.640940 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.616047 17323 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:53.645628 17438 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:53.647554 17339 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:53.649001 17446 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:53.652179 17406 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:53.652599 17515 log.cc:927] Time spent T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.690s	user 0.002s	sys 0.000s
I20260709 13:18:53.550071 17500 consensus_queue.cc:260] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:53.627153 17383 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:53.659348 17313 consensus_queue.cc:237] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:53.661372 17158 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:53.637603 17475 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:53.664083 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:53.666211 17323 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:53.657616 11512 raft_consensus.cc:1275] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:18:53.674737 17383 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:53.685117 17383 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:53.687906 11478 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:53.678833 10873 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 19.
I20260709 13:18:53.690126 17450 raft_consensus.cc:2764] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
W20260709 13:18:53.690411 17474 consensus_meta.cc:287] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.741s	user 0.005s	sys 0.003s
W20260709 13:18:53.669247 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 52601.6us
I20260709 13:18:53.690912 17450 consensus_queue.cc:1048] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:18:53.691273 17472 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:53.692764 17455 raft_consensus.cc:3055] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:18:53.693159 17455 raft_consensus.cc:740] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:53.694039 17455 consensus_queue.cc:260] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:53.695840 17406 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:53.696548 17269 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:53.696439 17406 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:53.697046 17269 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:53.697794 17450 consensus_queue.cc:1059] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:18:53.698747 17450 consensus_queue.cc:1243] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer e01ecd008edd4aa2b6b2b47efbc2ea87 log is divergent from this leader: its last log entry 19.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:18:53.699246 17472 raft_consensus.cc:3055] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:18:53.699433 17468 log.cc:927] Time spent T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 2.256s	user 0.023s	sys 0.026s
I20260709 13:18:53.699687 17472 raft_consensus.cc:740] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:53.700698 17472 consensus_queue.cc:260] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:53.701685 17472 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:53.701972 10767 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:53.671263 17327 raft_consensus.cc:2804] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:53.704437 11478 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:53.705545 17530 consensus_queue.cc:237] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:53.711328 17314 consensus_queue.cc:1048] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 13:18:53.713750 17543 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:53.714303 17543 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:53.706615 17411 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:18:53.716001 17411 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:53.716387 17411 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:53.716543 17543 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:53.705622 17406 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:53.718243 17406 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:53.718719 17406 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:53.720134 17210 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:53.721555 10872 raft_consensus.cc:1240] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:53.726156 10876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 1032) took 1492 ms. Trace:
W20260709 13:18:53.727741 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 121502us
I20260709 13:18:53.726680 10876 rpcz_store.cc:276] 0709 13:18:52.233942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:52.234189 (+   247us) service_pool.cc:224] Handling call
0709 13:18:52.234630 (+   441us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:18:52.294952 (+ 60322us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:52.294986 (+    34us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:18:52.316869 (+ 21883us) log.cc:844] Serialized 78 byte log entry
0709 13:18:52.850104 (+533235us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:18:53.095070 (+244966us) log.cc:844] Serialized 14 byte log entry
0709 13:18:53.095383 (+   313us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:53.095477 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:53.633655 (+538178us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:18:53.722290 (+ 88635us) raft_consensus.cc:1685] finished
0709 13:18:53.725918 (+  3628us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:53.726089 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24310,"prepare.queue_time_us":380960,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":2474,"raft.queue_time_us":44928,"raft.run_cpu_time_us":258,"raft.run_wall_time_us":10822,"spinlock_wait_cycles":25076608,"thread_start_us":710200,"threads_started":2,"wal-append.queue_time_us":623751}
I20260709 13:18:53.726156 10875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 1035) took 1481 ms. Trace:
I20260709 13:18:53.711974 17500 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:53.729611 10875 rpcz_store.cc:276] 0709 13:18:52.244538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:52.296308 (+ 51770us) service_pool.cc:224] Handling call
0709 13:18:52.323916 (+ 27608us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:52.324276 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:52.324316 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:52.336645 (+ 12329us) log.cc:844] Serialized 26 byte log entry
0709 13:18:52.964102 (+627457us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:52.964146 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:52.964218 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:53.652045 (+687827us) spinlock_profiling.cc:243] Waited 29.4 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:18:53.722435 (+ 70390us) raft_consensus.cc:1685] finished
0709 13:18:53.722465 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:53.726097 (+  3632us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":22372,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":82,"spinlock_wait_cycles":67599232,"thread_start_us":627001,"threads_started":1,"wal-append.queue_time_us":625153}
I20260709 13:18:53.675565 17364 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:53.725212 17314 raft_consensus.cc:2749] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:18:53.727627 17455 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:53.739710 17415 consensus_queue.cc:1048] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:18:53.726118 17452 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:53.745644 17338 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:53.747406 11478 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:53.754004 17406 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:18:53.755385 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:53.755591 10916 connection.cc:256] RPC call timeout handler was delayed by -1.1606781410000622s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1625s for a response
I20260709 13:18:53.727324 17327 raft_consensus.cc:697] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 17 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:53.737470 10841 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:53.743515 17480 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:53.743796 17450 consensus_queue.cc:1048] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:18:53.729281 17474 consensus_queue.cc:1048] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.015s
W20260709 13:18:53.744481 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.757650 10871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 1031) took 1524 ms. Trace:
I20260709 13:18:53.762732 10871 rpcz_store.cc:276] 0709 13:18:52.233631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:52.236697 (+  3066us) service_pool.cc:224] Handling call
0709 13:18:52.237362 (+   665us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:52.256446 (+ 19084us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:18:52.256479 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:52.257424 (+   945us) log.cc:844] Serialized 501 byte log entry
0709 13:18:52.374412 (+116988us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:52.374456 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:53.746520 (+1372064us) spinlock_profiling.cc:243] Waited 18.7 ms on lock 0x7b540037b254. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:53.746594 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:53.746663 (+    69us) raft_consensus.cc:1685] finished
0709 13:18:53.746684 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:53.757580 (+ 10896us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42937600,"thread_start_us":116643,"threads_started":1,"wal-append.queue_time_us":553388}
I20260709 13:18:53.765439 10872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2545) took 1651 ms. Trace:
I20260709 13:18:53.765894 10872 rpcz_store.cc:276] 0709 13:18:52.114337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:52.114484 (+   147us) service_pool.cc:224] Handling call
0709 13:18:52.114926 (+   442us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:53.724893 (+1609967us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:53.765150 (+ 40257us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b54001e0354. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:53.765365 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3694338816}
I20260709 13:18:53.744915 17452 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:53.755970 10916 connection.cc:256] RPC call timeout handler was delayed by -1.1559502330001123s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15900000000000003s for a response
I20260709 13:18:53.758927 10841 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:53.771859 10943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 2620) took 2685 ms. Trace:
I20260709 13:18:53.785156 10943 rpcz_store.cc:276] 0709 13:18:51.086306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:51.559794 (+473488us) service_pool.cc:224] Handling call
0709 13:18:52.318617 (+758823us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:52.338472 (+ 19855us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:53.771619 (+1433147us) spinlock_profiling.cc:243] Waited 758 ms on lock 0x7b54003d8e50. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:53.771761 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1744296704}
I20260709 13:18:53.820853 10916 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:53.827689 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 68885.1us
I20260709 13:18:53.828960 10841 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:53.737344 17265 consensus_queue.cc:1059] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:18:53.847587 10916 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:53.850371 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.768998 17312 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:53.786674 17210 consensus_queue.cc:1059] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.082s
I20260709 13:18:53.868832 17210 raft_consensus.cc:3055] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:53.869285 17210 raft_consensus.cc:740] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:53.768342 17304 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:53.872807 17304 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:53.873898 17265 raft_consensus.cc:3055] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:53.751922 17352 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:53.874256 17265 raft_consensus.cc:740] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:53.770493 17364 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:53.875167 17265 consensus_queue.cc:260] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:53.875878 17352 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:53.759981 17498 raft_consensus.cc:2804] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:18:53.888164 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 59248.1us
I20260709 13:18:53.891937 11511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56416 (request call id 1019) took 1783 ms. Trace:
I20260709 13:18:53.894188 11511 rpcz_store.cc:276] 0709 13:18:52.108422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:52.108547 (+   125us) service_pool.cc:224] Handling call
0709 13:18:52.122643 (+ 14096us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:52.392392 (+269749us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:53.891604 (+1499212us) spinlock_profiling.cc:243] Waited 218 ms on lock 0x7b54008b79d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:53.891854 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":501607808}
I20260709 13:18:53.785571 17313 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:53.770452 17475 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:53.887893 17498 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:53.913398 10792 raft_consensus.cc:1240] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:18:53.887676 17552 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:53.919147 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:53.770955 17388 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:53.915056 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:53.921792 10797 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 21.
I20260709 13:18:53.921876 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:53.922571 10794 raft_consensus.cc:2368] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 19. Current term is 20.
I20260709 13:18:53.846719 17158 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:18:53.922859 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:53.913715 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 66201.4us
W20260709 13:18:53.917522 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.917426 10793 raft_consensus.cc:1275] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:18:53.918113 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:53.924135 17312 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:53.952548 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:53.952399 17312 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:53.953342 10794 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 17.
I20260709 13:18:53.954329 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:53.954922 17089 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:53.955036 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:53.955358 17452 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:53.955789 17476 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:53.956228 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:53.845685 17282 consensus_queue.cc:1059] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.111s
I20260709 13:18:53.956854 10795 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 15.
I20260709 13:18:53.957275 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:53.957724 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:53.958022 10800 raft_consensus.cc:2393] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 21: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:53.915895 17497 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:53.960052 17497 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:53.888391 17440 consensus_queue.cc:1048] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.079s
I20260709 13:18:53.924451 17536 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:53.961989 17536 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:53.962404 17497 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:53.923183 17388 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:53.963016 17269 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:53.964215 17536 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:53.957285 17282 consensus_queue.cc:1243] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer e01ecd008edd4aa2b6b2b47efbc2ea87 log is divergent from this leader: its last log entry 18.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:18:53.965188 17269 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:53.769055 17387 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:53.967403 17387 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:53.916647 17539 raft_consensus.cc:2804] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:53.969715 17312 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:53.955533 17089 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:53.971910 17228 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:53.972515 17089 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:53.972433 17228 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:53.912420 17503 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:53.974030 17503 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:53.975282 10800 raft_consensus.cc:1275] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:18:53.975916 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:53.976102 17503 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:53.976600 10798 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
W20260709 13:18:53.976830 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 78361.1us
I20260709 13:18:53.977581 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:53.977954 11478 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:53.978165 10798 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 14.
I20260709 13:18:53.916752 17406 raft_consensus.cc:2749] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:18:53.956271 17476 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:53.980906 17476 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:53.981206 11512 raft_consensus.cc:1240] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:18:53.982048 11506 raft_consensus.cc:1275] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:18:53.916939 17313 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:53.983933 17313 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:53.984117 10794 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 24.
I20260709 13:18:53.985023 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:53.985581 10794 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
I20260709 13:18:53.986590 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:53.987211 10794 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 15.
I20260709 13:18:53.988672 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:53.995282 17451 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:53.870260 17210 consensus_queue.cc:260] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:53.879191 17327 consensus_queue.cc:237] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:53.996249 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:53.989508 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:53.997154 10798 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
I20260709 13:18:53.998172 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:53.998732 10798 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:18:53.998992 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 82301.5us
W20260709 13:18:53.999558 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.999969 10841 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:53.991119 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:18:53.991291 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.991253 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:53.992388 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:54.009053 10842 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:53.919206 17498 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:53.924806 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:54.011093 10797 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 19.
I20260709 13:18:54.012113 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:54.012948 10797 raft_consensus.cc:2393] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 17: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:54.013638 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:53.974606 17228 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:54.015897 17480 raft_consensus.cc:2804] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:54.020398 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:54.021065 10842 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:54.021744 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:54.024501 10792 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
I20260709 13:18:54.017664 10795 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
I20260709 13:18:54.023969 11506 raft_consensus.cc:1240] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:18:53.924402 10916 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:54.027803 17452 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:54.010684 17455 consensus_queue.cc:1048] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 13:18:54.028221 17452 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:54.028636 17452 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:18:54.033052 17526 log.cc:927] Time spent T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.592s	user 0.008s	sys 0.023s
I20260709 13:18:54.034291 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:54.014707 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:54.040779 10797 raft_consensus.cc:2393] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 19: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:54.032028 17552 raft_consensus.cc:2749] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:54.041769 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:54.043012 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 118677us
W20260709 13:18:54.044102 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:54.045332 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:54.046756 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:54.047420 10797 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
I20260709 13:18:54.048377 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:54.051589 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:54.056758 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:54.001703 17539 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:54.058743 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:54.059715 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:54.016237 17304 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:18:54.081035 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:54.081714 11509 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 17.
I20260709 13:18:54.082633 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:54.085382 11506 raft_consensus.cc:2393] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 16: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:54.022759 17480 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:54.071444 10916 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:54.060842 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:54.061363 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:54.078675 10793 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
I20260709 13:18:54.060151 17530 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:53.990731 17323 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:54.063658 10792 raft_consensus.cc:2368] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 21. Current term is 22.
I20260709 13:18:54.094550 10916 leader_election.cc:400] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 19. Current term is 20.
I20260709 13:18:54.096838 10916 leader_election.cc:403] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:18:54.097886 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:54.094991 10795 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:54.099255 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:54.075093 17455 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:54.078147 10798 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
I20260709 13:18:54.074891 17228 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:18:53.999118 17456 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:54.009235 17411 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:54.151279 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:54.152818 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:18:54.153276 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 58898.8us
I20260709 13:18:54.154258 10795 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
I20260709 13:18:54.071125 17539 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:54.156030 10916 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:54.160745 10916 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:54.161965 10916 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:54.163072 17383 raft_consensus.cc:2804] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:18:54.168717 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:54.087705 17304 raft_consensus.cc:697] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 22 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:54.105240 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:54.070945 17536 consensus_queue.cc:1048] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.069s
I20260709 13:18:54.083482 17406 consensus_queue.cc:1059] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.075s
W20260709 13:18:54.174162 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 214779us
I20260709 13:18:54.178992 10916 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:54.183275 10796 raft_consensus.cc:1240] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:18:54.123864 17282 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:18:54.096362 17530 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:54.098274 17480 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:54.209437 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:18:54.211122 10946 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41020 (request call id 2973) took 3203 ms (client timeout 1951 ms). Trace:
I20260709 13:18:54.214727 11509 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 22.
W20260709 13:18:54.215847 11507 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56416 (request call id 1013) took 2174 ms (client timeout 2127 ms). Trace:
W20260709 13:18:54.216801 11507 rpcz_store.cc:269] 0709 13:18:52.035915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:52.036028 (+   113us) service_pool.cc:224] Handling call
0709 13:18:54.208691 (+2172663us) spinlock_profiling.cc:243] Waited 68.3 ms on lock 0x7b540046d2d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:54.210072 (+  1381us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":156990336}
I20260709 13:18:54.167342 17338 raft_consensus.cc:2804] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:54.163662 17543 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:54.158741 17228 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
W20260709 13:18:54.214663 10946 rpcz_store.cc:269] 0709 13:18:50.980258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:50.980417 (+   159us) service_pool.cc:224] Handling call
0709 13:18:54.183949 (+3203532us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54000172d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:54.184090 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2346360192}
I20260709 13:18:54.208372 17282 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:54.175148 17383 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:54.184437 17497 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:54.210131 17416 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:18:54.237171 10801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 916) took 2022 ms. Trace:
I20260709 13:18:54.245081 10801 rpcz_store.cc:276] 0709 13:18:52.214892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:52.216901 (+  2009us) service_pool.cc:224] Handling call
0709 13:18:52.217161 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:52.385036 (+167875us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:54.236837 (+1851801us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b54002120d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:54.237098 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":319589248}
W20260709 13:18:54.238085 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 217194us
I20260709 13:18:54.207425 17476 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:54.214174 17530 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:54.211294 17552 raft_consensus.cc:2804] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:54.166105 17455 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:54.254640 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:54.263248 11507 raft_consensus.cc:2393] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 16: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:54.221983 17338 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:54.238972 17228 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:54.241287 17282 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:18:54.211536 10795 raft_consensus.cc:1275] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:18:54.242760 17158 raft_consensus.cc:2749] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:18:54.241470 17383 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:54.244910 17416 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:54.263844 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 73543.7us
I20260709 13:18:54.279453 10916 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:54.236063 17313 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:54.267921 17338 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:54.248111 17497 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:54.306372 11507 raft_consensus.cc:1275] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 13:18:54.304298 10867 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 2558) took 2179 ms (client timeout 2082 ms). Trace:
I20260709 13:18:54.259265 17552 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:18:54.308048 10867 rpcz_store.cc:269] 0709 13:18:52.125269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:52.212992 (+ 87723us) service_pool.cc:224] Handling call
0709 13:18:54.304110 (+2091118us) spinlock_profiling.cc:243] Waited 24.3 ms on lock 0x7b5400078354. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:54.304231 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":55803648}
I20260709 13:18:54.310827 17388 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:54.254472 17327 consensus_queue.cc:1048] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 13:18:54.279929 17416 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:18:54.259778 17476 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:54.321599 10916 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:54.322305 10842 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:54.330610 11507 raft_consensus.cc:1275] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:18:54.332363 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:54.333046 11509 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 19.
I20260709 13:18:54.332953 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:54.319612 17552 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:54.334066 10792 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 21.
I20260709 13:18:54.342514 17497 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:54.345429 17338 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:54.327481 17563 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:18:54.336449 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:54.349795 10797 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 19.
I20260709 13:18:54.348119 17562 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:54.337795 10795 raft_consensus.cc:1240] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:18:54.356616 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:54.357537 10797 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
I20260709 13:18:54.360121 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:54.360813 10792 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 17.
I20260709 13:18:54.348671 17563 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:54.363327 17563 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:54.367035 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:54.364184 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:54.262276 17304 consensus_queue.cc:237] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:54.348965 17407 raft_consensus.cc:2764] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
I20260709 13:18:54.374867 17388 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:54.365391 17323 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:54.371276 17411 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:54.381587 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:54.383683 10799 raft_consensus.cc:1240] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:18:54.390385 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:54.391355 17480 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:54.393447 17480 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:54.379415 17541 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:54.379119 17456 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:54.378676 17568 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:54.379801 17566 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:54.402452 10766 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:54.416061 17411 raft_consensus.cc:2749] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:54.421960 17539 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:54.425500 17228 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:54.426254 17313 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:54.426035 17452 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:54.427783 17438 raft_consensus.cc:3055] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:18:54.428202 17438 raft_consensus.cc:740] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:54.428150 17228 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:54.371548 17383 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:54.431207 17455 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:54.367897 17562 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:54.433466 17476 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:54.434070 17562 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:54.435104 17383 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:54.429422 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:54.435585 17446 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:54.429126 17438 consensus_queue.cc:260] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:54.436933 17438 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:18:54.406668 17338 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:18:54.430409 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 108190us
I20260709 13:18:54.439836 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:54.373682 11511 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 19.
I20260709 13:18:54.443722 10798 raft_consensus.cc:1240] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:18:54.445843 10842 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:54.447258 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:54.447479 10842 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:54.447971 10799 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 14.
I20260709 13:18:54.449777 10766 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:54.451012 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:54.451506 10842 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:54.456053 10915 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:54.461826 10915 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:54.463030 10915 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:54.467662 10799 raft_consensus.cc:1240] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:54.461862 17476 consensus_queue.cc:1048] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.015s
I20260709 13:18:54.444799 17568 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:54.473032 17568 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:54.444957 17566 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:54.474720 17539 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:54.447193 17383 consensus_queue.cc:1048] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 13:18:54.461112 17562 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:54.475796 17566 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:54.475875 17562 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:54.476418 17562 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:54.476305 17383 consensus_queue.cc:1048] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:18:54.478868 17327 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:54.479351 17327 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:54.479713 17327 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:54.481379 17282 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:54.455435 17228 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:54.482887 17407 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:18:54.483179 17228 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:54.483453 17407 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:54.483697 17228 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:54.483924 17407 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:18:54.487884 10842 leader_election.cc:400] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 election: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 21. Current term is 22.
I20260709 13:18:54.488260 10842 leader_election.cc:403] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 13:18:54.489217 10799 raft_consensus.cc:1240] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 16. Current term is 17. Ops: [16.14-16.14]
I20260709 13:18:54.487679 10766 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:54.468657 10915 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:54.498515 10766 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:54.490031 10842 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:18:54.538266 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:54.462095 17446 raft_consensus.cc:2749] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:18:54.555361 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 56930.1us
I20260709 13:18:54.563573 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:54.570421 10947 raft_consensus.cc:1240] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:18:54.571600 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:54.571807 10948 raft_consensus.cc:1275] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:18:54.572175 10947 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:18:54.573079 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:54.559584 10916 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:18:54.578647 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:54.576974 10874 raft_consensus.cc:1240] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:18:54.497071 17451 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:54.546674 10842 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:54.547993 17517 raft_consensus.cc:2804] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:54.542337 17448 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:18:54.496650 17386 raft_consensus.cc:2749] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:54.582628 10871 raft_consensus.cc:1240] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:18:54.584846 10874 raft_consensus.cc:1240] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:18:54.585361 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:54.547370 17089 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:18:54.540185 17566 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:54.585889 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:54.462235 17338 consensus_queue.cc:1059] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.116s
I20260709 13:18:54.580233 17447 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:54.603118 10941 raft_consensus.cc:2393] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 17: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:54.600903 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:54.604563 17447 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:54.605332 10949 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:18:54.604557 10870 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 21.
I20260709 13:18:54.605208 17566 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:54.600278 10868 raft_consensus.cc:1240] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:54.607401 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:54.608142 10876 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 20.
I20260709 13:18:54.603650 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:54.594697 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:54.609334 10947 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 22.
I20260709 13:18:54.581911 17411 raft_consensus.cc:2764] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 13:18:54.610698 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:54.610821 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:54.551548 17406 consensus_queue.cc:1048] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 13:18:54.606920 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:54.616217 10875 raft_consensus.cc:2393] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 19: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:54.618106 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:54.619257 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:54.590426 17269 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:54.599902 17383 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:18:54.581305 17416 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:54.548996 17498 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:54.591796 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:54.629944 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:54.630029 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:18:54.627031 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 67582.2us
I20260709 13:18:54.610344 17566 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:54.627218 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:54.634644 10947 raft_consensus.cc:3055] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:18:54.635406 10947 raft_consensus.cc:740] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:54.610935 17517 raft_consensus.cc:697] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:54.638900 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:54.633886 10916 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:54.640537 10874 raft_consensus.cc:2393] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 19: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:54.647339 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:54.647681 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:54.648105 10875 raft_consensus.cc:2368] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 16. Current term is 17.
I20260709 13:18:54.650929 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:18:54.650956 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:54.653326 10944 raft_consensus.cc:2368] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 21. Current term is 22.
I20260709 13:18:54.657634 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:54.658077 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:54.659348 10944 raft_consensus.cc:1240] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:54.622612 17338 raft_consensus.cc:3055] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:18:54.593614 17455 consensus_queue.cc:1048] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 13:18:54.636543 10947 consensus_queue.cc:260] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:54.663367 10876 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 24.
I20260709 13:18:54.665594 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:54.666767 10876 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 21.
W20260709 13:18:54.695622 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:54.701342 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:54.628970 17383 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:54.704635 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:54.705660 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:54.706267 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:54.707365 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:54.708683 10868 raft_consensus.cc:2393] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 18: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:54.708998 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:54.709496 10869 raft_consensus.cc:1240] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:18:54.705495 10916 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:54.709510 10875 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:54.708479 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:54.714994 10944 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:54.638057 17269 raft_consensus.cc:697] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:54.720337 10949 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 24.
I20260709 13:18:54.724684 10948 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
I20260709 13:18:54.725103 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:54.719177 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:54.650429 17411 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:54.658665 17497 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:54.730156 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:54.732815 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:54.663712 17338 raft_consensus.cc:740] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:54.742352 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:54.736053 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:54.788177 10868 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:54.710191 17383 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:18:54.702807 17089 consensus_queue.cc:1048] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:18:54.701083 17539 consensus_queue.cc:1048] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.084s
I20260709 13:18:54.734205 17406 raft_consensus.cc:2804] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:18:54.768306 10767 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:54.755125 11537 heartbeater.cc:646] Failed to heartbeat to 127.9.116.190:36007 (3 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.9.116.190:36007 timed out after 15.000s (SENT)
I20260709 13:18:54.761005 17578 consensus_queue.cc:1059] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 13:18:54.807013 17476 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:18:54.763597 17483 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:54.785054 10841 leader_election.cc:400] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 election: Vote denied by peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553) with higher term. Message: Invalid argument: T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 21. Current term is 22.
I20260709 13:18:54.785072 10948 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:54.783350 17452 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:54.808197 10767 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:54.809536 17386 consensus_queue.cc:1059] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:54.790339 17564 raft_consensus.cc:2804] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:54.793974 17447 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:54.827183 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 194740us
I20260709 13:18:54.773283 17517 consensus_queue.cc:237] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:54.821846 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:54.786525 17411 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:54.814134 10841 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:54.811556 17476 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:54.790658 17497 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:54.821854 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 116418us
I20260709 13:18:54.847149 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:54.827418 17564 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:54.851464 17452 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:18:54.855099 17573 log.cc:927] Time spent T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 13:18:54.855221 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:18:54.856305 17576 log.cc:927] Time spent T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 13:18:54.856640 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:54.851342 17578 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:54.857436 10950 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 16.
I20260709 13:18:54.857826 17578 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:54.853607 17452 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:54.859246 17575 log.cc:927] Time spent T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 13:18:54.860208 17578 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:54.860816 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:54.861263 17452 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:54.862002 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:54.863143 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:54.864120 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:54.864787 10869 raft_consensus.cc:2393] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 18: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:54.865559 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:54.862002 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:54.867420 10795 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:54.858438 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:54.823837 17364 consensus_queue.cc:1059] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:18:54.845753 17571 raft_consensus.cc:2804] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:18:54.871259 17571 raft_consensus.cc:697] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:54.871479 17364 raft_consensus.cc:3055] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:54.872008 17364 raft_consensus.cc:740] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:54.872149 17571 consensus_queue.cc:237] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:54.872874 17364 consensus_queue.cc:260] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:54.805524 17269 consensus_queue.cc:237] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:54.874181 17552 consensus_meta.cc:287] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.510s	user 0.004s	sys 0.002s
W20260709 13:18:54.878010 17543 consensus_meta.cc:287] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.639s	user 0.007s	sys 0.000s
I20260709 13:18:54.878445 17543 raft_consensus.cc:2749] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 19. Current term is 20.
I20260709 13:18:54.879756 17448 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 21. Current term is 22.
W20260709 13:18:54.880963 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:54.881762 17327 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:54.882314 17498 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:54.883718 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:54.841378 17476 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:18:54.884471 10798 raft_consensus.cc:2393] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 20: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:54.885100 10795 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
I20260709 13:18:54.885357 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:54.885999 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:54.871582 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:54.887544 17564 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:54.887840 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:54.888962 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:54.889356 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:54.890125 17406 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:54.890291 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:54.890460 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:54.891182 10950 raft_consensus.cc:2393] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 19: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:54.891258 10946 raft_consensus.cc:2393] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 19: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:54.891939 17282 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:54.892169 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:54.892799 10946 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 20.
I20260709 13:18:54.893605 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:54.893749 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:18:54.894769 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:54.894604 10875 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 21.
W20260709 13:18:54.849661 17577 log.cc:927] Time spent T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.187s	user 0.001s	sys 0.000s
I20260709 13:18:54.895779 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:54.897071 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:54.894635 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:54.851812 17416 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:54.898180 17525 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:54.779276 10949 raft_consensus.cc:2376] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17. Re-sending same reply.
I20260709 13:18:54.890570 17406 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:54.866535 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:54.954214 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:54.955204 10869 raft_consensus.cc:2393] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 19: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:54.956132 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:54.956714 10869 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:54.956967 17497 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:54.905159 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:54.959157 10801 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:54.958904 17525 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:54.960402 17416 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:54.961090 17525 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:54.960949 17416 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:54.963189 17416 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:54.963347 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:54.963989 10799 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:18:54.828660 17338 consensus_queue.cc:260] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:54.966351 17338 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:18:54.900596 17541 raft_consensus.cc:2749] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:54.907342 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:54.967483 10876 raft_consensus.cc:2410] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 17 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 13:18:54.967876 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:54.905875 10868 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 20.
I20260709 13:18:54.971800 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:54.972699 10868 raft_consensus.cc:2393] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 20: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:54.892089 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:18:54.909459 17530 consensus_meta.cc:287] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.642s	user 0.007s	sys 0.003s
I20260709 13:18:54.911099 10948 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
I20260709 13:18:54.974807 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:54.886595 10795 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
I20260709 13:18:54.911895 17503 consensus_queue.cc:1048] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:54.980262 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:54.916386 17210 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:54.923758 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:54.924041 10841 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:54.981060 10948 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:18:54.981716 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 57744.5us
I20260709 13:18:54.976924 11478 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:54.874222 17364 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:54.984062 11478 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:54.985086 10841 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:54.926246 10947 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:54.987970 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:54.988603 10795 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 19.
I20260709 13:18:54.989576 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:54.990206 10795 raft_consensus.cc:2368] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 16. Current term is 17.
I20260709 13:18:54.926771 17327 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:54.926874 17440 raft_consensus.cc:3055] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:18:54.878010 17563 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:54.870473 17228 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:54.878660 17407 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:54.931239 10767 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:55.010244 10869 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 18.
W20260709 13:18:55.010851 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 79736.6us
I20260709 13:18:54.886505 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:54.948190 10944 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
I20260709 13:18:54.949692 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:55.011706 10949 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:18:55.012326 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.013114 10767 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:55.005402 17475 raft_consensus.cc:2804] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:54.898262 17089 raft_consensus.cc:2749] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:18:54.991091 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:54.906018 11504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56410 (request call id 1029) took 1533 ms. Trace:
I20260709 13:18:55.015156 10950 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:54.993867 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:55.016582 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:54.993561 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:54.996935 10799 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 25.
I20260709 13:18:55.017922 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:55.020066 10949 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
I20260709 13:18:55.021673 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:55.015326 11504 rpcz_store.cc:276] 0709 13:18:53.373109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:53.373289 (+   180us) service_pool.cc:224] Handling call
0709 13:18:53.373675 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:53.570920 (+197245us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:54.905760 (+1334840us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b54001d1354. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:54.905961 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":443472768}
I20260709 13:18:54.986553 10841 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:55.023115 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:55.023800 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:55.008944 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:55.025574 10941 raft_consensus.cc:2368] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 19. Current term is 20.
I20260709 13:18:55.025820 10842 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:54.997350 17539 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:55.016860 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:55.031224 10795 raft_consensus.cc:2410] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:18:55.023938 11509 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:55.026291 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:55.031466 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:55.024817 10868 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 19.
I20260709 13:18:55.027148 10841 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:54.874646 17552 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:55.000466 10946 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 20.
I20260709 13:18:55.033392 10948 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
I20260709 13:18:55.037305 17475 raft_consensus.cc:697] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:55.037554 17498 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:18:55.038123 17498 raft_consensus.cc:697] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:55.038591 17228 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:55.038288 17475 consensus_queue.cc:237] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:55.039274 17448 raft_consensus.cc:2749] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:18:55.039041 17498 consensus_queue.cc:237] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:55.040365 17563 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:55.035161 10874 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 19.
I20260709 13:18:55.041930 10841 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:55.043013 17480 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:18:55.005834 17440 raft_consensus.cc:740] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:55.039062 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:55.044024 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:55.044338 10868 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 17.
I20260709 13:18:55.045188 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:55.045539 17440 consensus_queue.cc:260] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:55.046283 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:55.047040 10868 raft_consensus.cc:2393] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 18: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:55.007460 17450 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:18:55.037045 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:55.047748 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:55.049958 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:55.050810 11507 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 17.
I20260709 13:18:55.051892 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:55.052698 11504 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 21.
I20260709 13:18:55.054296 10842 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:55.021673 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:55.003424 10801 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
I20260709 13:18:55.034454 10800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2405) took 1111 ms. Trace:
I20260709 13:18:55.010923 10870 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 22.
I20260709 13:18:55.089438 11508 raft_consensus.cc:1240] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:55.104815 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:55.104542 10800 rpcz_store.cc:276] 0709 13:18:53.923478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:53.958570 (+ 35092us) service_pool.cc:224] Handling call
0709 13:18:53.969598 (+ 11028us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:53.975620 (+  6022us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:55.034221 (+1058601us) spinlock_profiling.cc:243] Waited 5.35 ms on lock 0x7b54003cd054. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:55.034398 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12306176}
I20260709 13:18:54.992909 17583 consensus_queue.cc:1048] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.073s
I20260709 13:18:54.987468 17530 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:55.005455 17587 raft_consensus.cc:2804] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:55.107113 10796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2389) took 1194 ms. Trace:
I20260709 13:18:55.115763 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2566) took 1948 ms. Trace:
I20260709 13:18:55.069818 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:55.142314 11512 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:18:55.141772 10873 rpcz_store.cc:276] 0709 13:18:53.166980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:53.689014 (+522034us) service_pool.cc:224] Handling call
0709 13:18:53.689363 (+   349us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:53.689743 (+   380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:53.689772 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:53.690337 (+   565us) log.cc:844] Serialized 52 byte log entry
0709 13:18:53.751789 (+ 61452us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:53.784170 (+ 32381us) log.cc:844] Serialized 14 byte log entry
0709 13:18:53.784390 (+   220us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:53.784462 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:53.908895 (+124433us) spinlock_profiling.cc:243] Waited 30.6 ms on lock 0x7b1800317f80. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc83f3 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:18:54.929249 (+1020354us) spinlock_profiling.cc:243] Waited 39.9 ms on lock 0x7b1800317f80. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc83f3 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:18:55.115511 (+186262us) raft_consensus.cc:1685] finished
0709 13:18:55.115537 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:55.115698 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52915,"prepare.queue_time_us":34552,"prepare.run_cpu_time_us":412,"prepare.run_wall_time_us":2782,"raft.queue_time_us":2578,"raft.run_cpu_time_us":265,"raft.run_wall_time_us":30565,"spinlock_wait_cycles":162154112,"thread_start_us":60811,"threads_started":1,"wal-append.queue_time_us":268642}
I20260709 13:18:54.970333 10916 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:55.105643 11511 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 20.
I20260709 13:18:55.008376 17415 consensus_queue.cc:1059] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.080s
I20260709 13:18:55.086828 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:55.138305 10841 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:55.026566 17497 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:18:55.056823 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:18:55.144528 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 174274us
I20260709 13:18:55.119756 11506 raft_consensus.cc:1240] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 13:18:55.056018 10947 raft_consensus.cc:1275] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 9. (term mismatch)
I20260709 13:18:55.141005 10796 rpcz_store.cc:276] 0709 13:18:53.912426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:53.912568 (+   142us) service_pool.cc:224] Handling call
0709 13:18:53.952326 (+ 39758us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:54.189396 (+237070us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:55.106774 (+917378us) spinlock_profiling.cc:243] Waited 231 ms on lock 0x7b54000bc7d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:55.107040 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":530669952}
I20260709 13:18:55.163597 10947 pending_rounds.cc:85] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 8
I20260709 13:18:55.058385 10842 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:55.148557 11509 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 20.
I20260709 13:18:55.151407 11477 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:55.163975 10947 pending_rounds.cc:107] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 18.9
I20260709 13:18:55.152762 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:55.164497 10947 raft_consensus.cc:2889] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:55.165246 10873 raft_consensus.cc:2393] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 19: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:55.155876 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:18:55.035595 17590 log.cc:927] Time spent T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:18:55.116811 10946 raft_consensus.cc:3055] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:55.166091 10946 raft_consensus.cc:740] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:55.166064 10876 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:18:55.168556 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 110299us
I20260709 13:18:55.169917 10842 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:55.165300 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:55.172134 11508 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 21.
I20260709 13:18:55.023173 17402 consensus_queue.cc:1059] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.091s
I20260709 13:18:55.089046 17525 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:55.091755 17541 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:18:55.177666 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:55.105286 17339 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:55.043565 17480 raft_consensus.cc:697] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:55.182338 10801 raft_consensus.cc:1275] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:18:55.182698 10876 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 18.
W20260709 13:18:55.183465 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.116232 17552 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:55.167507 17416 raft_consensus.cc:2804] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:55.174861 17510 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:18:55.046521 17440 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:55.140349 17587 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:55.186465 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:55.280815 10868 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:18:55.201573 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 313741us
I20260709 13:18:55.283093 10766 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:55.203884 10916 leader_election.cc:400] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 election: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 16. Current term is 17.
I20260709 13:18:55.283944 10916 leader_election.cc:403] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
W20260709 13:18:55.284044 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.187424 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:55.285666 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:55.209605 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:55.211105 11478 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:55.223242 11512 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 19.
I20260709 13:18:55.287986 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:55.288611 10874 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 19.
I20260709 13:18:55.173862 17456 raft_consensus.cc:2749] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:18:55.210482 10915 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:55.056213 17566 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:55.295940 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 92260.1us
I20260709 13:18:55.166569 17568 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:18:55.251173 10948 raft_consensus.cc:1240] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:55.289673 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:55.173672 17589 consensus_queue.cc:1059] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.104s
W20260709 13:18:55.292774 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 81787.7us
I20260709 13:18:55.303772 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:55.304025 11478 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:55.161589 17327 consensus_queue.cc:1059] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 13:18:55.180075 17381 consensus_queue.cc:1048] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.108s
I20260709 13:18:55.308902 10796 raft_consensus.cc:1240] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:18:55.310689 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:18:55.298465 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 88097.5us
I20260709 13:18:55.297726 10916 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:55.313017 10948 raft_consensus.cc:1240] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 23. Current term is 24. Ops: [23.15-23.15]
I20260709 13:18:55.314448 10873 raft_consensus.cc:2393] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 16: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:55.180956 17524 consensus_queue.cc:1059] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.113s
I20260709 13:18:55.208151 17541 raft_consensus.cc:697] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:55.208607 17416 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:18:55.331730 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.333250 11478 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:55.319828 10948 raft_consensus.cc:1240] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:55.320453 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:55.336151 10945 raft_consensus.cc:1240] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 18. Current term is 19. Ops: []
I20260709 13:18:55.288760 17587 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:55.196233 17339 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:55.344676 11478 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:55.305811 17282 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:18:55.360051 17282 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:18:55.360617 17451 consensus_meta.cc:287] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.771s	user 0.014s	sys 0.000s
I20260709 13:18:55.361198 17451 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:18:55.361634 17383 consensus_meta.cc:287] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.558s	user 0.007s	sys 0.003s
I20260709 13:18:55.362195 17383 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:55.364761 17383 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:55.364878 10942 raft_consensus.cc:1240] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:55.373442 17265 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:55.385048 10944 raft_consensus.cc:2393] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 22: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:55.387627 10948 raft_consensus.cc:1240] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 21. Current term is 22. Ops: []
I20260709 13:18:55.336489 17327 consensus_queue.cc:1243] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer 12682eb70bc84c76879e04a8628ee586 log is divergent from this leader: its last log entry 16.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:18:55.349850 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:55.302469 17438 raft_consensus.cc:2749] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 16. Current term is 17.
W20260709 13:18:55.414585 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 61250.7us
I20260709 13:18:55.415815 10948 raft_consensus.cc:1275] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 13:18:55.404465 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 59900.9us
I20260709 13:18:55.417352 10868 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 22.
W20260709 13:18:55.330462 17562 consensus_meta.cc:287] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.853s	user 0.012s	sys 0.000s
I20260709 13:18:55.420498 10950 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 21.
I20260709 13:18:55.422014 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:55.425364 10948 raft_consensus.cc:1240] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 22. Current term is 23. Ops: []
I20260709 13:18:55.427443 10915 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:55.343102 17416 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:55.432461 10947 raft_consensus.cc:2410] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 23 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 13:18:55.434692 10948 raft_consensus.cc:1240] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:55.349089 17583 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 13:18:55.398695 17564 consensus_meta.cc:287] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.511s	user 0.010s	sys 0.000s
I20260709 13:18:55.438630 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:55.360585 17282 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:18:55.401960 17407 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:55.415632 17452 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:55.429328 17478 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.016s	sys 0.044s
I20260709 13:18:55.452595 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:55.461383 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:55.489463 11506 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:18:55.400765 17530 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:55.360303 17566 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:55.459203 11510 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 19.
W20260709 13:18:55.478647 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.420823 17578 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader election won for term 25
W20260709 13:18:55.442193 17476 consensus_meta.cc:287] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.557s	user 0.009s	sys 0.001s
I20260709 13:18:55.492190 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:55.491079 10915 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:55.453523 17450 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:18:55.488778 17478 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.738s	user 0.017s	sys 0.044s
I20260709 13:18:55.461400 17438 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:18:55.516471 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.441721 17451 consensus_queue.cc:1059] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 13:18:55.532788 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:55.533906 10868 raft_consensus.cc:2393] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 24: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:55.447167 17383 consensus_queue.cc:1059] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 13:18:55.431102 17562 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:55.538683 11506 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 23.
I20260709 13:18:55.496495 17583 raft_consensus.cc:697] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 24 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:55.399829 17480 consensus_queue.cc:237] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:55.454312 17579 consensus_queue.cc:1048] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.089s
I20260709 13:18:55.510068 17578 raft_consensus.cc:697] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 25 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:55.493518 17564 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:55.524470 17566 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:18:55.553676 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:55.545444 17089 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:55.556028 10871 raft_consensus.cc:1813] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Handling vote request from an unknown peer cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:55.537842 17438 raft_consensus.cc:697] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 19 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:55.584859 10871 raft_consensus.cc:2393] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 18: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:55.498235 17476 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:55.495190 17452 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:55.511823 17593 raft_consensus.cc:2804] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:18:55.587407 17269 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.023s	sys 0.041s
I20260709 13:18:55.599313 10792 raft_consensus.cc:2393] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 22: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:55.599359 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:55.601855 10948 raft_consensus.cc:1275] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:18:55.605825 17472 raft_consensus.cc:2749] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:18:55.605851 17478 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.607082 17478 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:55.609573 17478 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:55.589033 17517 raft_consensus.cc:2764] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:18:55.523255 17541 consensus_queue.cc:237] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:55.594923 17566 raft_consensus.cc:697] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 19 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:18:55.603739 17269 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.023s	sys 0.042s
I20260709 13:18:55.623310 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:55.604466 17600 raft_consensus.cc:2749] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:18:55.634181 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 482810us
I20260709 13:18:55.634887 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:55.636300 11508 raft_consensus.cc:1240] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 22. Current term is 23. Ops: [22.17-22.17]
W20260709 13:18:55.636962 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 353919us
I20260709 13:18:55.637495 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:55.638231 11512 raft_consensus.cc:2393] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 19: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:55.638463 10868 raft_consensus.cc:2393] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 19: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:18:55.639215 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.646103 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:55.610392 17510 raft_consensus.cc:697] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:55.637701 10766 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:55.652679 11510 raft_consensus.cc:1240] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 18. Current term is 19. Ops: []
I20260709 13:18:55.657119 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:55.625780 17497 raft_consensus.cc:697] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:55.662127 10766 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:55.624259 17583 consensus_queue.cc:237] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:55.672116 17510 consensus_queue.cc:237] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:55.669433 10766 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:55.679087 17566 consensus_queue.cc:237] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:55.679916 17452 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:55.684235 17599 log.cc:927] Time spent T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
I20260709 13:18:55.657879 17607 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:18:55.686019 17564 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:55.686185 10949 raft_consensus.cc:2410] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 21 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 13:18:55.686439 17607 raft_consensus.cc:697] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:55.687423 17607 consensus_queue.cc:237] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:55.688344 10766 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:18:55.689587 17539 consensus_meta.cc:287] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.642s	user 0.005s	sys 0.005s
I20260709 13:18:55.689787 10766 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:55.690027 17539 raft_consensus.cc:2749] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:55.624102 17578 consensus_queue.cc:237] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 23.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:55.690632 17562 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:55.652783 17438 consensus_queue.cc:237] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:55.699232 10869 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 19.
I20260709 13:18:55.707674 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:18:55.708949 10767 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.008s	sys 0.016s
W20260709 13:18:55.709298 10767 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.008s	sys 0.016s
I20260709 13:18:55.712152 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:55.712348 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:55.713032 11505 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
I20260709 13:18:55.713362 10868 raft_consensus.cc:2393] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 16: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:18:55.715440 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 545589us
W20260709 13:18:55.714107 10765 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.017s	sys 0.000s
W20260709 13:18:55.717221 10765 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.017s	sys 0.000s
I20260709 13:18:55.717397 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:55.694175 17595 raft_consensus.cc:2764] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:18:55.719244 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:55.699599 17602 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:55.723701 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:55.721284 10915 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:55.721587 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:55.718008 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.725996 10841 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:55.726260 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 310520us
I20260709 13:18:55.735661 10841 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:55.737982 10841 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:55.730983 10842 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:55.740382 10916 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:55.735219 17564 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:18:55.773823 10950 raft_consensus.cc:1240] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 25 LEADER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 24. Current term is 25. Ops: []
I20260709 13:18:55.737066 17562 consensus_queue.cc:1059] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.037s
I20260709 13:18:55.772850 10916 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:55.778717 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 50866.6us
W20260709 13:18:55.811609 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 66841.1us
I20260709 13:18:55.813700 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:55.811689 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 73781.2us
W20260709 13:18:55.821049 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 48288.2us
I20260709 13:18:55.823029 10916 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:55.735824 17602 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:55.852207 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.819242 17595 raft_consensus.cc:2749] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:18:55.870930 10793 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.038s	sys 0.026s
I20260709 13:18:55.827471 10841 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:55.874966 10793 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.039s	sys 0.026s
W20260709 13:18:55.875039 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:55.877624 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 54668us
I20260709 13:18:55.880138 10916 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:55.883661 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 73021.9us
I20260709 13:18:55.771595 17539 raft_consensus.cc:2749] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:18:55.889571 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:55.894037 10916 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:55.896478 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:55.824729 17579 raft_consensus.cc:2749] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:55.816762 17536 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:18:55.869704 17451 raft_consensus.cc:2749] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:55.811393 17269 raft_consensus.cc:2749] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:18:55.925853 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 109727us
W20260709 13:18:55.881852 17210 consensus_meta.cc:287] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.887s	user 0.009s	sys 0.003s
I20260709 13:18:55.884714 17566 raft_consensus.cc:2749] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:18:55.809563 17452 consensus_queue.cc:1059] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.118s
W20260709 13:18:55.893728 17597 log.cc:927] Time spent T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.293s	user 0.000s	sys 0.001s
I20260709 13:18:55.778339 17440 consensus_queue.cc:1059] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.125s
I20260709 13:18:55.817153 17564 raft_consensus.cc:697] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 23 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:18:55.932468 17609 log.cc:927] Time spent T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.285s	user 0.000s	sys 0.001s
I20260709 13:18:55.888262 17476 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:55.933751 17541 raft_consensus.cc:2749] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:18:55.931098 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 43585.3us
I20260709 13:18:55.887125 17282 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:55.940387 17411 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:55.946713 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:55.970048 10767 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:55.930466 17438 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:55.930241 17536 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:55.903867 17593 raft_consensus.cc:697] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 18 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:55.827761 17587 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:55.962370 17608 log.cc:927] Time spent T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.263s	user 0.000s	sys 0.001s
I20260709 13:18:55.897135 17456 consensus_queue.cc:1048] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.051s
I20260709 13:18:55.826241 17571 raft_consensus.cc:2804] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:18:55.951615 10841 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.116.131:45553 timed out after 1.827s (SENT) [suppressed 22 similar messages]
I20260709 13:18:56.023850 10944 raft_consensus.cc:1240] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 21. Current term is 22. Ops: []
I20260709 13:18:55.973376 10868 raft_consensus.cc:2393] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 18: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:55.967303 17452 consensus_queue.cc:1243] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer 468a8fd46e824f62b9c2c0d3532a6d26 log is divergent from this leader: its last log entry 17.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:18:55.988716 10794 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 16.
I20260709 13:18:55.955009 17595 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:18:55.888736 17497 consensus_queue.cc:237] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:55.998368 17411 raft_consensus.cc:2749] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:18:56.000839 17623 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:18:56.002274 17416 consensus_meta.cc:287] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.540s	user 0.007s	sys 0.000s
I20260709 13:18:56.003851 17588 raft_consensus.cc:2955] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 17 LEADER]: Committing config change with OpId 17.13: config changed from index 8 to 13, VOTER cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:18:56.004492 17564 consensus_queue.cc:237] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:55.993712 17604 log.cc:927] Time spent T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.279s	user 0.000s	sys 0.001s
W20260709 13:18:56.006091 17612 log.cc:927] Time spent T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
I20260709 13:18:56.078094 10943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 2648) took 1515 ms. Trace:
I20260709 13:18:56.078590 10943 rpcz_store.cc:276] 0709 13:18:54.562623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:54.562818 (+   195us) service_pool.cc:224] Handling call
0709 13:18:54.563292 (+   474us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:54.563806 (+   514us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:54.563845 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:54.564453 (+   608us) log.cc:844] Serialized 26 byte log entry
0709 13:18:55.046298 (+481845us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b1800307000. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a550d1 00007f9c80e70a0f
0709 13:18:55.455919 (+409621us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:55.455959 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:55.456022 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:56.074092 (+618070us) raft_consensus.cc:1685] finished
0709 13:18:56.074113 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:56.078017 (+  3904us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":119,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":4912,"spinlock_wait_cycles":29271808,"thread_start_us":409390,"threads_started":1,"wal-append.queue_time_us":1066100}
I20260709 13:18:55.979494 17603 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.013403 10947 raft_consensus.cc:1275] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
W20260709 13:18:56.013460 17629 log.cc:927] Time spent T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:18:56.014048 11511 raft_consensus.cc:2393] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 15: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:56.015064 17587 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:18:56.015599 17601 log.cc:927] Time spent T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
W20260709 13:18:55.994067 17613 log.cc:927] Time spent T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 13:18:55.982312 17536 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:56.017602 17339 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:56.019630 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 329917us
I20260709 13:18:56.138450 17593 consensus_queue.cc:237] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:56.139351 11507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56404 (request call id 1108) took 1071 ms. Trace:
I20260709 13:18:56.139551 10766 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:56.139786 11507 rpcz_store.cc:276] 0709 13:18:55.068397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.069030 (+   633us) service_pool.cc:224] Handling call
0709 13:18:55.069322 (+   292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:55.514678 (+445356us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:55.514723 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:55.520306 (+  5583us) log.cc:844] Serialized 26 byte log entry
0709 13:18:55.817531 (+297225us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:18:55.817574 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:55.817657 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:56.139128 (+321471us) raft_consensus.cc:1685] finished
0709 13:18:56.139148 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:56.139290 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46146,"raft.queue_time_us":182129,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":16806,"thread_start_us":656613,"threads_started":2,"wal-append.queue_time_us":493291}
I20260709 13:18:56.028182 10916 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:56.141176 10766 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:55.898859 17578 raft_consensus.cc:2749] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:18:56.042578 10868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 2576) took 1386 ms. Trace:
I20260709 13:18:56.142375 10868 rpcz_store.cc:276] 0709 13:18:54.655740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.903796 (+1248056us) service_pool.cc:224] Handling call
0709 13:18:56.042503 (+138707us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:56.051416 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:56.047676 17406 consensus_meta.cc:287] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 1.062s	user 0.012s	sys 0.000s
I20260709 13:18:56.038082 17386 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.081197 11504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56404 (request call id 1107) took 1013 ms. Trace:
I20260709 13:18:56.073093 17623 raft_consensus.cc:697] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:56.043759 17595 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:56.090595 17571 raft_consensus.cc:697] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:18:56.091233 17089 consensus_meta.cc:287] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.505s	user 0.005s	sys 0.005s
I20260709 13:18:56.091774 17483 raft_consensus.cc:3055] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:18:56.092151 17602 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:56.090612 10867 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 2387) took 1514 ms (client timeout 1590 ms). Trace:
I20260709 13:18:56.095724 10694 catalog_manager.cc:5697] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:56.098182 10944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 3030) took 1247 ms. Trace:
I20260709 13:18:56.165877 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:56.173444 10944 rpcz_store.cc:276] 0709 13:18:54.892474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.386640 (+494166us) service_pool.cc:224] Handling call
0709 13:18:55.387054 (+   414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:56.024355 (+637301us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:56.097898 (+ 73543us) spinlock_profiling.cc:243] Waited 637 ms on lock 0x7b540039b054. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:56.098114 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1463972736}
I20260709 13:18:56.174463 10868 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 21.
I20260709 13:18:56.101330 17498 raft_consensus.cc:3055] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:18:56.103438 17569 raft_consensus.cc:3055] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:18:56.104526 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49290 (request call id 1043) took 1096 ms. Trace:
I20260709 13:18:56.147946 17476 raft_consensus.cc:2749] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:56.112854 11503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56416 (request call id 1033) took 1062 ms. Trace:
I20260709 13:18:56.113946 17568 raft_consensus.cc:3055] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:18:56.117887 17438 raft_consensus.cc:2749] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:56.119069 10942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 2650) took 1551 ms. Trace:
I20260709 13:18:56.197608 10942 rpcz_store.cc:276] 0709 13:18:54.567182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:54.567327 (+   145us) service_pool.cc:224] Handling call
0709 13:18:54.567587 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:55.365461 (+797874us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:56.118847 (+753386us) spinlock_profiling.cc:243] Waited 797 ms on lock 0x7b5400399754. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:56.119011 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1832960256}
I20260709 13:18:56.199175 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:56.064905 17416 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:56.128607 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 177009us
I20260709 13:18:56.117933 17407 raft_consensus.cc:3055] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:18:56.146130 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:56.220177 11509 raft_consensus.cc:2393] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 19: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:56.172549 17089 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:56.169515 17483 raft_consensus.cc:740] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:56.148485 11504 rpcz_store.cc:276] 0709 13:18:55.068018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.068213 (+   195us) service_pool.cc:224] Handling call
0709 13:18:55.068638 (+   425us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:55.114706 (+ 46068us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:55.114741 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:55.115297 (+   556us) log.cc:844] Serialized 26 byte log entry
0709 13:18:55.490079 (+374782us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:18:55.490115 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:55.490304 (+   189us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:56.081004 (+590700us) raft_consensus.cc:1685] finished
0709 13:18:56.081022 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:56.081128 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27300,"raft.queue_time_us":116922,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":9235,"thread_start_us":374455,"threads_started":1,"wal-append.queue_time_us":496872}
I20260709 13:18:56.081197 10795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2419) took 1578 ms. Trace:
I20260709 13:18:56.150977 10766 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:56.152973 17621 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:56.153772 17622 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.180045 17569 raft_consensus.cc:740] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:56.163654 17595 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:56.168576 17617 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:56.168373 17571 consensus_queue.cc:237] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:56.100517 10949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41014 (request call id 2663) took 1277 ms. Trace:
I20260709 13:18:56.294543 10949 rpcz_store.cc:276] 0709 13:18:54.823395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.021434 (+198039us) service_pool.cc:224] Handling call
0709 13:18:56.100293 (+1078859us) spinlock_profiling.cc:243] Waited 630 ms on lock 0x7b54000aafd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:56.100452 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1449560960}
I20260709 13:18:56.295910 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:56.296558 10949 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 16.
I20260709 13:18:56.297829 10949 raft_consensus.cc:1275] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 13:18:56.172767 10867 rpcz_store.cc:269] 0709 13:18:54.576088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:54.577325 (+  1237us) service_pool.cc:224] Handling call
0709 13:18:56.090380 (+1513055us) spinlock_profiling.cc:243] Waited 998 ms on lock 0x7b540006ccd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:56.090550 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2295433984}
I20260709 13:18:56.175104 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:56.299109 10799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 927) took 1815 ms. Trace:
W20260709 13:18:56.183461 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:56.180235 10869 rpcz_store.cc:276] 0709 13:18:55.008389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.290048 (+281659us) service_pool.cc:224] Handling call
0709 13:18:56.104357 (+814309us) spinlock_profiling.cc:243] Waited 355 ms on lock 0x7b540019ffd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:56.104467 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":817473792}
I20260709 13:18:56.299600 10799 rpcz_store.cc:276] 0709 13:18:54.483819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.027183 (+543364us) service_pool.cc:224] Handling call
0709 13:18:55.027752 (+   569us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:18:55.445094 (+417342us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:55.445127 (+    33us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:18:55.464951 (+ 19824us) log.cc:844] Serialized 553 byte log entry
0709 13:18:55.728216 (+263265us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:56.012778 (+284562us) spinlock_profiling.cc:243] Waited 5.69 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:18:56.024514 (+ 11736us) log.cc:844] Serialized 14 byte log entry
0709 13:18:56.024772 (+   258us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:56.024850 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:56.298762 (+273912us) raft_consensus.cc:1685] finished
0709 13:18:56.298893 (+   131us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:56.299049 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30763,"prepare.queue_time_us":554704,"raft.queue_time_us":307686,"raft.run_cpu_time_us":301,"raft.run_wall_time_us":27903,"spinlock_wait_cycles":13077888,"thread_start_us":818993,"threads_started":3,"wal-append.queue_time_us":392977}
I20260709 13:18:56.301565 10949 raft_consensus.cc:1240] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:18:56.301981 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:56.160940 17566 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:18:56.153200 17352 raft_consensus.cc:2804] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:18:56.189500 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:56.188341 11503 rpcz_store.cc:276] 0709 13:18:55.050238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.065944 (+ 15706us) service_pool.cc:224] Handling call
0709 13:18:55.087706 (+ 21762us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:55.489527 (+401821us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:55.489558 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:55.497014 (+  7456us) log.cc:844] Serialized 26 byte log entry
0709 13:18:55.716966 (+219952us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:18:55.722227 (+  5261us) log.cc:844] Serialized 14 byte log entry
0709 13:18:55.722448 (+   221us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:55.722517 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:56.112550 (+390033us) raft_consensus.cc:1685] finished
0709 13:18:56.112584 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:56.112773 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":127763,"prepare.queue_time_us":5084,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":190699,"raft.run_cpu_time_us":330,"raft.run_wall_time_us":66098,"thread_start_us":583846,"threads_started":2,"wal-append.queue_time_us":574899}
I20260709 13:18:56.167354 17641 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:56.195634 10871 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 2395) took 1595 ms (client timeout 1820 ms). Trace:
W20260709 13:18:56.331637 10871 rpcz_store.cc:269] 0709 13:18:54.600226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:54.606661 (+  6435us) service_pool.cc:224] Handling call
0709 13:18:56.195400 (+1588739us) spinlock_profiling.cc:243] Waited 942 ms on lock 0x7b540037b254. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:56.195562 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2166110592}
W20260709 13:18:56.195777 10792 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2424) took 1350 ms (client timeout 1662 ms). Trace:
I20260709 13:18:56.203390 10945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 1205) took 1609 ms. Trace:
I20260709 13:18:56.164034 17447 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.218109 10942 raft_consensus.cc:2393] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 22: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:56.163800 17594 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:56.221601 11509 raft_consensus.cc:1240] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 24 LEADER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 23. Current term is 24. Ops: [23.15-23.15]
I20260709 13:18:56.147154 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:18:56.331379 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 137947us
I20260709 13:18:56.246948 10943 raft_consensus.cc:1240] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:56.225023 10795 rpcz_store.cc:276] 0709 13:18:54.502683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.034536 (+531853us) service_pool.cc:224] Handling call
0709 13:18:55.034987 (+   451us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:55.035230 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:55.035254 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:55.035698 (+   444us) log.cc:844] Serialized 26 byte log entry
0709 13:18:55.560263 (+524565us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:18:55.560305 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:55.560380 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:56.080935 (+520555us) raft_consensus.cc:1685] finished
0709 13:18:56.080960 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:56.081128 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":396997,"threads_started":1,"wal-append.queue_time_us":590404}
I20260709 13:18:56.335414 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:56.178408 17498 raft_consensus.cc:740] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:56.129946 17603 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:56.246236 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:18:56.167212 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 174552us
I20260709 13:18:56.342289 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:56.277097 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 126232us
I20260709 13:18:56.343283 10766 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:56.343861 10950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 3036) took 1294 ms. Trace:
I20260709 13:18:56.277114 11504 raft_consensus.cc:1240] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 22. Current term is 23. Ops: [22.17-22.17]
I20260709 13:18:56.278666 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:56.161075 17406 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:56.196115 17568 raft_consensus.cc:740] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:56.219825 17407 raft_consensus.cc:740] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:56.299276 10944 raft_consensus.cc:2393] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 24: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:56.110370 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:56.286528 17562 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:56.300817 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:56.356734 10869 raft_consensus.cc:2393] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 22: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:56.160645 17411 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.344494 17483 consensus_queue.cc:260] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:56.360651 17411 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:56.220240 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:18:56.337303 17364 consensus_meta.cc:287] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 1.340s	user 0.014s	sys 0.000s
W20260709 13:18:56.332950 10792 rpcz_store.cc:269] 0709 13:18:54.845005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:54.893516 (+ 48511us) service_pool.cc:224] Handling call
0709 13:18:56.195544 (+1302028us) spinlock_profiling.cc:243] Waited 705 ms on lock 0x7b54002125d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:56.195699 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1620939136}
I20260709 13:18:56.339947 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:56.342687 10942 raft_consensus.cc:1240] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 18. Current term is 19. Ops: []
I20260709 13:18:56.349041 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:56.164518 17386 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:56.345165 10868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49290 (request call id 1045) took 1336 ms. Trace:
I20260709 13:18:56.376960 10868 rpcz_store.cc:276] 0709 13:18:55.008871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.213354 (+1204483us) service_pool.cc:224] Handling call
0709 13:18:56.345096 (+131742us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:56.342669 10916 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:56.381037 10916 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:56.344331 10950 rpcz_store.cc:276] 0709 13:18:55.049044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.421520 (+372476us) service_pool.cc:224] Handling call
0709 13:18:55.436341 (+ 14821us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:55.774287 (+337946us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:56.343581 (+569294us) spinlock_profiling.cc:243] Waited 337 ms on lock 0x7b54000ab4d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:56.343791 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":775639296}
I20260709 13:18:56.381871 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:56.282267 17497 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:56.353967 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:56.331308 17566 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:56.245582 17339 consensus_queue.cc:1059] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.087s
W20260709 13:18:56.360057 17640 log.cc:927] Time spent T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:18:56.333789 10945 rpcz_store.cc:276] 0709 13:18:54.594204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:54.598624 (+  4420us) service_pool.cc:224] Handling call
0709 13:18:54.598986 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:55.336736 (+737750us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:56.203114 (+866378us) spinlock_profiling.cc:243] Waited 733 ms on lock 0x7b540018ecd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:56.203311 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1686780672}
I20260709 13:18:56.275869 17621 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:56.364614 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:56.358765 10766 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:56.383679 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:56.384214 10874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 2587) took 1638 ms. Trace:
I20260709 13:18:56.385004 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:56.361280 17483 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:18:56.362514 17525 consensus_meta.cc:287] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 1.169s	user 0.013s	sys 0.000s
I20260709 13:18:56.441509 10947 raft_consensus.cc:1240] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 19. Current term is 20. Ops: []
I20260709 13:18:56.438091 10942 raft_consensus.cc:1275] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:18:56.409809 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:56.442890 10916 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:56.364094 17282 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:56.364656 17352 raft_consensus.cc:697] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:56.447567 10950 raft_consensus.cc:3055] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:18:56.337947 17641 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:56.347818 17447 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:56.361418 17569 consensus_queue.cc:260] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:56.287124 17622 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:56.571945 10950 raft_consensus.cc:740] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:56.572299 10799 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
I20260709 13:18:56.566228 10875 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 17.
I20260709 13:18:56.291352 17623 consensus_queue.cc:237] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:56.337049 17617 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:56.551681 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:56.502633 10868 raft_consensus.cc:1240] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:18:56.576282 10874 rpcz_store.cc:276] 0709 13:18:54.746022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.348712 (+1602690us) service_pool.cc:224] Handling call
0709 13:18:56.384152 (+ 35440us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:56.372038 17562 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:56.538962 17582 log.cc:927] Time spent T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 1.694s	user 0.006s	sys 0.011s
W20260709 13:18:56.581782 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 223086us
I20260709 13:18:56.447604 17327 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:18:56.530936 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:56.577052 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:56.589078 10868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2573) took 1934 ms. Trace:
I20260709 13:18:56.637279 10868 rpcz_store.cc:276] 0709 13:18:54.654911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.386701 (+1731790us) service_pool.cc:224] Handling call
0709 13:18:56.502360 (+115659us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:56.588672 (+ 86312us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:56.589010 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:56.638952 10871 raft_consensus.cc:2410] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 23 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 13:18:56.447672 17510 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.581023 10795 raft_consensus.cc:2393] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 15: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:18:56.540270 17643 log.cc:927] Time spent T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
W20260709 13:18:56.540957 17630 log.cc:927] Time spent T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.237s	user 0.000s	sys 0.001s
I20260709 13:18:56.411936 17566 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:18:56.539619 17637 log.cc:927] Time spent T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.289s	user 0.000s	sys 0.001s
I20260709 13:18:56.633445 17569 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:56.598688 10874 raft_consensus.cc:1240] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:56.672000 10874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2577) took 2016 ms. Trace:
W20260709 13:18:56.542302 17265 consensus_meta.cc:287] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 1.168s	user 0.011s	sys 0.003s
I20260709 13:18:56.617030 10876 raft_consensus.cc:1240] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 17. Current term is 18. Ops: []
W20260709 13:18:56.620477 10870 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 2569) took 1970 ms (client timeout 2137 ms). Trace:
W20260709 13:18:56.619798 10873 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49290 (request call id 1041) took 1611 ms (client timeout 1763 ms). Trace:
I20260709 13:18:56.624603 10766 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:18:56.582062 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 139312us
I20260709 13:18:56.577811 10950 consensus_queue.cc:260] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:56.678251 10873 rpcz_store.cc:269] 0709 13:18:55.007878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.251440 (+243562us) service_pool.cc:224] Handling call
0709 13:18:56.619551 (+1368111us) spinlock_profiling.cc:243] Waited 9.65 ms on lock 0x7b54001e0354. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:56.619721 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22197632}
W20260709 13:18:56.475852 17450 consensus_meta.cc:287] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.959s	user 0.017s	sys 0.001s
W20260709 13:18:56.639732 10794 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 3062) took 2625 ms (client timeout 1514 ms). Trace:
W20260709 13:18:56.680487 10794 rpcz_store.cc:269] 0709 13:18:53.993941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:53.996017 (+  2076us) service_pool.cc:224] Handling call
0709 13:18:56.618997 (+2622980us) spinlock_profiling.cc:243] Waited 1.94 s on lock 0x7b540024e5d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:56.619165 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4455433472}
I20260709 13:18:56.495220 17364 raft_consensus.cc:3055] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:18:56.487528 17578 raft_consensus.cc:2749] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:18:56.537413 17452 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.343336 17594 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:56.538138 17411 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:56.539615 17416 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:56.543251 17406 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:56.581595 17525 raft_consensus.cc:2749] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:18:56.488770 17498 consensus_queue.cc:260] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:56.486999 17642 raft_consensus.cc:2749] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:56.581581 17446 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.593662 17602 raft_consensus.cc:2749] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:18:56.438274 17625 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.683609 17450 raft_consensus.cc:2749] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:56.685300 17447 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:56.678891 10950 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:56.678329 10876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2581) took 2016 ms. Trace:
I20260709 13:18:56.692359 17617 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:56.681757 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 57283.8us
W20260709 13:18:56.656145 17593 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.007s	sys 0.015s
I20260709 13:18:56.828259 10766 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:18:56.488546 17629 log.cc:927] Time spent T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
W20260709 13:18:56.678092 10870 rpcz_store.cc:269] 0709 13:18:54.650562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:54.650686 (+   124us) service_pool.cc:224] Handling call
0709 13:18:56.620261 (+1969575us) spinlock_profiling.cc:243] Waited 235 ms on lock 0x7b540006d1d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:56.620411 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":541454720}
I20260709 13:18:56.693190 10793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3080) took 1757 ms. Trace:
I20260709 13:18:56.684377 10873 raft_consensus.cc:1240] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:18:56.691109 10872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 2390) took 2114 ms. Trace:
I20260709 13:18:56.679108 10916 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:56.538687 17407 consensus_queue.cc:260] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:56.646771 10869 raft_consensus.cc:1240] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
W20260709 13:18:56.730986 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:56.502717 17568 consensus_queue.cc:260] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:56.688961 17536 consensus_meta.cc:287] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.553s	user 0.003s	sys 0.005s
I20260709 13:18:56.690114 17415 consensus_queue.cc:1048] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.129s
W20260709 13:18:56.683439 17564 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.008s	sys 0.023s
I20260709 13:18:56.691691 17476 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:56.672644 10868 raft_consensus.cc:1240] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:56.806299 10867 raft_consensus.cc:2410] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 21 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 13:18:56.672453 10874 rpcz_store.cc:276] 0709 13:18:54.656009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.598022 (+1942013us) service_pool.cc:224] Handling call
0709 13:18:56.598429 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:56.671704 (+ 73275us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:56.671931 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:56.872740 10874 raft_consensus.cc:1240] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:56.671566 17524 consensus_queue.cc:1059] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.068s
I20260709 13:18:56.718333 17364 raft_consensus.cc:740] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:56.667173 17480 consensus_queue.cc:1059] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.104s
I20260709 13:18:56.726547 17452 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:56.852933 17498 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:18:56.871740 17564 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.747s	user 0.008s	sys 0.029s
I20260709 13:18:56.655496 17510 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:56.831421 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 152390us
I20260709 13:18:56.924103 10916 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:56.835644 10801 raft_consensus.cc:1240] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 24. Current term is 25. Ops: []
I20260709 13:18:56.837270 10794 raft_consensus.cc:1275] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 21 index: 9. (index mismatch)
I20260709 13:18:56.735426 17282 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:56.839469 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2574) took 2184 ms. Trace:
I20260709 13:18:56.931073 10869 rpcz_store.cc:276] 0709 13:18:54.655192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.592042 (+1936850us) service_pool.cc:224] Handling call
0709 13:18:56.646284 (+ 54242us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:56.839042 (+192758us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:56.839387 (+   345us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:56.839695 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 2418) took 2085 ms. Trace:
I20260709 13:18:56.932565 10873 rpcz_store.cc:276] 0709 13:18:54.754409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.679475 (+1925066us) service_pool.cc:224] Handling call
0709 13:18:56.684068 (+  4593us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:56.839444 (+155376us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:56.839641 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:56.849604 10766 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:56.854907 11505 raft_consensus.cc:1240] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:56.656831 17327 raft_consensus.cc:697] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:56.853029 17269 consensus_queue.cc:1059] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.051s
I20260709 13:18:56.832402 10793 rpcz_store.cc:276] 0709 13:18:54.936296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.265216 (+328920us) service_pool.cc:224] Handling call
0709 13:18:55.280332 (+ 15116us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:55.875402 (+595070us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:55.875433 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:55.889134 (+ 13701us) log.cc:844] Serialized 52 byte log entry
0709 13:18:56.160515 (+271381us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:56.160623 (+   108us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:56.161037 (+   414us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:56.692965 (+531928us) raft_consensus.cc:1685] finished
0709 13:18:56.692990 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:56.693135 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":258247,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":10102,"thread_start_us":812966,"threads_started":2,"wal-append.queue_time_us":756124}
I20260709 13:18:56.830554 10872 rpcz_store.cc:276] 0709 13:18:54.576933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:54.577281 (+   348us) service_pool.cc:224] Handling call
0709 13:18:54.577609 (+   328us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:54.577956 (+   347us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:54.577984 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:54.578511 (+   527us) log.cc:844] Serialized 52 byte log entry
0709 13:18:54.723728 (+145217us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:54.723790 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:54.723884 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:56.223632 (+1499748us) spinlock_profiling.cc:243] Waited 2.22 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:18:56.690631 (+466999us) raft_consensus.cc:1685] finished
0709 13:18:56.690669 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:56.691004 (+   335us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19058,"raft.queue_time_us":2006,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":6865,"spinlock_wait_cycles":5104512,"thread_start_us":116305,"threads_started":1,"wal-append.queue_time_us":201596}
I20260709 13:18:56.954468 10872 raft_consensus.cc:1240] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 17. Current term is 18. Ops: [16.13-17.14]
I20260709 13:18:56.866231 10798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 928) took 2382 ms. Trace:
W20260709 13:18:56.836634 17618 log.cc:927] Time spent T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.332s	user 0.001s	sys 0.001s
W20260709 13:18:56.842625 17593 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.007s	sys 0.015s
I20260709 13:18:56.877537 10874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 2425) took 2013 ms. Trace:
I20260709 13:18:56.878394 10868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 2415) took 2125 ms. Trace:
I20260709 13:18:56.836614 17406 raft_consensus.cc:2749] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:56.735596 17458 consensus_queue.cc:1059] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.118s
I20260709 13:18:56.819911 10876 rpcz_store.cc:276] 0709 13:18:54.662422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.616446 (+1954024us) service_pool.cc:224] Handling call
0709 13:18:56.616804 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:56.678052 (+ 61248us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:56.678274 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:56.971901 10876 raft_consensus.cc:1240] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:56.972623 10876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 1052) took 1952 ms. Trace:
I20260709 13:18:56.973089 10876 rpcz_store.cc:276] 0709 13:18:55.020502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.971408 (+1950906us) service_pool.cc:224] Handling call
0709 13:18:56.971730 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:56.972306 (+   576us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:56.972546 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:56.681113 11507 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 16.
I20260709 13:18:56.692647 17352 consensus_queue.cc:237] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:56.975214 17611 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.975724 17611 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:56.811693 17446 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:56.825613 10792 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
I20260709 13:18:56.869563 17536 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:56.871511 17476 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:56.901789 17480 raft_consensus.cc:3055] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:56.991235 10801 raft_consensus.cc:1275] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 13:18:56.902742 17646 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:56.902913 17455 raft_consensus.cc:3055] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:18:56.904271 17650 log.cc:927] Time spent T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:18:56.848042 17622 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:57.003036 17641 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:57.003865 11508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56404 (request call id 1116) took 1230 ms. Trace:
I20260709 13:18:57.004365 11508 rpcz_store.cc:276] 0709 13:18:55.773752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.775023 (+  1271us) service_pool.cc:224] Handling call
0709 13:18:55.775382 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:55.775599 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:55.775621 (+    22us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:55.782479 (+  6858us) log.cc:844] Serialized 26 byte log entry
0709 13:18:56.153646 (+371167us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:18:56.153684 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:56.153767 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:57.003564 (+849797us) raft_consensus.cc:1685] finished
0709 13:18:57.003596 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:57.003795 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":250395,"threads_started":1,"wal-append.queue_time_us":789587}
W20260709 13:18:56.909482 17615 log.cc:927] Time spent T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 1.013s	user 0.003s	sys 0.000s
I20260709 13:18:56.912698 17402 raft_consensus.cc:3055] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:18:57.006251 17402 raft_consensus.cc:740] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:56.859192 17617 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:18:56.995112 17458 consensus_queue.cc:1243] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer cb0428cc55984daba1ba63744a5102a2 log is divergent from this leader: its last log entry 20.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:18:57.006994 17617 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:57.007519 17617 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:18:57.007887 10800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2420) took 2499 ms. Trace:
I20260709 13:18:57.008255 10800 rpcz_store.cc:276] 0709 13:18:54.508440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.138647 (+630207us) service_pool.cc:224] Handling call
0709 13:18:55.139034 (+   387us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:55.153561 (+ 14527us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:55.153606 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:55.166912 (+ 13306us) log.cc:844] Serialized 26 byte log entry
0709 13:18:55.621428 (+454516us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:18:55.621487 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:55.621581 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:57.007668 (+1386087us) raft_consensus.cc:1685] finished
0709 13:18:57.007689 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:57.007831 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14119,"raft.queue_time_us":276906,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":29645,"thread_start_us":370606,"threads_started":1,"wal-append.queue_time_us":1148926}
I20260709 13:18:57.010532 17621 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:57.011214 17639 log.cc:927] Time spent T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.397s	user 0.001s	sys 0.000s
I20260709 13:18:56.926236 10950 raft_consensus.cc:1275] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
W20260709 13:18:56.949679 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 100203us
W20260709 13:18:57.006938 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:56.857133 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 202486us
W20260709 13:18:57.014671 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:56.957206 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:56.958617 10872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 1051) took 1948 ms. Trace:
I20260709 13:18:56.955145 10798 rpcz_store.cc:276] 0709 13:18:54.484072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.031558 (+547486us) service_pool.cc:224] Handling call
0709 13:18:55.034754 (+  3196us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:55.518518 (+483764us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:55.518573 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:55.523745 (+  5172us) log.cc:844] Serialized 26 byte log entry
0709 13:18:55.772217 (+248472us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:55.772273 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:55.772349 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:56.865510 (+1093161us) raft_consensus.cc:1685] finished
0709 13:18:56.865645 (+   135us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:56.866072 (+   427us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2130,"raft.queue_time_us":299210,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":13491,"thread_start_us":687202,"threads_started":2,"wal-append.queue_time_us":650299}
W20260709 13:18:56.959074 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 162572us
I20260709 13:18:56.962584 10874 rpcz_store.cc:276] 0709 13:18:54.863713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.872191 (+2008478us) service_pool.cc:224] Handling call
0709 13:18:56.872537 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:56.877216 (+  4679us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:56.877467 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:56.818446 17625 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:57.021485 17625 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:56.993716 17476 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:56.962986 10868 rpcz_store.cc:276] 0709 13:18:54.753381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.638320 (+1884939us) service_pool.cc:224] Handling call
0709 13:18:56.669449 (+ 31129us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:56.878106 (+208657us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:56.878329 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:57.022317 17646 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:57.023756 10868 raft_consensus.cc:1240] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:18:57.024845 17646 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:56.967597 17578 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.027072 17578 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:57.027786 17510 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:57.028364 17327 consensus_queue.cc:237] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:57.029151 17578 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:56.974717 10876 raft_consensus.cc:1240] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:57.030098 10876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 1053) took 1889 ms. Trace:
W20260709 13:18:56.980392 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 318549us
W20260709 13:18:56.967207 17654 log.cc:927] Time spent T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.458s	user 0.001s	sys 0.000s
W20260709 13:18:56.902360 17627 log.cc:927] Time spent T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.177s	user 0.000s	sys 0.000s
I20260709 13:18:56.993247 10801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3085) took 1009 ms. Trace:
I20260709 13:18:57.041366 10801 rpcz_store.cc:276] 0709 13:18:55.984139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.967490 (+983351us) service_pool.cc:224] Handling call
0709 13:18:56.990889 (+ 23399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:56.992955 (+  2066us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:56.993177 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:56.982424 10869 raft_consensus.cc:1275] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 13:18:57.042065 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.042538 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 1038) took 2036 ms. Trace:
I20260709 13:18:57.042919 10869 rpcz_store.cc:276] 0709 13:18:55.006355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.932012 (+1925657us) service_pool.cc:224] Handling call
0709 13:18:56.982060 (+ 50048us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:57.042297 (+ 60237us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.042484 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:56.960258 17623 raft_consensus.cc:2749] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:18:57.000238 17655 log.cc:927] Time spent T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.471s	user 0.000s	sys 0.001s
I20260709 13:18:57.000310 10796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 930) took 1191 ms. Trace:
I20260709 13:18:56.981225 17603 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:56.963522 17564 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:57.015481 10872 rpcz_store.cc:276] 0709 13:18:55.010258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.953719 (+1943461us) service_pool.cc:224] Handling call
0709 13:18:56.954146 (+   427us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:56.958291 (+  4145us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:56.958551 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:57.019052 17620 log.cc:927] Time spent T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.253s	user 0.000s	sys 0.001s
I20260709 13:18:57.018692 17402 consensus_queue.cc:260] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:57.075604 17402 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:57.040688 17452 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:18:57.021586 17588 consensus_meta.cc:287] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.918s	user 0.001s	sys 0.009s
I20260709 13:18:57.032984 10874 raft_consensus.cc:1275] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:18:57.078904 10874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2590) took 1028 ms. Trace:
I20260709 13:18:57.079284 10874 rpcz_store.cc:276] 0709 13:18:56.050556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.032377 (+981821us) service_pool.cc:224] Handling call
0709 13:18:57.032677 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:57.078597 (+ 45920us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.078848 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:57.080230 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:57.080829 10874 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 19.
I20260709 13:18:57.081693 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:57.033229 10873 raft_consensus.cc:1240] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 18. Current term is 20. Ops: []
I20260709 13:18:57.082340 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 2584) took 2368 ms. Trace:
I20260709 13:18:57.030434 10876 rpcz_store.cc:276] 0709 13:18:55.141035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.974079 (+1833044us) service_pool.cc:224] Handling call
0709 13:18:56.974477 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:57.029829 (+ 55352us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.030042 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:57.083698 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:18:57.084218 17595 consensus_meta.cc:287] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.779s	user 0.010s	sys 0.000s
I20260709 13:18:57.084340 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 1042) took 2076 ms. Trace:
W20260709 13:18:57.085114 17497 consensus_meta.cc:287] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.675s	user 0.008s	sys 0.000s
I20260709 13:18:56.935473 17563 raft_consensus.cc:2749] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:18:57.035902 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:57.086484 10870 raft_consensus.cc:1240] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:18:57.086817 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:57.035390 10949 raft_consensus.cc:1240] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:18:57.087497 10942 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 20.
I20260709 13:18:57.036463 10945 raft_consensus.cc:1275] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:18:56.995394 17480 raft_consensus.cc:740] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:57.088989 17670 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.089517 17670 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:57.089753 17480 consensus_queue.cc:260] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:57.090922 11511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56410 (request call id 1053) took 1383 ms. Trace:
I20260709 13:18:57.091005 17480 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:57.091228 11505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56404 (request call id 1119) took 1130 ms. Trace:
I20260709 13:18:56.997061 17562 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:56.996941 17455 raft_consensus.cc:740] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:57.052168 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:18:57.037760 17651 log.cc:927] Time spent T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.666s	user 0.002s	sys 0.000s
I20260709 13:18:57.050611 10796 rpcz_store.cc:276] 0709 13:18:55.809294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.812325 (+  3031us) service_pool.cc:224] Handling call
0709 13:18:55.887760 (+ 75435us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:56.339080 (+451320us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:56.339126 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:56.348601 (+  9475us) log.cc:844] Serialized 26 byte log entry
0709 13:18:56.431792 (+ 83191us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:18:56.539010 (+107218us) log.cc:844] Serialized 14 byte log entry
0709 13:18:56.580671 (+ 41661us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:56.580752 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:57.000027 (+419275us) raft_consensus.cc:1685] finished
0709 13:18:57.000055 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:57.000244 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41385,"prepare.queue_time_us":107076,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":154096,"raft.run_cpu_time_us":265,"raft.run_wall_time_us":28414,"thread_start_us":522127,"threads_started":2,"wal-append.queue_time_us":601180}
I20260709 13:18:57.057812 10798 raft_consensus.cc:1240] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 16. Current term is 17. Ops: [16.14-16.14]
I20260709 13:18:57.059194 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:57.050909 17587 consensus_queue.cc:1048] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.009s
I20260709 13:18:57.054842 10946 consensus_queue.cc:260] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:57.161537 10946 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:57.040722 17472 consensus_queue.cc:1048] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.020s
I20260709 13:18:57.061466 10801 raft_consensus.cc:1275] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:18:57.047742 17593 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.970921 17364 consensus_queue.cc:260] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:57.062983 17543 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 20 current term: 20 is higher.
I20260709 13:18:57.063071 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:57.198678 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:57.063705 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:57.199756 10869 raft_consensus.cc:1813] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Handling vote request from an unknown peer cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:57.200412 10869 raft_consensus.cc:2393] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 18: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:57.201359 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:57.201936 10869 raft_consensus.cc:2376] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 22. Re-sending same reply.
I20260709 13:18:57.190949 17587 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:57.064373 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:57.203235 10948 raft_consensus.cc:2393] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 20: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
W20260709 13:18:57.069250 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.060055 17541 consensus_queue.cc:1048] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.012s
I20260709 13:18:57.206115 10871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49290 (request call id 1049) took 2196 ms. Trace:
I20260709 13:18:57.206605 10871 rpcz_store.cc:276] 0709 13:18:55.009821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.342033 (+1332212us) service_pool.cc:224] Handling call
0709 13:18:57.205912 (+863879us) spinlock_profiling.cc:243] Waited 295 ms on lock 0x7b54000997d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:57.206050 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":679546112}
I20260709 13:18:57.074483 17564 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:57.084726 17595 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:57.085529 17497 raft_consensus.cc:2749] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:57.092958 10867 rpcz_store.cc:276] 0709 13:18:54.713761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.330841 (+1617080us) service_pool.cc:224] Handling call
0709 13:18:57.082136 (+751295us) spinlock_profiling.cc:243] Waited 422 ms on lock 0x7b54000f73d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:57.082285 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":970967808}
I20260709 13:18:57.086817 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:57.209754 11507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56416 (request call id 1034) took 1065 ms. Trace:
W20260709 13:18:57.115664 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.091678 11505 rpcz_store.cc:276] 0709 13:18:55.960950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.961125 (+   175us) service_pool.cc:224] Handling call
0709 13:18:55.982394 (+ 21269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:56.935333 (+952939us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.091009 (+155676us) spinlock_profiling.cc:243] Waited 872 ms on lock 0x7b54004d3fd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:57.091175 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2006028672}
I20260709 13:18:57.092247 10941 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 17.
I20260709 13:18:57.211930 11512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56410 (request call id 1056) took 1493 ms. Trace:
I20260709 13:18:57.091380 11511 rpcz_store.cc:276] 0709 13:18:55.707291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.707419 (+   128us) service_pool.cc:224] Handling call
0709 13:18:57.090699 (+1383280us) spinlock_profiling.cc:243] Waited 305 ms on lock 0x7b5400691c54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:57.090842 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":702348544}
I20260709 13:18:57.092106 10876 raft_consensus.cc:2376] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 24. Re-sending same reply.
I20260709 13:18:57.152292 10942 raft_consensus.cc:1240] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 17. Current term is 18. Ops: []
W20260709 13:18:57.153231 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.212467 11512 rpcz_store.cc:276] 0709 13:18:55.718905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.719011 (+   106us) service_pool.cc:224] Handling call
0709 13:18:57.211697 (+1492686us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54005f96d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:57.211857 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2870200704}
W20260709 13:18:57.218603 10875 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 2420) took 2414 ms (client timeout 1851 ms). Trace:
W20260709 13:18:57.219166 10875 rpcz_store.cc:269] 0709 13:18:54.804076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:54.896795 (+ 92719us) service_pool.cc:224] Handling call
0709 13:18:57.218381 (+2321586us) spinlock_profiling.cc:243] Waited 1.49 s on lock 0x7b540009b5d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:57.218519 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3427912576}
I20260709 13:18:57.220427 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:57.221280 10875 raft_consensus.cc:2368] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 19. Current term is 20.
I20260709 13:18:57.158738 10949 raft_consensus.cc:1275] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 16 index: 13. (term mismatch)
I20260709 13:18:57.153041 17304 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.181248 17476 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:57.155643 17566 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:57.166942 17472 consensus_queue.cc:1243] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer cb0428cc55984daba1ba63744a5102a2 log is divergent from this leader: its last log entry 21.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:18:57.165621 17455 consensus_queue.cc:260] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:57.091727 17670 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:57.235234 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:57.236086 10950 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 16.
I20260709 13:18:57.171540 17611 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:57.168092 17617 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:57.193606 10946 raft_consensus.cc:1275] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:18:57.239074 10946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 3010) took 2595 ms. Trace:
I20260709 13:18:57.239500 10946 rpcz_store.cc:276] 0709 13:18:54.643733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.058944 (+415211us) service_pool.cc:224] Handling call
0709 13:18:55.116668 (+ 57724us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:57.054503 (+1937835us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b5400310c20. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80e3306f 00007f9c80eabf60 00007f9c80ead7de 00007f9c80eb7b49 00007f9c80eb8416 00007f9c80eb4d8f 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62
0709 13:18:57.238716 (+184213us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.238995 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":362163072}
I20260709 13:18:57.196247 17364 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:57.197150 10947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 3040) took 1090 ms. Trace:
I20260709 13:18:57.237003 17304 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:57.248456 17536 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:57.210316 11507 rpcz_store.cc:276] 0709 13:18:56.144586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.146877 (+  2291us) service_pool.cc:224] Handling call
0709 13:18:57.209537 (+1062660us) spinlock_profiling.cc:243] Waited 346 ms on lock 0x7b54005014d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:57.209679 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":796747008}
I20260709 13:18:57.225698 10949 pending_rounds.cc:85] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 12
I20260709 13:18:57.207026 17587 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:57.209955 10944 raft_consensus.cc:2393] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 17: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:57.247395 10947 rpcz_store.cc:276] 0709 13:18:56.107081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.125040 (+ 17959us) service_pool.cc:224] Handling call
0709 13:18:56.125311 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:56.577146 (+451835us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.196859 (+619713us) spinlock_profiling.cc:243] Waited 261 ms on lock 0x7b540039a154. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:57.197076 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":600848640}
I20260709 13:18:57.254395 17455 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:57.197386 17593 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:57.259037 17588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:46842 (request call id 8) took 8557 ms. Trace:
I20260709 13:18:57.238169 17563 consensus_queue.cc:1059] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 13:18:57.260393 17402 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.265403 17675 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:57.267976 17669 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:57.282914 17675 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:57.330292 17564 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:57.335304 17476 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:57.337509 17566 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:57.338656 17595 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:57.340350 17593 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:57.341387 17617 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:57.342190 17587 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:57.285346 17669 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:57.345153 17669 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:57.347373 17675 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:18:57.348784 17678 log.cc:927] Time spent T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:18:57.272475 17588 rpcz_store.cc:276] 0709 13:18:48.681865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.682030 (+   165us) service_pool.cc:224] Handling call
0709 13:18:48.759545 (+ 77515us) spinlock_profiling.cc:243] Waited 53.3 ms on lock 0x7b4c000d0400. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7f181a20 00007f9c7f192a29 00007f9c7f158644 00007f9c7f17ff2d 00007f9c7f17fb93 00007f9c80e0e1af 00007f9c80e0df3e 00007f9c80e93888 00007f9c80eb075c 00007f9c80eb176a 00007f9c80eb091b
0709 13:18:48.760485 (+   940us) log.cc:844] Serialized 501 byte log entry
0709 13:18:49.553339 (+792854us) spinlock_profiling.cc:243] Waited 4.42 ms on lock 0x7b1800369940. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7dd42c0a 00007f9c87482219 00007f9c7ddcaa40 00007f9c7ddc54e4 00007f9c7ddc5455 00007f9c80e0eb0b 00007f9c80e93b63 00007f9c80ebb752 00007f9c80eb39fa
0709 13:18:57.239072 (+7685733us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":223,"mutex_wait_us":315135,"raft.queue_time_us":458289,"raft.run_cpu_time_us":1917,"raft.run_wall_time_us":227386,"spinlock_wait_cycles":132813952,"thread_start_us":334275,"threads_started":1,"wal-append.queue_time_us":368598,"wal-append.run_cpu_time_us":3277,"wal-append.run_wall_time_us":1198500}
W20260709 13:18:57.384464 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.034063 10766 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:57.165932 10796 raft_consensus.cc:1275] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:18:57.182745 10792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 3087) took 1198 ms. Trace:
W20260709 13:18:57.164831 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.387746 10800 raft_consensus.cc:1240] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 21. Current term is 22. Ops: []
I20260709 13:18:57.387581 10792 rpcz_store.cc:276] 0709 13:18:55.984647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.373868 (+389221us) service_pool.cc:224] Handling call
0709 13:18:57.182470 (+808602us) spinlock_profiling.cc:243] Waited 249 ms on lock 0x7b54000d7354. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:57.182648 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":571599616}
W20260709 13:18:57.103183 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:57.387733 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 353555us
W20260709 13:18:57.392129 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:57.395704 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 238057us
I20260709 13:18:57.397159 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:57.397786 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.084743 10873 rpcz_store.cc:276] 0709 13:18:55.008123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.933442 (+1925319us) service_pool.cc:224] Handling call
0709 13:18:56.962818 (+ 29376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:57.082707 (+119889us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.084110 (+  1403us) spinlock_profiling.cc:243] Waited 70.2 ms on lock 0x7b540006c2d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:57.084284 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":161503104}
W20260709 13:18:57.399493 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 343687us
I20260709 13:18:57.063934 10797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 926) took 2580 ms. Trace:
I20260709 13:18:57.400388 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:57.401631 10873 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 20.
I20260709 13:18:57.402143 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:57.207993 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:57.214110 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:57.405885 17304 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:57.407878 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.408069 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:57.182597 10874 raft_consensus.cc:2410] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 23 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 16.
I20260709 13:18:57.415396 17089 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.417507 17411 raft_consensus.cc:2804] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:57.418131 17411 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:57.418592 17411 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:57.405959 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:57.419329 17498 consensus_queue.cc:1059] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:18:57.420281 10869 raft_consensus.cc:2393] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 16: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:57.421416 17498 raft_consensus.cc:3055] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:18:57.421211 17562 consensus_queue.cc:1059] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:18:57.421904 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:57.422421 17562 consensus_queue.cc:1243] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer cb0428cc55984daba1ba63744a5102a2 log is divergent from this leader: its last log entry 25.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:18:57.163479 10801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3090) took 1071 ms. Trace:
I20260709 13:18:57.423408 17562 raft_consensus.cc:3055] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:18:57.424120 17562 raft_consensus.cc:740] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:57.424384 17438 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.278863 17402 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:57.419600 17536 consensus_queue.cc:1048] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:57.413667 10872 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 22.
W20260709 13:18:57.408609 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.395290 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:57.388419 10796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3091) took 1266 ms. Trace:
I20260709 13:18:57.398537 10871 raft_consensus.cc:2393] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 19: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:57.426419 10792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3092) took 1288 ms. Trace:
I20260709 13:18:57.423516 10801 rpcz_store.cc:276] 0709 13:18:56.092165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.042124 (+949959us) service_pool.cc:224] Handling call
0709 13:18:57.061130 (+ 19006us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:57.163204 (+102074us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.163417 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:57.432585 10792 rpcz_store.cc:276] 0709 13:18:56.122227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.390475 (+1268248us) service_pool.cc:224] Handling call
0709 13:18:57.392082 (+  1607us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:57.393654 (+  1572us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:57.393684 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:57.393780 (+    96us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:57.393813 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.409481 (+ 15668us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:57.410191 (+   710us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:57.393638 10766 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:57.425545 17438 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:57.432387 10796 rpcz_store.cc:276] 0709 13:18:56.121958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.165272 (+1043314us) service_pool.cc:224] Handling call
0709 13:18:57.165616 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:57.387416 (+221800us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.388135 (+   719us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:57.444916 10875 raft_consensus.cc:2393] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 21: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:57.421948 17498 raft_consensus.cc:740] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:57.250655 10949 pending_rounds.cc:107] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 13:18:57.454876 10869 raft_consensus.cc:2376] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 19. Re-sending same reply.
I20260709 13:18:57.465158 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:57.466946 17480 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.467389 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:57.467921 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:57.468066 17480 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:57.469185 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:57.470189 10873 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:57.401217 10797 rpcz_store.cc:276] 0709 13:18:54.483563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.025372 (+541809us) service_pool.cc:224] Handling call
0709 13:18:55.025723 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:55.442155 (+416432us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:55.442188 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:55.442669 (+   481us) log.cc:844] Serialized 26 byte log entry
0709 13:18:55.721780 (+279111us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:55.721816 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:55.721880 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:57.063700 (+1341820us) raft_consensus.cc:1685] finished
0709 13:18:57.063723 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:57.063870 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":271488,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":14164,"thread_start_us":614153,"threads_started":2,"wal-append.queue_time_us":1096397}
I20260709 13:18:57.473933 11511 raft_consensus.cc:1240] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 18 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:18:57.474926 17304 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:57.479300 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:57.480163 17304 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:57.481750 10875 raft_consensus.cc:1275] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:18:57.482084 10798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 933) took 1459 ms. Trace:
I20260709 13:18:57.430572 17089 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:57.479017 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.486012 17089 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:57.487630 10797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3095) took 1342 ms. Trace:
I20260709 13:18:57.467389 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:57.482502 10798 rpcz_store.cc:276] 0709 13:18:56.022556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.015874 (+993318us) service_pool.cc:224] Handling call
0709 13:18:57.018900 (+  3026us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:57.161028 (+142128us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.481438 (+320410us) spinlock_profiling.cc:243] Waited 38.7 ms on lock 0x7b540026b954. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:57.481898 (+   460us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":88905088}
I20260709 13:18:57.489646 10797 rpcz_store.cc:276] 0709 13:18:56.145651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.473950 (+1328299us) service_pool.cc:224] Handling call
0709 13:18:57.474549 (+   599us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:57.474954 (+   405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:57.487123 (+ 12169us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:57.487218 (+    95us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:57.487287 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.487354 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:57.487508 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:57.491258 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.494450 10867 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
I20260709 13:18:57.495992 10796 raft_consensus.cc:1240] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 LEADER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:18:57.496681 10867 raft_consensus.cc:1217] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 LEADER]: Deduplicated request from leader. Original: 14.10->[16.11-16.11]   Dedup: 16.11->[]
I20260709 13:18:57.497013 10796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 935) took 1361 ms. Trace:
I20260709 13:18:57.497327 10867 raft_consensus.cc:1240] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:18:57.497448 10796 rpcz_store.cc:276] 0709 13:18:56.135220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.459982 (+1324762us) service_pool.cc:224] Handling call
0709 13:18:57.495497 (+ 35515us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:57.496638 (+  1141us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.496918 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:57.496050 10949 raft_consensus.cc:2889] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:57.498404 17498 consensus_queue.cc:260] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:57.500828 17498 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:18:57.501566 17438 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:57.501770 10869 raft_consensus.cc:1240] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:57.502413 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 1057) took 1418 ms. Trace:
I20260709 13:18:57.502746 10869 rpcz_store.cc:276] 0709 13:18:56.083858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.488346 (+1404488us) service_pool.cc:224] Handling call
0709 13:18:57.501386 (+ 13040us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:57.502169 (+   783us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.502361 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:57.503633 11507 raft_consensus.cc:1275] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
W20260709 13:18:57.504021 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 110816us
I20260709 13:18:57.504585 10869 raft_consensus.cc:1275] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 13:18:57.509011 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.510277 17673 consensus_queue.cc:1059] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:18:57.447069 17476 consensus_queue.cc:1059] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.087s
I20260709 13:18:57.511819 10694 catalog_manager.cc:5697] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 12 to 16, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 16 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:57.516335 17646 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.517324 17646 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:57.519893 10916 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:57.520284 17646 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:57.521876 17566 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:18:57.523274 17566 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:57.528337 11505 raft_consensus.cc:1275] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:18:57.528680 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:57.529762 11507 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:18:57.531441 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:57.532203 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:57.532419 11508 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 22.
I20260709 13:18:57.533499 17566 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:57.534406 17589 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.535411 17589 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:57.538380 17589 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:57.539613 17562 consensus_queue.cc:260] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 24.12, Last appended by leader: 12, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:57.493767 10875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 1056) took 1407 ms. Trace:
I20260709 13:18:57.550216 10875 rpcz_store.cc:276] 0709 13:18:56.083564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.480739 (+1397175us) service_pool.cc:224] Handling call
0709 13:18:57.481188 (+   449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:57.487055 (+  5867us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.491059 (+  4004us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:57.554308 17647 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.554929 11503 raft_consensus.cc:1275] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:18:57.555274 17647 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:57.556311 17386 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:57.558202 17647 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:57.558650 17480 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:57.558985 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.559340 17564 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.560143 17564 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:57.560596 10916 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:57.565207 10796 raft_consensus.cc:1275] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
W20260709 13:18:57.565348 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 60138.8us
I20260709 13:18:57.571295 10796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3096) took 1220 ms. Trace:
I20260709 13:18:57.574360 10796 rpcz_store.cc:276] 0709 13:18:56.346025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.498498 (+1152473us) service_pool.cc:224] Handling call
0709 13:18:57.514173 (+ 15675us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:57.565762 (+ 51589us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.565978 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:57.565243 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 56816.7us
I20260709 13:18:57.550235 17304 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:57.563079 17589 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:57.523447 17621 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:18:57.590272 17564 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:57.622170 10872 raft_consensus.cc:1275] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:18:57.622574 17619 raft_consensus.cc:3055] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:18:57.623114 17619 raft_consensus.cc:740] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:57.624670 10868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2593) took 1573 ms. Trace:
I20260709 13:18:57.624214 10797 raft_consensus.cc:2955] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Committing config change with OpId 17.13: config changed from index 8 to 13, VOTER cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:18:57.625721 10868 rpcz_store.cc:276] 0709 13:18:56.051434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.024675 (+973241us) service_pool.cc:224] Handling call
0709 13:18:57.024975 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:57.025169 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:18:57.215806 (+190637us) log.cc:844] Serialized 14 byte log entry
0709 13:18:57.623848 (+408042us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:57.623964 (+   116us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:57.624087 (+   123us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.624208 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:57.624422 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13475,"prepare.queue_time_us":252229,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"thread_start_us":553047,"threads_started":2}
I20260709 13:18:57.634020 17411 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:57.638387 17636 log.cc:927] Time spent T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.607s	user 0.008s	sys 0.000s
I20260709 13:18:57.659993 10767 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:57.639052 17411 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:57.660980 17594 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:57.572240 17670 consensus_queue.cc:1048] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 13:18:57.661530 17411 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:57.662129 10767 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:57.663110 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:57.663982 10796 raft_consensus.cc:2393] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 17: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:57.665192 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:57.665971 17402 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:57.668571 10943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 3037) took 1562 ms. Trace:
I20260709 13:18:57.669064 10943 rpcz_store.cc:276] 0709 13:18:56.106270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.122912 (+ 16642us) service_pool.cc:224] Handling call
0709 13:18:56.152431 (+ 29519us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:56.334871 (+182440us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.667977 (+1333106us) spinlock_profiling.cc:243] Waited 93.3 ms on lock 0x7b54003589d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:57.668327 (+   350us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":214512512}
I20260709 13:18:57.639212 10798 raft_consensus.cc:1275] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:18:57.671391 10798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 936) took 1307 ms. Trace:
I20260709 13:18:57.671635 17594 raft_consensus.cc:2749] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:18:57.672201 10801 raft_consensus.cc:3055] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:18:57.672644 10801 raft_consensus.cc:740] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:57.672719 10767 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:57.673759 10801 consensus_queue.cc:260] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:57.674824 17386 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:18:57.675130 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 55867.7us
I20260709 13:18:57.671792 10798 rpcz_store.cc:276] 0709 13:18:56.364409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.567820 (+1203411us) service_pool.cc:224] Handling call
0709 13:18:57.603947 (+ 36127us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:57.671093 (+ 67146us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.671322 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:57.677192 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:57.677343 17381 consensus_queue.cc:1048] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:18:57.675478 17594 consensus_queue.cc:1059] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 13:18:57.678089 17680 consensus_queue.cc:1059] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:18:57.679189 17210 raft_consensus.cc:3055] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:57.679311 17680 raft_consensus.cc:3055] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:18:57.679668 17210 raft_consensus.cc:740] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:57.679836 17680 raft_consensus.cc:740] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:57.680034 10916 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:57.640172 17682 log.cc:927] Time spent T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.366s	user 0.001s	sys 0.000s
I20260709 13:18:57.644102 11477 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:57.649587 17562 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:18:57.680915 17210 consensus_queue.cc:260] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:57.652884 17446 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:57.653765 17415 consensus_queue.cc:1059] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:18:57.703120 17415 raft_consensus.cc:3055] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:18:57.703514 17415 raft_consensus.cc:740] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:57.704562 17415 consensus_queue.cc:260] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:57.706079 17415 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:57.654883 17564 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:57.707239 17564 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:57.709879 17564 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:18:57.659042 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 52765.1us
I20260709 13:18:57.683511 11507 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 22.
W20260709 13:18:57.650082 17681 log.cc:927] Time spent T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.298s	user 0.000s	sys 0.001s
I20260709 13:18:57.666579 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:57.688943 10873 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 20.
I20260709 13:18:57.696722 17451 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.701573 17447 raft_consensus.cc:3055] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:57.658831 10870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 2419) took 2904 ms. Trace:
I20260709 13:18:57.688393 17673 raft_consensus.cc:2749] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:18:57.727708 10870 rpcz_store.cc:276] 0709 13:18:54.754720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.852024 (+2097304us) service_pool.cc:224] Handling call
0709 13:18:57.086286 (+234262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:57.086917 (+   631us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.658460 (+571543us) spinlock_profiling.cc:243] Waited 234 ms on lock 0x7b54000f73d0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:57.658652 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":537592064}
W20260709 13:18:57.730118 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 53111.9us
I20260709 13:18:57.740141 10916 leader_election.cc:400] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 19. Current term is 20.
I20260709 13:18:57.739825 17670 consensus_queue.cc:1059] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:18:57.740998 10796 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 20.
I20260709 13:18:57.742234 17411 raft_consensus.cc:2749] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:18:57.742835 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.743309 17451 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:57.681188 17680 consensus_queue.cc:260] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:57.747696 17680 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:18:57.749267 17282 consensus_meta.cc:287] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.814s	user 0.006s	sys 0.010s
I20260709 13:18:57.749402 17552 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:57.749887 17282 raft_consensus.cc:2749] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:57.750147 17552 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:57.751015 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 107293us
I20260709 13:18:57.751210 17402 raft_consensus.cc:2804] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:57.751706 17402 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:57.752085 17402 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:57.742308 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:57.755264 10685 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 171abdebfc4142cda86de477a5c89a7e with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 9)
I20260709 13:18:57.756402 10943 pending_rounds.cc:85] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 13
I20260709 13:18:57.756765 10943 pending_rounds.cc:107] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 21.14
I20260709 13:18:57.757040 10943 raft_consensus.cc:2889] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:57.757367 17455 consensus_queue.cc:1048] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:18:57.759353 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:57.760154 10795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 931) took 1946 ms. Trace:
I20260709 13:18:57.760211 10946 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 20.
I20260709 13:18:57.760582 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:57.730197 17447 raft_consensus.cc:740] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:57.761348 10876 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:57.761919 17447 consensus_queue.cc:260] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:57.763013 11478 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:57.763249 17447 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:57.763979 17407 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:18:57.764420 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 171064us
I20260709 13:18:57.767629 10868 raft_consensus.cc:1240] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:18:57.768033 10945 raft_consensus.cc:1240] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 19. Current term is 20. Ops: []
I20260709 13:18:57.769062 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:57.769578 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:57.770532 10950 raft_consensus.cc:2393] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 22: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:57.771152 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:57.771517 17210 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:57.771696 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:57.772360 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:57.773044 10872 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 18.
I20260709 13:18:57.773998 10874 raft_consensus.cc:1275] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 17. (term mismatch)
I20260709 13:18:57.774364 10872 raft_consensus.cc:1275] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 13:18:57.772501 10944 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 16.
I20260709 13:18:57.774993 10869 raft_consensus.cc:1240] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:18:57.774384 10874 pending_rounds.cc:85] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 16
I20260709 13:18:57.775977 10874 pending_rounds.cc:107] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 22.17
I20260709 13:18:57.776225 10874 raft_consensus.cc:2889] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:57.770440 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:57.624666 17619 consensus_queue.cc:260] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 21.9, Last appended by leader: 9, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:57.779294 10871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 1054) took 1696 ms. Trace:
I20260709 13:18:57.779779 17619 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:18:57.780856 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.779742 10871 rpcz_store.cc:276] 0709 13:18:56.082936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.469091 (+1386155us) service_pool.cc:224] Handling call
0709 13:18:57.469676 (+   585us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:57.470293 (+   617us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:18:57.507355 (+ 37062us) log.cc:844] Serialized 14 byte log entry
0709 13:18:57.723806 (+216451us) log.cc:844] Serialized 14 byte log entry
0709 13:18:57.778964 (+ 55158us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:57.779015 (+    51us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:18:57.779075 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.779136 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:57.779237 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54914,"prepare.queue_time_us":40603,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":100,"raft.queue_time_us":51960,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":22044,"thread_start_us":170751,"threads_started":1}
W20260709 13:18:57.781358 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:57.782800 17686 log.cc:927] Time spent T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.456s	user 0.002s	sys 0.002s
I20260709 13:18:57.784579 10767 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:57.785105 17402 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:57.727339 17594 consensus_queue.cc:1059] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.063s
W20260709 13:18:57.782007 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:57.793483 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.787196 10876 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 20.
I20260709 13:18:57.791608 10766 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:57.766577 11478 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:57.788901 17402 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:57.812819 10797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 937) took 1147 ms. Trace:
I20260709 13:18:57.813262 10797 rpcz_store.cc:276] 0709 13:18:56.665854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.543290 (+877436us) service_pool.cc:224] Handling call
0709 13:18:57.589488 (+ 46198us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:57.623101 (+ 33613us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:18:57.623614 (+   513us) log.cc:844] Serialized 14 byte log entry
0709 13:18:57.801773 (+178159us) log.cc:844] Serialized 14 byte log entry
0709 13:18:57.804258 (+  2485us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:57.804344 (+    86us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:18:57.812458 (+  8114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.812570 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:57.812748 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2080,"prepare.queue_time_us":8218,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":58,"raft.queue_time_us":12774,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":119}
I20260709 13:18:57.817619 10766 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:57.819149 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:57.820657 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:57.822026 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:57.826948 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:57.827351 10800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3088) took 1842 ms. Trace:
I20260709 13:18:57.823675 17603 raft_consensus.cc:2804] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:57.828083 17603 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:57.827790 10800 rpcz_store.cc:276] 0709 13:18:55.984914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.009711 (+1024797us) service_pool.cc:224] Handling call
0709 13:18:57.010046 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:57.389045 (+378999us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.827059 (+438014us) spinlock_profiling.cc:243] Waited 377 ms on lock 0x7b54001f8bd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:57.827274 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":866781056}
I20260709 13:18:57.828457 17536 raft_consensus.cc:3055] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:57.828653 17603 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:57.829020 17536 raft_consensus.cc:740] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:57.829291 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:57.830049 10800 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 17.
I20260709 13:18:57.832127 17622 consensus_queue.cc:1059] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 13:18:57.822757 10796 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:57.833302 17622 raft_consensus.cc:3055] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:18:57.833787 17622 raft_consensus.cc:740] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:57.824085 17327 raft_consensus.cc:3055] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:18:57.830158 17536 consensus_queue.cc:260] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:57.835332 17327 raft_consensus.cc:740] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:57.835480 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:57.825362 17670 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.836328 17670 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:57.836436 17327 consensus_queue.cc:260] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:57.837850 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.786772 17641 consensus_queue.cc:1059] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 13:18:57.790513 17594 consensus_queue.cc:1243] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Peer 468a8fd46e824f62b9c2c0d3532a6d26 log is divergent from this leader: its last log entry 18.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:18:57.839435 17641 consensus_queue.cc:1243] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer 468a8fd46e824f62b9c2c0d3532a6d26 log is divergent from this leader: its last log entry 18.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:18:57.811151 17411 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:18:57.786904 17588 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:18:57.823309 17617 consensus_queue.cc:1048] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.004s
I20260709 13:18:57.827924 10873 raft_consensus.cc:2376] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 17. Re-sending same reply.
I20260709 13:18:57.826809 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:57.845841 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:57.838527 17670 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:57.745587 17566 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:57.840045 17623 consensus_queue.cc:1059] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
W20260709 13:18:57.849431 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.795102 10795 rpcz_store.cc:276] 0709 13:18:55.813515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.657086 (+843571us) service_pool.cc:224] Handling call
0709 13:18:57.183815 (+526729us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:57.184098 (+   283us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:18:57.184448 (+   350us) log.cc:844] Serialized 14 byte log entry
0709 13:18:57.606577 (+422129us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:57.682093 (+ 75516us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:18:57.726978 (+ 44885us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.735336 (+  8358us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:57.759775 (+ 24439us) spinlock_profiling.cc:243] Waited 526 ms on lock 0x7b54000d7350. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:57.759941 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":6175,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":3671,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":20502,"spinlock_wait_cycles":1210366848,"thread_start_us":382226,"threads_started":1,"wal-append.queue_time_us":474979}
I20260709 13:18:57.854244 17402 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.854851 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:57.855412 10950 raft_consensus.cc:1275] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:18:57.854940 17402 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:57.856606 10945 raft_consensus.cc:1275] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:18:57.809441 17647 consensus_queue.cc:1048] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.048s
I20260709 13:18:57.857046 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:57.857543 17402 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:57.857753 10946 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
I20260709 13:18:57.809890 17646 raft_consensus.cc:2749] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:18:57.859225 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:57.855760 10944 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 22.
I20260709 13:18:57.861581 10916 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:57.861725 10800 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:57.862838 10916 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:57.863919 17552 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:57.864490 17552 raft_consensus.cc:2804] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:18:57.864826 17552 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:57.864879 17673 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:18:57.865259 17552 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:18:57.865336 17673 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:57.865793 17673 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:57.843127 17588 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:57.866403 17455 raft_consensus.cc:2749] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:18:57.840121 17411 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:57.869190 10797 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 22.
I20260709 13:18:57.874632 10801 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:57.866998 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:57.866581 17588 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:18:57.875780 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.878116 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:57.826735 17568 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:18:57.881942 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:57.884446 10800 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 20.
I20260709 13:18:57.891054 10948 raft_consensus.cc:1240] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:18:57.891017 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:57.891665 17670 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.892417 10842 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:57.892357 17670 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:57.893608 17525 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.894121 17525 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:57.894775 17386 raft_consensus.cc:2749] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:18:57.895054 17475 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.896281 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:18:57.898168 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.874150 17411 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:18:57.906453 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.890154 17622 consensus_queue.cc:260] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:57.897720 17475 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:57.911157 10946 raft_consensus.cc:1240] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:57.913216 10945 raft_consensus.cc:1275] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 13:18:57.894723 17670 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:57.921752 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 155329us
I20260709 13:18:57.917608 17622 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:18:57.934309 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.939872 10792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3093) took 1817 ms. Trace:
W20260709 13:18:57.941730 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 72391.7us
I20260709 13:18:57.923892 17364 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.942899 10795 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
I20260709 13:18:57.927085 17536 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:18:57.952708 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.953692 10796 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 18.
I20260709 13:18:57.941309 10792 rpcz_store.cc:276] 0709 13:18:56.122483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.438036 (+1315553us) service_pool.cc:224] Handling call
0709 13:18:57.438743 (+   707us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:57.515828 (+ 77085us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:57.515864 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:57.516632 (+   768us) log.cc:844] Serialized 26 byte log entry
0709 13:18:57.794768 (+278136us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:57.794833 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.794902 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:57.939642 (+144740us) raft_consensus.cc:1685] finished
0709 13:18:57.939665 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:57.939812 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":76192,"raft.queue_time_us":76634,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":181,"thread_start_us":209822,"threads_started":1,"wal-append.queue_time_us":340986}
I20260709 13:18:57.900537 10800 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3"
dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
mode: GRACEFUL
new_leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
 from {username='slave'} at 127.0.0.1:40148
I20260709 13:18:57.961551 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:57.933913 17475 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:57.965083 10797 raft_consensus.cc:1240] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:57.971041 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:57.971841 10799 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 3081) took 1937 ms (client timeout 1567 ms). Trace:
I20260709 13:18:57.957506 17402 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:57.972349 10799 rpcz_store.cc:269] 0709 13:18:55.971288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.301719 (+330431us) service_pool.cc:224] Handling call
0709 13:18:57.905590 (+1603871us) spinlock_profiling.cc:243] Waited 80.3 ms on lock 0x7b5400386154. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:18:57.908267 (+  2677us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":184613760}
I20260709 13:18:57.972875 17402 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:57.973858 10799 raft_consensus.cc:1240] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:18:57.977208 10916 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:57.958335 17564 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:18:57.981667 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.982038 17455 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.982638 17450 consensus_queue.cc:1059] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:18:58.040061 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:57.992039 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.993399 10694 catalog_manager.cc:5697] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:57.969244 17364 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:57.983271 17589 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:58.007767 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:57.987777 17607 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:58.009852 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.967759 17525 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:57.992468 10944 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:58.070084 10798 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 22.
I20260709 13:18:58.040544 10842 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:58.045145 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:58.047341 10801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3094) took 1902 ms. Trace:
I20260709 13:18:58.051084 17646 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.081305 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:58.051437 17450 raft_consensus.cc:3055] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:18:58.051143 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 80373.2us
I20260709 13:18:58.051314 17564 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:58.088634 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:58.052019 17498 consensus_meta.cc:287] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.551s	user 0.008s	sys 0.008s
I20260709 13:18:58.054325 10800 raft_consensus.cc:606] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Received request to transfer leadership to TS 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:58.054598 10792 raft_consensus.cc:1240] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:18:58.056054 17566 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:58.057197 17603 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:58.091033 10800 raft_consensus.cc:614] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:58.051084 17438 raft_consensus.cc:2749] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:18:58.071771 10867 raft_consensus.cc:1275] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
W20260709 13:18:58.074133 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.075785 17587 consensus_queue.cc:1048] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 13:18:58.090713 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:58.080188 10694 catalog_manager.cc:5697] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 17 to 18, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 18 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:18:58.081226 10699 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42968 (request call id 248) took 27939 ms (client timeout 15000 ms). Trace:
I20260709 13:18:58.072075 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:58.082842 10801 rpcz_store.cc:276] 0709 13:18:56.145385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.460009 (+1314624us) service_pool.cc:224] Handling call
0709 13:18:57.508924 (+ 48915us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:57.672106 (+163182us) spinlock_profiling.cc:243] Waited 6.55 ms on lock 0x7b54003050a0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80e2cdc0 00007f9c80eb71c0 00007f9c80eb8231 00007f9c80eb4d8f 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c
0709 13:18:57.789292 (+117186us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b58004ff4d0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80e110aa 00007f9c80e93447 00007f9c80eabfe7 00007f9c80ead7de 00007f9c80eb7b49 00007f9c80eb8416 00007f9c80eb4d8f 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4
0709 13:18:57.864897 (+ 75605us) spinlock_profiling.cc:243] Waited 51.4 ms on lock 0x7b54003050a0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80e2aa1d 00007f9c80e111a4 00007f9c80e93447 00007f9c80eabfe7 00007f9c80ead7de 00007f9c80eb7b49 00007f9c80eb8416 00007f9c80eb4d8f 00007f9c80eb483e 00007f9c85e79419
0709 13:18:58.042094 (+177197us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:58.042134 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:58.042195 (+    61us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:58.047032 (+  4837us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:58.047156 (+   124us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:58.047289 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":89,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":16996,"spinlock_wait_cycles":164428416}
I20260709 13:18:58.084517 10793 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 19.
I20260709 13:18:58.086460 10950 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 17.
I20260709 13:18:57.981379 10796 raft_consensus.cc:1217] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Deduplicated request from leader. Original: 14.10->[16.11-16.11]   Dedup: 16.11->[]
I20260709 13:18:58.064783 17621 raft_consensus.cc:2749] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:18:58.092518 10916 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:58.092878 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:58.093954 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:58.094076 17646 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:58.095014 17539 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:18:58.099201 10799 raft_consensus.cc:1275] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
W20260709 13:18:58.099452 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 141294us
I20260709 13:18:58.100240 17525 raft_consensus.cc:2749] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:18:58.100976 17603 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:58.073604 17476 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
I20260709 13:18:58.047338 17455 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:58.112214 10949 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 16.
I20260709 13:18:58.089341 17450 raft_consensus.cc:740] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:58.115846 11478 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:58.115630 17611 consensus_queue.cc:1059] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:18:58.120260 17564 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:58.121065 10792 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:18:58.123374 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.128842 17607 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:58.127828 17594 raft_consensus.cc:3055] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:18:58.137229 10796 raft_consensus.cc:1240] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
W20260709 13:18:58.139235 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.133693 11477 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:58.151216 10948 raft_consensus.cc:1275] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:18:58.153205 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:58.175560 10793 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
I20260709 13:18:58.153889 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:58.167799 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:58.161557 10941 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 19.
I20260709 13:18:58.168913 10942 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:58.169039 17617 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.169602 17622 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:58.169859 17589 raft_consensus.cc:2749] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:18:58.170384 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:58.171283 17611 raft_consensus.cc:3055] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:18:58.171350 17675 raft_consensus.cc:2749] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:18:58.147460 17566 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.163650 10944 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 21.
I20260709 13:18:58.191488 10949 raft_consensus.cc:1240] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
W20260709 13:18:58.195182 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 105063us
I20260709 13:18:58.134030 17588 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:58.190718 10801 raft_consensus.cc:2376] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 16. Re-sending same reply.
I20260709 13:18:58.194700 10950 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 17.
I20260709 13:18:58.122607 17552 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:58.173496 17455 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:58.204165 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:58.204948 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 89279.9us
I20260709 13:18:58.176692 17594 raft_consensus.cc:740] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:58.208599 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:58.208149 17525 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.201669 17210 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:58.216607 17525 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:58.219527 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 127162us
I20260709 13:18:58.198184 17611 raft_consensus.cc:740] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:58.217743 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 2452) took 1171 ms. Trace:
I20260709 13:18:58.224877 10842 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:58.223031 10867 rpcz_store.cc:276] 0709 13:18:57.046645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.498436 (+451791us) service_pool.cc:224] Handling call
0709 13:18:57.508070 (+  9634us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:58.107491 (+599421us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:58.215594 (+108103us) spinlock_profiling.cc:243] Waited 554 ms on lock 0x7b540006d6d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:58.217669 (+  2075us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1274486016}
I20260709 13:18:58.228327 17619 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.230549 17438 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:58.231197 17438 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:58.231098 17619 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:58.194607 17562 consensus_queue.cc:1048] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:18:58.218844 17210 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:58.240760 10801 raft_consensus.cc:1275] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 13:18:58.243238 10795 raft_consensus.cc:1275] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:18:58.228595 17670 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.214911 17622 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:58.169636 17450 consensus_queue.cc:260] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:58.319077 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.323438 10875 raft_consensus.cc:1275] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 15. (term mismatch)
I20260709 13:18:58.225939 17675 consensus_queue.cc:1059] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 13:18:58.229715 17566 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:58.323846 10875 pending_rounds.cc:85] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 14
I20260709 13:18:58.336102 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:58.337519 10875 pending_rounds.cc:107] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 23.15
I20260709 13:18:58.337869 10875 raft_consensus.cc:2889] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:58.336835 11512 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 22.
I20260709 13:18:58.320246 17364 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:58.234061 17617 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:58.236400 17623 consensus_queue.cc:1048] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.098s
W20260709 13:18:58.344090 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 142738us
I20260709 13:18:58.339023 17579 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.335584 17587 raft_consensus.cc:3055] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:18:58.354343 11511 raft_consensus.cc:1240] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 18. Current term is 19. Ops: []
I20260709 13:18:58.359543 11507 raft_consensus.cc:1240] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 21. Current term is 22. Ops: []
W20260709 13:18:58.363288 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.364260 10766 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:58.342281 17525 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:58.367337 17571 raft_consensus.cc:2804] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:58.350996 17536 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.365048 17688 raft_consensus.cc:2804] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:18:58.383037 17688 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:58.383512 17688 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:58.386262 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:58.386814 17571 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:58.387048 10949 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 24.
I20260709 13:18:58.387362 17571 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:58.388005 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:58.388171 17327 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:58.388648 10949 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
I20260709 13:18:58.389487 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:58.390097 17588 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:58.391149 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:58.392091 10950 raft_consensus.cc:2368] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 19. Current term is 20.
I20260709 13:18:58.393399 17364 consensus_queue.cc:1048] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:58.394122 17475 consensus_queue.cc:1048] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:18:58.395015 17364 consensus_queue.cc:1059] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:18:58.396173 17364 raft_consensus.cc:3055] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:18:58.396370 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:58.396576 17364 raft_consensus.cc:740] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:58.397581 17364 consensus_queue.cc:260] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:58.399547 10945 raft_consensus.cc:1240] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 21. Current term is 22. Ops: [21.9-21.9]
I20260709 13:18:58.400390 17675 consensus_queue.cc:1048] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:18:58.400714 17455 consensus_queue.cc:1048] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:18:58.402355 17438 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:58.403614 17607 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:58.405561 17210 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:58.406963 10799 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 24.
I20260709 13:18:58.407159 17566 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:18:58.407917 17700 log.cc:927] Time spent T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.229s	user 0.000s	sys 0.001s
I20260709 13:18:58.408573 17536 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:58.409185 17402 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:58.358261 17594 consensus_queue.cc:260] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:58.412158 17594 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:58.369361 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:58.340948 17411 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:58.372030 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.377002 17269 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.319793 17611 consensus_queue.cc:260] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:58.381852 17701 log.cc:927] Time spent T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
I20260709 13:18:58.356546 17587 raft_consensus.cc:740] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:58.415899 17622 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:58.341423 17670 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:58.418524 17411 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:58.351251 17451 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:58.433008 10841 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:58.433364 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:58.433652 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:58.434237 10870 raft_consensus.cc:2393] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 18: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:58.434334 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:58.435173 10873 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 22.
I20260709 13:18:58.432682 17269 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:58.435712 17364 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:18:58.434343 10867 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 16.
I20260709 13:18:58.433030 17611 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:58.419284 10944 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:58.357173 17680 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:58.134933 10699 rpcz_store.cc:269] 0709 13:18:30.168202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:30.180552 (+ 12350us) service_pool.cc:224] Handling call
0709 13:18:58.080991 (+27900439us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:49.249918 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:49.250578 (+   660us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:49.250625 (+    47us) write_op.cc:435] Acquired schema lock
0709 13:18:49.250645 (+    20us) tablet.cc:662] Decoding operations
0709 13:18:49.252246 (+  1601us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:49.252358 (+   112us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:49.252384 (+    26us) tablet.cc:685] Acquiring locks for 18 operations
0709 13:18:49.253824 (+  1440us) tablet.cc:701] Row locks acquired
0709 13:18:49.253874 (+    50us) write_op.cc:260] PREPARE: finished
0709 13:18:49.254039 (+   165us) write_op.cc:270] Start()
0709 13:18:49.254194 (+   155us) write_op.cc:276] Timestamp: P: 1783603129253979 usec, L: 0
0709 13:18:49.254243 (+    49us) op_driver.cc:348] REPLICATION: starting
0709 13:18:49.255083 (+   840us) log.cc:844] Serialized 7166 byte log entry
0709 13:18:49.469046 (+213963us) op_driver.cc:464] REPLICATION: finished
0709 13:18:49.790634 (+321588us) write_op.cc:301] APPLY: starting
0709 13:18:49.790743 (+   109us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:50.026104 (+235361us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:50.026224 (+   120us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:50.147392 (+121168us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:50.147826 (+   434us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=36,key_file_lookups=36,delta_file_lookups=36,mrs_lookups=0
0709 13:18:50.147903 (+    77us) write_op.cc:312] APPLY: finished
0709 13:18:50.148396 (+   493us) log.cc:844] Serialized 164 byte log entry
0709 13:18:50.148590 (+   194us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:50.148770 (+   180us) write_op.cc:454] Released schema lock
0709 13:18:50.148840 (+    70us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:50.149308 (+   468us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:50.149339 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":738973,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":289062,"apply.run_cpu_time_us":26055,"apply.run_wall_time_us":358676,"cfile_cache_hit":74,"cfile_cache_hit_bytes":79403,"num_ops":18,"prepare.queue_time_us":213698,"prepare.run_cpu_time_us":25877,"prepare.run_wall_time_us":30682,"raft.queue_time_us":108082,"raft.run_cpu_time_us":14273,"raft.run_wall_time_us":181839,"replication_time_us":189416,"spinlock_wait_cycles":228864,"thread_start_us":316189,"threads_started":4,"wal-append.queue_time_us":25258,"wal-append.run_cpu_time_us":34231,"wal-append.run_wall_time_us":1873140}]]}
I20260709 13:18:58.422538 17450 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:58.358304 17673 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:58.356462 17579 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:58.460304 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:58.460796 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.445125 17680 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:58.463567 10876 raft_consensus.cc:1240] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:18:58.465157 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:58.465844 10876 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:18:58.466912 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.466457 17680 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:58.473623 10876 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 22.
W20260709 13:18:58.480535 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 47702.2us
I20260709 13:18:58.482554 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:58.484048 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:58.479910 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.437580 17552 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:58.437204 17269 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:58.490713 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:18:58.500229 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 62676.1us
I20260709 13:18:58.495839 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:58.456913 17562 raft_consensus.cc:2749] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:58.502545 10942 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 17.
W20260709 13:18:58.507047 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.504221 10867 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 19.
I20260709 13:18:58.509716 11507 raft_consensus.cc:2393] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 15: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
W20260709 13:18:58.516117 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 123036us
I20260709 13:18:58.516664 10870 raft_consensus.cc:2393] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 20: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:58.517355 10947 raft_consensus.cc:1275] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:18:58.452898 17407 consensus_queue.cc:1048] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.054s
I20260709 13:18:58.519351 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:58.492398 17566 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.525596 10942 raft_consensus.cc:2393] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 17: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:58.505395 17619 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:58.509924 17588 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.532526 10944 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 18.
I20260709 13:18:58.529438 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:58.535048 11511 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:58.536255 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:58.539203 10842 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:58.540822 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:58.541563 10944 raft_consensus.cc:2393] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 21: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
W20260709 13:18:58.546229 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 61690.2us
I20260709 13:18:58.547935 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:58.548792 11507 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
I20260709 13:18:58.531126 17587 consensus_queue.cc:260] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:58.555799 17587 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:58.431195 10694 catalog_manager.cc:5697] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:58.518988 17552 consensus_queue.cc:1048] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.039s
I20260709 13:18:58.536878 17566 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:58.588151 10792 raft_consensus.cc:1275] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 13:18:58.591816 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.540190 17588 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:58.571300 17619 raft_consensus.cc:2764] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader election decision vote started in defunct term 19: won
W20260709 13:18:58.596060 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 62723.4us
I20260709 13:18:58.571333 17455 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
W20260709 13:18:58.601572 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.602102 10842 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:18:58.605042 17702 log.cc:927] Time spent T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
I20260709 13:18:58.606559 17589 consensus_queue.cc:1048] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:18:58.608323 17588 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:58.609519 10916 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:58.592864 17402 consensus_queue.cc:1059] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.018s
W20260709 13:18:58.611773 17703 log.cc:927] Time spent T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.266s	user 0.004s	sys 0.000s
I20260709 13:18:58.590620 17617 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:58.625602 10915 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:58.594219 17708 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:58.621317 17706 log.cc:927] Time spent T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 13:18:58.614985 17455 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:58.634503 10694 catalog_manager.cc:5697] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 11 to 13, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 13 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:18:58.636842 17619 consensus_queue.cc:1048] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:58.638875 17619 consensus_queue.cc:1048] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:58.639688 10916 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:58.641376 10916 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:58.643097 17564 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:18:58.636294 10694 catalog_manager.cc:5697] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 15 to 16, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 16 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:58.644760 11511 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
I20260709 13:18:58.628487 17680 raft_consensus.cc:2804] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:58.649386 10873 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 22.
I20260709 13:18:58.628677 17407 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:18:58.686309 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:58.649492 17402 raft_consensus.cc:3055] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:18:58.688127 17402 raft_consensus.cc:740] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:58.690763 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:58.633764 17451 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:18:58.693950 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 68439.1us
I20260709 13:18:58.695446 10796 raft_consensus.cc:1275] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
W20260709 13:18:58.696084 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.696520 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:58.702247 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:58.703078 10872 raft_consensus.cc:2393] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 20: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:58.706591 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:58.649829 17588 consensus_queue.cc:1048] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.046s
I20260709 13:18:58.652995 17536 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:58.695854 10796 pending_rounds.cc:85] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 9
I20260709 13:18:58.722370 10796 pending_rounds.cc:107] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:18:58.722677 10796 raft_consensus.cc:2889] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:58.684057 17673 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:18:58.694648 17706 log.cc:927] Time spent T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:18:58.652112 17670 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:58.720846 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 79526.1us
I20260709 13:18:58.682943 17680 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:58.689796 17402 consensus_queue.cc:260] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:58.649441 17438 consensus_queue.cc:1059] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.056s
I20260709 13:18:58.736425 10876 raft_consensus.cc:1275] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 13:18:58.632607 17708 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:58.733385 10867 raft_consensus.cc:1275] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:18:58.740960 17708 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:58.732700 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:58.743903 11509 raft_consensus.cc:1275] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 14. (term mismatch)
I20260709 13:18:58.744341 11509 pending_rounds.cc:85] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586: Aborting all ops after (but not including) 13
I20260709 13:18:58.744429 17455 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:58.703014 17564 raft_consensus.cc:697] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 20 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:58.746078 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:58.747097 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:58.747370 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:58.744577 11509 pending_rounds.cc:107] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586: Aborting uncommitted NO_OP operation due to leader change: 17.14
I20260709 13:18:58.748466 10794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2432) took 1092 ms. Trace:
I20260709 13:18:58.748733 11509 raft_consensus.cc:2889] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:58.749475 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:58.750465 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:58.751250 11512 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:58.748312 11508 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 15.
W20260709 13:18:58.757308 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.747677 10872 raft_consensus.cc:2376] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 22. Re-sending same reply.
I20260709 13:18:58.736483 17402 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:58.734349 17680 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:18:58.770321 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.754242 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:58.771049 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:58.768424 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:58.748803 10794 rpcz_store.cc:276] 0709 13:18:57.656057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.924866 (+268809us) service_pool.cc:224] Handling call
0709 13:18:57.955877 (+ 31011us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:57.981733 (+ 25856us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:18:57.982098 (+   365us) log.cc:844] Serialized 14 byte log entry
0709 13:18:58.244752 (+262654us) log.cc:844] Serialized 14 byte log entry
0709 13:18:58.376603 (+131851us) spinlock_profiling.cc:243] Waited 2.61 ms on lock 0x7b48002e0158. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c85ddeaa1 00007f9c85dd5b37 00007f9c85dd59ca 00007f9c85ecd26f 00007f9c85eccf37 00007f9c85ee155c 00007f9c85ee1508 00007f9c85ee1488 00007f9c85ee1443 00007f9c85ee06ec 00007f9c84e0aa96
0709 13:18:58.380888 (+  4285us) spinlock_profiling.cc:243] Waited 1.71 ms on lock 0x7b1800367ea0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc8522 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:18:58.747968 (+367080us) spinlock_profiling.cc:243] Waited 40.2 ms on lock 0x7b1800367ea0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7dd42c0a 00007f9c87482219 00007f9c7ddcaa40 00007f9c7ddc54e4 00007f9c7ddc5455 00007f9c80ead060 00007f9c80ebf716 00007f9c80ed0287 00007f9c80ed0208
0709 13:18:58.748120 (+   152us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:58.748184 (+    64us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:58.748232 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:58.748300 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:58.748412 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9894,"prepare.queue_time_us":1897,"prepare.run_cpu_time_us":102,"prepare.run_wall_time_us":3230,"raft.queue_time_us":13120,"raft.run_cpu_time_us":2167,"raft.run_wall_time_us":127670,"spinlock_wait_cycles":102464384,"thread_start_us":250525,"threads_started":1,"wal-append.queue_time_us":445143}
W20260709 13:18:58.769461 17709 log.cc:927] Time spent T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 13:18:58.769904 17641 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:18:58.779600 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 445483us
W20260709 13:18:58.773679 17714 log.cc:927] Time spent T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:18:58.781031 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:58.774523 17617 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:58.774926 17721 raft_consensus.cc:3055] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:18:58.774515 17711 log.cc:927] Time spent T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
I20260709 13:18:58.775660 17688 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:58.776090 17646 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:58.778218 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:58.769724 17564 consensus_queue.cc:237] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:58.779289 10875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 1059) took 1371 ms. Trace:
I20260709 13:18:58.780887 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:58.820492 10867 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 16.
I20260709 13:18:58.820726 10874 raft_consensus.cc:1275] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:18:58.774019 17622 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:58.828190 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:58.790923 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.784081 11511 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 16.
I20260709 13:18:58.786486 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:58.785701 17721 raft_consensus.cc:740] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:58.782502 17407 raft_consensus.cc:697] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 24 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:58.821034 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:58.843225 10876 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 19.
I20260709 13:18:58.820253 10875 rpcz_store.cc:276] 0709 13:18:57.396461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.568299 (+171838us) service_pool.cc:224] Handling call
0709 13:18:57.610916 (+ 42617us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:58.349786 (+738870us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:58.767028 (+417242us) spinlock_profiling.cc:243] Waited 632 ms on lock 0x7b540009b0d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:58.767225 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1452931840}
W20260709 13:18:58.822214 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 48660us
I20260709 13:18:58.824482 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:58.792312 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:58.779049 17089 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:58.838404 10948 raft_consensus.cc:1240] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 19. Current term is 20. Ops: []
I20260709 13:18:58.781904 17641 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:58.840523 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:58.851100 11509 raft_consensus.cc:2393] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 17: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:58.853626 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:58.854519 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:58.855958 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:58.858014 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:58.859313 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:58.860190 10798 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 16.
I20260709 13:18:58.863765 10793 raft_consensus.cc:1240] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:18:58.848834 17451 raft_consensus.cc:697] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:58.823431 17566 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:18:58.867604 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.867617 10874 raft_consensus.cc:1275] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:18:58.869531 10869 raft_consensus.cc:1240] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 21. Current term is 22. Ops: [18.13-21.15]
I20260709 13:18:58.873288 10799 raft_consensus.cc:2410] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 23 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 9.
I20260709 13:18:58.874938 17708 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.875340 17721 consensus_queue.cc:260] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:58.875527 17708 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:58.876753 17721 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:58.876304 17089 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:58.878072 17708 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:58.880353 10694 catalog_manager.cc:5697] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:18:58.882074 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.836900 17589 consensus_queue.cc:1048] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.035s
I20260709 13:18:58.845335 17617 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:58.875519 11504 raft_consensus.cc:1240] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 20. Current term is 21. Ops: []
I20260709 13:18:58.850350 17641 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:58.915692 10916 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:58.917318 17670 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.918020 17438 raft_consensus.cc:2804] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:58.917953 17670 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:58.918715 17438 raft_consensus.cc:697] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 17 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:58.919907 17438 consensus_queue.cc:237] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:58.920925 17670 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:58.922588 17617 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:58.923029 17673 consensus_queue.cc:1048] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:18:58.909644 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:58.923810 17673 consensus_queue.cc:1243] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer 468a8fd46e824f62b9c2c0d3532a6d26 log is divergent from this leader: its last log entry 24.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:18:58.924157 10867 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 17.
I20260709 13:18:58.888655 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:58.925918 10794 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 16.
I20260709 13:18:58.928143 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:58.928972 10801 raft_consensus.cc:2410] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:18:58.887991 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:58.917488 10842 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:18:58.892168 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.908886 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:58.931491 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:58.931578 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.932662 10796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3098) took 1249 ms. Trace:
I20260709 13:18:58.933161 10796 rpcz_store.cc:276] 0709 13:18:57.683882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.202123 (+518241us) service_pool.cc:224] Handling call
0709 13:18:58.220315 (+ 18192us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:58.725277 (+504962us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:58.902568 (+177291us) spinlock_profiling.cc:243] Waited 475 ms on lock 0x7b5400211bd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:18:58.932587 (+ 30019us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1091804544}
I20260709 13:18:58.891377 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:58.846146 17646 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.932092 10793 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
I20260709 13:18:58.883896 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:58.948737 11512 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 21.
W20260709 13:18:58.950865 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.938024 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:18:58.901085 17716 log.cc:927] Time spent T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.166s	user 0.000s	sys 0.000s
I20260709 13:18:58.952958 17670 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:18:58.958925 17670 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:58.959321 17670 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:18:58.841964 17622 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:58.961378 17622 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:58.962246 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:58.962975 17364 consensus_queue.cc:1048] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:18:58.963469 10868 raft_consensus.cc:1240] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:18:58.954674 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:58.965363 10869 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 19.
I20260709 13:18:58.956212 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:58.966264 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:58.966498 11507 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
I20260709 13:18:58.947072 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:58.962379 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:58.964732 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:58.965844 10841 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:58.967875 11504 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 16.
W20260709 13:18:58.967839 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 222735us
I20260709 13:18:58.964107 10694 catalog_manager.cc:5697] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 15 to 16, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 16 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:18:58.969192 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:58.972599 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:58.969753 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:58.900104 17719 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.970999 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:58.974839 11506 raft_consensus.cc:2410] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:18:58.974984 10801 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 18.
I20260709 13:18:58.971868 10799 raft_consensus.cc:2410] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:18:58.890383 17708 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.974982 11478 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:58.957751 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:58.950816 10872 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 22.
I20260709 13:18:58.978709 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:58.979844 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:58.979938 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:58.980645 10868 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 16.
W20260709 13:18:58.981828 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:58.973358 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 609130us
I20260709 13:18:58.986075 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:58.986725 10793 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 25.
I20260709 13:18:58.989508 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:58.990720 10872 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:18:58.992108 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:58.989647 10841 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:58.993364 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:58.891987 17725 consensus_queue.cc:1048] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.129s
I20260709 13:18:58.994932 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:58.974754 17536 raft_consensus.cc:2804] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:58.996966 10792 raft_consensus.cc:2393] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 22: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:18:58.997628 11478 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:18:58.997728 10801 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
I20260709 13:18:58.995770 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:59.004621 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:59.005734 10868 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 22.
I20260709 13:18:59.007179 17588 raft_consensus.cc:2804] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:59.007752 17588 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:59.008231 17588 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:59.022336 11478 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:59.023581 17415 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:18:59.023835 11478 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:59.024189 17415 raft_consensus.cc:697] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 22 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:18:59.024914 17611 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:18:59.025303 17611 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:58.999648 10766 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:59.030879 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:58.901787 17407 consensus_queue.cc:237] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:59.025509 11478 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:59.026111 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 26 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:58.973366 17475 consensus_queue.cc:1048] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:18:59.031401 10766 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:59.035285 10871 raft_consensus.cc:2410] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 26 because replica has last-logged OpId of term: 24 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 11.
I20260709 13:18:58.998689 11504 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
I20260709 13:18:58.994272 17455 raft_consensus.cc:2764] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 13:18:59.032732 17327 raft_consensus.cc:2804] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:59.054777 11505 raft_consensus.cc:2410] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:18:59.028414 17536 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:59.053881 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:58.979172 17646 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:59.033222 17611 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:58.993079 17708 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:59.058092 17536 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:59.063582 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:59.083762 10792 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 15.
W20260709 13:18:59.079190 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.978974 17719 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:59.027333 17562 consensus_queue.cc:1048] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:18:59.037817 17617 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:18:59.087779 17617 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:59.088307 17617 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:59.057335 10916 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:59.101523 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 44251.8us
I20260709 13:18:59.104171 10766 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:59.107610 10797 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
I20260709 13:18:59.042451 17622 consensus_queue.cc:1048] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.055s
I20260709 13:18:59.103726 10841 leader_election.cc:400] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Vote denied by peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553) with higher term. Message: Invalid argument: T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 19. Current term is 20.
I20260709 13:18:59.081291 17402 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:59.025303 17415 consensus_queue.cc:237] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:59.054504 17364 consensus_queue.cc:1059] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 13:18:59.120584 10766 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:59.121109 10916 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:59.079960 17619 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:59.132563 17727 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:59.133015 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:59.134631 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:59.137300 10798 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 19.
I20260709 13:18:59.137679 17210 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:59.138218 17210 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:59.140354 17210 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:18:59.143554 17571 consensus_meta.cc:287] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.756s	user 0.011s	sys 0.002s
I20260709 13:18:59.143913 17680 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:59.144042 17571 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:59.145257 10875 raft_consensus.cc:1240] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 18. Current term is 19. Ops: []
I20260709 13:18:59.146375 17571 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:59.147094 10874 raft_consensus.cc:2393] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 18: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:18:59.146652 17641 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:59.148326 10800 raft_consensus.cc:1240] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:18:59.148998 17675 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:59.149534 17675 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:59.149973 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:59.150852 17089 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:59.151160 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:59.151643 17675 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:59.153743 17641 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:59.156026 11508 raft_consensus.cc:1240] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:59.157521 17708 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:59.130764 17402 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:59.157521 17450 consensus_meta.cc:287] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.681s	user 0.008s	sys 0.001s
I20260709 13:18:59.159154 10766 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:59.160069 17402 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:18:59.160979 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.131346 17386 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:18:59.163789 17386 raft_consensus.cc:697] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:59.164742 17386 consensus_queue.cc:237] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:59.165691 17327 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:59.166059 10766 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:59.166165 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:59.166978 17607 raft_consensus.cc:2804] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:59.167085 17622 raft_consensus.cc:2804] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:18:59.167637 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:59.156070 17719 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:59.168468 10950 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 20: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:59.125749 17728 raft_consensus.cc:2804] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:59.169327 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:59.169512 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:59.169697 17728 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:59.170136 17728 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:18:59.170333 17587 consensus_meta.cc:287] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.614s	user 0.004s	sys 0.006s
I20260709 13:18:59.138230 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:59.170998 17623 consensus_queue.cc:1048] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:18:59.171787 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:59.172492 10944 raft_consensus.cc:2393] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 16: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:59.173170 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 26 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:59.173374 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:59.174062 17646 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:59.174523 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:59.106108 17688 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:59.055869 17451 consensus_queue.cc:237] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:59.172643 17210 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:59.171113 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:59.176285 17210 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:59.178990 17476 raft_consensus.cc:2749] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:59.129058 10842 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:18:59.168504 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 110465us
I20260709 13:18:59.151746 10800 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:18:59.183835 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:59.184643 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.170626 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:59.174590 17646 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.021885 17588 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:59.130429 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:59.190325 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:59.123515 10872 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 23.
I20260709 13:18:59.170478 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:18:59.119691 17539 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.010s	sys 0.018s
I20260709 13:18:59.119412 17455 raft_consensus.cc:2749] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:18:59.192060 10796 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 18.
I20260709 13:18:59.205456 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:59.208745 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 24 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:59.168098 17579 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:18:59.197203 17539 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.011s	sys 0.019s
I20260709 13:18:59.189432 17619 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:59.212085 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:59.227067 10942 raft_consensus.cc:1275] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:18:59.248095 10875 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:59.230480 10794 raft_consensus.cc:1240] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 20. Current term is 21. Ops: []
W20260709 13:18:59.235471 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 69498.6us
I20260709 13:18:59.239014 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:59.166208 17327 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:59.264003 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:18:59.222070 17730 log.cc:927] Time spent T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
W20260709 13:18:59.195549 17696 log.cc:927] Time spent T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.869s	user 0.004s	sys 0.004s
W20260709 13:18:59.210287 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 93179us
I20260709 13:18:59.325881 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:59.326305 10916 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:59.150665 17670 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:59.216213 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:59.327734 10872 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
I20260709 13:18:59.237820 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:18:59.328688 10793 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 17.
I20260709 13:18:59.193493 17498 raft_consensus.cc:2764] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 13:18:59.250330 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:59.238503 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:59.254774 10800 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 21.
I20260709 13:18:59.193773 17727 raft_consensus.cc:2749] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:18:59.330391 11510 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
W20260709 13:18:59.332527 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 125461us
I20260709 13:18:59.334007 11509 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 20.
I20260709 13:18:59.341027 17607 raft_consensus.cc:697] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:59.342482 17607 consensus_queue.cc:237] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:59.345319 17622 raft_consensus.cc:697] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:18:59.349730 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:18:59.350960 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 325529us
W20260709 13:18:59.357880 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 228887us
I20260709 13:18:59.223143 17729 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.362509 10944 raft_consensus.cc:2410] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
W20260709 13:18:59.263825 17701 log.cc:927] Time spent T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:18:59.361033 10842 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:59.233367 17623 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:59.368525 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.240864 17588 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:59.240689 17562 raft_consensus.cc:2804] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:59.371941 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:59.351863 11478 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:59.346496 17622 consensus_queue.cc:237] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:59.246934 10694 catalog_manager.cc:5697] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 12 to 13, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 13 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:18:59.209841 17646 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:59.378077 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:18:59.363263 17734 log.cc:927] Time spent T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:18:59.380999 10916 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
W20260709 13:18:59.385339 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:59.385342 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.386579 10766 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:59.387739 10941 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 16.
I20260709 13:18:59.377466 17562 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:59.369274 17727 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:59.391829 10842 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:59.394951 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:59.402984 10942 raft_consensus.cc:2410] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:18:59.398496 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:59.395208 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:59.404143 11505 raft_consensus.cc:1275] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:18:59.404292 11510 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 23.
I20260709 13:18:59.399770 11478 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:59.406647 11478 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:18:59.408746 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:59.409363 10942 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
I20260709 13:18:59.410607 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:59.400653 17407 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:18:59.395733 17588 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:18:59.370566 17729 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:59.397431 17562 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:59.405557 17727 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:59.424530 10766 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:59.431708 10766 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:59.436086 17603 raft_consensus.cc:2804] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:59.379513 17623 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:59.415089 17738 raft_consensus.cc:2804] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:59.446743 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:59.415439 17089 raft_consensus.cc:2749] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:18:59.451025 10944 raft_consensus.cc:2410] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:18:59.445050 17603 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:18:59.436124 17587 consensus_queue.cc:1059] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.016s
I20260709 13:18:59.455828 17708 raft_consensus.cc:3055] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:18:59.403811 17415 consensus_queue.cc:1048] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.023s
I20260709 13:18:59.464195 10916 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:18:59.464227 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:59.427321 17450 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:18:59.422626 17756 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:18:59.423326 17407 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:18:59.425642 17588 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:18:59.478390 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 46793.1us
I20260709 13:18:59.485313 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 21 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:18:59.394604 17745 consensus_queue.cc:1048] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.051s
W20260709 13:18:59.563503 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.567576 10942 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 22.
I20260709 13:18:59.434587 17727 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:18:59.552134 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:59.570537 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 131259us
I20260709 13:18:59.487600 17708 raft_consensus.cc:740] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:18:59.463575 17210 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:59.579691 10766 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:59.468652 17680 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:59.465674 17738 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:59.572089 17756 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:59.475171 17744 raft_consensus.cc:2749] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:18:59.485548 17603 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:18:59.588559 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.566834 17670 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:59.572284 17588 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:18:59.572423 17745 consensus_queue.cc:1243] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer 12682eb70bc84c76879e04a8628ee586 log is divergent from this leader: its last log entry 20.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:18:59.572292 17450 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:59.572201 17407 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:18:59.613057 11478 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:18:59.611879 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.611675 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:18:59.614426 10942 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:18:59.615403 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 168734us
I20260709 13:18:59.592957 17738 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:59.590939 17571 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:18:59.621656 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:59.627868 10876 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 18.
I20260709 13:18:59.634516 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:18:59.635197 10870 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:59.636243 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 24 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:18:59.636858 10872 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 23.
I20260709 13:18:59.576558 17089 consensus_queue.cc:1048] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.041s
I20260709 13:18:59.599568 17564 raft_consensus.cc:2804] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:18:59.598417 17210 raft_consensus.cc:2804] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:59.610090 17539 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.595443 17756 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:59.617308 17758 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.617883 17450 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:59.621417 17579 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.633801 17587 raft_consensus.cc:2804] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:59.627674 17571 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:59.640900 17619 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:59.614740 17594 consensus_queue.cc:1059] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.034s
W20260709 13:18:59.639599 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 56887.1us
I20260709 13:18:59.674039 10942 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 17.
I20260709 13:18:59.647420 17210 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:59.676982 10875 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 21.
I20260709 13:18:59.679625 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:18:59.659143 17740 log.cc:927] Time spent T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:18:59.656183 17564 raft_consensus.cc:697] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 20 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:18:59.689483 10947 raft_consensus.cc:2393] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 16: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
W20260709 13:18:59.692852 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 49621.9us
W20260709 13:18:59.699347 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 55584.1us
I20260709 13:18:59.700126 11478 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:59.685565 10686 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 2b82f18992434ed5ba58b805107060d4 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 468a8fd46e824f62b9c2c0d3532a6d26 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 20 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:18:59.668784 17587 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:18:59.676131 17571 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:18:59.703280 10686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 73830.9us
I20260709 13:18:59.683808 17744 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:18:59.589254 10694 catalog_manager.cc:5697] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:59.687093 17210 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:59.684809 17646 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:59.707947 11478 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:18:59.711894 10916 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:18:59.721627 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 329899us
I20260709 13:18:59.668928 17579 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:59.731413 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 92399.5us
I20260709 13:18:59.744942 10841 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:59.709806 17739 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.709358 17587 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:59.739883 10842 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:59.709556 17455 raft_consensus.cc:2749] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:59.662601 17539 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:18:59.751917 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 44050.2us
I20260709 13:18:59.752449 10841 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:18:59.670060 17758 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:59.769150 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:18:59.772910 11504 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:18:59.769435 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:59.669584 17708 consensus_queue.cc:260] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:18:59.787827 11509 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 16.
I20260709 13:18:59.788725 11512 raft_consensus.cc:1240] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 13:18:59.791249 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:59.792423 11511 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 20: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:18:59.793320 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:59.794935 10916 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:18:59.799140 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:59.799539 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:18:59.753309 17729 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:18:59.765458 17731 raft_consensus.cc:2804] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:59.799820 11510 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 18.
I20260709 13:18:59.802834 17762 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:18:59.811306 17748 log.cc:927] Time spent T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 13:18:59.811631 17752 consensus_queue.cc:1059] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:18:59.813331 17611 consensus_meta.cc:287] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.713s	user 0.009s	sys 0.000s
I20260709 13:18:59.787295 17670 raft_consensus.cc:2804] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:59.815021 11504 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 23.
I20260709 13:18:59.765640 17739 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:18:59.820160 17617 consensus_meta.cc:287] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.731s	user 0.009s	sys 0.002s
I20260709 13:18:59.735881 17744 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:18:59.766649 17763 consensus_queue.cc:1059] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.076s
W20260709 13:18:59.831228 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:59.833283 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 80914.8us
I20260709 13:18:59.766397 17765 raft_consensus.cc:2749] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:18:59.808283 17539 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:59.775215 17619 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:18:59.880007 10841 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:18:59.881604 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 182288us
W20260709 13:18:59.953310 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:59.951153 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 173432us
W20260709 13:18:59.992327 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 112407us
I20260709 13:19:00.001969 10841 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:18:59.855072 17708 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:59.835500 17731 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:19:00.058022 17741 log.cc:927] Time spent T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.267s	user 0.001s	sys 0.000s
I20260709 13:18:59.877741 17611 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:00.059295 17750 log.cc:927] Time spent T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.282s	user 0.001s	sys 0.000s
I20260709 13:19:00.060827 17611 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:00.061671 11507 raft_consensus.cc:1240] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:18:59.948235 17738 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:00.063232 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 262009us
W20260709 13:18:59.992630 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.834429 17623 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:59.857689 17670 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:59.947988 17758 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:18:59.944949 17579 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:18:59.977092 17589 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.977694 17451 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:00.056437 17731 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:00.073971 10916 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:00.078293 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 21 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:00.078310 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:00.103909 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:59.837417 17588 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:00.110857 10793 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:00.073601 17670 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:00.120080 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:00.120888 10792 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
I20260709 13:18:59.962288 17756 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:00.064316 17763 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:59.854104 17721 raft_consensus.cc:2804] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:18:59.807597 17455 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.958698 17727 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:00.110188 10796 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 22.
I20260709 13:18:59.925818 17617 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:00.110459 10795 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:18:59.977699 17603 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:00.114821 10767 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.004s	sys 0.012s
I20260709 13:19:00.114488 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:18:59.958199 17407 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:00.154759 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.157618 10799 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:00.158097 10767 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.005s	sys 0.012s
W20260709 13:19:00.158535 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 672385us
I20260709 13:19:00.088764 11503 raft_consensus.cc:2393] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 18: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:18:59.836959 17564 consensus_queue.cc:237] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:00.141530 11478 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:19:00.109717 17438 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:00.160185 11478 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:00.122581 17282 raft_consensus.cc:2749] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:19:00.110101 17753 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.010s	sys 0.005s
I20260709 13:19:00.167769 10915 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:00.172411 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:00.144857 17763 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:00.142740 17611 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:00.228859 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:00.150671 17579 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:00.179791 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:00.234236 10792 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 15.
W20260709 13:19:00.164491 17536 consensus_meta.cc:287] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 1.052s	user 0.011s	sys 0.000s
W20260709 13:19:00.174188 17753 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.010s	sys 0.005s
I20260709 13:19:00.112679 17451 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:00.114437 17589 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:00.246306 17752 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:00.249102 17562 consensus_meta.cc:287] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.811s	user 0.007s	sys 0.001s
I20260709 13:19:00.251542 17617 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:00.252030 17562 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:00.253444 17720 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.008s	sys 0.008s
I20260709 13:19:00.253774 17407 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:00.254208 17562 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:00.249617 10870 consensus_meta.cc:287] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.608s	user 0.005s	sys 0.001s
W20260709 13:19:00.250455 17773 log.cc:927] Time spent T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:19:00.255081 17589 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:00.251477 17752 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:00.255846 10945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 2706) took 1091 ms. Trace:
I20260709 13:19:00.256914 10945 rpcz_store.cc:276] 0709 13:18:59.164754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.174052 (+  9298us) service_pool.cc:224] Handling call
0709 13:18:59.174357 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:59.347387 (+173030us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:59.347419 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:59.370063 (+ 22644us) log.cc:844] Serialized 26 byte log entry
0709 13:18:59.856588 (+486525us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:18:59.856635 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:59.856703 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:00.255610 (+398907us) raft_consensus.cc:1685] finished
0709 13:19:00.255633 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:00.255791 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":73874,"raft.queue_time_us":90195,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":34639,"thread_start_us":434230,"threads_started":1,"wal-append.queue_time_us":716910}
I20260709 13:19:00.258069 17752 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:00.256198 17589 raft_consensus.cc:2804] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:19:00.258724 10870 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 17.
I20260709 13:19:00.259263 17589 raft_consensus.cc:697] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 23 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:00.260383 17589 consensus_queue.cc:237] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:00.263576 17777 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:00.250088 17587 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:00.258268 17767 log.cc:927] Time spent T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.146s	user 0.000s	sys 0.000s
W20260709 13:19:00.272764 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.286080 10799 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
W20260709 13:19:00.238706 17728 consensus_meta.cc:287] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 1.068s	user 0.015s	sys 0.000s
I20260709 13:19:00.289515 10793 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
I20260709 13:19:00.167186 17438 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:00.251379 17751 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.710s	user 0.006s	sys 0.015s
I20260709 13:19:00.228611 17763 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:00.254437 17407 consensus_queue.cc:1059] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.035s
W20260709 13:19:00.302182 17751 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.857s	user 0.006s	sys 0.016s
W20260709 13:19:00.263613 17768 log.cc:927] Time spent T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:19:00.231177 17762 raft_consensus.cc:697] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:19:00.300587 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 298720us
I20260709 13:19:00.160944 17455 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:00.303256 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:00.253801 17720 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.808s	user 0.008s	sys 0.008s
I20260709 13:19:00.292630 10794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 944) took 1316 ms. Trace:
I20260709 13:19:00.289251 17680 raft_consensus.cc:2764] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:19:00.289062 17777 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:00.371153 10794 rpcz_store.cc:276] 0709 13:18:58.976173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.358997 (+382824us) service_pool.cc:224] Handling call
0709 13:18:59.375622 (+ 16625us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:59.390364 (+ 14742us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:18:59.406304 (+ 15940us) log.cc:844] Serialized 14 byte log entry
0709 13:19:00.103420 (+697116us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b180036eda0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a5523a 00007f9c80ebfa2a
0709 13:19:00.291906 (+188486us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:00.292009 (+   103us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:00.292125 (+   116us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:00.292257 (+   132us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:00.292512 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":19899,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":2677,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":9579,"spinlock_wait_cycles":24990464,"thread_start_us":188001,"threads_started":1}
I20260709 13:19:00.289997 10767 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:00.240940 17611 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:00.242476 17536 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:00.239224 17579 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:00.378988 17617 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:19:00.379880 17764 log.cc:927] Time spent T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
W20260709 13:19:00.379773 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 89901.3us
W20260709 13:19:00.380561 17771 log.cc:927] Time spent T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.269s	user 0.000s	sys 0.000s
I20260709 13:19:00.382283 10841 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:19:00.393240 17721 raft_consensus.cc:697] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 18 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:00.314093 17738 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:00.383338 17739 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:00.434729 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:00.435528 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:00.436226 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:00.439078 10875 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 18.
I20260709 13:19:00.439788 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:00.440022 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:00.441555 10871 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 20: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:00.300606 17728 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:00.449226 10867 raft_consensus.cc:2393] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 19: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:00.449723 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:00.452412 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:00.452843 10767 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:00.453568 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:00.390054 17777 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:00.457793 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 305768us
I20260709 13:19:00.429955 17415 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:19:00.459686 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 44818.2us
I20260709 13:19:00.464260 10842 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:00.464809 10916 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
W20260709 13:19:00.468818 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 86635us
I20260709 13:19:00.489009 10797 raft_consensus.cc:1240] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 17. Current term is 18. Ops: []
W20260709 13:19:00.493273 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 51968us
I20260709 13:19:00.452848 17753 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:00.459717 17622 raft_consensus.cc:2749] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:00.449445 17780 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:19:00.468789 17571 consensus_meta.cc:287] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.745s	user 0.009s	sys 0.003s
I20260709 13:19:00.461054 17588 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:00.463181 17744 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:00.470814 17769 log.cc:927] Time spent T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.303s	user 0.000s	sys 0.001s
W20260709 13:19:00.524453 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 60282.4us
I20260709 13:19:00.497526 17756 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:00.466495 17787 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:00.465497 17727 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:00.463584 17415 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:00.495642 17589 raft_consensus.cc:3055] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:19:00.522701 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 58214.8us
I20260709 13:19:00.525166 10842 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:00.499714 17587 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:00.508301 17791 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:00.526120 10946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 2697) took 1364 ms. Trace:
W20260709 13:19:00.512349 17772 log.cc:927] Time spent T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.275s	user 0.001s	sys 0.001s
W20260709 13:19:00.535967 17779 log.cc:927] Time spent T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.143s	user 0.000s	sys 0.000s
I20260709 13:19:00.542213 10842 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:00.546234 10842 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:19:00.547389 10842 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:00.533463 17539 raft_consensus.cc:2804] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:00.539288 10946 rpcz_store.cc:276] 0709 13:18:59.161853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.165217 (+  3364us) service_pool.cc:224] Handling call
0709 13:18:59.165582 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:59.210033 (+ 44451us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:59.210062 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:59.210527 (+   465us) log.cc:844] Serialized 26 byte log entry
0709 13:18:59.677230 (+466703us) spinlock_profiling.cc:243] Waited 15.4 ms on lock 0x7b18003d2520. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a550d1 00007f9c80e70a0f
0709 13:18:59.778914 (+101684us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:59.779082 (+   168us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:59.779329 (+   247us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:00.521937 (+742608us) raft_consensus.cc:1685] finished
0709 13:19:00.521967 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:00.526057 (+  4090us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44025,"raft.queue_time_us":58144,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":22104,"spinlock_wait_cycles":35406080,"thread_start_us":101367,"threads_started":1,"wal-append.queue_time_us":811973}
I20260709 13:19:00.537135 17589 raft_consensus.cc:740] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:00.519167 17571 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:00.575351 10945 raft_consensus.cc:1240] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:00.538108 17415 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:00.540046 17603 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:00.574185 10795 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 25.
I20260709 13:19:00.541921 17791 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:00.496415 17762 consensus_queue.cc:237] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:00.593355 10794 raft_consensus.cc:1240] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:00.555389 17756 raft_consensus.cc:2804] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:00.591022 10792 raft_consensus.cc:1240] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:19:00.536396 17744 consensus_queue.cc:1048] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:19:00.594592 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:00.518151 17753 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:00.592694 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:19:00.631556 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 127989us
I20260709 13:19:00.553013 17795 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:19:00.588682 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 420951us
I20260709 13:19:00.559976 17451 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:00.600646 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:00.578755 17766 raft_consensus.cc:2804] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:00.577018 17539 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:00.639567 17539 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:00.640355 17789 log.cc:927] Time spent T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 13:19:00.640874 17776 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:00.641402 17776 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:00.643980 17571 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:00.592547 17791 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:00.645004 17455 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:00.596280 17587 consensus_queue.cc:1059] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.051s
I20260709 13:19:00.633955 17536 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:00.598567 17756 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:00.624545 17794 consensus_queue.cc:1048] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.062s
I20260709 13:19:00.593465 17438 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:00.646744 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:00.665578 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:00.581736 17799 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:19:00.661482 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.671761 10945 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 18.
I20260709 13:19:00.578406 17562 raft_consensus.cc:2804] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:00.628620 10795 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:00.626224 10842 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:00.536528 17787 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:00.533463 17721 consensus_queue.cc:237] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:00.635087 10792 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:00.634138 10874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 2479) took 1431 ms. Trace:
I20260709 13:19:00.525250 17780 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:00.685338 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 59242.8us
W20260709 13:19:00.656131 17327 consensus_meta.cc:287] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 1.290s	user 0.006s	sys 0.007s
W20260709 13:19:00.646926 17731 consensus_meta.cc:287] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.527s	user 0.007s	sys 0.004s
W20260709 13:19:00.652921 17790 log.cc:927] Time spent T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
W20260709 13:19:00.652258 17775 log.cc:927] Time spent T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
I20260709 13:19:00.693406 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:00.700220 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:00.634486 17795 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:00.651660 17798 log.cc:927] Time spent T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:19:00.647542 17587 raft_consensus.cc:3055] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:19:00.704476 10796 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:19:00.658140 17756 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:00.659789 17402 raft_consensus.cc:3055] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:00.706120 10795 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
I20260709 13:19:00.693228 17766 raft_consensus.cc:697] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:00.663157 17728 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:00.674491 17579 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:00.652227 17619 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:00.665230 17801 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:00.739562 17788 consensus_queue.cc:1059] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 13:19:00.740007 17619 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:19:00.699441 17450 consensus_meta.cc:287] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 1.024s	user 0.008s	sys 0.005s
I20260709 13:19:00.690320 10874 rpcz_store.cc:276] 0709 13:18:59.203175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.203342 (+   167us) service_pool.cc:224] Handling call
0709 13:18:59.226229 (+ 22887us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:59.361321 (+135092us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:59.361355 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:59.361857 (+   502us) log.cc:844] Serialized 26 byte log entry
0709 13:18:59.765724 (+403867us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:18:59.765768 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:59.765835 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:00.633801 (+867966us) raft_consensus.cc:1685] finished
0709 13:19:00.633831 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:00.634045 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":106276,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":127,"thread_start_us":471987,"threads_started":2,"wal-append.queue_time_us":664227}
I20260709 13:19:00.727541 17402 raft_consensus.cc:740] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:00.718775 17475 consensus_queue.cc:1059] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 13:19:00.740808 17450 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:00.740365 17801 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:00.742062 11505 raft_consensus.cc:1275] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:19:00.744686 17450 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:00.745208 17475 raft_consensus.cc:3055] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:00.745638 17475 raft_consensus.cc:740] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:00.747939 17801 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:00.748884 17475 consensus_queue.cc:260] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:00.693518 17536 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:19:00.686750 17799 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:00.728974 10842 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:00.706560 10916 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:00.729244 10792 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
I20260709 13:19:00.658458 17603 consensus_queue.cc:1048] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 13:19:00.717239 17587 raft_consensus.cc:740] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
W20260709 13:19:00.740720 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 52778.6us
I20260709 13:19:00.765569 10842 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:00.703920 17562 raft_consensus.cc:697] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:00.757812 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:00.758443 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:00.769068 11507 raft_consensus.cc:1813] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Handling vote request from an unknown peer cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:00.717646 17795 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:00.763868 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:00.787425 11510 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:00.782687 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:00.758621 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:00.788220 11504 raft_consensus.cc:2376] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 20. Re-sending same reply.
I20260709 13:19:00.769120 11511 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 18.
I20260709 13:19:00.754751 17475 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:00.795388 10873 raft_consensus.cc:1275] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:19:00.709236 17571 consensus_queue.cc:1059] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:19:00.704196 17451 consensus_queue.cc:1048] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 13:19:00.796813 17451 consensus_queue.cc:1243] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer e01ecd008edd4aa2b6b2b47efbc2ea87 log is divergent from this leader: its last log entry 19.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:19:00.703337 17327 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:00.807000 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:00.776746 17587 consensus_queue.cc:260] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:00.777791 17788 raft_consensus.cc:2804] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:00.777980 11507 raft_consensus.cc:2410] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:19:00.778299 17364 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:19:00.778440 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 71996.9us
I20260709 13:19:00.778970 17801 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:00.779539 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:00.780576 17780 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:00.780579 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:00.854686 10767 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:19:00.865331 11509 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 20.
I20260709 13:19:00.784364 17787 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:00.753310 17210 consensus_meta.cc:287] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 1.002s	user 0.008s	sys 0.004s
I20260709 13:19:00.902712 17210 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:00.792451 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:00.907239 10795 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
I20260709 13:19:00.794035 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:00.797245 17562 consensus_queue.cc:237] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:00.798497 17766 consensus_queue.cc:237] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:00.764843 17799 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:00.788107 10949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 2708) took 1131 ms. Trace:
I20260709 13:19:00.915027 10949 rpcz_store.cc:276] 0709 13:18:59.656647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.656766 (+   119us) service_pool.cc:224] Handling call
0709 13:18:59.677864 (+ 21098us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:59.678233 (+   369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:59.678262 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:59.705190 (+ 26928us) log.cc:844] Serialized 26 byte log entry
0709 13:19:00.165484 (+460294us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:00.165535 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:00.165611 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:00.787844 (+622233us) raft_consensus.cc:1685] finished
0709 13:19:00.787871 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:00.788031 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":926,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":5900,"thread_start_us":391985,"threads_started":1,"wal-append.queue_time_us":711401}
I20260709 13:19:00.801855 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:00.802486 17805 consensus_queue.cc:1059] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:19:00.803300 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:19:00.804245 17708 consensus_meta.cc:287] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.748s	user 0.008s	sys 0.004s
W20260709 13:19:00.806005 17670 consensus_meta.cc:287] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.650s	user 0.009s	sys 0.001s
I20260709 13:19:00.807020 10946 raft_consensus.cc:1240] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:00.808159 17728 consensus_queue.cc:1048] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:19:00.709883 17731 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:00.813321 17415 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:00.814558 11506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56404 (request call id 1137) took 1042 ms. Trace:
I20260709 13:19:00.815603 17327 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:00.934695 10869 raft_consensus.cc:2410] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:19:00.776217 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:00.817130 11504 raft_consensus.cc:2393] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 20: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
W20260709 13:19:00.818050 10686 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 2b82f18992434ed5ba58b805107060d4 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 468a8fd46e824f62b9c2c0d3532a6d26 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 20 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:00.820636 17793 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:00.821233 17752 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:00.824663 11510 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
I20260709 13:19:00.857319 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:00.800832 17798 log.cc:927] Time spent T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:19:00.860718 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:00.871867 10796 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 26.
I20260709 13:19:00.864737 11512 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 21.
I20260709 13:19:00.872888 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:00.804584 17617 raft_consensus.cc:2749] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:00.888710 17539 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:00.786933 17536 raft_consensus.cc:697] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:00.918591 11508 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 22.
I20260709 13:19:00.906493 17210 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:00.929215 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:00.936404 10686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 118422us
I20260709 13:19:00.849002 17571 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:00.959081 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 138473us
I20260709 13:19:00.801196 17744 consensus_queue.cc:1048] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.124s
I20260709 13:19:00.941282 10871 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 18.
W20260709 13:19:00.945735 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:00.945583 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 121641us
I20260709 13:19:00.980727 10841 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:01.001664 10793 raft_consensus.cc:1240] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:19:01.002127 10841 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:01.003754 10841 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:00.942371 10916 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:19:01.005043 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 62780.5us
I20260709 13:19:00.934456 11506 rpcz_store.cc:276] 0709 13:18:59.771843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.771963 (+   120us) service_pool.cc:224] Handling call
0709 13:18:59.830670 (+ 58707us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:59.831026 (+   356us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:59.831064 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:59.885293 (+ 54229us) log.cc:844] Serialized 26 byte log entry
0709 13:19:00.297935 (+412642us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:19:00.297980 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:00.298050 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:00.814291 (+516241us) raft_consensus.cc:1685] finished
0709 13:19:00.814317 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:00.814491 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":209502,"threads_started":1,"wal-append.queue_time_us":492061}
I20260709 13:19:01.006510 10916 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:00.951584 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:00.950404 10867 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 19.
I20260709 13:19:00.929265 17587 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:00.925212 17801 raft_consensus.cc:697] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:00.930652 17386 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:00.960098 11478 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:19:01.011973 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 51946.8us
I20260709 13:19:01.012825 11478 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:01.010586 10916 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:00.962216 17498 raft_consensus.cc:3055] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:00.962492 17415 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:00.964772 10766 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:19:01.021207 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 56492.2us
I20260709 13:19:01.022150 10915 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:00.965432 10795 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2"
dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
mode: GRACEFUL
new_leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
 from {username='slave'} at 127.0.0.1:40148
I20260709 13:19:00.968827 17731 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:01.024317 17455 raft_consensus.cc:2749] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:19:00.969287 17327 consensus_queue.cc:1059] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:19:01.039719 10797 raft_consensus.cc:1240] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:00.969527 17451 consensus_queue.cc:1048] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:19:01.027320 17794 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:01.027627 17725 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:01.045428 17725 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:01.036623 17438 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:01.047750 17725 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:00.952796 17752 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:01.048656 17756 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:01.049925 17752 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:00.971535 17536 consensus_queue.cc:237] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:01.051208 10873 raft_consensus.cc:2393] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 22: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:01.051546 17756 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:01.052798 17455 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:01.053480 17455 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:00.971892 17751 raft_consensus.cc:3055] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:00.939448 17670 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:01.054980 17751 raft_consensus.cc:740] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:19:01.029340 17802 log.cc:927] Time spent T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.390s	user 0.001s	sys 0.001s
I20260709 13:19:00.975337 17799 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:01.030859 17564 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:01.056353 17455 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:01.056643 10875 raft_consensus.cc:2393] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 18: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:01.026425 17778 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:01.032922 17808 raft_consensus.cc:2804] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:01.057319 17799 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:00.977632 17539 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:00.982273 17603 raft_consensus.cc:3055] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:19:01.055898 17751 consensus_queue.cc:260] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:01.056371 17564 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:01.058277 17603 raft_consensus.cc:740] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:01.058748 17564 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:01.033304 17450 consensus_queue.cc:1048] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 13:19:01.059494 17603 consensus_queue.cc:260] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:00.951020 17793 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:00.983398 17571 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:00.984397 17619 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:00.816839 17791 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:00.987910 17759 log.cc:927] Time spent T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 1.348s	user 0.003s	sys 0.003s
I20260709 13:19:00.991143 17753 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:00.992638 17777 consensus_meta.cc:287] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.527s	user 0.006s	sys 0.004s
I20260709 13:19:01.067759 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:01.068523 10800 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
I20260709 13:19:01.068388 17777 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:01.058728 17751 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:01.070099 17793 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:01.070662 17777 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:00.928965 17787 consensus_queue.cc:1048] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:19:01.071303 10870 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 18.
W20260709 13:19:00.992939 17763 consensus_meta.cc:287] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.638s	user 0.008s	sys 0.002s
I20260709 13:19:01.072202 17763 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:01.072666 17571 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:01.073200 17438 raft_consensus.cc:697] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 21 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:01.073609 10794 raft_consensus.cc:1240] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 24. Current term is 25. Ops: []
I20260709 13:19:01.074331 17763 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:00.993511 17589 consensus_queue.cc:260] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:01.075134 17670 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:01.075286 17589 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:01.076090 10868 raft_consensus.cc:2368] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 18. Current term is 19.
I20260709 13:19:00.998852 10842 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:19:01.076664 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 77914.2us
I20260709 13:19:00.999449 10799 raft_consensus.cc:1240] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:00.990190 17795 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:00.961241 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:01.000517 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:01.077955 10796 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 16.
I20260709 13:19:01.024128 10795 raft_consensus.cc:606] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Received request to transfer leadership to TS 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:00.959160 17788 raft_consensus.cc:697] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:01.027117 10915 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:00.959499 17805 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:01.057842 17808 raft_consensus.cc:697] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 17 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:01.025660 10841 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:01.080873 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 55325.8us
I20260709 13:19:01.063010 10797 raft_consensus.cc:1275] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
W20260709 13:19:01.021603 11478 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.116.129:39685 timed out after 1.571s (SENT) [suppressed 12 similar messages]
W20260709 13:19:01.084066 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 62484.7us
W20260709 13:19:01.084282 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 57240.1us
I20260709 13:19:01.038522 17760 raft_consensus.cc:2749] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:01.040205 17680 raft_consensus.cc:2749] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:01.070895 17794 raft_consensus.cc:697] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 20 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:01.074363 17438 consensus_queue.cc:237] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:01.078207 10792 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:01.078670 10795 raft_consensus.cc:614] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:01.025410 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:01.023296 10916 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:01.082571 10842 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:01.083019 10841 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:01.085870 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 231288us
I20260709 13:19:01.013387 17729 raft_consensus.cc:2804] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:01.024557 17498 raft_consensus.cc:740] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:01.120606 17729 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:01.121153 17729 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:01.121403 17498 consensus_queue.cc:260] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:01.122879 17498 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:01.101322 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:01.127164 10796 raft_consensus.cc:1813] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Handling vote request from an unknown peer cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:01.128008 10796 raft_consensus.cc:2410] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:19:01.104247 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:01.100669 10915 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:01.111346 10801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3128) took 1034 ms. Trace:
I20260709 13:19:01.129211 11510 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 18.
I20260709 13:19:01.129350 10801 rpcz_store.cc:276] 0709 13:19:00.077186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:00.079003 (+  1817us) service_pool.cc:224] Handling call
0709 13:19:00.079445 (+   442us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:00.079876 (+   431us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:00.079914 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:00.080519 (+   605us) log.cc:844] Serialized 26 byte log entry
0709 13:19:00.500842 (+420323us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:00.500902 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:00.500994 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:01.028555 (+527561us) spinlock_profiling.cc:243] Waited 30.2 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:19:01.081193 (+ 52638us) raft_consensus.cc:1685] finished
0709 13:19:01.081217 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:01.111246 (+ 30029us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":938,"raft.run_wall_time_us":23035,"spinlock_wait_cycles":90028672,"thread_start_us":349628,"threads_started":1,"wal-append.queue_time_us":523163}
I20260709 13:19:01.070981 17801 consensus_queue.cc:237] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 20.9, Last appended by leader: 9, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:01.117288 10767 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:01.130769 17787 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:01.131501 17787 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:01.131728 17776 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:01.111091 17411 raft_consensus.cc:3055] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:01.132838 17411 raft_consensus.cc:740] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:01.132350 17788 consensus_queue.cc:237] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:01.102262 17793 raft_consensus.cc:2749] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:01.133766 17411 consensus_queue.cc:260] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:01.119336 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:19:01.119988 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 96795.1us
I20260709 13:19:01.135090 17619 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:01.063271 17386 raft_consensus.cc:697] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:01.135494 10916 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:01.131851 17787 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:01.136281 17386 consensus_queue.cc:237] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:01.100133 17725 raft_consensus.cc:2804] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:01.137351 17327 raft_consensus.cc:2749] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:19:01.137604 17725 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:01.138016 17725 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:01.138617 11509 raft_consensus.cc:1240] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 22 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 21. Current term is 22. Ops: [19.14-21.15]
I20260709 13:19:01.139190 10915 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:01.065415 17603 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:01.142974 17731 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:01.143545 17721 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:01.145042 10944 raft_consensus.cc:1275] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:19:01.080606 17451 raft_consensus.cc:3055] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:01.151103 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:01.153872 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:01.143373 17794 consensus_queue.cc:237] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:01.156469 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:01.146104 10916 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:01.158042 10916 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:01.145527 10767 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:01.131753 11504 raft_consensus.cc:1240] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:19:01.147055 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:01.119074 10841 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:01.147820 11508 raft_consensus.cc:1275] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:19:01.104549 17805 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:01.144002 17721 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:01.085450 17670 raft_consensus.cc:2804] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:01.080219 17622 raft_consensus.cc:2749] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:01.080000 17617 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:01.156575 10792 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
W20260709 13:19:01.151041 10798 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39442 (request call id 946) took 2110 ms (client timeout 1999 ms). Trace:
W20260709 13:19:01.191182 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.160635 10795 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:01.161978 11509 raft_consensus.cc:1275] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 13:19:01.191452 10798 rpcz_store.cc:269] 0709 13:18:59.041071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.137975 (+ 96904us) service_pool.cc:224] Handling call
0709 13:19:01.150842 (+2012867us) spinlock_profiling.cc:243] Waited 629 ms on lock 0x7b5400294d54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:01.150971 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1446883072}
I20260709 13:19:01.151700 17619 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:01.167470 10947 raft_consensus.cc:1275] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:19:01.169111 10841 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:01.203226 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:01.204553 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:01.155889 17327 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:01.169785 10793 raft_consensus.cc:2393] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 18: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:01.170277 10767 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:01.172178 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:01.207270 10941 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:01.172851 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:01.207808 10767 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:01.208037 10943 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:01.101609 17364 raft_consensus.cc:697] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:01.183525 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:01.208738 10949 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 16.
I20260709 13:19:01.183936 11503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56416 (request call id 1052) took 1399 ms. Trace:
I20260709 13:19:01.167819 17407 raft_consensus.cc:2749] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:01.186030 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:01.173731 17721 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:01.143371 17731 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:01.189670 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:01.145391 17753 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:19:01.164765 10801 raft_consensus.cc:1240] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 25. Current term is 26. Ops: [23.12-25.13]
I20260709 13:19:01.162628 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:19:01.233939 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 54265.4us
I20260709 13:19:01.192853 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:01.234549 10948 raft_consensus.cc:2393] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 17: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:01.146267 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:01.235085 11506 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 21.
I20260709 13:19:01.152112 17451 raft_consensus.cc:740] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:01.163008 17799 raft_consensus.cc:2749] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:01.159322 17820 raft_consensus.cc:2804] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:01.211133 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:01.259511 11511 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
I20260709 13:19:01.188633 10796 raft_consensus.cc:1240] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:19:01.211520 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:19:01.219475 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:01.219992 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 48495us
I20260709 13:19:01.220297 17831 consensus_queue.cc:1048] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:19:01.209470 11503 rpcz_store.cc:276] 0709 13:18:59.768467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.768576 (+   109us) service_pool.cc:224] Handling call
0709 13:19:01.167679 (+1399103us) spinlock_profiling.cc:243] Waited 270 ms on lock 0x7b54005f8a54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:01.167833 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":620094336}
I20260709 13:19:01.184522 17670 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:01.173604 17680 raft_consensus.cc:2749] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:01.174587 17744 raft_consensus.cc:2749] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:01.226543 10842 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:01.228889 10792 raft_consensus.cc:1275] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:19:01.197484 17210 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:01.231894 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.231906 17830 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:01.232455 10766 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:01.234686 11508 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:01.240600 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:01.222086 17562 raft_consensus.cc:2749] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:01.240628 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:01.257092 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:01.257591 10798 pending_rounds.cc:85] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 9
I20260709 13:19:01.257766 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:01.277590 11509 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
I20260709 13:19:01.277817 11504 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:01.278067 10800 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
I20260709 13:19:01.277750 10797 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:19:01.258297 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:01.279145 10794 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:19:01.277799 10798 pending_rounds.cc:107] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 19.10
I20260709 13:19:01.280711 10798 raft_consensus.cc:2889] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:01.258708 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:01.255663 10795 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 23.
I20260709 13:19:01.270043 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:01.271965 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:01.273224 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:01.273934 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:01.283495 10793 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:01.275197 10796 raft_consensus.cc:1240] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:01.275306 10916 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:01.238346 17731 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:01.241408 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:01.285573 11506 raft_consensus.cc:2393] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 18: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:01.289816 11510 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 22.
I20260709 13:19:01.268944 17826 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:01.283222 10801 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
W20260709 13:19:01.285878 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 59414.6us
W20260709 13:19:01.295742 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 63385.1us
I20260709 13:19:01.297043 10766 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:01.298099 10766 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:01.295831 10841 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:19:01.288014 11507 raft_consensus.cc:2393] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 20: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:01.307922 10766 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:01.211596 17619 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:01.259299 17820 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:01.232908 17793 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:01.288550 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 80829.6us
I20260709 13:19:01.202172 17617 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:01.260011 17728 raft_consensus.cc:2749] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:19:01.272650 17758 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.101s	user 0.002s	sys 0.011s
I20260709 13:19:01.316426 10841 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:19:01.276962 17670 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:01.236478 17327 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:01.277005 17753 raft_consensus.cc:697] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:01.318390 17680 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:01.321101 17744 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:01.313750 17819 raft_consensus.cc:2804] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:01.330319 17801 consensus_queue.cc:1059] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.086s
I20260709 13:19:01.292109 17210 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:01.321148 17820 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:01.313505 17571 consensus_queue.cc:1048] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.062s
I20260709 13:19:01.298993 17830 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:01.331758 10767 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:01.436663 10797 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
I20260709 13:19:01.339474 10793 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
W20260709 13:19:01.413316 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.417022 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:01.423589 11477 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:01.424690 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:01.470927 10801 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:01.472314 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:01.473678 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:01.474344 10942 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 16.
I20260709 13:19:01.475003 17776 raft_consensus.cc:2804] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:01.475512 17776 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:01.475934 17776 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:01.343135 10841 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:19:01.333453 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:19:01.479372 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 136342us
I20260709 13:19:01.480358 10841 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:19:01.332860 11504 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
I20260709 13:19:01.422220 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:01.482136 10944 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 20.
W20260709 13:19:01.471072 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 47567us
I20260709 13:19:01.483719 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:01.484566 17415 consensus_queue.cc:1059] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:19:01.485620 17777 raft_consensus.cc:3055] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:01.485562 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:01.486037 11504 raft_consensus.cc:1275] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:19:01.486153 17777 raft_consensus.cc:740] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:01.353408 17327 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:01.485075 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:01.488867 10947 raft_consensus.cc:1240] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:01.490298 10946 raft_consensus.cc:1240] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
W20260709 13:19:01.370193 17747 log.cc:927] Time spent T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 1.660s	user 0.001s	sys 0.003s
W20260709 13:19:01.449743 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 141920us
I20260709 13:19:01.449954 11508 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 16.
I20260709 13:19:01.450755 10941 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 18.
I20260709 13:19:01.437911 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:01.466185 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:19:01.360514 17822 log.cc:927] Time spent T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.150s	user 0.000s	sys 0.000s
W20260709 13:19:01.419324 17792 log.cc:927] Time spent T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.334s	user 0.003s	sys 0.000s
W20260709 13:19:01.335170 17758 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.166s	user 0.002s	sys 0.011s
I20260709 13:19:01.411100 17791 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:01.418118 17210 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:01.470553 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 195309us
I20260709 13:19:01.369494 17829 raft_consensus.cc:2804] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:19:01.516579 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.407627 17801 raft_consensus.cc:3055] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:01.344506 17826 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:01.331171 17832 raft_consensus.cc:2804] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:01.562763 10801 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 20.
W20260709 13:19:01.575063 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 77600.7us
I20260709 13:19:01.357661 17725 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:01.407027 17819 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:01.375767 17680 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:01.342680 17756 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:01.368588 17579 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:01.430128 10794 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
I20260709 13:19:01.588909 10944 raft_consensus.cc:2410] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 22 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:19:01.407785 17805 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:01.415251 17795 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:01.590008 10947 raft_consensus.cc:1240] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:01.300065 17808 consensus_queue.cc:237] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:01.467129 17771 log.cc:927] Time spent T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 13:19:01.353618 17821 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:01.497786 17760 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:01.631753 17791 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:01.354591 17670 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:01.494186 17762 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:19:01.576064 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 244372us
I20260709 13:19:01.623564 10766 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:01.498291 17777 consensus_queue.cc:260] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:01.629730 10948 raft_consensus.cc:1275] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:19:01.668572 10767 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:01.579772 17833 raft_consensus.cc:2764] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:19:01.630899 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:01.669586 10946 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:01.639041 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:01.671365 17787 consensus_meta.cc:287] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.535s	user 0.010s	sys 0.000s
I20260709 13:19:01.495076 17729 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:01.609800 17801 raft_consensus.cc:740] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:01.671842 17787 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:01.673046 17801 consensus_queue.cc:260] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:01.619270 17826 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:01.355504 17753 consensus_queue.cc:237] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 24.12, Last appended by leader: 12, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:01.674813 17787 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:01.675104 17826 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:01.336269 17793 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:01.629500 17758 raft_consensus.cc:3055] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:01.678965 17758 raft_consensus.cc:740] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:01.680048 17758 consensus_queue.cc:260] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:01.681275 17758 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:19:01.632143 17751 consensus_meta.cc:287] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.562s	user 0.008s	sys 0.002s
I20260709 13:19:01.618552 17832 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:01.656062 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:01.497586 17728 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:01.635424 17680 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:01.633684 17819 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:01.603366 17829 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:01.363250 17364 consensus_queue.cc:237] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:01.674698 17801 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:01.692857 17829 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:01.694392 17832 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:01.695901 17728 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:01.696290 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:01.696936 17793 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:01.704929 10946 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
W20260709 13:19:01.703190 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 96427.5us
I20260709 13:19:01.707060 10916 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:01.707708 17728 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:01.708027 17842 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:01.710316 17842 consensus_queue.cc:1059] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:01.711526 17282 raft_consensus.cc:3055] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:19:01.711971 17282 raft_consensus.cc:740] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:01.713871 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:01.713701 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 90215.1us
I20260709 13:19:01.714980 10946 raft_consensus.cc:2393] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 20: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:01.681427 17729 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:01.717898 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 49421.2us
I20260709 13:19:01.630143 17210 consensus_queue.cc:1059] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 13:19:01.720669 10767 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:01.653906 17831 consensus_queue.cc:1048] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.071s
I20260709 13:19:01.650149 17821 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:01.724920 10916 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:01.661164 17760 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:01.756907 10767 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:01.751561 10916 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:01.713692 17619 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:01.713274 17762 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:01.657620 17805 consensus_queue.cc:1048] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.094s
W20260709 13:19:01.750869 17603 consensus_meta.cc:287] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.609s	user 0.011s	sys 0.000s
I20260709 13:19:01.642325 17756 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:01.785780 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:01.789964 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 74329.7us
I20260709 13:19:01.789956 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:01.732649 17623 raft_consensus.cc:2804] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:19:01.777048 17564 consensus_meta.cc:287] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.713s	user 0.009s	sys 0.008s
W20260709 13:19:01.801049 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.774176 17760 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:01.807540 10801 raft_consensus.cc:1240] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:01.753656 17821 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:01.779343 17729 consensus_queue.cc:1048] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
W20260709 13:19:01.807881 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 51096.3us
I20260709 13:19:01.812304 10766 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:01.815237 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:01.782289 17762 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:01.781214 17834 raft_consensus.cc:2764] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:19:01.660053 17840 consensus_queue.cc:1048] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.132s
I20260709 13:19:01.779094 17386 raft_consensus.cc:3055] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:01.733348 17841 raft_consensus.cc:2749] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:19:01.791648 10793 raft_consensus.cc:1240] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:19:01.778936 17728 raft_consensus.cc:2749] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:01.714284 17791 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:01.806293 17792 log.cc:927] Time spent T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:19:01.855011 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:01.802076 17623 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:01.800697 17438 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:19:01.721911 17282 consensus_queue.cc:260] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 25.13, Last appended by leader: 13, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:01.810427 17794 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:01.809641 17617 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:01.811756 17838 raft_consensus.cc:2749] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:19:01.885390 10943 consensus_meta.cc:287] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.664s	user 0.012s	sys 0.000s
I20260709 13:19:01.905037 10801 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14"
dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
mode: GRACEFUL
new_leader_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:40148
I20260709 13:19:01.907474 10943 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 20.
I20260709 13:19:01.907636 10801 raft_consensus.cc:606] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Received request to transfer leadership to TS cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:01.908200 10801 raft_consensus.cc:614] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:01.909803 11505 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 23.
I20260709 13:19:01.872164 17744 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:01.869383 17603 raft_consensus.cc:2804] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:01.872951 17386 raft_consensus.cc:740] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:01.809069 17564 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:01.913050 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.917619 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:01.918448 10945 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
I20260709 13:19:01.922405 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:01.883826 17623 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:01.920771 10876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 1080) took 2130 ms. Trace:
I20260709 13:19:01.868958 17832 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:01.929297 10876 rpcz_store.cc:276] 0709 13:18:59.786146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.786279 (+   133us) service_pool.cc:224] Handling call
0709 13:18:59.798030 (+ 11751us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:59.829093 (+ 31063us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:18:59.929734 (+100641us) log.cc:844] Serialized 14 byte log entry
0709 13:19:00.455948 (+526214us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:00.456403 (+   455us) log.cc:844] Serialized 26 byte log entry
0709 13:19:00.456526 (+   123us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:19:00.463928 (+  7402us) log.cc:844] Serialized 14 byte log entry
0709 13:19:00.516428 (+ 52500us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:00.516504 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:00.990671 (+474167us) spinlock_profiling.cc:243] Waited 5.41 ms on lock 0x7b18003881e0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc8557 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:19:01.066024 (+ 75353us) spinlock_profiling.cc:243] Waited 4.01 ms on lock 0x7b18003881e0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7dd962a6 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc83f3 00007f9c7ddca7c1
0709 13:19:01.418559 (+352535us) spinlock_profiling.cc:243] Waited 7.21 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:19:01.916223 (+497664us) raft_consensus.cc:1685] finished
0709 13:19:01.916247 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:01.916377 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":187594,"prepare.queue_time_us":155178,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":71,"raft.queue_time_us":312427,"raft.run_cpu_time_us":374,"raft.run_wall_time_us":17681,"spinlock_wait_cycles":38268288,"thread_start_us":296205,"threads_started":1,"wal-append.queue_time_us":468116}
I20260709 13:19:01.909579 17438 raft_consensus.cc:697] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 23 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:01.825026 17402 consensus_queue.cc:260] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:01.917888 10767 connection.cc:256] RPC call timeout handler was delayed by -1.0204204209999261s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17320000000000002s for a response
I20260709 13:19:01.884455 17836 consensus_queue.cc:1059] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.314s
I20260709 13:19:01.936379 17725 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:01.925454 17603 raft_consensus.cc:697] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:01.978700 17611 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:01.981544 17851 log.cc:927] Time spent T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
W20260709 13:19:01.983680 17843 log.cc:927] Time spent T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.211s	user 0.000s	sys 0.001s
W20260709 13:19:01.985242 17837 log.cc:927] Time spent T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
I20260709 13:19:01.987174 10792 raft_consensus.cc:1240] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:01.987325 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:01.983032 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:01.962746 17282 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:19:01.988487 17402 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:02.009133 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:02.011966 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:19:02.017964 10767 leader_election.cc:336] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Timed out: RequestConsensusVote RPC to 127.9.116.130:37895 timed out after 1.732s (SENT)
I20260709 13:19:02.018626 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:02.023546 10798 raft_consensus.cc:2393] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 18: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:02.006136 17680 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:02.024287 17854 log.cc:927] Time spent T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.117s	user 0.001s	sys 0.001s
I20260709 13:19:02.026378 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:19:02.063211 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.033186 17832 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:02.034899 17839 log.cc:927] Time spent T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.254s	user 0.000s	sys 0.001s
I20260709 13:19:02.037230 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:02.064873 10793 raft_consensus.cc:2410] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 22 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 13:19:02.065939 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:02.039845 17756 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:02.042053 11477 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:19:02.043229 17845 log.cc:927] Time spent T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 13:19:02.044636 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 21 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:19:02.045538 10869 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 2622) took 1633 ms (client timeout 1732 ms). Trace:
I20260709 13:19:02.046310 10868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 2626) took 1633 ms. Trace:
I20260709 13:19:02.073060 10868 rpcz_store.cc:276] 0709 13:19:00.413095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:00.413198 (+   103us) service_pool.cc:224] Handling call
0709 13:19:02.046110 (+1632912us) spinlock_profiling.cc:243] Waited 637 ms on lock 0x7b540037b254. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:02.046256 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1465509248}
I20260709 13:19:02.047235 10875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 2628) took 1633 ms. Trace:
I20260709 13:19:02.047446 17791 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:02.074272 10875 rpcz_store.cc:276] 0709 13:19:00.413668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:00.453360 (+ 39692us) service_pool.cc:224] Handling call
0709 13:19:02.047036 (+1593676us) spinlock_profiling.cc:243] Waited 602 ms on lock 0x7b5400077954. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:02.047167 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1385045888}
W20260709 13:19:02.075026 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.048432 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:19:02.018306 17731 consensus_meta.cc:287] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.704s	user 0.007s	sys 0.004s
I20260709 13:19:02.009689 17670 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:02.072786 10793 raft_consensus.cc:2393] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 16: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
W20260709 13:19:02.072317 10869 rpcz_store.cc:269] 0709 13:19:00.411870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:00.414331 (+  2461us) service_pool.cc:224] Handling call
0709 13:19:02.045313 (+1630982us) spinlock_profiling.cc:243] Waited 484 ms on lock 0x7b5400098dd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:02.045454 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1112754048}
I20260709 13:19:02.077071 17611 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:02.080204 17611 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:02.080435 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:02.081558 17801 consensus_queue.cc:1059] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:19:02.083655 17859 raft_consensus.cc:3055] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:02.084100 17859 raft_consensus.cc:740] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:02.084977 17859 consensus_queue.cc:260] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:02.086308 17859 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:02.087819 17721 consensus_meta.cc:287] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.846s	user 0.011s	sys 0.000s
I20260709 13:19:02.088279 17721 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:02.090973 10942 raft_consensus.cc:1240] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:02.093637 10942 raft_consensus.cc:1240] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 13:19:02.096807 17830 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:02.098075 17564 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:02.056164 17799 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:02.099166 17799 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:02.101450 17799 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:02.101889 17589 consensus_meta.cc:287] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 1.026s	user 0.008s	sys 0.001s
I20260709 13:19:02.103858 10872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49290 (request call id 1089) took 1283 ms. Trace:
I20260709 13:19:02.104298 10872 rpcz_store.cc:276] 0709 13:19:00.820203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:00.932888 (+112685us) service_pool.cc:224] Handling call
0709 13:19:02.103646 (+1170758us) spinlock_profiling.cc:243] Waited 772 ms on lock 0x7b5400078854. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:02.103795 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1776623488}
I20260709 13:19:02.105738 17819 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:02.108318 17411 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:19:02.110471 17776 consensus_meta.cc:287] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.634s	user 0.005s	sys 0.002s
I20260709 13:19:02.077503 17438 consensus_queue.cc:237] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:02.090504 17721 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:01.986586 17386 consensus_queue.cc:260] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:02.080612 17611 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:02.111744 11505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56404 (request call id 1143) took 1354 ms. Trace:
W20260709 13:19:02.116577 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.121120 11505 rpcz_store.cc:276] 0709 13:19:00.757643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:00.775897 (+ 18254us) service_pool.cc:224] Handling call
0709 13:19:02.111520 (+1335623us) spinlock_profiling.cc:243] Waited 557 ms on lock 0x7b540055e6d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:02.111666 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1280856832}
I20260709 13:19:02.062698 17729 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.072108 17788 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.113078 17579 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:19:02.136566 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.144856 10942 raft_consensus.cc:1240] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:02.137037 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:02.072515 17829 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:02.086544 17826 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:02.113409 17847 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.150929 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:02.160001 10798 raft_consensus.cc:2393] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 18: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:02.159816 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:02.144846 17386 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:02.136027 17832 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:19:02.160391 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 103534us
I20260709 13:19:02.117640 17731 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:02.146186 17579 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:02.163002 10766 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:02.121793 17776 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:02.142235 17794 raft_consensus.cc:697] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 20 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:02.165033 17282 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.205017 17282 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:02.207252 17282 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:02.210255 10794 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 21.
I20260709 13:19:02.211615 17794 consensus_queue.cc:237] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:02.154989 17805 consensus_queue.cc:1048] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:19:02.200165 17857 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:02.213505 17857 raft_consensus.cc:697] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:02.214398 17857 consensus_queue.cc:237] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:02.196355 17670 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.203819  9682 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:19:02.165594 17847 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:02.212630 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:02.141433 17729 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:02.202323 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:19:02.212785 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.223196 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:02.191095 17721 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.226224 10801 raft_consensus.cc:2393] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 19: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:02.230741 17721 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:02.231772 17840 consensus_queue.cc:1059] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:19:02.232868 17840 raft_consensus.cc:3055] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:02.233101 17721 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:02.233392 17731 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:02.194222 17858 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:02.233230 17840 raft_consensus.cc:740] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:02.145467 17603 consensus_queue.cc:237] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:02.244828 10948 raft_consensus.cc:2410] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:19:02.146574 17788 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:02.194958 17832 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:02.255752 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:02.257519 10798 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 20.
I20260709 13:19:02.226347 17670 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:02.271620 10916 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:19:02.259333 17820 consensus_meta.cc:287] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.762s	user 0.012s	sys 0.000s
I20260709 13:19:02.280083 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:02.281193 10798 raft_consensus.cc:2368] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for earlier term 18. Current term is 19.
W20260709 13:19:02.284893 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:02.287475 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 1.10939e+06us
I20260709 13:19:02.269855 17808 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.293083 10915 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:02.290591 10766 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:02.294349 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:02.295583 10766 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:02.296460 10942 raft_consensus.cc:1275] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 9. (term mismatch)
I20260709 13:19:02.296991 10942 pending_rounds.cc:85] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 8
I20260709 13:19:02.297309 10942 pending_rounds.cc:107] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 17.9
I20260709 13:19:02.297609 10942 raft_consensus.cc:2889] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:19:02.299117 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:02.299827 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.247268 17858 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:02.306155 11505 raft_consensus.cc:1240] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:19:02.278312 17438 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:02.307443 10766 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:02.308367 17805 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:19:02.309937 17829 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:02.310497 17776 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:02.311110 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:02.311728 17821 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:02.312314 17788 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:02.313067 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:02.313838 17821 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:02.313956 11509 raft_consensus.cc:2410] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:19:02.304112 17808 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:02.315086 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:02.316726 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:02.314388 17858 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:02.317768 10946 raft_consensus.cc:1240] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:02.287228 17680 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:02.320462 10842 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:02.278029 17820 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:02.307857 17833 raft_consensus.cc:2749] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:02.326185 10842 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:02.328534 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:02.329099 10841 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:02.329649 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:02.331833 10942 raft_consensus.cc:2393] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 17: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:02.334024 17826 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:02.335458 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.336758 17847 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:02.337344 17670 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:02.337350 17756 consensus_queue.cc:1059] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:02.339046 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:02.339716 10945 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:02.303146 17411 consensus_queue.cc:1048] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.028s
W20260709 13:19:02.344254 17850 log.cc:927] Time spent T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.356s	user 0.002s	sys 0.000s
I20260709 13:19:02.344471 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:02.345333 10794 raft_consensus.cc:2393] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 18: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:02.321136 17210 raft_consensus.cc:2804] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:02.346115 17210 raft_consensus.cc:697] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:02.346351 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:02.375625 17670 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:02.377897 11505 raft_consensus.cc:1275] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:19:02.317047 17819 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:02.375020 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 48874.6us
I20260709 13:19:02.423300 10842 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:02.367372 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:02.424813 10792 raft_consensus.cc:2393] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 17: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:02.370406 17451 consensus_queue.cc:260] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:02.383129  9682 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:19:02.322896 17808 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:02.358958 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:02.362073 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:02.384627 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:02.347394 17210 consensus_queue.cc:237] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:02.380512 17611 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:02.404330 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.330399 17536 raft_consensus.cc:2804] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:02.382181 17729 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:02.383955 17758 consensus_meta.cc:287] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.702s	user 0.013s	sys 0.000s
I20260709 13:19:02.308985 17438 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:02.297084 17840 consensus_queue.cc:260] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:02.369623 17820 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:02.332916 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:02.391127 17832 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:02.355015 10916 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:02.370141 10797 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:02.376468 17498 raft_consensus.cc:3055] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:02.441269 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:02.441604 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:02.441943 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:02.441414 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 86480.5us
I20260709 13:19:02.446029 10941 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 18.
I20260709 13:19:02.389451 17623 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:02.447201 17498 raft_consensus.cc:740] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:02.447252 10949 raft_consensus.cc:2393] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 19: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:02.448057 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:02.449713 10795 raft_consensus.cc:2393] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 20: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:19:02.451117 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.415589 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:02.360630 10874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49290 (request call id 1086) took 1541 ms. Trace:
I20260709 13:19:02.451699 10874 rpcz_store.cc:276] 0709 13:19:00.818881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:00.819913 (+  1032us) service_pool.cc:224] Handling call
0709 13:19:02.360353 (+1540440us) spinlock_profiling.cc:243] Waited 678 ms on lock 0x7b5400076054. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:02.360551 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1558920448}
I20260709 13:19:02.416489 10794 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 22.
W20260709 13:19:02.386547 17762 consensus_meta.cc:287] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.520s	user 0.006s	sys 0.003s
W20260709 13:19:02.417124 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.319943 17282 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:02.425879 10842 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:02.428952 10945 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
I20260709 13:19:02.423074 17821 raft_consensus.cc:2804] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:02.422883 17670 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:02.429643 17788 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:02.454241 10942 raft_consensus.cc:2368] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 16. Current term is 17.
W20260709 13:19:02.455832 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.455891 10842 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:19:02.414976 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 85946.9us
I20260709 13:19:02.462673 10797 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 17.
I20260709 13:19:02.464423 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:02.466252 10766 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:02.470537 10793 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 18.
I20260709 13:19:02.471274 10796 raft_consensus.cc:1240] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 22. Current term is 23. Ops: [20.14-22.15]
I20260709 13:19:02.473116 10796 raft_consensus.cc:1275] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 13:19:02.475276 10801 raft_consensus.cc:1240] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:02.475714 10841 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:02.477129 10801 raft_consensus.cc:1275] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 22 index: 10. (index mismatch)
I20260709 13:19:02.440552 17438 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:02.483716 10798 raft_consensus.cc:2393] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 17: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:02.441323 17536 raft_consensus.cc:697] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:02.454823 17451 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:02.485329 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:02.473580 10796 pending_rounds.cc:85] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 12
I20260709 13:19:02.492965 10796 pending_rounds.cc:107] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 13:19:02.493268 10796 raft_consensus.cc:2889] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:02.452278 17729 raft_consensus.cc:2804] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:02.498697 10793 raft_consensus.cc:1275] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 15. (term mismatch)
I20260709 13:19:02.499228 10793 pending_rounds.cc:85] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 14
I20260709 13:19:02.499531 10793 pending_rounds.cc:107] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 21.15
I20260709 13:19:02.499828 10793 raft_consensus.cc:2889] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:02.502362 10872 raft_consensus.cc:1240] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:02.497709 17562 raft_consensus.cc:2804] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:02.506577 17833 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:02.469827 17808 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.469484 17819 raft_consensus.cc:2804] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:02.481482 17670 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:02.513746 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:02.487865 17840 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:02.507174 17833 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:02.518348 17833 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:02.519107 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 21 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:02.519801 10800 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:02.519731 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:02.520720 10949 raft_consensus.cc:2393] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 19: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:02.512687 17587 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:02.523998 17587 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:02.525995 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:02.527068 10872 raft_consensus.cc:2410] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:19:02.465488 17762 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:02.529985 17762 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:02.539390 17756 consensus_queue.cc:1059] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:19:02.543946 17579 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:02.545356 10942 raft_consensus.cc:1240] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 22. Current term is 23. Ops: []
W20260709 13:19:02.545713 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.548120 17623 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:02.548897 17819 raft_consensus.cc:697] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:02.550148 17819 consensus_queue.cc:237] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:02.551218 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:02.552070 10942 raft_consensus.cc:2393] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 17: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:02.553794 17587 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:02.488479 17603 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:02.557524 10799 raft_consensus.cc:2410] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:19:02.492751 17821 raft_consensus.cc:697] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:02.558967 17821 consensus_queue.cc:237] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:02.564158 17611 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:02.491566 17498 consensus_queue.cc:260] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:02.567420 17758 raft_consensus.cc:3055] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:19:02.567830 17758 raft_consensus.cc:740] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:02.488560 17858 consensus_queue.cc:1048] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.083s
I20260709 13:19:02.568650 17758 consensus_queue.cc:260] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:02.494648 17729 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:02.569995 17729 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:02.571254 17498 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:02.503772 17791 raft_consensus.cc:2749] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:02.578352 10943 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
I20260709 13:19:02.580111 17579 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:02.582108 17536 consensus_queue.cc:237] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:02.586687 17760 consensus_meta.cc:287] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.755s	user 0.012s	sys 0.000s
I20260709 13:19:02.587545 10800 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 23.
I20260709 13:19:02.587361 17760 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:02.590173 17760 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:02.594825 10870 raft_consensus.cc:1240] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:02.595255 10869 raft_consensus.cc:1240] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 25. Current term is 26. Ops: [23.12-25.13]
I20260709 13:19:02.596330 17808 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:02.598507 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 88174.3us
I20260709 13:19:02.610152 11478 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:02.600324 10868 raft_consensus.cc:2393] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 24: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:02.617323 11478 leader_election.cc:400] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 election: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for earlier term 18. Current term is 19.
I20260709 13:19:02.617933 11478 leader_election.cc:403] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:19:02.619060 10793 raft_consensus.cc:1275] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:19:02.623004 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:02.623847 10874 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:02.625629 10875 raft_consensus.cc:1240] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 23. Current term is 24. Ops: [23.17-23.17]
I20260709 13:19:02.510077 17776 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:02.628376 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:02.630918 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:02.632355 10867 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 18.
I20260709 13:19:02.632966 10873 raft_consensus.cc:2393] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 21: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:02.599545 17603 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:02.480875 17847 consensus_queue.cc:1059] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.131s
I20260709 13:19:02.636606 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:02.626502 17611 raft_consensus.cc:2749] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:02.648432 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:02.630666 17739 raft_consensus.cc:2749] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for earlier term 18. Current term is 19.
I20260709 13:19:02.637996 17603 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:02.652422 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:02.631379 11478 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:02.654121 10873 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
I20260709 13:19:02.656504 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:02.663021 11512 raft_consensus.cc:1275] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:19:02.663214 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:02.655860 17587 raft_consensus.cc:2749] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:19:02.664191 11509 raft_consensus.cc:2410] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:19:02.664908 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:02.671563 11505 raft_consensus.cc:1275] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:19:02.672798 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:02.673520 11506 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 18.
I20260709 13:19:02.673015 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:02.675868 10793 raft_consensus.cc:2410] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:19:02.678759 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:02.679162 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:19:02.695238 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.638240 17776 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:02.703445 11506 raft_consensus.cc:1240] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:19:02.704981 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:02.714635 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:19:02.715034 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.717691 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:02.718989 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:02.720369 11507 raft_consensus.cc:2393] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 21: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:02.721873 10874 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 19.
I20260709 13:19:02.725800 10874 raft_consensus.cc:1275] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:19:02.726286 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:02.731127 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:02.757887 11505 raft_consensus.cc:2393] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 18: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:02.731021 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:02.758868 11509 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:02.732679 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:02.696508 17562 raft_consensus.cc:697] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:02.707072 17788 raft_consensus.cc:697] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:02.765409 10873 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 18.
I20260709 13:19:02.743037 11508 raft_consensus.cc:1240] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 23. Current term is 24. Ops: []
I20260709 13:19:02.742398 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:02.702479 17729 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:02.745576 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.746718 11504 raft_consensus.cc:1240] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:19:02.745666 17840 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.751566 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:19:02.752007 10686 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 2b82f18992434ed5ba58b805107060d4 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 468a8fd46e824f62b9c2c0d3532a6d26 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 20 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:02.752614 17808 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:02.756098 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.797235 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:02.765471 17760 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.763899 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:19:02.768175 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.768460 11510 raft_consensus.cc:1275] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 13:19:02.742642 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 143142us
I20260709 13:19:02.769243 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:02.804653 11505 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:02.776235 10870 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 18.
W20260709 13:19:02.780596 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.793138 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:02.807345 10873 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:02.808080 11478 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:02.796144 10686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 44183.2us
I20260709 13:19:02.797572 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:02.797961 10874 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 21.
I20260709 13:19:02.756690 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:02.810447 11508 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 17.
I20260709 13:19:02.810918 11503 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:02.798240 11511 raft_consensus.cc:2393] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 19: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:02.798616 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:02.799762 11504 raft_consensus.cc:1275] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:19:02.797134 10868 raft_consensus.cc:1240] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:02.765578 17763 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:02.813061 11509 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 19.
W20260709 13:19:02.817551 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:02.857782 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 49804.5us
I20260709 13:19:02.860152 11478 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:02.812299 11506 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 17.
I20260709 13:19:02.869292 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:02.869925 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:02.869975 10870 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:19:02.871670 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 363914us
I20260709 13:19:02.873803 10873 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 17.
I20260709 13:19:02.875288 11478 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:02.875792 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:02.883461 10916 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:02.862737 17587 raft_consensus.cc:2764] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:19:02.899817 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:02.900720 11508 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
I20260709 13:19:02.867596 17808 raft_consensus.cc:2749] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:19:02.904023 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:02.904922 10793 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 18.
I20260709 13:19:02.905692 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:02.892799 10870 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 20.
I20260709 13:19:02.914367 11505 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 21.
I20260709 13:19:02.894057 11478 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:02.893201 10916 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:02.802431 17840 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:02.882424 17739 raft_consensus.cc:2749] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:19:02.917629 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:02.921594 11511 raft_consensus.cc:2368] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 20. Current term is 21.
I20260709 13:19:02.919255 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:02.923230 11506 raft_consensus.cc:2393] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 19: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:02.923722 10916 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:02.928584 17729 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:02.928670 10950 raft_consensus.cc:1275] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:19:02.886417 17857 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.935119 17857 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:02.936779 17887 log.cc:927] Time spent T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:19:02.937469 17857 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:02.940007 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:02.903041 17451 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:19:02.948745 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.949790 10842 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:02.816890 17760 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:02.886125 17847 consensus_queue.cc:1048] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.053s
I20260709 13:19:02.945436 10793 raft_consensus.cc:2393] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 19: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:02.915184 17680 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.882751 17536 consensus_queue.cc:1059] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.071s
I20260709 13:19:02.908761 17579 consensus_queue.cc:1059] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.050s
I20260709 13:19:02.902767 17791 consensus_queue.cc:1059] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.049s
I20260709 13:19:02.903265 17766 consensus_queue.cc:1048] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 13:19:02.888192 17763 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:02.957007 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 63072.5us
I20260709 13:19:02.986977 11478 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:02.958001 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:02.958969 11508 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:02.886552 17801 consensus_queue.cc:1048] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.057s
I20260709 13:19:02.956693 10916 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:02.938256 17722 raft_consensus.cc:2749] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:02.908030 17788 consensus_queue.cc:237] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:02.976657 17729 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:02.975961 10874 raft_consensus.cc:2393] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 17: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:02.956208 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:02.979146 17611 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.880996 17562 consensus_queue.cc:237] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:02.945082 11503 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 22.
I20260709 13:19:02.960602 17587 raft_consensus.cc:2749] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:02.988119 10950 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 20.
I20260709 13:19:02.947008 17858 consensus_queue.cc:1048] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.037s
I20260709 13:19:02.966444 10842 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:03.045785 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 79478.3us
I20260709 13:19:03.046870 10842 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:03.048411 10842 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:02.997017 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:03.003221 10869 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:02.993810 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:03.008533 11478 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:03.052627 10793 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:19:03.059588 10870 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 16.
I20260709 13:19:02.944242 17877 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:02.996847 17880 log.cc:927] Time spent T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 13:19:03.069092 17603 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:03.032032 11508 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 23.
I20260709 13:19:03.008164 17791 consensus_queue.cc:1243] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer cb0428cc55984daba1ba63744a5102a2 log is divergent from this leader: its last log entry 19.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 13:19:02.999586 17670 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:03.003062 17763 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:03.079254 17763 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:03.080062 17763 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:03.080644 17763 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:02.976400 17680 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:02.961987 17438 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:03.001070 17801 raft_consensus.cc:2804] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:03.004649 17840 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:03.012179 17729 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:19:03.094916 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:03.095268 17729 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:03.064924 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:03.097815 10870 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:03.011756 17739 raft_consensus.cc:2749] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:19:03.099572 17739 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:03.100044 17739 raft_consensus.cc:697] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 19 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:03.039919 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:03.100843 10872 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:02.978334 17879 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:19:03.024160 11509 raft_consensus.cc:1240] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:03.028767 17611 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:03.076260 17791 raft_consensus.cc:2749] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:03.077117 10871 raft_consensus.cc:1240] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:03.108897 10944 raft_consensus.cc:1240] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:03.114880 17611 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:03.078477 17670 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:03.103806 17801 raft_consensus.cc:697] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 17 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:03.056948 17847 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:03.057281 17498 raft_consensus.cc:2749] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:03.101688 10799 raft_consensus.cc:2393] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 19: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:03.116290 17847 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:03.116567 17801 consensus_queue.cc:237] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:03.118423 17847 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:03.119153 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:03.119808 10950 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:03.124020 17760 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:03.126121 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:03.128151 10793 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
I20260709 13:19:03.129047 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:03.105266 17739 consensus_queue.cc:237] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:03.106145 17890 log.cc:927] Time spent T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 13:19:03.106345 17879 raft_consensus.cc:697] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:03.107754 17438 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:03.107808 17603 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:03.070118 10801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2482) took 1047 ms. Trace:
I20260709 13:19:03.074183 17877 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:03.134104 10872 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 17.
W20260709 13:19:03.146621 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:03.137354 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 180741us
I20260709 13:19:03.160467 10801 rpcz_store.cc:276] 0709 13:19:02.022725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:02.479714 (+456989us) service_pool.cc:224] Handling call
0709 13:19:02.480030 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:02.482358 (+  2328us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:02.482388 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:02.500853 (+ 18465us) log.cc:844] Serialized 26 byte log entry
0709 13:19:02.735824 (+234971us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:02.735884 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:02.735977 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:03.069858 (+333881us) raft_consensus.cc:1685] finished
0709 13:19:03.069881 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:03.070053 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1942,"raft.queue_time_us":4752,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":11895,"thread_start_us":159621,"threads_started":1,"wal-append.queue_time_us":191912}
W20260709 13:19:03.162653 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:03.164702 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 116391us
I20260709 13:19:03.170902 10842 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:03.172008 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:03.174564 10842 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:03.146337 10870 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 19.
I20260709 13:19:03.183295 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:03.185555 10867 raft_consensus.cc:1813] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Handling vote request from an unknown peer cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:03.184448 10801 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 21.
I20260709 13:19:03.183605 10842 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:03.196269 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:03.199306 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:03.188086 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:03.202265 10872 raft_consensus.cc:2410] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 22 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 13:19:03.202579 10950 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 19.
I20260709 13:19:03.174118 17763 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:03.203688 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:03.200002 10801 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 17.
I20260709 13:19:03.205690 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:03.201987 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 193564us
I20260709 13:19:03.205576 10841 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:03.207439 17760 consensus_queue.cc:1048] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.002s
I20260709 13:19:03.209671 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:03.210464 17680 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:03.211817 10766 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:03.223863 17603 raft_consensus.cc:2749] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:03.214870 17879 consensus_queue.cc:237] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 21.9, Last appended by leader: 9, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:03.218174 17877 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:03.187129 17670 raft_consensus.cc:2749] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:03.227996 17603 consensus_queue.cc:1048] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:19:03.228708 10766 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:03.229557 17603 consensus_queue.cc:1048] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:19:03.229761 17210 raft_consensus.cc:2804] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:03.230976 17210 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:03.231387 17210 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:03.232864 17729 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:03.235245 17729 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:03.181690 17894 raft_consensus.cc:2749] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:03.246021 10795 raft_consensus.cc:1275] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:19:03.245934 10946 raft_consensus.cc:3055] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:19:03.250288 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:03.250375 10946 raft_consensus.cc:740] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:03.251196 10793 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
W20260709 13:19:03.256633 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 51115.4us
I20260709 13:19:03.260335 10873 raft_consensus.cc:2368] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 16. Current term is 17.
I20260709 13:19:03.270432 10869 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 19.
I20260709 13:19:03.267269 10916 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:03.175639 17840 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:03.285841 10842 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:19:03.286175 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 57552.2us
I20260709 13:19:03.246734 17897 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:03.206511 17788 raft_consensus.cc:2804] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:03.287268 17897 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:03.288085 17840 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:03.248272 17438 consensus_queue.cc:1059] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.044s
I20260709 13:19:03.296322 10766 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:03.297055 10841 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:03.285804 10794 raft_consensus.cc:1275] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:19:03.292979 10795 raft_consensus.cc:1275] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 26 index: 13. (index mismatch)
I20260709 13:19:03.298831 10916 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:03.251695 10946 consensus_queue.cc:260] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:03.293354 17788 raft_consensus.cc:697] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:03.289402 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:03.293658 17847 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:03.245985 17451 raft_consensus.cc:2749] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:03.305578 10941 raft_consensus.cc:1240] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 20 LEADER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:03.299576 10766 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:03.284408 17760 raft_consensus.cc:2749] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:19:03.298439 17857 raft_consensus.cc:2749] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:19:03.300458 10841 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:19:03.303120 10916 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:19:03.301898 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 53980.6us
I20260709 13:19:03.307058 10794 raft_consensus.cc:1275] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:19:03.298913 17894 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:03.292579 17840 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:03.330281 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:03.298642 17877 raft_consensus.cc:2749] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:19:03.333943 10841 leader_election.cc:400] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 election: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 20. Current term is 21.
I20260709 13:19:03.336627 10795 raft_consensus.cc:1275] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:19:03.339982 10841 leader_election.cc:403] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:19:03.306562 17847 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:03.368391 17847 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:03.375597 17896 raft_consensus.cc:2804] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:03.376449 17896 raft_consensus.cc:697] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 19 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:19:03.377085 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:03.377955 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:03.377676 17896 consensus_queue.cc:237] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:03.380484 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:03.305784 17438 raft_consensus.cc:2804] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:03.381107 10950 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:03.338936 10916 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:03.334168 17411 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:03.334909 17894 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:03.340826 17794 consensus_queue.cc:1048] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:19:03.389336 17894 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:03.341287 17801 consensus_queue.cc:1048] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:19:03.341926 10915 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:03.391894 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 50065.2us
I20260709 13:19:03.393224 10915 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:03.343420 17897 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:03.344089 10794 raft_consensus.cc:1240] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 22. Current term is 23. Ops: [20.9-22.10]
I20260709 13:19:03.355202 17777 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:03.357144 17758 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:03.358623 17856 raft_consensus.cc:3055] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:03.367048 17788 consensus_queue.cc:237] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:03.378778 17861 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:03.379684 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:03.379909 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 46842.3us
I20260709 13:19:03.422745 10943 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
I20260709 13:19:03.381145 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:03.337838 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:03.423089 10841 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:03.378580 10868 raft_consensus.cc:2376] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 18. Re-sending same reply.
W20260709 13:19:03.383800 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 44938.3us
I20260709 13:19:03.421178 10950 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
I20260709 13:19:03.301630 17680 consensus_queue.cc:1048] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.035s
I20260709 13:19:03.382685 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:03.386739 10947 raft_consensus.cc:1240] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:03.387298 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:03.429935 10942 raft_consensus.cc:2393] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 20: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:03.398564 17794 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:03.403846 10792 raft_consensus.cc:1240] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:03.402540 10798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2490) took 1122 ms. Trace:
I20260709 13:19:03.432845 10798 rpcz_store.cc:276] 0709 13:19:02.280226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:02.281954 (+  1728us) service_pool.cc:224] Handling call
0709 13:19:03.402306 (+1120352us) spinlock_profiling.cc:243] Waited 185 ms on lock 0x7b5400212ad4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:03.402469 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":426261888}
I20260709 13:19:03.376547 17579 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:03.433858 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:19:03.427961 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:03.432292 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:03.383955 17670 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:03.404333 17897 raft_consensus.cc:2804] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:03.391937 17729 raft_consensus.cc:2749] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:03.451017 11508 raft_consensus.cc:1275] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:19:03.404793 17411 raft_consensus.cc:697] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:03.458958 10916 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:03.305552 17210 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:03.460682 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 66368us
I20260709 13:19:03.461412 10915 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:03.461537 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:03.418184 17856 raft_consensus.cc:740] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:03.465256 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:03.467156 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:03.467545 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:03.467893 10943 raft_consensus.cc:2393] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 19: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:03.468784 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:03.468832 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:03.470118 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:03.470419 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 162247us
I20260709 13:19:03.437994 17762 raft_consensus.cc:2749] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:03.470747 10943 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
I20260709 13:19:03.456833 17622 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:03.474372 11508 raft_consensus.cc:1275] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:19:03.456447 17794 raft_consensus.cc:697] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 21 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:03.475827 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:03.478390 10872 raft_consensus.cc:2393] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 17: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:03.452577 17897 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:03.480726 11507 raft_consensus.cc:1240] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 13:19:03.484786 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:03.485713 11508 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 20.
I20260709 13:19:03.487629 11506 raft_consensus.cc:1275] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:19:03.488051 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:03.488708 11507 raft_consensus.cc:1240] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 13:19:03.440610 17861 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:03.454154 17579 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:03.492017 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:03.492815 10873 raft_consensus.cc:2376] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 18. Re-sending same reply.
I20260709 13:19:03.494999 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:03.495959 10872 raft_consensus.cc:2410] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 21 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 13:19:03.496722 17840 consensus_queue.cc:1059] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:19:03.487326 17729 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:03.507628 10944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 3107) took 1208 ms. Trace:
I20260709 13:19:03.508137 10944 rpcz_store.cc:276] 0709 13:19:02.299650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:02.434449 (+134799us) service_pool.cc:224] Handling call
0709 13:19:02.451822 (+ 17373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:03.109273 (+657451us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:03.507355 (+398082us) spinlock_profiling.cc:243] Waited 657 ms on lock 0x7b54003304d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:19:03.507562 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1510707072}
I20260709 13:19:03.509838 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:03.513048 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:03.514276 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:03.514971 10944 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
I20260709 13:19:03.515532 10798 raft_consensus.cc:1275] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:19:03.473567 17438 raft_consensus.cc:697] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 17 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:19:03.477289 17819 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.876s	user 0.016s	sys 0.018s
I20260709 13:19:03.517009 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:03.525620 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:03.525620 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:03.525739 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:03.529292 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:03.527114 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:03.530057 10795 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 21.
I20260709 13:19:03.479305 17801 raft_consensus.cc:2804] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:03.487074 17897 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:03.542184 17856 consensus_queue.cc:260] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:03.480417 17622 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:03.526285 17729 raft_consensus.cc:697] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:03.526561 10794 raft_consensus.cc:1275] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:19:03.547086 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:03.547533 10801 raft_consensus.cc:1275] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:19:03.547931 10872 raft_consensus.cc:2410] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 21 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:19:03.532135 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:03.537989 17801 raft_consensus.cc:697] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 18 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:19:03.549489 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 112072us
I20260709 13:19:03.550350 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:03.551184 10798 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
I20260709 13:19:03.553179 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:19:03.531217 17819 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.977s	user 0.017s	sys 0.019s
I20260709 13:19:03.555289 10948 raft_consensus.cc:1275] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:19:03.552316 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:03.543708 17856 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:19:03.556553 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 95207.8us
W20260709 13:19:03.558413 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 64721.9us
I20260709 13:19:03.560900 10766 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:03.561273 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:03.562693 10795 raft_consensus.cc:1240] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 13:19:03.563118 10841 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:03.567040 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:03.568924 11506 raft_consensus.cc:1275] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:19:03.572335 11506 raft_consensus.cc:3055] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:03.572767 11506 raft_consensus.cc:740] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:03.574720 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:03.575376 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:03.576023 10798 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 18.
I20260709 13:19:03.575399 11507 raft_consensus.cc:2393] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 21: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:03.577524 17917 raft_consensus.cc:3055] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:19:03.577996 17917 raft_consensus.cc:740] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:03.578480 17919 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:03.579027 17917 consensus_queue.cc:260] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:03.579492 17911 consensus_queue.cc:1048] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:19:03.580348 17917 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:03.545912 17210 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:03.573638 11506 consensus_queue.cc:260] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:03.606127 11506 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:03.608625 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:03.614341 10944 raft_consensus.cc:2376] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20. Re-sending same reply.
I20260709 13:19:03.611788 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:03.617307 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:03.616267 10798 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 26.
I20260709 13:19:03.621520 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:03.621891 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:03.622540 10944 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 19.
I20260709 13:19:03.595522 17670 raft_consensus.cc:2749] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 20. Current term is 21.
I20260709 13:19:03.633147 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:03.639840 10872 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 18.
I20260709 13:19:03.623610 10801 raft_consensus.cc:1240] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:03.579037 17919 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:03.627911 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:03.642398 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:03.611402 17210 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:03.643062 10874 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:03.629601 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:03.643743 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:03.647190 10869 raft_consensus.cc:2410] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 19 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:19:03.642333 10799 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 21.
I20260709 13:19:03.592574 17840 consensus_queue.cc:1048] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.030s
W20260709 13:19:03.616953 17907 log.cc:927] Time spent T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 13:19:03.650110 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:03.668470 10874 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 23.
I20260709 13:19:03.650518 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:03.644754 17916 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:03.656198 17919 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:03.643605 17883 consensus_queue.cc:1048] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.015s
I20260709 13:19:03.643198 17763 raft_consensus.cc:2804] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:03.582063 17794 consensus_queue.cc:237] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:03.659252 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:03.659952 17622 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:03.663525 17579 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:03.646154 17210 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:03.664644 17898 log.cc:927] Time spent T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.325s	user 0.001s	sys 0.006s
I20260709 13:19:03.665943 10875 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 22.
I20260709 13:19:03.582963 17766 consensus_queue.cc:1059] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.061s
I20260709 13:19:03.667904 10944 raft_consensus.cc:1275] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:19:03.650297 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:03.669350 10869 raft_consensus.cc:2376] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 19. Re-sending same reply.
W20260709 13:19:03.685312 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:03.683691 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 122887us
I20260709 13:19:03.704276 10872 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:03.705415 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:03.690364 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:03.706220 10869 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 20.
I20260709 13:19:03.708877 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:03.711601 10766 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:03.575646 17411 consensus_queue.cc:237] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:03.651103 17327 consensus_queue.cc:1059] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 13:19:03.731006 10872 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 22.
I20260709 13:19:03.731945 10766 leader_election.cc:400] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 election: Vote denied by peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553) with higher term. Message: Invalid argument: T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 16. Current term is 17.
I20260709 13:19:03.732023 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:03.732627 10766 leader_election.cc:403] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:19:03.732908 10872 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 17.
W20260709 13:19:03.675771 17885 log.cc:927] Time spent T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:19:03.696821 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:03.713171 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:03.693182 17916 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:03.735358 17919 raft_consensus.cc:2749] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553) with higher term. Message: Invalid argument: T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 16. Current term is 17.
I20260709 13:19:03.740924 17916 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:03.735390 17622 raft_consensus.cc:2749] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:03.742548 10867 raft_consensus.cc:1240] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:03.742888 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:03.743917 10868 raft_consensus.cc:2393] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 19: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:03.744185 10867 raft_consensus.cc:1240] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 22. Current term is 23. Ops: []
W20260709 13:19:03.745337 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:03.745656 10867 raft_consensus.cc:1275] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 9. (term mismatch)
I20260709 13:19:03.746021 10867 pending_rounds.cc:85] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 8
I20260709 13:19:03.746234 10867 pending_rounds.cc:107] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 17.9
I20260709 13:19:03.746469 10867 raft_consensus.cc:2889] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:03.682721 17680 consensus_queue.cc:1048] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:19:03.682590 17903 consensus_queue.cc:1048] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.045s
W20260709 13:19:03.730334 10797 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2478) took 1940 ms (client timeout 1545 ms). Trace:
W20260709 13:19:03.758528 10797 rpcz_store.cc:269] 0709 13:19:01.789548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:01.789712 (+   164us) service_pool.cc:224] Handling call
0709 13:19:03.730091 (+1940379us) spinlock_profiling.cc:243] Waited 579 ms on lock 0x7b54003ccb54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:03.730240 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1332029824}
I20260709 13:19:03.734555 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:03.666559 17897 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:03.704237 17210 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:03.693786 17817 consensus_queue.cc:1048] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.055s
I20260709 13:19:03.645386 17801 consensus_queue.cc:237] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:03.747702 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:03.774154 11512 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 19.
I20260709 13:19:03.694588 10873 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 21.
I20260709 13:19:03.728505 10944 raft_consensus.cc:1275] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:19:03.750778 10869 raft_consensus.cc:1240] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 22. Current term is 23. Ops: [22.11-22.11]
I20260709 13:19:03.641541 17438 consensus_queue.cc:237] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:03.759434 10867 raft_consensus.cc:1240] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 21. Current term is 22. Ops: [19.12-21.13]
I20260709 13:19:03.737934 17920 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:03.761015 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:03.762440 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:03.761842 11478 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:03.682152 17914 consensus_queue.cc:1048] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.101s
I20260709 13:19:03.652902 17729 consensus_queue.cc:237] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:03.775432 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 127115us
I20260709 13:19:03.734920 17282 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:03.776130 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2638) took 1005 ms. Trace:
I20260709 13:19:03.803941 10869 rpcz_store.cc:276] 0709 13:19:02.770499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:03.750084 (+979585us) service_pool.cc:224] Handling call
0709 13:19:03.750489 (+   405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:03.775851 (+ 25362us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:03.776076 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:03.776857 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 141193us
I20260709 13:19:03.740330 17579 raft_consensus.cc:3055] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:19:03.738123 17901 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:03.737170 10872 raft_consensus.cc:1240] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:03.754593 17861 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:03.784610 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2639) took 1013 ms. Trace:
I20260709 13:19:03.787648 11478 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:03.806175 10869 raft_consensus.cc:1240] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 18. Current term is 19. Ops: [16.13-18.14]
I20260709 13:19:03.812139 10797 raft_consensus.cc:1275] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:19:03.812837 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:03.785105 17896 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:03.793418 17900 raft_consensus.cc:2749] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:19:03.817271 10944 raft_consensus.cc:1275] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 26 index: 13. (term mismatch)
I20260709 13:19:03.820163 10872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 2498) took 1072 ms. Trace:
I20260709 13:19:03.739171 17834 consensus_queue.cc:1059] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 22.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.054s
W20260709 13:19:03.828766 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 91645.3us
I20260709 13:19:03.822621 10867 rpcz_store.cc:276] 0709 13:19:02.773408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:03.754914 (+981506us) service_pool.cc:224] Handling call
0709 13:19:03.755330 (+   416us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:03.779345 (+ 24015us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:03.784533 (+  5188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:03.751986 17894 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:03.841130 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2641) took 1067 ms. Trace:
I20260709 13:19:03.816171 17282 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:03.864729 10867 raft_consensus.cc:1240] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 13:19:03.861466 10869 rpcz_store.cc:276] 0709 13:19:02.773951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:03.805397 (+1031446us) service_pool.cc:224] Handling call
0709 13:19:03.805855 (+   458us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:03.834413 (+ 28558us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:03.841027 (+  6614us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:03.878108 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2642) took 1101 ms. Trace:
I20260709 13:19:03.878509 10867 rpcz_store.cc:276] 0709 13:19:02.776849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:03.864131 (+1087282us) service_pool.cc:224] Handling call
0709 13:19:03.864513 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:03.877820 (+ 13307us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:03.878036 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:03.879082 10841 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:03.879374 10872 rpcz_store.cc:276] 0709 13:19:02.747282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:03.736615 (+989333us) service_pool.cc:224] Handling call
0709 13:19:03.737003 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:03.819881 (+ 82878us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:03.820094 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:03.879945 10867 raft_consensus.cc:1275] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:19:03.866485 17808 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:03.799911 17920 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:03.873005 10871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 2493) took 1169 ms. Trace:
I20260709 13:19:03.874984 17776 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:03.880649 11506 raft_consensus.cc:1275] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:19:03.814677 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:03.820742 10842 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:03.882037 10869 raft_consensus.cc:1240] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:03.878942 10944 pending_rounds.cc:85] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 12
I20260709 13:19:03.919207 10944 pending_rounds.cc:107] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 25.13
I20260709 13:19:03.919538 10944 raft_consensus.cc:2889] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:03.920073 10916 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:03.920617 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:03.776161 17847 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:03.816072 17579 raft_consensus.cc:740] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:03.883769 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 2510) took 1071 ms. Trace:
I20260709 13:19:03.929850 10867 rpcz_store.cc:276] 0709 13:19:02.811988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:03.879315 (+1067327us) service_pool.cc:224] Handling call
0709 13:19:03.879621 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:03.880552 (+   931us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:03.883698 (+  3146us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:03.930553 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 109945us
I20260709 13:19:03.931123 10867 raft_consensus.cc:1240] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 24. Current term is 25. Ops: []
I20260709 13:19:03.908972 10871 rpcz_store.cc:276] 0709 13:19:02.703582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:02.714953 (+ 11371us) service_pool.cc:224] Handling call
0709 13:19:02.728087 (+ 13134us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:03.108568 (+380481us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:03.872603 (+764035us) spinlock_profiling.cc:243] Waited 349 ms on lock 0x7b5400316ad4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:19:03.872909 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":802256768}
W20260709 13:19:03.840994 17880 log.cc:927] Time spent T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:19:03.922297 10868 raft_consensus.cc:1240] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:03.923816 17894 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:03.927078 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2643) took 1137 ms. Trace:
I20260709 13:19:03.935837 10869 rpcz_store.cc:276] 0709 13:19:02.790195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:03.878726 (+1088531us) service_pool.cc:224] Handling call
0709 13:19:03.881813 (+  3087us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:03.926732 (+ 44919us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:03.927007 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:03.817314 17924 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:03.932724 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:03.932878 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 2514) took 1062 ms. Trace:
I20260709 13:19:03.940970 10867 rpcz_store.cc:276] 0709 13:19:02.870442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:03.930645 (+1060203us) service_pool.cc:224] Handling call
0709 13:19:03.930967 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:03.932521 (+  1554us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:03.932721 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:03.931449 10842 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:03.853653 17840 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:03.942978 10841 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:03.944228 10944 raft_consensus.cc:2410] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 21 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:19:03.819083 17883 consensus_queue.cc:1048] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.108s
I20260709 13:19:03.909483 17758 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:19:03.909703 17808 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:03.950017 17840 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:03.953822 17927 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:19:03.956391 17909 log.cc:927] Time spent T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.348s	user 0.001s	sys 0.000s
I20260709 13:19:03.959131 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:03.959758 17536 consensus_queue.cc:1048] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:19:03.951149 10916 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:03.936445 17562 raft_consensus.cc:2749] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:03.941514 17407 raft_consensus.cc:2804] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:03.854943 17896 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:03.832247 17901 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:03.984597 17901 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:03.986063 10801 raft_consensus.cc:1275] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 13:19:03.968773 17777 consensus_meta.cc:287] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.563s	user 0.010s	sys 0.001s
I20260709 13:19:03.975926 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:03.982959 10767 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:03.996757 10796 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 19.
I20260709 13:19:03.977191 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:03.981539 10916 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:19:03.981869 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:03.986752 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:03.974915 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:03.963865 17927 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:03.965623 17808 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:03.936311 17912 consensus_queue.cc:1059] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.041s
I20260709 13:19:03.960390 17766 consensus_queue.cc:1048] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 13:19:03.998272 10794 raft_consensus.cc:2410] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
W20260709 13:19:03.997373 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 209834us
I20260709 13:19:03.998258 10867 raft_consensus.cc:2393] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 17: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:03.998925 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:04.000078 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:04.018123 10799 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 20.
I20260709 13:19:03.939944 17670 consensus_queue.cc:1048] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.038s
I20260709 13:19:03.978055 17407 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:04.030421 17936 consensus_queue.cc:1048] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:04.031446 17936 consensus_queue.cc:1243] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer e01ecd008edd4aa2b6b2b47efbc2ea87 log is divergent from this leader: its last log entry 19.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:19:04.002043 11509 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
I20260709 13:19:04.032294 17899 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.032656 17919 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.032881 17899 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:04.033259 17919 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:04.002106 10795 raft_consensus.cc:1240] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:04.004930 10798 raft_consensus.cc:2393] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 18: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
W20260709 13:19:04.005985 17922 log.cc:927] Time spent T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
I20260709 13:19:04.037490 10792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3171) took 1354 ms. Trace:
I20260709 13:19:04.037892 10792 rpcz_store.cc:276] 0709 13:19:02.683306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:03.243949 (+560643us) service_pool.cc:224] Handling call
0709 13:19:03.403566 (+159617us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:03.432481 (+ 28915us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:04.037293 (+604812us) spinlock_profiling.cc:243] Waited 159 ms on lock 0x7b54000bbdd0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:04.037429 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":366088960}
W20260709 13:19:04.006516 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.006229 17920 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:04.006520 17896 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:04.009645 17923 log.cc:927] Time spent T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:19:03.947623 17924 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:04.010110 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:03.982008 17911 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:19:03.971073 17903 consensus_queue.cc:1048] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 13:19:04.011930 10868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 2503) took 1263 ms. Trace:
I20260709 13:19:04.008682 17927 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:19:04.013922 17929 log.cc:927] Time spent T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:19:04.015125 17861 raft_consensus.cc:2804] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:03.950428 17817 consensus_queue.cc:1059] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.046s
I20260709 13:19:03.997958 10797 raft_consensus.cc:2393] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 17: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:04.010304 17914 consensus_queue.cc:1048] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 13:19:04.027963 10801 raft_consensus.cc:2393] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 19: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:04.028815 17834 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.030982 17407 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:04.070974 17817 consensus_queue.cc:1059] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:19:04.073858 10801 raft_consensus.cc:1240] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 24. Current term is 25. Ops: [24.14-24.14]
I20260709 13:19:04.040781 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:04.075311 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:04.076406 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:04.042321 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:04.078085 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:04.002939 17579 consensus_queue.cc:260] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 22.10, Last appended by leader: 10, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:03.997936 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 113298us
I20260709 13:19:04.048014 10796 raft_consensus.cc:2393] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 17: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:04.010365 17819 raft_consensus.cc:3055] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:19:04.012326 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.047982 17911 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:04.046015 17920 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.027554 17897 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:04.079476 10797 raft_consensus.cc:1275] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:19:04.080256 10766 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:04.084381 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:04.096560 10798 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:04.050114 10868 rpcz_store.cc:276] 0709 13:19:02.748659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:03.744653 (+995994us) service_pool.cc:224] Handling call
0709 13:19:03.745054 (+   401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:03.936736 (+191682us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:04.011687 (+ 74951us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b54000f73d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:19:04.011869 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":407140608}
I20260709 13:19:04.078140 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:04.097800 10795 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 19.
I20260709 13:19:04.089133 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:04.091924 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:04.091866 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:04.109766 10873 raft_consensus.cc:1275] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:19:04.051827 17940 raft_consensus.cc:2749] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:04.095718 10766 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:19:04.119845 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.070258 17834 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:04.086390 17819 raft_consensus.cc:740] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:04.087038 17861 raft_consensus.cc:697] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 21 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:04.143119 10841 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:19:04.093546 17911 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:19:04.141956 10943 raft_consensus.cc:1275] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:19:04.059758 17758 raft_consensus.cc:697] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:19:04.145381 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 49781us
I20260709 13:19:04.148226 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:04.150981 10841 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:04.164776 10841 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:19:04.166396 10870 raft_consensus.cc:1275] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:19:04.154929 10798 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 19.
W20260709 13:19:04.157397 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.110862 17579 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:04.170416 10766 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:19:04.174216 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.156229 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 27 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:04.116266 17916 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:04.089375 17857 consensus_queue.cc:1048] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.060s
I20260709 13:19:04.164234 17622 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:04.182358 17766 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:04.147506 17756 consensus_queue.cc:1059] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
W20260709 13:19:04.227989 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 124269us
I20260709 13:19:04.141597 17936 consensus_queue.cc:1048] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 13:19:04.168205 17847 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:04.179193 17817 raft_consensus.cc:2749] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:04.106947 17920 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:04.144143 17842 consensus_queue.cc:1048] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 13:19:04.167837 17877 raft_consensus.cc:2804] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:04.184855 17916 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:04.206287 17919 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:04.240062 10766 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:04.210469 17940 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:04.267081 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:04.273344 17722 raft_consensus.cc:3055] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:19:04.273880 17722 raft_consensus.cc:740] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:04.274891 17722 consensus_queue.cc:260] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:04.275997 17722 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:04.207058 17788 consensus_queue.cc:1048] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 13:19:04.212168 17894 consensus_queue.cc:1048] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 13:19:04.235948 17622 raft_consensus.cc:697] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:04.247692 17916 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:04.246534 17840 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:04.247040 17877 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:04.306998 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 324081us
I20260709 13:19:04.270212 17935 raft_consensus.cc:2804] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:19:04.345225 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.278863 17940 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:04.347172 17935 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:04.251282 17819 consensus_queue.cc:260] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:04.347350 17877 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:04.351039 10767 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:04.352072 17920 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:04.408288 17935 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:04.354811 17536 consensus_queue.cc:1048] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:19:04.356237 10871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 2518) took 1186 ms. Trace:
I20260709 13:19:04.277680 17861 consensus_queue.cc:237] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:04.410696 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:04.340636 17856 consensus_meta.cc:287] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.766s	user 0.012s	sys 0.000s
I20260709 13:19:04.337427 17680 raft_consensus.cc:2749] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:04.412122 10941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 1253) took 2026 ms. Trace:
I20260709 13:19:04.246292 17282 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:04.424978 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 186948us
I20260709 13:19:04.443081 10941 rpcz_store.cc:276] 0709 13:19:02.385807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:02.465718 (+ 79911us) service_pool.cc:224] Handling call
0709 13:19:02.494842 (+ 29124us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:03.306295 (+811453us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:04.411850 (+1105555us) spinlock_profiling.cc:243] Waited 810 ms on lock 0x7b54000177d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:19:04.412057 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1864092544}
I20260709 13:19:04.336825 17777 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.298739 17758 consensus_queue.cc:237] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:04.435046 10871 rpcz_store.cc:276] 0709 13:19:03.169684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:03.932029 (+762345us) service_pool.cc:224] Handling call
0709 13:19:03.932431 (+   402us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:03.935501 (+  3070us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:19:04.044345 (+108844us) log.cc:844] Serialized 14 byte log entry
0709 13:19:04.288567 (+244222us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:04.336734 (+ 48167us) log.cc:844] Serialized 26 byte log entry
0709 13:19:04.336908 (+   174us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:19:04.336955 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:04.337049 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:04.355950 (+ 18901us) raft_consensus.cc:1685] finished
0709 13:19:04.355982 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:04.356164 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14525,"prepare.queue_time_us":249732,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":373,"raft.queue_time_us":59871,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":7876,"spinlock_wait_cycles":2473216,"thread_start_us":294352,"threads_started":2,"wal-append.queue_time_us":292576}
I20260709 13:19:04.409632 17940 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:04.444420 10916 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:04.436782 17536 consensus_queue.cc:1243] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer cb0428cc55984daba1ba63744a5102a2 log is divergent from this leader: its last log entry 23.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:19:04.449836 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:04.449302 10916 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:04.441237 17924 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:04.441107 17766 raft_consensus.cc:697] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:19:04.468513 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:04.468820 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 132596us
I20260709 13:19:04.448300 17856 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.474092 17766 consensus_queue.cc:237] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:04.464345 17327 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:04.464257 17729 raft_consensus.cc:2804] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:04.467816 17842 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.442196 17622 consensus_queue.cc:237] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:04.450062 17777 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:04.492744 17729 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:04.500618 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.500953 17899 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:04.501596 10800 raft_consensus.cc:1275] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:19:04.489234 17327 raft_consensus.cc:697] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 19 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:04.484727 17856 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:04.508030 17949 log.cc:927] Time spent T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
W20260709 13:19:04.508818 17953 log.cc:927] Time spent T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:19:04.513949 17763 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:04.528385 10797 raft_consensus.cc:1275] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:19:04.529317 10795 raft_consensus.cc:1275] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:19:04.530651 10794 raft_consensus.cc:1275] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:19:04.532502 10793 raft_consensus.cc:1275] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 21 index: 9. (index mismatch)
I20260709 13:19:04.534368 17965 consensus_queue.cc:1048] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:19:04.543615 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:04.497897 17950 log.cc:927] Time spent T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
W20260709 13:19:04.588482 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.590281 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:04.545715 17729 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:19:04.576638 17917 consensus_meta.cc:287] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.980s	user 0.013s	sys 0.000s
I20260709 13:19:04.555500 17847 consensus_queue.cc:1048] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 13:19:04.621472 10842 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:04.622105 17883 consensus_queue.cc:1048] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:19:04.622795 17562 consensus_queue.cc:1048] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:19:04.587455 17817 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:04.535529 17842 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:04.597693 17763 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:04.658627 17959 log.cc:927] Time spent T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:19:04.661998 10801 raft_consensus.cc:1275] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 13:19:04.662101 17808 consensus_meta.cc:287] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.626s	user 0.010s	sys 0.000s
W20260709 13:19:04.663089 17964 log.cc:927] Time spent T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:19:04.664978 10944 raft_consensus.cc:1240] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 21 LEADER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 13:19:04.667541 10944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 3129) took 1018 ms. Trace:
I20260709 13:19:04.667915 10944 rpcz_store.cc:276] 0709 13:19:03.648901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:03.945274 (+296373us) service_pool.cc:224] Handling call
0709 13:19:03.945644 (+   370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:04.665414 (+719770us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:04.667259 (+  1845us) spinlock_profiling.cc:243] Waited 719 ms on lock 0x7b540018ecd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:19:04.667477 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1653690240}
I20260709 13:19:04.669585 17411 raft_consensus.cc:3055] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:19:04.670032 17411 raft_consensus.cc:740] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:04.670994 17411 consensus_queue.cc:260] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:04.672386 17411 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:19:04.675050 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.575187 17327 consensus_queue.cc:237] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:04.677841 17940 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:04.679122 10876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 2482) took 2085 ms. Trace:
I20260709 13:19:04.679597 10876 rpcz_store.cc:276] 0709 13:19:02.593383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:02.599111 (+  5728us) service_pool.cc:224] Handling call
0709 13:19:02.602361 (+  3250us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:02.613051 (+ 10690us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:02.613099 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:03.073541 (+460442us) spinlock_profiling.cc:243] Waited 2.24 ms on lock 0x7b18004123c0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7dd962a6 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7f7aea11 00007f9c7f7be372 00007f9c80ef0a01 00007f9c80eb543a
0709 13:19:04.113096 (+1039555us) spinlock_profiling.cc:243] Waited 2.08 ms on lock 0x7b2c00029f28. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7f7aea11 00007f9c7f7be372 00007f9c80ef0a01 00007f9c80eb543a 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2
0709 13:19:04.113683 (+   587us) log.cc:844] Serialized 52 byte log entry
0709 13:19:04.335136 (+221453us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:04.335197 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:04.335304 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:04.678857 (+343553us) raft_consensus.cc:1685] finished
0709 13:19:04.678882 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:04.679050 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3704,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":12309,"spinlock_wait_cycles":9941760,"thread_start_us":147986,"threads_started":1,"wal-append.queue_time_us":267944}
I20260709 13:19:04.681484 17498 consensus_queue.cc:1048] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:19:04.684393 17949 log.cc:927] Time spent T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 13:19:04.687808 17911 consensus_meta.cc:287] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.506s	user 0.010s	sys 0.000s
I20260709 13:19:04.688267 17911 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:04.690806 17911 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:04.654624 10874 raft_consensus.cc:2393] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 19: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:19:04.671491 17956 log.cc:927] Time spent T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.190s	user 0.001s	sys 0.000s
I20260709 13:19:04.691284 17817 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:04.693339 17842 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:04.695214 17210 consensus_meta.cc:287] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.916s	user 0.007s	sys 0.005s
I20260709 13:19:04.695636 17210 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:04.696725 10801 raft_consensus.cc:1240] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 20. Current term is 21. Ops: [20.14-20.14]
W20260709 13:19:04.697501 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.588218 17891 raft_consensus.cc:3055] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:04.698146 17891 raft_consensus.cc:740] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:04.699241 17891 consensus_queue.cc:260] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:04.644397 17902 consensus_queue.cc:1048] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 13:19:04.700695 17282 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:04.701640 17891 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:04.702346 17856 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:04.694542 17817 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:04.701327 17210 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:04.662667 17808 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:04.716945 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 95555.2us
I20260709 13:19:04.728418 10944 raft_consensus.cc:1217] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: Deduplicated request from leader. Original: 19.10->[21.11-26.13]   Dedup: 21.11->[24.12-26.13]
I20260709 13:19:04.734616 10944 raft_consensus.cc:1240] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 26. Current term is 27. Ops: [21.11-26.13]
I20260709 13:19:04.735898 10869 raft_consensus.cc:1240] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:04.706740 17877 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:04.716434 17763 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:04.711234 17927 consensus_meta.cc:287] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.650s	user 0.006s	sys 0.004s
I20260709 13:19:04.739130 17927 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:04.740367 17763 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:04.741227 17927 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:04.742756 17927 consensus_queue.cc:1059] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:19:04.743014 17899 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.743561 17877 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:04.743831 17927 consensus_queue.cc:1243] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer cb0428cc55984daba1ba63744a5102a2 log is divergent from this leader: its last log entry 23.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:19:04.744518 17877 raft_consensus.cc:3055] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:19:04.744917 10867 raft_consensus.cc:2393] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 22: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:04.744984 17877 raft_consensus.cc:740] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:04.745973 17877 consensus_queue.cc:260] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 26.13, Last appended by leader: 13, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:04.666778 17407 consensus_meta.cc:287] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.596s	user 0.006s	sys 0.002s
I20260709 13:19:04.743598 17899 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:04.747087 17407 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:04.748445 17899 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:04.749099 17407 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:04.723872 17579 consensus_meta.cc:287] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.545s	user 0.008s	sys 0.004s
I20260709 13:19:04.736969 17817 consensus_queue.cc:1048] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
W20260709 13:19:04.756402 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.776599 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:04.776142 10876 raft_consensus.cc:1240] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 23. Current term is 24. Ops: [23.17-23.17]
I20260709 13:19:04.751447 10947 raft_consensus.cc:1240] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 18. Current term is 19. Ops: []
W20260709 13:19:04.772589 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.761755 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:04.772321 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:04.714110 17808 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:04.777985 10948 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:04.716739 17562 raft_consensus.cc:2749] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:19:04.751046 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:04.751958 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:04.784237 10944 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Advancing to term 22
I20260709 13:19:04.781431 10942 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:04.791553 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:04.793577 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:04.795285 10867 raft_consensus.cc:1240] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:04.796576 10867 raft_consensus.cc:1275] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:19:04.792443 10876 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 19.
I20260709 13:19:04.799793 10871 raft_consensus.cc:1275] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
W20260709 13:19:04.800546 10686 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 2b82f18992434ed5ba58b805107060d4 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 468a8fd46e824f62b9c2c0d3532a6d26 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 20 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:04.819108 10942 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
I20260709 13:19:04.819290 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:04.780540 17763 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:04.820586 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:04.823637 10943 raft_consensus.cc:2410] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:19:04.822432 10872 raft_consensus.cc:1240] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:04.824361 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:04.825367 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 27 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:04.825942 10942 raft_consensus.cc:2393] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 27: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:04.824234 10841 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:04.777220 17883 consensus_queue.cc:1059] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.038s
I20260709 13:19:04.828577 10943 raft_consensus.cc:1240] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:04.834195 10942 raft_consensus.cc:1275] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:19:04.835014 10944 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 22.
I20260709 13:19:04.836076 10841 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:04.836707 10944 raft_consensus.cc:1240] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:04.837685 10944 raft_consensus.cc:1240] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:04.735410 17916 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:04.779122 17729 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:04.779001 17935 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:04.848564 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:04.849607 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.802476 17282 consensus_queue.cc:1048] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.041s
I20260709 13:19:04.843395 17562 raft_consensus.cc:2804] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:04.842005 17766 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.846886 17883 consensus_queue.cc:1243] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer cb0428cc55984daba1ba63744a5102a2 log is divergent from this leader: its last log entry 21.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 13:19:04.864099 10794 raft_consensus.cc:1240] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 13:19:04.864825 10950 raft_consensus.cc:1240] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:19:04.843111 17821 raft_consensus.cc:3055] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:04.845417 17758 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:04.871567 10949 raft_consensus.cc:1240] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:19:04.833034 17763 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:04.882371 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:04.873551 17562 raft_consensus.cc:697] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:04.894876 17562 consensus_queue.cc:237] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:04.895601 17970 log.cc:927] Time spent T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 13:19:04.894893 17971 log.cc:927] Time spent T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:19:04.903957 10948 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
I20260709 13:19:04.872380 17407 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.873021 17821 raft_consensus.cc:740] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:04.932576 10944 raft_consensus.cc:1275] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 22 index: 10. Preceding OpId from leader: term: 23 index: 10. (term mismatch)
I20260709 13:19:04.933027 10944 pending_rounds.cc:85] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 9
I20260709 13:19:04.933251 10944 pending_rounds.cc:107] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 22.10
I20260709 13:19:04.933475 10944 raft_consensus.cc:2889] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:04.935046 10948 raft_consensus.cc:1240] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 21 LEADER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 13:19:04.888680 17758 raft_consensus.cc:697] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:04.892383 17940 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:04.938674 17788 consensus_queue.cc:1048] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:04.939672 17788 consensus_queue.cc:1243] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer cb0428cc55984daba1ba63744a5102a2 log is divergent from this leader: its last log entry 20.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:19:04.926867 17407 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:04.946010 17407 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:04.947360 17821 consensus_queue.cc:260] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:04.948582 17821 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:04.873198 17766 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:04.953351 17766 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:04.957765 10874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 2648) took 2072 ms. Trace:
W20260709 13:19:04.958083 17975 log.cc:927] Time spent T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:19:04.958228 10874 rpcz_store.cc:276] 0709 13:19:02.884900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:03.708647 (+823747us) service_pool.cc:224] Handling call
0709 13:19:04.957536 (+1248889us) spinlock_profiling.cc:243] Waited 945 ms on lock 0x7b540006b8d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:04.957691 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2173081984}
I20260709 13:19:04.960268 10872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2647) took 2091 ms. Trace:
I20260709 13:19:04.960681 10872 rpcz_store.cc:276] 0709 13:19:02.868714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:03.880478 (+1011764us) service_pool.cc:224] Handling call
0709 13:19:03.939316 (+ 58838us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:04.824729 (+885413us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:04.960030 (+135301us) spinlock_profiling.cc:243] Waited 883 ms on lock 0x7b5400316fd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:19:04.960206 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2030524288}
W20260709 13:19:04.965582 17974 log.cc:927] Time spent T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.242s	user 0.002s	sys 0.000s
W20260709 13:19:04.966547 10875 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 2490) took 2291 ms (client timeout 1902 ms). Trace:
I20260709 13:19:04.980125 17758 consensus_queue.cc:237] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:04.978195 17777 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:04.965673 17729 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:04.970521 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.981352 17877 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:19:04.981098 10875 rpcz_store.cc:269] 0709 13:19:02.687298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:02.688131 (+   833us) service_pool.cc:224] Handling call
0709 13:19:04.966284 (+2278153us) spinlock_profiling.cc:243] Waited 960 ms on lock 0x7b5400109ad4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:04.966429 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2208148864}
I20260709 13:19:04.987742 17896 raft_consensus.cc:3055] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:04.996196 17936 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:05.118637 10945 pending_rounds.cc:85] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 8
I20260709 13:19:05.118996 10945 pending_rounds.cc:107] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 20.9
W20260709 13:19:04.997242 17945 log.cc:927] Time spent T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.720s	user 0.000s	sys 0.000s
I20260709 13:19:05.001272 10872 raft_consensus.cc:1275] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 19 index: 16. (term mismatch)
I20260709 13:19:05.119240 10945 raft_consensus.cc:2889] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:05.119505 10872 pending_rounds.cc:85] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 15
I20260709 13:19:05.119817 10872 pending_rounds.cc:107] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 20.16
I20260709 13:19:05.120055 10872 raft_consensus.cc:2889] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:05.002282 10874 raft_consensus.cc:1275] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:19:05.005518 10871 raft_consensus.cc:1275] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:19:05.006115 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 27 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:05.006523 17670 raft_consensus.cc:3055] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:05.006748 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:05.122468 10867 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 20.
I20260709 13:19:05.015911 17899 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:05.016639 17808 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:05.021697 17834 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:05.031666 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:05.033012 10946 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:05.026546 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:19:05.040557 11509 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56410 (request call id 1102) took 1610 ms (client timeout 2057 ms). Trace:
I20260709 13:19:05.056200 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:05.057039 11478 leader_election.cc:336] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Timed out: RequestConsensusVote RPC to 127.9.116.129:39685 timed out after 1.572s (SENT)
W20260709 13:19:05.058511 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.058931 17903 consensus_queue.cc:1059] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:19:05.059059 17944 consensus_queue.cc:1059] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:19:04.966972 17935 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:05.062074 17763 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:05.066051 10950 raft_consensus.cc:1240] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:19:03.511490  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:19:05.172185 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 115210us
I20260709 13:19:05.067229 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:05.068799 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:05.069044 17819 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:05.072149 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:05.074509 17920 raft_consensus.cc:2749] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:19:05.074826 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 618554us
I20260709 13:19:05.076118 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 2521) took 1880 ms. Trace:
I20260709 13:19:05.176251 10949 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 19.
I20260709 13:19:05.188711 10869 rpcz_store.cc:276] 0709 13:19:03.196101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:03.936640 (+740539us) service_pool.cc:224] Handling call
0709 13:19:03.936930 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:04.736501 (+799571us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:05.075857 (+339356us) spinlock_profiling.cc:243] Waited 799 ms on lock 0x7b5400077454. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:19:05.076044 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1836929408}
W20260709 13:19:05.074579 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 723596us
W20260709 13:19:05.078300 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.088354 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:05.190830 10948 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
I20260709 13:19:05.193164 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:19:05.194104 10767 connection.cc:256] RPC call timeout handler was delayed by -1.4181004140000368s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1549s for a response
I20260709 13:19:05.090440 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:05.122854 17896 raft_consensus.cc:740] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:05.113948 17821 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.117544 17891 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.123358 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:05.172821 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:05.131928 17670 raft_consensus.cc:740] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:05.177264 10794 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
W20260709 13:19:05.181353 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.216012 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:05.216813 10794 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:05.182646 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:05.198930 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:05.113263 17777 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:19:05.199951 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.201037 10801 raft_consensus.cc:2393] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 22: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
W20260709 13:19:05.194612 10767 connection.cc:256] RPC call timeout handler was delayed by -1.404346569999916s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2189s for a response
W20260709 13:19:05.223374 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.209410 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:05.224404 10916 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:05.224738 10869 raft_consensus.cc:2410] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:19:05.225654 10916 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:05.223296 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:19:05.199215 11509 rpcz_store.cc:269] 0709 13:19:03.430576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:03.430874 (+   298us) service_pool.cc:224] Handling call
0709 13:19:05.040262 (+1609388us) spinlock_profiling.cc:243] Waited 539 ms on lock 0x7b54005f96d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:05.040457 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1240806656}
I20260709 13:19:05.170493 17928 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.251550 10871 raft_consensus.cc:1275] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:19:05.260075 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:05.263372 10946 raft_consensus.cc:1275] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 14. (term mismatch)
I20260709 13:19:05.263936 10946 pending_rounds.cc:85] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 13
I20260709 13:19:05.263825 10916 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:05.264452 10946 pending_rounds.cc:107] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 24.14
I20260709 13:19:05.264822 10946 raft_consensus.cc:2889] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:05.211885 10699 catalog_manager.cc:5697] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 11 to 17, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 17 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:05.274118 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:05.275856 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.276708 10916 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:05.199572 17847 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.279120 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:05.291087 10801 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 22.
I20260709 13:19:05.291435 10946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 3110) took 2906 ms. Trace:
I20260709 13:19:05.280037 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:05.200747 17917 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.300302 10699 catalog_manager.cc:5697] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 11 to 19, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 19 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:05.291989 10946 rpcz_store.cc:276] 0709 13:19:02.359114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:02.448758 (+ 89644us) service_pool.cc:224] Handling call
0709 13:19:02.466763 (+ 18005us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:03.245839 (+779076us) spinlock_profiling.cc:243] Waited 1.5 ms on lock 0x7b540015cfa0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80e2cdc0 00007f9c80eb71c0 00007f9c80eb8231 00007f9c80eb4d8f 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c
0709 13:19:05.032607 (+1786768us) spinlock_profiling.cc:243] Waited 563 ms on lock 0x7b580098b1d0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80e110aa 00007f9c80e93447 00007f9c80eabfe7 00007f9c80ead7de 00007f9c80eb7b49 00007f9c80eb8416 00007f9c80eb4d8f 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4
0709 13:19:05.265419 (+232812us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:05.265687 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1297346048}
I20260709 13:19:05.293812 10916 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:05.307904 10916 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:19:05.138621 17936 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:05.298578 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 27 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:05.302071 10699 catalog_manager.cc:5697] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 16 to 20. New cstate: current_term: 20 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:05.314273 10916 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:05.296135 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:05.319373 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:05.141088 17899 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:05.309087 17579 raft_consensus.cc:2749] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:05.343523 17722 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:05.344173 17722 raft_consensus.cc:697] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:05.345054 17722 consensus_queue.cc:237] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:05.346081 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:05.328164 17327 raft_consensus.cc:2804] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:05.347326 17327 raft_consensus.cc:697] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 19 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:05.348290 17327 consensus_queue.cc:237] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:05.348701 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:05.328564 10694 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42506 (request call id 157) took 46161 ms (client timeout 15000 ms). Trace:
I20260709 13:19:05.266925 17897 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.356951 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:05.357510 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.357885 10872 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 19.
I20260709 13:19:05.271622 17407 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:05.253890 17891 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:05.338900 10916 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:19:05.327798 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:05.341969 10794 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
I20260709 13:19:05.169363 17808 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:05.351212 10694 rpcz_store.cc:269] 0709 13:18:19.148537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:19.148877 (+   340us) service_pool.cc:224] Handling call
0709 13:19:05.303430 (+46154553us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:00.378809 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:00.379541 (+   732us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:00.379598 (+    57us) write_op.cc:435] Acquired schema lock
0709 13:19:00.379619 (+    21us) tablet.cc:662] Decoding operations
0709 13:19:00.380782 (+  1163us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:00.380895 (+   113us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:00.380931 (+    36us) tablet.cc:685] Acquiring locks for 12 operations
0709 13:19:00.382158 (+  1227us) tablet.cc:701] Row locks acquired
0709 13:19:00.382214 (+    56us) write_op.cc:260] PREPARE: finished
0709 13:19:00.395552 (+ 13338us) write_op.cc:270] Start()
0709 13:19:00.434534 (+ 38982us) write_op.cc:276] Timestamp: P: 1783603140382318 usec, L: 0
0709 13:19:00.452493 (+ 17959us) op_driver.cc:348] REPLICATION: starting
0709 13:19:00.583475 (+130982us) log.cc:844] Serialized 4813 byte log entry
0709 13:19:00.777807 (+194332us) spinlock_profiling.cc:243] Waited 18.2 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c7ddab0ac 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b
0709 13:19:02.035880 (+1258073us) spinlock_profiling.cc:243] Waited 17.2 ms on lock 0x7b18003e3f80. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc5455 00007f9c80e32190 00007f9c80e2bae7 00007f9c80e2add9 00007f9c80e35930
0709 13:19:02.231663 (+195783us) op_driver.cc:464] REPLICATION: finished
0709 13:19:02.356891 (+125228us) write_op.cc:301] APPLY: starting
0709 13:19:02.357011 (+   120us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:02.532452 (+175441us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:02.532510 (+    58us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:03.041881 (+509371us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:03.042266 (+   385us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=24,key_file_lookups=24,delta_file_lookups=24,mrs_lookups=0
0709 13:19:03.042334 (+    68us) write_op.cc:312] APPLY: finished
0709 13:19:03.042767 (+   433us) log.cc:844] Serialized 113 byte log entry
0709 13:19:03.068493 (+ 25726us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:03.074295 (+  5802us) write_op.cc:454] Released schema lock
0709 13:19:03.074390 (+    95us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:03.076714 (+  2324us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:03.076751 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":6518903,"reported_tablets":31,"child_traces":[["op",{"apply.queue_time_us":95879,"apply.run_cpu_time_us":23398,"apply.run_wall_time_us":719806,"cfile_cache_hit":50,"cfile_cache_hit_bytes":54311,"num_ops":12,"prepare.queue_time_us":413047,"prepare.run_cpu_time_us":123779,"prepare.run_wall_time_us":640682,"raft.queue_time_us":1186266,"raft.run_cpu_time_us":13372,"raft.run_wall_time_us":55628,"replication_time_us":1779003,"spinlock_wait_cycles":81458816,"thread_start_us":693456,"threads_started":4,"wal-append.queue_time_us":435930,"wal-append.run_cpu_time_us":13890,"wal-append.run_wall_time_us":3077065}]]}
I20260709 13:19:05.278601 17777 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:05.289172 17856 raft_consensus.cc:2749] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:05.325345 10699 catalog_manager.cc:5697] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 16 to 21, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 21 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:05.375321 17808 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:05.310087 17842 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:19:05.350378 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:05.372994 10868 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 22.
I20260709 13:19:05.367992 17903 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.254045 17821 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:05.395328 10874 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 21.
I20260709 13:19:05.310945 17847 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:05.383453 17729 raft_consensus.cc:2764] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:19:05.371765 17897 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:05.371994 17861 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.288697 17928 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:05.375355 17899 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:05.384889 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:05.321903 17917 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:05.385645 17407 raft_consensus.cc:697] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 18 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:05.438570 17916 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:05.389156 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 50405.5us
I20260709 13:19:05.380487 10699 catalog_manager.cc:5697] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 14 to 18. New cstate: current_term: 18 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
W20260709 13:19:05.435487 17883 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.093s	sys 0.234s
I20260709 13:19:05.397868 17842 raft_consensus.cc:697] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 23 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:05.398872 17903 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:05.521286 11477 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.092s	sys 0.112s
I20260709 13:19:05.378378 17896 consensus_queue.cc:260] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:05.526602 10699 catalog_manager.cc:5697] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 14 to 17. New cstate: current_term: 17 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:05.391108 17670 consensus_queue.cc:260] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:05.533681 11477 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.092s	sys 0.113s
I20260709 13:19:05.548988 11512 raft_consensus.cc:1240] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 20. Current term is 21. Ops: [20.16-20.16]
W20260709 13:19:05.528330 17836 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 25 current term: 25 is higher.
W20260709 13:19:05.526288 17883 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.094s	sys 0.236s
W20260709 13:19:05.528257 17451 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.017s	sys 0.047s
I20260709 13:19:05.557703 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:05.563961 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:19:05.579945 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.584579 17897 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:05.585431 17897 consensus_queue.cc:1059] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:19:05.548785 17808 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.564109 17579 consensus_queue.cc:1048] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 13:19:05.587018 17808 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:05.563663 17282 consensus_queue.cc:1059] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 13:19:05.589306 17808 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:05.589345 17282 raft_consensus.cc:3055] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:05.589988 17282 raft_consensus.cc:740] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:19:05.591317 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.563467 10699 catalog_manager.cc:5697] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 16 to 21, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 21 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:05.563764 11507 raft_consensus.cc:1275] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:19:05.595697 10699 catalog_manager.cc:5697] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:05.509719 17861 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:05.549182 17760 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:05.564018 17451 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.018s	sys 0.048s
I20260709 13:19:05.555037 17899 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:05.572144 17977 log.cc:927] Time spent T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.388s	user 0.001s	sys 0.000s
I20260709 13:19:05.602536 17944 raft_consensus.cc:3055] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:05.603238 17766 raft_consensus.cc:3055] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:19:05.611110 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 408119us
W20260709 13:19:05.634522 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.611881 17935 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
W20260709 13:19:05.615965 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.528713 17729 consensus_queue.cc:1059] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.063s
I20260709 13:19:05.622830 17891 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:05.574662 17896 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:05.574424 17670 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:05.573048 17777 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:05.542948 17856 consensus_queue.cc:1059] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.058s
I20260709 13:19:05.622560 10841 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:19:05.689805 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 67316.6us
I20260709 13:19:05.623862 17965 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.623548 17760 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:05.611820 17411 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.636559 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:05.616753 17282 consensus_queue.cc:260] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:05.636983 10699 catalog_manager.cc:5697] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 16 to 20, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 20 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:05.638289 10944 raft_consensus.cc:1275] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:19:05.649277 10766 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:19:05.660313 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.628005 17944 raft_consensus.cc:740] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:05.712854 17944 consensus_queue.cc:260] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:05.714283 17944 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:05.632244 17936 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:05.715083 17936 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:05.715583 17936 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:05.625403 17763 consensus_queue.cc:1059] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 13:19:05.695422 10792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2522) took 1237 ms. Trace:
I20260709 13:19:05.632066 17766 raft_consensus.cc:740] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:05.647085 17916 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.599267 17407 consensus_queue.cc:237] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:05.625586 17927 consensus_queue.cc:1059] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 13:19:05.629346 17808 consensus_queue.cc:1059] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 13:19:05.651304 17579 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.637259 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:05.653604 17451 raft_consensus.cc:3055] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 13:19:05.688926 17982 log.cc:927] Time spent T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.257s	user 0.001s	sys 0.000s
I20260709 13:19:05.651530 17622 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:05.735334 17622 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:05.737457 17622 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:05.717067 10792 rpcz_store.cc:276] 0709 13:19:04.458073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.460945 (+  2872us) service_pool.cc:224] Handling call
0709 13:19:04.461379 (+   434us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:05.074957 (+613578us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:05.074988 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:05.075500 (+   512us) log.cc:844] Serialized 26 byte log entry
0709 13:19:05.390152 (+314652us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:05.390211 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:05.390305 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:05.695098 (+304793us) raft_consensus.cc:1685] finished
0709 13:19:05.695136 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:05.695339 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":680747,"raft.queue_time_us":613882,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":93,"spinlock_wait_cycles":775680,"thread_start_us":313877,"threads_started":1,"wal-append.queue_time_us":539540}
I20260709 13:19:05.738853 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:05.738901 17579 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:05.630295 17899 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:05.741277 17899 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:05.741694 17579 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:05.717010 17821 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:05.684580 17756 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.751165 17861 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:05.751053 17756 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:05.704670 17562 consensus_queue.cc:1048] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.025s
W20260709 13:19:05.724937 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 75766.5us
I20260709 13:19:05.753037 17903 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:05.701128 17891 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:05.631498 17842 consensus_queue.cc:237] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:05.733865 17777 raft_consensus.cc:3055] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:05.730821 17282 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:05.734258 10947 raft_consensus.cc:2393] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 22: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:05.731626 17451 raft_consensus.cc:740] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:05.679719 17935 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:05.708103 10699 catalog_manager.cc:5697] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 18 to 21, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 21 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:19:05.704699 17603 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.021s	sys 0.043s
I20260709 13:19:05.750116 17760 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:05.705003 17965 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:05.761552 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.717244 17729 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:05.728114 17857 raft_consensus.cc:2749] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:05.732936 17883 raft_consensus.cc:3055] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:19:05.789390 17603 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.022s	sys 0.043s
I20260709 13:19:05.711827 17847 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:05.782546 10767 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.010s	sys 0.059s
W20260709 13:19:05.786085 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.745014 17928 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:05.788040 10699 catalog_manager.cc:5697] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 14 to 17. New cstate: current_term: 17 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:05.759130 17891 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:05.755223 17927 raft_consensus.cc:3055] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:05.816854 17927 raft_consensus.cc:740] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:19:05.817879 17983 log.cc:927] Time spent T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 13:19:05.817744 17927 consensus_queue.cc:260] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:05.819042 17927 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:05.820024 10799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3190) took 1291 ms. Trace:
I20260709 13:19:05.820405 10799 rpcz_store.cc:276] 0709 13:19:04.528129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.531131 (+  3002us) service_pool.cc:224] Handling call
0709 13:19:04.531519 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:05.075392 (+543873us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:05.075427 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:05.075918 (+   491us) log.cc:844] Serialized 26 byte log entry
0709 13:19:05.519923 (+444005us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:05.519965 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:05.520042 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:05.819809 (+299767us) raft_consensus.cc:1685] finished
0709 13:19:05.819830 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:05.819967 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":543493,"raft.queue_time_us":544096,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":80,"thread_start_us":443530,"threads_started":1,"wal-append.queue_time_us":706036}
I20260709 13:19:05.822201 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:05.728588 17916 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:05.761081 17777 raft_consensus.cc:740] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:05.708784 17411 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:05.814663 10767 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.011s	sys 0.059s
I20260709 13:19:05.813392 17836 raft_consensus.cc:3055] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:19:05.831388 17836 raft_consensus.cc:740] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:05.832111 17411 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:05.833084 10793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3195) took 1289 ms. Trace:
I20260709 13:19:05.833566 17777 consensus_queue.cc:260] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:05.833577 10793 rpcz_store.cc:276] 0709 13:19:04.543178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.608124 (+ 64946us) service_pool.cc:224] Handling call
0709 13:19:04.613447 (+  5323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:05.077624 (+464177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:05.077670 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:05.120432 (+ 42762us) log.cc:844] Serialized 26 byte log entry
0709 13:19:05.632224 (+511792us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:05.632271 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:05.632359 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:05.832805 (+200446us) raft_consensus.cc:1685] finished
0709 13:19:05.832837 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:05.833012 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":894124,"raft.queue_time_us":504541,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":7437,"thread_start_us":431732,"threads_started":1,"wal-append.queue_time_us":661825}
I20260709 13:19:05.839251 17836 consensus_queue.cc:260] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:05.840706 17836 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:05.842224 17916 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:05.842999 10792 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 17.
I20260709 13:19:05.775813 17919 consensus_queue.cc:1059] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 13:19:05.816401 17891 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:05.789687 17917 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:05.808892 17883 raft_consensus.cc:740] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:05.813855 10699 catalog_manager.cc:5697] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:05.786638 17840 consensus_queue.cc:1059] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.037s
I20260709 13:19:05.789337 17940 consensus_queue.cc:1059] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:19:05.874962 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:05.847784 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:05.782459 17622 consensus_queue.cc:1059] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 13:19:05.853018 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:19:05.829952 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 607311us
I20260709 13:19:05.814388 17756 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:05.856683 10699 catalog_manager.cc:5697] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 12 to 16, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 16 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
W20260709 13:19:05.873437 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 315487us
W20260709 13:19:06.015865 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 62898.2us
W20260709 13:19:05.927482 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:06.016729 10767 leader_election.cc:336] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Timed out: RequestConsensusVote RPC to 127.9.116.130:37895 timed out after 2.055s (SENT)
I20260709 13:19:05.927443 10799 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 22.
I20260709 13:19:05.787297 17766 consensus_queue.cc:260] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:06.055086 10916 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:06.067394 11478 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.891s	user 0.020s	sys 0.048s
W20260709 13:19:06.069029 11478 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.893s	user 0.020s	sys 0.048s
I20260709 13:19:05.954557 10842 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:19:06.070585 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 116102us
W20260709 13:19:06.071543 10942 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.953s	user 0.004s	sys 0.040s
I20260709 13:19:06.071478 10842 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:19:06.071956 10942 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.954s	user 0.004s	sys 0.040s
W20260709 13:19:05.982879 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:05.928491 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.810793 17729 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:06.018529 10799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3211) took 1139 ms. Trace:
I20260709 13:19:06.076128 10799 rpcz_store.cc:276] 0709 13:19:04.879131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.017602 (+1138471us) service_pool.cc:224] Handling call
0709 13:19:06.017900 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:06.018122 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:06.018160 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:06.018229 (+    69us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:06.018274 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:06.018342 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:06.018462 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:06.077970 10799 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:05.762007 17924 consensus_queue.cc:1048] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 13:19:05.840737 17935 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:06.032150 10793 raft_consensus.cc:1240] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:06.091040 10793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3210) took 1236 ms. Trace:
I20260709 13:19:06.091487 10793 rpcz_store.cc:276] 0709 13:19:04.854505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.015135 (+1160630us) service_pool.cc:224] Handling call
0709 13:19:06.031961 (+ 16826us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:06.090691 (+ 58730us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:06.090970 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:06.033442 10699 catalog_manager.cc:5697] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 10 to 20, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 20 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:06.093533 10699 catalog_manager.cc:5697] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 18 to 21. New cstate: current_term: 21 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:19:06.094914 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.069414 10916 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:06.073408 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 56729.1us
W20260709 13:19:06.074003 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 897888us
W20260709 13:19:06.174381 10767 connection.cc:256] RPC call timeout handler was delayed by -1.1467878159999145s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1836s for a response
I20260709 13:19:06.177577 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:06.178009 11505 raft_consensus.cc:1275] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:19:06.178252 11506 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:06.178925 17899 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:06.179450 17899 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:06.180106 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:06.180852 11505 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 20.
I20260709 13:19:06.181527 17899 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:06.181999 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:06.182825 11508 raft_consensus.cc:1240] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 23. Current term is 24. Ops: []
I20260709 13:19:06.183959 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:06.182718 11505 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 16.
I20260709 13:19:06.179445 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:06.185077 11511 raft_consensus.cc:1275] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:19:06.185333 11504 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:06.184588 11503 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:19:06.083978 10842 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:19:06.191424 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 107538us
W20260709 13:19:06.086436 10686 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 2b82f18992434ed5ba58b805107060d4 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 468a8fd46e824f62b9c2c0d3532a6d26 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 20 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
W20260709 13:19:06.192090 10686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 105734us
I20260709 13:19:06.148085 10793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2525) took 1136 ms. Trace:
I20260709 13:19:06.192749 10793 rpcz_store.cc:276] 0709 13:19:05.011212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.092459 (+1081247us) service_pool.cc:224] Handling call
0709 13:19:06.146687 (+ 54228us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:06.147622 (+   935us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:06.147665 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:06.147725 (+    60us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:06.147770 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:06.147846 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:06.148013 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:06.144686 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.145413 10699 catalog_manager.cc:5697] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 12 to 18, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 18 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
W20260709 13:19:06.194772 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.195360 10699 catalog_manager.cc:5697] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 16 to 24, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 24 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:06.197270 10699 catalog_manager.cc:5697] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 11 to 18, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 18 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:19:06.111474 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.950094 17883 consensus_queue.cc:260] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:06.200734 17883 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:06.201265 11503 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 19.
I20260709 13:19:06.201992 11504 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 22.
I20260709 13:19:06.202250 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:06.202909 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:06.202909 10699 catalog_manager.cc:5697] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 14 to 18. New cstate: current_term: 18 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
W20260709 13:19:06.208885 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 139542us
I20260709 13:19:06.199818 11478 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:19:06.212292 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:06.219033 11477 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.030s	sys 0.080s
W20260709 13:19:06.220196 11477 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.030s	sys 0.080s
I20260709 13:19:06.214777 11503 raft_consensus.cc:2393] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 21: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:06.215950 10699 catalog_manager.cc:5697] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 13 to 17, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 17 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:06.216513 11506 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 18.
I20260709 13:19:06.228327 11506 raft_consensus.cc:1275] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:19:05.845779 17451 consensus_queue.cc:260] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:06.232100 11503 raft_consensus.cc:1275] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:19:06.233883 10799 raft_consensus.cc:1275] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:19:06.215497 11504 raft_consensus.cc:1813] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Handling vote request from an unknown peer cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:06.218654 11478 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:19:06.221732 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 662165us
W20260709 13:19:06.243850 10841 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.056s	sys 0.052s
I20260709 13:19:06.300607 11504 raft_consensus.cc:2393] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 19: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:19:06.300207 10841 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.058s	sys 0.058s
W20260709 13:19:06.302345 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 611024us
I20260709 13:19:06.086980 17756 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:06.087373 17861 raft_consensus.cc:2749] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:06.304837 10799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2524) took 1294 ms. Trace:
I20260709 13:19:06.305408 10799 rpcz_store.cc:276] 0709 13:19:05.010826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.077418 (+1066592us) service_pool.cc:224] Handling call
0709 13:19:06.077826 (+   408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:06.304208 (+226382us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:06.304693 (+   485us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:06.303843 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 85425.7us
I20260709 13:19:06.306703 11478 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:06.301537 11477 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:06.088340 17936 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:06.080466 17766 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:19:06.329005 10794 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.701s	user 0.048s	sys 0.064s
W20260709 13:19:06.330936 10794 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.048s	sys 0.064s
W20260709 13:19:06.143769 17977 log.cc:927] Time spent T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 13:19:06.219280 17848 raft_consensus.cc:2749] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:06.337143 11505 raft_consensus.cc:1240] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:06.227047 17919 raft_consensus.cc:3055] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:06.312173 10699 catalog_manager.cc:5697] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 2 to 20. New cstate: current_term: 20 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:19:06.215595 17899 raft_consensus.cc:2749] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:06.244702 17821 raft_consensus.cc:2804] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:19:06.339679 17758 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.708s	user 0.029s	sys 0.084s
I20260709 13:19:06.244122 17411 raft_consensus.cc:3055] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:19:06.341154 10801 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.027s	sys 0.023s
W20260709 13:19:06.395641 10801 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.033s	sys 0.026s
I20260709 13:19:06.307997 17756 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:06.227469 17932 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:06.339983 17996 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:06.398007 10795 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.793s	user 0.031s	sys 0.029s
I20260709 13:19:06.346591 17919 raft_consensus.cc:740] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:06.339223 17861 consensus_queue.cc:1048] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 13:19:06.340793 17847 raft_consensus.cc:2749] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:19:06.219643 17562 consensus_queue.cc:1175] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" responder_term: 18 status { last_received { term: 18 index: 9 } last_committed_idx: 8 last_received_current_leader { term: 18 index: 9 } }
W20260709 13:19:06.356674 17722 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.735s	user 0.022s	sys 0.013s
W20260709 13:19:06.415652 17722 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.797s	user 0.022s	sys 0.018s
I20260709 13:19:06.415931 17985 raft_consensus.cc:2804] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:06.416359 17985 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:06.416749 17985 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:06.340728 17936 raft_consensus.cc:2749] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:06.422230 17936 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:06.344015 17451 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:19:06.404507 10795 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.800s	user 0.031s	sys 0.029s
W20260709 13:19:06.360865 17758 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.735s	user 0.029s	sys 0.084s
I20260709 13:19:06.344544 17729 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:06.425633 10766 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.001s	sys 0.042s
W20260709 13:19:06.460687 10766 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.002s	sys 0.043s
W20260709 13:19:06.461019 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 684163us
I20260709 13:19:06.411263 17996 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:06.408021 17821 raft_consensus.cc:697] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:06.344558 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:06.357141 17411 raft_consensus.cc:740] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:19:06.454145 10798 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.840s	user 0.006s	sys 0.006s
W20260709 13:19:06.529551 10798 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.916s	user 0.006s	sys 0.006s
I20260709 13:19:06.354032 10699 catalog_manager.cc:5697] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 9 to 15, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 15 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:06.532207 10699 catalog_manager.cc:5697] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 15 to 19. New cstate: current_term: 19 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:19:06.397639 17965 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:06.397032 17760 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:06.404479 17756 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:19:06.348701 17856 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.017s	sys 0.019s
I20260709 13:19:06.423103 17936 raft_consensus.cc:697] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 21 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
W20260709 13:19:06.429147 11512 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.827s	user 0.003s	sys 0.053s
W20260709 13:19:06.574087 11512 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.972s	user 0.003s	sys 0.053s
I20260709 13:19:06.406808 18004 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:19:06.460003 17670 consensus_meta.cc:287] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.777s	user 0.006s	sys 0.005s
W20260709 13:19:06.569320 17987 log.cc:927] Time spent T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.551s	user 0.000s	sys 0.001s
W20260709 13:19:06.474831 17944 consensus_meta.cc:287] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.760s	user 0.008s	sys 0.004s
W20260709 13:19:06.456310 17920 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.843s	user 0.022s	sys 0.035s
W20260709 13:19:06.573231 17856 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.869s	user 0.018s	sys 0.019s
I20260709 13:19:06.637356 10867 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 21.
I20260709 13:19:06.559741 17924 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:06.638892 10767 leader_election.cc:400] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 election: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 16. Current term is 17.
I20260709 13:19:06.531814 17916 consensus_queue.cc:1048] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.054s
I20260709 13:19:06.521569 17758 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:06.608371 17920 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.995s	user 0.024s	sys 0.037s
I20260709 13:19:06.412761 17932 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:06.648543 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 429161us
I20260709 13:19:06.559617 17834 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:06.580071 17917 raft_consensus.cc:3055] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 13:19:06.598177 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.558490 10699 catalog_manager.cc:5697] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 10 to 15, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 15 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:06.527359 10799 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
I20260709 13:19:06.575449 17965 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:06.641610 10767 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:19:06.642884 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:06.571666 11509 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.514s	user 0.010s	sys 0.010s
I20260709 13:19:06.584685 18004 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:06.558048 17919 consensus_queue.cc:260] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:06.663355 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.654981 10916 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:19:06.648516 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 340346us
I20260709 13:19:06.640592 17856 consensus_queue.cc:1243] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer 468a8fd46e824f62b9c2c0d3532a6d26 log is divergent from this leader: its last log entry 19.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:19:06.571457 17760 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:06.682480 18004 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:19:06.682324 17991 log.cc:927] Time spent T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.454s	user 0.001s	sys 0.001s
I20260709 13:19:06.716136 11509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56410 (request call id 1112) took 1140 ms. Trace:
I20260709 13:19:06.740692 10795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3196) took 2191 ms. Trace:
I20260709 13:19:06.740658 11509 rpcz_store.cc:276] 0709 13:19:05.547714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:05.547905 (+   191us) service_pool.cc:224] Handling call
0709 13:19:05.576136 (+ 28231us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:05.576427 (+   291us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:19:06.057387 (+480960us) log.cc:844] Serialized 14 byte log entry
0709 13:19:06.682360 (+624973us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:06.687316 (+  4956us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:19:06.687380 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:06.687464 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:06.687609 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3393,"prepare.queue_time_us":884433,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":44,"raft.queue_time_us":32107,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":26521,"thread_start_us":1076162,"threads_started":2}
I20260709 13:19:06.716677 17808 consensus_queue.cc:1059] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:06.723109 18010 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:19:06.723626 18006 log.cc:927] Time spent T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.075s	user 0.001s	sys 0.001s
I20260709 13:19:06.648948 17883 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:06.682632 17327 consensus_queue.cc:1048] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.033s
I20260709 13:19:06.725541 17857 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:06.717025 17997 consensus_queue.cc:1048] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:19:06.647636 17944 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:06.675058 17903 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
W20260709 13:19:06.728853 17995 log.cc:927] Time spent T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.376s	user 0.002s	sys 0.000s
W20260709 13:19:06.731165 18000 log.cc:927] Time spent T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.168s	user 0.000s	sys 0.000s
I20260709 13:19:06.682964 17965 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:06.733632 17932 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:06.812433 10914 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.025s	sys 0.041s
I20260709 13:19:06.687776 17821 consensus_queue.cc:237] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.9, Last appended by leader: 9, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:06.731190 10867 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 2539) took 1726 ms (client timeout 2102 ms). Trace:
W20260709 13:19:06.814867 10867 rpcz_store.cc:269] 0709 13:19:05.004531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:05.194077 (+189546us) service_pool.cc:224] Handling call
0709 13:19:05.346008 (+151931us) spinlock_profiling.cc:243] Waited 8.88 ms on lock 0x7b5c0004d38c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c8481f161 00007f9c849c0a79 00007f9c85e7a9b6 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:06.730980 (+1384972us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54002d3f54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:06.731116 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2922788608}
W20260709 13:19:06.715605 18005 log.cc:927] Time spent T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
W20260709 13:19:06.737684 11478 connection.cc:256] RPC call timeout handler was delayed by -1.5400916240000697s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2198s for a response
I20260709 13:19:06.741269 10795 rpcz_store.cc:276] 0709 13:19:04.543373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.608194 (+ 64821us) service_pool.cc:224] Handling call
0709 13:19:04.608576 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:05.076768 (+468192us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:19:05.138816 (+ 62048us) log.cc:844] Serialized 14 byte log entry
0709 13:19:05.597207 (+458391us) spinlock_profiling.cc:243] Waited 43.9 ms on lock 0x7b1800393a60. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7dd42c0a 00007f9c87482219 00007f9c84d8958f 00007f9c7ddc8571 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd
0709 13:19:05.604464 (+  7257us) spinlock_profiling.cc:243] Waited 7.07 ms on lock 0x7b1800393a60. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a5523a 00007f9c80ebfa2a
0709 13:19:06.469219 (+864755us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:06.530624 (+ 61405us) log.cc:844] Serialized 26 byte log entry
0709 13:19:06.530753 (+   129us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:06.530939 (+   186us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:06.531006 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:06.734979 (+203973us) raft_consensus.cc:1685] finished
0709 13:19:06.735004 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:06.735142 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":613092,"prepare.queue_time_us":38820,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":9463,"raft.queue_time_us":487219,"raft.run_cpu_time_us":272,"raft.run_wall_time_us":12126,"spinlock_wait_cycles":117272576,"thread_start_us":849387,"threads_started":1,"wal-append.queue_time_us":1430828}
W20260709 13:19:06.816212 11478 connection.cc:256] RPC call timeout handler was delayed by -1.514891624000029s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22260000000000002s for a response
W20260709 13:19:06.816720 11478 leader_election.cc:336] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Timed out: RequestConsensusVote RPC to 127.9.116.129:39685 timed out after 1.684s (SENT)
I20260709 13:19:06.748553 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:06.816895 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:06.817149 11478 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:06.746752 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.752102 18021 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:19:06.819453 11478 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.116.129:39685 timed out after 1.829s (SENT) [suppressed 25 similar messages]
I20260709 13:19:06.752013 17729 consensus_queue.cc:1048] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:19:06.753710 10800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3192) took 2214 ms. Trace:
I20260709 13:19:06.823320 10800 rpcz_store.cc:276] 0709 13:19:04.539330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.539471 (+   141us) service_pool.cc:224] Handling call
0709 13:19:04.539837 (+   366us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:05.099161 (+559324us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:05.099217 (+    56us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:05.100049 (+   832us) log.cc:844] Serialized 52 byte log entry
0709 13:19:05.649320 (+549271us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:05.649362 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:05.649430 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:06.753408 (+1103978us) raft_consensus.cc:1685] finished
0709 13:19:06.753439 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:06.753636 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":611809,"raft.queue_time_us":554878,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":21567,"spinlock_wait_cycles":1229568,"thread_start_us":448440,"threads_started":1,"wal-append.queue_time_us":705668}
I20260709 13:19:06.825742 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:06.826237 10800 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:06.645727 17924 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:06.775701 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 114390us
I20260709 13:19:06.749123 17856 consensus_queue.cc:1048] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
W20260709 13:19:06.746104 17836 consensus_meta.cc:287] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.905s	user 0.014s	sys 0.000s
I20260709 13:19:06.752179 18011 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:06.649623 17758 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:06.740294 17919 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:06.740032 17899 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:06.683727 17834 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:06.645987 17411 consensus_queue.cc:260] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:06.748298 18010 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:06.808485 10798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3203) took 2082 ms. Trace:
I20260709 13:19:06.653542 17917 raft_consensus.cc:740] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:19:06.812988 10914 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.026s	sys 0.041s
I20260709 13:19:06.780993 17808 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:06.817598 10799 raft_consensus.cc:2410] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 24 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 12.
I20260709 13:19:06.751550 10797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3197) took 2208 ms. Trace:
W20260709 13:19:06.807461 17891 consensus_meta.cc:287] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.932s	user 0.006s	sys 0.005s
I20260709 13:19:06.818883 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:06.820946 10916 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:06.809820 17944 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:06.909094 10942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 3146) took 1819 ms. Trace:
W20260709 13:19:06.834298 11478 connection.cc:256] RPC call timeout handler was delayed by -1.47109285800002s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1899s for a response
I20260709 13:19:06.801604 17883 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:06.804224 17928 consensus_queue.cc:1048] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.026s
W20260709 13:19:06.828490 11477 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.003s	sys 0.004s
W20260709 13:19:06.886171 17877 consensus_meta.cc:287] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 1.745s	user 0.015s	sys 0.000s
I20260709 13:19:06.857335 10796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3202) took 2132 ms. Trace:
I20260709 13:19:06.827386 17932 raft_consensus.cc:2749] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:06.862058 10800 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 23.
I20260709 13:19:06.804039 17857 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:06.855793 17996 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:06.748684 17936 consensus_queue.cc:237] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:06.840567 17896 consensus_meta.cc:287] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 1.159s	user 0.012s	sys 0.000s
I20260709 13:19:06.907378 10795 raft_consensus.cc:2410] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 24 because replica has last-logged OpId of term: 23 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 15.
I20260709 13:19:06.907887 10797 rpcz_store.cc:276] 0709 13:19:04.543537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.612036 (+ 68499us) service_pool.cc:224] Handling call
0709 13:19:04.612381 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:05.077070 (+464689us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:05.077114 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:05.100516 (+ 23402us) log.cc:844] Serialized 26 byte log entry
0709 13:19:05.504649 (+404133us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:05.504693 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:05.504766 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:06.751313 (+1246547us) raft_consensus.cc:1685] finished
0709 13:19:06.751335 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:06.751486 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":547148,"raft.queue_time_us":499196,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":115,"thread_start_us":403714,"threads_started":1,"wal-append.queue_time_us":727705}
W20260709 13:19:06.911072 11478 connection.cc:256] RPC call timeout handler was delayed by -1.1724853840000833s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16820000000000002s for a response
I20260709 13:19:06.844409 18013 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:06.946420 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:06.947320 11503 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 23.
I20260709 13:19:06.892884 10798 rpcz_store.cc:276] 0709 13:19:04.725583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:05.297069 (+571486us) service_pool.cc:224] Handling call
0709 13:19:05.297466 (+   397us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:05.612863 (+315397us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:05.612897 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:05.613373 (+   476us) log.cc:844] Serialized 26 byte log entry
0709 13:19:06.530123 (+916750us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:06.530187 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:06.530305 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:06.808257 (+277952us) raft_consensus.cc:1685] finished
0709 13:19:06.808278 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:06.808420 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":314972,"raft.queue_time_us":317555,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":2346,"thread_start_us":916332,"threads_started":1,"wal-append.queue_time_us":1026701}
I20260709 13:19:06.950006 10798 raft_consensus.cc:1240] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:06.950867 10798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2532) took 1340 ms. Trace:
I20260709 13:19:06.951285 10798 rpcz_store.cc:276] 0709 13:19:05.610759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.949376 (+1338617us) service_pool.cc:224] Handling call
0709 13:19:06.949765 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:06.950495 (+   730us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:06.950768 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:06.914050 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.909503 10942 rpcz_store.cc:276] 0709 13:19:05.089445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:05.089587 (+   142us) service_pool.cc:224] Handling call
0709 13:19:05.089920 (+   333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:05.095426 (+  5506us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:05.095466 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:05.112086 (+ 16620us) log.cc:844] Serialized 26 byte log entry
0709 13:19:06.144396 (+1032310us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:06.144440 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:06.144516 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:06.908807 (+764291us) raft_consensus.cc:1685] finished
0709 13:19:06.908833 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:06.909026 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5107,"raft.queue_time_us":5663,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":5784,"thread_start_us":954173,"threads_started":1,"wal-append.queue_time_us":1294296}
I20260709 13:19:06.954478 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:06.959592 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:06.830351 18021 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:06.833535 10840 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.000s	sys 0.007s
W20260709 13:19:06.988241 10840 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.756s	user 0.000s	sys 0.007s
W20260709 13:19:06.902043 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.852113 10767 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:19:06.988592 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:06.880126 10841 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.015s	sys 0.028s
W20260709 13:19:06.918313 11477 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.003s	sys 0.004s
I20260709 13:19:06.922817 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:19:06.989007 10841 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.668s	user 0.016s	sys 0.029s
W20260709 13:19:06.989456 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 687949us
I20260709 13:19:06.989634 10800 raft_consensus.cc:2410] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 22 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 15.
W20260709 13:19:06.989936 10841 connection.cc:256] RPC call timeout handler was delayed by -1.0228349119997802s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2057s for a response
I20260709 13:19:06.855499 18026 raft_consensus.cc:2749] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:06.809945 17903 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:06.926116 10876 raft_consensus.cc:2410] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 27 because replica has last-logged OpId of term: 26 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 12.
I20260709 13:19:06.994903 10800 raft_consensus.cc:1275] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 13. (term mismatch)
I20260709 13:19:06.995285 10800 pending_rounds.cc:85] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 12
I20260709 13:19:06.995522 10800 pending_rounds.cc:107] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 22.13
W20260709 13:19:06.926285 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 105395us
W20260709 13:19:06.910096 18003 log.cc:927] Time spent T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.258s	user 0.000s	sys 0.001s
W20260709 13:19:06.911778 17819 consensus_meta.cc:287] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 1.718s	user 0.011s	sys 0.004s
I20260709 13:19:06.996548 10916 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:06.934521 10797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2531) took 1324 ms. Trace:
I20260709 13:19:07.015134 10797 rpcz_store.cc:276] 0709 13:19:05.610515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.933453 (+1322938us) service_pool.cc:224] Handling call
0709 13:19:06.933873 (+   420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:06.934099 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:06.934136 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:06.934194 (+    58us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:19:06.934239 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:06.934312 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:06.934447 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:07.017422 10797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3218) took 1362 ms. Trace:
I20260709 13:19:07.017833 10797 rpcz_store.cc:276] 0709 13:19:05.655587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:07.016415 (+1360828us) service_pool.cc:224] Handling call
0709 13:19:07.016789 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:07.016999 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:07.017034 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:07.017091 (+    57us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:07.017137 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:07.017215 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.017355 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:06.938264 11504 raft_consensus.cc:1240] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:06.939733 11506 raft_consensus.cc:1240] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:06.943019 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:06.945228 10794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3206) took 2210 ms. Trace:
I20260709 13:19:07.020447 10794 rpcz_store.cc:276] 0709 13:19:04.734368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:05.376533 (+642165us) service_pool.cc:224] Handling call
0709 13:19:05.376914 (+   381us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:05.613317 (+236403us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:05.613362 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:05.614009 (+   647us) log.cc:844] Serialized 52 byte log entry
0709 13:19:05.831015 (+217006us) spinlock_profiling.cc:243] Waited 2.17 ms on lock 0x7b48002e0158. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c85ddeaa1 00007f9c85dd5b37 00007f9c85dd59ca 00007f9c85ecd26f 00007f9c85eccf37 00007f9c85ee155c 00007f9c85ee1508 00007f9c85ee1488 00007f9c85ee1443 00007f9c85ee06ec 00007f9c84e0aa96
0709 13:19:06.333280 (+502265us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:06.333346 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:06.333448 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:06.944926 (+611478us) raft_consensus.cc:1685] finished
0709 13:19:06.944947 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:06.945149 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":235976,"raft.queue_time_us":238174,"raft.run_cpu_time_us":4675,"raft.run_wall_time_us":213527,"spinlock_wait_cycles":4997376,"thread_start_us":704964,"threads_started":1,"wal-append.queue_time_us":906647}
I20260709 13:19:06.869979 17899 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:06.928117 17327 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:19:06.961522 17766 consensus_meta.cc:287] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.624s	user 0.011s	sys 0.000s
I20260709 13:19:06.964859 10795 raft_consensus.cc:1240] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 13:19:06.901580 18010 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:06.971371 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:07.091629 11511 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:07.092191 17722 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.092805 17722 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:06.989012 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.095062 17899 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:07.095130 17722 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:07.095747 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.097342 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:06.979584 10799 raft_consensus.cc:1275] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:19:07.098125 10869 raft_consensus.cc:2393] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 23: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:07.098214 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:07.098532 10799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3215) took 1465 ms. Trace:
I20260709 13:19:07.098883 10867 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 23.
I20260709 13:19:07.099254 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:07.099736 11512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56410 (request call id 1114) took 1551 ms. Trace:
I20260709 13:19:07.100358 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:07.101084 10873 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 24.
I20260709 13:19:07.100071 11512 rpcz_store.cc:276] 0709 13:19:05.551527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:05.555920 (+  4393us) service_pool.cc:224] Handling call
0709 13:19:05.598240 (+ 42320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:05.601095 (+  2855us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:19:06.574548 (+973453us) log.cc:844] Serialized 14 byte log entry
0709 13:19:06.815762 (+241214us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:06.825923 (+ 10161us) log.cc:844] Serialized 26 byte log entry
0709 13:19:06.826345 (+   422us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:06.826377 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:06.826440 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:07.099550 (+273110us) raft_consensus.cc:1685] finished
0709 13:19:07.099570 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.099686 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2439,"prepare.queue_time_us":963859,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":66,"raft.queue_time_us":104332,"raft.run_cpu_time_us":324,"raft.run_wall_time_us":32305,"thread_start_us":1139424,"threads_started":2,"wal-append.queue_time_us":486389}
I20260709 13:19:07.101985 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:06.920540 10796 rpcz_store.cc:276] 0709 13:19:04.725232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.725370 (+   138us) service_pool.cc:224] Handling call
0709 13:19:04.744083 (+ 18713us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:05.118543 (+374460us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:05.118572 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:05.119079 (+   507us) log.cc:844] Serialized 26 byte log entry
0709 13:19:05.369289 (+250210us) spinlock_profiling.cc:243] Waited 1.25 ms on lock 0x7b1800393ac0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a550d1 00007f9c80e70a0f
0709 13:19:05.852572 (+483283us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:05.852615 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:05.852684 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:06.728200 (+875516us) spinlock_profiling.cc:243] Waited 1.42 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:19:06.857046 (+128846us) raft_consensus.cc:1685] finished
0709 13:19:06.857080 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:06.857254 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":806586,"raft.queue_time_us":374856,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":10127,"spinlock_wait_cycles":6151296,"thread_start_us":483084,"threads_started":1,"wal-append.queue_time_us":1089109}
I20260709 13:19:06.985483 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:07.103529 11503 raft_consensus.cc:2393] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 23: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:06.995779 10800 raft_consensus.cc:2889] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:07.104593 10800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3217) took 1456 ms. Trace:
I20260709 13:19:07.098902 10799 rpcz_store.cc:276] 0709 13:19:05.633201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.955573 (+1322372us) service_pool.cc:224] Handling call
0709 13:19:06.979192 (+ 23619us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:07.098290 (+119098us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:07.098480 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:07.105047 10800 rpcz_store.cc:276] 0709 13:19:05.648566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.994216 (+1345650us) service_pool.cc:224] Handling call
0709 13:19:06.994531 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:07.104351 (+109820us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:07.104542 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:06.914876 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.999918 10947 raft_consensus.cc:1275] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:19:07.054365 17916 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:07.001742 10942 raft_consensus.cc:1240] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:07.002707 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:07.109229 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:07.109843 10874 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 19.
I20260709 13:19:06.943101 18024 consensus_queue.cc:1048] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.154s
W20260709 13:19:06.971349 18015 log.cc:927] Time spent T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 13:19:07.025429 10945 raft_consensus.cc:1275] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:19:06.910377 17808 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:07.028317 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 176310us
I20260709 13:19:07.132550 10767 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:06.848523 18011 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:07.135815 11511 raft_consensus.cc:2410] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 20 because replica has last-logged OpId of term: 19 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 15.
W20260709 13:19:06.972061 18008 log.cc:927] Time spent T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.320s	user 0.000s	sys 0.001s
I20260709 13:19:07.056079 10795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3216) took 1419 ms. Trace:
I20260709 13:19:07.139118 10795 rpcz_store.cc:276] 0709 13:19:05.636583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.964140 (+1327557us) service_pool.cc:224] Handling call
0709 13:19:06.964552 (+   412us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:07.055802 (+ 91250us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:07.056024 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:06.971134 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:07.056336 10797 raft_consensus.cc:1275] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 14. (term mismatch)
I20260709 13:19:07.141569 10797 pending_rounds.cc:85] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 13
I20260709 13:19:07.141975 10797 pending_rounds.cc:107] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 18.14
I20260709 13:19:07.142328 10797 raft_consensus.cc:2889] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:07.030825 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:07.057922 18020 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:07.058454 17927 consensus_meta.cc:287] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 1.239s	user 0.012s	sys 0.000s
W20260709 13:19:07.059429 18014 log.cc:927] Time spent T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 13:19:06.917789 17891 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:07.062814 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 66333.7us
I20260709 13:19:07.168188 10916 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:07.168924 10767 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:07.031823 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:07.065106 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:07.065879 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:07.066118 10944 raft_consensus.cc:1275] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
W20260709 13:19:07.069594 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.994051 17670 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.070201 11510 raft_consensus.cc:1275] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
W20260709 13:19:07.013223 17985 consensus_meta.cc:287] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.596s	user 0.012s	sys 0.000s
I20260709 13:19:07.070024 17996 raft_consensus.cc:2804] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:07.132325 18022 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.186065 10870 raft_consensus.cc:1275] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 13:19:07.077442 18025 log.cc:927] Time spent T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.163s	user 0.000s	sys 0.000s
I20260709 13:19:07.078095 17763 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.070152 18019 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.060460 17756 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:06.961342 18033 consensus_queue.cc:1059] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.140s
I20260709 13:19:07.079958 17760 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:07.025321 17877 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.273322 17763 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:07.284745 17877 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:07.286274 17763 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:07.088951 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:07.166905 18020 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:06.999032 10798 raft_consensus.cc:1275] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:19:07.003628 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:07.202800 17670 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:07.111099 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:07.061663 17819 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.288776 18034 consensus_queue.cc:1048] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.070s
I20260709 13:19:07.288868 17819 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:07.290019 17670 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:07.177119 17840 raft_consensus.cc:2749] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:19:07.291066 17819 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:07.109350 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:07.291821 17282 consensus_meta.cc:287] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 1.504s	user 0.012s	sys 0.000s
I20260709 13:19:07.023658 10794 raft_consensus.cc:1217] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Deduplicated request from leader. Original: 16.11->[17.12-22.15]   Dedup: 18.13->[20.14-22.15]
I20260709 13:19:07.292407 10794 pending_rounds.cc:85] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 13
I20260709 13:19:07.292646 10794 pending_rounds.cc:107] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 19.14
I20260709 13:19:07.117904 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:07.292887 10794 raft_consensus.cc:2889] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:07.293784 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:07.294385 10874 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 23.
I20260709 13:19:07.295269 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:07.295816 10874 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 24.
I20260709 13:19:06.996455 17932 consensus_queue.cc:1059] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.092s
W20260709 13:19:07.111548 18009 log.cc:927] Time spent T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.331s	user 0.000s	sys 0.001s
I20260709 13:19:07.144296 11506 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 16.
W20260709 13:19:07.145798 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 159926us
I20260709 13:19:07.311079 11478 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:07.146770 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:07.147768 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.183629 17996 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:07.170821 10950 raft_consensus.cc:2410] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 24 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
I20260709 13:19:07.171283 11504 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 21.
W20260709 13:19:07.160133 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 163958us
I20260709 13:19:07.183040 10800 raft_consensus.cc:1240] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 18. Current term is 19. Ops: [18.15-18.15]
W20260709 13:19:07.316198 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.316968 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:07.187350 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:07.227311 17924 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:06.965003 18013 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:07.194989 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:07.321265 10916 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:07.322062 10870 raft_consensus.cc:2393] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 21: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:07.195377 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:07.322273 10916 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:19:07.310765 18008 log.cc:927] Time spent T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
I20260709 13:19:07.202445 10795 raft_consensus.cc:1275] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:19:07.203328 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:07.229810 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:07.231575 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.148697 17603 raft_consensus.cc:3055] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:19:07.272764 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:07.273320 11507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56404 (request call id 1175) took 1098 ms. Trace:
I20260709 13:19:07.309355 17821 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.133325 17899 consensus_queue.cc:1048] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.084s
I20260709 13:19:07.318625 17996 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:07.352470 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:19:07.280316 17451 consensus_meta.cc:287] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.823s	user 0.011s	sys 0.000s
I20260709 13:19:07.283133 18036 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:07.283859 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:07.287158 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:07.354647 10944 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 24.
I20260709 13:19:07.288201 10948 raft_consensus.cc:2410] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 22 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 13:19:07.061125 17729 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.288277 17758 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:07.103852 17944 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:07.296603 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:07.176764 17928 raft_consensus.cc:2749] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:19:07.192261 17965 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:07.312731 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:07.365643 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.157694 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:07.315006 11478 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:07.317946 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.317848 10873 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 18.
I20260709 13:19:07.192867 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:07.319662 17847 consensus_queue.cc:1048] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:07.286664 17877 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:07.322688 10875 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 21.
I20260709 13:19:07.195683 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:07.325701 10872 raft_consensus.cc:2410] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 20 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:19:07.327034 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:07.392118 10797 raft_consensus.cc:1813] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Handling vote request from an unknown peer cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:07.327673 10870 raft_consensus.cc:1275] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 13:19:07.392352 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 77433.1us
I20260709 13:19:07.393095 11478 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:07.326503 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:07.267593 18022 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:07.265812 18019 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:07.307978 18011 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:07.203276 17985 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:07.333788 10792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3212) took 2445 ms. Trace:
I20260709 13:19:07.334918 18013 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:07.219561 17856 consensus_queue.cc:1059] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.101s
I20260709 13:19:07.323246 18037 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.336335 17834 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:07.336338 17891 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:07.218530 17722 consensus_queue.cc:1048] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.102s
I20260709 13:19:07.320101 18026 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:07.330024 11507 rpcz_store.cc:276] 0709 13:19:06.175035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.175208 (+   173us) service_pool.cc:224] Handling call
0709 13:19:06.175659 (+   451us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:06.176098 (+   439us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:06.176136 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:06.176686 (+   550us) log.cc:844] Serialized 26 byte log entry
0709 13:19:06.561264 (+384578us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:19:06.561305 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:06.561385 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:07.273109 (+711724us) raft_consensus.cc:1685] finished
0709 13:19:07.273128 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.273256 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6030,"raft.run_cpu_time_us":73,"raft.run_wall_time_us":73,"thread_start_us":384190,"threads_started":1,"wal-append.queue_time_us":630560}
I20260709 13:19:07.334988 10801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3204) took 2609 ms. Trace:
I20260709 13:19:07.279522 17857 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:07.330304 18030 log.cc:927] Time spent T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.182s	user 0.000s	sys 0.000s
I20260709 13:19:07.361501 10874 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 24.
I20260709 13:19:07.329771 17282 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:07.329690 17603 raft_consensus.cc:740] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:07.374573 11505 raft_consensus.cc:1275] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:19:07.374904 17758 consensus_queue.cc:1048] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:19:07.376766 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:07.452854 10944 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
I20260709 13:19:07.377310 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:07.378382 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:07.453963 10841 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:07.454514 10947 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
I20260709 13:19:07.455482 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:07.456524 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:07.457299 10947 raft_consensus.cc:2410] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 23 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 13:19:07.458038 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:07.380000 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:07.459123 10946 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 27.
I20260709 13:19:07.381541 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:07.382226 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:07.388640 10942 raft_consensus.cc:2393] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 18: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:07.328691 11504 raft_consensus.cc:1275] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:19:07.461275 10941 raft_consensus.cc:2410] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 19 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
W20260709 13:19:07.395174 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.462133 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:07.397055 10797 raft_consensus.cc:2393] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 19: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:07.463744 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:07.464607 10797 raft_consensus.cc:2410] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 23 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
W20260709 13:19:07.465119 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.465500 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:07.400410 10766 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:07.406991 11478 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
W20260709 13:19:07.412724 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 236870us
I20260709 13:19:07.467283 17857 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:07.413023 10792 rpcz_store.cc:276] 0709 13:19:04.888462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.055349 (+1166887us) service_pool.cc:224] Handling call
0709 13:19:06.055798 (+   449us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:06.074181 (+ 18383us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:06.074226 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:06.074832 (+   606us) log.cc:844] Serialized 26 byte log entry
0709 13:19:06.581373 (+506541us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:06.581418 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:06.581492 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:07.333551 (+752059us) raft_consensus.cc:1685] finished
0709 13:19:07.333572 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.333720 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":87929,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":24919,"thread_start_us":272978,"threads_started":1,"wal-append.queue_time_us":599865}
I20260709 13:19:07.467754 10767 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:07.466660 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:07.468364 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:07.469554 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:07.469545 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:07.470628 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:07.471025 18038 raft_consensus.cc:2804] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:07.471474 10792 raft_consensus.cc:2410] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 21 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:19:07.471448 18038 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:07.471964 18038 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:07.416520 10916 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:07.371848 17451 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:19:07.389029 17883 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:07.438750 10842 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:07.351432 17821 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:07.396914 17899 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:07.444229 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:07.396057 17903 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:07.362936 17729 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:07.470247 10767 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:07.473229 17451 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:07.475778 17451 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:07.476428 17729 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:07.476475 18011 raft_consensus.cc:2749] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:07.413339 18044 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:07.461148 10943 raft_consensus.cc:2410] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 22 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 13:19:07.273574 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:07.478506 10799 raft_consensus.cc:2410] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 26 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
W20260709 13:19:07.467178 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 66850.8us
I20260709 13:19:07.474165 18013 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:19:07.479192 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.431758 10801 rpcz_store.cc:276] 0709 13:19:04.725766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:05.360516 (+634750us) service_pool.cc:224] Handling call
0709 13:19:05.364870 (+  4354us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:05.619718 (+254848us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:19:05.649780 (+ 30062us) log.cc:844] Serialized 14 byte log entry
0709 13:19:06.398132 (+748352us) log.cc:844] Serialized 14 byte log entry
0709 13:19:06.398368 (+   236us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:06.399120 (+   752us) log.cc:844] Serialized 52 byte log entry
0709 13:19:06.399464 (+   344us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:06.399511 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:06.399604 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:06.725267 (+325663us) spinlock_profiling.cc:243] Waited 2.83 ms on lock 0x7b48002e0158. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c85ddeaa1 00007f9c85dd5b37 00007f9c85dd59ca 00007f9c85ecd26f 00007f9c85eccf37 00007f9c85ee155c 00007f9c85ee1508 00007f9c85ee1488 00007f9c85ee1443 00007f9c85ee06ec 00007f9c84e0aa96
0709 13:19:07.334755 (+609488us) raft_consensus.cc:1685] finished
0709 13:19:07.334797 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.334930 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":244778,"prepare.queue_time_us":34748,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":95,"raft.queue_time_us":252417,"raft.run_cpu_time_us":8423,"raft.run_wall_time_us":330204,"spinlock_wait_cycles":6519680,"thread_start_us":700962,"threads_started":1,"wal-append.queue_time_us":875341}
W20260709 13:19:07.437399 17842 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.488s	user 0.017s	sys 0.022s
I20260709 13:19:07.450989 17927 consensus_queue.cc:1059] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 19.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:19:07.467303 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 60409.3us
I20260709 13:19:07.481220 17899 raft_consensus.cc:2749] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:07.472777 10792 raft_consensus.cc:1275] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:19:07.484894 10767 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:07.486045 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:07.475232 17857 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:07.491096 11478 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:19:07.431440 18026 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
W20260709 13:19:07.496498 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 57830.7us
I20260709 13:19:07.500419 17903 consensus_queue.cc:1048] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:07.443543 17819 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.531420 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:07.516109 10767 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:07.446085 17282 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:07.495146 17857 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:07.501368 18044 raft_consensus.cc:2749] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:19:07.489845 17842 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.541s	user 0.018s	sys 0.022s
I20260709 13:19:07.496112 17766 raft_consensus.cc:2749] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:07.534433 10767 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:07.363768 18036 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:07.432040 17327 consensus_queue.cc:1048] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.064s
I20260709 13:19:07.451195 17891 consensus_queue.cc:1048] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
W20260709 13:19:07.537194 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 46002.5us
I20260709 13:19:07.450351 18021 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:07.564646 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:07.567764 17927 raft_consensus.cc:3055] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:07.567826 18044 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:19:07.568295 17927 raft_consensus.cc:740] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:07.568960 17985 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:07.569425 18042 log.cc:927] Time spent T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 13:19:07.569270 17927 consensus_queue.cc:260] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:07.569995 17603 consensus_queue.cc:260] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:07.571118 17603 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:07.571686 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:07.572279 10792 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:07.438033 18037 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:07.570554 17927 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:07.579445 11478 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
W20260709 13:19:07.594966 10915 connection.cc:256] RPC call timeout handler was delayed by -1.3844407929998397s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15660000000000002s for a response
W20260709 13:19:07.576112 17834 consensus_queue.cc:1175] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "12682eb70bc84c76879e04a8628ee586" responder_term: 22 status { last_received { term: 21 index: 16 } last_committed_idx: 16 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 21. Current term is 22. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:19:07.600708 17985 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:07.602402 17985 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:07.602861 17985 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:07.606962 18046 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.607570 18046 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:07.609838 18046 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:07.564617 17622 raft_consensus.cc:2749] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:07.618145 17756 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:07.616670 18041 log.cc:927] Time spent T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.245s	user 0.000s	sys 0.001s
W20260709 13:19:07.620074 17994 log.cc:927] Time spent T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 1.205s	user 0.005s	sys 0.007s
I20260709 13:19:07.531571 17883 consensus_queue.cc:1048] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.089s
I20260709 13:19:07.541813 17819 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:07.627614 11478 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:07.532397 18026 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:07.531689 17808 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:07.542393 17965 raft_consensus.cc:2804] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:07.655198 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:07.568536 18044 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:07.543008 17282 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:07.501978 17729 raft_consensus.cc:2804] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:19:07.643921 17919 consensus_meta.cc:287] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.774s	user 0.010s	sys 0.000s
I20260709 13:19:07.614598 18013 raft_consensus.cc:2749] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:19:07.662163 17916 consensus_meta.cc:287] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.539s	user 0.002s	sys 0.008s
I20260709 13:19:07.658861 17997 raft_consensus.cc:2749] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:07.659161 18044 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:07.630604 18010 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:07.688242 18038 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:07.692397 18044 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:07.675362 18037 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:07.694087 18019 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:07.694240 18044 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:07.697520 18004 consensus_meta.cc:287] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.944s	user 0.010s	sys 0.000s
I20260709 13:19:07.697551 18026 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:07.699343 18010 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:07.699991 18004 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:07.734340 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 169044us
I20260709 13:19:07.736857 10766 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:07.738588 10766 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:19:07.739398 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:07.759944 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.760215 10799 raft_consensus.cc:1275] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 13:19:07.760852 10799 pending_rounds.cc:85] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 12
I20260709 13:19:07.761169 10799 pending_rounds.cc:107] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 13:19:07.761571 10799 raft_consensus.cc:2889] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:19:07.773453 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 242416us
I20260709 13:19:07.775056 10842 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:07.775475 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:07.781227 18082 consensus_queue.cc:1048] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:19:07.654583 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:07.657725 11506 raft_consensus.cc:1275] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:19:07.656085 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:07.652482 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:07.813249 11510 raft_consensus.cc:2410] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 21 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
I20260709 13:19:07.815917 10871 raft_consensus.cc:1240] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:07.817175 18053 raft_consensus.cc:2804] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:07.817935 18053 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:07.818472 18053 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:07.824136 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 289724us
W20260709 13:19:07.824754 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 408145us
I20260709 13:19:07.825089 17819 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:07.825745 10767 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:07.662652 17729 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:07.827363 17729 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:07.829123 17451 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:07.830946 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.831372 10947 raft_consensus.cc:2393] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 25: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:07.831765 18013 raft_consensus.cc:2804] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:07.832247 18013 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:07.832297 17942 raft_consensus.cc:2749] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:19:07.832747 18013 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:07.830296 10792 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
I20260709 13:19:07.669412 17916 raft_consensus.cc:2749] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:07.847081 10800 raft_consensus.cc:2410] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 20 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:19:07.798252 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:07.805755 17903 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.848246 11504 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 19.
I20260709 13:19:07.848383 17903 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:07.806672 18019 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.807531 18029 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.851332 18029 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:07.853142 10868 raft_consensus.cc:1240] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:07.851373 18019 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:07.856721 18029 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:07.858935 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:07.860062 11504 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 19.
I20260709 13:19:07.861620 10767 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:19:07.775497 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:07.841873 17942 raft_consensus.cc:2749] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:07.843307 11507 raft_consensus.cc:2393] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 25: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:07.700613 17996 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:07.860586 17903 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:07.864992 17579 raft_consensus.cc:3055] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:07.811442 17883 raft_consensus.cc:2749] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:07.839639 17985 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:07.798233 18004 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:07.873420 17996 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:07.874151 18044 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.877198 17917 consensus_queue.cc:260] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:07.878492 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 103724us
I20260709 13:19:07.881759 17579 raft_consensus.cc:740] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:07.883051 10872 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:07.892834 17579 consensus_queue.cc:260] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:07.895853 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:07.897986 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:07.883352 17932 raft_consensus.cc:2749] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:07.898037 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:07.899780 10873 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:07.899919 18044 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:07.901605 18038 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:07.903065 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.904510 18026 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:07.908504 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:07.910218 10766 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:07.911187 18026 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:07.911685 17917 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:07.912151 10946 raft_consensus.cc:1275] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:19:07.910693 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:07.907058 17985 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:07.907051 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.912072 18026 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:07.927101 18019 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:07.933051 17579 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:07.930605 17451 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:07.991799 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:19:07.992072 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 130838us
I20260709 13:19:07.992738 10868 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:07.993315 10767 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:07.995285 18039 consensus_queue.cc:1059] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:19:07.996693 18039 raft_consensus.cc:3055] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:19:07.997102 18039 raft_consensus.cc:740] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:07.999112 17920 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:07.999709 17920 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:08.000147 17920 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:08.000468 17924 raft_consensus.cc:2749] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:08.002655 18026 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:08.005363 18039 consensus_queue.cc:260] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:08.007179 17819 consensus_queue.cc:1048] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:08.007531 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:08.007977 18022 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:08.009187 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:08.009967 10875 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:08.009678 17819 consensus_queue.cc:1048] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:19:08.010964 18080 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:08.011508 18080 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:08.011921 18080 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:08.017910 18036 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:08.019085 18020 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:07.976550 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.982942 10794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 982) took 1285 ms. Trace:
W20260709 13:19:08.036391 18083 log.cc:927] Time spent T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
W20260709 13:19:08.038897 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 45858us
I20260709 13:19:08.040375 10767 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:08.041870 10867 raft_consensus.cc:1275] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:19:08.044802 17282 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:08.047879 10842 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:08.051275 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.052147 11507 raft_consensus.cc:1240] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:19:08.052806 18039 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:08.062104 10875 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 20.
I20260709 13:19:08.067857 10873 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 25.
I20260709 13:19:08.073829 10842 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:19:08.075551 17603 consensus_meta.cc:287] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.504s	user 0.008s	sys 0.010s
I20260709 13:19:08.042030 17932 raft_consensus.cc:2749] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:19:08.076756 10767 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:08.077524 18029 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:08.078999 18026 consensus_queue.cc:1048] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:19:08.084393 18054 raft_consensus.cc:3055] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:19:08.084959 18054 raft_consensus.cc:740] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:08.087049 18034 consensus_queue.cc:1059] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:19:08.088377 18034 raft_consensus.cc:2749] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:08.089349 18034 raft_consensus.cc:3055] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:08.089929 18034 raft_consensus.cc:740] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:08.025297 18044 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:08.098237 10872 raft_consensus.cc:2468] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 20.
I20260709 13:19:08.100194 10868 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 21.
W20260709 13:19:08.098587 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 153511us
I20260709 13:19:08.104038 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:08.104694 10873 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 21.
I20260709 13:19:08.112273 10942 raft_consensus.cc:2410] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 22 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
W20260709 13:19:08.113250 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.113605 17282 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:08.115587 18053 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:08.116866 10876 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 2535) took 3113 ms (client timeout 2157 ms). Trace:
W20260709 13:19:08.117331 10876 rpcz_store.cc:269] 0709 13:19:05.003363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:05.005873 (+  2510us) service_pool.cc:224] Handling call
0709 13:19:08.116405 (+3110532us) spinlock_profiling.cc:243] Waited 1.8 s on lock 0x7b540006d6d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:08.116645 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4148468992}
W20260709 13:19:08.118747 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.086238 18054 consensus_queue.cc:260] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:08.120747 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.121116 18054 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:08.123131 17729 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:08.091066 18029 raft_consensus.cc:697] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:08.126574 10798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2534) took 2511 ms. Trace:
I20260709 13:19:08.127038 18029 consensus_queue.cc:237] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:08.129429 17821 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:08.132539 17965 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:08.133203 17965 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:08.127197 10798 rpcz_store.cc:276] 0709 13:19:05.614607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.952282 (+1337675us) service_pool.cc:224] Handling call
0709 13:19:06.952639 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:07.287683 (+335044us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:08.125957 (+838274us) spinlock_profiling.cc:243] Waited 45.2 ms on lock 0x7b54002120d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:19:08.126344 (+   387us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":104034048}
W20260709 13:19:08.136230 17927 consensus_meta.cc:287] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.505s	user 0.007s	sys 0.007s
I20260709 13:19:08.148342 17891 raft_consensus.cc:2955] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Committing config change with OpId 17.13: config changed from index 8 to 13, VOTER cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
W20260709 13:19:08.160483 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.161643 17896 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:08.162424 17896 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:08.165167 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:08.166064 10948 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
I20260709 13:19:08.167147 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:08.169299 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 706323us
I20260709 13:19:08.170050 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:08.170864 10943 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
I20260709 13:19:08.171115 10841 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:08.172757 10841 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:08.178162 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:08.139912 17903 raft_consensus.cc:2749] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:08.179190 18053 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:08.179745 17927 raft_consensus.cc:2749] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:08.180583 17927 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:08.181007 17927 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:08.181366 17927 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:08.170418 11477 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:08.184178 18036 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:08.187280 17856 consensus_queue.cc:1048] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:19:08.187219 18034 consensus_queue.cc:260] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:08.191493 18034 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:08.095757 18013 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:08.192065 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:08.031554 10794 rpcz_store.cc:276] 0709 13:19:06.637530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:07.022577 (+385047us) service_pool.cc:224] Handling call
0709 13:19:07.023394 (+   817us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:19:07.293539 (+270145us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:19:07.293858 (+   319us) log.cc:844] Serialized 14 byte log entry
0709 13:19:07.839838 (+545980us) log.cc:844] Serialized 14 byte log entry
0709 13:19:07.842284 (+  2446us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:07.845212 (+  2928us) log.cc:844] Serialized 52 byte log entry
0709 13:19:07.845422 (+   210us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:19:07.863674 (+ 18252us) log.cc:844] Serialized 14 byte log entry
0709 13:19:07.867613 (+  3939us) log.cc:844] Serialized 14 byte log entry
0709 13:19:07.882907 (+ 15294us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:07.883475 (+   568us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:07.883534 (+    59us) raft_consensus.cc:1685] finished
0709 13:19:07.883546 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.922940 (+ 39394us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8499,"prepare.queue_time_us":21205,"prepare.run_cpu_time_us":192,"prepare.run_wall_time_us":195,"raft.queue_time_us":70427,"raft.run_cpu_time_us":881,"raft.run_wall_time_us":4592,"thread_start_us":498305,"threads_started":2,"wal-append.queue_time_us":397461}
I20260709 13:19:08.214851 11478 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:19:08.215020 10795 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 19.
I20260709 13:19:08.216189 18013 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:08.216593 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:08.217072 10799 raft_consensus.cc:1240] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 21. Current term is 22. Ops: [21.17-21.17]
I20260709 13:19:08.217336 10868 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 16.
I20260709 13:19:08.217644 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:19:08.218389 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 45731.7us
I20260709 13:19:08.218392 17935 raft_consensus.cc:2749] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:19:08.219363 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.219797 10800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2542) took 1027 ms. Trace:
I20260709 13:19:08.220211 10800 rpcz_store.cc:276] 0709 13:19:07.191878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:07.316695 (+124817us) service_pool.cc:224] Handling call
0709 13:19:08.219554 (+902859us) spinlock_profiling.cc:243] Waited 529 ms on lock 0x7b5400211bd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:08.219718 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1215944448}
I20260709 13:19:08.218436 10871 raft_consensus.cc:2393] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 25: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:08.221405 10916 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:08.225525 17861 raft_consensus.cc:2749] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:19:08.219413 10841 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:19:08.237627 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 117873us
I20260709 13:19:08.256296 17936 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:08.257158 17936 raft_consensus.cc:697] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 20 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:08.258471 17936 consensus_queue.cc:237] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:08.265859 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 27 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:08.153116 17729 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:08.219838 10915 connection.cc:256] RPC call timeout handler was delayed by -1.2845407929999055s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16770000000000002s for a response
I20260709 13:19:08.165210 17896 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:08.275350 17857 consensus_meta.cc:287] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.589s	user 0.005s	sys 0.013s
I20260709 13:19:08.276018 17857 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:08.277014 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:08.277710 11507 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:08.277794 17965 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:08.279142 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:08.283370 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:08.284811 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:08.285295 17857 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:08.285691 10795 raft_consensus.cc:2393] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 18: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:08.285743 10841 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:08.286149 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:08.287017 10876 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 21.
I20260709 13:19:08.287611 17411 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:08.288074 10841 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:08.290963 17935 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:08.284369 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:08.292007 11478 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:19:08.298099 10873 raft_consensus.cc:1275] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:19:08.300081 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:08.301198 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:19:08.296917 10915 connection.cc:256] RPC call timeout handler was delayed by -1.2017407929997717s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1769s for a response
W20260709 13:19:08.302215 10915 connection.cc:256] RPC call timeout handler was delayed by -1.1801407929997367s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17930000000000001s for a response
W20260709 13:19:08.307927 10915 connection.cc:256] RPC call timeout handler was delayed by -1.0880344340000647s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16710000000000003s for a response
I20260709 13:19:08.308816 11510 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:08.304661 10916 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:08.314872 18044 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:19:08.315565 18044 raft_consensus.cc:697] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 25 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:08.316659 18044 consensus_queue.cc:237] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:08.319532 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:08.314601 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:08.321640 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:08.322430 10795 raft_consensus.cc:2393] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 24: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:08.323685 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:08.324582 10795 raft_consensus.cc:2393] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 21: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:08.325528 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:08.328148 17840 raft_consensus.cc:3055] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:08.329383 17840 raft_consensus.cc:740] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:08.330564 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:19:08.330957 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.330449 17840 consensus_queue.cc:260] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:08.331372 10799 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:08.331956 17840 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:08.326666 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:08.332731 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:08.333143 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:08.333410 10794 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 20.
I20260709 13:19:08.333652 10795 raft_consensus.cc:2376] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 23. Re-sending same reply.
I20260709 13:19:08.334606 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:08.335397 10792 raft_consensus.cc:2393] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 20: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:08.336404 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:08.339068 10950 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:08.339848 10868 raft_consensus.cc:2410] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 27 because replica has last-logged OpId of term: 26 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 12.
I20260709 13:19:08.340991 17756 raft_consensus.cc:2749] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:19:08.345667 18084 log.cc:927] Time spent T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.404s	user 0.001s	sys 0.002s
W20260709 13:19:08.346460 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:08.348197 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 60234.8us
W20260709 13:19:08.348969 18095 log.cc:927] Time spent T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
I20260709 13:19:08.284830 17928 consensus_queue.cc:1048] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.065s
I20260709 13:19:08.351748 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:08.284313 17920 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:08.353448 17821 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:19:08.355278 10876 raft_consensus.cc:1275] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
W20260709 13:19:08.355495 18093 log.cc:927] Time spent T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 13:19:08.356343 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:19:08.356793 18092 log.cc:927] Time spent T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.134s	user 0.002s	sys 0.000s
I20260709 13:19:08.357277 10875 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:08.304981 18013 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:08.359081 10872 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:08.359279 18013 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:08.359326 10949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 3157) took 1346 ms. Trace:
I20260709 13:19:08.334460 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:08.364729 10949 rpcz_store.cc:276] 0709 13:19:07.012484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:07.014193 (+  1709us) service_pool.cc:224] Handling call
0709 13:19:07.014625 (+   432us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:07.063580 (+ 48955us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:19:07.921466 (+857886us) spinlock_profiling.cc:243] Waited 19.4 ms on lock 0x7b18004031e0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc5455 00007f9c848ad3c7 00007f9c80ebf9a7 00007f9c80ecf0bf 00007f9c80ecf038
0709 13:19:07.977909 (+ 56443us) log.cc:844] Serialized 14 byte log entry
0709 13:19:08.159300 (+181391us) log.cc:844] Serialized 14 byte log entry
0709 13:19:08.159727 (+   427us) log.cc:844] Serialized 14 byte log entry
0709 13:19:08.159954 (+   227us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:08.160711 (+   757us) log.cc:844] Serialized 26 byte log entry
0709 13:19:08.160874 (+   163us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:08.160912 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:08.161123 (+   211us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:08.356103 (+194980us) spinlock_profiling.cc:243] Waited 47.7 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:19:08.358894 (+  2791us) raft_consensus.cc:1685] finished
0709 13:19:08.358915 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:08.359102 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":159968,"prepare.queue_time_us":266395,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":90,"raft.queue_time_us":104924,"raft.run_cpu_time_us":656,"raft.run_wall_time_us":95311,"spinlock_wait_cycles":154335360,"thread_start_us":175698,"threads_started":2,"wal-append.queue_time_us":244071}
I20260709 13:19:08.371457 10793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2529) took 2761 ms. Trace:
I20260709 13:19:08.394043 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:08.378774 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:08.302860 17857 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:08.367815  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:08.397919 10801 raft_consensus.cc:2393] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 20: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
W20260709 13:19:08.400064 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.402860 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:08.406365 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:08.411494 10941 raft_consensus.cc:2393] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 25: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:08.411626 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:08.368276 17821 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:08.413671 11504 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 21: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:08.417829 11478 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:19:08.417940 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:08.419443 10873 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 19.
I20260709 13:19:08.413496 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:08.410318 17861 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:08.422297 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:08.416013 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:08.394066 10793 rpcz_store.cc:276] 0709 13:19:05.609966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.193702 (+583736us) service_pool.cc:224] Handling call
0709 13:19:06.194086 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:06.246217 (+ 52131us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:06.246247 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:06.246894 (+   647us) log.cc:844] Serialized 26 byte log entry
0709 13:19:08.079673 (+1832779us) spinlock_profiling.cc:243] Waited 1.07 ms on lock 0x7b180043de60. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a550d1 00007f9c80e70a0f
0709 13:19:08.217730 (+138057us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:08.217786 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:08.217873 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:08.370943 (+153070us) raft_consensus.cc:1685] finished
0709 13:19:08.370964 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:08.371218 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51764,"raft.queue_time_us":108168,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":28375,"spinlock_wait_cycles":2452352,"thread_start_us":116172,"threads_started":1,"wal-append.queue_time_us":1998533}
I20260709 13:19:08.403620 17920 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:08.428313 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:08.430673 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:08.417245 17821 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:19:08.310985 17935 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:08.441818 11478 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:19:08.452097 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:08.414237 17857 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:08.440050 18055 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:08.459569 10801 raft_consensus.cc:2393] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 22: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
W20260709 13:19:08.463828 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 47973.1us
I20260709 13:19:08.467553 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:08.467553 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:08.445972 17928 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:08.468621 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:08.469810 10794 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 20.
I20260709 13:19:08.470187 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:08.422501 17861 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:08.475684 17928 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:08.475873 18013 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:08.476200 17928 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:08.478380 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:08.479633 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:08.479835 17861 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:08.480105 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:19:08.481235 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.481287 11478 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
W20260709 13:19:08.481895 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.460889 18026 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:08.444501 17729 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:08.484999 18026 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:08.488952 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:08.493655 10841 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:08.498133 10867 raft_consensus.cc:1275] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:19:08.499917 10872 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 24.
I20260709 13:19:08.485074 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:19:08.504101 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 233665us
I20260709 13:19:08.506206 11504 raft_consensus.cc:2393] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 24: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:08.506399 10873 raft_consensus.cc:2393] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 21: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:08.507714 10875 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 19.
I20260709 13:19:08.513479 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:08.506641 11512 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
I20260709 13:19:08.497824 18026 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:08.493145 17965 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:08.519073 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.518777 17777 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:08.521747 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:08.523042 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:08.516574 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:08.526217 10797 raft_consensus.cc:1275] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:19:08.528740 10874 raft_consensus.cc:1240] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:08.530652 10916 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:08.532816 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:08.535032 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:08.535953 10801 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
I20260709 13:19:08.528785 10870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 2560) took 1160 ms. Trace:
I20260709 13:19:08.513384 18029 raft_consensus.cc:2749] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:08.534231 18013 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:08.539525 10686 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 2b82f18992434ed5ba58b805107060d4 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 468a8fd46e824f62b9c2c0d3532a6d26 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 20 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:08.539947 18013 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:08.542114 18013 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:08.542989 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:08.549616 11507 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 18.
I20260709 13:19:08.554888 18045 raft_consensus.cc:2804] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:08.555362 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:08.555648 18045 raft_consensus.cc:697] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 20 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:08.556111 10793 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
I20260709 13:19:08.557185 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:08.556833 18045 consensus_queue.cc:237] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:08.558086 10793 raft_consensus.cc:2410] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 22 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 13:19:08.536882 17861 consensus_queue.cc:1048] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 13:19:08.536737 10870 rpcz_store.cc:276] 0709 13:19:07.282255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:07.577188 (+294933us) service_pool.cc:224] Handling call
0709 13:19:07.577545 (+   357us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:07.596979 (+ 19434us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:07.597013 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:07.597666 (+   653us) log.cc:844] Serialized 52 byte log entry
0709 13:19:07.991391 (+393725us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:19:07.991469 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:07.991569 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:08.344712 (+353143us) spinlock_profiling.cc:243] Waited 21.8 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:19:08.429942 (+ 85230us) raft_consensus.cc:1685] finished
0709 13:19:08.429974 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:08.442937 (+ 12963us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19033,"raft.queue_time_us":20026,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":101,"spinlock_wait_cycles":50126720,"thread_start_us":148259,"threads_started":1,"wal-append.queue_time_us":343680}
I20260709 13:19:08.568495 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:08.569478 10801 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
I20260709 13:19:08.577157 18013 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:08.577832 18013 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:08.580122 18013 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:08.580654 18013 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:08.581126 18013 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:08.583065 18011 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:08.583614 17935 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:08.584733 18013 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:08.583666 18011 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:08.589598 18011 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:08.592509 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.596108 11510 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 22.
W20260709 13:19:08.530220 18080 consensus_meta.cc:287] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.514s	user 0.010s	sys 0.004s
I20260709 13:19:08.570720 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:08.616431 10841 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:08.617883 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:08.618650 10799 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 21.
I20260709 13:19:08.522116 17729 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:08.620185 10795 raft_consensus.cc:1240] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 20. Current term is 21. Ops: []
W20260709 13:19:08.621990 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:08.621852 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 91277.1us
I20260709 13:19:08.626379 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:08.622512 10915 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:08.597442 18039 raft_consensus.cc:2804] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:08.638767 10793 raft_consensus.cc:2410] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 24 because replica has last-logged OpId of term: 22 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 10.
I20260709 13:19:08.639056 10950 raft_consensus.cc:1275] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:19:08.664808 18055 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:08.665442 18055 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:19:08.674088 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:08.678738 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.687253 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:08.694093 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 54204us
W20260709 13:19:08.699816 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 108116us
I20260709 13:19:08.701026 11477 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:08.634017 17996 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:08.723874 10867 raft_consensus.cc:1275] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:19:08.639540 18039 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:08.692621 17965 raft_consensus.cc:2749] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:08.612296 18080 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:08.745863 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 201075us
I20260709 13:19:08.736080 18011 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:19:08.752295 18103 log.cc:927] Time spent T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 13:19:08.752516 18076 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:08.717814 17861 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:08.778715 10916 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:19:08.769306 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 152984us
I20260709 13:19:08.782997 18076 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:08.788929 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:08.771116 11503 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 23.
I20260709 13:19:08.774828 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:08.764202 10801 raft_consensus.cc:1275] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:19:08.735659 18039 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:08.794713 10870 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 23.
I20260709 13:19:08.826946 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:08.828003 10801 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 21.
W20260709 13:19:08.820417 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 93976.5us
I20260709 13:19:08.835259 18011 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:08.837780 18011 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:08.838330 18098 log.cc:927] Time spent T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.191s	user 0.000s	sys 0.005s
I20260709 13:19:08.633085 17927 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:08.840319 10795 raft_consensus.cc:1240] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:08.840497 17927 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:08.841084 18044 raft_consensus.cc:2749] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:19:08.841769 17411 consensus_meta.cc:287] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.554s	user 0.008s	sys 0.002s
I20260709 13:19:08.842303 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:08.855567 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 154660us
I20260709 13:19:08.832103 17896 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 13:19:08.845112 17891 consensus_meta.cc:287] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.645s	user 0.010s	sys 0.000s
I20260709 13:19:08.787415 17861 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:19:08.846369 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.866607 10841 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:08.828711 18036 consensus_meta.cc:287] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.644s	user 0.005s	sys 0.009s
I20260709 13:19:08.817041 17857 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:08.825553 17840 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:08.882238 10867 raft_consensus.cc:2393] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 22: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:08.876767 18036 raft_consensus.cc:2749] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:08.889474 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:08.869979 17861 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:08.894173 10793 raft_consensus.cc:1240] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 13:19:08.870223 17896 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:08.888902 17834 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:08.896932 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:08.891970 17840 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:08.898835 11477 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:08.913460 11477 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:08.894821 17927 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:08.915575 11477 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:08.917701 18036 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:08.911705 17834 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:08.903810 17896 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:08.920463 10800 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 18.
I20260709 13:19:08.929898 10867 raft_consensus.cc:1240] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 20. Current term is 21. Ops: [18.11-20.12]
I20260709 13:19:08.937309 17821 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:08.940465 10869 raft_consensus.cc:1240] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 18. Current term is 19. Ops: [18.9-18.9]
I20260709 13:19:08.941795 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:08.943194 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:08.943322 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:08.944710 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 470043us
I20260709 13:19:08.922919 17777 raft_consensus.cc:2749] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:08.933060 18036 raft_consensus.cc:2749] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:19:08.963382 18036 consensus_queue.cc:1048] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:19:08.965227 18036 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:08.965871 18036 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:08.967777 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:08.974887 18080 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:08.954707 10766 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:08.958137 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:08.958922 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:08.959559 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:08.960210 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:08.960453 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:08.960795 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:08.961967 17834 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:08.979499 10869 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 21.
I20260709 13:19:08.982312 10766 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:08.983667 10873 raft_consensus.cc:2393] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 21: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:08.983772 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:08.987032 10800 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:08.948135 17729 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:08.922444 17928 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:08.985286 10876 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 19.
I20260709 13:19:08.989326 18080 raft_consensus.cc:2804] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:09.015466 10867 raft_consensus.cc:1240] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 13:19:08.984375 17927 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:09.017194 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:09.005416 17834 raft_consensus.cc:2804] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:09.008963 17756 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:09.043231 10767 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:19:09.019800 11511 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.016s	sys 0.056s
I20260709 13:19:09.009990 17891 consensus_queue.cc:1048] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 13:19:09.035533 17411 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:09.024710 18080 raft_consensus.cc:697] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:19:09.049371 11511 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.016s	sys 0.057s
I20260709 13:19:09.049363 17840 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:09.056186 17756 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:09.037180 17834 raft_consensus.cc:697] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:19:09.063467 18107 log.cc:927] Time spent T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:19:09.071331 18055 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:09.045794 17821 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 28 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:09.074803 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:09.075708 18034 consensus_meta.cc:287] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.884s	user 0.010s	sys 0.003s
I20260709 13:19:09.028685 17927 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:09.085700 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:09.092935 18036 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:09.072896 18104 log.cc:927] Time spent T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.235s	user 0.002s	sys 0.002s
W20260709 13:19:09.081033 18026 consensus_meta.cc:287] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.536s	user 0.007s	sys 0.002s
I20260709 13:19:09.120883 10796 raft_consensus.cc:2410] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
W20260709 13:19:09.122088 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:09.110168 18102 log.cc:927] Time spent T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.262s	user 0.000s	sys 0.003s
I20260709 13:19:09.086416 17896 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:09.129830 10841 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:09.110656 17840 raft_consensus.cc:2749] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:19:09.132211 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:09.116276 17928 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:09.149169 10841 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:09.097555 17411 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:09.163864 17996 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:09.167665 17996 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:09.167253 17834 consensus_queue.cc:237] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:09.169163 17756 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:09.171813 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 142039us
I20260709 13:19:09.173521 10767 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:09.174325 18011 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:09.175814 17857 consensus_queue.cc:1048] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:19:09.168654 18054 consensus_meta.cc:287] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 1.047s	user 0.013s	sys 0.004s
I20260709 13:19:09.176147 17411 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:09.180011 10800 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
I20260709 13:19:09.158707 17821 raft_consensus.cc:2804] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:09.129796 18026 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:09.184676 17756 raft_consensus.cc:2804] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:09.200173 17756 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:09.200759 17756 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:09.201877 17896 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:09.199007 11504 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 21.
I20260709 13:19:09.189123 17821 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:09.177798 17928 consensus_queue.cc:1048] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.038s
I20260709 13:19:09.217949 17411 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.177302 18080 consensus_queue.cc:237] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:09.184573 17996 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:09.219753 17821 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:09.241178 17927 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:09.282930 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:09.283545 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:09.278700 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 101296us
W20260709 13:19:09.258598 17985 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.779s	user 0.010s	sys 0.027s
I20260709 13:19:09.264544 17896 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:09.244135 17411 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:09.278080 18055 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:09.292804 17985 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.813s	user 0.011s	sys 0.027s
I20260709 13:19:09.304705 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:09.315222 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:09.316113 10947 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:19:09.319177 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:09.323364 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:09.324324 10946 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:09.327270 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:09.327809 18076 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:09.328119 10944 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
W20260709 13:19:09.337284 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 45054.2us
I20260709 13:19:09.338160 10767 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:19:09.339206 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 380699us
W20260709 13:19:09.340396 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 472500us
I20260709 13:19:09.347187 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:09.351030 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:09.351719 10949 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:19:09.354964 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 411991us
I20260709 13:19:09.367857 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:09.368947 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:09.369856 11505 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 22.
I20260709 13:19:09.372053 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:09.373359 11503 raft_consensus.cc:1275] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 13:19:09.373721 11503 pending_rounds.cc:85] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586: Aborting all ops after (but not including) 9
I20260709 13:19:09.373934 11503 pending_rounds.cc:107] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:19:09.374146 11503 raft_consensus.cc:2889] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:09.372754 11504 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 21.
I20260709 13:19:09.329931 17896 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:09.389914 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:09.353976 18026 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:09.392407 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:09.401413 11512 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:09.401801 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:09.340296 18039 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:09.372486 18034 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.372236 18045 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.405874 10950 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
I20260709 13:19:09.413384 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:09.417188 11509 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 18.
W20260709 13:19:09.427014 18105 log.cc:927] Time spent T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.584s	user 0.001s	sys 0.003s
W20260709 13:19:09.428383 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 90294.4us
I20260709 13:19:09.429682 17756 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:09.431005 17996 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:09.410334 18108 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.437433 18108 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:09.439821 18108 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:09.418673 17857 raft_consensus.cc:2749] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:09.440408 18039 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:09.441190 10947 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 28.
I20260709 13:19:09.441385 17840 consensus_queue.cc:1048] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:19:09.442070 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:09.417706 18045 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:09.446761 11506 raft_consensus.cc:2410] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 20 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:19:09.446561 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:09.447579 11510 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 18.
I20260709 13:19:09.448828 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:09.452050 10946 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
I20260709 13:19:09.403936 17861 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:09.458441 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:09.459313 10950 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 22.
I20260709 13:19:09.467276 11512 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 24.
I20260709 13:19:09.470616 11510 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 23.
W20260709 13:19:09.479866 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:09.455350 18034 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:09.484133 11509 raft_consensus.cc:1240] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:09.483525 18122 consensus_queue.cc:1048] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.004s
I20260709 13:19:09.482903 17411 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:09.496729 10949 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 22.
I20260709 13:19:09.498018 10947 raft_consensus.cc:1240] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 21. Current term is 22. Ops: [19.14-21.15]
I20260709 13:19:09.498728 17411 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.499490 17411 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:09.503243 17411 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:09.520419 18045 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:09.520339 17756 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:09.528497 18034 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:09.511757 17896 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:09.530769 11478 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:09.531975 18045 raft_consensus.cc:2749] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:09.532874 11478 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:09.554687 10945 raft_consensus.cc:1240] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 24. Current term is 25. Ops: []
W20260709 13:19:09.556723 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:09.533939 18045 raft_consensus.cc:2749] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:19:09.576974 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 44235.8us
I20260709 13:19:09.616968 11478 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:09.678121 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:09.616982 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 275133us
W20260709 13:19:09.683439 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 254271us
I20260709 13:19:09.671113 18034 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:09.685274 10767 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:09.675029 17861 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:09.688958 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 79234.3us
I20260709 13:19:09.626130 17821 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:09.672111 18109 raft_consensus.cc:2749] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:09.700680 10841 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:19:09.704516 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:09.704097 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 306740us
I20260709 13:19:09.710960 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:09.714979 11478 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:09.712126 10949 raft_consensus.cc:3055] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:09.716039 10949 raft_consensus.cc:740] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:09.719937 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:09.693439 18034 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:09.762969 10916 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:09.755283 10949 consensus_queue.cc:260] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:09.773948 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 53209.1us
W20260709 13:19:09.774250 18127 log.cc:927] Time spent T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 13:19:09.774976 10915 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:09.740676 17411 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:09.775787 17411 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:09.776180 17411 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:19:09.778453 18125 log.cc:927] Time spent T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:19:09.759227 18109 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:09.780367 18109 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:09.780778 18109 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:19:09.783390 18114 log.cc:927] Time spent T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
W20260709 13:19:09.784320 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 83713.9us
W20260709 13:19:09.788082 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 79918.2us
I20260709 13:19:09.774353 10949 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:09.790684 11511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56410 (request call id 1130) took 1309 ms. Trace:
I20260709 13:19:09.738566 18129 consensus_queue.cc:1059] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 22.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.072s
I20260709 13:19:09.804867 11478 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:09.794490 11511 rpcz_store.cc:276] 0709 13:19:08.476422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:08.477046 (+   624us) service_pool.cc:224] Handling call
0709 13:19:08.477494 (+   448us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:09.108081 (+630587us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:09.108145 (+    64us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:09.125610 (+ 17465us) log.cc:844] Serialized 26 byte log entry
0709 13:19:09.557884 (+432274us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:09.557942 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:09.558028 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:09.785584 (+227556us) raft_consensus.cc:1685] finished
0709 13:19:09.785607 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:09.785771 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4668,"raft.queue_time_us":42667,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":7144,"thread_start_us":998344,"threads_started":2,"wal-append.queue_time_us":537780}
I20260709 13:19:09.811395 17896 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:19:09.836222 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 61423.6us
I20260709 13:19:09.837343 10915 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:09.790330 18080 consensus_queue.cc:1059] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:19:09.843416 11478 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:09.849699 11478 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:09.851081 18044 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:19:09.851125 18122 raft_consensus.cc:2749] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:09.851874 18044 raft_consensus.cc:697] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 23 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
W20260709 13:19:09.852793 10766 leader_election.cc:336] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Timed out: RequestConsensusVote RPC to 127.9.116.131:45553 timed out after 1.508s (SENT)
I20260709 13:19:09.852885 18044 consensus_queue.cc:237] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:09.854879 18080 raft_consensus.cc:3055] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:09.855363 18080 raft_consensus.cc:740] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:09.856334 18080 consensus_queue.cc:260] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:09.857074 18151 raft_consensus.cc:2749] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:09.857931 18139 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.858052 18151 consensus_queue.cc:1048] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:19:09.863121 18142 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
W20260709 13:19:09.861019 18124 log.cc:927] Time spent T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 13:19:09.858614 18139 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:09.862095 18141 log.cc:927] Time spent T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:19:09.830811 18120 raft_consensus.cc:2749] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:09.839936 18113 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:19:09.883610 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:09.884711 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:09.872803 10949 raft_consensus.cc:1275] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
W20260709 13:19:09.907440 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 57856.4us
I20260709 13:19:09.915179 10949 pending_rounds.cc:85] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 10
I20260709 13:19:09.846498 18147 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:09.822654 17834 consensus_queue.cc:1059] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.082s
I20260709 13:19:09.847380 17896 raft_consensus.cc:697] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:09.843662 18149 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:09.921087 10796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39442 (request call id 991) took 1600 ms. Trace:
I20260709 13:19:09.919031 18080 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:09.941639 10796 rpcz_store.cc:276] 0709 13:19:08.315821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:08.319075 (+  3254us) service_pool.cc:224] Handling call
0709 13:19:09.915746 (+1596671us) spinlock_profiling.cc:243] Waited 798 ms on lock 0x7b540024ead4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:09.915931 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1835292416}
W20260709 13:19:09.925930 10766 leader_election.cc:336] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Timed out: RequestConsensusVote RPC to 127.9.116.131:45553 timed out after 1.586s (SENT)
I20260709 13:19:09.907189 17821 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:09.940521 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 134244us
I20260709 13:19:09.944320 10766 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:09.949757 10916 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:19:09.959784 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 172165us
I20260709 13:19:09.961176 11478 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:09.964591 10916 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:09.917045 10949 pending_rounds.cc:107] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 13:19:09.965659 10949 raft_consensus.cc:2889] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:09.872201 18142 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:09.943938 18116 raft_consensus.cc:2749] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:09.936841 18113 raft_consensus.cc:697] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 21 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:09.995807 10945 raft_consensus.cc:3055] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:09.996333 10945 raft_consensus.cc:740] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:10.001178 11477 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:19:09.998308 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 161102us
W20260709 13:19:10.005131 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:09.972134 17777 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:09.952754 17411 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:09.999100 17834 raft_consensus.cc:3055] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:09.953276 18147 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:10.028709 17834 raft_consensus.cc:740] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:10.032325 10841 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:10.000212 18034 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:10.035401 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 74346.4us
I20260709 13:19:10.008944 18011 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:10.011723 18120 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:10.047976 10915 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:10.050014 11478 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:10.055889 17928 consensus_queue.cc:1048] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.016s
I20260709 13:19:10.057106 17834 consensus_queue.cc:260] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:10.058432 18129 raft_consensus.cc:2749] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 13:19:10.058619 17834 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:10.040812 18034 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:10.059885 18034 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:10.062292 18034 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:10.041007 17996 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:10.063189 17996 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:10.063575 17996 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:19:10.067533 18146 log.cc:927] Time spent T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
W20260709 13:19:10.068862 18145 log.cc:927] Time spent T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 13:19:09.966315 18149 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:10.059875 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:10.104696 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 178789us
I20260709 13:19:10.015146 18055 consensus_queue.cc:1048] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.068s
I20260709 13:19:10.025231 18039 consensus_queue.cc:1059] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 24.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.058s
W20260709 13:19:10.107393 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.047068 18120 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:10.032373 18122 consensus_queue.cc:1059] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.058s
W20260709 13:19:10.115227 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 81420.3us
W20260709 13:19:10.129287 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 106146us
W20260709 13:19:10.130965 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.116716 18120 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:10.105253 18161 raft_consensus.cc:2749] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:10.037506 17896 consensus_queue.cc:237] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:10.124801 18166 raft_consensus.cc:2804] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:19:10.165123 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 200603us
I20260709 13:19:10.169504 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:10.150460 18139 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:10.117820 18113 consensus_queue.cc:237] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:10.176931 10916 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:10.138833 18109 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:10.162377 18161 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:10.160392 18044 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:10.186714 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:10.203641 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 57974.3us
I20260709 13:19:10.202245 10916 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:10.177578 18139 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:10.205698 18161 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:10.214519 18139 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:10.176446 18149 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:10.175251 18166 raft_consensus.cc:697] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 18 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:10.216497 18161 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:10.217353 18166 consensus_queue.cc:237] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:10.208052 18156 log.cc:927] Time spent T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:19:10.208390 18169 consensus_queue.cc:1048] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.045s
I20260709 13:19:10.209096 18158 consensus_queue.cc:1048] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.006s
I20260709 13:19:10.213698 10841 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:10.225422 10841 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:10.227012 10841 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:10.228089 18055 raft_consensus.cc:2749] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:10.228895 18055 raft_consensus.cc:2749] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:10.229624 18055 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:10.234519 10841 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:19:10.237995 18161 raft_consensus.cc:2804] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:10.238161 18170 consensus_queue.cc:1048] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 13:19:10.201726 18054 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:10.251302 18054 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:10.251741 18054 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:10.253983 18147 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:10.259927 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.260895 18168 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:10.261583 18168 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:10.264349 18168 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:10.238515 18161 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:10.264987 18161 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:19:10.265337 18160 log.cc:927] Time spent T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.235s	user 0.001s	sys 0.000s
I20260709 13:19:10.265945 18142 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:10.269795 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 57334.1us
I20260709 13:19:10.242476 17777 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:19:10.272001 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 71715us
W20260709 13:19:10.284016 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 234132us
I20260709 13:19:10.281787 10916 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:10.206727 18122 raft_consensus.cc:2749] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:10.268536 17996 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:10.325657 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 43960.8us
I20260709 13:19:10.320713 18158 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 13:19:10.330055 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 46320.3us
I20260709 13:19:10.288904 18147 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:10.337963 10841 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
W20260709 13:19:10.340227 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 51345.4us
I20260709 13:19:10.329099 18174 raft_consensus.cc:3055] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:19:10.326637 17411 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:10.318738 18044 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:10.347043 10841 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:19:10.350872 10841 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
W20260709 13:19:10.353107 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.353137 10841 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:10.338536 18173 raft_consensus.cc:2804] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:10.346493 18174 raft_consensus.cc:740] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:10.349967 18149 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:19:10.340211 18147 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:10.370851 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.373229 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:10.357010 18019 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:10.370283 10766 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:10.355204 18158 raft_consensus.cc:697] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 24 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:10.376192 10801 raft_consensus.cc:1240] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:19:10.373975 10794 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 19.
I20260709 13:19:10.377900 10766 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:10.380692 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:10.381628 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:10.390733 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:10.383742 10873 raft_consensus.cc:1275] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 13:19:10.384094 10876 raft_consensus.cc:1275] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:19:10.384619 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:10.385938 10841 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:10.372299 18011 raft_consensus.cc:2749] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:10.388335 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:10.393350 10792 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 18.
I20260709 13:19:10.389009 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:10.389484 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:19:10.390587 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.389215 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:10.401090 10796 raft_consensus.cc:1275] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 13. (term mismatch)
I20260709 13:19:10.403398 10871 raft_consensus.cc:2410] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:19:10.402848 10796 pending_rounds.cc:85] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 12
I20260709 13:19:10.358719 18142 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:10.406662 10799 raft_consensus.cc:1240] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 23. Current term is 24. Ops: []
I20260709 13:19:10.398162 10794 raft_consensus.cc:1275] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:19:10.391439 10795 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 22.
I20260709 13:19:10.401862 10797 raft_consensus.cc:2393] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 21: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:10.403955 10796 pending_rounds.cc:107] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 19.13
I20260709 13:19:10.374466 17965 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:10.405823 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:10.405447 18158 consensus_queue.cc:237] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:10.373073 18173 raft_consensus.cc:697] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 19 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:10.400553 17834 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:10.389429 18019 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:10.407114 17996 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:10.400524 18168 consensus_queue.cc:1059] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.007s
I20260709 13:19:10.407925 18109 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:10.409880 10872 raft_consensus.cc:1275] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 13:19:10.411473 18172 log.cc:927] Time spent T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:19:10.389776 18054 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:10.456753 18168 raft_consensus.cc:3055] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:10.457267 18168 raft_consensus.cc:740] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:10.415127 10796 raft_consensus.cc:2889] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:10.457617 10941 raft_consensus.cc:1275] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:19:10.459136 18155 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:10.459415 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:10.459686 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:10.459848 18155 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:10.460944 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:10.395217 18177 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:10.463172 18177 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:10.459440 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:10.420497 17834 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:10.420306 17857 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:10.464756 10799 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 20.
I20260709 13:19:10.465297 18177 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:10.465118 17857 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:10.464757 17834 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:10.419559 10867 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:10.419205 17965 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:10.467082 17965 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:10.468639 18139 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:10.427742 18019 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:10.475237 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:10.475529 18175 raft_consensus.cc:3055] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:19:10.475952 18175 raft_consensus.cc:740] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:10.476174 11506 raft_consensus.cc:2393] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 23: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:10.476984 18175 consensus_queue.cc:260] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:10.397436 17411 raft_consensus.cc:2804] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:10.479311 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:10.479645 17411 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:10.479946 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:10.431193 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:10.480576 10796 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 22.
I20260709 13:19:10.428874 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:10.443815 10947 raft_consensus.cc:1240] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 21. Current term is 22. Ops: [19.14-21.15]
I20260709 13:19:10.483490 10795 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 18.
I20260709 13:19:10.449000 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:10.485430 10867 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 17.
I20260709 13:19:10.459255 11507 raft_consensus.cc:1275] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:19:10.486773 18039 consensus_queue.cc:1048] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:19:10.462616 11503 raft_consensus.cc:1275] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:19:10.487285 18173 consensus_queue.cc:237] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:10.488996 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:10.489650 10799 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:10.389891 18149 raft_consensus.cc:697] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:10.457701 18054 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:10.462482 18155 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:10.480104 17411 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:10.480635 10797 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 23.
I20260709 13:19:10.500456 10867 raft_consensus.cc:1240] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 18. Current term is 19. Ops: []
W20260709 13:19:10.433164 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 57251.8us
I20260709 13:19:10.505524 10842 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:10.429736 18055 raft_consensus.cc:2749] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:10.439229 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:10.481148 10766 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:10.506526 18154 consensus_queue.cc:1048] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:10.452316 17821 raft_consensus.cc:2749] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:10.508108 17756 consensus_queue.cc:1048] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:19:10.508486 18163 consensus_queue.cc:1048] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:19:10.510452 10868 raft_consensus.cc:1240] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:19:10.445377 10944 raft_consensus.cc:1275] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 13:19:10.485539 10949 raft_consensus.cc:2393] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 21: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:10.463534 18120 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:10.449514 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:10.516532 10792 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 20.
I20260709 13:19:10.492311 11478 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:10.423689 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:10.518038 10794 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:10.435945 17729 raft_consensus.cc:3055] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:19:10.491492 18170 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:10.509755 17756 raft_consensus.cc:2749] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:10.532505 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:10.519901 18155 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:19:10.533213 10873 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 21.
I20260709 13:19:10.534099 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:10.535723 18170 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:10.537957 18170 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:10.478637 18175 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:19:10.520401 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 45651.3us
I20260709 13:19:10.458251 18168 consensus_queue.cc:260] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:10.539191 18168 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:10.541168 11478 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:10.541530 17834 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:10.520750 10842 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:10.521971 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:10.513645 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:10.498705 10945 consensus_queue.cc:260] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 12, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:10.526839 18164 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:10.493168 18174 consensus_queue.cc:260] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:10.527006 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 73919.7us
I20260709 13:19:10.519670 17729 raft_consensus.cc:740] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:10.527518 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:10.527716 18149 consensus_queue.cc:237] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:10.599131 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:10.529474 10942 raft_consensus.cc:1240] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:10.600093 10797 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:10.529928 18039 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:10.531592 17857 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:10.543303 18080 consensus_meta.cc:287] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.601s	user 0.014s	sys 0.000s
I20260709 13:19:10.543062 18076 consensus_queue.cc:1048] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:19:10.543869 11511 raft_consensus.cc:1240] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 13:19:10.543789 18177 consensus_queue.cc:1048] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:10.544711 18045 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:10.545553 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:10.546085 18155 consensus_queue.cc:1048] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:19:10.542304 18122 consensus_queue.cc:1059] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:19:10.546455 18011 consensus_queue.cc:1048] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:19:10.547395 18147 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:10.547679 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:10.614404 11507 raft_consensus.cc:2376] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 24. Re-sending same reply.
I20260709 13:19:10.549310 18120 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:10.553253 10870 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 21.
I20260709 13:19:10.554232 18157 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:10.616720 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:10.513473 10799 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 21.
I20260709 13:19:10.494922 18139 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:10.573755 10950 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:10.576977 11508 raft_consensus.cc:2393] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 23: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:10.581409 10945 raft_consensus.cc:3060] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:19:10.603996 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.623597 11478 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:10.554960 17777 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:10.608438 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 87757.8us
I20260709 13:19:10.606618 10841 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:10.613477 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:10.626495 10873 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:10.604470 17861 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:10.604424 18039 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:10.604096 17927 raft_consensus.cc:2749] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:10.634848 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:10.631767 10916 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:10.624022 10766 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:19:10.627712 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.608088 17996 consensus_queue.cc:1048] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.023s
I20260709 13:19:10.598958 18164 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:10.616945 18176 consensus_queue.cc:1048] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.055s
I20260709 13:19:10.658438 18044 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:10.656060 18151 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:10.607887 18045 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:10.609211 17756 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:10.580843 18109 consensus_queue.cc:1059] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.038s
I20260709 13:19:10.633020 10842 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:10.634217 10841 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:10.658020 17965 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:10.640810 17861 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:10.618654 18142 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:10.627916 10794 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 22.
I20260709 13:19:10.635382 18120 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:10.676215 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 52274us
W20260709 13:19:10.674926 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 67033.3us
I20260709 13:19:10.632117 17729 consensus_queue.cc:260] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:10.679169 10870 raft_consensus.cc:1240] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:10.635491 10868 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:10.631114 18139 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:10.684625 10950 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
I20260709 13:19:10.687458 10945 raft_consensus.cc:1275] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 15. (term mismatch)
I20260709 13:19:10.690852 18045 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:10.683813 18151 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:10.689625 10873 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 22.
I20260709 13:19:10.641064 18039 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:19:10.697383 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 65679.5us
I20260709 13:19:10.629544 18157 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:10.635927 18147 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:10.700083 18113 raft_consensus.cc:2804] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:10.689520 18155 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:19:10.657276 17834 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:10.701550 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 67403.3us
I20260709 13:19:10.743659 10916 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:10.701746 18164 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:10.702066 18163 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:10.702190 10948 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:19:10.702847 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 69881.3us
I20260709 13:19:10.703366 17927 raft_consensus.cc:2749] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:19:10.668331 18076 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:19:10.716423 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.719599 10945 pending_rounds.cc:85] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 14
I20260709 13:19:10.750221 10945 pending_rounds.cc:107] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 19.15
I20260709 13:19:10.631181 17777 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:10.718698 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:10.719733 18187 log.cc:927] Time spent T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 13:19:10.763185 18076 raft_consensus.cc:697] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 21 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:10.656673 18019 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:10.764190 18076 consensus_queue.cc:237] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:10.764940 18174 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:10.765882 18019 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:10.732270 18151 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:10.766286 17965 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:10.767797 18139 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:10.753227 18155 raft_consensus.cc:697] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:10.769359 18155 consensus_queue.cc:237] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:10.717288 18142 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:10.747881 18163 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:10.772418 18163 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:10.772683 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:10.775836 10873 raft_consensus.cc:1240] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:10.740641 17729 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:10.779523 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:10.720427 18120 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:10.742400 10841 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:10.746687 10842 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:10.706609 17756 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:10.750427 10766 leader_election.cc:336] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Timed out: RequestConsensusVote RPC to 127.9.116.131:45553 timed out after 1.662s (SENT)
I20260709 13:19:10.750495 10945 raft_consensus.cc:2889] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:10.756377 18169 raft_consensus.cc:2749] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:10.757431 18175 raft_consensus.cc:3055] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:10.757740 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:10.791810 18175 raft_consensus.cc:740] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:10.792065 10949 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 21.
I20260709 13:19:10.793258 17756 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:10.741020 10950 raft_consensus.cc:1240] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 20. Current term is 22. Ops: []
I20260709 13:19:10.739106 18113 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:10.795010 18113 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:10.795719 10692 catalog_manager.cc:5697] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 14 to 15, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 15 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:10.796617 17411 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:10.797906 10800 raft_consensus.cc:2368] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for earlier term 18. Current term is 19.
W20260709 13:19:10.799226 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.798995 18055 consensus_queue.cc:1048] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:10.801151 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:10.802896 18019 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:10.760406 17777 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:10.782334 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.781201 10841 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:19:10.787137 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.807771 10842 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:10.787353 10868 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 25.
W20260709 13:19:10.788342 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 49714.9us
I20260709 13:19:10.721480 18161 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:10.790243 18157 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:10.790596 18120 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:10.792814 18175 consensus_queue.cc:260] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:10.778846 10948 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 21.
I20260709 13:19:10.805593 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:10.812057 10797 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 23.
I20260709 13:19:10.737963 18054 consensus_queue.cc:1048] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.042s
I20260709 13:19:10.816720 10867 raft_consensus.cc:1240] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 23. Current term is 24. Ops: []
I20260709 13:19:10.817503 10841 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
W20260709 13:19:10.820290 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.783620 17927 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 13:19:10.838151 18154 raft_consensus.cc:2749] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:10.819831 18109 consensus_queue.cc:1048] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.019s
W20260709 13:19:10.836472 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.833182 17411 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:10.842907 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:10.843787 10943 raft_consensus.cc:1240] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:10.844614 10945 raft_consensus.cc:1240] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 21 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 13:19:10.844774 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:10.845633 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:10.845974 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:10.846594 10943 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 22.
I20260709 13:19:10.847028 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:10.847816 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:10.848270 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:10.848538 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:10.848893 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:10.848584 10943 raft_consensus.cc:2393] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 18: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:10.849476 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:10.850069 10949 raft_consensus.cc:1240] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:10.851336 10943 raft_consensus.cc:1240] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:19:10.843106 10797 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a"
dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
mode: GRACEFUL
new_leader_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:40148
I20260709 13:19:10.812484 18176 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:10.797376 18151 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:10.853068 18176 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:10.852317 17861 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:10.854089 10797 raft_consensus.cc:606] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Received request to transfer leadership to TS cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:10.852443 10949 raft_consensus.cc:1240] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 19 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:10.753484 18147 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:10.878371 18147 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:10.862627 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.875897 10797 raft_consensus.cc:614] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:19:10.859319 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:10.907439 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 71052us
I20260709 13:19:10.833439 17756 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:10.912477 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:10.922560 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 299002us
I20260709 13:19:10.820087 17821 consensus_queue.cc:1059] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.091s
I20260709 13:19:10.819639 17891 consensus_queue.cc:1059] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 11, Time since last communication: 0.091s
I20260709 13:19:10.934628 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:10.935375 10794 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 19.
I20260709 13:19:10.910929 18175 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:10.939885 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:10.926801 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:10.941978 10949 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 21.
I20260709 13:19:10.928097 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:10.943014 10945 raft_consensus.cc:2368] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 19. Current term is 20.
I20260709 13:19:10.842046 18161 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:10.911303 18019 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:10.953908 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:10.926414 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.965808 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:10.966518 10869 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 24.
W20260709 13:19:10.987856 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.010316 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:11.011412 10950 raft_consensus.cc:2368] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 21. Current term is 22.
I20260709 13:19:11.013356 10941 raft_consensus.cc:1217] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Deduplicated request from leader. Original: 17.12->[18.13-18.13]   Dedup: 18.13->[]
W20260709 13:19:11.017967 10766 leader_election.cc:336] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Timed out: RequestConsensusVote RPC to 127.9.116.131:45553 timed out after 1.825s (SENT)
I20260709 13:19:11.019449 10766 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:11.019688 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:11.016764 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 49276.5us
W20260709 13:19:11.015522 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 55244.4us
I20260709 13:19:10.909291 18157 consensus_queue.cc:1059] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.081s
I20260709 13:19:11.022930 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:11.023295 11478 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:11.029938 11478 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:11.025743 10941 raft_consensus.cc:1240] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:19:11.034435 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:10.915033 18113 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:10.952847 17756 raft_consensus.cc:697] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:11.037566 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:11.042196 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:11.029599 18116 consensus_queue.cc:1059] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 13:19:11.026968 10692 catalog_manager.cc:5697] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 13 to 15, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 15 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:11.027344 18076 consensus_queue.cc:1048] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.018s
I20260709 13:19:11.050006 18176 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:11.050292 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:11.039863 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:11.062423 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:11.052687 10950 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 21.
W20260709 13:19:11.055308 10766 leader_election.cc:336] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Timed out: RequestConsensusVote RPC to 127.9.116.131:45553 timed out after 2.164s (SENT)
I20260709 13:19:11.068352 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:11.056109 10916 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:11.073140 18201 raft_consensus.cc:2749] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:19:11.073413 10766 leader_election.cc:336] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Timed out: RequestConsensusVote RPC to 127.9.116.131:45553 timed out after 2.024s (SENT)
I20260709 13:19:11.054893 18019 raft_consensus.cc:697] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:11.073956 10766 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:19:11.074205 10699 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42968 (request call id 249) took 12137 ms (client timeout 15000 ms). Trace:
I20260709 13:19:11.075028 18019 consensus_queue.cc:237] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:11.078922 18176 consensus_queue.cc:1048] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:19:11.079092 11478 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:11.080538 11478 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:11.081329 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:11.082212 11478 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:11.082751 18164 raft_consensus.cc:3055] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:19:11.082973 18201 raft_consensus.cc:2804] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:11.083248 18164 raft_consensus.cc:740] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:11.083674 18201 raft_consensus.cc:697] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 17 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:11.084357 18164 consensus_queue.cc:260] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:11.084739 18201 consensus_queue.cc:237] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:11.084968 17756 consensus_queue.cc:237] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:11.036823 18120 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:11.086967 18122 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:11.087530 18122 raft_consensus.cc:697] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 21 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:11.088938 10946 raft_consensus.cc:2393] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 20: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:11.082257 10942 raft_consensus.cc:2410] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:19:11.089504 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:11.090126 10950 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 22.
I20260709 13:19:11.063128 10947 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 18.
I20260709 13:19:11.091362 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:11.130885 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 76332.7us
I20260709 13:19:11.131907 10916 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:19:11.133282 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 51165.1us
I20260709 13:19:11.093808 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:11.134074 11478 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:11.137729 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:11.069690 17927 raft_consensus.cc:697] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 28 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:10.967193 18205 consensus_queue.cc:1048] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.130s
I20260709 13:19:11.029254 18174 consensus_queue.cc:1048] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.061s
I20260709 13:19:11.139760 10916 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:11.145742 10916 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:11.148850 18054 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:11.149317 18054 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:11.149648 18054 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:11.085688 18164 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:11.152382 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:11.060395 17411 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:11.153096 10794 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Advancing to term 22
I20260709 13:19:11.033152 18142 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:11.154454 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:11.155400 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:11.160300 10944 raft_consensus.cc:2393] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 18: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:11.132961 17777 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:11.060139 18170 consensus_queue.cc:1059] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.091s
I20260709 13:19:11.142591 18120 raft_consensus.cc:2804] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:19:11.150146 18044 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:11.180540 10795 raft_consensus.cc:1240] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 20. Current term is 21. Ops: [20.16-20.16]
W20260709 13:19:11.165555 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 284541us
I20260709 13:19:11.180830 11478 leader_election.cc:400] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for earlier term 18. Current term is 19.
I20260709 13:19:11.181375 11478 leader_election.cc:403] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:11.145972 18116 raft_consensus.cc:2804] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:11.168174 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:11.163288 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:11.171631 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:11.172632 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:11.175664 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:11.175733 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:11.176085 10873 raft_consensus.cc:1240] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:11.176676 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:11.164460 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:11.146638 18158 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:11.165387 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:11.181154 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:11.166747 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:11.194567 10942 raft_consensus.cc:2393] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 28: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:11.190857 10872 raft_consensus.cc:2393] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 20: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:19:11.187328 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.195009 10870 raft_consensus.cc:1240] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:11.191159 10869 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:11.191380 11507 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:11.192824 11511 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 19.
I20260709 13:19:11.173532 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:11.199663 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:11.200582 10943 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 19.
I20260709 13:19:11.200723 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:11.190310 11503 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:11.201567 10873 raft_consensus.cc:2393] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 21: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:11.159653 17996 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:11.192488 10801 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 19.
W20260709 13:19:11.214915 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.176577 18120 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:11.238689 18120 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:11.180778 18139 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:11.088387 18122 consensus_queue.cc:237] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:11.234566 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.166240 18161 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:11.236815 10794 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 22.
I20260709 13:19:11.198923 10796 raft_consensus.cc:2410] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:19:11.242700 10876 raft_consensus.cc:1240] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 LEADER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 21. Current term is 22. Ops: []
W20260709 13:19:11.247715 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.189579 18116 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:11.195281 18158 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:11.250192 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:11.251717 18200 log.cc:927] Time spent T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
I20260709 13:19:11.251812 18207 consensus_queue.cc:1059] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:19:11.074609 10699 rpcz_store.cc:269] 0709 13:18:58.892654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.894145 (+  1491us) service_pool.cc:224] Handling call
0709 13:19:11.021460 (+12127315us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:07.452697 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:07.638233 (+185536us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:07.638276 (+    43us) write_op.cc:435] Acquired schema lock
0709 13:19:07.638301 (+    25us) tablet.cc:662] Decoding operations
0709 13:19:07.640147 (+  1846us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:07.640215 (+    68us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:07.640246 (+    31us) tablet.cc:685] Acquiring locks for 24 operations
0709 13:19:07.698625 (+ 58379us) tablet.cc:701] Row locks acquired
0709 13:19:07.698644 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:19:07.698839 (+   195us) write_op.cc:270] Start()
0709 13:19:07.698966 (+   127us) write_op.cc:276] Timestamp: P: 1783603147698747 usec, L: 0
0709 13:19:07.698987 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:19:07.699727 (+   740us) log.cc:844] Serialized 9489 byte log entry
0709 13:19:07.799545 (+ 99818us) op_driver.cc:464] REPLICATION: finished
0709 13:19:08.149399 (+349854us) spinlock_profiling.cc:243] Waited 3.96 ms on lock 0x7b180034e880. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc83f3 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:19:08.149620 (+   221us) write_op.cc:301] APPLY: starting
0709 13:19:08.149760 (+   140us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:09.023591 (+873831us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:09.035966 (+ 12375us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:09.761280 (+725314us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:09.783277 (+ 21997us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=48,key_file_lookups=48,delta_file_lookups=48,mrs_lookups=0
0709 13:19:09.783372 (+    95us) write_op.cc:312] APPLY: finished
0709 13:19:09.784091 (+   719us) log.cc:844] Serialized 212 byte log entry
0709 13:19:09.940574 (+156483us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:10.082324 (+141750us) write_op.cc:454] Released schema lock
0709 13:19:10.109652 (+ 27328us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:10.167024 (+ 57372us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:10.171948 (+  4924us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":230,"reported_tablets":50,"spinlock_wait_cycles":779008,"child_traces":[["op",{"apply.queue_time_us":177870,"apply.run_cpu_time_us":63234,"apply.run_wall_time_us":2006000,"cfile_cache_hit":98,"cfile_cache_hit_bytes":104495,"num_ops":24,"prepare.queue_time_us":378640,"prepare.run_cpu_time_us":9726,"prepare.run_wall_time_us":267103,"raft.queue_time_us":905,"raft.run_cpu_time_us":15788,"raft.run_wall_time_us":388725,"replication_time_us":8107,"spinlock_wait_cycles":9112448,"thread_start_us":602990,"threads_started":5,"wal-append.queue_time_us":375512,"wal-append.run_cpu_time_us":3521,"wal-append.run_wall_time_us":2116449}]]}
W20260709 13:19:11.237841 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 356542us
I20260709 13:19:11.233955 18151 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:11.263703 10944 raft_consensus.cc:1275] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
W20260709 13:19:11.272856 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.275934 10794 raft_consensus.cc:1240] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:11.264245 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:11.259822 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:11.188795 17777 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:11.263721 17834 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:11.194258 18039 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:11.307508 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:11.304450 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.286805 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:11.307555 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:11.309307 10945 raft_consensus.cc:2393] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 18: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
W20260709 13:19:11.296590 10766 leader_election.cc:336] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Timed out: RequestConsensusVote RPC to 127.9.116.131:45553 timed out after 1.965s (SENT)
I20260709 13:19:11.311436 10766 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:11.247741 10841 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:11.282657 10796 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 21.
I20260709 13:19:11.295867 11507 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 19.
I20260709 13:19:11.336045 11503 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 24.
I20260709 13:19:11.259477 18116 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:11.337316 10869 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 22.
I20260709 13:19:11.339643 10872 raft_consensus.cc:1275] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:19:11.257638 18158 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:19:11.345058 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.341799 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:11.338546 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:11.370139 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:11.372112 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:11.362869 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 115189us
I20260709 13:19:11.375003 10692 catalog_manager.cc:5697] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
W20260709 13:19:11.376184 18195 log.cc:927] Time spent T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.349s	user 0.002s	sys 0.000s
I20260709 13:19:11.377370 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:11.371111 10867 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 20.
I20260709 13:19:11.379745 10841 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:11.389834 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:11.253028 18139 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:11.392388 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 231189us
W20260709 13:19:11.365096 17729 consensus_meta.cc:287] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.561s	user 0.010s	sys 0.000s
I20260709 13:19:11.394773 11478 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:19:11.398113 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.403232 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:11.337869 18109 raft_consensus.cc:2749] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for earlier term 18. Current term is 19.
I20260709 13:19:11.396238 11478 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:11.330945 18168 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:11.327001 18155 raft_consensus.cc:2749] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:11.264734 18161 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:11.257341 17927 consensus_queue.cc:237] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 26.13, Last appended by leader: 13, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:11.427417 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 47756.6us
I20260709 13:19:11.426641 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:11.440831 10872 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 21.
I20260709 13:19:11.434368 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:11.390751 18204 raft_consensus.cc:2804] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:11.434401 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:11.444131 10945 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 23.
W20260709 13:19:11.448351 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.338907 17834 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:11.423628 18147 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:11.427386 18206 log.cc:927] Time spent T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.391s	user 0.000s	sys 0.002s
W20260709 13:19:11.430104 18199 log.cc:927] Time spent T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:19:11.427862 18201 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:19:11.431528 18175 raft_consensus.cc:2804] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:11.474463 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:11.425076 18113 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:11.490875 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:11.469175 10874 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 22.
W20260709 13:19:11.493240 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 52844.9us
I20260709 13:19:11.499210 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:11.502009 10945 raft_consensus.cc:2368] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 21. Current term is 22.
I20260709 13:19:11.495975 10869 raft_consensus.cc:2393] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 24: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
W20260709 13:19:11.473920 18045 consensus_meta.cc:287] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.738s	user 0.007s	sys 0.003s
I20260709 13:19:11.465044 18211 raft_consensus.cc:2804] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:11.531054 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:11.540467 11503 raft_consensus.cc:1240] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:11.540896 10870 raft_consensus.cc:2393] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 22: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:11.492125 18175 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:11.538973 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:11.526568 18163 consensus_meta.cc:287] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.754s	user 0.011s	sys 0.000s
I20260709 13:19:11.539315 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:11.552302 10872 raft_consensus.cc:2368] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 21. Current term is 22.
W20260709 13:19:11.525290 18208 log.cc:927] Time spent T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:19:11.551571 18175 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:11.433279 10692 catalog_manager.cc:5697] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:11.474126 18147 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:11.556974 18147 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:11.538008 18045 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:11.557699 18147 raft_consensus.cc:2749] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:11.496881 18201 raft_consensus.cc:697] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 23 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:11.559092 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:11.559204 18163 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:11.559995 11504 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 21: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:11.559592 18201 consensus_queue.cc:237] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:11.533933 18215 log.cc:927] Time spent T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 13:19:11.562618 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:11.532143 18139 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:11.531069 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:11.563280 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:11.540066 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:11.563519 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:11.564337 10869 raft_consensus.cc:2393] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 23: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:11.433883 18168 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:11.539340 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:11.566146 11509 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:11.562562 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:11.563606 11511 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:11.567170 10867 raft_consensus.cc:2393] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 21: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:11.491894 18173 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:11.568051 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:11.556087 10692 catalog_manager.cc:5697] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 13 to 18. New cstate: current_term: 18 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:11.538420 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:11.571663 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:11.573729 11512 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:19:11.576152 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.505061 18204 raft_consensus.cc:697] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:11.584147 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:11.585078 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:19:11.561918 18214 log.cc:927] Time spent T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.177s	user 0.001s	sys 0.000s
I20260709 13:19:11.589592 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:11.587005 10944 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 18.
I20260709 13:19:11.599439 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:11.599260 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:11.570861 18209 log.cc:927] Time spent T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
W20260709 13:19:11.582134 18216 log.cc:927] Time spent T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 13:19:11.622668 10943 raft_consensus.cc:2368] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 19. Current term is 20.
W20260709 13:19:11.623356 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 326786us
I20260709 13:19:11.624189 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:11.627032 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:11.628026 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:11.631692 10801 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:19:11.565652 18163 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:11.638989 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:11.639747 10796 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
W20260709 13:19:11.633832 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.642619 10766 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:11.583205 18139 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:11.648133 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:11.573092 18211 raft_consensus.cc:697] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:11.650005 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:11.650775 10797 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:19:11.585916 17411 raft_consensus.cc:2749] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:11.659471 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:11.646224 10766 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:11.664664 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:11.662833 10766 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:11.667804 11512 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 21.
W20260709 13:19:11.647955 18215 log.cc:927] Time spent T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:19:11.640522 18147 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:11.676927 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:11.677907 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:11.670526 10796 raft_consensus.cc:2393] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 20: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:11.683338 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:11.694700 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:11.695533 10950 raft_consensus.cc:2393] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 20: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:11.684805 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:11.685693 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:11.686349 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:11.697315 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:11.687376 10799 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 22.
I20260709 13:19:11.582479 18173 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:11.701462 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.744287 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:11.739547 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:11.749277 10868 raft_consensus.cc:2393] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 19: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:11.674038 18045 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:11.752362 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:11.753113 10793 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 17.
I20260709 13:19:11.750273 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:11.753959 10873 raft_consensus.cc:2393] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 21: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:11.686759 18039 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:11.746517 18209 log.cc:927] Time spent T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:19:11.669065 18139 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:11.763370 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:11.765866 10944 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 25.
I20260709 13:19:11.761875 10797 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 28.
I20260709 13:19:11.738360 17411 raft_consensus.cc:2804] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:11.764091 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:11.793074 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:11.816695 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.739907 17996 raft_consensus.cc:697] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:11.743562 18147 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:11.801892 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:11.828792 10792 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.009s	sys 0.009s
W20260709 13:19:11.829416 11508 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.011s	sys 0.007s
I20260709 13:19:11.757121 18169 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:11.801005 18164 consensus_meta.cc:287] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.641s	user 0.010s	sys 0.001s
I20260709 13:19:11.853681 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:11.843695 10950 raft_consensus.cc:3055] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:11.871367 10950 raft_consensus.cc:740] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:11.876088 18169 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:11.878212 18169 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:11.873663 10871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 2588) took 1054 ms. Trace:
W20260709 13:19:11.880036 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.882815 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:11.883421 10872 raft_consensus.cc:1240] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:19:11.880790 10868 raft_consensus.cc:1275] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
W20260709 13:19:11.864310 11508 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.011s	sys 0.007s
I20260709 13:19:11.854104 10794 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Advancing to term 21
W20260709 13:19:11.828394 17756 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.000s	sys 0.023s
I20260709 13:19:11.884991 10942 raft_consensus.cc:1275] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 17. (term mismatch)
I20260709 13:19:11.886893 10942 pending_rounds.cc:85] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 16
I20260709 13:19:11.887301 10942 pending_rounds.cc:107] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 21.17
I20260709 13:19:11.887626 10942 raft_consensus.cc:2889] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:11.886106 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:11.802718 17411 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:11.879181 10871 rpcz_store.cc:276] 0709 13:19:10.800712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:10.812018 (+ 11306us) service_pool.cc:224] Handling call
0709 13:19:10.836416 (+ 24398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:10.836607 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:19:11.061830 (+225223us) log.cc:844] Serialized 14 byte log entry
0709 13:19:11.854521 (+792691us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:11.854569 (+    48us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:11.854615 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.854684 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:11.854842 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39220,"prepare.queue_time_us":1774,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":58199,"raft.run_cpu_time_us":54,"raft.run_wall_time_us":55,"spinlock_wait_cycles":8948608,"thread_start_us":977949,"threads_started":3,"wal-append.queue_time_us":567961}
W20260709 13:19:11.805657 18054 consensus_meta.cc:287] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.656s	user 0.010s	sys 0.001s
W20260709 13:19:11.865231 10792 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.009s	sys 0.009s
I20260709 13:19:11.904628 10873 raft_consensus.cc:1275] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
W20260709 13:19:11.881975 18209 log.cc:927] Time spent T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
W20260709 13:19:11.901911 18044 consensus_meta.cc:287] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.708s	user 0.011s	sys 0.000s
W20260709 13:19:11.899631 17756 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.000s	sys 0.024s
I20260709 13:19:11.922019 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:11.929708 10800 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 21: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:11.923352 10872 raft_consensus.cc:1240] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:19:11.925266 10795 raft_consensus.cc:1275] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 13:19:11.923632 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.763154 18204 consensus_queue.cc:237] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:11.917145 10950 consensus_queue.cc:260] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:11.899312 18233 raft_consensus.cc:2804] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:11.926206 10794 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 21.
I20260709 13:19:11.937976 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:11.945381 10943 raft_consensus.cc:2410] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:19:11.939327 10950 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:11.933827 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:11.910341 17411 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:11.918756 18161 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:11.948482 10916 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:11.952059 18136 consensus_queue.cc:1059] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 13:19:11.952805 18170 consensus_queue.cc:1059] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:19:11.952277 10692 catalog_manager.cc:5697] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 10 to 15, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 15 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:11.955943 18157 raft_consensus.cc:3055] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:11.956449 18157 raft_consensus.cc:740] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:11.934020 18044 raft_consensus.cc:2749] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:11.957917 18173 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:11.968052 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:11.924278 18232 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:19:11.979421 10947 raft_consensus.cc:2410] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 20 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:19:11.981515 10950 raft_consensus.cc:1275] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
W20260709 13:19:11.983047 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.998258 11509 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 22.
I20260709 13:19:12.000558 10948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 1297) took 1069 ms. Trace:
I20260709 13:19:11.899999 18211 consensus_queue.cc:237] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:11.970902 18116 consensus_meta.cc:287] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.582s	user 0.011s	sys 0.000s
I20260709 13:19:11.914486 18054 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:12.004501 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 56081.7us
I20260709 13:19:12.001096 10948 rpcz_store.cc:276] 0709 13:19:10.931502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:10.931721 (+   219us) service_pool.cc:224] Handling call
0709 13:19:10.932022 (+   301us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:10.932358 (+   336us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:19:11.008571 (+ 76213us) log.cc:844] Serialized 14 byte log entry
0709 13:19:11.083666 (+ 75095us) spinlock_profiling.cc:243] Waited 19.1 ms on lock 0x7b1800372f40. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc83f3 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:19:11.569456 (+485790us) log.cc:844] Serialized 14 byte log entry
0709 13:19:11.569635 (+   179us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:11.585335 (+ 15700us) log.cc:844] Serialized 26 byte log entry
0709 13:19:11.585486 (+   151us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:19:11.585527 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.585592 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:12.000323 (+414731us) raft_consensus.cc:1685] finished
0709 13:19:12.000346 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:12.000490 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23227,"prepare.queue_time_us":33543,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":5521,"raft.queue_time_us":536389,"raft.run_cpu_time_us":501,"raft.run_wall_time_us":62962,"spinlock_wait_cycles":43874176,"thread_start_us":509324,"threads_started":2,"wal-append.queue_time_us":337593}
I20260709 13:19:11.943871 17821 consensus_queue.cc:1059] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
W20260709 13:19:11.994956 18120 consensus_meta.cc:287] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.756s	user 0.009s	sys 0.002s
W20260709 13:19:12.007477 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 344939us
I20260709 13:19:12.010170 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:11.957449 18157 consensus_queue.cc:260] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:12.028407 10792 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:12.028482 11511 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 23.
W20260709 13:19:12.045037 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 243757us
I20260709 13:19:12.018800 18229 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:19:12.017614 18227 raft_consensus.cc:3055] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:19:11.963027 17996 consensus_queue.cc:237] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:12.092375 10945 raft_consensus.cc:2368] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 21. Current term is 22.
I20260709 13:19:12.094719 10801 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
W20260709 13:19:12.092043 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 66646.1us
I20260709 13:19:11.954416 10692 catalog_manager.cc:5697] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 11 to 15, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 15 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:12.103974 10792 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 21.
W20260709 13:19:12.110534 18175 consensus_meta.cc:287] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.556s	user 0.010s	sys 0.000s
I20260709 13:19:12.111208 18225 consensus_queue.cc:1059] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:19:12.111135 18175 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:12.112757 18225 raft_consensus.cc:3055] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:19:12.113299 18225 raft_consensus.cc:740] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:12.114007 18175 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:12.092043 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 46268.5us
I20260709 13:19:12.135011 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:12.039178 18120 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:12.023484 18116 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:12.114501 18225 consensus_queue.cc:260] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:12.137540 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:12.141994 10950 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:19:12.138087 18232 raft_consensus.cc:697] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:12.138845 18261 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.038971 18136 consensus_queue.cc:1059] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.078s
I20260709 13:19:12.104455 18173 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:12.138628 10916 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:12.102890 18227 raft_consensus.cc:740] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:12.140645 17411 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:12.147905 10842 thread.cc:641] server-negotiator [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.002s	sys 0.009s
I20260709 13:19:12.144433 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:12.152959 10842 thread.cc:608] server-negotiator [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.002s	sys 0.010s
I20260709 13:19:12.153311 10943 raft_consensus.cc:2410] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 23 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 13:19:12.109242 18229 raft_consensus.cc:697] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 25 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:12.141186 18147 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:12.214704 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:12.147701 18233 raft_consensus.cc:697] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:19:12.223527 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:12.203205 18168 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:12.147379 18261 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:12.202678 18173 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:12.174333 18231 consensus_queue.cc:1048] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.125s
I20260709 13:19:12.270762 18232 consensus_queue.cc:237] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:12.319739 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 148610us
I20260709 13:19:12.321312 10916 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:19:12.331184 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 156963us
I20260709 13:19:12.331790 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:19:12.318639 10766 leader_election.cc:336] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Timed out: RequestConsensusVote RPC to 127.9.116.131:45553 timed out after 1.511s (SENT)
I20260709 13:19:12.327455 10801 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:19:12.339381 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:12.339929 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:12.336036 17411 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:12.340713 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:12.341204 18236 log.cc:927] Time spent T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.203s	user 0.004s	sys 0.004s
I20260709 13:19:12.342109 10950 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 21.
W20260709 13:19:12.344022 18158 consensus_meta.cc:287] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.955s	user 0.016s	sys 0.001s
I20260709 13:19:12.344164 11508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56416 (request call id 1115) took 1196 ms. Trace:
I20260709 13:19:12.344440 18158 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:12.344523 11508 rpcz_store.cc:276] 0709 13:19:11.148064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.148186 (+   122us) service_pool.cc:224] Handling call
0709 13:19:11.169893 (+ 21707us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:11.181168 (+ 11275us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:11.181198 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:11.194038 (+ 12840us) log.cc:844] Serialized 26 byte log entry
0709 13:19:11.906140 (+712102us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:11.906409 (+   269us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.909355 (+  2946us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:12.343924 (+434569us) raft_consensus.cc:1685] finished
0709 13:19:12.343947 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:12.344103 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4290,"raft.queue_time_us":63712,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":7108,"thread_start_us":634506,"threads_started":1,"wal-append.queue_time_us":766833}
W20260709 13:19:12.333689 10766 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.116.131:45553 timed out after 1.558s (SENT) [suppressed 41 similar messages]
W20260709 13:19:12.341276 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:12.320250 18147 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:12.350859 10766 leader_election.cc:336] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Timed out: RequestConsensusVote RPC to 127.9.116.131:45553 timed out after 1.640s (SENT)
W20260709 13:19:12.351475 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:12.352669 10801 raft_consensus.cc:1275] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
W20260709 13:19:12.357081 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:12.346609 18158 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:12.350133 18239 log.cc:927] Time spent T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.186s	user 0.000s	sys 0.008s
W20260709 13:19:12.364770 10766 leader_election.cc:336] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Timed out: RequestConsensusVote RPC to 127.9.116.131:45553 timed out after 1.648s (SENT)
W20260709 13:19:12.373138 10766 leader_election.cc:336] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Timed out: RequestConsensusVote RPC to 127.9.116.131:45553 timed out after 2.620s (SENT)
I20260709 13:19:12.340716 18169 raft_consensus.cc:2804] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:12.369630 18233 consensus_queue.cc:237] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:12.394337 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:12.394953 10801 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:12.350257 18204 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:12.362820 18244 log.cc:927] Time spent T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.255s	user 0.007s	sys 0.002s
I20260709 13:19:12.333139 18230 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:12.391031 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:12.391954 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:12.339099 18225 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:12.408120 10792 raft_consensus.cc:1240] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 19. Current term is 20. Ops: []
W20260709 13:19:12.388217 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:12.399870 10796 raft_consensus.cc:1240] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 LEADER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 13:19:12.406224 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:12.413518 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:12.420285 10801 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 21.
I20260709 13:19:12.398100 18169 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:12.423344 10842 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:12.428606 10766 leader_election.cc:336] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Timed out: RequestConsensusVote RPC to 127.9.116.131:45553 timed out after 1.607s (SENT)
W20260709 13:19:12.432819 10766 leader_election.cc:336] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Timed out: RequestConsensusVote RPC to 127.9.116.131:45553 timed out after 1.710s (SENT)
I20260709 13:19:12.366844 18147 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:12.437811 10842 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:12.439033 10842 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:12.423156 18230 raft_consensus.cc:697] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:12.431533 18169 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:12.473570 10842 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:12.412191 18204 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:12.469098 18247 log.cc:927] Time spent T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 13:19:12.440578 18161 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:12.476946 10792 raft_consensus.cc:1240] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:12.476092 18151 raft_consensus.cc:2804] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:12.431712 18229 consensus_queue.cc:237] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:12.488057 18252 log.cc:927] Time spent T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 13:19:12.488389 18240 log.cc:927] Time spent T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.185s	user 0.005s	sys 0.003s
I20260709 13:19:12.434168 18054 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:12.441192 18159 raft_consensus.cc:2749] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:12.441589 18120 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:12.500483 10950 raft_consensus.cc:1275] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:19:12.477665 18256 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.508301 11507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56410 (request call id 1144) took 1004 ms. Trace:
I20260709 13:19:12.421187 18227 consensus_queue.cc:260] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:12.497471 18055 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:12.497818 18151 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:12.525301 11506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56416 (request call id 1114) took 1377 ms. Trace:
I20260709 13:19:12.492833 18248 consensus_queue.cc:1048] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:19:12.534754 18173 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:12.508733 11507 rpcz_store.cc:276] 0709 13:19:11.499823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.502041 (+  2218us) service_pool.cc:224] Handling call
0709 13:19:11.502360 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:11.502699 (+   339us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:19:11.502751 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:11.513031 (+ 10280us) log.cc:844] Serialized 26 byte log entry
0709 13:19:11.866752 (+353721us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:11.866827 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.866905 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:12.504419 (+637514us) raft_consensus.cc:1685] finished
0709 13:19:12.504438 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:12.504599 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":35421,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":12731,"thread_start_us":304918,"threads_started":1,"wal-append.queue_time_us":442449}
I20260709 13:19:12.542857 18151 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:12.547451 10947 raft_consensus.cc:1240] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 23. Current term is 24. Ops: []
W20260709 13:19:12.546120 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 72641.4us
I20260709 13:19:12.542500 10766 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:12.540519 11506 rpcz_store.cc:276] 0709 13:19:11.147708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.147852 (+   144us) service_pool.cc:224] Handling call
0709 13:19:11.149328 (+  1476us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:11.433944 (+284616us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:11.433977 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:11.492956 (+ 58979us) log.cc:844] Serialized 52 byte log entry
0709 13:19:11.835706 (+342750us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:11.835774 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.835904 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:12.525038 (+689134us) raft_consensus.cc:1685] finished
0709 13:19:12.525060 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:12.525237 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19962,"raft.queue_time_us":18682,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":12189,"thread_start_us":585867,"threads_started":2,"wal-append.queue_time_us":515319}
I20260709 13:19:12.549520 18259 raft_consensus.cc:2764] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:19:12.549497 10842 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:12.551532 10842 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:12.541119 18148 raft_consensus.cc:3055] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:12.560217 17834 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:12.554507 18212 raft_consensus.cc:2804] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:12.568861 18212 raft_consensus.cc:697] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:12.570137 18212 consensus_queue.cc:237] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:12.575624 18210 raft_consensus.cc:2749] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:19:12.578949 18210 consensus_queue.cc:1048] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:19:12.581622 18207 consensus_queue.cc:1059] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:19:12.583184 18154 raft_consensus.cc:3055] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:12.583544 18154 raft_consensus.cc:740] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:12.584407 18154 consensus_queue.cc:260] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:12.586120 18154 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:12.568140 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 2713) took 1038 ms. Trace:
I20260709 13:19:12.546528 18227 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:12.545845 18261 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:12.594574 10869 rpcz_store.cc:276] 0709 13:19:11.529637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.564905 (+ 35268us) service_pool.cc:224] Handling call
0709 13:19:12.567881 (+1002976us) spinlock_profiling.cc:243] Waited 603 ms on lock 0x7b540009b5d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:12.568040 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1386235776}
I20260709 13:19:12.611967 10947 raft_consensus.cc:1240] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:12.611865 10916 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:12.617585 10916 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:12.620507 10766 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:12.623207 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:12.626556 10916 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:12.555553 18148 raft_consensus.cc:740] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:12.628021 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:12.539561 18256 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:12.635471 10766 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:12.637401 10766 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:12.609359 18159 consensus_queue.cc:1059] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.021s
I20260709 13:19:12.618573 18158 raft_consensus.cc:2749] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:19:12.629937 18251 log.cc:927] Time spent T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.217s	user 0.001s	sys 0.000s
I20260709 13:19:12.640631 10766 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:12.631639 17861 raft_consensus.cc:2804] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:12.647060 17411 raft_consensus.cc:2749] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:12.649555 10798 raft_consensus.cc:1275] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:19:12.648300 10766 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:12.628232 18261 consensus_queue.cc:1048] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.019s
I20260709 13:19:12.637048 17834 raft_consensus.cc:2749] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:12.654625 10766 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:12.643016 18259 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:12.657310 17411 raft_consensus.cc:2749] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:12.641587 17996 raft_consensus.cc:2749] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:12.652541 18271 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.659087 18259 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:12.661720 18148 consensus_queue.cc:260] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:12.650727 17861 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:12.593153 18230 consensus_queue.cc:237] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:12.671060 18262 raft_consensus.cc:2749] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:12.671576 18234 consensus_queue.cc:1059] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
W20260709 13:19:12.672189 18226 log.cc:927] Time spent T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.630s	user 0.003s	sys 0.000s
I20260709 13:19:12.672554 18260 consensus_queue.cc:1059] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.128s
I20260709 13:19:12.650259 18159 consensus_queue.cc:1243] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer e01ecd008edd4aa2b6b2b47efbc2ea87 log is divergent from this leader: its last log entry 24.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:19:12.674361 18147 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:12.661895 18212 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.688532 18212 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:12.685339 10949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41014 (request call id 2810) took 1124 ms. Trace:
I20260709 13:19:12.691558 10949 rpcz_store.cc:276] 0709 13:19:11.560938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.562191 (+  1253us) service_pool.cc:224] Handling call
0709 13:19:12.685075 (+1122884us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54000a96d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:12.685249 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2374011648}
I20260709 13:19:12.678963 17861 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:12.696831 18151 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:12.686519 18270 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:12.699156 18270 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:12.701555 18270 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:12.702374 17891 raft_consensus.cc:3055] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:12.661428 18250 raft_consensus.cc:2804] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:12.702838 17891 raft_consensus.cc:740] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:12.703076 18250 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:12.703620 18250 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:12.674772 10766 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:12.706447 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:12.707429 10767 leader_election.cc:336] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Timed out: RequestConsensusVote RPC to 127.9.116.130:37895 timed out after 1.511s (SENT)
I20260709 13:19:12.708071 10767 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:12.707487 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:12.709546 10947 raft_consensus.cc:1240] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:12.680558 18161 raft_consensus.cc:2749] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:12.711076 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:12.680428 18148 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:12.676270 18259 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:12.712818 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:12.713523 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:12.715423 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:12.717361 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:12.717626 10767 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:12.717911 10868 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 21.
I20260709 13:19:12.710340 17996 raft_consensus.cc:3055] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:12.703888 17891 consensus_queue.cc:260] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:12.723172 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:12.723747 17891 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:12.723968 18139 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:12.724014 10941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41014 (request call id 2809) took 1163 ms. Trace:
I20260709 13:19:12.724821 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:12.725406 10949 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 23.
I20260709 13:19:12.714267 18147 raft_consensus.cc:3055] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:12.726333 18147 raft_consensus.cc:740] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:12.724817 10941 rpcz_store.cc:276] 0709 13:19:11.560631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.572400 (+ 11769us) service_pool.cc:224] Handling call
0709 13:19:12.723013 (+1150613us) spinlock_profiling.cc:243] Waited 3.54 ms on lock 0x7b5800112598. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd90a22 00007f9c86ca9c09 00007f9c85e03ee0 00007f9c85ecd0ab 00007f9c85ecd131 00007f9c85e7b0a0 00007f9c85e7a984 00007f9c7f3f56d4 00007f9c7f3f5662
0709 13:19:12.723952 (+   939us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8142720}
I20260709 13:19:12.727397 18147 consensus_queue.cc:260] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:12.728662 18147 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:12.719120 18233 raft_consensus.cc:2749] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:19:12.720230 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 45506.3us
I20260709 13:19:12.687041 18120 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:12.731246 11504 raft_consensus.cc:1275] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:19:12.731609 18120 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:12.732772 18113 raft_consensus.cc:3055] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:12.733172 18113 raft_consensus.cc:740] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:12.733891 18120 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:12.696900 18116 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:12.735128 18261 raft_consensus.cc:2749] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:12.735910 10944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41014 (request call id 2815) took 1114 ms. Trace:
I20260709 13:19:12.736336 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:12.736275 10944 rpcz_store.cc:276] 0709 13:19:11.621690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.623858 (+  2168us) service_pool.cc:224] Handling call
0709 13:19:12.735757 (+1111899us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b5400170cd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:12.735861 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":309668736}
W20260709 13:19:12.737355 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:12.739068 10872 raft_consensus.cc:3055] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:19:12.739431 10872 raft_consensus.cc:740] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:12.743701 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:12.742650 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:12.744925 10944 raft_consensus.cc:2393] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 28: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:12.707360 18212 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:12.746239 10766 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:12.734338 18169 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:12.746635 10767 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:12.775239 10841 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:12.788343 10841 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:12.791298 10767 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:12.740160 10872 consensus_queue.cc:260] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:12.780251 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:12.721633 17996 raft_consensus.cc:740] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:12.733960 18113 consensus_queue.cc:260] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:12.795116 18113 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:12.796068 18212 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:19:12.796953 18168 raft_consensus.cc:2749] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:19:12.797024 18212 raft_consensus.cc:697] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:12.798172 18212 consensus_queue.cc:237] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:12.799930 18224 log.cc:927] Time spent T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 1.109s	user 0.010s	sys 0.000s
I20260709 13:19:12.793201 10767 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:12.801895 18168 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:12.803253 10767 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:12.781517 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:12.804298 10943 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 20.
I20260709 13:19:12.805284 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:12.806438 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:12.783151 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:12.678292 18271 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:12.809612 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:12.810366 10944 raft_consensus.cc:2393] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 20: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:12.810585 18271 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:12.783969 11506 raft_consensus.cc:1240] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 23. Current term is 24. Ops: [23.13-23.13]
W20260709 13:19:12.811340 10799 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2566) took 1654 ms (client timeout 1759 ms). Trace:
I20260709 13:19:12.784991 10766 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:19:12.811761 10799 rpcz_store.cc:269] 0709 13:19:11.156681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.156824 (+   143us) service_pool.cc:224] Handling call
0709 13:19:12.811184 (+1654360us) spinlock_profiling.cc:243] Waited 495 ms on lock 0x7b54000bd1d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:12.811286 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1138060672}
I20260709 13:19:12.785609 11504 raft_consensus.cc:1240] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 23. Current term is 24. Ops: []
I20260709 13:19:12.785768 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:12.773494 18261 raft_consensus.cc:2749] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:12.776986 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:12.792752 11507 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
I20260709 13:19:12.773245 18262 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.775444 18151 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:12.803411 10942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 2797) took 1567 ms. Trace:
W20260709 13:19:12.806932 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:12.781430 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:12.794540 17996 consensus_queue.cc:260] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:12.807996 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 1144) took 1345 ms. Trace:
I20260709 13:19:12.823246 10767 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:12.811851 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:12.816305 10800 raft_consensus.cc:1275] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 28 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 26 index: 13. Preceding OpId from leader: term: 28 index: 14. (index mismatch)
W20260709 13:19:12.817914 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:12.818096 10950 raft_consensus.cc:2368] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 20. Current term is 21.
I20260709 13:19:12.819921 10870 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 20.
I20260709 13:19:12.813221 10766 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:12.782675 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:12.824429 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:12.825244 10872 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:12.825348 18260 raft_consensus.cc:2749] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:12.825899 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:12.787336 18233 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:19:12.822223 10942 rpcz_store.cc:276] 0709 13:19:11.235612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.243551 (+  7939us) service_pool.cc:224] Handling call
0709 13:19:11.267534 (+ 23983us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:11.903352 (+635818us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:12.803141 (+899789us) spinlock_profiling.cc:243] Waited 617 ms on lock 0x7b54002a53d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:19:12.803352 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1418969856}
I20260709 13:19:12.833041 17411 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:12.833482 10793 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:12.787441 18139 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:19:12.836539 17411 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:12.836995 17411 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:12.838289 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:12.839430 10800 raft_consensus.cc:2410] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 20 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:19:12.823764 10873 rpcz_store.cc:276] 0709 13:19:11.462832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.914267 (+451435us) service_pool.cc:224] Handling call
0709 13:19:11.920204 (+  5937us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:11.920631 (+   427us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:19:12.096894 (+176263us) log.cc:844] Serialized 14 byte log entry
0709 13:19:12.318547 (+221653us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:12.701529 (+382982us) spinlock_profiling.cc:243] Waited 3.75 ms on lock 0x7b1800343720. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc8522 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:19:12.802123 (+100594us) spinlock_profiling.cc:243] Waited 2.04 ms on lock 0x7b2c00029f28. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7f7aea11 00007f9c7f7be372 00007f9c80ef0a01 00007f9c80eb543a 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2
0709 13:19:12.802507 (+   384us) log.cc:844] Serialized 26 byte log entry
0709 13:19:12.802654 (+   147us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:12.802687 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:12.802749 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:12.807803 (+  5054us) raft_consensus.cc:1685] finished
0709 13:19:12.807822 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:12.807941 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":284094,"prepare.run_cpu_time_us":10191,"prepare.run_wall_time_us":102978,"raft.queue_time_us":59513,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":2671,"spinlock_wait_cycles":45492096,"thread_start_us":314053,"threads_started":2,"wal-append.queue_time_us":338242}
I20260709 13:19:12.839996 10767 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:12.812526 18154 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:12.844245 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:12.845115 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:12.786741 18120 consensus_queue.cc:1048] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.043s
I20260709 13:19:12.849637 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:12.850464 10868 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 19.
I20260709 13:19:12.852862 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:12.834909 18168 raft_consensus.cc:2749] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:19:12.834669 17834 raft_consensus.cc:2749] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:12.867274 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:12.837870 18139 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:12.826465 18271 consensus_queue.cc:1048] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.033s
I20260709 13:19:12.870688 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:12.856918 17996 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:12.856453 18233 raft_consensus.cc:2749] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:12.875797 10872 raft_consensus.cc:1275] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 13. (term mismatch)
I20260709 13:19:12.883186 10872 pending_rounds.cc:85] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 12
I20260709 13:19:12.856791 18154 raft_consensus.cc:697] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:12.837812 18161 consensus_queue.cc:1059] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.031s
I20260709 13:19:12.836933 18262 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:12.880868 17411 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:12.871770 18139 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:19:12.892395 10872 pending_rounds.cc:107] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 23.13
I20260709 13:19:12.892788 10872 raft_consensus.cc:2889] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:19:12.892921 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 53055.4us
I20260709 13:19:12.893227 10949 raft_consensus.cc:1275] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:19:12.894255 10767 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:12.897244 17411 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:12.901893 10794 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 3259) took 1275 ms (client timeout 1604 ms). Trace:
W20260709 13:19:12.902262 10794 rpcz_store.cc:269] 0709 13:19:11.626685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.626809 (+   124us) service_pool.cc:224] Handling call
0709 13:19:12.901720 (+1274911us) spinlock_profiling.cc:243] Waited 223 ms on lock 0x7b54001aa9d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:12.901833 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":512115072}
I20260709 13:19:12.903242 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:12.904394 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:12.905747 10767 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:12.907665 10767 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:12.907980 10872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2722) took 1375 ms. Trace:
I20260709 13:19:12.913122 10944 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
I20260709 13:19:12.913492 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54948 (request call id 2702) took 1641 ms. Trace:
I20260709 13:19:12.913916 10867 rpcz_store.cc:276] 0709 13:19:11.263101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.402879 (+139778us) service_pool.cc:224] Handling call
0709 13:19:12.904255 (+1501376us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b54003174d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:12.904365 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":370941696}
W20260709 13:19:12.914932 18149 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 24 current term: 24 is higher.
I20260709 13:19:12.915622 18261 raft_consensus.cc:2749] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:12.908896 10767 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:12.918466 10767 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:12.908360 10872 rpcz_store.cc:276] 0709 13:19:11.532153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.931107 (+398954us) service_pool.cc:224] Handling call
0709 13:19:11.935096 (+  3989us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:12.738976 (+803880us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b54003ea920. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80e2cdc0 00007f9c80eb71c0 00007f9c80eb8231 00007f9c80eb4d8f 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c
0709 13:19:12.907737 (+168761us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:12.907925 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":308947840}
I20260709 13:19:12.925724 17411 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:12.925740 10767 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:12.903182 18212 raft_consensus.cc:3055] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:19:12.900403 18233 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:12.933915 17411 raft_consensus.cc:2749] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:12.891456 17861 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:12.903865 18055 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:12.947068 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:12.951087 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:12.951552 18122 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.951797 10794 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:12.952270 18122 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:12.929699 18261 raft_consensus.cc:2749] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:12.958356 10869 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 21.
I20260709 13:19:12.901073 18154 consensus_queue.cc:237] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:12.942973 18212 raft_consensus.cc:740] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:12.960664 17411 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:19:12.963303 18279 log.cc:927] Time spent T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 13:19:12.964150 18278 log.cc:927] Time spent T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:19:12.968825 17411 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:12.972091 17411 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:12.954605 18122 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:12.976946 17861 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:12.972651 18169 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:12.978746 10868 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 20.
I20260709 13:19:12.990021 18262 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:12.990283 18177 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.990854 17729 raft_consensus.cc:3055] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:19:12.991361 17729 raft_consensus.cc:740] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:12.991052 18177 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:12.992260 17729 consensus_queue.cc:260] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:12.993378 17729 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:12.997292 17777 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:12.998124 17777 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.989089 10794 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 22.
I20260709 13:19:12.982300 10875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 2587) took 2182 ms. Trace:
I20260709 13:19:13.008358 17411 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:12.981616 10945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41014 (request call id 2818) took 1238 ms. Trace:
I20260709 13:19:12.986199 10793 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 20.
I20260709 13:19:13.009650 10945 rpcz_store.cc:276] 0709 13:19:11.743274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.743945 (+   671us) service_pool.cc:224] Handling call
0709 13:19:12.981385 (+1237440us) spinlock_profiling.cc:243] Waited 343 ms on lock 0x7b54002a58d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:12.981549 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":789926144}
I20260709 13:19:13.006273 10916 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:13.012326 10916 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:13.006996 10875 rpcz_store.cc:276] 0709 13:19:10.800309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:10.801204 (+   895us) service_pool.cc:224] Handling call
0709 13:19:10.801589 (+   385us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:19:10.801992 (+   403us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:10.802021 (+    29us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:19:10.802668 (+   647us) log.cc:844] Serialized 78 byte log entry
0709 13:19:11.177111 (+374443us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:11.234014 (+ 56903us) log.cc:844] Serialized 14 byte log entry
0709 13:19:11.249122 (+ 15108us) log.cc:844] Serialized 14 byte log entry
0709 13:19:11.249336 (+   214us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.249408 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:12.807207 (+1557799us) spinlock_profiling.cc:243] Waited 22.3 ms on lock 0x7b180036c400. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc83f3 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:19:12.979463 (+172256us) spinlock_profiling.cc:243] Waited 5.06 ms on lock 0x7b2400124028. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dc4e9c4 00007f9c7dc4cacb 00007f9c7de07dc9 00007f9c7de07a55 00007f9c7de06fb0 00007f9c80a7d488 00007f9c80a7dbe8 00007f9c80a55398 00007f9c80a5076a 00007f9c80a4fbb6
0709 13:19:12.982022 (+  2559us) raft_consensus.cc:1685] finished
0709 13:19:12.982046 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:12.982206 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26362,"prepare.queue_time_us":101447,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":74,"raft.queue_time_us":39536,"raft.run_cpu_time_us":304,"raft.run_wall_time_us":71662,"spinlock_wait_cycles":129702912,"thread_start_us":355591,"threads_started":1,"wal-append.queue_time_us":473819}
I20260709 13:19:13.019194 18169 raft_consensus.cc:2804] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:13.021822 18169 raft_consensus.cc:697] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:13.022696 18169 consensus_queue.cc:237] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:13.023258 18142 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:13.027081 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:13.030009 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:13.030436 17411 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:13.028247 10842 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:19:13.031525 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:12.967819 18212 consensus_queue.cc:260] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:13.030684 10794 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 22.
I20260709 13:19:13.033697 18227 raft_consensus.cc:2804] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:13.033739 18149 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.994099 18177 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:13.041764 10916 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:13.037984 18148 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:13.099239 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:13.099566 18148 raft_consensus.cc:697] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:13.049623 18259 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:13.051144 18212 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:13.051633 18227 raft_consensus.cc:697] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 20 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:13.057304 18250 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:13.059333 18139 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:13.011415 17777 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:13.117931 17777 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:13.061606 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:13.061386 18149 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:13.063411 18164 raft_consensus.cc:3055] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:19:13.119762 18164 raft_consensus.cc:740] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:13.120841 18149 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:13.069236 18054 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:13.069433 10874 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 2595) took 1912 ms (client timeout 1759 ms). Trace:
W20260709 13:19:13.069905 18202 log.cc:927] Time spent T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 1.829s	user 0.004s	sys 0.011s
I20260709 13:19:13.071265 18238 consensus_queue.cc:1048] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:19:13.076402 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:19:13.124171 10874 rpcz_store.cc:269] 0709 13:19:11.157381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.157521 (+   140us) service_pool.cc:224] Handling call
0709 13:19:13.069232 (+1911711us) spinlock_profiling.cc:243] Waited 276 ms on lock 0x7b54000f73d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:13.069358 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":633807104}
I20260709 13:19:13.125684 10795 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
W20260709 13:19:13.085386 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:13.086524 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:13.088948 18157 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:13.092962 10799 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:13.097671 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:13.099985 10793 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Advancing to term 22
W20260709 13:19:13.099229 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 57553.7us
I20260709 13:19:13.099172 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 29 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:19:13.101593 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 176025us
W20260709 13:19:13.104543 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 145331us
I20260709 13:19:13.107528 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:13.129280 10797 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:13.040556 18122 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:13.122351 10800 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 23.
I20260709 13:19:13.107388 18170 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:19:13.131323 10766 leader_election.cc:336] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Timed out: RequestConsensusVote RPC to 127.9.116.131:45553 timed out after 2.162s (SENT)
I20260709 13:19:13.133376 10798 raft_consensus.cc:2410] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 29 because replica has last-logged OpId of term: 26 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 12.
W20260709 13:19:13.135509 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:13.134320 10766 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:19:13.136184 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:13.137243 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:13.114487 18260 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:13.137919 10944 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:13.139952 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:13.141669 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:13.142493 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:13.143003 10795 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:13.144888 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:13.145354 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:13.147137 10950 raft_consensus.cc:3055] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:19:13.147507 10950 raft_consensus.cc:740] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:13.121083 18164 consensus_queue.cc:260] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:13.145464 10942 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 19.
I20260709 13:19:13.150712 10950 consensus_queue.cc:260] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:13.154852 10799 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
I20260709 13:19:13.156275 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:13.156782 11508 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:13.137187 18170 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:13.138068 18229 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:13.149791 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:13.165599 11509 raft_consensus.cc:1240] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 13:19:13.160224 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:13.163848 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:13.140574 18177 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:13.164991 10797 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 22.
I20260709 13:19:13.154832 10793 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 22.
I20260709 13:19:13.166236 10950 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:13.166699 10798 raft_consensus.cc:2393] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 20: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:13.167088 11506 raft_consensus.cc:2393] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 18: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:13.167573 10766 leader_election.cc:400] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Vote denied by peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553) with higher term. Message: Invalid argument: T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 20. Current term is 21.
I20260709 13:19:13.169000 10766 leader_election.cc:403] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:13.171376 11512 raft_consensus.cc:1240] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:13.172827 17411 raft_consensus.cc:2749] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553) with higher term. Message: Invalid argument: T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 20. Current term is 21.
I20260709 13:19:13.173002 10766 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:13.145958 18113 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:13.173895 17411 raft_consensus.cc:2749] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:13.169348 10916 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:13.174942 10766 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:13.134441 18054 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:13.172444 18245 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:13.178737 17834 raft_consensus.cc:2804] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:13.186573 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:13.149757 18230 consensus_queue.cc:1048] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:19:13.190119 10792 raft_consensus.cc:1240] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:13.173884 18177 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:13.191077 18227 consensus_queue.cc:237] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:13.192521 18259 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:13.192754 18177 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:13.171159 18229 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:13.182878 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:13.169904 18295 raft_consensus.cc:2749] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:13.178800 17729 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:13.187006 18113 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:13.195637 17729 raft_consensus.cc:697] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:19:13.195986 18277 log.cc:927] Time spent T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.325s	user 0.002s	sys 0.000s
I20260709 13:19:13.196622 17729 consensus_queue.cc:237] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:13.143972 18260 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:13.200132 18139 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 26 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:13.200592 18260 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:13.201102 18204 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:13.201376 11509 raft_consensus.cc:1240] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 21. Current term is 22. Ops: [19.16-21.17]
I20260709 13:19:13.202121 18256 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:13.203014 18250 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:13.203576 11512 raft_consensus.cc:1217] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Deduplicated request from leader. Original: 18.15->[20.16-21.17]   Dedup: 20.16->[21.17-21.17]
I20260709 13:19:13.185498 17834 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:13.165253 18170 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:13.182531 11503 raft_consensus.cc:1217] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 20 LEADER]: Deduplicated request from leader. Original: 18.14->[19.15-19.15]   Dedup: 19.15->[]
I20260709 13:19:13.205226 11503 raft_consensus.cc:1240] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 20 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:19:13.204197 11512 raft_consensus.cc:1240] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 21. Current term is 22. Ops: [20.16-21.17]
I20260709 13:19:13.184724 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:13.183137 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:13.208056 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:13.208536 10947 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 22.
I20260709 13:19:13.208148 10798 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 21.
I20260709 13:19:13.209594 10767 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:13.213634 10767 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:13.211454 11477 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:13.137828 18122 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:13.216832 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:13.179509 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:13.230119 18221 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:13.230015 18256 consensus_queue.cc:1048] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:19:13.230800 18221 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:13.231267 18221 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:13.209975 17834 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:13.151669 18169 consensus_queue.cc:1048] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.027s
I20260709 13:19:13.235246 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:13.235956 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:13.236796 11503 raft_consensus.cc:1240] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 23. Current term is 24. Ops: []
I20260709 13:19:13.237335 11507 raft_consensus.cc:1240] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:19:13.236743 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:13.238235 11510 raft_consensus.cc:1240] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 23. Current term is 25. Ops: []
I20260709 13:19:13.238945 11503 raft_consensus.cc:1240] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 23 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:13.239044 10874 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:53438
I20260709 13:19:13.239209 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:13.239632 10874 raft_consensus.cc:606] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Received request to transfer leadership to TS cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:13.239962 11510 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 21.
I20260709 13:19:13.240365 11503 raft_consensus.cc:1275] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:19:13.240561 11507 raft_consensus.cc:1240] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:13.240747 18250 raft_consensus.cc:2804] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:13.241159 18250 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:13.241528 18250 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:13.241681 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:13.242326 10870 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 17.
I20260709 13:19:13.177126 18164 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:13.243577 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:13.244120 10868 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:13.245138 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:13.245857 11505 raft_consensus.cc:2393] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 20: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:13.172786 18148 consensus_queue.cc:237] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:13.251694 10795 raft_consensus.cc:2410] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 20 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:19:13.254083 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 29 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:13.240020 10874 raft_consensus.cc:614] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:13.255302 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:13.227111 11508 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 24.
I20260709 13:19:13.277472 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:13.277613 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:13.270391 10767 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:13.311148 11509 raft_consensus.cc:1275] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 13:19:13.312633 10868 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 22.
I20260709 13:19:13.286234 10767 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:13.314177 18233 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:13.283180 10842 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:13.315181 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:13.315940 11512 raft_consensus.cc:1240] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 21 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:13.318382 10842 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:13.182861 18245 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:13.321698 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:13.323132 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:13.329444 10875 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 20.
I20260709 13:19:13.331645 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:19:13.337639 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:13.347559 10867 raft_consensus.cc:1240] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 21. Current term is 22. Ops: [18.11-21.12]
I20260709 13:19:13.348241 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:13.349351 10873 raft_consensus.cc:2393] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 23: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:13.350512 10801 raft_consensus.cc:1240] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 23. Current term is 24. Ops: [23.12-23.12]
I20260709 13:19:13.351455 10767 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:13.352185 10801 raft_consensus.cc:1240] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 22. Current term is 23. Ops: []
W20260709 13:19:13.352941 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:13.353395 10767 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:13.353735 10801 raft_consensus.cc:1240] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 23. Current term is 24. Ops: []
I20260709 13:19:13.357026 11478 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:13.217558 18168 raft_consensus.cc:2749] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:13.359728 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:13.360695 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:13.368024 10876 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:13.368534 10867 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:13.371359 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:13.371971 11508 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 20.
I20260709 13:19:13.278477 18235 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:13.376398 10793 raft_consensus.cc:1240] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:13.370754 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:13.316256 18233 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:13.315225 18295 raft_consensus.cc:2804] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:13.380820 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:13.381428 11509 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:13.382166 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:13.389178 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:13.365204 11478 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:13.215350 18207 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:13.367236 11505 raft_consensus.cc:1275] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:19:13.312206 18238 consensus_queue.cc:1048] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.072s
I20260709 13:19:13.396464 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:13.406134 10875 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:13.372969 18287 raft_consensus.cc:2749] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:19:13.373342 18234 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:13.373063 18154 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:13.367969 18155 raft_consensus.cc:2804] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:13.368088 18231 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:13.375063 18260 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:13.440773 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:13.441658 11512 raft_consensus.cc:2393] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 21: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:13.410363 10871 raft_consensus.cc:1275] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:19:13.411505 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:13.413908 10950 raft_consensus.cc:1275] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:19:13.407001 11478 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:13.444679 11478 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:13.372728 18142 consensus_queue.cc:1059] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 13:19:13.385316 18233 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:13.433702 11509 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 20.
I20260709 13:19:13.433310 10867 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 22.
I20260709 13:19:13.387713 18149 raft_consensus.cc:2804] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:19:13.418149 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:13.526362 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:13.459491 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:13.395875 18289 raft_consensus.cc:2749] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:13.397552 18295 raft_consensus.cc:697] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:13.448702 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:13.530704 10873 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 22.
I20260709 13:19:13.385464 17834 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:13.385438 18235 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:13.448089 18175 raft_consensus.cc:2749] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:19:13.449083 18231 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:13.431423 18204 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:13.462858 17777 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:13.502543 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:13.503670 10796 raft_consensus.cc:1240] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 17. Current term is 18. Ops: []
W20260709 13:19:13.504436 10686 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 2b82f18992434ed5ba58b805107060d4 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 468a8fd46e824f62b9c2c0d3532a6d26 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 21 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:13.550400 10799 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 20.
I20260709 13:19:13.504535 18229 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:13.506444 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:13.437626 18155 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:13.447814 18287 raft_consensus.cc:2749] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:13.410274 18207 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:13.563935 18207 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:13.465180 18148 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:13.565193 18148 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:13.568141 18148 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:13.463157 18154 raft_consensus.cc:697] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:13.569743 18154 consensus_queue.cc:237] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:13.475395 18260 raft_consensus.cc:697] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:13.528512 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:13.529045 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:13.513641 18230 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:13.572963 18260 consensus_queue.cc:237] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:13.573378 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:13.573985 10867 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 25.
I20260709 13:19:13.535775 10944 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 21.
I20260709 13:19:13.485677 18054 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:13.542017 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:13.440878 18234 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:13.578704 18234 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:13.543530 11478 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:19:13.544519 10875 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 22.
I20260709 13:19:13.579666 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:13.579949 18122 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:13.580396 10867 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 20.
W20260709 13:19:13.580920 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:13.581264 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:13.525556 18149 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:13.581936 18149 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:13.582268 10867 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d32f5e230e464e61b23a41962708d120"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:53438
W20260709 13:19:13.579684 18303 log.cc:927] Time spent T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 13:19:13.550877 10686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 46524.8us
I20260709 13:19:13.559113 10794 raft_consensus.cc:1240] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:13.585974 10870 raft_consensus.cc:2410] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 24 because replica has last-logged OpId of term: 23 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 15.
I20260709 13:19:13.560498 17777 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:13.555989 18155 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:13.573077 18230 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:13.580862 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:13.588580 18230 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:13.533483 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:13.589263 10873 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 24.
I20260709 13:19:13.588291 18221 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:13.591122 10875 raft_consensus.cc:1275] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 13:19:13.592597 18221 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:13.594923 17777 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:13.595654 10867 raft_consensus.cc:606] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Received request to transfer leadership to TS cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:13.598695 18155 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:13.599298 18245 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:13.600350 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:13.601222 11512 raft_consensus.cc:2393] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 23: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:13.602730 18155 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:13.511035 18296 consensus_queue.cc:1048] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.384s
W20260709 13:19:13.609001 18306 log.cc:927] Time spent T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 13:19:13.611341 18302 log.cc:927] Time spent T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:19:13.614730 18113 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:13.607402 11504 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 25.
I20260709 13:19:13.604110 10876 raft_consensus.cc:2468] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 25.
I20260709 13:19:13.628485 10876 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
 from {username='slave'} at 127.0.0.1:53438
I20260709 13:19:13.629273 10876 raft_consensus.cc:606] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Received request to transfer leadership to TS e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:13.629737 10876 raft_consensus.cc:614] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:13.636927 10874 raft_consensus.cc:1240] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 23. Current term is 25. Ops: []
W20260709 13:19:13.638351 18304 log.cc:927] Time spent T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:19:13.639642 18204 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:13.642719 10796 raft_consensus.cc:1240] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 17. Current term is 18. Ops: []
W20260709 13:19:13.647149 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:13.653476 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:13.556074 10916 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:13.654671 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:13.657680 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:13.658838 10944 raft_consensus.cc:2410] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:19:13.662643 10800 raft_consensus.cc:1275] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:19:13.658615 10842 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:13.673635 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:13.674649 10796 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 22.
W20260709 13:19:13.675423 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 119467us
I20260709 13:19:13.676194 10916 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:13.677644 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:13.642516 18204 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:13.679482 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 136056us
I20260709 13:19:13.682992 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:13.683632 10871 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:13.686024 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:13.687727 10916 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:13.688584 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:13.684386 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:13.691491 10941 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:13.648093 18234 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:13.618251 18149 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:13.664677 18256 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:19:13.713798 11506 consensus_meta.cc:287] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.526s	user 0.007s	sys 0.003s
I20260709 13:19:13.722054 11506 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 21.
I20260709 13:19:13.724362 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:13.725179 10869 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 21.
I20260709 13:19:13.715327 18248 raft_consensus.cc:3055] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:13.647594 18295 consensus_queue.cc:237] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:13.681113 18113 raft_consensus.cc:2749] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:19:13.680763 18148 raft_consensus.cc:2804] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:13.727466 10916 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:13.720652 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:19:13.735858 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 53110.1us
I20260709 13:19:13.736328 10794 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 20.
I20260709 13:19:13.694919 17996 raft_consensus.cc:2749] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:19:13.694449 18122 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:19:13.739303 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 84230.5us
I20260709 13:19:13.736801 11478 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:19:13.743602 10872 raft_consensus.cc:1240] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 18. Current term is 20. Ops: []
I20260709 13:19:13.742111 10941 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 23.
W20260709 13:19:13.719077 18298 log.cc:927] Time spent T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.346s	user 0.000s	sys 0.003s
I20260709 13:19:13.745978 11478 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:19:13.742821 10766 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:13.747157 10916 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:13.747822 11478 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:19:13.709010 17729 raft_consensus.cc:2749] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:19:13.744673 11506 raft_consensus.cc:1240] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 13:19:13.716264 17834 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:13.752422 10766 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:13.752887 11478 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:13.755048 18149 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:13.730973 18248 raft_consensus.cc:740] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:13.808794 18248 consensus_queue.cc:260] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:13.744618 18155 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:13.814672 18155 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:13.817487 18155 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:13.785558 18120 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:13.829428 18120 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:13.829865 18120 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:13.831074 10795 raft_consensus.cc:1275] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:19:13.836392 10874 raft_consensus.cc:1275] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
W20260709 13:19:13.837126 18170 consensus_meta.cc:287] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.522s	user 0.006s	sys 0.002s
I20260709 13:19:13.837572 18170 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:13.760201 10916 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:13.748301 18148 raft_consensus.cc:697] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:13.783826 18262 consensus_queue.cc:1048] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.003s
I20260709 13:19:13.788296 18235 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:13.788396 18213 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:13.785157 10841 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:19:13.721150 18234 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:13.788820 11505 raft_consensus.cc:1275] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 13:19:13.783895 18221 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:19:13.758742 18232 consensus_queue.cc:1048] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 13:19:13.789409 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:13.789829 10794 raft_consensus.cc:1240] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 13:19:13.790191 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:13.877995 11507 raft_consensus.cc:2410] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 25 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
W20260709 13:19:13.791986 18157 consensus_meta.cc:287] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.657s	user 0.009s	sys 0.000s
I20260709 13:19:13.794216 11506 raft_consensus.cc:1275] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:19:13.748387 18296 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:13.745214 18177 consensus_queue.cc:1048] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 13:19:13.729521 18256 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:13.758530 18164 raft_consensus.cc:2749] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:13.744843 18122 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:13.735368 18245 consensus_queue.cc:1059] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 13:19:13.786185 11478 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:13.759215 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 26 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:19:13.849498 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 89373.6us
I20260709 13:19:13.853179 17729 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:19:13.758873 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:19:13.855685 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 70625us
I20260709 13:19:13.867877 10871 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 18.
I20260709 13:19:13.759630 17777 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:19:13.905519 17777 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:13.901340 18122 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:13.905988 17777 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:19:13.789013 18175 raft_consensus.cc:2804] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:19:13.911188 18175 raft_consensus.cc:697] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 25 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:13.912197 18175 consensus_queue.cc:237] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:13.914208 18310 consensus_queue.cc:1048] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:19:13.915138 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 26 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:13.915723 18170 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:13.872021 18235 raft_consensus.cc:2804] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:13.874217 18234 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:13.855331 18213 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:13.784090 18113 raft_consensus.cc:2749] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:13.879791 18262 raft_consensus.cc:2749] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:19:13.881065 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 128740us
I20260709 13:19:13.900045 10948 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 21.
W20260709 13:19:13.902897 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 116802us
I20260709 13:19:13.905014 18044 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:19:13.916646 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:13.916149 10842 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:13.923801 10949 raft_consensus.cc:1275] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:19:13.878520 18221 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:13.918665 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:13.959841 10949 raft_consensus.cc:1275] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 13:19:13.926604 10943 raft_consensus.cc:1275] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:19:13.926400 17965 consensus_queue.cc:1059] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:19:13.923110 18235 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:13.961910 10841 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:13.927255 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:13.927409 10947 raft_consensus.cc:1275] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 28 index: 14. (index mismatch)
I20260709 13:19:13.959132 18221 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:13.927942 18233 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:14.004534 18233 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:13.928982 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:14.006457 10798 raft_consensus.cc:2393] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 21: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:13.930948 10942 raft_consensus.cc:1275] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:19:13.931118 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:13.939997 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:13.930758 17777 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:14.010138 17777 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 26 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:13.847777 18148 consensus_queue.cc:237] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:13.895262 18296 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:13.942760 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 410226us
I20260709 13:19:13.946940 10842 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:19:14.012077 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 65232.1us
I20260709 13:19:13.926687 18234 consensus_queue.cc:1048] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:13.918213 11477 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:13.901796 18287 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:13.905534 18245 raft_consensus.cc:3055] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:19:13.963390 18212 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:13.959540 17834 consensus_queue.cc:1048] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.007s
I20260709 13:19:13.964239 18235 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:13.967116 17965 raft_consensus.cc:3055] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:19:13.967649 18142 consensus_queue.cc:1048] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.004s
I20260709 13:19:13.970757 18149 raft_consensus.cc:2749] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:13.933456 17861 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:13.976197 18120 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:14.087668 18204 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:14.088297 18204 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:14.088461 18120 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:14.088785 18204 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:13.981422 18122 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:13.961391 18155 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:14.089560 18155 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:14.040056 18245 raft_consensus.cc:740] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:13.943028 18044 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:14.059154 18287 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:14.010972 11509 raft_consensus.cc:1240] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 25. Current term is 26. Ops: []
W20260709 13:19:14.015168 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 97067.1us
I20260709 13:19:14.024537 17777 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 13:19:14.020658 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:13.969991 18213 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:13.943207 18300 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:13.931406 18207 consensus_queue.cc:1048] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.083s
I20260709 13:19:14.019358 18296 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:14.098838 10915 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:19:14.107717 10839 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.025s	sys 0.025s
W20260709 13:19:14.111354 10839 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.026s	sys 0.025s
I20260709 13:19:14.111356 10915 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:19:14.112403 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:14.060240 18212 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:14.081502 17965 raft_consensus.cc:740] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:19:14.099582 18297 log.cc:927] Time spent T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.727s	user 0.000s	sys 0.002s
I20260709 13:19:14.091974 18044 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:14.117632 10915 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:14.130730 10873 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.014s	sys 0.017s
I20260709 13:19:14.091764 18155 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:14.090760 18122 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:14.139088 10873 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.014s	sys 0.018s
I20260709 13:19:14.153677 17965 consensus_queue.cc:260] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:14.097677 18300 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:14.108001 18231 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:14.094853 17777 raft_consensus.cc:697] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 23 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:14.117929 18113 raft_consensus.cc:2749] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:19:14.126163 18212 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:14.118180 18170 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:19:14.167220 18250 consensus_meta.cc:287] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.925s	user 0.012s	sys 0.002s
I20260709 13:19:14.094061 18287 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:14.294552 18204 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:14.309746 18155 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:14.179208 18229 raft_consensus.cc:2749] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:19:14.181768 18233 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:14.110482 17861 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:14.091671 18245 consensus_queue.cc:260] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:14.342365 18154 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.009s	sys 0.005s
W20260709 13:19:14.333093 10793 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.700s	user 0.013s	sys 0.000s
W20260709 13:19:14.353178 10793 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.720s	user 0.013s	sys 0.000s
W20260709 13:19:14.335738 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 218195us
W20260709 13:19:14.332720 10796 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.000s	sys 0.012s
W20260709 13:19:14.354890 10796 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.001s	sys 0.012s
W20260709 13:19:14.332656 18076 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.008s	sys 0.005s
W20260709 13:19:14.356122 10945 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.801s	user 0.002s	sys 0.011s
W20260709 13:19:14.356685 10945 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.810s	user 0.002s	sys 0.011s
W20260709 13:19:14.359843 18154 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.009s	sys 0.005s
I20260709 13:19:14.158597 18221 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:14.343365 17965 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:14.327385 18155 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:14.343214 18287 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:14.314924 18250 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:14.346272 18300 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:14.488430 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:14.456725 18256 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:14.368270 18076 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.008s	sys 0.005s
I20260709 13:19:14.346028 18238 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:14.346109 18232 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:19:14.496310 18175 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.012s	sys 0.035s
I20260709 13:19:14.346447 18233 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:14.403033 18295 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.010s	sys 0.008s
W20260709 13:19:14.543835 10871 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.004s	sys 0.045s
W20260709 13:19:14.544376 10871 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.004s	sys 0.045s
W20260709 13:19:14.388792 18260 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.010s	sys 0.038s
I20260709 13:19:14.457193 18245 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:19:14.411760 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 57220.6us
I20260709 13:19:14.535810 10801 raft_consensus.cc:1240] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 19. Current term is 20. Ops: []
W20260709 13:19:14.506098 18225 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.012s	sys 0.036s
I20260709 13:19:14.452550 18142 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:14.537222 11478 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.020s	sys 0.034s
W20260709 13:19:14.555804 11478 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.020s	sys 0.035s
I20260709 13:19:14.557546 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:19:14.502660 18174 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.811s	user 0.020s	sys 0.035s
I20260709 13:19:14.504421 18139 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:14.537128 18175 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.016s	sys 0.035s
W20260709 13:19:14.540174 10841 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.012s	sys 0.031s
I20260709 13:19:14.559182 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:14.559782 11512 raft_consensus.cc:1275] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
W20260709 13:19:14.543790 18295 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.010s	sys 0.015s
W20260709 13:19:14.552963 18225 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.012s	sys 0.039s
W20260709 13:19:14.562758 10841 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.013s	sys 0.031s
W20260709 13:19:14.560462 10868 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.519s	user 0.012s	sys 0.059s
W20260709 13:19:14.563860 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 602000us
I20260709 13:19:14.566138 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:14.573158 11512 raft_consensus.cc:2393] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 21: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
W20260709 13:19:14.552323 18260 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.732s	user 0.011s	sys 0.044s
I20260709 13:19:14.575264 10915 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:14.582679 10915 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:14.575819 11510 raft_consensus.cc:1240] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 24. Current term is 25. Ops: [24.13-24.13]
I20260709 13:19:14.575733 11478 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:14.577950 10842 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.023s	sys 0.034s
W20260709 13:19:14.587412 10842 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.023s	sys 0.034s
W20260709 13:19:14.587815 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 574551us
I20260709 13:19:14.539726 18238 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:14.507779 18287 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:14.590766 18148 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.023s	sys 0.023s
I20260709 13:19:14.620822 10797 raft_consensus.cc:1240] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:14.541904 18232 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:14.633447 10800 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.027s	sys 0.020s
W20260709 13:19:14.666353 10800 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.539s	user 0.030s	sys 0.023s
I20260709 13:19:14.667268 10800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 1027) took 1125 ms. Trace:
I20260709 13:19:14.591704 18231 raft_consensus.cc:2749] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:14.473939 17777 consensus_queue.cc:237] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:14.565887 18174 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.878s	user 0.021s	sys 0.036s
W20260709 13:19:14.620179 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:14.591197 18175 raft_consensus.cc:2749] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:19:14.667656 10800 rpcz_store.cc:276] 0709 13:19:13.542205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:13.716863 (+174658us) service_pool.cc:224] Handling call
0709 13:19:13.717165 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:13.717382 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:19:14.127272 (+409890us) log.cc:844] Serialized 14 byte log entry
0709 13:19:14.666891 (+539619us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:14.666959 (+    68us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:14.667011 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:14.667086 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:14.667206 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":20322,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":25,"thread_start_us":948697,"threads_started":2}
I20260709 13:19:14.691929 18315 consensus_queue.cc:1048] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:14.562645 18229 consensus_queue.cc:1059] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.089s
I20260709 13:19:14.692824 18155 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:14.693104 10870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2743) took 1054 ms. Trace:
I20260709 13:19:14.693532 10870 rpcz_store.cc:276] 0709 13:19:13.638580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:13.839778 (+201198us) service_pool.cc:224] Handling call
0709 13:19:13.840157 (+   379us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:13.840661 (+   504us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:13.840700 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:13.841286 (+   586us) log.cc:844] Serialized 26 byte log entry
0709 13:19:14.334467 (+493181us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:14.334525 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:14.334619 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:14.692885 (+358266us) raft_consensus.cc:1685] finished
0709 13:19:14.692906 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:14.693041 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9574,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":1594,"thread_start_us":492686,"threads_started":1,"wal-append.queue_time_us":751441}
I20260709 13:19:14.695392 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:14.696123 10870 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:19:14.567358 18139 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:14.650873 18225 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:19:14.658087 18148 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.024s	sys 0.023s
I20260709 13:19:14.581049 18113 consensus_queue.cc:1048] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.088s
I20260709 13:19:14.714525 10874 raft_consensus.cc:1240] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:14.716384 10874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 2624) took 1179 ms. Trace:
I20260709 13:19:14.716717 10874 rpcz_store.cc:276] 0709 13:19:13.536598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:13.837384 (+300786us) service_pool.cc:224] Handling call
0709 13:19:13.837716 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:14.715009 (+877293us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:14.716118 (+  1109us) spinlock_profiling.cc:243] Waited 877 ms on lock 0x7b540009abd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:19:14.716321 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2016030080}
I20260709 13:19:14.587132 18288 consensus_queue.cc:1048] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.094s
I20260709 13:19:14.676656 18204 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:14.564467 18142 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:14.580879 18122 consensus_queue.cc:1059] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.088s
I20260709 13:19:14.580552 18170 consensus_queue.cc:1059] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.094s
W20260709 13:19:14.706645 18318 log.cc:927] Time spent T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 13:19:14.707487 18250 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:14.709515 18248 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:14.717927 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:14.752324 10874 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:14.693130 18229 raft_consensus.cc:3055] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:19:14.744621 10872 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.731s	user 0.018s	sys 0.025s
W20260709 13:19:14.754189 10872 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.740s	user 0.019s	sys 0.025s
I20260709 13:19:14.750836 10870 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 26.
I20260709 13:19:14.723390 18122 raft_consensus.cc:3055] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:14.764051 10870 raft_consensus.cc:1240] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 17. Current term is 18. Ops: [16.13-17.14]
I20260709 13:19:14.726264 18170 consensus_queue.cc:1243] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer 12682eb70bc84c76879e04a8628ee586 log is divergent from this leader: its last log entry 23.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 13:19:14.722738 18142 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:14.777128 18322 log.cc:927] Time spent T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.113s	user 0.002s	sys 0.000s
W20260709 13:19:14.758085 18325 log.cc:927] Time spent T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:19:14.780431 10796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3296) took 1529 ms. Trace:
I20260709 13:19:14.726488 18155 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:14.781719 17861 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:14.782207 18155 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:14.784621 10801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2587) took 1870 ms. Trace:
I20260709 13:19:14.785135 18155 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:14.785029 10801 rpcz_store.cc:276] 0709 13:19:12.914727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:13.361544 (+446817us) service_pool.cc:224] Handling call
0709 13:19:13.361969 (+   425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:14.550659 (+1188690us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:14.784382 (+233723us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54003bd3d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:19:14.784562 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2699173120}
I20260709 13:19:14.787173 10793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2589) took 1871 ms. Trace:
I20260709 13:19:14.787744 10793 rpcz_store.cc:276] 0709 13:19:12.915418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:13.377494 (+462076us) service_pool.cc:224] Handling call
0709 13:19:13.377870 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:13.632878 (+255008us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:19:13.633214 (+   336us) log.cc:844] Serialized 14 byte log entry
0709 13:19:14.370626 (+737412us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:14.370708 (+    82us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:19:14.370800 (+    92us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:14.786846 (+416046us) spinlock_profiling.cc:243] Waited 255 ms on lock 0x7b5400387554. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:19:14.786909 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:14.787100 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":184,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":54,"raft.queue_time_us":16604,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":115,"spinlock_wait_cycles":585924736,"thread_start_us":720135,"threads_started":1,"wal-append.queue_time_us":929934}
W20260709 13:19:14.787381 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:14.796039 18311 log.cc:927] Time spent T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.710s	user 0.000s	sys 0.014s
I20260709 13:19:14.796164 18170 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:14.797041 18170 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:14.798990 18231 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:14.812332 10874 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 21.
I20260709 13:19:14.758239 18229 raft_consensus.cc:740] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:14.752955 18221 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:14.743628 18225 raft_consensus.cc:697] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 21 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:14.780975 10796 rpcz_store.cc:276] 0709 13:19:13.251151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:13.693378 (+442227us) service_pool.cc:224] Handling call
0709 13:19:13.702411 (+  9033us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:13.720902 (+ 18491us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:19:13.720940 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:13.721418 (+   478us) log.cc:844] Serialized 26 byte log entry
0709 13:19:14.355485 (+634067us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:14.355540 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:14.355607 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:14.780142 (+424535us) raft_consensus.cc:1685] finished
0709 13:19:14.780169 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:14.780352 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":102,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":7786,"thread_start_us":606082,"threads_started":1,"wal-append.queue_time_us":851590}
I20260709 13:19:14.766270 18122 raft_consensus.cc:740] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:14.702543 18260 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:14.862769 10798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39442 (request call id 1024) took 1653 ms. Trace:
I20260709 13:19:14.863242 10798 rpcz_store.cc:276] 0709 13:19:13.209311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:13.210910 (+  1599us) service_pool.cc:224] Handling call
0709 13:19:14.862564 (+1651654us) spinlock_profiling.cc:243] Waited 651 ms on lock 0x7b54000bc2d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:14.862707 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1498075008}
I20260709 13:19:14.865954 10797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2580) took 1997 ms. Trace:
I20260709 13:19:14.812287 18120 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:19:14.881400 10948 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41014 (request call id 2826) took 2474 ms (client timeout 2115 ms). Trace:
W20260709 13:19:14.881840 10948 rpcz_store.cc:269] 0709 13:19:12.390552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:12.390701 (+   149us) service_pool.cc:224] Handling call
0709 13:19:14.864358 (+2473657us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b5400173754. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:14.864455 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3442981120}
I20260709 13:19:14.866366 10797 rpcz_store.cc:276] 0709 13:19:12.868161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:13.348892 (+480731us) service_pool.cc:224] Handling call
0709 13:19:13.383303 (+ 34411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:14.658250 (+1274947us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:14.865708 (+207458us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b54003bddd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:19:14.865892 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2842191232}
I20260709 13:19:14.821220 17861 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:14.869951 18334 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
W20260709 13:19:14.871527 18329 log.cc:927] Time spent T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 13:19:14.872430 18321 log.cc:927] Time spent T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
W20260709 13:19:14.913455 18324 log.cc:927] Time spent T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.251s	user 0.000s	sys 0.001s
I20260709 13:19:14.916544 18295 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:14.917237 18250 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:14.926108 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:15.006681 10874 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:15.011348 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:14.926409 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:14.927577 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 344978us
W20260709 13:19:15.016151 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:14.928189 11478 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:19:14.932091 10868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 2608) took 1617 ms. Trace:
W20260709 13:19:15.017047 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 88975.4us
I20260709 13:19:14.934142 18113 raft_consensus.cc:3055] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Stepping down as leader of term 22
W20260709 13:19:14.934922 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 344609us
I20260709 13:19:14.940369 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:14.941768 11509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56404 (request call id 1220) took 1400 ms. Trace:
I20260709 13:19:14.942562 10793 raft_consensus.cc:1240] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:14.942773 10796 raft_consensus.cc:1275] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:19:14.944355 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:14.946086 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:14.948433 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:14.950642 18122 consensus_queue.cc:260] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:14.953984 10945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 1308) took 1746 ms. Trace:
I20260709 13:19:14.955348 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:15.035296 10795 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 21.
I20260709 13:19:14.956995 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:14.960361 17834 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:14.962282 18256 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:14.963793 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 342527us
I20260709 13:19:14.967150 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 2622) took 1467 ms. Trace:
I20260709 13:19:15.055737 10867 rpcz_store.cc:276] 0709 13:19:13.499652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:13.611892 (+112240us) service_pool.cc:224] Handling call
0709 13:19:13.612269 (+   377us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:13.612640 (+   371us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:13.612685 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:13.613706 (+  1021us) log.cc:844] Serialized 52 byte log entry
0709 13:19:14.064278 (+450572us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:19:14.567607 (+503329us) log.cc:844] Serialized 14 byte log entry
0709 13:19:14.571857 (+  4250us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:14.571939 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:14.783414 (+211475us) spinlock_profiling.cc:243] Waited 8.22 ms on lock 0x7b18003d2a00. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc83f3 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:19:14.966683 (+183269us) raft_consensus.cc:1685] finished
0709 13:19:14.966860 (+   177us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:14.967068 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26008,"prepare.queue_time_us":611141,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":104777,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":6729,"spinlock_wait_cycles":18910464,"thread_start_us":931140,"threads_started":2,"wal-append.queue_time_us":863699}
I20260709 13:19:14.969694 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:14.970737 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:15.059204 10948 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 22.
W20260709 13:19:14.976311 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:14.966383 17861 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:14.901719 18233 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:14.899875 18155 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:15.004659 10871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 1158) took 1065 ms. Trace:
I20260709 13:19:15.022326 10874 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 21.
W20260709 13:19:15.007268 18212 consensus_meta.cc:287] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.722s	user 0.012s	sys 0.000s
I20260709 13:19:14.864946 18260 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:15.037169 10800 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:15.035787 11509 rpcz_store.cc:276] 0709 13:19:13.540940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:13.541114 (+   174us) service_pool.cc:224] Handling call
0709 13:19:13.915480 (+374366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:14.092755 (+177275us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:14.940452 (+847697us) spinlock_profiling.cc:243] Waited 95.3 ms on lock 0x7b54001d1354. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:19:14.941557 (+  1105us) spinlock_profiling.cc:243] Waited 355 ms on lock 0x7b54001d1350. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:14.941706 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1035526144}
I20260709 13:19:14.907099 18120 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:14.975513 18334 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:15.043555 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 2619) took 1566 ms. Trace:
I20260709 13:19:14.879321 18229 consensus_queue.cc:260] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:15.051518 10799 raft_consensus.cc:2393] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 21: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:15.034830 10945 rpcz_store.cc:276] 0709 13:19:13.207073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:13.207188 (+   115us) service_pool.cc:224] Handling call
0709 13:19:13.207485 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:13.384537 (+177052us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:19:13.546470 (+161933us) log.cc:844] Serialized 14 byte log entry
0709 13:19:14.357366 (+810896us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:14.357447 (+    81us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:19:14.357513 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:14.905853 (+548340us) spinlock_profiling.cc:243] Waited 18.2 ms on lock 0x7b18004690c0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7dd962a6 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419
0709 13:19:14.953706 (+ 47853us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b540018e2d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:19:14.953756 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:14.953916 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14745,"prepare.queue_time_us":350984,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":55,"raft.queue_time_us":138,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":90479,"spinlock_wait_cycles":448468480,"thread_start_us":972011,"threads_started":2,"wal-append.queue_time_us":1041526}
I20260709 13:19:14.831843 18231 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:15.067123 10767 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:15.024362 18234 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:15.137194 10873 rpcz_store.cc:276] 0709 13:19:13.476716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:13.589891 (+113175us) service_pool.cc:224] Handling call
0709 13:19:13.590172 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:13.590356 (+   184us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:19:14.172300 (+581944us) log.cc:844] Serialized 14 byte log entry
0709 13:19:15.043158 (+870858us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:15.043215 (+    57us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:19:15.043277 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:15.043343 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:15.043476 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10480,"prepare.queue_time_us":939112,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":55918,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":81,"thread_start_us":1451597,"threads_started":3,"wal-append.queue_time_us":840083}
I20260709 13:19:15.016764 18170 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:15.041321 18310 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:15.041877 11478 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
W20260709 13:19:15.147387 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 105589us
I20260709 13:19:15.044423 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:15.031247 18076 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:19:15.148799 11478 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:19:15.058501 18175 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:15.171119 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:15.013813 18250 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:15.173166 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 117629us
I20260709 13:19:15.158097 10876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 1151) took 1630 ms. Trace:
I20260709 13:19:15.139447  9682 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
W20260709 13:19:15.161978 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:15.037467 18113 raft_consensus.cc:740] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:15.165719 18174 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:19:15.165931 18342 log.cc:927] Time spent T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
I20260709 13:19:15.166036 18287 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:15.166553 18352 consensus_queue.cc:1059] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 13:19:15.167205 18231 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:15.168258 18170 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:15.168555 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:15.168825 10947 raft_consensus.cc:3060] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:19:15.169521 18338 log.cc:927] Time spent T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.180s	user 0.000s	sys 0.000s
I20260709 13:19:15.008649 18295 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:15.138686 18321 log.cc:927] Time spent T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 13:19:15.063354 18318 log.cc:927] Time spent T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:19:15.174193 10876 rpcz_store.cc:276] 0709 13:19:13.527859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:13.690984 (+163125us) service_pool.cc:224] Handling call
0709 13:19:13.691537 (+   553us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:13.701559 (+ 10022us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:13.701596 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:13.702197 (+   601us) log.cc:844] Serialized 52 byte log entry
0709 13:19:14.016013 (+313816us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:19:14.016070 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:14.016161 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:14.912843 (+896682us) spinlock_profiling.cc:243] Waited 4.96 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:19:15.138580 (+225737us) raft_consensus.cc:1685] finished
0709 13:19:15.138607 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:15.158029 (+ 19422us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1135,"raft.queue_time_us":10615,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":4566,"spinlock_wait_cycles":11410304,"thread_start_us":292301,"threads_started":1,"wal-append.queue_time_us":839019}
I20260709 13:19:15.136183 18346 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:15.053300 18225 consensus_queue.cc:237] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:15.135701 18233 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:15.061484 18238 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:15.016995 10868 rpcz_store.cc:276] 0709 13:19:13.314890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:13.359805 (+ 44915us) service_pool.cc:224] Handling call
0709 13:19:13.386145 (+ 26340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:13.652604 (+266459us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:19:14.040723 (+388119us) log.cc:844] Serialized 14 byte log entry
0709 13:19:14.564175 (+523452us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:14.564252 (+    77us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:14.564295 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:14.915103 (+350808us) spinlock_profiling.cc:243] Waited 5.79 ms on lock 0x7b18003356e0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc8522 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:19:14.918433 (+  3330us) spinlock_profiling.cc:243] Waited 3.02 ms on lock 0x7b18003356e0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7dd962a6 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419
0709 13:19:14.931832 (+ 13399us) spinlock_profiling.cc:243] Waited 266 ms on lock 0x7b540006d1d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:19:14.931904 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:14.932032 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":77297,"prepare.queue_time_us":903655,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":337,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":10004,"spinlock_wait_cycles":632418432,"thread_start_us":910430,"threads_started":2,"wal-append.queue_time_us":816238}
I20260709 13:19:15.201596 10800 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 21.
I20260709 13:19:15.058820 18288 consensus_queue.cc:1059] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.112s
W20260709 13:19:15.221565 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 154512us
I20260709 13:19:15.232241 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:15.286677 10873 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:15.183917 18289 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:15.068881 18212 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:15.193403 18328 log.cc:927] Time spent T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.328s	user 0.000s	sys 0.001s
W20260709 13:19:15.286262 18344 log.cc:927] Time spent T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 13:19:15.203696 18204 consensus_queue.cc:1059] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 23.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.080s
I20260709 13:19:15.270481 10872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 1157) took 1331 ms. Trace:
I20260709 13:19:15.327997 10872 rpcz_store.cc:276] 0709 13:19:13.939190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:13.940613 (+  1423us) service_pool.cc:224] Handling call
0709 13:19:13.940993 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:13.941239 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:19:14.754647 (+813408us) log.cc:844] Serialized 14 byte log entry
0709 13:19:15.270000 (+515353us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:15.270077 (+    77us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:19:15.270140 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:15.270244 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:15.270404 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":173453,"prepare.queue_time_us":613292,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":187769,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":7805,"thread_start_us":1124445,"threads_started":2,"wal-append.queue_time_us":510297}
I20260709 13:19:15.329834 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:15.330950 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:15.060104 18232 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:15.064898 18256 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:15.229274 18333 log.cc:927] Time spent T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.463s	user 0.001s	sys 0.000s
I20260709 13:19:15.278260 18205 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:15.352972 18205 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:15.355628 17861 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:15.355824 18205 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:15.356602 17861 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:15.357323 17861 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:15.359656 18238 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:15.360225 18238 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:15.278918 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:15.278280 18346 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:15.223085 18229 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:15.227017 18139 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:15.230934 18261 raft_consensus.cc:2804] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:15.252506 18231 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:15.172695 18234 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:15.264254 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:15.188812 18175 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:15.319904 10947 raft_consensus.cc:1275] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:19:15.132918 18155 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:15.060846 17834 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:15.335172 10873 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 23.
I20260709 13:19:15.428545 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:15.429216 11510 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 22.
I20260709 13:19:15.429584 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:15.430182 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:15.430655 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:15.183347 18310 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:15.431362 11506 raft_consensus.cc:1275] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
W20260709 13:19:15.389091 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:15.287336 18207 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:15.338747 18289 raft_consensus.cc:697] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 21 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
W20260709 13:19:15.346113 18354 log.cc:927] Time spent T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:19:15.336902 18355 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:15.338538 18113 consensus_queue.cc:260] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:15.265040 18170 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:15.362362 18238 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:15.370829 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:15.371101 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:15.379724 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:15.481783 11512 raft_consensus.cc:2376] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 24. Re-sending same reply.
I20260709 13:19:15.482239 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:15.483121 10948 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:19:15.483210 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:15.382125 11478 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:15.337168 10872 raft_consensus.cc:2393] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 21: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:15.428593 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:15.387440 10867 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:15.343909 18363 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:15.373271 18361 log.cc:927] Time spent T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 13:19:15.379565 18174 raft_consensus.cc:697] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 21 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:15.373924 18287 consensus_queue.cc:1048] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.060s
I20260709 13:19:15.343678 18232 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:15.435988 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:19:15.427454 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 253515us
I20260709 13:19:15.222414 18076 raft_consensus.cc:697] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 26 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:15.440407 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:15.496563 10873 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 21.
I20260709 13:19:15.363413 18367 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:19:15.460608 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:15.476935 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:15.291301 18233 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:15.484447 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 102406us
I20260709 13:19:15.436225 18366 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:15.493486 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 57753.1us
I20260709 13:19:15.428845 18139 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:15.497468 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:15.498245 11510 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 20.
I20260709 13:19:15.501645 10945 raft_consensus.cc:1275] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 13:19:15.485646 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:15.503721 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:19:15.495101 18343 log.cc:927] Time spent T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.362s	user 0.000s	sys 0.001s
I20260709 13:19:15.516867 10945 pending_rounds.cc:85] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 12
I20260709 13:19:15.522240 10945 pending_rounds.cc:107] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 13:19:15.522518 10945 raft_consensus.cc:2889] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:15.435117 18261 raft_consensus.cc:697] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:15.519681 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:15.516031 10873 raft_consensus.cc:2368] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 19. Current term is 20.
I20260709 13:19:15.526360 10943 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:15.524693 10948 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 26.
I20260709 13:19:15.525179 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:15.529479 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:15.435318 18231 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:15.531451 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:15.536885 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:15.537638 10868 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 24.
I20260709 13:19:15.498194 17777 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:15.541432 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:15.542050 10873 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 20.
I20260709 13:19:15.543031 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:15.543565 10873 raft_consensus.cc:1813] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Handling vote request from an unknown peer cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:15.543977 10873 raft_consensus.cc:2368] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 20. Current term is 21.
I20260709 13:19:15.443900 18207 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:15.554647 10867 raft_consensus.cc:2410] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 25 because replica has last-logged OpId of term: 24 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 12.
I20260709 13:19:15.548151 10841 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:15.563526 10873 raft_consensus.cc:1275] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:19:15.515184 18370 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:15.573369 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:15.577175 10944 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 23.
I20260709 13:19:15.578881 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:15.576718 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 75942.6us
I20260709 13:19:15.579835 10944 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 22.
I20260709 13:19:15.536850 18120 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:15.538985 18155 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:15.473975 18355 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:15.572844 18241 consensus_queue.cc:1048] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:19:15.496580 18363 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:15.538550 18373 consensus_queue.cc:1059] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 13:19:15.520817 18366 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:15.525249 18139 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:15.551247 17777 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:15.582733 10946 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 19.
W20260709 13:19:15.591478 18235 consensus_meta.cc:287] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 1.487s	user 0.010s	sys 0.003s
I20260709 13:19:15.586323 18346 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:15.630594 18233 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:15.595011 18295 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:15.595103 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:15.595762 18076 consensus_queue.cc:237] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:15.597110 18044 consensus_meta.cc:287] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 1.417s	user 0.003s	sys 0.008s
I20260709 13:19:15.597230 18377 raft_consensus.cc:2749] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:19:15.552294 18250 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:15.600749 10876 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 17.
I20260709 13:19:15.500030 18232 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:15.605734 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:15.588622 18296 consensus_meta.cc:287] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 1.470s	user 0.006s	sys 0.005s
I20260709 13:19:15.663084 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:15.663460 18296 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:15.601715 18245 consensus_meta.cc:287] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 1.041s	user 0.006s	sys 0.007s
W20260709 13:19:15.587908 18359 log.cc:927] Time spent T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.267s	user 0.000s	sys 0.000s
W20260709 13:19:15.587560 18364 log.cc:927] Time spent T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.217s	user 0.001s	sys 0.000s
W20260709 13:19:15.589021 18347 log.cc:927] Time spent T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:19:15.064536 10871 rpcz_store.cc:276] 0709 13:19:13.939523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:14.009002 (+ 69479us) service_pool.cc:224] Handling call
0709 13:19:14.009368 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:14.009613 (+   245us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:19:14.547038 (+537425us) log.cc:844] Serialized 14 byte log entry
0709 13:19:15.004137 (+457099us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:15.004328 (+   191us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:15.004372 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:15.004447 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:15.004581 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":81033,"prepare.queue_time_us":578667,"prepare.run_cpu_time_us":3597,"prepare.run_wall_time_us":16260,"raft.queue_time_us":93258,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":31658,"thread_start_us":912596,"threads_started":2}
I20260709 13:19:15.584259 18310 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:15.623201 10868 raft_consensus.cc:1240] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 22. Current term is 23. Ops: [22.12-22.12]
I20260709 13:19:15.628955 17861 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:15.630290 18317 log.cc:927] Time spent T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 1.088s	user 0.008s	sys 0.002s
I20260709 13:19:15.514070 18367 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:15.631317 18261 consensus_queue.cc:237] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:15.632493 18256 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:15.589159 18207 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:15.580140 18334 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:15.585548 18175 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:15.594348 10943 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 19.
I20260709 13:19:15.649847 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:15.651773 10944 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 20.
I20260709 13:19:15.653508 10948 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 20.
I20260709 13:19:15.654393 10841 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:15.657531 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:15.552021 18289 consensus_queue.cc:237] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:15.662528 18374 consensus_queue.cc:1048] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:19:15.664063 10794 raft_consensus.cc:2410] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 21 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:19:15.583590 17834 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:15.667097 11503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56416 (request call id 1131) took 1107 ms. Trace:
I20260709 13:19:15.703876 10795 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 18.
W20260709 13:19:15.706046 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 51711.4us
I20260709 13:19:15.675009 10869 raft_consensus.cc:1240] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 LEADER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:15.582762 18174 consensus_queue.cc:237] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:15.716099 10871 raft_consensus.cc:1240] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 25. Current term is 26. Ops: [25.16-25.16]
I20260709 13:19:15.717720 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:15.712173 11503 rpcz_store.cc:276] 0709 13:19:14.559427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:14.559543 (+   116us) service_pool.cc:224] Handling call
0709 13:19:14.581673 (+ 22130us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:14.582153 (+   480us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:19:14.996943 (+414790us) log.cc:844] Serialized 14 byte log entry
0709 13:19:15.293777 (+296834us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:15.324673 (+ 30896us) log.cc:844] Serialized 26 byte log entry
0709 13:19:15.324910 (+   237us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:19:15.324962 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:15.325055 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:15.666866 (+341811us) raft_consensus.cc:1685] finished
0709 13:19:15.666891 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:15.667038 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55722,"prepare.queue_time_us":757267,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":135092,"raft.run_cpu_time_us":344,"raft.run_wall_time_us":57779,"thread_start_us":592916,"threads_started":3,"wal-append.queue_time_us":285691}
I20260709 13:19:15.719854 10841 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:15.640813 18212 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:15.727413 10841 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:15.666710 18296 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:15.653978 18235 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:15.731660 18235 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:15.724265 18371 log.cc:927] Time spent T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:19:15.733771 10873 raft_consensus.cc:1240] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:15.674091 18366 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:15.667270 18204 consensus_queue.cc:1048] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.068s
W20260709 13:19:15.735936 18380 log.cc:927] Time spent T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:19:15.736444 18207 consensus_queue.cc:1048] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:15.737342 18207 consensus_queue.cc:1243] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer cb0428cc55984daba1ba63744a5102a2 log is divergent from this leader: its last log entry 23.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:19:15.714645 18154 raft_consensus.cc:2749] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:15.675695 18139 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:15.741984 10875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 2620) took 2262 ms. Trace:
I20260709 13:19:15.735045 18355 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:15.742487 10875 rpcz_store.cc:276] 0709 13:19:13.479716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:13.595901 (+116185us) service_pool.cc:224] Handling call
0709 13:19:13.596368 (+   467us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:13.596858 (+   490us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:13.596898 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:13.597642 (+   744us) log.cc:844] Serialized 52 byte log entry
0709 13:19:14.063354 (+465712us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:14.573860 (+510506us) log.cc:844] Serialized 14 byte log entry
0709 13:19:14.923086 (+349226us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:14.924574 (+  1488us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:15.629661 (+705087us) spinlock_profiling.cc:243] Waited 3.55 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:19:15.741622 (+111961us) raft_consensus.cc:1685] finished
0709 13:19:15.741742 (+   120us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:15.741914 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":570969,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":43,"raft.queue_time_us":122748,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":168,"spinlock_wait_cycles":8155520,"thread_start_us":1297854,"threads_started":3,"wal-append.queue_time_us":861443}
I20260709 13:19:15.744036 10868 raft_consensus.cc:1275] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:19:15.739012 18363 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:15.718393 18260 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:15.751368 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:15.756192 10946 raft_consensus.cc:2393] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 24: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:15.756815 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:15.757270 10915 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:15.666975 18238 consensus_queue.cc:1048] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.079s
I20260709 13:19:15.661228 18044 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:15.746194 18212 raft_consensus.cc:2749] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:15.753814 18377 raft_consensus.cc:2749] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:15.774216 17861 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:19:15.792469 10916 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.008s	sys 0.040s
W20260709 13:19:15.793419 10916 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.008s	sys 0.040s
W20260709 13:19:15.794276 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:15.794858 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:15.795315 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:15.795881 11511 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 22: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:15.778385 18335 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:15.796233 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 91358.8us
I20260709 13:19:15.799862 18365 consensus_queue.cc:1059] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:19:15.760368 18211 consensus_queue.cc:1048] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 13:19:15.773468 18363 consensus_queue.cc:1059] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.022s
I20260709 13:19:15.808590 10916 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:15.810024 10916 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:15.779251 18310 consensus_queue.cc:1048] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.021s
I20260709 13:19:15.793267 17861 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:15.801043 18231 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:15.802942 18335 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:15.835698 18353 raft_consensus.cc:3055] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:19:15.836192 18353 raft_consensus.cc:740] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:15.837138 18353 consensus_queue.cc:260] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 23.10, Last appended by leader: 10, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:15.838186 18353 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:19:15.840338 18248 consensus_meta.cc:287] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 1.081s	user 0.011s	sys 0.000s
I20260709 13:19:15.840951 18205 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:15.841471 18205 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:15.813745 17861 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:15.816542 18365 raft_consensus.cc:3055] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:19:15.851547 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:15.831043 18231 raft_consensus.cc:2804] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:15.859388 18231 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:15.859885 18231 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:15.860203 18225 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:15.860760 18225 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:15.854238 18232 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:15.864317 18044 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:15.864892 18225 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:15.864812 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:15.865711 11504 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 23.
I20260709 13:19:15.865972 18170 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:15.867085 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:19:15.865688 10916 connection.cc:256] RPC call timeout handler was delayed by -1.0943477739999707s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:19:15.867755 11504 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 19.
I20260709 13:19:15.867112 10915 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:15.868667 18225 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:15.869237 18225 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:15.870172 10915 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:15.871953 18225 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:15.872574 18225 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:15.872926 18225 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:15.873258 18225 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:15.848217 18365 raft_consensus.cc:740] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:15.875038 10916 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:15.875689 18365 consensus_queue.cc:260] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:15.876374 18142 raft_consensus.cc:3055] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:15.876879 18142 raft_consensus.cc:740] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:15.877031 18365 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:19:15.877822 18142 consensus_queue.cc:260] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:15.854723 18395 consensus_queue.cc:1059] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:15.880167 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:15.884908 10795 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 22: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:15.817261 18376 consensus_queue.cc:1059] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.083s
I20260709 13:19:15.871974 18335 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:15.886132 18229 raft_consensus.cc:2804] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:15.914907 18310 raft_consensus.cc:2749] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:15.917577 18225 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:15.920313 18363 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:15.920910 18363 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:15.921264 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:15.921808 11504 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:15.924587 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 1154) took 2389 ms. Trace:
I20260709 13:19:15.924984 10869 rpcz_store.cc:276] 0709 13:19:13.535571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:13.736585 (+201014us) service_pool.cc:224] Handling call
0709 13:19:13.740662 (+  4077us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:15.712722 (+1972060us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:15.924346 (+211624us) spinlock_profiling.cc:243] Waited 1.93 s on lock 0x7b5400078854. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:19:15.924526 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4447896448}
I20260709 13:19:15.928275 18352 raft_consensus.cc:3055] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:15.928721 18352 raft_consensus.cc:740] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:15.929598 18352 consensus_queue.cc:260] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:15.893989 18142 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:15.907102 18229 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:15.935170 18229 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:19:15.903529 18392 log.cc:927] Time spent T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:19:15.941663 18367 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:15.958987 11504 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 24.
I20260709 13:19:15.960822 10915 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:15.956166  9682 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:19:15.966151 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 91185us
I20260709 13:19:15.960103 18352 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:15.973657 18363 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:15.973371 17861 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:15.979013 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:15.948885 18205 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:15.979699 11512 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:15.986624 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:15.987434 10795 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:15.998291 10801 raft_consensus.cc:1240] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:16.000833 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:16.001585 11504 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 18.
I20260709 13:19:16.001813 10801 raft_consensus.cc:1275] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
I20260709 13:19:15.948065 18225 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:16.014210 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:16.017457 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:16.017704 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:16.018473 10801 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
I20260709 13:19:16.019084 10794 raft_consensus.cc:1275] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:19:16.020915 18395 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:19:16.022920 10797 raft_consensus.cc:2393] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 24: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:16.026925 11509 raft_consensus.cc:1813] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 21 LEADER]: Handling vote request from an unknown peer cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:16.024904 10795 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
I20260709 13:19:16.003052 18367 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:16.000311 17861 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:16.039750 10795 raft_consensus.cc:1275] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:19:16.007035 18245 raft_consensus.cc:2749] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:16.007233 18232 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:16.006744 18379 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:16.046109 10794 raft_consensus.cc:1240] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 25. Current term is 26. Ops: []
I20260709 13:19:16.043084 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:16.047562 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:16.042513 11512 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 23.
I20260709 13:19:16.063781 10795 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 23.
I20260709 13:19:16.064098 10797 raft_consensus.cc:1275] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:19:16.064327 10799 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:16.095075 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:16.048173 18395 raft_consensus.cc:697] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 24 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:16.096092 10795 raft_consensus.cc:2410] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 21 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 13:19:16.068679 17861 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:16.065181 18174 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:16.099346 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:16.038825 18231 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:16.098611 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 54184.2us
I20260709 13:19:16.101378 10915 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:16.104879 10795 raft_consensus.cc:1275] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:19:16.050189 18367 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:16.107127 18120 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:16.109077 18120 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:16.102242 18385 consensus_queue.cc:1059] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.012s
I20260709 13:19:16.111943 18120 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:16.065662 18232 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:16.120024 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:16.120733 18376 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:16.121407 18376 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:16.119585 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:16.124218 10796 raft_consensus.cc:1813] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Handling vote request from an unknown peer cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:16.066658 18379 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:16.122630 18396 log.cc:927] Time spent T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 13:19:16.125176 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:16.126130 10796 raft_consensus.cc:2368] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 20. Current term is 21.
W20260709 13:19:16.128153 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 89563us
I20260709 13:19:16.158882 10799 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 23.
I20260709 13:19:16.159518 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:16.160293 10799 raft_consensus.cc:1240] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:16.104737 17861 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:16.108013 18174 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:16.167268 10799 raft_consensus.cc:1275] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:19:16.160126 18385 consensus_queue.cc:1243] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer 468a8fd46e824f62b9c2c0d3532a6d26 log is divergent from this leader: its last log entry 18.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:19:16.129516 18400 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:19:16.191335 10796 raft_consensus.cc:1275] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:19:16.207574 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:16.208832 10796 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:19:16.215932 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:16.227891 18404 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:16.228677 18174 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:16.229561 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:16.230334 10799 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 18.
I20260709 13:19:16.228724 18404 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:16.210990 18413 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:16.232733 18413 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:16.235345 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:16.236366 10800 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 22: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:16.236531 17861 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:16.212997 18076 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:16.238379 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:16.239092 18350 raft_consensus.cc:993] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26: : Instructing follower cb0428cc55984daba1ba63744a5102a2 to start an election
I20260709 13:19:16.239234 10795 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 23.
I20260709 13:19:16.239862 18232 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:16.240681 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:16.241668 10795 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 24.
I20260709 13:19:16.241796 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:16.243076 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:16.243700 10797 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 18.
I20260709 13:19:16.244550 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:19:16.244942 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:16.197875 18367 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:16.237411 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:16.234642 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:16.247694 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:16.238157 18076 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:16.199262 18231 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:16.256281 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:16.250109 10800 raft_consensus.cc:2410] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 21 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 13:19:16.247367 10794 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 21.
I20260709 13:19:16.258104 10795 raft_consensus.cc:2393] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 19: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:16.263568 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:16.265409 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:16.266961 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:16.280215 10794 raft_consensus.cc:1275] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:19:16.168677 18395 consensus_queue.cc:237] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:16.280194 11477 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:16.296628 17861 consensus_queue.cc:1048] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:19:16.300616 18406 log.cc:927] Time spent T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 13:19:16.331022 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:16.305646 18310 raft_consensus.cc:3055] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:16.309291 10766 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:16.310714 18232 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:16.311304 18261 consensus_queue.cc:1048] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:19:16.312155 18260 consensus_queue.cc:1059] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:19:16.321360 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:19:16.327517 18381 log.cc:927] Time spent T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
W20260709 13:19:16.330009 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 44437.3us
W20260709 13:19:16.333810 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 53710.7us
I20260709 13:19:16.333861 10867 raft_consensus.cc:1275] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:19:16.255671 18356 raft_consensus.cc:1081] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Signalling peer cb0428cc55984daba1ba63744a5102a2 to start an election
I20260709 13:19:16.234105 18404 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:16.297401 18379 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:16.348060 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 2626) took 1343 ms. Trace:
I20260709 13:19:16.361191 10867 rpcz_store.cc:276] 0709 13:19:15.004987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:15.557484 (+552497us) service_pool.cc:224] Handling call
0709 13:19:16.320929 (+763445us) spinlock_profiling.cc:243] Waited 19.7 ms on lock 0x7b5800177098. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd90a22 00007f9c86ca9c09 00007f9c85e03ee0 00007f9c85ecd0ab 00007f9c85ecd131 00007f9c85e8efc0 00007f9c85e79929 00007f9c85e793d2 00007f9c7f3f0bd4
0709 13:19:16.321207 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:16.347674 (+ 26467us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:16.347975 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":45373824}
I20260709 13:19:16.345312 10842 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:16.338455 11478 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:16.357012 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:16.359027 11504 raft_consensus.cc:1240] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 22. Current term is 23. Ops: [19.16-22.18]
I20260709 13:19:16.335945 18310 raft_consensus.cc:740] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:16.342002 18120 consensus_queue.cc:1059] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 13:19:16.367782 18310 consensus_queue.cc:260] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:16.368933 18310 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:16.369794 18229 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:16.371131 18044 consensus_queue.cc:1048] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:19:16.334748 18413 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:16.374071 10842 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:16.344108 17777 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:16.381529 10796 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 23.
I20260709 13:19:16.378962 11507 raft_consensus.cc:1240] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 23. Current term is 24. Ops: [22.16-23.17]
I20260709 13:19:16.333334 18241 consensus_queue.cc:1048] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 13:19:16.380151 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:16.384536 11512 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 24.
I20260709 13:19:16.340332 18232 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:16.376024 18142 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:16.376124 18379 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:16.393827 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:16.395170 11478 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:16.398662 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:16.379658 18350 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:16.401705 11478 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:16.396718 10842 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:16.405426 11512 raft_consensus.cc:2393] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 25: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:16.400460 11507 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 21.
I20260709 13:19:16.406179 18376 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:16.392012 18044 raft_consensus.cc:2804] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:16.379464 18315 consensus_queue.cc:1048] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.025s
I20260709 13:19:16.422279 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:16.420565 10796 raft_consensus.cc:1275] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:19:16.397718 18204 raft_consensus.cc:2804] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:16.399992 18142 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:16.405131 18379 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:16.415663 18356 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:19:16.419093 18044 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:16.396217 18205 consensus_queue.cc:1048] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:19:16.416074 18350 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:16.413230 18353 consensus_queue.cc:1048] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 13:19:16.427721 18288 raft_consensus.cc:2804] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:16.429539 18142 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:16.432217 17777 raft_consensus.cc:2749] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:16.466058 11509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56404 (request call id 1222) took 1068 ms. Trace:
I20260709 13:19:16.501457 11509 rpcz_store.cc:276] 0709 13:19:15.368818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:15.368964 (+   146us) service_pool.cc:224] Handling call
0709 13:19:16.437126 (+1068162us) spinlock_profiling.cc:243] Waited 596 ms on lock 0x7b54006dd154. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:16.437307 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1370919680}
I20260709 13:19:16.436810 18044 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:19:16.469281 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:16.479246 18363 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:16.488237 18315 consensus_queue.cc:1059] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:19:16.489224 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:16.489923 18407 log.cc:927] Time spent T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.293s	user 0.000s	sys 0.001s
I20260709 13:19:16.493497 18353 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:16.514472 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:16.468286 18174 raft_consensus.cc:2804] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:16.495971 18350 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:16.467216 18288 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:16.498629 18076 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:16.433197 18379 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:19:16.500001 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:16.433051 18356 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:16.502287 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 251578us
I20260709 13:19:16.462390 11505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56416 (request call id 1130) took 1903 ms. Trace:
I20260709 13:19:16.526136 18400 raft_consensus.cc:697] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 23 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:16.535953 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:16.537891 10916 leader_election.cc:400] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 20. Current term is 21.
I20260709 13:19:16.538300 10916 leader_election.cc:403] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:16.513863 18122 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:16.527216 18288 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:16.532531 11505 rpcz_store.cc:276] 0709 13:19:14.558999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:14.559108 (+   109us) service_pool.cc:224] Handling call
0709 13:19:14.571301 (+ 12193us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:14.571555 (+   254us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 13:19:14.992707 (+421152us) log.cc:844] Serialized 14 byte log entry
0709 13:19:16.452047 (+1459340us) spinlock_profiling.cc:243] Waited 3.62 ms on lock 0x7b18004cc960. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c80a5d4bc 00007f9c7ddab48b 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a5523a
0709 13:19:16.461694 (+  9647us) log.cc:844] Serialized 14 byte log entry
0709 13:19:16.461915 (+   221us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:16.461976 (+    61us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:19:16.462045 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:16.462141 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:16.462309 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25740,"prepare.queue_time_us":704104,"prepare.run_cpu_time_us":98,"prepare.run_wall_time_us":100,"raft.queue_time_us":34556,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":20040,"spinlock_wait_cycles":47507968,"thread_start_us":618304,"threads_started":2,"wal-append.queue_time_us":504071}
I20260709 13:19:16.534106 18356 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:16.570885 10916 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:16.574846 10916 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:16.514222 18363 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:16.523720 18353 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:16.557376 18229 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:16.559257 18385 consensus_queue.cc:1059] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 13:19:16.561663 18416 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:19:16.584568 10916 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:19:16.631819 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 322578us
W20260709 13:19:16.636518 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 52036us
I20260709 13:19:16.636564 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:16.636924 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:16.637611 10948 raft_consensus.cc:2393] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 20: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:16.638332 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:16.564790 18245 consensus_queue.cc:1048] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.049s
I20260709 13:19:16.641215 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:16.640360 18419 raft_consensus.cc:3055] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:19:16.638981 10950 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 17.
W20260709 13:19:16.602510 18352 consensus_meta.cc:287] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.622s	user 0.009s	sys 0.000s
I20260709 13:19:16.646404 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:16.647728 10950 raft_consensus.cc:1275] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
W20260709 13:19:16.648991 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:16.581812 18376 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:19:16.598400 18170 raft_consensus.cc:3055] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:16.640528 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:16.652464 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:16.653043 10943 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:19:16.654217 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 217350us
I20260709 13:19:16.642233 10945 raft_consensus.cc:2393] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 21: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:16.639249 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:16.655781 10942 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 19.
I20260709 13:19:16.642522 18419 raft_consensus.cc:740] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:16.592577 18234 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:16.613236 18353 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:16.613519 18413 raft_consensus.cc:2804] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:19:16.645421 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 242152us
I20260709 13:19:16.596400 18113 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:16.595049 18232 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:16.576711 18399 consensus_queue.cc:1048] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.063s
W20260709 13:19:16.603339 18414 log.cc:927] Time spent T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 13:19:16.668781 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:16.647996 18229 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:16.659150 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:16.672165 10842 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:16.672241 10792 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 18.
I20260709 13:19:16.660023 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:16.674114 10793 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 22: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:16.660758 10798 raft_consensus.cc:1240] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 19. Current term is 20. Ops: [19.10-19.10]
I20260709 13:19:16.642648 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:16.663941 18385 consensus_queue.cc:1048] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:19:16.686576 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:16.686960 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:16.669559 10914 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.006s	sys 0.015s
W20260709 13:19:16.689224 10914 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.007s	sys 0.015s
I20260709 13:19:16.650935 18376 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:16.650915 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:16.674960 10943 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 22.
I20260709 13:19:16.694213 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:16.694973 10875 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:16.695350 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:16.696980 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 29 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:16.677471 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:16.697795 10943 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 28.
I20260709 13:19:16.698071 10800 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:16.677996 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:16.678277 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:16.678925 10767 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:16.679121 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:16.680682 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:16.700897 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:16.680824 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:16.613849 18400 consensus_queue.cc:237] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:16.701934 10797 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 22.
I20260709 13:19:16.657359 18204 raft_consensus.cc:697] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:16.707428 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:16.707841 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:16.708676 10867 raft_consensus.cc:2410] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 25 because replica has last-logged OpId of term: 23 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 11.
I20260709 13:19:16.708729 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:16.681667 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:16.709681 10872 raft_consensus.cc:2393] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 24: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:16.683537 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:16.710489 10874 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 21.
I20260709 13:19:16.683905 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:16.685778 10796 raft_consensus.cc:1240] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:16.706606 10767 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:16.687834 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:16.681366 10766 connection.cc:256] RPC call timeout handler was delayed by -1.6745123850000709s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.197s for a response
I20260709 13:19:16.712754 10869 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 22.
W20260709 13:19:16.713450 10766 connection.cc:256] RPC call timeout handler was delayed by -1.5332123850000698s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2127s for a response
I20260709 13:19:16.702886 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:16.652626 18170 raft_consensus.cc:740] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:16.705268 10945 raft_consensus.cc:2393] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 21: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:16.706357 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:16.669469 10794 raft_consensus.cc:2376] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 23. Re-sending same reply.
I20260709 13:19:16.715289 10943 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 21.
I20260709 13:19:16.716482 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:16.716660 10876 raft_consensus.cc:1240] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 24. Current term is 25. Ops: []
I20260709 13:19:16.716724 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:16.717458 10944 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 18.
W20260709 13:19:16.717810 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 316148us
W20260709 13:19:16.719775 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 61467.3us
I20260709 13:19:16.721124 10916 leader_election.cc:400] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 20. Current term is 21.
I20260709 13:19:16.721124 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:16.723089 10800 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 24.
I20260709 13:19:16.723512 10950 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
I20260709 13:19:16.724373 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:16.724757 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:16.671241 18229 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:16.669200 18413 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:16.736320 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:16.725502 10873 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 22.
I20260709 13:19:16.737061 10950 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:16.737905 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:16.726922 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:16.734095 10875 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 22.
W20260709 13:19:16.717590 10766 connection.cc:256] RPC call timeout handler was delayed by -1.4461408460001621s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22870000000000001s for a response
W20260709 13:19:16.740224 10766 connection.cc:256] RPC call timeout handler was delayed by -1.315798889000007s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15280000000000002s for a response
I20260709 13:19:16.741005 10766 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:16.705385 18376 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:16.750329 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:16.745056 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:16.752044 10945 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:16.754611 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:16.757653 18419 consensus_queue.cc:260] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:16.758198 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:16.759063 18419 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:16.758836 18170 consensus_queue.cc:260] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:16.759382 18310 consensus_queue.cc:1175] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" responder_term: 23 status { last_received { term: 20 index: 11 } last_committed_idx: 11 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 22. Current term is 23. Ops: [22.12-22.12]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:19:16.760257 18204 consensus_queue.cc:237] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:16.720158 18363 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:16.764344 18352 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:19:16.764887 18352 raft_consensus.cc:697] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:16.765780 18352 consensus_queue.cc:237] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:16.744391 18413 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:16.767134 18241 raft_consensus.cc:3055] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:19:16.767555 18241 raft_consensus.cc:740] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:16.768388 18241 consensus_queue.cc:260] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:16.769397 18241 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:19:16.775530 10870 raft_consensus.cc:1275] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:19:16.763059 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:16.757889 18142 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:16.779361 10799 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
I20260709 13:19:16.781788 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:16.742630 18229 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:16.797211 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:16.796219 10950 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
I20260709 13:19:16.804579 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:16.805154 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:16.805894 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:16.806720 10867 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 23.
I20260709 13:19:16.808633 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:16.809455 10876 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 20.
I20260709 13:19:16.738544 18365 consensus_queue.cc:1048] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.062s
I20260709 13:19:16.814513 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:16.778380 18350 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:16.821274 10873 raft_consensus.cc:2393] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 21: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:16.817936 10950 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 23.
I20260709 13:19:16.822844 10945 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 24.
I20260709 13:19:16.823004 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:16.823992 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:16.783474 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:16.824575 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:16.827781 10950 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
W20260709 13:19:16.822077 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 64751.4us
I20260709 13:19:16.783108 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:16.843050 10875 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 19.
I20260709 13:19:16.860549 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:16.861598 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 29 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:16.865371 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:16.864423 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:16.804201 18310 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:16.868333 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:16.863749 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:16.901491 10842 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:16.871810 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:16.876008 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:16.904135 10842 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:16.904901 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:16.905423 10842 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:16.878295 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:16.881273 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:16.857547 18420 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:16.906726 18420 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:16.906984 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:16.895350 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:16.907629 10870 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 21.
I20260709 13:19:16.821792 18399 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:16.908635 10867 raft_consensus.cc:1275] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:19:16.908385 18399 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:16.905594 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:16.786582 18351 consensus_queue.cc:1048] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.112s
I20260709 13:19:16.897774 18288 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:16.900153 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:16.905689 10874 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 22: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:16.900293 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:16.822491 18365 consensus_queue.cc:1243] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer 468a8fd46e824f62b9c2c0d3532a6d26 log is divergent from this leader: its last log entry 19.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:19:16.907120 18420 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:16.894160 18376 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:16.902441 10869 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 22.
I20260709 13:19:16.802335 18353 consensus_queue.cc:1059] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.062s
I20260709 13:19:16.910477 18399 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:16.786090 18231 consensus_queue.cc:1059] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.164s
I20260709 13:19:16.910580 10945 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 22: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:16.913951 10868 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 22.
I20260709 13:19:16.928001 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:16.918666 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:16.936020 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:16.933681 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:16.916564 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:16.936990 10869 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 24.
I20260709 13:19:16.927191 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:16.920719 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:16.934806 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 29 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:16.923445 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:16.926173 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:16.937983 10947 raft_consensus.cc:2393] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 25: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:16.938766 10876 raft_consensus.cc:2376] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 23. Re-sending same reply.
I20260709 13:19:16.941859 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:16.930904 11478 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:16.946771 11505 raft_consensus.cc:2393] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 24: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:16.858563 18350 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:16.948153 11510 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 22: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:16.955989 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:16.957067 10945 raft_consensus.cc:2410] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 21 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:19:16.957057 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:16.957959 10874 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 25.
I20260709 13:19:16.920584 18420 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:16.959280 18424 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:16.973150 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:16.974104 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:16.929513 18260 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.047s	sys 0.095s
I20260709 13:19:16.912457 18288 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:16.992180 11477 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:16.972582 18424 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:17.000701 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:17.001825 10876 raft_consensus.cc:2393] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 20: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:17.004282 18424 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:16.920418 18044 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:17.005473 11507 raft_consensus.cc:1240] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:17.005527 18174 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:17.006146 18174 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:17.006506 18044 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:17.007194 18288 raft_consensus.cc:2749] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:19:17.011022 18205 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:19:17.011659 18205 raft_consensus.cc:697] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:17.012684 18205 consensus_queue.cc:237] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 22.11, Last appended by leader: 11, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:17.014325 10947 raft_consensus.cc:1240] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:17.019802 18044 raft_consensus.cc:2749] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:17.024595 11509 raft_consensus.cc:2410] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:19:17.024719 11504 raft_consensus.cc:3055] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:19:17.025252 11504 raft_consensus.cc:740] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:17.026074 11504 consensus_queue.cc:260] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:17.027067 18221 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.027397 11504 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:17.027788 18221 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:16.986481 18260 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.048s	sys 0.096s
I20260709 13:19:17.031600 18260 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:17.032163 18260 raft_consensus.cc:697] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:17.033159 18260 consensus_queue.cc:237] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:17.035768 18427 raft_consensus.cc:2804] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:17.036275 18427 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:17.036635 18427 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:17.035794 18221 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:17.038873 10945 raft_consensus.cc:1240] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 23. Current term is 24. Ops: []
I20260709 13:19:17.036434 18170 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:17.041096 11506 raft_consensus.cc:1275] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 14. (term mismatch)
I20260709 13:19:17.040755 18174 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:17.041513 11506 pending_rounds.cc:85] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586: Aborting all ops after (but not including) 13
I20260709 13:19:17.041807 11506 pending_rounds.cc:107] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586: Aborting uncommitted NO_OP operation due to leader change: 17.14
I20260709 13:19:17.042054 11506 raft_consensus.cc:2889] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:17.019517 18424 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:17.047811 10945 raft_consensus.cc:1240] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 24 LEADER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 23. Current term is 24. Ops: [23.17-23.17]
I20260709 13:19:17.050631 10947 raft_consensus.cc:1217] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Deduplicated request from leader. Original: 19.10->[21.11-28.14]   Dedup: 21.11->[24.12-28.14]
I20260709 13:19:17.053196 10947 pending_rounds.cc:85] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 11
I20260709 13:19:17.061158 10947 pending_rounds.cc:107] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 23.12
I20260709 13:19:17.061781 10947 raft_consensus.cc:2889] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:17.054611 10916 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:19:17.057976 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 322855us
I20260709 13:19:16.912640 18310 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:17.048799 18315 raft_consensus.cc:3055] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:19:17.069235 10916 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:17.071969 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:16.937727 18423 consensus_queue.cc:1048] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.118s
I20260709 13:19:17.078708 18310 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:17.079564 18404 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:17.080425 10941 raft_consensus.cc:1275] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 17. (term mismatch)
I20260709 13:19:17.081094 10941 pending_rounds.cc:85] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 16
I20260709 13:19:17.081408 10941 pending_rounds.cc:107] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 22.17
I20260709 13:19:17.081700 18376 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:17.081697 10941 raft_consensus.cc:2889] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:17.083356 18350 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:17.086019 10941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 1319) took 2075 ms. Trace:
I20260709 13:19:17.086292 10916 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:17.067763 18395 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.087170 18351 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:19:17.086621 10941 rpcz_store.cc:276] 0709 13:19:15.010409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:15.011340 (+   931us) service_pool.cc:224] Handling call
0709 13:19:15.055934 (+ 44594us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:17.082669 (+2026735us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:17.085724 (+  3055us) spinlock_profiling.cc:243] Waited 2.02 s on lock 0x7b54001e3554. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:19:17.085950 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4654667008}
I20260709 13:19:17.080489 18363 raft_consensus.cc:2804] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:17.097905 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:17.098728 10942 raft_consensus.cc:2410] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 23 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 13:19:17.098444 10841 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:19:17.098754 10874 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 22: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:17.111841 10874 raft_consensus.cc:1275] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 13:19:17.124320 10841 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:17.111748 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 370820us
I20260709 13:19:17.126013 10766 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:17.127753 10766 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:17.127944 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:17.070899 18315 raft_consensus.cc:740] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:17.088191 18234 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:17.146775 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:17.108327 18363 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:17.158439 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:17.140378 18399 raft_consensus.cc:3055] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:16.988097 18413 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:17.106806 18376 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:19:17.129288 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 422761us
I20260709 13:19:17.113970 18419 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:17.172008 10767 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:17.212390 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:17.114274 18351 raft_consensus.cc:697] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 23 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:17.160660 18363 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:17.176766 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:17.164794 10916 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:17.171777 10941 raft_consensus.cc:1240] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 20. Current term is 21. Ops: [20.10-20.10]
I20260709 13:19:17.173118 10869 raft_consensus.cc:1275] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:19:17.250567 18351 consensus_queue.cc:237] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:17.125568 18142 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:17.251745 18142 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.252224 18142 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:17.186022 10872 raft_consensus.cc:1240] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:17.254752 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:17.255707 10941 raft_consensus.cc:2410] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 23 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:19:17.087376 18395 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:17.248029 10841 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:17.279248 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 114546us
I20260709 13:19:17.280012 10841 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:17.206877 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:17.199987 10868 pending_rounds.cc:85] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 11
I20260709 13:19:17.162647 10874 raft_consensus.cc:1275] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:19:17.284066 10868 pending_rounds.cc:107] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 19.12
I20260709 13:19:17.284471 10868 raft_consensus.cc:2889] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:17.249408 10867 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Advancing to term 25
I20260709 13:19:17.270411 18315 consensus_queue.cc:260] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.18, Last appended by leader: 18, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:17.274976 18424 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:17.155639 18174 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:17.098068 18356 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:17.179699 18175 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 22 current term: 22 is higher.
W20260709 13:19:17.287230 10797 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.016s	sys 0.016s
I20260709 13:19:17.288112 10948 raft_consensus.cc:2393] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 19: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:17.162475 18350 raft_consensus.cc:2749] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:17.318799 18421 raft_consensus.cc:2749] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:17.319737 18421 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:19:17.320276 18421 raft_consensus.cc:697] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:17.320660 10950 raft_consensus.cc:1240] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:17.321302 18421 consensus_queue.cc:237] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:17.323140 18353 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:17.161129 18429 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:17.324102 18429 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:17.324532 18232 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:17.324592 18429 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:17.325172 18232 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:17.325647 18232 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:17.326879 10916 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:17.327184 18315 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:17.327646 18170 consensus_queue.cc:1048] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:19:17.328233 18395 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:17.328517 10916 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:17.329326 18231 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:17.329696 18231 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:17.330018 18231 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:17.330235 10916 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:17.331271 18413 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:17.277484 18419 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:17.334134 18419 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:17.338721 10916 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:17.292055 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 318127us
I20260709 13:19:17.129823 18241 consensus_queue.cc:1059] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.147s
W20260709 13:19:17.299329 10797 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.522s	user 0.016s	sys 0.016s
I20260709 13:19:17.163079 18399 raft_consensus.cc:740] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:17.304690 10944 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d32f5e230e464e61b23a41962708d120"
dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:41020
I20260709 13:19:17.305697 10867 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 25.
I20260709 13:19:17.286091 17861 raft_consensus.cc:2749] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:17.160954 18234 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:17.314988 18356 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:17.323608 18353 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:17.283476 18416 consensus_meta.cc:287] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.643s	user 0.011s	sys 0.000s
I20260709 13:19:17.180343 18367 raft_consensus.cc:3055] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:19:17.350713 10916 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:17.349274 11478 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:17.228266 10697 catalog_manager.cc:5697] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:17.356648 10950 raft_consensus.cc:1275] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 26 index: 16. (term mismatch)
I20260709 13:19:17.335362 18424 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:17.256455 18376 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:17.346431 18385 raft_consensus.cc:2764] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:19:17.362383 10950 pending_rounds.cc:85] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 15
I20260709 13:19:17.374732 10950 pending_rounds.cc:107] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 25.16
I20260709 13:19:17.375244 10950 raft_consensus.cc:2889] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:17.346742 18350 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:17.352845 18353 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:19:17.366891 10875 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.015s	sys 0.040s
W20260709 13:19:17.353879 18261 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.015s	sys 0.044s
I20260709 13:19:17.381880 11478 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:17.353277 18395 raft_consensus.cc:2749] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:19:17.386327 10875 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.015s	sys 0.040s
I20260709 13:19:17.353470 18367 raft_consensus.cc:740] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:17.360391 18356 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.354643 18416 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 20. Current term is 21.
I20260709 13:19:17.394145 10945 raft_consensus.cc:1240] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 23 LEADER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 22. Current term is 23. Ops: [22.18-22.18]
I20260709 13:19:17.357678 18142 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:17.401214 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 495817us
W20260709 13:19:17.416664 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:17.362084 18310 raft_consensus.cc:2804] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:17.362352 18372 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:19:17.384923 18204 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.006s	sys 0.022s
I20260709 13:19:17.393628 17861 consensus_queue.cc:1048] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.019s
W20260709 13:19:17.402354 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 410258us
W20260709 13:19:17.411755 18379 consensus_meta.cc:287] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.858s	user 0.012s	sys 0.009s
I20260709 13:19:17.438162 11477 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:17.393149 18261 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.015s	sys 0.044s
I20260709 13:19:17.423751 10950 raft_consensus.cc:1275] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:19:17.425269 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:19:17.440958 11475 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.014s	sys 0.017s
I20260709 13:19:17.384913  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:19:17.418116 10764 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.028s	sys 0.032s
I20260709 13:19:17.399039 18433 raft_consensus.cc:3055] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:19:17.434473 10945 raft_consensus.cc:1240] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 25. Current term is 26. Ops: []
W20260709 13:19:17.409229 18229 consensus_meta.cc:287] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.587s	user 0.007s	sys 0.003s
I20260709 13:19:17.439014 11504 raft_consensus.cc:1275] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:19:17.389302 10916 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:17.369669 18420 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:17.443634 10764 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.029s	sys 0.033s
I20260709 13:19:17.422233 18310 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:17.446220 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:17.430830 10697 catalog_manager.cc:5697] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 9 to 17, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 17 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
W20260709 13:19:17.453445 11475 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.014s	sys 0.017s
I20260709 13:19:17.460026 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:17.394742 18288 consensus_queue.cc:1048] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.043s
W20260709 13:19:17.464190 10793 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.686s	user 0.006s	sys 0.015s
I20260709 13:19:17.406174 18174 raft_consensus.cc:2749] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:17.385426 18424 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:17.485277 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 29 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:17.410486 17777 consensus_queue.cc:1048] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.052s
I20260709 13:19:17.471004 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:17.471414 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:17.467868 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:17.479054 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:19:17.480937 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 91766.2us
I20260709 13:19:17.410267 18300 consensus_queue.cc:1048] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.052s
W20260709 13:19:17.483777 10793 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.706s	user 0.007s	sys 0.015s
I20260709 13:19:17.467531 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:17.490480 11506 raft_consensus.cc:1275] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:19:17.458957 18433 raft_consensus.cc:740] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:17.414520 18356 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:17.531770 10795 raft_consensus.cc:2393] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 24: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:17.533808 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:17.534526 10801 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:19:17.526341 18205 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.004s	sys 0.010s
I20260709 13:19:17.466346 18229 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:17.537040 18076 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.015s	sys 0.017s
W20260709 13:19:17.480517 18397 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.015s	sys 0.017s
I20260709 13:19:17.478061 18372 raft_consensus.cc:2749] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:17.481618 18310 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:17.535880 11477 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:19:17.540323 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:17.540799 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:17.543514 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:17.487452 18385 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:17.546730 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:17.399410 18399 consensus_queue.cc:260] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:17.459438 18379 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:17.581969 10946 raft_consensus.cc:2393] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 22: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:17.527443 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:19:17.565018 10943 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.002s	sys 0.014s
I20260709 13:19:17.565152 10796 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 23.
W20260709 13:19:17.584286 10943 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.003s	sys 0.014s
I20260709 13:19:17.585338 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:17.586151 10796 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:17.582700 10798 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 21.
W20260709 13:19:17.562093 18397 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.015s	sys 0.020s
I20260709 13:19:17.571012 11509 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 24.
I20260709 13:19:17.576032 11477 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:17.558211 18205 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.005s	sys 0.010s
W20260709 13:19:17.559193 18076 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.015s	sys 0.017s
I20260709 13:19:17.524914 10916 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:17.611900 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 87034.8us
I20260709 13:19:17.614199 10916 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:17.583849 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 456198us
I20260709 13:19:17.480075 18367 consensus_queue.cc:260] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:17.604921 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:19:17.606769 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:17.638360 10766 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:17.609972 18372 raft_consensus.cc:2749] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:17.581223 18174 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:19:17.634335 10792 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.504s	user 0.004s	sys 0.016s
I20260709 13:19:17.608734 10801 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 25.
I20260709 13:19:17.642270 10916 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:19:17.644393 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:19:17.647099 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 475170us
I20260709 13:19:17.656309 18395 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:19:17.656900 18395 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:17.658250 10916 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:17.661823 18435 consensus_queue.cc:1048] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:17.657127 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:17.665333 10767 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:17.666333 18438 raft_consensus.cc:2804] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:17.666481 10767 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:17.666903 18438 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:17.667436 18438 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:17.668645 10946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41014 (request call id 2864) took 1022 ms. Trace:
I20260709 13:19:17.669070 10946 rpcz_store.cc:276] 0709 13:19:16.645966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:16.646108 (+   142us) service_pool.cc:224] Handling call
0709 13:19:17.668402 (+1022294us) spinlock_profiling.cc:243] Waited 934 ms on lock 0x7b54001e1c54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:17.668573 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2148737280}
I20260709 13:19:17.670357 10949 raft_consensus.cc:1240] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:17.674105 18350 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.674665 18350 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:17.676088 18438 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:17.677170 18350 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:17.615597 18385 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.652530 10766 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:17.686432 10767 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:17.630838 18400 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:19:17.713693 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 73622.5us
W20260709 13:19:17.616144 18413 consensus_queue.cc:1175] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "12682eb70bc84c76879e04a8628ee586" responder_term: 19 status { last_received { term: 19 index: 14 } last_committed_idx: 14 last_received_current_leader { term: 19 index: 14 } }
I20260709 13:19:17.649096 17861 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:19:17.617388 18419 consensus_queue.cc:1048] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.105s
I20260709 13:19:17.588330 18417 consensus_queue.cc:1048] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.064s
I20260709 13:19:17.713819 18434 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:17.657397 18395 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:19:17.732031 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 45608.4us
I20260709 13:19:17.758348 10792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3322) took 1081 ms. Trace:
I20260709 13:19:17.678594 18438 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:17.740736 11506 raft_consensus.cc:1240] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:17.741525 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:17.710839 18170 raft_consensus.cc:2749] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:17.717402 18400 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:17.762393 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:17.762560 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:17.763367 10949 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:17.761348 10767 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:17.758957 10792 rpcz_store.cc:276] 0709 13:19:16.676631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:16.682342 (+  5711us) service_pool.cc:224] Handling call
0709 13:19:16.682765 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:16.683094 (+   329us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:19:17.130079 (+446985us) log.cc:844] Serialized 14 byte log entry
0709 13:19:17.648179 (+518100us) log.cc:844] Serialized 14 byte log entry
0709 13:19:17.746223 (+ 98044us) spinlock_profiling.cc:243] Waited 32.5 ms on lock 0x7b18003adc80. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7dd42c0a 00007f9c87482219 00007f9c7ddcaa40 00007f9c7ddc54e4 00007f9c7ddc5455 00007f9c848ad3c7 00007f9c80ebf9a7 00007f9c80ecf0bf 00007f9c80ecf038
0709 13:19:17.746538 (+   315us) log.cc:844] Serialized 14 byte log entry
0709 13:19:17.746773 (+   235us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:17.750473 (+  3700us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:19:17.750552 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:17.750652 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:17.758258 (+  7606us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18427,"prepare.queue_time_us":795858,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":55,"raft.queue_time_us":2767,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":70677,"spinlock_wait_cycles":200678912,"thread_start_us":938880,"threads_started":2}
I20260709 13:19:17.768078 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:17.769057 10792 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 21.
W20260709 13:19:17.754180 10873 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.018s	sys 0.020s
I20260709 13:19:17.634843 18449 consensus_queue.cc:1048] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.129s
W20260709 13:19:17.760672 10871 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.024s	sys 0.046s
W20260709 13:19:17.729471 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 77015.8us
W20260709 13:19:17.774408 10871 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.024s	sys 0.047s
I20260709 13:19:17.775359 10766 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:19:17.771070 10873 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.657s	user 0.019s	sys 0.020s
I20260709 13:19:17.739822 18367 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:17.748596 17861 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:17.738612 18376 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:17.790849 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:17.592500 18433 consensus_queue.cc:260] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:17.770213 18260 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:17.683720 18450 consensus_queue.cc:1048] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.118s
I20260709 13:19:17.747793 18441 raft_consensus.cc:2764] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election decision vote started in defunct term 19: won
I20260709 13:19:17.784523 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:17.777420 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:17.786091 10766 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:17.792193 17861 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:17.757850 18434 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:17.773870 18400 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:19:17.777841 18351 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.012s	sys 0.021s
I20260709 13:19:17.748287 18372 consensus_queue.cc:1048] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 13:19:17.810722 10792 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:17.743052 18234 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:17.810137 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:17.730535 10796 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 25.
I20260709 13:19:17.796284 18420 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:17.783154 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:17.820158 10874 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 21.
I20260709 13:19:17.803699 18376 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:19:17.825546 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:19:17.811745 18442 log.cc:927] Time spent T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:19:17.833822 10869 raft_consensus.cc:2368] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 24. Current term is 25.
I20260709 13:19:17.803280 18424 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:17.812021 18437 log.cc:927] Time spent T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.246s	user 0.001s	sys 0.000s
I20260709 13:19:17.723018 18385 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:17.805224 18260 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
W20260709 13:19:17.812652 18445 log.cc:927] Time spent T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.167s	user 0.000s	sys 0.000s
W20260709 13:19:17.825302 18351 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.013s	sys 0.022s
I20260709 13:19:17.822368 10842 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:17.810509 18462 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:17.864147 18462 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:17.864631 18462 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:17.823877 18434 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:17.823503 18441 raft_consensus.cc:2749] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:17.892050 10797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3328) took 1199 ms. Trace:
I20260709 13:19:17.892522 10797 rpcz_store.cc:276] 0709 13:19:16.692502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:16.775396 (+ 82894us) service_pool.cc:224] Handling call
0709 13:19:16.775710 (+   314us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:16.776038 (+   328us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:16.776063 (+    25us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:16.776594 (+   531us) log.cc:844] Serialized 52 byte log entry
0709 13:19:17.344787 (+568193us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:17.344828 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:17.344895 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:17.891769 (+546874us) raft_consensus.cc:1685] finished
0709 13:19:17.891794 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:17.891984 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":540896,"raft.run_cpu_time_us":62,"raft.run_wall_time_us":61,"thread_start_us":567788,"threads_started":1,"wal-append.queue_time_us":746118}
I20260709 13:19:17.893525 18241 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:17.893931 18241 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:17.894302 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:17.894491 18241 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:17.893724 18385 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:17.849959 10949 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 23.
W20260709 13:19:17.833410 18231 consensus_meta.cc:287] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.503s	user 0.012s	sys 0.000s
W20260709 13:19:17.894878 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 72598.9us
I20260709 13:19:17.899962 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:17.926851 10797 raft_consensus.cc:2410] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 24 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
I20260709 13:19:17.810571 18356 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:17.812232 18465 raft_consensus.cc:2749] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:19:17.926836 11477 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:17.935194 10800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3327) took 1253 ms. Trace:
I20260709 13:19:17.951831 10800 rpcz_store.cc:276] 0709 13:19:16.682194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:16.752011 (+ 69817us) service_pool.cc:224] Handling call
0709 13:19:16.753088 (+  1077us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:17.112773 (+359685us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:17.112827 (+    54us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:17.113650 (+   823us) log.cc:844] Serialized 52 byte log entry
0709 13:19:17.617570 (+503920us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:19:17.617782 (+   212us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:17.618079 (+   297us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:17.934938 (+316859us) raft_consensus.cc:1685] finished
0709 13:19:17.934963 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:17.935120 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":528267,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":29677,"thread_start_us":844328,"threads_started":2,"wal-append.queue_time_us":801416}
I20260709 13:19:17.961302 10842 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:17.967232 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:17.968525 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:17.971095 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:17.971954 10796 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 21.
I20260709 13:19:17.838821 18229 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:17.848852 18379 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:17.942206 18363 consensus_meta.cc:287] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.692s	user 0.012s	sys 0.000s
I20260709 13:19:17.943250 18120 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:17.935194 10799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3331) took 1182 ms. Trace:
I20260709 13:19:17.846940 18260 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:17.937443 18462 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:17.898640 18399 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:17.900856 18433 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:17.940871 10767 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:19:18.029026 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 88239.9us
I20260709 13:19:17.849285 18376 raft_consensus.cc:697] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 25 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:17.997913 18463 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:19:17.962541 18205 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:18.049847 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:18.032245 10792 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 19.
I20260709 13:19:18.005084 10799 rpcz_store.cc:276] 0709 13:19:16.752725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:16.791345 (+ 38620us) service_pool.cc:224] Handling call
0709 13:19:16.816050 (+ 24705us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:17.108622 (+292572us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:17.108654 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:17.109149 (+   495us) log.cc:844] Serialized 26 byte log entry
0709 13:19:17.651011 (+541862us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:17.651063 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:17.651141 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:17.934962 (+283821us) raft_consensus.cc:1685] finished
0709 13:19:17.934982 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:17.935120 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9958,"raft.queue_time_us":500961,"raft.run_cpu_time_us":62,"raft.run_wall_time_us":62,"thread_start_us":770839,"threads_started":2,"wal-append.queue_time_us":808036}
I20260709 13:19:18.000859 10842 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:17.967232 18449 raft_consensus.cc:2804] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 13:19:17.992547 18427 consensus_meta.cc:287] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.955s	user 0.019s	sys 0.000s
I20260709 13:19:18.024178 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:18.046905 10767 leader_election.cc:400] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 election: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 24. Current term is 25.
I20260709 13:19:18.062592 10767 leader_election.cc:403] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 election: Cancelling election due to peer responding with higher term
I20260709 13:19:18.050990 10798 raft_consensus.cc:1240] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:18.052832 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:18.063560 10870 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 18.
W20260709 13:19:18.001386 18426 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.015s	sys 0.006s
I20260709 13:19:18.057106 10944 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:18.067003 10944 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:18.057266 10943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 3253) took 1353 ms. Trace:
W20260709 13:19:18.057284 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 56516.5us
I20260709 13:19:17.949345 18231 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:18.060885 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:18.066000 10797 raft_consensus.cc:1240] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 22. Current term is 23. Ops: [22.15-22.15]
W20260709 13:19:18.082659 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 52752.2us
I20260709 13:19:18.083736 10767 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:19:18.083550 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 502654us
I20260709 13:19:18.094643 10869 pending_rounds.cc:85] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 15
I20260709 13:19:18.095057 10869 pending_rounds.cc:107] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 22.16
I20260709 13:19:18.095332 10869 raft_consensus.cc:2889] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:18.072288 10943 rpcz_store.cc:276] 0709 13:19:16.703650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:17.016898 (+313248us) service_pool.cc:224] Handling call
0709 13:19:17.048467 (+ 31569us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:17.048688 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:19:17.584662 (+535974us) log.cc:844] Serialized 14 byte log entry
0709 13:19:18.056860 (+472198us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:18.056927 (+    67us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:18.056979 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:18.057069 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:18.057198 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3941,"prepare.queue_time_us":755703,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":2973,"raft.queue_time_us":42101,"thread_start_us":971706,"threads_started":2}
I20260709 13:19:18.051093 18379 raft_consensus.cc:2804] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:18.101030 10915 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:19:18.066725 18426 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.015s	sys 0.006s
I20260709 13:19:18.066000 18356 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:18.106585 11478 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:18.019910 18363 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:18.100080 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 359628us
I20260709 13:19:18.108479 10916 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:18.005277 18120 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:18.069029 18463 raft_consensus.cc:697] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:19:18.080194 18261 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.003s	sys 0.024s
I20260709 13:19:18.116513 10944 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:18.080267 18449 raft_consensus.cc:697] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 23 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:18.093868 18465 raft_consensus.cc:2749] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Leader election lost for term 24. Reason: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 24. Current term is 25.
I20260709 13:19:18.129992 11478 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:18.132212 11478 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:18.146946 10841 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.866s	user 0.021s	sys 0.034s
W20260709 13:19:18.148468 10841 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.867s	user 0.021s	sys 0.034s
W20260709 13:19:18.118717 18310 consensus_meta.cc:287] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.537s	user 0.009s	sys 0.001s
I20260709 13:19:18.152140 10948 raft_consensus.cc:1240] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:18.107668 18441 raft_consensus.cc:2749] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:18.065361 18205 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:18.167641 10868 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.882s	user 0.028s	sys 0.027s
I20260709 13:19:18.121264 18372 raft_consensus.cc:2804] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:19:18.157600 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:18.124342 18429 consensus_meta.cc:287] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.771s	user 0.012s	sys 0.000s
I20260709 13:19:18.167821 18310 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:18.111091 18356 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:18.121577 18261 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.004s	sys 0.025s
I20260709 13:19:18.080351 18427 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:18.119531 18379 raft_consensus.cc:697] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:18.153707 18450 raft_consensus.cc:2749] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:19:18.125910 18426 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:18.162990 18175 raft_consensus.cc:2804] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:19:18.180296 18353 consensus_meta.cc:287] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.773s	user 0.009s	sys 0.004s
W20260709 13:19:18.178633 18232 consensus_meta.cc:287] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.853s	user 0.015s	sys 0.000s
I20260709 13:19:18.172712 18174 raft_consensus.cc:2749] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:18.182565 18372 raft_consensus.cc:697] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 19 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:18.185575 18356 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:18.220712 18175 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:18.243777 18464 log.cc:927] Time spent T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
I20260709 13:19:18.254138 18175 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:18.158505 18376 consensus_queue.cc:237] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:18.183095 18429 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:18.283085 18466 log.cc:927] Time spent T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.233s	user 0.001s	sys 0.000s
W20260709 13:19:18.284031 18471 log.cc:927] Time spent T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 13:19:18.284893 18470 log.cc:927] Time spent T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 13:19:18.264796 18467 log.cc:927] Time spent T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.220s	user 0.001s	sys 0.000s
W20260709 13:19:18.277897 18395 consensus_meta.cc:287] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.502s	user 0.010s	sys 0.001s
I20260709 13:19:18.224743 18426 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:18.232172 18353 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:18.262925 18315 consensus_meta.cc:287] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.935s	user 0.009s	sys 0.005s
I20260709 13:19:18.290524 18462 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:18.237879 18463 consensus_queue.cc:237] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:18.244472 18449 consensus_queue.cc:237] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:18.357661 18476 log.cc:927] Time spent T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
W20260709 13:19:18.359365 18493 log.cc:927] Time spent T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:19:18.309366 18395 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:18.364012 18427 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:18.372829 18426 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:18.150401 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 870546us
W20260709 13:19:18.166891 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:18.168056 10868 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.883s	user 0.028s	sys 0.027s
I20260709 13:19:18.472268 18424 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:18.469717 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:18.496009 10874 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:18.495610 18424 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:18.456544 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:18.499593 18424 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:18.470690 10876 raft_consensus.cc:2393] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 22: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:18.438552 10794 raft_consensus.cc:1275] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:19:18.471868 10945 raft_consensus.cc:1240] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 25. Current term is 26. Ops: []
I20260709 13:19:18.461031 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:18.516822 18424 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:18.518005 18424 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:18.521317 18424 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:18.521559 18353 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:18.523506 11508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56416 (request call id 1142) took 1594 ms. Trace:
I20260709 13:19:18.523998 11508 rpcz_store.cc:276] 0709 13:19:16.928725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:16.928888 (+   163us) service_pool.cc:224] Handling call
0709 13:19:16.935950 (+  7062us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:17.438497 (+502547us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:17.438529 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:17.478751 (+ 40222us) log.cc:844] Serialized 26 byte log entry
0709 13:19:17.917115 (+438364us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:17.917174 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:17.917350 (+   176us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:18.522936 (+605586us) raft_consensus.cc:1685] finished
0709 13:19:18.522973 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:18.523280 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2076,"raft.queue_time_us":66852,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":101,"thread_start_us":885783,"threads_started":2,"wal-append.queue_time_us":604637}
I20260709 13:19:18.528894 18120 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:18.432468 10766 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.006s	sys 0.008s
I20260709 13:19:18.471040 10941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 3265) took 1697 ms. Trace:
I20260709 13:19:18.463095 10871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 2667) took 1267 ms. Trace:
W20260709 13:19:18.445134 10801 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.006s	sys 0.020s
W20260709 13:19:18.550261 10766 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.738s	user 0.007s	sys 0.008s
I20260709 13:19:18.542256 10941 rpcz_store.cc:276] 0709 13:19:16.773231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:17.270237 (+497006us) service_pool.cc:224] Handling call
0709 13:19:17.270659 (+   422us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:17.311472 (+ 40813us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:17.311510 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:17.312173 (+   663us) log.cc:844] Serialized 52 byte log entry
0709 13:19:17.684721 (+372548us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:17.684785 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:17.684890 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:18.470253 (+785363us) raft_consensus.cc:1685] finished
0709 13:19:18.470308 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:18.470705 (+   397us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34320,"raft.queue_time_us":35690,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":92,"thread_start_us":372070,"threads_started":1,"wal-append.queue_time_us":636197}
W20260709 13:19:18.388348 18434 consensus_meta.cc:287] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.505s	user 0.012s	sys 0.000s
I20260709 13:19:18.581755 18434 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:18.585163 18363 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:18.544456 10871 rpcz_store.cc:276] 0709 13:19:17.195266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:17.196562 (+  1296us) service_pool.cc:224] Handling call
0709 13:19:17.197110 (+   548us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:17.774966 (+577856us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:17.775017 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:17.790167 (+ 15150us) log.cc:844] Serialized 52 byte log entry
0709 13:19:18.156347 (+366180us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:18.173125 (+ 16778us) log.cc:844] Serialized 14 byte log entry
0709 13:19:18.462178 (+289053us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:18.462381 (+   203us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:18.462500 (+   119us) raft_consensus.cc:1685] finished
0709 13:19:18.462518 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:18.462830 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":340215,"prepare.queue_time_us":21807,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":449541,"raft.run_cpu_time_us":476,"raft.run_wall_time_us":7922,"thread_start_us":866203,"threads_started":2,"wal-append.queue_time_us":569803}
I20260709 13:19:18.459079 10841 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:18.575037 10801 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.006s	sys 0.021s
W20260709 13:19:18.385699 18468 log.cc:927] Time spent T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.267s	user 0.000s	sys 0.005s
I20260709 13:19:18.320405 18310 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:18.595832 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:18.597970 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:19:18.384439 18475 log.cc:927] Time spent T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 13:19:18.611191 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 1170) took 1521 ms. Trace:
I20260709 13:19:18.607712 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:18.235728 18232 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:18.618211 10794 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:18.620292 17777 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:18.620748 18232 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:18.621217 17777 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:18.623133 18205 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:18.517392 18372 consensus_queue.cc:237] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:18.624909 17777 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:18.626034 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 167550us
I20260709 13:19:18.627573 18509 raft_consensus.cc:2749] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:18.628515 10841 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:19:18.629494 18449 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:18.631100 18449 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:18.631893 10801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2633) took 1159 ms. Trace:
I20260709 13:19:18.632366 10801 rpcz_store.cc:276] 0709 13:19:17.472454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:17.669868 (+197414us) service_pool.cc:224] Handling call
0709 13:19:17.670283 (+   415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:17.670664 (+   381us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:17.670703 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:17.878548 (+207845us) spinlock_profiling.cc:243] Waited 18.9 ms on lock 0x7b2c000014c8. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7f7aea11 00007f9c7f7be372 00007f9c80ef0a01 00007f9c80eb543a 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2
0709 13:19:17.879131 (+   583us) log.cc:844] Serialized 26 byte log entry
0709 13:19:18.631145 (+752014us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:18.631206 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:18.631449 (+   243us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:18.631523 (+    74us) raft_consensus.cc:1685] finished
0709 13:19:18.631540 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:18.631811 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":167154,"raft.queue_time_us":56648,"raft.run_cpu_time_us":273,"raft.run_wall_time_us":141468,"spinlock_wait_cycles":43547520,"thread_start_us":712182,"threads_started":1,"wal-append.queue_time_us":488032}
I20260709 13:19:18.633986 18449 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:18.637801 18463 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:18.638343 18463 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:18.638811 18463 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:18.643314 18435 raft_consensus.cc:2804] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:18.644193 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:18.638540 18175 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:18.645743 18435 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:18.647346 18435 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:18.648355 10793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3330) took 1932 ms. Trace:
I20260709 13:19:18.646314 10872 raft_consensus.cc:1240] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:18.648712 10798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3338) took 1111 ms. Trace:
I20260709 13:19:18.656333 10798 rpcz_store.cc:276] 0709 13:19:17.537283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:17.995432 (+458149us) service_pool.cc:224] Handling call
0709 13:19:18.050801 (+ 55369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:18.066277 (+ 15476us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:18.648348 (+582071us) spinlock_profiling.cc:243] Waited 54.9 ms on lock 0x7b540026b950. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:18.648545 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":126344064}
I20260709 13:19:18.657742 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:18.648676 10799 raft_consensus.cc:2393] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 20: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:18.611912 10873 rpcz_store.cc:276] 0709 13:19:17.089728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:17.090053 (+   325us) service_pool.cc:224] Handling call
0709 13:19:17.090457 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:17.090703 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 13:19:17.776619 (+685916us) log.cc:844] Serialized 14 byte log entry
0709 13:19:18.609864 (+833245us) log.cc:844] Serialized 14 byte log entry
0709 13:19:18.610244 (+   380us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:18.610359 (+   115us) raft_consensus.cc:1645] Marking committed up to 18
0709 13:19:18.610482 (+   123us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:18.610631 (+   149us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:18.610933 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":84017,"prepare.queue_time_us":540330,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":85179,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":21162,"thread_start_us":1390334,"threads_started":3,"wal-append.queue_time_us":478875}
I20260709 13:19:18.658291 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:18.650815 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:18.648730 10793 rpcz_store.cc:276] 0709 13:19:16.717123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:16.777623 (+ 60500us) service_pool.cc:224] Handling call
0709 13:19:16.777930 (+   307us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:17.523884 (+745954us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:19:18.003877 (+479993us) log.cc:844] Serialized 14 byte log entry
0709 13:19:18.438969 (+435092us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:18.440220 (+  1251us) log.cc:844] Serialized 26 byte log entry
0709 13:19:18.440465 (+   245us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:18.440513 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:18.440660 (+   147us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:18.647927 (+207267us) raft_consensus.cc:1685] finished
0709 13:19:18.647951 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:18.648145 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":39213,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":4263,"raft.queue_time_us":643650,"raft.run_cpu_time_us":643,"raft.run_wall_time_us":35231,"spinlock_wait_cycles":152832,"thread_start_us":1619849,"threads_started":3,"wal-append.queue_time_us":254097}
I20260709 13:19:18.381695 18379 consensus_queue.cc:237] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:18.653589 10797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2634) took 1180 ms. Trace:
I20260709 13:19:18.664608 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:18.669240 10798 raft_consensus.cc:2393] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 21: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
W20260709 13:19:18.647851 18241 consensus_meta.cc:287] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.752s	user 0.009s	sys 0.004s
I20260709 13:19:18.677021 10874 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 23.
I20260709 13:19:18.662715 10944 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:18.673010 10797 rpcz_store.cc:276] 0709 13:19:17.472746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:17.969823 (+497077us) service_pool.cc:224] Handling call
0709 13:19:17.970227 (+   404us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:18.086221 (+115994us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:18.652896 (+566675us) spinlock_profiling.cc:243] Waited 95.4 ms on lock 0x7b5400386154. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:19:18.653358 (+   462us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":219497344}
I20260709 13:19:18.669591 18205 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:18.682524 18205 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:18.395709 18429 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:18.689395 10870 raft_consensus.cc:1240] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 22. Current term is 23. Ops: [20.14-22.15]
I20260709 13:19:18.690373 18483 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:18.692159 18483 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:18.695765 18469 log.cc:927] Time spent T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.575s	user 0.000s	sys 0.003s
W20260709 13:19:18.695614 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 52530.8us
I20260709 13:19:18.704651 10869 raft_consensus.cc:1275] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 22 index: 18. Preceding OpId from leader: term: 23 index: 18. (term mismatch)
I20260709 13:19:18.705057 10869 pending_rounds.cc:85] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 17
I20260709 13:19:18.705308 10869 pending_rounds.cc:107] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 22.18
I20260709 13:19:18.705570 10869 raft_consensus.cc:2889] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:19:18.695638 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 909605us
I20260709 13:19:18.709895 10873 raft_consensus.cc:2393] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 21: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:18.703299 10792 raft_consensus.cc:1275] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:19:18.665652 18142 consensus_queue.cc:1048] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 13:19:18.711179 10944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:41020 (request call id 3269) took 1914 ms. Trace:
I20260709 13:19:18.703521 18434 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:18.720428 10944 rpcz_store.cc:276] 0709 13:19:16.774186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:17.304547 (+530361us) service_pool.cc:224] Handling call
0709 13:19:18.660843 (+1356296us) spinlock_profiling.cc:243] Waited 713 ms on lock 0x7b54001f2f54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eaca3d 00007f9c85e7ccd0 00007f9c7f401254 00007f9c7f4011e2 00007f9c7f401111 00007f9c7f40108c 00007f9c7f400342 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:18.688223 (+ 27380us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1638850944}
I20260709 13:19:18.724664 10942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 1331) took 1825 ms. Trace:
I20260709 13:19:18.725445 18192 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:18.730219 10942 rpcz_store.cc:276] 0709 13:19:16.899572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:17.354370 (+454798us) service_pool.cc:224] Handling call
0709 13:19:17.354818 (+   448us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:17.355266 (+   448us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:17.355306 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:17.357965 (+  2659us) log.cc:844] Serialized 26 byte log entry
0709 13:19:17.792622 (+434657us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:19:17.792662 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:17.792727 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:18.694524 (+901797us) spinlock_profiling.cc:243] Waited 26.1 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:19:18.698442 (+  3918us) raft_consensus.cc:1685] finished
0709 13:19:18.698462 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:18.724465 (+ 26003us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":104,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":1732,"spinlock_wait_cycles":60034816,"thread_start_us":356410,"threads_started":1,"wal-append.queue_time_us":649927}
I20260709 13:19:18.739589 18443 consensus_queue.cc:1059] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:19:18.740162 10766 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:18.742082 18076 raft_consensus.cc:3055] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:18.742547 18076 raft_consensus.cc:740] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:18.743851 18076 consensus_queue.cc:260] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:18.744511 18433 consensus_meta.cc:287] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.694s	user 0.013s	sys 0.004s
I20260709 13:19:18.745100 18076 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:19:18.747073 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 116678us
I20260709 13:19:18.747498 10872 raft_consensus.cc:1275] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 17. (term mismatch)
I20260709 13:19:18.748029 10872 pending_rounds.cc:85] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 16
I20260709 13:19:18.748368 10872 pending_rounds.cc:107] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 23.17
I20260709 13:19:18.748685 10872 raft_consensus.cc:2889] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:18.687044 18241 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:18.761127 18192 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:18.765317 18367 consensus_meta.cc:287] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.975s	user 0.012s	sys 0.007s
I20260709 13:19:18.766278 18192 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:18.768806 10875 raft_consensus.cc:1240] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 13:19:18.769989 18175 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:18.786500 17861 consensus_meta.cc:287] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.951s	user 0.010s	sys 0.007s
W20260709 13:19:18.789470 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 688322us
I20260709 13:19:18.806646 17861 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:18.807428 10841 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:18.808522 18509 raft_consensus.cc:2749] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:19:18.797920 10874 raft_consensus.cc:1240] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 13:19:18.809582 17861 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:18.787572 18350 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.011s	sys 0.006s
W20260709 13:19:18.810849 18350 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.012s	sys 0.008s
I20260709 13:19:18.811291 18232 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:18.791962 18356 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:18.812064 18232 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:18.812722 10871 raft_consensus.cc:1240] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 21. Current term is 22. Ops: [21.15-21.15]
W20260709 13:19:18.813920 18400 consensus_meta.cc:287] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.976s	user 0.001s	sys 0.017s
I20260709 13:19:18.794448 18205 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:18.802318 18435 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:18.792995 18483 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:18.811825 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 156104us
I20260709 13:19:18.786657 10796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 1050) took 1017 ms. Trace:
I20260709 13:19:18.816268 10868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 2668) took 1493 ms. Trace:
I20260709 13:19:18.792244 10947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 3262) took 2019 ms. Trace:
W20260709 13:19:18.796300 10876 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49290 (request call id 1174) took 1638 ms (client timeout 1946 ms). Trace:
I20260709 13:19:18.835335 18413 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:18.835597 10876 rpcz_store.cc:269] 0709 13:19:17.157944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:17.158111 (+   167us) service_pool.cc:224] Handling call
0709 13:19:18.795984 (+1637873us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54001090d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:18.796214 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3013321600}
I20260709 13:19:18.836169 18413 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:18.801939 18231 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:18.839466 10794 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
I20260709 13:19:18.814513 18232 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:18.790292 18229 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:19:18.852335 18229 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:18.852835 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:19:18.853374 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:18.853633 11509 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 18.
I20260709 13:19:18.853869 18435 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:18.797068 18260 consensus_meta.cc:287] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.751s	user 0.007s	sys 0.009s
I20260709 13:19:18.853075 18356 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:18.790633 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 682102us
I20260709 13:19:18.839007 18440 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:18.814405 18400 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:18.863447 11508 raft_consensus.cc:1275] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 13:19:18.864377 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:18.809597 18351 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:18.803535 18395 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:18.849479 18434 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:19:18.867714 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 127757us
I20260709 13:19:18.825942 10796 rpcz_store.cc:276] 0709 13:19:17.768968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.442545 (+673577us) service_pool.cc:224] Handling call
0709 13:19:18.443146 (+   601us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:18.443734 (+   588us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 13:19:18.444323 (+   589us) log.cc:844] Serialized 14 byte log entry
0709 13:19:18.785951 (+341628us) spinlock_profiling.cc:243] Waited 3.17 ms on lock 0x7b180019ccc0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7dd42c0a 00007f9c87482219 00007f9c7ddcaa40 00007f9c7ddc54e4 00007f9c7ddc5455 00007f9c80ead060 00007f9c80ebf716 00007f9c80ed0287 00007f9c80ed0208
0709 13:19:18.786148 (+   197us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:18.786217 (+    69us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:19:18.786278 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:18.786356 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:18.786499 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28643,"prepare.queue_time_us":49728,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":1298,"raft.queue_time_us":219410,"raft.run_cpu_time_us":70,"raft.run_wall_time_us":71,"spinlock_wait_cycles":7300352,"thread_start_us":126268,"threads_started":1,"wal-append.queue_time_us":225535}
I20260709 13:19:18.852859 18229 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:18.831251 10947 rpcz_store.cc:276] 0709 13:19:16.772576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:17.050009 (+277433us) service_pool.cc:224] Handling call
0709 13:19:17.050401 (+   392us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:19:17.091107 (+ 40706us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:19:17.564853 (+473746us) log.cc:844] Serialized 14 byte log entry
0709 13:19:18.058142 (+493289us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:19:18.058867 (+   725us) log.cc:844] Serialized 78 byte log entry
0709 13:19:18.059009 (+   142us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:19:18.061847 (+  2838us) log.cc:844] Serialized 14 byte log entry
0709 13:19:18.087429 (+ 25582us) log.cc:844] Serialized 14 byte log entry
0709 13:19:18.284972 (+197543us) spinlock_profiling.cc:243] Waited 14.3 ms on lock 0x7b18002e6360. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc83f3 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:19:18.667863 (+382891us) spinlock_profiling.cc:243] Waited 7.9 ms on lock 0x7b18002e6360. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc83f3 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:19:18.788708 (+120845us) spinlock_profiling.cc:243] Waited 4.14 ms on lock 0x7b18002e6360. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7dd42c0a 00007f9c87482219 00007f9c84d8958f 00007f9c7ddc8571 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd
0709 13:19:18.791448 (+  2740us) log.cc:844] Serialized 14 byte log entry
0709 13:19:18.791720 (+   272us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:18.791838 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:18.791896 (+    58us) raft_consensus.cc:1685] finished
0709 13:19:18.791910 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:18.792084 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":874642,"prepare.queue_time_us":943753,"prepare.run_cpu_time_us":191,"prepare.run_wall_time_us":215,"raft.queue_time_us":763326,"raft.run_cpu_time_us":671,"raft.run_wall_time_us":28479,"spinlock_wait_cycles":60567424,"thread_start_us":909316,"threads_started":2,"wal-append.queue_time_us":596275}
I20260709 13:19:18.792816 18417 raft_consensus.cc:2749] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:19:18.838675 18413 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:18.867928 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:18.877673 10767 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:18.827337 10868 rpcz_store.cc:276] 0709 13:19:17.197954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:17.199109 (+  1155us) service_pool.cc:224] Handling call
0709 13:19:17.199663 (+   554us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:18.461923 (+1262260us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:18.461991 (+    68us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:18.463435 (+  1444us) log.cc:844] Serialized 52 byte log entry
0709 13:19:18.508959 (+ 45524us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:18.608687 (+ 99728us) log.cc:844] Serialized 14 byte log entry
0709 13:19:18.609045 (+   358us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:18.609207 (+   162us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:18.676306 (+ 67099us) raft_consensus.cc:1685] finished
0709 13:19:18.676336 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:18.691527 (+ 15191us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":141759,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":73066,"raft.run_cpu_time_us":308,"raft.run_wall_time_us":8662,"thread_start_us":1221200,"threads_started":2,"wal-append.queue_time_us":125945}
I20260709 13:19:18.889928 10795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2632) took 1403 ms. Trace:
I20260709 13:19:18.889576 18440 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:18.895046 18434 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:18.895092 18351 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:18.896266 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:18.896657 18400 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:18.897346 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:18.901306 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:18.905961 10941 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 18.
W20260709 13:19:18.939302 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 88894.4us
I20260709 13:19:18.941502 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:18.947654 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:18.951678 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 92394.2us
I20260709 13:19:18.954100 18395 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:18.955150 18367 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:19:18.955811 18367 raft_consensus.cc:697] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:18.956956 18367 consensus_queue.cc:237] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:18.957755 18076 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:18.958426 18076 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:18.961863 18351 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:18.962530 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 45370.7us
I20260709 13:19:18.963047 18433 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:18.963718 18433 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:18.965167 18232 consensus_queue.cc:1048] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:19:18.965906 10766 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:18.966238 18433 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:18.961889 10942 raft_consensus.cc:2393] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 19: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
W20260709 13:19:18.967072 18437 log.cc:927] Time spent T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 13:19:18.888549 18175 consensus_queue.cc:1059] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.069s
I20260709 13:19:18.958523 10916 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:18.968465 18175 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:18.969225 10916 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:18.969123 18175 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:18.971000 10871 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:18.971081 18400 raft_consensus.cc:2749] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:18.972255 18400 raft_consensus.cc:2749] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:18.973305 10842 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:18.969398 17777 raft_consensus.cc:3055] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:19:18.974985 17777 raft_consensus.cc:740] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:18.966938 18513 consensus_queue.cc:1059] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.036s
I20260709 13:19:18.962698 18441 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:18.976157 17777 consensus_queue.cc:260] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:18.978039 10949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 1339) took 1026 ms. Trace:
I20260709 13:19:18.978978 10916 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:18.978483 10949 rpcz_store.cc:276] 0709 13:19:17.951268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:17.951420 (+   152us) service_pool.cc:224] Handling call
0709 13:19:17.971307 (+ 19887us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:17.979905 (+  8598us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:19:18.005299 (+ 25394us) log.cc:844] Serialized 14 byte log entry
0709 13:19:18.521677 (+516378us) spinlock_profiling.cc:243] Waited 8.18 ms on lock 0x7b48002415d8. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c85ddeaa1 00007f9c85dd5b37 00007f9c85dd59ca 00007f9c85ecd26f 00007f9c85eccf37 00007f9c85ee155c 00007f9c85ee1508 00007f9c85ee1488 00007f9c85ee1443 00007f9c85ee06ec 00007f9c84e0aa96
0709 13:19:18.906062 (+384385us) log.cc:844] Serialized 14 byte log entry
0709 13:19:18.942809 (+ 36747us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:18.954264 (+ 11455us) log.cc:844] Serialized 26 byte log entry
0709 13:19:18.954428 (+   164us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:19:18.977103 (+ 22675us) log.cc:844] Serialized 14 byte log entry
0709 13:19:18.977355 (+   252us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:18.977566 (+   211us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:18.977608 (+    42us) raft_consensus.cc:1685] finished
0709 13:19:18.977627 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:18.977848 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":929181,"prepare.queue_time_us":71809,"prepare.run_cpu_time_us":124,"prepare.run_wall_time_us":128,"raft.queue_time_us":616004,"raft.run_cpu_time_us":7064,"raft.run_wall_time_us":543669,"spinlock_wait_cycles":18824064,"thread_start_us":292118,"threads_started":2,"wal-append.queue_time_us":313333}
I20260709 13:19:18.981863 18400 raft_consensus.cc:2749] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:19:18.978008 17777 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:19:18.984041 10766 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:18.976475 18076 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:18.985026 18463 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:18.899569 10795 rpcz_store.cc:276] 0709 13:19:17.472132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:17.532540 (+ 60408us) service_pool.cc:224] Handling call
0709 13:19:17.532894 (+   354us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:17.548628 (+ 15734us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:17.548673 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:18.003395 (+454722us) spinlock_profiling.cc:243] Waited 68.1 ms on lock 0x7b2c000014c8. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7f7aea11 00007f9c7f7be372 00007f9c80ef0a01 00007f9c80eb543a 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2
0709 13:19:18.016581 (+ 13186us) log.cc:844] Serialized 52 byte log entry
0709 13:19:18.501201 (+484620us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:19:18.501716 (+   515us) log.cc:844] Serialized 14 byte log entry
0709 13:19:18.700690 (+198974us) log.cc:844] Serialized 14 byte log entry
0709 13:19:18.818395 (+117705us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:18.830678 (+ 12283us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:18.858159 (+ 27481us) raft_consensus.cc:1685] finished
0709 13:19:18.858188 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:18.875283 (+ 17095us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":228687,"prepare.queue_time_us":18518,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":70,"raft.queue_time_us":276688,"raft.run_cpu_time_us":560,"raft.run_wall_time_us":39520,"spinlock_wait_cycles":156546304,"thread_start_us":499338,"threads_started":2,"wal-append.queue_time_us":414432}
I20260709 13:19:18.874889 18260 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:18.981134 10916 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:18.990983 10869 raft_consensus.cc:1240] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 23. Current term is 24. Ops: []
I20260709 13:19:18.991099 18260 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:18.992931 18463 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:18.987459 18175 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:18.986150 18395 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:18.995337 18413 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:18.996167 18395 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:18.996026 18413 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:18.996852 18395 raft_consensus.cc:2749] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:18.997040 10916 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:19:18.998440 18413 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:18.998973 18351 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:18.999464 18351 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:18.999886 18351 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:19.001368 10871 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 19.
I20260709 13:19:19.003252 18486 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:19.007089 18261 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:19.007845 18261 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:19.010066 18261 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:19.013922 18379 raft_consensus.cc:2804] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:19.014384 18379 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:18.998605 10916 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:19:19.014755 18379 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:19.014629 18260 consensus_queue.cc:1048] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:19.015437 18260 consensus_queue.cc:1243] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer cb0428cc55984daba1ba63744a5102a2 log is divergent from this leader: its last log entry 21.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:19:19.017323 10766 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:19:19.019599 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 46455.7us
W20260709 13:19:19.026804 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 59918.6us
I20260709 13:19:19.030222 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:19.041301 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:19.026701 18433 raft_consensus.cc:2804] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:19.031620 18395 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:19.047426 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:19.045035 10795 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 18.
I20260709 13:19:19.048503 18395 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:19.054291 18435 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:19.054888 18435 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:19.055342 18435 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:19.060322 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:19.054495 18395 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:19.061757 18350 consensus_queue.cc:1059] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:19:19.063097 18205 raft_consensus.cc:3055] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:19.063488 18205 raft_consensus.cc:740] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:19.064668 18205 consensus_queue.cc:260] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:19.066831 18204 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:19.067773 10948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 3270) took 2293 ms. Trace:
I20260709 13:19:19.068143 10948 rpcz_store.cc:276] 0709 13:19:16.774435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:17.316649 (+542214us) service_pool.cc:224] Handling call
0709 13:19:17.317058 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:18.159759 (+842701us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:19.067277 (+907518us) spinlock_profiling.cc:243] Waited 835 ms on lock 0x7b54003589d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:19:19.067572 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1919896832}
I20260709 13:19:19.067435 18204 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:19.066454 18434 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:19.070962 18426 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:19.071208 18204 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:19.071682 18426 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:19.074054 18426 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:19.075539 10875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 1183) took 1020 ms. Trace:
I20260709 13:19:19.075992 10875 rpcz_store.cc:276] 0709 13:19:18.055586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.086923 (+ 31337us) service_pool.cc:224] Handling call
0709 13:19:18.108450 (+ 21527us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:18.786842 (+678392us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:19.075247 (+288405us) spinlock_profiling.cc:243] Waited 660 ms on lock 0x7b540037b254. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:19:19.075474 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1517387136}
I20260709 13:19:19.076428 18120 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:19.077054 18120 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:19.079130 11509 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 20.
I20260709 13:19:19.080873 18433 raft_consensus.cc:697] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:19.078259 18351 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:19.099186 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:19.099433 18433 consensus_queue.cc:237] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:19.100248 10875 raft_consensus.cc:2393] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 23: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:19.083724 18120 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:19.105336 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:19.107530 10871 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 25.
I20260709 13:19:19.111028 18413 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:19.111817 18413 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:19.114169 18413 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:19.123524 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 125004us
W20260709 13:19:19.141089 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 90749us
I20260709 13:19:19.141283 18463 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:19.144254 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:19:19.145388 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 128228us
I20260709 13:19:19.147953 18422 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:19.148679 18422 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:19.149506 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:19.148880 18463 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:19.150619 18229 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:19.156665 18229 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:19.158836 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:19.159412 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:19.159735 10794 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:19.160545 10799 raft_consensus.cc:2393] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 25: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:19.161456 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:19.162089 18440 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:19.159495 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:19.162286 10799 raft_consensus.cc:2410] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 24 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 13:19:19.166761 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:19.167160 10793 raft_consensus.cc:1275] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:19:19.154597 18241 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:19.177263 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:19.180754 10767 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:19.183825 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:19.180202 10870 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 23.
I20260709 13:19:19.192694 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:19.193583 11506 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 22.
W20260709 13:19:19.194190 18399 consensus_meta.cc:287] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 1.058s	user 0.017s	sys 0.003s
I20260709 13:19:19.196918 18422 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:19.197168 10915 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:19.197595 18413 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:19.198252 18413 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:19.199153 18400 raft_consensus.cc:2804] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:19.199651 18400 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:19.200007 18400 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:19.194844 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:19.179627 18486 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:19:19.233811 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:19.245944 18231 consensus_queue.cc:1048] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:19:19.253471 18395 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:19.253954 18530 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:19:19.255489 18530 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:19.255925 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:19.274422 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:19:19.286006 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:19.286460 10794 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 22.
W20260709 13:19:19.267772 18523 log.cc:927] Time spent T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
W20260709 13:19:19.294762 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 127024us
I20260709 13:19:19.255980 18530 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:19:19.290776 11478 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.116.130:37895 timed out after 1.946s (SENT) [suppressed 23 similar messages]
I20260709 13:19:19.296952 10915 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:19.303251 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:19.305954 10869 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 24.
I20260709 13:19:19.307487 10871 raft_consensus.cc:3055] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:19.308473 10871 raft_consensus.cc:740] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:19.292757 18229 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:19.307318 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:19.309937 18351 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:19.330853 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:19.311228 11477 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:19:19.330065 18529 log.cc:927] Time spent T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 13:19:19.342900 10766 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:19:19.343858 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 86295.5us
I20260709 13:19:19.345060 10948 raft_consensus.cc:2393] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 20: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:19.345825 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:19.346501 10794 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 20.
I20260709 13:19:19.333905 10871 consensus_queue.cc:260] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:19.343043 11508 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 26.
W20260709 13:19:19.348994 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 57818.4us
I20260709 13:19:19.401172 10841 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:19.402997 18532 consensus_queue.cc:1059] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:19.403928 18532 raft_consensus.cc:2749] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:19.404670 18532 raft_consensus.cc:3055] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:19.405062 18532 raft_consensus.cc:740] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:19.405900 18532 consensus_queue.cc:260] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:19.331971 18229 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:19.409013 18229 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:19.346560 18413 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:19.349721 18422 raft_consensus.cc:2804] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:19.410153 18422 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:19.354952 10801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 1054) took 1584 ms. Trace:
I20260709 13:19:19.410997 10801 rpcz_store.cc:276] 0709 13:19:17.770736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.634519 (+863783us) service_pool.cc:224] Handling call
0709 13:19:18.664934 (+ 30415us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:18.815031 (+150097us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:18.815060 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:18.815784 (+   724us) log.cc:844] Serialized 52 byte log entry
0709 13:19:19.019157 (+203373us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:19:19.036521 (+ 17364us) log.cc:844] Serialized 14 byte log entry
0709 13:19:19.087722 (+ 51201us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:19.087822 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:19.354717 (+266895us) raft_consensus.cc:1685] finished
0709 13:19:19.354738 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:19.354895 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":185741,"prepare.queue_time_us":16907,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":55,"raft.queue_time_us":241237,"raft.run_cpu_time_us":424,"raft.run_wall_time_us":50084,"thread_start_us":251570,"threads_started":2,"wal-append.queue_time_us":186460}
W20260709 13:19:19.355118 18528 log.cc:927] Time spent T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.212s	user 0.001s	sys 0.000s
I20260709 13:19:19.362668 18463 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:19.354704 18533 log.cc:927] Time spent T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 13:19:19.363608 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:19.363694 18521 log.cc:927] Time spent T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.254s	user 0.001s	sys 0.001s
I20260709 13:19:19.364208 18419 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:19.417091 18419 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:19.417531 18419 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:19.367213 10942 raft_consensus.cc:1275] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 13:19:19.363476 18441 raft_consensus.cc:2749] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:19.386729 18429 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:19.410609 18422 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:19.421938 18429 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:19.428994 18429 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:19.433660 10841 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:19.435408 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:19.435842 18532 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:19.436399 10796 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 24.
I20260709 13:19:19.439669 18463 raft_consensus.cc:2749] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:19.441911 10767 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:19.442996 18441 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:19.443410 18441 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:19.443814 18441 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:19.444630 18513 consensus_queue.cc:1059] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:19.446122 18142 raft_consensus.cc:3055] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:19:19.446667 18142 raft_consensus.cc:740] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:19.442996 18241 consensus_queue.cc:1059] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 13:19:19.448160 18241 raft_consensus.cc:3055] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:19.447679 18142 consensus_queue.cc:260] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.10, Last appended by leader: 10, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:19.448523 18241 raft_consensus.cc:740] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:19.448885 18142 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:19.449618 18241 consensus_queue.cc:260] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:19.452950 10697 catalog_manager.cc:5697] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:19:19.455677 10697 catalog_manager.cc:5697] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 8 to 14. New cstate: current_term: 14 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:19.460683 18241 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:19.461846 18205 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:19:19.469779 18524 log.cc:927] Time spent T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.439s	user 0.002s	sys 0.012s
I20260709 13:19:19.373466 18379 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:19.388835 10945 raft_consensus.cc:1240] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 23. Current term is 24. Ops: []
W20260709 13:19:19.374027 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 50050.3us
I20260709 13:19:19.472867 18379 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:19.473219 10766 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:19.369055 18435 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:19.474931 10766 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:19.374244 18261 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:19.475575 18435 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:19.476305 18432 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 13:19:19.379637 10871 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:19.394286 10800 raft_consensus.cc:1275] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:19:19.476908 18432 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:19.477380 18432 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 28 FOLLOWER]: Advancing to term 29
W20260709 13:19:19.387423 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:19.401261 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 250684us
I20260709 13:19:19.447680 18400 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:19.476400 18435 raft_consensus.cc:2749] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:19.393245 18395 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:19.393960 10793 raft_consensus.cc:3055] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:19:19.488924 10793 raft_consensus.cc:740] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:19.491480 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:19:19.492509 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 48116.7us
I20260709 13:19:19.491070 18120 consensus_queue.cc:1048] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 13, Time since last communication: 0.003s
I20260709 13:19:19.496490 10916 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:19.466662 18441 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:19.478919 18261 raft_consensus.cc:697] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:19.397464 18530 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:19.487526 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:19.517318 10842 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:19.497457 18395 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:19:19.524679 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 49900.7us
I20260709 13:19:19.529606 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:19.518949 10793 consensus_queue.cc:260] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:19.544502 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 253757us
I20260709 13:19:19.545187 10799 raft_consensus.cc:1240] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:19.545353 10798 raft_consensus.cc:2393] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 26: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:19.552054 10801 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 25.
I20260709 13:19:19.562901 18531 consensus_queue.cc:1048] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.045s
I20260709 13:19:19.542302 18483 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:19.566628 18483 raft_consensus.cc:697] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:19.567808 18483 consensus_queue.cc:237] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:19.570271 10766 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:19.583598 10842 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:19.593497 11478 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:19.607535 18433 consensus_queue.cc:1059] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:19.607942 10766 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:19.608894 18433 raft_consensus.cc:3055] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:19.609328 18433 raft_consensus.cc:740] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:19.610164 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:19.611153 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:19.611506 18120 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:19.611073 18433 consensus_queue.cc:260] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:19.611868 11508 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:19.612929 18433 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:19.611795 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:19.632021 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:19.636720 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:19.597551 18531 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:19:19.636782 10766 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:19:19.639204 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 118143us
I20260709 13:19:19.641767 18465 raft_consensus.cc:2804] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:19.641921 18462 raft_consensus.cc:2804] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:19.642287 18465 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:19.642849 18465 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:19.646342 10871 raft_consensus.cc:1275] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:19:19.642463 18462 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:19.645112 18435 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:19.654585 11508 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 25.
I20260709 13:19:19.652557 10842 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:19.639603 18531 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:19.632550 17777 consensus_queue.cc:1048] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 13:19:19.680241 18513 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:19:19.680819 18513 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:19.681209 18513 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:19:19.685384 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 74914.8us
I20260709 13:19:19.686349 10766 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:19.687431 18486 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:19.687922 10766 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:19.688967 10766 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:19.670650 18531 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:19.689803 18486 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:19.661132 18462 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:19:19.663785 18509 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 19 current term: 19 is higher.
I20260709 13:19:19.753957 18352 consensus_queue.cc:1059] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.094s
I20260709 13:19:19.763338 18441 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:19.785096 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:19.786067 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:19.786969 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:19.787948 10916 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:19.648995 18174 consensus_queue.cc:1059] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.206s
I20260709 13:19:19.789579 18530 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 26 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:19.696823 18379 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:19.821396 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:19.709862 18400 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:19.832731 10948 raft_consensus.cc:2368] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 23. Current term is 24.
I20260709 13:19:19.822683 10950 raft_consensus.cc:2376] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 19. Re-sending same reply.
I20260709 13:19:19.853971 18509 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:19.867168 10793 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:19.827221 10949 raft_consensus.cc:2376] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 22. Re-sending same reply.
I20260709 13:19:19.802949 17777 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:19.901171 10941 raft_consensus.cc:2368] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 21. Current term is 22.
W20260709 13:19:19.919263 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 180167us
I20260709 13:19:19.931846 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:19.878630 18417 raft_consensus.cc:2749] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:19.939366 18400 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:19.940308 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:19.794914 18419 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:19.936838 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:19.656919 18261 consensus_queue.cc:237] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:19.820598 18422 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:19.940032 18400 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:19.975209 18400 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:20.000772 10947 raft_consensus.cc:2393] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 21: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:20.023054 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:20.033543 18205 consensus_meta.cc:287] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.571s	user 0.010s	sys 0.000s
I20260709 13:19:19.968132 18174 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:19.927441 17777 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:19.995766 18232 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:20.038642 18435 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:20.030370 18531 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:20.056983 18432 consensus_meta.cc:287] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.579s	user 0.005s	sys 0.008s
W20260709 13:19:20.068017 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 415508us
W20260709 13:19:20.083999 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:20.054235 18174 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:20.090754 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 497326us
I20260709 13:19:20.091809 11478 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:20.080386 18120 raft_consensus.cc:2749] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:19:20.101068 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:20.067332 18232 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:19.930996 18509 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:20.128263 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:20.129995 10868 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:19:20.132234 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:20.133126 10871 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:20.069878 18432 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:20.088626 18483 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:20.123034 18356 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.021s	sys 0.083s
W20260709 13:19:20.154541 18356 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.021s	sys 0.084s
I20260709 13:19:20.156366 18486 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:20.160023 18513 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:20.162915 18541 log.cc:927] Time spent T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:19:20.168089 18232 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:20.169883 18422 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:20.122624 18465 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:20.176386 18422 raft_consensus.cc:2804] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:20.190891 18422 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:20.191358 18422 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:20.193785 18175 raft_consensus.cc:3055] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:19:20.194378 18175 raft_consensus.cc:740] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:20.195355 18463 raft_consensus.cc:3055] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:20.195850 18463 raft_consensus.cc:740] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:20.195514 18175 consensus_queue.cc:260] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:20.196759 18463 consensus_queue.cc:260] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:20.197419 18175 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:20.197840 18463 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:20.162485 18513 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:20.180517 18433 consensus_meta.cc:287] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.567s	user 0.008s	sys 0.004s
I20260709 13:19:20.172278 18120 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:20.186051 10841 connection.cc:256] RPC call timeout handler was delayed by -1.6530028409999886s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24209999999999998s for a response
I20260709 13:19:20.168148 18174 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:20.233174 10871 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 24.
I20260709 13:19:20.242247 10799 raft_consensus.cc:1275] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:19:20.214824 18448 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:20.240566 18120 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:20.183911 18483 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:20.252852 17777 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:20.264668 18232 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:20.302560 10766 connection.cc:256] RPC call timeout handler was delayed by -1.6967136149999078s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16140000000000002s for a response
W20260709 13:19:20.304589 10766 connection.cc:256] RPC call timeout handler was delayed by -1.6400136150000435s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16770000000000002s for a response
I20260709 13:19:20.304875 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:20.305042 10766 connection.cc:256] RPC call timeout handler was delayed by -1.5563136149999082s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17700000000000002s for a response
W20260709 13:19:20.305543 10766 connection.cc:256] RPC call timeout handler was delayed by -1.3934136149998722s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19510000000000002s for a response
I20260709 13:19:20.305581 10947 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
W20260709 13:19:20.307969 10766 connection.cc:256] RPC call timeout handler was delayed by -1.0802136150000479s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2299s for a response
I20260709 13:19:20.271654 18486 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:20.274235 18531 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:20.328727 10944 raft_consensus.cc:1275] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 12. (term mismatch)
I20260709 13:19:20.329206 10944 pending_rounds.cc:85] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 11
I20260709 13:19:20.329458 10944 pending_rounds.cc:107] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 22.12
I20260709 13:19:20.329697 10944 raft_consensus.cc:2889] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:20.331127 18486 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:20.333734 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:20.334723 18509 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:20.335520 18232 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:20.271289 18448 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:20.271813 18205 consensus_queue.cc:1059] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.045s
I20260709 13:19:20.319259 10841 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:20.316888 17777 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:20.352690 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:20.338259 18232 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:20.355857 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:20.354825 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:20.361637 18175 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:20.363687 18120 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:20.364650 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 59489.2us
I20260709 13:19:20.354547 18548 raft_consensus.cc:3055] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:19:20.365530 18509 raft_consensus.cc:2749] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:20.366887 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:19:20.343227 18462 consensus_meta.cc:287] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.540s	user 0.012s	sys 0.000s
I20260709 13:19:20.376878 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:20.385021 10871 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 17.
I20260709 13:19:20.374429 10944 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 26.
I20260709 13:19:20.379264 10766 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:20.377627 18548 raft_consensus.cc:740] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:20.382282 10950 raft_consensus.cc:2393] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 21: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:20.382863 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 29 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:20.382957 10841 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:20.388497 10793 raft_consensus.cc:1275] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 13:19:20.389462 10793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 1065) took 1130 ms. Trace:
I20260709 13:19:20.389943 10793 rpcz_store.cc:276] 0709 13:19:19.259176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:19.338276 (+ 79100us) service_pool.cc:224] Handling call
0709 13:19:19.393784 (+ 55508us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:19.866920 (+473136us) spinlock_profiling.cc:243] Waited 312 ms on lock 0x7b5800413fd0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80e110aa 00007f9c80e93447 00007f9c80eabfe7 00007f9c80ead7de 00007f9c80eb7b49 00007f9c80eb8416 00007f9c80eb4d8f 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4
0709 13:19:20.389118 (+522198us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:20.389390 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":717108608}
I20260709 13:19:20.384289 10767 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:20.405135 18435 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:19:20.405624 18435 raft_consensus.cc:697] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:20.405730 10767 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:20.392657 10946 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:20.394699 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:20.358768 18432 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 29 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:20.400362 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:20.408149 10948 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 21.
I20260709 13:19:20.402262 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:20.402463 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:19:20.401105 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 309395us
I20260709 13:19:20.396998 18465 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:20.413384 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:20.416618 10947 raft_consensus.cc:2393] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 25: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:20.376343 17777 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:20.414126 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:20.412725 18142 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:20.418074 10944 raft_consensus.cc:2393] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 26: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:20.416728 10870 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 19.
I20260709 13:19:20.403721 18530 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:20.424188 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:20.425175 10871 raft_consensus.cc:1240] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:20.426667 18205 raft_consensus.cc:2749] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:20.428833 18400 consensus_queue.cc:1059] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:19:20.427012 18142 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:20.397342 18175 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:20.393821 18462 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:20.442539 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 252258us
I20260709 13:19:20.443161 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:20.430960 18241 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:20.444653 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:20.415478 18545 consensus_queue.cc:1048] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.024s
I20260709 13:19:20.448060 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:20.457715 11509 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 23.
I20260709 13:19:20.447923 18548 consensus_queue.cc:260] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:20.448730 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:20.448974 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:20.463920 11508 raft_consensus.cc:2393] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 26: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:20.449172 18395 raft_consensus.cc:3055] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:19:20.450165 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:20.451186 11504 raft_consensus.cc:1275] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:19:20.452005 17777 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:20.453682 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:19:20.454556 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 71714.9us
I20260709 13:19:20.454938 18419 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:20.417922 18174 consensus_queue.cc:1048] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.021s
I20260709 13:19:20.462328 10869 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 19.
W20260709 13:19:20.465399 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 294589us
I20260709 13:19:20.465049 10800 raft_consensus.cc:1240] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 21. Current term is 22. Ops: []
W20260709 13:19:20.466149 18546 log.cc:927] Time spent T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 13:19:20.475766 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:20.422032 18435 consensus_queue.cc:237] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:20.422741 18432 raft_consensus.cc:2804] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:20.481918 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:20.464985 18395 raft_consensus.cc:740] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:19:20.487761 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:20.495702 11509 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 25.
I20260709 13:19:20.501226 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:20.501865 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:20.501295 18554 consensus_queue.cc:1059] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 13:19:20.503026 18395 consensus_queue.cc:260] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:20.504050 18395 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:20.471765 10868 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 26.
I20260709 13:19:20.514770 10947 raft_consensus.cc:1275] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:19:20.492915 18548 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:19:20.523607 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 117961us
I20260709 13:19:20.526005 10767 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:20.530608 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:20.531296 10868 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 22.
I20260709 13:19:20.535048 10947 raft_consensus.cc:1240] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:20.487697 18432 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:20.474227 18241 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:20.498888 18486 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:20.544100 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:20.544202 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:20.546907 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 54399.1us
I20260709 13:19:20.547487 10943 raft_consensus.cc:2393] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 25: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:20.547667 11478 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:20.541093 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:20.552380 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:20.539480 18432 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:20.549976 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 24 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:20.536047 18465 raft_consensus.cc:2804] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:20.549404 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:20.556571 10948 raft_consensus.cc:2393] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 19: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:20.559687 11478 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:20.558157 10945 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 20.
I20260709 13:19:20.569530 18556 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:20.569917 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:20.570214 18556 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:20.570760 10949 raft_consensus.cc:2393] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 21: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:20.571978 18422 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:20.572584 18556 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:20.574294 18422 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:20.544153 18448 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:20.539918 18557 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:20.556274 18419 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:20.594244 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 94182.9us
W20260709 13:19:20.595947 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 249918us
I20260709 13:19:20.560560 18545 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:20.560801 18465 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:20.595186 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:20.600076 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:20.602085 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:20.572016 18554 raft_consensus.cc:2749] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:20.612100 10943 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:20.618165 10947 raft_consensus.cc:2410] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 23 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 13:19:20.622622 10949 raft_consensus.cc:1240] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
W20260709 13:19:20.623488 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 71561.5us
I20260709 13:19:20.625653 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:20.629431 10946 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
I20260709 13:19:20.610486 18465 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:20.580470 17777 consensus_queue.cc:1059] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.042s
I20260709 13:19:20.618655 18142 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:20.634187 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 74593.3us
I20260709 13:19:20.635557 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:20.637950 11478 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:20.632812 11477 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:19:20.639854 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:20.643275 18553 log.cc:927] Time spent T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:19:20.645776 17777 raft_consensus.cc:3055] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:19:20.646173 17777 raft_consensus.cc:740] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:20.646348 10942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 1356) took 1131 ms. Trace:
I20260709 13:19:20.647094 17777 consensus_queue.cc:260] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:20.647676 18558 raft_consensus.cc:2749] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:20.648553 18558 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:20.649014 18558 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:20.651122 18558 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:20.653402 18379 consensus_meta.cc:287] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.774s	user 0.006s	sys 0.006s
I20260709 13:19:20.653808 18379 raft_consensus.cc:2749] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:20.621695 18419 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:20.656854 18419 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:20.657902 18241 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:20.612928 18557 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:20.658981 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 46864.7us
I20260709 13:19:20.660034 10841 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:19:20.660550 18557 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:20.660945 10841 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:20.664856 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 63300.2us
I20260709 13:19:20.666429 10766 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:20.637104 18462 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:20.671792 18432 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:20.675091 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:20.675750 10868 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:20.671921 18465 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:20.678094 10841 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:20.683318 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:19:20.679049 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 46304.7us
I20260709 13:19:20.684129 10869 raft_consensus.cc:2410] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 22 because replica has last-logged OpId of term: 21 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 9.
I20260709 13:19:20.684679 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:20.685009 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:20.685429 10872 raft_consensus.cc:2393] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 19: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:20.685685 10874 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:20.685127 10870 raft_consensus.cc:1240] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 23. Current term is 24. Ops: []
I20260709 13:19:20.680967 11478 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:20.682374 10943 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 23.
I20260709 13:19:20.690032 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:20.646718 10942 rpcz_store.cc:276] 0709 13:19:19.514515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:19.518446 (+  3931us) service_pool.cc:224] Handling call
0709 13:19:19.541829 (+ 23383us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:19.554578 (+ 12749us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 13:19:19.600601 (+ 46023us) log.cc:844] Serialized 14 byte log entry
0709 13:19:19.974282 (+373681us) spinlock_profiling.cc:243] Waited 9.61 ms on lock 0x7b180046df80. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a5523a 00007f9c80ebfa2a
0709 13:19:20.517161 (+542879us) log.cc:844] Serialized 14 byte log entry
0709 13:19:20.537634 (+ 20473us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:20.538830 (+  1196us) log.cc:844] Serialized 26 byte log entry
0709 13:19:20.539025 (+   195us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:19:20.539057 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:20.545227 (+  6170us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:20.646123 (+100896us) raft_consensus.cc:1685] finished
0709 13:19:20.646145 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:20.646288 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22349,"prepare.queue_time_us":247766,"prepare.run_cpu_time_us":107,"prepare.run_wall_time_us":3202,"raft.queue_time_us":76766,"raft.run_cpu_time_us":346,"raft.run_wall_time_us":7781,"spinlock_wait_cycles":22092160,"thread_start_us":542391,"threads_started":2,"wal-append.queue_time_us":867138}
I20260709 13:19:20.693243 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:20.695273 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:20.648561 17777 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:20.697434 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:20.702148 10872 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:19:20.679557 18463 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:20.704317 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:20.705324 10841 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:20.679881 18548 raft_consensus.cc:2749] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:19:20.707787 10867 raft_consensus.cc:2393] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 21: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:20.685956 18554 raft_consensus.cc:2804] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:20.709234 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: true dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:20.709645 11509 pending_rounds.cc:85] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586: Aborting all ops after (but not including) 14
I20260709 13:19:20.709801 10869 raft_consensus.cc:3055] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:20.710037 11509 pending_rounds.cc:107] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586: Aborting uncommitted NO_OP operation due to leader change: 21.15
I20260709 13:19:20.710263 10869 raft_consensus.cc:740] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:20.710412 11509 raft_consensus.cc:2889] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:20.711068 10869 consensus_queue.cc:260] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:20.712041 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:20.712934 18554 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:20.713186 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:20.713397 18554 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:20.713994 10799 raft_consensus.cc:2393] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 21: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:20.714743 18561 raft_consensus.cc:2749] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:20.715096 18205 raft_consensus.cc:2749] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:19:20.717067 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:20.718317 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:20.718961 11510 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 22.
I20260709 13:19:20.686100 18241 consensus_queue.cc:1048] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.035s
I20260709 13:19:20.689024 18379 consensus_queue.cc:1048] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 13:19:20.723455 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:20.724282 10942 raft_consensus.cc:2393] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 25: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:20.725216 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:20.725389 18552 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:20.725943 18552 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:20.726401 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:20.729053 10949 raft_consensus.cc:1240] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:20.729194 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:20.727517 18569 raft_consensus.cc:3055] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 13:19:20.725349 10916 connection.cc:256] RPC call timeout handler was delayed by -1.5022037670000827s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1575s for a response
W20260709 13:19:20.739435 10916 connection.cc:256] RPC call timeout handler was delayed by -1.2277037670000936s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.188s for a response
I20260709 13:19:20.720947 18545 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:20.732481 10872 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 26.
I20260709 13:19:20.733127 10766 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:20.734683 10870 raft_consensus.cc:2393] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 25: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:20.733866 10874 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 22.
I20260709 13:19:20.737680 10868 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 18.
I20260709 13:19:20.728197 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:20.741777 10793 raft_consensus.cc:1240] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 23. Current term is 24. Ops: []
I20260709 13:19:20.697568 18549 raft_consensus.cc:2764] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Leader pre-election decision vote started in defunct term 25: won
I20260709 13:19:20.747424 10876 raft_consensus.cc:1240] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:20.739986 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:20.746708 10870 raft_consensus.cc:1275] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:19:20.748466 10949 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 22.
I20260709 13:19:20.707248 18120 raft_consensus.cc:2804] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:20.705471 18463 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:20.754060 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:20.754508 11478 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:20.751180 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:20.751294 10841 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:20.753521 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:20.754683 10949 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
I20260709 13:19:20.761137 18435 raft_consensus.cc:2764] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election decision vote started in defunct term 19: won
I20260709 13:19:20.739089 18569 raft_consensus.cc:740] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:20.767829 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:20.764004 18545 consensus_queue.cc:1059] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 13, Time since last communication: 0.007s
I20260709 13:19:20.771768 11508 raft_consensus.cc:1240] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:20.771761 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:20.772907 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:20.773273 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:20.773488 11510 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:20.774456 10799 raft_consensus.cc:2393] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 21: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:20.771812 10766 leader_election.cc:400] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 election: Vote denied by peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553) with higher term. Message: Invalid argument: T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 21. Current term is 22.
I20260709 13:19:20.748896 18432 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:20.742880 18531 consensus_queue.cc:1048] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 13:19:20.757437 18463 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:20.788424 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:20.795085 10766 leader_election.cc:400] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 election: Vote denied by peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553) with higher term. Message: Invalid argument: T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 23. Current term is 24.
I20260709 13:19:20.797307 10766 leader_election.cc:403] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
W20260709 13:19:20.797871 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 51372us
I20260709 13:19:20.803279 18261 raft_consensus.cc:2749] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553) with higher term. Message: Invalid argument: T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 23. Current term is 24.
I20260709 13:19:20.804915 11507 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 24.
W20260709 13:19:20.805554 18551 log.cc:927] Time spent T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.350s	user 0.000s	sys 0.003s
I20260709 13:19:20.750365 18567 consensus_queue.cc:1059] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.036s
I20260709 13:19:20.765551 18447 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:19:20.798702 10841 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:20.764029 18120 raft_consensus.cc:697] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 19 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:20.813567 11510 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 25.
I20260709 13:19:20.813748 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:20.818956 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:20.827847 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:20.835825 10872 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 19.
I20260709 13:19:20.819195 18569 consensus_queue.cc:260] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:20.825271 18530 raft_consensus.cc:2749] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:20.783983 18549 raft_consensus.cc:2749] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:19:20.844008 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:20.830761 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:20.827502 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:20.833949 10767 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:20.832962 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:20.828157 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:20.845513 10874 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 22.
I20260709 13:19:20.841977 10766 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:20.842391 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:20.828712 10873 raft_consensus.cc:1240] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 25. Current term is 26. Ops: []
I20260709 13:19:20.844223 10798 raft_consensus.cc:2393] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 19: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:20.843724 18352 raft_consensus.cc:2749] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:19:20.832763 18447 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:20.848069 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:20.848489 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:20.848822 18432 consensus_queue.cc:1048] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:20.849334 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:20.850055 10795 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 20.
I20260709 13:19:20.848508 18447 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:20.850976 18563 raft_consensus.cc:2804] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:20.851527 18563 raft_consensus.cc:697] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:20.853093 18567 raft_consensus.cc:3055] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:19:20.852613 18563 consensus_queue.cc:237] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:20.854199 18569 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:19:20.854892 18572 log.cc:927] Time spent T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 13:19:20.857203 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:20.873857 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:20.874042 10872 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 20.
W20260709 13:19:20.858589 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 59975.5us
I20260709 13:19:20.826725 18552 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:20.822261 18554 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:20.878624 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:20.859611 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:20.859735 10766 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:20.879532 11510 raft_consensus.cc:2393] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 19: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:20.880017 10868 raft_consensus.cc:1240] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:20.858201 18120 consensus_queue.cc:237] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:20.845889 10767 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:20.860229 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:20.860453 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:19:20.890154 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 44349.9us
I20260709 13:19:20.861514 10949 raft_consensus.cc:2393] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 20: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:20.847224 10870 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 26.
I20260709 13:19:20.765689 18174 consensus_queue.cc:1048] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.049s
I20260709 13:19:20.851703 10916 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:20.845746 10867 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 18.
I20260709 13:19:20.865505 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:20.867321 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:20.891702 10798 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:20.872964 10792 raft_consensus.cc:1240] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:20.872848 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:20.892679 10841 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:20.882494 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:20.890976 10767 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:20.893365 11507 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:20.893970 11504 raft_consensus.cc:2393] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 21: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:20.867254 17777 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:20.884038 18447 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:20.885329 10766 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:20.885931 18432 raft_consensus.cc:2749] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:19:20.886387 18530 consensus_queue.cc:1059] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:19:20.907272 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:20.888777 18554 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:20.890282 10794 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:20.860998 18261 raft_consensus.cc:2749] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:20.892834 10797 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:20.867477 18352 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:20.882831 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: true dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:20.895562 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:20.897125 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:20.898638 10767 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:19:20.899761 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 65390.4us
I20260709 13:19:20.932890 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:20.900197 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:20.933529 10870 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 21.
I20260709 13:19:20.866514 18567 raft_consensus.cc:740] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:20.902649 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:20.905298 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:20.905809 10766 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:20.907460 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:20.862643 18462 consensus_queue.cc:1059] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.052s
I20260709 13:19:20.937795 11507 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 21.
I20260709 13:19:20.974964 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 24 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:20.910408 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:19:20.916224 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:20.916478 10801 raft_consensus.cc:1275] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 13:19:20.893723 18548 consensus_queue.cc:1048] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 13:19:20.987411 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:20.988376 11512 raft_consensus.cc:2368] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 24. Current term is 25.
I20260709 13:19:20.989315 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:20.990000 11512 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:19:20.990978 18465 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:20.992486 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:20.993831 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:20.995074 18566 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:20.995790 18566 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:20.916708 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:20.998281 10949 raft_consensus.cc:2393] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 19: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:20.998186 18483 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:20.999156 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 29 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:20.999092 18432 consensus_queue.cc:1048] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.019s
I20260709 13:19:21.000056 10945 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:19:21.000415 11505 raft_consensus.cc:1240] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 24. Current term is 25. Ops: []
I20260709 13:19:21.001508 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:20.917636 18447 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:20.896298 18448 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:19:21.003244 18448 raft_consensus.cc:697] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:19:21.004643 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 98932.4us
I20260709 13:19:21.006036 10766 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:20.899621 17777 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:21.006762 18175 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:21.007428 11507 raft_consensus.cc:1240] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:21.008353 18400 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:21.008646 17777 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:21.009120 18400 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:21.010167 10792 raft_consensus.cc:2393] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 18: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:21.010960 18395 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:21.011225 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:21.011673 18395 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:21.012951 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:21.014245 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:21.014422 18395 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:20.918503 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:19:20.927784 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 76186.2us
I20260709 13:19:20.919128 18435 raft_consensus.cc:2749] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:19:20.903419 18509 raft_consensus.cc:2749] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:19:20.999436 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:20.917039 11508 raft_consensus.cc:1240] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 24. Current term is 25. Ops: []
I20260709 13:19:21.004272 18448 consensus_queue.cc:237] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:20.898322 18379 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:21.008479 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:20.904942 18552 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:20.980406 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:21.081655 10872 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 20.
W20260709 13:19:21.082180 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 50365.3us
I20260709 13:19:20.974174 18352 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:20.976135 18561 raft_consensus.cc:2804] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:21.002729 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:21.086387 10875 raft_consensus.cc:2393] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 22: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:19:21.019681 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:21.020133 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:21.020429 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:21.089028 10874 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:20.906617 10871 raft_consensus.cc:1275] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 11. (term mismatch)
I20260709 13:19:21.090075 10871 pending_rounds.cc:85] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 10
I20260709 13:19:21.090355 10871 pending_rounds.cc:107] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 20.11
I20260709 13:19:21.090597 10871 raft_consensus.cc:2889] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:20.919144 18557 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:21.026314 18422 raft_consensus.cc:3055] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:19:21.094414 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:21.097365 11510 raft_consensus.cc:2376] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 26. Re-sending same reply.
W20260709 13:19:21.027181 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:21.098980 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:21.027571 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:21.028509 10795 raft_consensus.cc:1240] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:20.980922 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:21.018654 11478 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:19:20.919953 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:20.980528 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:19:21.105757 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 87188.9us
I20260709 13:19:20.998214 18566 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:21.019735 10766 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:21.031421 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:21.033932 10792 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:19:21.036625 10794 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 22.
I20260709 13:19:20.936523 10799 raft_consensus.cc:2393] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 20: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:21.040203 11511 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 22.
W20260709 13:19:20.979003 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 97126.1us
I20260709 13:19:21.116291 10841 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:21.124182 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:21.125042 10872 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 26.
W20260709 13:19:21.128684 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 109032us
I20260709 13:19:21.080716 18261 raft_consensus.cc:2749] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:19:21.062316 11507 raft_consensus.cc:2393] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 21: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:21.080907 18379 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:21.080197 10949 raft_consensus.cc:2393] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 22: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:21.022634 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:21.092267 11504 raft_consensus.cc:2393] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 22: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:21.092500 11505 raft_consensus.cc:2393] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 21: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:21.004678 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:21.051818 18432 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:19:21.108942 10801 raft_consensus.cc:2393] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 21: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:21.112669 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:21.148695 10948 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 26.
I20260709 13:19:21.142560 10873 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 24.
I20260709 13:19:21.122763 10915 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:21.150344 11477 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:21.151248 10915 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:21.151264 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:21.157429 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:21.109207 18422 raft_consensus.cc:740] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
W20260709 13:19:21.167017 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 50835us
I20260709 13:19:21.168139 10841 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:21.116314 18447 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:21.160552 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:21.152668 18435 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:21.162823 18558 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:19:21.163038 18552 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:19:21.152630 18509 raft_consensus.cc:2804] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:21.163618 10948 raft_consensus.cc:1275] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:19:21.163836 10941 raft_consensus.cc:1240] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:21.164127 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:21.177055 10841 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:21.123358 18561 raft_consensus.cc:697] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:19:21.179071 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 245080us
I20260709 13:19:21.182575 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:20.981133 10798 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 24.
I20260709 13:19:21.052502 10797 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 25.
I20260709 13:19:21.150837 18379 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:21.189302 10871 raft_consensus.cc:1240] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 13:19:21.224509 10949 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:21.222234 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:21.228276 10797 raft_consensus.cc:1275] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:19:21.229291 10874 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 21.
I20260709 13:19:21.115957 18557 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:21.230540 10874 raft_consensus.cc:1240] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 23. Current term is 24. Ops: []
I20260709 13:19:21.231122 10873 raft_consensus.cc:2410] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 23 because replica has last-logged OpId of term: 21 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 16.
W20260709 13:19:21.231855 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 54922.8us
W20260709 13:19:21.187954 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 67700.2us
I20260709 13:19:21.130216 18417 consensus_queue.cc:1048] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.066s
I20260709 13:19:21.149035 18566 consensus_queue.cc:1059] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.081s
I20260709 13:19:21.236553 18531 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:19:21.239269 18531 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:21.239717 18531 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:19:21.188107 18548 raft_consensus.cc:2749] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:21.241770 18563 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:21.242389 18563 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:21.243008 10872 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:21.243770 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:21.244606 18563 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:21.245175 11503 raft_consensus.cc:1240] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:21.245487 18563 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:21.246075 18563 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:21.248109 10944 raft_consensus.cc:1240] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:21.248881 18563 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:21.249346 10944 raft_consensus.cc:1240] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:21.250514 10944 raft_consensus.cc:1240] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 24. Current term is 25. Ops: []
I20260709 13:19:21.246529 11503 raft_consensus.cc:1275] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:19:21.250612 10872 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 25.
I20260709 13:19:21.253578 11512 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 26.
I20260709 13:19:21.254858 10944 raft_consensus.cc:1275] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 15. (term mismatch)
I20260709 13:19:21.255887 18531 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:21.256507 10944 pending_rounds.cc:85] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 14
I20260709 13:19:21.256901 10944 pending_rounds.cc:107] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 22.15
I20260709 13:19:21.257194 10944 raft_consensus.cc:2889] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:21.258726 10944 raft_consensus.cc:1275] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
W20260709 13:19:21.258865 18544 log.cc:927] Time spent T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.992s	user 0.002s	sys 0.001s
I20260709 13:19:21.259330 18531 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 27 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:21.260142 10944 raft_consensus.cc:1240] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:21.261670 10869 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:21.261879 10944 raft_consensus.cc:1275] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:19:21.264117 18352 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:21.265723 18417 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:19:21.266304 18417 raft_consensus.cc:697] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 25 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:21.267128 11478 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:21.173822 18447 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:21.237452 10841 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:21.187438 18483 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.183445 18558 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:21.222919 18462 raft_consensus.cc:2749] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:19:21.274595 10948 raft_consensus.cc:1217] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Deduplicated request from leader. Original: 17.10->[18.11-22.13]   Dedup: 18.11->[20.12-22.13]
I20260709 13:19:21.281430 11478 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:19:21.224244 18509 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:21.279109 10841 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:21.300999 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:21.237164 18400 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:21.299499 10948 raft_consensus.cc:1240] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 22. Current term is 23. Ops: [18.11-22.13]
I20260709 13:19:21.272811 18352 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.309000 10841 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:21.273643 18447 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:21.315296 10841 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:21.317575 10869 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 20.
I20260709 13:19:21.183313 18435 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:21.334896 10841 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:21.289322 18576 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:19:21.303879 18558 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:19:21.339613 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:21.300748 18531 raft_consensus.cc:2749] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:21.178834 18567 consensus_queue.cc:260] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:21.304409 18575 raft_consensus.cc:2804] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:21.302446 18509 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:21.340090 18532 raft_consensus.cc:2749] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 13:19:21.357847 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 206680us
I20260709 13:19:21.365751 10841 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:21.341527 18582 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.286384 18483 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:21.387966 10945 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 29.
W20260709 13:19:21.387573 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 157795us
I20260709 13:19:21.394573 10766 leader_election.cc:400] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 election: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 24. Current term is 25.
I20260709 13:19:21.363081 10797 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
I20260709 13:19:21.339768 18548 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:19:21.362315 18574 log.cc:927] Time spent T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
I20260709 13:19:21.333709 18241 raft_consensus.cc:2749] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:19:21.359117 18576 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:21.366621 18350 raft_consensus.cc:2749] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:21.416570 10947 raft_consensus.cc:1240] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 29 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 28. Current term is 29. Ops: []
W20260709 13:19:21.416626 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 50960.7us
I20260709 13:19:21.417200 18548 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:21.360898 18462 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:21.430490 18462 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:21.432806 18462 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:21.339190 18352 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:21.360491 18552 raft_consensus.cc:697] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 24 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:21.301611 18422 consensus_queue.cc:260] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:21.421432 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 183105us
I20260709 13:19:21.399961 18575 raft_consensus.cc:697] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 21 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
W20260709 13:19:21.427103 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 58429.2us
I20260709 13:19:21.426888 18582 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:21.391428 18561 consensus_queue.cc:237] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:21.433990 18548 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:19:21.428465 18569 consensus_meta.cc:287] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.547s	user 0.008s	sys 0.002s
I20260709 13:19:21.429451 18583 consensus_queue.cc:1048] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:19:21.422591 18576 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:21.415339 10915 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:21.439851 10842 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:21.422268 18432 raft_consensus.cc:697] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:21.448319 10841 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:21.424939 18532 raft_consensus.cc:2749] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:21.453400 10949 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 21.
W20260709 13:19:21.463927 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 50342.6us
I20260709 13:19:21.304617 18417 consensus_queue.cc:237] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:21.444258 18577 log.cc:927] Time spent T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.222s	user 0.000s	sys 0.002s
I20260709 13:19:21.460582 10766 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:21.462404 10800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3375) took 1381 ms. Trace:
I20260709 13:19:21.464972 10915 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:21.466352 10841 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:21.467113 18350 raft_consensus.cc:2749] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:19:21.463778 18241 raft_consensus.cc:2804] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:21.469276 10800 rpcz_store.cc:276] 0709 13:19:20.080587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:20.080735 (+   148us) service_pool.cc:224] Handling call
0709 13:19:20.100494 (+ 19759us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:20.476380 (+375886us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:21.462080 (+985700us) spinlock_profiling.cc:243] Waited 364 ms on lock 0x7b54001aa9d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:19:21.462320 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":837319424}
I20260709 13:19:21.474349 11510 raft_consensus.cc:1240] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 19 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:19:21.444621 18549 consensus_queue.cc:1048] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 11, Time since last communication: 0.026s
W20260709 13:19:21.485562 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:21.493772 10766 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:21.472285 18531 raft_consensus.cc:2764] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:19:21.491235 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:21.446295 18175 consensus_queue.cc:1059] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 13, Time since last communication: 0.025s
W20260709 13:19:21.495729 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:21.491451 18580 raft_consensus.cc:2749] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:21.470834 18395 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:21.490070 18241 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:21.499162 18463 consensus_meta.cc:287] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.675s	user 0.008s	sys 0.003s
W20260709 13:19:21.514591 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:21.498711 10766 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:21.465037 18462 consensus_queue.cc:1059] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.034s
I20260709 13:19:21.468153 18594 consensus_queue.cc:1048] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.041s
I20260709 13:19:21.497686 18350 consensus_queue.cc:1059] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 13:19:21.500310 18583 raft_consensus.cc:2749] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:19:21.533103 18588 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.532518 18603 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:19:21.493818 18532 consensus_queue.cc:1048] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 13:19:21.544322 18419 consensus_queue.cc:1059] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:19:21.512465 18241 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:21.545925 10792 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 26.
W20260709 13:19:21.550360 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 51723.1us
I20260709 13:19:21.471287 18593 consensus_queue.cc:1059] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.058s
I20260709 13:19:21.535517 18350 raft_consensus.cc:3055] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 13:19:21.555204 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 84678us
I20260709 13:19:21.558836 10916 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:21.563140 10800 raft_consensus.cc:1240] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:21.563000 10697 catalog_manager.cc:5697] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 8 to 11, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 11 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:19:21.518894 18463 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:21.554041 18580 raft_consensus.cc:2749] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:19:21.576501 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 98281.9us
I20260709 13:19:21.577632 10916 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:21.556144 18350 raft_consensus.cc:740] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:21.545284 18566 consensus_queue.cc:1059] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.022s
I20260709 13:19:21.585597 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:21.582919 10916 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:21.566277 18435 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:21.617676 10867 raft_consensus.cc:2393] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 24: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:21.565975 18603 raft_consensus.cc:697] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 26 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:21.620699 10942 raft_consensus.cc:3055] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:19:21.627599 10942 raft_consensus.cc:740] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:19:21.659149 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:21.610720 10941 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 20.
I20260709 13:19:21.536264 18575 consensus_queue.cc:237] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:21.575196 18429 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:21.576506 18395 raft_consensus.cc:697] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 21 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:21.612234 18419 raft_consensus.cc:2804] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:21.561868 18549 consensus_queue.cc:1059] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.059s
I20260709 13:19:21.569895 18593 consensus_queue.cc:1059] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.051s
I20260709 13:19:21.521102 18552 consensus_queue.cc:237] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:21.549609 18588 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:21.572434 18548 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:21.717837 10942 consensus_queue.cc:260] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:21.611100 18462 consensus_queue.cc:1059] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.142s
I20260709 13:19:21.608901 18447 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:21.565893 18432 consensus_queue.cc:237] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:21.773383 18609 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.775177  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:19:21.775561 18602 log.cc:927] Time spent T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:19:21.776284 18610 raft_consensus.cc:2804] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:21.776872 18610 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:21.777360 18610 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:21.783280 18603 consensus_queue.cc:237] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:21.789489 18352 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:21.790200 18352 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.791126 18611 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:21.791841 18611 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:21.792308 18611 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:21.792693 18395 consensus_queue.cc:237] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:21.795138 10692 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42956 (request call id 262) took 43499 ms (client timeout 15000 ms). Trace:
I20260709 13:19:21.722494 18350 consensus_queue.cc:260] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:21.758082 18582 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:21.799911 18465 raft_consensus.cc:3055] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:21.800330 18465 raft_consensus.cc:740] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:21.801151 18465 consensus_queue.cc:260] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:21.802311 18465 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:21.802811 10798 raft_consensus.cc:1240] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:21.795158 10942 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:21.762431 18419 raft_consensus.cc:697] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:21.749979 18429 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:21.810500 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:21.820906 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:21.822881 11511 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 22.
I20260709 13:19:21.824028 11478 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:21.822208 11505 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 25.
I20260709 13:19:21.838274 11478 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:21.850521 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:21.812079 18549 raft_consensus.cc:3055] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:19:21.809195 18593 raft_consensus.cc:3055] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:19:21.871331 18593 raft_consensus.cc:740] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:21.872177 18593 consensus_queue.cc:260] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:21.874347 10686 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 2b82f18992434ed5ba58b805107060d4 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 468a8fd46e824f62b9c2c0d3532a6d26 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 22 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:21.849054 18545 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:19:21.876080 18548 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:21.838064 18598 log.cc:927] Time spent T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.324s	user 0.001s	sys 0.001s
I20260709 13:19:21.877259 18588 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:21.879451 18241 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:21.881709 18241 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:21.882308 17777 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:21.774273 18609 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:21.813495 11506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56404 (request call id 1262) took 1049 ms. Trace:
I20260709 13:19:21.870415 18419 consensus_queue.cc:237] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:21.854489 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:21.790696 18352 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:21.890978 11506 rpcz_store.cc:276] 0709 13:19:20.764459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:20.764578 (+   119us) service_pool.cc:224] Handling call
0709 13:19:20.770612 (+  6034us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:19:20.791517 (+ 20905us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:20.792067 (+   550us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:19:20.802936 (+ 10869us) log.cc:844] Serialized 78 byte log entry
0709 13:19:21.464658 (+661722us) spinlock_profiling.cc:243] Waited 4.76 ms on lock 0x7b18003091c0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a550d1 00007f9c80e70a0f
0709 13:19:21.553839 (+ 89181us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:19:21.759654 (+205815us) log.cc:844] Serialized 14 byte log entry
0709 13:19:21.760025 (+   371us) log.cc:844] Serialized 14 byte log entry
0709 13:19:21.761193 (+  1168us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:21.761381 (+   188us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:21.813171 (+ 51790us) raft_consensus.cc:1685] finished
0709 13:19:21.813215 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:21.813420 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":742,"prepare.queue_time_us":329835,"raft.queue_time_us":2142,"raft.run_cpu_time_us":507,"raft.run_wall_time_us":46540,"spinlock_wait_cycles":10945792,"thread_start_us":302462,"threads_started":3,"wal-append.queue_time_us":915673}
I20260709 13:19:21.857817 18549 raft_consensus.cc:740] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:19:21.920738 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 82655.4us
W20260709 13:19:21.922032 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:21.884524 18545 raft_consensus.cc:697] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 25 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
W20260709 13:19:21.922632 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 356174us
I20260709 13:19:21.927603 10841 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:19:21.925979 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 343148us
I20260709 13:19:21.931349 10841 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:21.931627 10916 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:19:21.929404 10950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 3294) took 1427 ms. Trace:
I20260709 13:19:21.939226 17777 raft_consensus.cc:2749] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:21.916477 18588 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:21.965615 10871 raft_consensus.cc:1275] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:19:21.968644 10766 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:21.884562 18592 consensus_queue.cc:1059] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.083s
I20260709 13:19:21.939350 18241 raft_consensus.cc:2749] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:21.968819 18587 raft_consensus.cc:2764] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:19:21.981072 10766 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:21.970948 18463 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:21.939754 18557 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:21.795789 10692 rpcz_store.cc:269] 0709 13:18:38.303480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:38.382247 (+ 78767us) service_pool.cc:224] Handling call
0709 13:18:38.771514 (+389267us) spinlock_profiling.cc:243] Waited 90.9 ms on lock 0x7b54002fce54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1f090 00007f9c80ebdf3a 00007f9c86c741cc 00007f9c86d7237e 00007f9c852332a4 00007f9c85233232 00007f9c85233161 00007f9c852330dc 00007f9c85232392 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:19:21.794829 (+43023315us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:12.679542 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:12.683901 (+  4359us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:12.683951 (+    50us) write_op.cc:435] Acquired schema lock
0709 13:19:12.683970 (+    19us) tablet.cc:662] Decoding operations
0709 13:19:12.684860 (+   890us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:12.684950 (+    90us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:12.684982 (+    32us) tablet.cc:685] Acquiring locks for 7 operations
0709 13:19:12.685656 (+   674us) tablet.cc:701] Row locks acquired
0709 13:19:12.685705 (+    49us) write_op.cc:260] PREPARE: finished
0709 13:19:12.685863 (+   158us) write_op.cc:270] Start()
0709 13:19:12.686020 (+   157us) write_op.cc:276] Timestamp: P: 1783603152685804 usec, L: 0
0709 13:19:12.686043 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:19:12.778124 (+ 92081us) log.cc:844] Serialized 2845 byte log entry
0709 13:19:12.982816 (+204692us) op_driver.cc:464] REPLICATION: finished
0709 13:19:13.178917 (+196101us) write_op.cc:301] APPLY: starting
0709 13:19:13.185696 (+  6779us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:13.355855 (+170159us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:13.355968 (+   113us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:13.745532 (+389564us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:13.790024 (+ 44492us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=14,key_file_lookups=14,delta_file_lookups=14,mrs_lookups=0
0709 13:19:13.790084 (+    60us) write_op.cc:312] APPLY: finished
0709 13:19:13.790400 (+   316us) log.cc:844] Serialized 73 byte log entry
0709 13:19:13.790589 (+   189us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:13.790732 (+   143us) write_op.cc:454] Released schema lock
0709 13:19:13.790815 (+    83us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:13.791285 (+   470us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:13.791321 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":75976,"reported_tablets":49,"spinlock_wait_cycles":209005440,"child_traces":[["op",{"apply.queue_time_us":184296,"apply.run_cpu_time_us":13346,"apply.run_wall_time_us":618880,"cfile_cache_hit":30,"cfile_cache_hit_bytes":33401,"num_ops":7,"prepare.queue_time_us":118768,"prepare.run_cpu_time_us":24648,"prepare.run_wall_time_us":225763,"raft.queue_time_us":17059,"raft.run_cpu_time_us":13531,"raft.run_wall_time_us":167495,"replication_time_us":296561,"thread_start_us":240716,"threads_started":4,"wal-append.queue_time_us":123640,"wal-append.run_cpu_time_us":5943,"wal-append.run_wall_time_us":1926316}]]}
I20260709 13:19:21.963697 18435 consensus_queue.cc:1048] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.035s
I20260709 13:19:21.998122 18563 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 13:19:21.967121 18582 consensus_queue.cc:1048] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.080s
I20260709 13:19:21.977365 18448 consensus_queue.cc:1059] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.075s
I20260709 13:19:21.974543 18261 consensus_queue.cc:1048] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.079s
I20260709 13:19:22.057789 17777 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.966537 18613 consensus_queue.cc:1059] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.110s
W20260709 13:19:22.061759 18615 log.cc:927] Time spent T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:19:21.942900 18614 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:19:22.054476 18379 consensus_meta.cc:287] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.786s	user 0.011s	sys 0.002s
I20260709 13:19:21.979146 18588 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:22.101111 17777 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:22.101389 18576 consensus_meta.cc:287] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.633s	user 0.009s	sys 0.003s
W20260709 13:19:22.128196 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 147161us
I20260709 13:19:22.132663 17777 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:22.133081 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 325159us
I20260709 13:19:22.113317 18561 raft_consensus.cc:2804] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:22.103713 18563 raft_consensus.cc:697] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 29 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:22.115424 18614 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:22.139791 18561 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:22.149601 18561 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:22.149480 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:19:22.150107 18623 log.cc:927] Time spent T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:19:22.150333 11506 raft_consensus.cc:2376] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 22. Re-sending same reply.
W20260709 13:19:22.152343 18621 log.cc:927] Time spent T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:19:22.153573 10800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3388) took 1124 ms. Trace:
I20260709 13:19:22.059608 18549 consensus_queue.cc:260] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:22.156340 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:22.157158 11509 raft_consensus.cc:1240] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:22.157176 10766 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:22.158480 10766 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:22.163427 10875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49290 (request call id 1197) took 1266 ms. Trace:
I20260709 13:19:22.163949 10875 rpcz_store.cc:276] 0709 13:19:20.896686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:20.896812 (+   126us) service_pool.cc:224] Handling call
0709 13:19:22.163153 (+1266341us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b54001090d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:22.163346 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":246144896}
W20260709 13:19:22.133116 18558 consensus_meta.cc:287] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.741s	user 0.008s	sys 0.003s
I20260709 13:19:22.152401 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:22.134676 18582 raft_consensus.cc:3055] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:22.059516 18545 consensus_queue.cc:237] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:22.174150 10942 raft_consensus.cc:1275] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 15. (term mismatch)
W20260709 13:19:22.175817 18625 log.cc:927] Time spent T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
W20260709 13:19:22.176291 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:22.181910 10942 pending_rounds.cc:85] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 14
I20260709 13:19:22.140560 18613 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:22.192272 10942 pending_rounds.cc:107] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 23.15
I20260709 13:19:22.192727 10942 raft_consensus.cc:2889] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:22.153945 10800 rpcz_store.cc:276] 0709 13:19:21.029081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:21.564016 (+534935us) service_pool.cc:224] Handling call
0709 13:19:21.564340 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:21.564737 (+   397us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:21.564772 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:21.565288 (+   516us) log.cc:844] Serialized 26 byte log entry
0709 13:19:21.923846 (+358558us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:21.923886 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:21.923952 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:22.153322 (+229370us) raft_consensus.cc:1685] finished
0709 13:19:22.153348 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:22.153501 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3620,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":3233,"thread_start_us":357327,"threads_started":1,"wal-append.queue_time_us":378063}
I20260709 13:19:22.128551 18379 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:22.154649 18614 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:22.134202 18576 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:22.178962 18582 raft_consensus.cc:740] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:22.190147 18352 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:22.211838 10796 raft_consensus.cc:1240] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:22.200915 18261 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:19:22.242867 18617 log.cc:927] Time spent T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:19:22.245720 18561 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:22.248522 11503 raft_consensus.cc:1240] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 24. Current term is 25. Ops: []
I20260709 13:19:22.186882 18558 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:22.204155 18549 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:22.198939 10875 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
 from {username='slave'} at 127.0.0.1:53438
I20260709 13:19:22.251061 10875 raft_consensus.cc:606] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Received request to transfer leadership to TS e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:22.251519 10875 raft_consensus.cc:614] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:19:22.246713 18509 consensus_meta.cc:287] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.823s	user 0.009s	sys 0.000s
I20260709 13:19:22.270255 10842 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:22.262135 18563 consensus_queue.cc:237] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 28.14, Last appended by leader: 14, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:22.272830 18463 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:19:22.273406 18463 raft_consensus.cc:697] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:22.274351 18463 consensus_queue.cc:237] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:22.277871 18636 consensus_queue.cc:1048] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:22.282121 10798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3379) took 1280 ms. Trace:
I20260709 13:19:22.282683 10798 rpcz_store.cc:276] 0709 13:19:21.001701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:21.231172 (+229471us) service_pool.cc:224] Handling call
0709 13:19:21.231559 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:21.803207 (+571648us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:22.281769 (+478562us) spinlock_profiling.cc:243] Waited 567 ms on lock 0x7b54003cd554. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:19:22.282034 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1304153856}
I20260709 13:19:22.198189 18613 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:22.280539 18590 raft_consensus.cc:3055] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:22.285924 18590 raft_consensus.cc:740] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:22.286831 18613 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:22.287369 18631 log.cc:927] Time spent T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 13:19:22.286955 18590 consensus_queue.cc:260] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:22.288403 18590 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:22.289726 18576 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:22.303615 18618 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:19:22.304251 18628 log.cc:927] Time spent T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:19:22.306892 18561 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:22.309309 18611 consensus_meta.cc:287] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.516s	user 0.010s	sys 0.002s
I20260709 13:19:22.309759 18611 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:22.302548 18588 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:22.316496 10797 raft_consensus.cc:1275] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
W20260709 13:19:22.317408 18610 consensus_meta.cc:287] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.539s	user 0.010s	sys 0.000s
I20260709 13:19:22.318087 18611 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:22.322139 18531 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:22.322746 18531 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:22.317346 10797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3381) took 1315 ms. Trace:
I20260709 13:19:22.323740 10797 rpcz_store.cc:276] 0709 13:19:21.002305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:21.420189 (+417884us) service_pool.cc:224] Handling call
0709 13:19:22.315811 (+895622us) spinlock_profiling.cc:243] Waited 1.88 ms on lock 0x7b5800002798. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd90a22 00007f9c86ca9c09 00007f9c85e03ee0 00007f9c85ecd0ab 00007f9c85ecd131 00007f9c85e8efc0 00007f9c85e79929 00007f9c85e793d2 00007f9c7f3f0bd4
0709 13:19:22.316090 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:22.317014 (+   924us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:22.317277 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4331904}
I20260709 13:19:22.324857 18531 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:22.327008 18558 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 27 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:22.327299 18429 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:22.327840 18429 consensus_queue.cc:1048] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:19:22.331569 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:22.332288 10792 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 22.
W20260709 13:19:22.368168 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 169907us
I20260709 13:19:22.369786 10766 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:22.370672 10799 raft_consensus.cc:1240] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 LEADER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:22.370811 10766 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:22.371796 18645 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:22.372285 18645 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:22.372467 10799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3382) took 1369 ms. Trace:
I20260709 13:19:22.372694 18645 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:22.372911 10799 rpcz_store.cc:276] 0709 13:19:21.002570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:21.431263 (+428693us) service_pool.cc:224] Handling call
0709 13:19:21.436432 (+  5169us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:22.371241 (+934809us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:22.372232 (+   991us) spinlock_profiling.cc:243] Waited 934 ms on lock 0x7b54002111d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:19:22.372407 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2148136192}
I20260709 13:19:22.373625 11503 raft_consensus.cc:1240] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 24. Current term is 25. Ops: []
I20260709 13:19:22.374873 18589 raft_consensus.cc:3055] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:22.376186 11503 raft_consensus.cc:1240] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:22.281483 18509 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:22.380663 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:22.297163 18609 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:22.383284 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:22.385285 10797 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 21.
I20260709 13:19:22.375360 18589 raft_consensus.cc:740] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:19:22.380354 18465 consensus_meta.cc:287] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.573s	user 0.005s	sys 0.005s
I20260709 13:19:22.317936 18610 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:22.389976 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:22.393594 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:22.365033 18531 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:22.380153 18448 consensus_queue.cc:1243] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer cb0428cc55984daba1ba63744a5102a2 log is divergent from this leader: its last log entry 21.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:19:22.404809 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:22.405570 10797 raft_consensus.cc:2393] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 23: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:22.398411 10799 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:19:22.399792 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:22.395818 18589 consensus_queue.cc:260] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:22.419255 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:22.421880 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:22.423898 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:22.424585 10794 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
I20260709 13:19:22.425660 11504 raft_consensus.cc:1275] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 13:19:22.426017 10842 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:22.426863 18548 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:22.427277 18548 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:22.427636 18548 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:22.428308 10842 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:22.430164 18619 raft_consensus.cc:2804] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:22.430713 18619 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:22.431192 18619 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:22.433043 18610 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:22.407838 18531 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:22.300374 18582 consensus_queue.cc:260] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:22.406029 18483 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:22.452612 18531 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:22.452358 18614 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:22.451004 10795 raft_consensus.cc:1240] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 29 LEADER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 28. Current term is 29. Ops: []
I20260709 13:19:22.451010 11504 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca"
dest_uuid: "12682eb70bc84c76879e04a8628ee586"
mode: GRACEFUL
new_leader_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:57818
I20260709 13:19:22.455096 11504 raft_consensus.cc:606] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 23 LEADER]: Received request to transfer leadership to TS cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:22.456365 10799 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 21.
I20260709 13:19:22.457567 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:22.461788 10794 raft_consensus.cc:2393] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 20: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:22.461524 10842 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:22.462109 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:22.443492 18589 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:22.449054 18261 raft_consensus.cc:2749] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:22.450980 18637 consensus_queue.cc:1059] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.024s
I20260709 13:19:22.483170 18583 raft_consensus.cc:3055] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 LEADER]: Stepping down as leader of term 24
W20260709 13:19:22.484629 18594 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 24 current term: 24 is higher.
I20260709 13:19:22.485683 10794 raft_consensus.cc:1240] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:22.483920 18645 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:22.485837 18618 raft_consensus.cc:2749] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:22.471912 18531 raft_consensus.cc:2804] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:22.492493 10797 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 22.
I20260709 13:19:22.483850 18583 raft_consensus.cc:740] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:22.495014 18531 raft_consensus.cc:697] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:19:22.496122 18585 log.cc:927] Time spent T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 1.104s	user 0.003s	sys 0.002s
W20260709 13:19:22.496990 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:22.501688 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:22.498657 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:22.504673 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:22.505517 11503 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 22.
I20260709 13:19:22.507491 11510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56416 (request call id 1155) took 1492 ms. Trace:
I20260709 13:19:22.507859 11510 rpcz_store.cc:276] 0709 13:19:21.014777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:21.116874 (+102097us) service_pool.cc:224] Handling call
0709 13:19:21.152098 (+ 35224us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:21.483541 (+331443us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:22.507252 (+1023711us) spinlock_profiling.cc:243] Waited 322 ms on lock 0x7b5400691c54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:19:22.507445 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":740205312}
I20260709 13:19:22.510361 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:22.509486 10841 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:19:22.476869 18548 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:22.513839 10875 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:53438
I20260709 13:19:22.515173 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:22.515231 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:22.516381 11511 raft_consensus.cc:2410] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 24 because replica has last-logged OpId of term: 23 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 17.
I20260709 13:19:22.518182 10841 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:22.519824 18545 raft_consensus.cc:993] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586: : Instructing follower cb0428cc55984daba1ba63744a5102a2 to start an election
I20260709 13:19:22.519810 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:22.520542 18545 raft_consensus.cc:1081] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 23 LEADER]: Signalling peer cb0428cc55984daba1ba63744a5102a2 to start an election
I20260709 13:19:22.520678 11506 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
I20260709 13:19:22.517308 18352 raft_consensus.cc:2804] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:22.523378 18352 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:22.523756 18352 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:21.935281 10950 rpcz_store.cc:276] 0709 13:19:20.501677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:20.514660 (+ 12983us) service_pool.cc:224] Handling call
0709 13:19:20.518853 (+  4193us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:20.519090 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:19:21.717795 (+1198705us) spinlock_profiling.cc:243] Waited 3.78 ms on lock 0x7b18002de6e0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c80a5d4bc 00007f9c7ddab48b 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc5455 00007f9c848ad3c7 00007f9c80ebf9a7 00007f9c80ecf0bf
0709 13:19:21.717951 (+   156us) log.cc:844] Serialized 14 byte log entry
0709 13:19:21.923516 (+205565us) log.cc:844] Serialized 14 byte log entry
0709 13:19:21.923719 (+   203us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:21.923765 (+    46us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:21.928946 (+  5181us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:21.929072 (+   126us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:21.929300 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":112482,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":1655,"raft.queue_time_us":6991,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":5167,"spinlock_wait_cycles":34217728,"thread_start_us":224260,"threads_started":2}
I20260709 13:19:22.538334 11478 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:22.538676 18548 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:22.524047 18614 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:22.539202 18624 consensus_queue.cc:1048] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:19:22.541910 18581 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:19:22.543145 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:22.543983 10841 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:22.544988 18532 raft_consensus.cc:2804] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:22.545403 18532 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:22.545737 18532 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:22.549829 18581 raft_consensus.cc:2749] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:22.541512 18619 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:22.551810 10867 raft_consensus.cc:1275] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:19:22.552166 18619 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:22.553076 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:22.553678 18532 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:22.554441 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:22.554940 18614 raft_consensus.cc:2749] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:22.555224 10872 raft_consensus.cc:2393] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 22: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:19:22.555835 18651 log.cc:927] Time spent T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
W20260709 13:19:22.557564 18650 log.cc:927] Time spent T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:19:22.558282 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:22.558671 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:22.558271 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:22.561426 18350 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:22.563632 18531 consensus_queue.cc:237] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:22.564929 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:22.565528 10799 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:22.567065 10872 raft_consensus.cc:1240] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 18. Current term is 19. Ops: [16.13-18.14]
I20260709 13:19:22.532537 18593 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:22.556173 10869 raft_consensus.cc:1240] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:22.570117 10796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2668) took 1569 ms. Trace:
W20260709 13:19:22.571004 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 629439us
I20260709 13:19:22.572257 10916 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:19:22.571599 18583 consensus_queue.cc:260] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:22.583848 10796 rpcz_store.cc:276] 0709 13:19:21.000784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:21.140076 (+139292us) service_pool.cc:224] Handling call
0709 13:19:21.165433 (+ 25357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:22.234933 (+1069500us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:22.569793 (+334860us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54000bc7d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:19:22.570034 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2400408320}
I20260709 13:19:22.616180 10799 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 23.
I20260709 13:19:22.616909 10792 raft_consensus.cc:1217] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Deduplicated request from leader. Original: 19.14->[21.15-21.15]   Dedup: 21.15->[]
I20260709 13:19:22.627234 10792 raft_consensus.cc:1240] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 13:19:22.628829 10792 raft_consensus.cc:1240] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:22.613543 10916 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:22.631073 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:22.631508 10869 raft_consensus.cc:1240] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 21. Current term is 22. Ops: [21.11-21.11]
I20260709 13:19:22.631299 10873 raft_consensus.cc:1240] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 26. Current term is 27. Ops: []
I20260709 13:19:22.634728 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:22.642853 10947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 3302) took 1938 ms. Trace:
I20260709 13:19:22.643463 10947 rpcz_store.cc:276] 0709 13:19:20.704021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:21.170369 (+466348us) service_pool.cc:224] Handling call
0709 13:19:21.413650 (+243281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:21.435306 (+ 21656us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:22.506522 (+1071216us) spinlock_profiling.cc:243] Waited 2.65 ms on lock 0x7b54000ab4d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:19:22.642541 (+136019us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b54000ab4d0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:22.642747 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":564541568}
I20260709 13:19:22.645794 11511 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 22.
I20260709 13:19:22.656875 10949 raft_consensus.cc:1240] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 22. Current term is 23. Ops: [22.14-22.14]
I20260709 13:19:22.655588 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:22.658917 10943 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 23.
I20260709 13:19:22.660235 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:22.660892 10945 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:22.661022 10948 raft_consensus.cc:1240] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:22.662825 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:22.662846 10943 raft_consensus.cc:1240] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 25 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 24. Current term is 25. Ops: []
I20260709 13:19:22.654654 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:22.671661 10875 raft_consensus.cc:606] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 LEADER]: Received request to transfer leadership to TS cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:22.673167 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:22.673337 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:22.674561 10798 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 21.
I20260709 13:19:22.679548 10946 raft_consensus.cc:1240] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:19:22.679643 10944 raft_consensus.cc:1275] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:19:22.669894 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:22.657763 18587 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:19:22.701565 18379 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:22.702066 18587 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:22.685575 10797 raft_consensus.cc:1240] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:22.686494 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:22.696452 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 83040.7us
I20260709 13:19:22.703743 10949 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 21.
I20260709 13:19:22.702569 18587 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:19:22.704758 10916 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:22.707024 18419 consensus_queue.cc:1059] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:19:22.708137 18419 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:22.708626 18419 raft_consensus.cc:697] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 20 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:22.709496 18419 consensus_queue.cc:237] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:22.711066 18592 raft_consensus.cc:3055] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:19:22.711499 18592 raft_consensus.cc:740] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:19:22.711632 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:22.712389 18592 consensus_queue.cc:260] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:22.713766 18592 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:22.697692 18594 raft_consensus.cc:2804] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:22.734239 10871 raft_consensus.cc:1240] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:22.695941 18611 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:22.695927 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:22.718618 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:22.743660 10945 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 21.
I20260709 13:19:22.743970 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:22.725885 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:22.657495 18552 raft_consensus.cc:3055] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 24 LEADER]: Stepping down as leader of term 24
W20260709 13:19:22.735031 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 73496.8us
I20260709 13:19:22.751231 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:22.752112 10799 raft_consensus.cc:2410] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 23 because replica has last-logged OpId of term: 21 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 16.
I20260709 13:19:22.662460 18395 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:22.754634 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:22.759426 10792 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 20.
I20260709 13:19:22.759752 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:22.644937 18417 consensus_queue.cc:1059] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.089s
I20260709 13:19:22.748198 10797 raft_consensus.cc:1240] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 13:19:22.748432 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:22.750150 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:22.766649 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:22.766625 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:22.756629 10943 raft_consensus.cc:1275] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:19:22.768658 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:22.756510 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:22.767956 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:22.760877 10871 raft_consensus.cc:2393] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 24: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:22.761595 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:22.770432 10869 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 20.
I20260709 13:19:22.770529 18465 consensus_queue.cc:1048] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:19:22.771071 10871 raft_consensus.cc:1240] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 23. Current term is 24. Ops: [23.18-23.18]
I20260709 13:19:22.771505 10869 raft_consensus.cc:1240] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 24. Current term is 25. Ops: []
I20260709 13:19:22.772465 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:22.773644 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:22.779044 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:22.681963 18583 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:22.730958 18486 raft_consensus.cc:3055] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:19:22.764539 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:22.747017 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:22.805677 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:22.780468 10949 raft_consensus.cc:1275] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:19:22.781021 10766 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:22.761770 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:22.806872 10872 raft_consensus.cc:2393] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 22: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:22.774703 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:22.807302 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:22.808037 10869 raft_consensus.cc:2393] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 22: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:22.784099 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 29 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:22.770968 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 29 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:22.784778 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:22.802651 10945 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
I20260709 13:19:22.733969 18594 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:22.820695 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:22.798596 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:22.822907 10799 raft_consensus.cc:2410] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 22 because replica has last-logged OpId of term: 20 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 9.
I20260709 13:19:22.801250 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:22.823429 10943 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:22.764267 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:22.815790 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:22.816975 10947 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 22.
I20260709 13:19:22.818005 18555 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:22.819100 10916 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:22.826866 11505 raft_consensus.cc:1275] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 13:19:22.796958 10796 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 19.
I20260709 13:19:22.750841 18552 raft_consensus.cc:740] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:22.758389 18589 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:22.758598 18530 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:22.828801 11511 raft_consensus.cc:1240] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 25 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 24. Current term is 25. Ops: []
I20260709 13:19:22.832991 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:22.833146 10868 raft_consensus.cc:1275] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 13:19:22.833597 18639 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:22.834544 10872 raft_consensus.cc:1275] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:19:22.780310 18590 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:22.834666 18532 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:22.914891 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:22.916404 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:19:22.916982 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:22.917187 10870 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:22.766870 18611 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:22.838305 18447 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:22.927405 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:22.830009 18594 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:22.839792 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:22.935922 10874 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 23: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:22.841702 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:22.842473 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:22.845359 10869 raft_consensus.cc:2393] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 22: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:22.851800 10946 raft_consensus.cc:1240] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
W20260709 13:19:22.847013 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:22.842718 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:22.921643 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:22.940858 10799 raft_consensus.cc:3060] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:22.915820 10948 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 21.
I20260709 13:19:22.927587 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:22.942044 11506 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:22.804039 18486 raft_consensus.cc:740] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:22.928390 10945 raft_consensus.cc:2393] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 23: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:22.928790 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:22.929546 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:22.931878 18552 consensus_queue.cc:260] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:22.957829 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:22.959029 11478 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:22.840948 18261 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:22.960592 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:22.966180 10947 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
I20260709 13:19:22.966599 10792 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:22.961823 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:22.782454 18395 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:22.912959 18588 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:22.852262 18548 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:19:22.969952 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 150933us
I20260709 13:19:22.980161 10916 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:22.805020 18432 consensus_queue.cc:1059] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.156s
I20260709 13:19:22.980146 10948 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 23: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:22.972247 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:22.970064 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:22.984200 10796 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:22.844218 18581 consensus_queue.cc:1059] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 15, Time since last communication: 0.125s
I20260709 13:19:22.924865 18530 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:22.974277 18552 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:22.975224 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:22.960590 18417 raft_consensus.cc:2749] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:19:22.989158 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:22.923429 18618 consensus_queue.cc:1059] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.056s
I20260709 13:19:23.016551 10946 raft_consensus.cc:3060] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:22.999650 10943 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 21.
I20260709 13:19:22.973404 11478 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:22.964567 18557 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:22.966243 18261 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:22.976379 18609 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:22.988304 18619 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:22.939056 18448 consensus_queue.cc:1048] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.054s
I20260709 13:19:23.009634 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:22.974903 18483 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:22.930083 18589 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:22.922307 18555 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:22.924134 18639 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:23.020839 10799 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 21.
I20260709 13:19:23.061265 10799 raft_consensus.cc:1240] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:23.061784 10796 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
I20260709 13:19:22.923756 18590 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:23.044394 18618 raft_consensus.cc:3055] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:19:22.996397 18622 raft_consensus.cc:2749] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:23.042855 18557 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:23.042898 18261 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:19:23.048218 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 74933.8us
I20260709 13:19:23.056447 10949 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:19:23.061133 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 81050.3us
I20260709 13:19:23.022220 10797 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 21.
I20260709 13:19:22.982693 18588 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:23.042925 18549 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:23.050652 18352 consensus_meta.cc:287] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.526s	user 0.011s	sys 0.001s
I20260709 13:19:23.027367 10792 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 22.
I20260709 13:19:22.991825 18548 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:23.077986 10946 raft_consensus.cc:2468] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 22.
I20260709 13:19:23.088388 10799 raft_consensus.cc:1240] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:23.091317 10915 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:23.102007 11505 raft_consensus.cc:1240] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 26. Current term is 27. Ops: []
I20260709 13:19:23.155586 18417 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:23.108175 10797 raft_consensus.cc:1275] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 14. (term mismatch)
I20260709 13:19:23.156293 10797 pending_rounds.cc:85] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 13
I20260709 13:19:23.157087 10797 pending_rounds.cc:107] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 18.14
I20260709 13:19:23.157374 10797 raft_consensus.cc:2889] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:23.060992 18545 raft_consensus.cc:2749] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:19:23.085232 18656 log.cc:927] Time spent T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
W20260709 13:19:23.144742 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 57379.1us
W20260709 13:19:23.115586 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:23.151629 11511 raft_consensus.cc:1240] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 23. Current term is 24. Ops: []
I20260709 13:19:23.065984 18618 raft_consensus.cc:740] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:23.042412 18465 consensus_queue.cc:1048] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.028s
W20260709 13:19:23.158317 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 67120us
I20260709 13:19:23.086115 18350 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.056857 18619 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:23.050907 18609 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:23.161595 10916 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:23.193136 10874 raft_consensus.cc:1275] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 13:19:23.194149 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 237026us
I20260709 13:19:23.018153 18486 consensus_queue.cc:260] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:23.060468 18483 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:23.159934 18633 raft_consensus.cc:3055] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:19:23.156601 18417 raft_consensus.cc:697] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:23.106527 18581 consensus_queue.cc:1059] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.077s
W20260709 13:19:23.166364 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 313758us
I20260709 13:19:23.144879 18549 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:23.198823 10949 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 23.
I20260709 13:19:23.197535 10767 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:23.200438 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:23.228044 10946 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 20.
I20260709 13:19:23.151310 18352 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:23.213478 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 51964.8us
I20260709 13:19:23.199615 10800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3390) took 2109 ms. Trace:
I20260709 13:19:23.223421 10766 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:19:23.207873 18659 log.cc:927] Time spent T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
I20260709 13:19:23.235060 18563 raft_consensus.cc:2764] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 13:19:23.195292 18419 consensus_queue.cc:1059] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.041s
I20260709 13:19:23.231197 10916 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:23.236755 18419 raft_consensus.cc:3055] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 24 LEADER]: Stepping down as leader of term 24
W20260709 13:19:23.229283 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:23.230957 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:23.237885 10767 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:23.237128 10870 raft_consensus.cc:2468] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 21.
I20260709 13:19:23.238560 10916 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:23.208184 18463 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.200114 18636 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:19:23.231997 10766 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:23.236107 18352 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:23.191879 18350 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:23.212702 18633 raft_consensus.cc:740] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:23.220707 18486 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:23.230834 10800 rpcz_store.cc:276] 0709 13:19:21.089743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:22.193552 (+1103809us) service_pool.cc:224] Handling call
0709 13:19:22.205658 (+ 12106us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:22.205967 (+   309us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 13:19:22.239206 (+ 33239us) log.cc:844] Serialized 14 byte log entry
0709 13:19:22.696723 (+457517us) spinlock_profiling.cc:243] Waited 7.95 ms on lock 0x7b18003eaac0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a5523a 00007f9c80ebfa2a
0709 13:19:22.955130 (+258407us) log.cc:844] Serialized 14 byte log entry
0709 13:19:23.198350 (+243220us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:23.198421 (+    71us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:19:23.198490 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:23.199360 (+   870us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:23.199540 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":33516,"prepare.run_cpu_time_us":134,"prepare.run_wall_time_us":15248,"raft.queue_time_us":139875,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":8314,"spinlock_wait_cycles":74013824,"thread_start_us":475665,"threads_started":2,"wal-append.queue_time_us":827076}
W20260709 13:19:23.237977 18660 log.cc:927] Time spent T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.304s	user 0.000s	sys 0.001s
I20260709 13:19:23.256114 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:23.230468 18400 raft_consensus.cc:2749] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:23.204082 18587 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:23.237120 18419 raft_consensus.cc:740] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:23.236132 18549 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:23.242620 18593 raft_consensus.cc:2749] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:19:23.265753 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:23.248612 18532 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.245028 18395 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:23.275166 11511 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:23.228896 18664 consensus_queue.cc:1048] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.038s
W20260709 13:19:23.283509 18662 log.cc:927] Time spent T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:19:23.284605 18483 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:23.286054 18666 log.cc:927] Time spent T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:19:23.285883 18670 consensus_queue.cc:1059] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 26.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:19:23.286626 18400 raft_consensus.cc:2749] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:23.286690 18678 raft_consensus.cc:2749] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:23.289631 18609 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:23.290624 18619 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:23.290832 18661 log.cc:927] Time spent T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 13:19:23.293285 18567 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:23.293537 18594 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:23.294719 18557 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:23.304600 10943 raft_consensus.cc:1240] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 26. Current term is 27. Ops: []
W20260709 13:19:23.307050 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 61159.6us
I20260709 13:19:23.309259 18419 consensus_queue.cc:260] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:23.306030 18589 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:23.308004 10945 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca"
dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:36714
I20260709 13:19:23.355294 10945 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:23.355685 10945 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:19:23.330979 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 149122us
I20260709 13:19:23.329089 11506 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
I20260709 13:19:23.274721 18530 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:23.252056 18463 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:23.321982 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:23.359251 11505 raft_consensus.cc:2393] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 27: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
W20260709 13:19:23.361454 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:23.363737 10841 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:23.366489 10841 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:23.366485 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 127976us
I20260709 13:19:23.241034 18618 consensus_queue.cc:260] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:23.370846 10841 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:23.303442 18532 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:23.336138 18555 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:23.356238 11511 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 23.
W20260709 13:19:23.375032 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 143111us
I20260709 13:19:23.381763 10916 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:23.368649 11477 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:23.349265 18670 raft_consensus.cc:2804] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:19:23.391183 10767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:23.392817 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:19:23.393110 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:23.393626 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:23.393536 10793 raft_consensus.cc:2376] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 21. Re-sending same reply.
I20260709 13:19:23.272917 18417 consensus_queue.cc:237] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:23.398528 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:23.399128 10793 raft_consensus.cc:2376] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 19. Re-sending same reply.
I20260709 13:19:23.396284 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:23.400069 10792 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 21.
I20260709 13:19:23.368888 18619 raft_consensus.cc:2749] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 13:19:23.373241 18549 raft_consensus.cc:2749] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:23.367880 18588 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:23.429206 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:23.380541 18548 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:23.430258 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:19:23.388711 18657 log.cc:927] Time spent T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.229s	user 0.000s	sys 0.001s
I20260709 13:19:23.386413 18419 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:23.383965 18352 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:23.383832 18676 raft_consensus.cc:2749] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:19:23.421054 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:23.387194 18530 raft_consensus.cc:3055] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:19:23.433003 11511 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 24.
I20260709 13:19:23.416244 10945 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:23.436723 11507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56416 (request call id 1164) took 1058 ms. Trace:
I20260709 13:19:23.435267 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:23.442458 10798 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:23.320264 18633 consensus_queue.cc:260] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:23.373489 18672 consensus_queue.cc:1048] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.071s
I20260709 13:19:23.430485 18618 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:23.431635 18589 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.442484 10946 raft_consensus.cc:1240] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 23. Current term is 24. Ops: []
I20260709 13:19:23.406379 18545 raft_consensus.cc:2804] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:23.368230 18609 consensus_queue.cc:1059] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 28.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.038s
I20260709 13:19:23.461923 18545 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:23.462282 18609 raft_consensus.cc:3055] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 13:19:23.462378 18545 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:23.462776 18609 raft_consensus.cc:740] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:23.463800 18582 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:23.463691 18609 consensus_queue.cc:260] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 28.14, Last appended by leader: 14, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:23.465093 18609 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 28 FOLLOWER]: Advancing to term 29
W20260709 13:19:23.447508 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 65832.9us
I20260709 13:19:23.467331 10916 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:23.468518 18681 raft_consensus.cc:2749] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:19:23.469156 18681 consensus_queue.cc:1048] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:19:23.469728 10916 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:23.470273 18674 consensus_queue.cc:1059] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.195s
I20260709 13:19:23.432652 18555 consensus_queue.cc:1059] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.038s
I20260709 13:19:23.471298 18683 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.471807 18683 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:23.397030 18678 consensus_queue.cc:1048] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.059s
I20260709 13:19:23.477012 10948 raft_consensus.cc:1275] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 13:19:23.478242 18261 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:23.479266 18592 consensus_meta.cc:287] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.765s	user 0.013s	sys 0.000s
I20260709 13:19:23.480523 18674 raft_consensus.cc:3055] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:19:23.480981 18674 raft_consensus.cc:740] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:19:23.483418 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:23.397465 18593 consensus_queue.cc:1048] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
W20260709 13:19:23.487702 18667 log.cc:927] Time spent T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.263s	user 0.001s	sys 0.000s
I20260709 13:19:23.437150 18588 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.489756 18588 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:23.443279 18352 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:23.457159 18350 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:23.457554 18463 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:23.439987 18549 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.436195 18447 raft_consensus.cc:2749] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:23.443873 18530 raft_consensus.cc:740] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:23.459707 18594 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:23.460501 18663 log.cc:927] Time spent T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.223s	user 0.001s	sys 0.000s
I20260709 13:19:23.460939 18680 raft_consensus.cc:2749] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:23.461249 18532 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:23.440258 18548 consensus_queue.cc:1048] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 13:19:23.481848 18674 consensus_queue.cc:260] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:23.455597 10945 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:23.491845 10946 raft_consensus.cc:1275] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:19:23.492507 10916 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:23.474524 18683 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:23.428980 18400 consensus_queue.cc:1059] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.026s
I20260709 13:19:23.470525 18681 raft_consensus.cc:2749] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:23.520777 18352 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:23.522202 18530 consensus_queue.cc:260] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:23.523972 18530 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:19:23.527489 18583 consensus_meta.cc:287] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.707s	user 0.011s	sys 0.002s
I20260709 13:19:23.528352 10801 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 25.
I20260709 13:19:23.502285 18633 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:23.504014 18587 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:23.545564 18509 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:23.512111 18350 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:23.547511 18350 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:23.494722 18589 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:23.499979 18549 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:23.553188 18674 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:23.555660 18680 raft_consensus.cc:2749] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:19:23.396807 18563 consensus_queue.cc:1175] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" responder_term: 23 status { last_received { term: 20 index: 13 } last_committed_idx: 13 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 22. Current term is 23. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:19:23.557869 18400 raft_consensus.cc:3055] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:19:23.558431 18400 raft_consensus.cc:740] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:23.559554 18400 consensus_queue.cc:260] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.19, Last appended by leader: 19, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:23.496866 10798 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 22.
W20260709 13:19:23.564563 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 51126us
I20260709 13:19:23.566004 10916 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:23.444456 11507 rpcz_store.cc:276] 0709 13:19:22.378744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:22.378951 (+   207us) service_pool.cc:224] Handling call
0709 13:19:22.388690 (+  9739us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:22.388904 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:19:22.404798 (+ 15894us) log.cc:844] Serialized 14 byte log entry
0709 13:19:23.372840 (+968042us) spinlock_profiling.cc:243] Waited 2.13 ms on lock 0x7b18004c6ae0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a5523a 00007f9c80ebfa2a
0709 13:19:23.435884 (+ 63044us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:23.435941 (+    57us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:19:23.435986 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:23.436481 (+   495us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:23.436649 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":605,"prepare.queue_time_us":15832,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":42,"spinlock_wait_cycles":4903168,"thread_start_us":62194,"threads_started":1}
W20260709 13:19:23.498624 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 104693us
I20260709 13:19:23.445401 18175 consensus_queue.cc:1048] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.054s
I20260709 13:19:23.480566 18261 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:23.568143 11478 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:19:23.574599 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:23.576335 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 187369us
I20260709 13:19:23.578073 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:23.581012 10941 raft_consensus.cc:2368] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 20. Current term is 21.
I20260709 13:19:23.584803 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:23.585114 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:23.586331 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:23.587323 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 205276us
I20260709 13:19:23.580675 18587 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:19:23.599180 10945 raft_consensus.cc:1275] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:19:23.595324 10950 raft_consensus.cc:1217] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Deduplicated request from leader. Original: 17.10->[18.11-22.13]   Dedup: 18.11->[20.12-22.13]
I20260709 13:19:23.591607 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:23.593755 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:23.601549 10950 raft_consensus.cc:1240] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 22. Current term is 23. Ops: [18.11-22.13]
I20260709 13:19:23.602094 10944 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
I20260709 13:19:23.602489 10766 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:23.603032 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:23.603267 18379 raft_consensus.cc:2749] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:23.604115 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:23.605306 10944 raft_consensus.cc:3055] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:23.605202 10766 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:23.605692 10944 raft_consensus.cc:740] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:23.606040 18379 raft_consensus.cc:2749] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:23.600358 18261 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.571456 18680 consensus_queue.cc:1059] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 13:19:23.609279 10941 raft_consensus.cc:2393] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 27: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:23.610544 10941 raft_consensus.cc:1275] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 29 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 28 index: 14. Preceding OpId from leader: term: 29 index: 15. (index mismatch)
I20260709 13:19:23.607674 18261 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:23.585203 18588 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:23.612453 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 44387us
I20260709 13:19:23.616282 11478 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:23.619177 10841 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:23.622123 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:23.618954 18549 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:23.627441 18395 raft_consensus.cc:3055] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:19:23.628059 18395 raft_consensus.cc:740] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:23.628966 18395 consensus_queue.cc:260] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:23.630280 18395 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:23.631374 18350 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:23.631937 18174 raft_consensus.cc:3055] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:23.632398 18174 raft_consensus.cc:740] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:23.633296 18174 consensus_queue.cc:260] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:23.634418 18174 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:23.636129 18611 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:23.639122 18645 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:23.639976 18691 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:23.643642 10795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3389) took 2554 ms. Trace:
I20260709 13:19:23.644083 10795 rpcz_store.cc:276] 0709 13:19:21.089466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:22.169701 (+1080235us) service_pool.cc:224] Handling call
0709 13:19:22.190376 (+ 20675us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:22.454807 (+264431us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:23.643376 (+1188569us) spinlock_profiling.cc:243] Waited 260 ms on lock 0x7b54000d9654. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:19:23.643580 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":598915840}
I20260709 13:19:23.645356 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:23.645591 10798 raft_consensus.cc:1275] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:19:23.646692 18400 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:19:23.646998 10800 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 20.
I20260709 13:19:23.647944 18558 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.648504 18558 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:23.646848 18587 raft_consensus.cc:697] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 25 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:23.649713 11512 raft_consensus.cc:1275] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 15. (term mismatch)
I20260709 13:19:23.650051 11512 pending_rounds.cc:85] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586: Aborting all ops after (but not including) 14
I20260709 13:19:23.650266 11512 pending_rounds.cc:107] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586: Aborting uncommitted NO_OP operation due to leader change: 21.15
I20260709 13:19:23.650641 11512 raft_consensus.cc:2889] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:23.650290 18587 consensus_queue.cc:237] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 23.10, Last appended by leader: 10, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:23.651147 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:23.652434 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:23.653079 18637 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.653820 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:23.653662 18637 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:23.654817 18693 raft_consensus.cc:2804] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:23.655184 18261 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:23.610638 18545 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:23.657611 18545 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:23.658982 18175 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.660095 18639 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:23.659822 18175 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:23.663492 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 44448.9us
I20260709 13:19:23.663810 10916 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:23.664309 10841 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:23.665963 10841 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:23.667035 18692 raft_consensus.cc:2749] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:19:23.667770 18692 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:23.669809 18637 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:23.671180 11512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56410 (request call id 1197) took 2109 ms. Trace:
I20260709 13:19:23.671547 10947 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 21.
I20260709 13:19:23.627556 18589 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:23.662113 18175 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:23.674525 10841 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:23.587538 18670 raft_consensus.cc:697] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:23.606472 10944 consensus_queue.cc:260] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:23.680452 18590 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:23.681175 18603 raft_consensus.cc:3055] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 25 LEADER]: Stepping down as leader of term 25
W20260709 13:19:23.683198 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 78069.4us
I20260709 13:19:23.683507 10801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39442 (request call id 1074) took 1012 ms. Trace:
I20260709 13:19:23.696130 10801 rpcz_store.cc:276] 0709 13:19:22.670911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:22.671140 (+   229us) service_pool.cc:224] Handling call
0709 13:19:23.683331 (+1012191us) spinlock_profiling.cc:243] Waited 785 ms on lock 0x7b54000e8b54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:23.683452 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1805382528}
I20260709 13:19:23.653190 10793 raft_consensus.cc:2393] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 20: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:23.624986 18681 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.697082 10766 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:23.687348 10767 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:23.646634 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: true dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:19:23.684145 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:23.699332 10795 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:23.671578 11512 rpcz_store.cc:276] 0709 13:19:21.561435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:21.561631 (+   196us) service_pool.cc:224] Handling call
0709 13:19:21.562058 (+   427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:23.651116 (+2089058us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:23.652734 (+  1618us) spinlock_profiling.cc:243] Waited 2.09 s on lock 0x7b54006dd154. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:19:23.671094 (+ 18360us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4800342912}
I20260709 13:19:23.700795 10841 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:23.701858 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:23.708233 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:23.708350 10916 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:23.710234 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:23.711092 10793 raft_consensus.cc:2393] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 18: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
W20260709 13:19:23.712847 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 96658.6us
W20260709 13:19:23.714592 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:23.715355 18582 raft_consensus.cc:2749] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:19:23.715632 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:23.717973 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:23.718632 10945 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:23.714057 11478 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:23.720280 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:23.721709 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:23.722373 10867 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 23.
I20260709 13:19:23.724705 10795 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 24.
I20260709 13:19:23.726567 10767 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:23.727392 18588 consensus_queue.cc:1048] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:23.727630 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:23.728313 10875 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 21.
I20260709 13:19:23.729321 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:23.729916 10875 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 22.
W20260709 13:19:23.730677 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:23.730912 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:23.731314 10876 raft_consensus.cc:1275] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 29 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 28 index: 14. Preceding OpId from leader: term: 29 index: 15. (index mismatch)
I20260709 13:19:23.731879 18590 raft_consensus.cc:2749] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:19:23.709584 18350 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:19:23.732512 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:23.732930 18350 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.711795 18681 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:23.733196 10868 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 19.
I20260709 13:19:23.733430 18350 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:23.733990 18676 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.734480 18676 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:23.750089 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:23.747740 10799 raft_consensus.cc:1275] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 13:19:23.739817 10801 raft_consensus.cc:1275] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 11. (term mismatch)
I20260709 13:19:23.738564 10945 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 21.
I20260709 13:19:23.756908 10801 pending_rounds.cc:85] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting all ops after (but not including) 10
I20260709 13:19:23.757166 10801 pending_rounds.cc:107] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting uncommitted NO_OP operation due to leader change: 21.11
I20260709 13:19:23.757457 10801 raft_consensus.cc:2889] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:23.730721 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:23.752611 10869 raft_consensus.cc:1275] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:19:23.741246 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:23.743522 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:23.740859 10794 raft_consensus.cc:1275] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 13:19:23.759454 10871 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:53438
I20260709 13:19:23.745335 10793 raft_consensus.cc:1275] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:19:23.760803 10767 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:23.698484 18603 raft_consensus.cc:740] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:23.762408 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:23.773066 10767 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:23.773324 10945 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 23.
I20260709 13:19:23.774768 10916 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:23.776324 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 62329.8us
I20260709 13:19:23.776856 10795 raft_consensus.cc:1240] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 25. Current term is 26. Ops: [23.12-25.13]
I20260709 13:19:23.768016 10841 leader_election.cc:400] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 election: Vote denied by peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553) with higher term. Message: Invalid argument: T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 20. Current term is 21.
I20260709 13:19:23.778365 10841 leader_election.cc:403] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:19:23.778272 10916 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:23.780354 10767 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:23.780658 10766 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:23.786556 10916 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:23.706956 18693 raft_consensus.cc:697] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 21 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:23.713338 18609 raft_consensus.cc:2804] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:23.712483 18637 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:23.748478 18261 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:23.788918 10842 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:23.735649 18681 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:23.702612 18555 consensus_queue.cc:1059] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.077s
I20260709 13:19:23.802348 10916 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:23.735252 18509 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.698038 18549 consensus_queue.cc:1059] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.079s
I20260709 13:19:23.797255 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:23.789147 18395 raft_consensus.cc:2749] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:19:23.783902 10767 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:23.811522 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:23.782708 18639 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:23.775698 18175 raft_consensus.cc:2804] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:23.736475 18676 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:23.817924 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:23.836565 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 52774.4us
I20260709 13:19:23.843298 10916 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:23.787739 18618 raft_consensus.cc:2749] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:23.819736 10798 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
I20260709 13:19:23.702836 18589 consensus_queue.cc:1048] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.079s
I20260709 13:19:23.795769 18680 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:23.715304 18688 consensus_queue.cc:1059] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 15, Time since last communication: 0.096s
I20260709 13:19:23.809228 18699 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:23.789499 18582 raft_consensus.cc:2804] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:23.836355 18614 raft_consensus.cc:2749] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553) with higher term. Message: Invalid argument: T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 20. Current term is 21.
I20260709 13:19:23.808272 18622 raft_consensus.cc:2749] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:23.803110 18674 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:19:23.848845 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:23.932090 10945 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 21.
W20260709 13:19:23.930935 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 142418us
I20260709 13:19:23.810528 18261 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:23.781654 18633 consensus_queue.cc:1059] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.055s
I20260709 13:19:23.872689 10916 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:23.908565 18690 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.748077 18694 consensus_queue.cc:1048] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.063s
I20260709 13:19:23.811352 18637 raft_consensus.cc:697] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:19:23.956738 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 167880us
I20260709 13:19:23.842509 10799 raft_consensus.cc:3060] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:23.843137 18448 raft_consensus.cc:2804] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:23.842343 18639 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:23.961244 18639 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:23.810827 18670 consensus_queue.cc:237] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:23.971159 10948 raft_consensus.cc:2410] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 27 because replica has last-logged OpId of term: 26 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 18.
I20260709 13:19:23.934114 10766 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:23.972975 10766 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:23.846585 18592 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:19:23.974201 18618 raft_consensus.cc:2749] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:19:23.975184 18618 raft_consensus.cc:2804] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:23.975548 18618 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:23.975876 18618 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:19:23.924610 18657 log.cc:927] Time spent T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 13:19:23.956588 18699 raft_consensus.cc:697] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 22 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:23.982357 18699 consensus_queue.cc:237] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:23.954290 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 81716.1us
I20260709 13:19:23.920471 18175 raft_consensus.cc:697] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 19 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:23.993772 18175 consensus_queue.cc:237] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:23.994974 18448 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:23.995442 18448 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:23.923477 18395 raft_consensus.cc:2749] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:23.929108 18582 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:23.958770 10842 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:19:24.000094 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 159889us
I20260709 13:19:23.958911 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 26 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:19:23.959532 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:24.002665 10798 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 25.
I20260709 13:19:23.839483 18509 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:23.848134 18463 consensus_queue.cc:1059] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.091s
I20260709 13:19:24.008414 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:24.013461 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:24.014369 11503 raft_consensus.cc:1813] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Handling vote request from an unknown peer cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:23.930269 18603 consensus_queue.cc:260] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 25.13, Last appended by leader: 13, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:24.045233 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:23.938350 18350 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:23.957103 18680 raft_consensus.cc:2804] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:19:24.047068 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 279240us
I20260709 13:19:23.960094 18549 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 13:19:24.040427 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:24.045897 10799 raft_consensus.cc:2468] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 23.
I20260709 13:19:24.046113 11507 raft_consensus.cc:2393] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 21: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:24.044221 10916 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:24.055601 18587 raft_consensus.cc:2804] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:19:23.939148 18261 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:24.015151 11503 raft_consensus.cc:2410] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 22 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:19:24.056353 18587 raft_consensus.cc:697] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 24 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:24.057315 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:24.057528 18622 consensus_queue.cc:1048] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:19:23.931712 18674 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:24.059020 18674 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:24.059314 18395 consensus_queue.cc:1048] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 13:19:24.052224 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 45028.3us
I20260709 13:19:24.058902 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:24.060860 11503 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 22.
I20260709 13:19:24.060798 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:24.061522 11506 raft_consensus.cc:1275] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 13:19:24.061163 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:24.061836 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:24.062312 11510 raft_consensus.cc:2393] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 20: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:24.061481 10875 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 22.
I20260709 13:19:24.053618 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:24.055558 10915 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:24.067644 10766 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:24.069307 11504 raft_consensus.cc:1275] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:19:24.060120 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:24.071136 10841 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:24.072665 18633 consensus_queue.cc:1059] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 22.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:19:24.059053 18417 consensus_queue.cc:1048] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 13:19:24.075096 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:24.075171 18680 raft_consensus.cc:697] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 21 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:24.076784 18680 consensus_queue.cc:237] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:24.079362 18603 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:19:24.081259 10915 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:24.083294 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: true dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:24.083914 11511 raft_consensus.cc:3055] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:19:24.084227 11511 raft_consensus.cc:740] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:24.084324 18350 consensus_queue.cc:1048] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:19:24.085014 11511 consensus_queue.cc:260] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:24.085935 11511 raft_consensus.cc:3060] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:24.057350 18587 consensus_queue.cc:237] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:24.086925 18687 consensus_queue.cc:1048] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:19:24.004935 18592 raft_consensus.cc:3055] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:19:24.091079 10915 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:24.092084 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:24.095950 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:24.096792 10766 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:24.008289 18619 raft_consensus.cc:2804] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:24.007496 18582 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:24.010035 18486 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:23.999747 18645 consensus_queue.cc:1048] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.093s
I20260709 13:19:24.010138 18688 consensus_queue.cc:1059] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.092s
I20260709 13:19:24.016646 18400 consensus_queue.cc:1048] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.087s
W20260709 13:19:24.145560 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:24.145740 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 54767.4us
I20260709 13:19:24.100704 18596 raft_consensus.cc:2749] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:23.956950 18690 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:24.119130 10871 raft_consensus.cc:606] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Received request to transfer leadership to TS cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:24.127437 11511 raft_consensus.cc:2468] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 24.
I20260709 13:19:24.095357 18592 raft_consensus.cc:740] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:24.102267 18589 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:24.155860 10871 raft_consensus.cc:614] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:24.096583 18549 raft_consensus.cc:697] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:24.118042 18530 raft_consensus.cc:2749] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:19:24.160440 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 66047.3us
I20260709 13:19:23.957372 18693 consensus_queue.cc:237] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:24.115242 18417 raft_consensus.cc:2749] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:24.152365 18609 raft_consensus.cc:697] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:24.171347 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:24.147231 18622 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:24.149976 18664 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:24.105408 18419 consensus_queue.cc:1048] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.056s
I20260709 13:19:24.147531 18687 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:24.118861 18486 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:24.166553 18352 raft_consensus.cc:2804] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:24.167802 18589 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:24.204632 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:24.209482 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:24.170977 18614 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:24.192490 18688 consensus_queue.cc:1243] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Peer e01ecd008edd4aa2b6b2b47efbc2ea87 log is divergent from this leader: its last log entry 18.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:19:24.191248 18532 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:19:24.229733 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:24.229763 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 133079us
I20260709 13:19:24.203671 18352 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:24.203262 18687 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:24.239360 18702 log.cc:927] Time spent T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
I20260709 13:19:24.236835 18695 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:24.235958 18609 consensus_queue.cc:237] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:24.240163 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 26 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:24.210947 18589 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:19:24.242812 18649 log.cc:927] Time spent T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 1.761s	user 0.010s	sys 0.005s
I20260709 13:19:24.243152 18352 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:24.243332 18704 log.cc:927] Time spent T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 13:19:24.181892 18637 consensus_queue.cc:237] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:24.244503 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:24.201639 18622 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:24.204569 18639 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:24.215339 18530 consensus_queue.cc:1048] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.036s
W20260709 13:19:24.250496 18692 consensus_meta.cc:287] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.582s	user 0.007s	sys 0.004s
I20260709 13:19:24.243624 18706 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:24.223186 18614 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:24.246192 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:24.279271 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:24.281699 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 117225us
I20260709 13:19:24.284689 18619 raft_consensus.cc:697] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:24.286151 18670 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:24.287204 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 109480us
I20260709 13:19:24.288005 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:24.288331 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:24.307108 18509 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:24.232343 18532 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:24.181318 18596 consensus_queue.cc:1048] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.094s
I20260709 13:19:24.268128 18175 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:24.369740 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:24.382189 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:24.423256 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:24.423823 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:24.425113 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:24.425625 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:24.426173 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:24.426169 10874 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:19:24.429157 18722 log.cc:927] Time spent T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:19:24.431025 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:24.430280 10916 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:24.431734 10945 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 23.
I20260709 13:19:24.267943 18618 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:24.433393 10949 raft_consensus.cc:2410] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 23 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 13:19:24.308652 18717 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:24.391079 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:24.463132 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:24.407322 18619 consensus_queue.cc:237] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:24.465158 18618 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:24.409041 18709 log.cc:927] Time spent T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.179s	user 0.000s	sys 0.000s
W20260709 13:19:24.409945 18714 log.cc:927] Time spent T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.170s	user 0.000s	sys 0.000s
W20260709 13:19:24.410830 18718 log.cc:927] Time spent T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
W20260709 13:19:24.415148 18707 log.cc:927] Time spent T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.252s	user 0.000s	sys 0.000s
I20260709 13:19:24.415449 18175 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:24.416669 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:24.417309 18486 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:24.375964 18716 log.cc:927] Time spent T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 13:19:24.419071 18610 raft_consensus.cc:3055] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:19:24.419564 11509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56416 (request call id 1166) took 2033 ms. Trace:
I20260709 13:19:24.420243 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:24.400022 18670 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:24.422619 18690 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:24.443070 18639 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:24.376869 18706 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:24.448292 18687 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:24.250802 18549 consensus_queue.cc:237] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.10, Last appended by leader: 10, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:24.449270 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:24.449340 18582 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:24.450547 18622 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:24.452433 18614 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:24.384302 10841 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
W20260709 13:19:24.497757 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 113583us
I20260709 13:19:24.457396 18530 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:24.502264 18530 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:24.505755 18734 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:24.506600 18734 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:24.509379 18548 raft_consensus.cc:3055] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:24.509958 18548 raft_consensus.cc:740] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:24.486971 18557 raft_consensus.cc:2749] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:19:24.510998 18548 consensus_queue.cc:260] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:24.512446 18548 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:24.513335 18422 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:24.518082 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:24.312589 18692 raft_consensus.cc:2749] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:24.520812 18582 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:24.394392 18350 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:24.458739 18731 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:24.523281 18731 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:24.525259 18614 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:24.525856 18614 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:24.526270 18731 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:24.273101 18695 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:24.397193 10876 raft_consensus.cc:1240] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 13:19:24.536698 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:24.537606 10801 raft_consensus.cc:2410] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 23 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 13:19:24.537968 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 2706) took 1798 ms. Trace:
I20260709 13:19:24.538117 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:24.542577 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:24.476056 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:24.482985 11503 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 22.
I20260709 13:19:24.487200 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:24.480908 11509 rpcz_store.cc:276] 0709 13:19:22.385804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:22.385938 (+   134us) service_pool.cc:224] Handling call
0709 13:19:22.386342 (+   404us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:22.386872 (+   530us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:19:22.386898 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:22.387372 (+   474us) log.cc:844] Serialized 26 byte log entry
0709 13:19:22.404042 (+ 16670us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:22.404084 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:22.404155 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:24.205329 (+1801174us) spinlock_profiling.cc:243] Waited 4.84 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:19:24.419236 (+213907us) raft_consensus.cc:1685] finished
0709 13:19:24.419269 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:24.419497 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1462,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":2863,"spinlock_wait_cycles":11131136,"thread_start_us":16274,"threads_started":1,"wal-append.queue_time_us":91217}
I20260709 13:19:24.492605 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:24.545169 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:24.545621 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:24.546036 10876 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 24.
I20260709 13:19:24.546808 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:24.547410 10945 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 23.
W20260709 13:19:24.480293 18691 consensus_meta.cc:287] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.837s	user 0.008s	sys 0.004s
I20260709 13:19:24.547953 10871 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:24.548556 18691 raft_consensus.cc:2749] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:24.495536 10942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 2912) took 1840 ms. Trace:
I20260709 13:19:24.496620 10868 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:19:24.462857 18717 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:24.550084 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:24.528400 18614 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:24.528859 18733 raft_consensus.cc:2804] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:19:24.530995 18692 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:24.532483 18706 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:24.522387 18350 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:24.534667 18530 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:24.536129 18175 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:24.454725 10874 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 27.
I20260709 13:19:24.538437 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:24.585332 10868 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 27.
I20260709 13:19:24.586483 10874 raft_consensus.cc:1240] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 25. Current term is 26. Ops: [25.13-25.13]
I20260709 13:19:24.540820 18695 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:24.461824 18261 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:24.548174 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 27 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:24.596328 10945 raft_consensus.cc:2410] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 27 because replica has last-logged OpId of term: 25 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 11.
W20260709 13:19:24.555555 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:24.549492 10942 rpcz_store.cc:276] 0709 13:19:22.655257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:22.656495 (+  1238us) service_pool.cc:224] Handling call
0709 13:19:22.656936 (+   441us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:22.698350 (+ 41414us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:22.698401 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:22.699066 (+   665us) log.cc:844] Serialized 26 byte log entry
0709 13:19:22.933955 (+234889us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:19:23.248596 (+314641us) log.cc:844] Serialized 14 byte log entry
0709 13:19:24.484077 (+1235481us) spinlock_profiling.cc:243] Waited 40.9 ms on lock 0x7b540015cfa0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80e30733 00007f9c80eb5dbf 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c
0709 13:19:24.484134 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:24.484236 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:24.484307 (+    71us) raft_consensus.cc:1685] finished
0709 13:19:24.495312 (+ 11005us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:24.495488 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75842,"prepare.queue_time_us":395956,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":58,"raft.queue_time_us":182206,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":35236,"spinlock_wait_cycles":94173568,"thread_start_us":659786,"threads_started":3,"wal-append.queue_time_us":446012,"wal-append.run_cpu_time_us":4415,"wal-append.run_wall_time_us":1325548}
I20260709 13:19:24.548050 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:24.597991 10867 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 23.
I20260709 13:19:24.599103 10871 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 22.
I20260709 13:19:24.538349 10873 rpcz_store.cc:276] 0709 13:19:22.739983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:22.798691 (+ 58708us) service_pool.cc:224] Handling call
0709 13:19:22.820293 (+ 21602us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:22.853868 (+ 33575us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:19:23.488784 (+634916us) spinlock_profiling.cc:243] Waited 2.53 ms on lock 0x7b1800170920. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7dd42c0a 00007f9c87482219 00007f9c7ddcaa40 00007f9c7ddc54e4 00007f9c7ddc5455 00007f9c848ad3c7 00007f9c80ebf9a7 00007f9c80ecf0bf 00007f9c80ecf038
0709 13:19:23.689308 (+200524us) log.cc:844] Serialized 14 byte log entry
0709 13:19:23.954581 (+265273us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:23.955524 (+   943us) log.cc:844] Serialized 26 byte log entry
0709 13:19:23.955703 (+   179us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:19:23.955999 (+   296us) log.cc:844] Serialized 14 byte log entry
0709 13:19:23.972157 (+ 16158us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:23.972243 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:24.537773 (+565530us) raft_consensus.cc:1685] finished
0709 13:19:24.537793 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:24.537915 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34990,"prepare.queue_time_us":575265,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":71,"raft.queue_time_us":53025,"raft.run_cpu_time_us":295,"raft.run_wall_time_us":72397,"spinlock_wait_cycles":5853312,"thread_start_us":363020,"threads_started":2,"wal-append.queue_time_us":449051}
I20260709 13:19:24.554231 10766 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:24.480710 18610 raft_consensus.cc:740] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:24.578150 11509 raft_consensus.cc:1275] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 13:19:24.470254 18674 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:24.604591 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:24.606447 10942 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:19:24.581874 18717 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:24.583215 10876 raft_consensus.cc:1275] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:19:24.585554 10801 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 25.
I20260709 13:19:24.599270 10867 raft_consensus.cc:1275] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 23 index: 10. Preceding OpId from leader: term: 25 index: 11. (index mismatch)
I20260709 13:19:24.605113 10873 raft_consensus.cc:1240] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 23. Current term is 24. Ops: [23.18-23.18]
W20260709 13:19:24.600443 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 46276.4us
W20260709 13:19:24.610335 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:24.610984 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:24.611202 10874 raft_consensus.cc:1240] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 26. Current term is 27. Ops: []
I20260709 13:19:24.527724 18734 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:24.603778 10872 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 21.
I20260709 13:19:24.605510 10868 raft_consensus.cc:1275] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 13:19:24.579535 18733 raft_consensus.cc:697] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:24.617065 10798 raft_consensus.cc:2393] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 24: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:24.567269 18352 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:24.608196 18530 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:24.635864 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:24.624604 11478 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:24.624819 18733 consensus_queue.cc:237] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:24.625911 18670 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:24.631704 18589 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:24.637617 10801 raft_consensus.cc:2393] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 22: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:24.578303 18692 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:24.643761 10942 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 23.
I20260709 13:19:24.601984 10871 raft_consensus.cc:1240] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 26. Current term is 27. Ops: []
W20260709 13:19:24.621039 18448 consensus_meta.cc:287] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.625s	user 0.010s	sys 0.003s
I20260709 13:19:24.653952 10766 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:24.653872 18448 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:24.655249 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:24.657019 18448 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:24.658262 10799 raft_consensus.cc:2393] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 20: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:24.669806 10875 raft_consensus.cc:2368] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 18. Current term is 19.
I20260709 13:19:24.636032 18530 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:24.677565 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:24.678407 10801 raft_consensus.cc:1813] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Handling vote request from an unknown peer cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:24.679433 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:24.681123 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:24.683540 10876 raft_consensus.cc:1275] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 13:19:24.686235 10801 raft_consensus.cc:2393] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 22: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:24.669432 18175 raft_consensus.cc:2764] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Leader pre-election decision vote started in defunct term 24: won
I20260709 13:19:24.689545 10796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 1079) took 1881 ms. Trace:
I20260709 13:19:24.698290 10796 rpcz_store.cc:276] 0709 13:19:22.808686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.063890 (+255204us) service_pool.cc:224] Handling call
0709 13:19:23.064148 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:23.104426 (+ 40278us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:23.104473 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:23.104532 (+    59us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:19:23.104576 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:24.689301 (+1584725us) spinlock_profiling.cc:243] Waited 40 ms on lock 0x7b5400295254. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:19:24.689354 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:24.689487 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":92024576}
I20260709 13:19:24.685552 18670 raft_consensus.cc:2804] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:19:24.707304 18670 raft_consensus.cc:697] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:24.708292 18670 consensus_queue.cc:237] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:24.712005 18692 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:24.712785 10948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41020 (request call id 3328) took 1128 ms. Trace:
I20260709 13:19:24.675403 18610 consensus_queue.cc:260] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:24.713199 10948 rpcz_store.cc:276] 0709 13:19:23.584408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.584541 (+   133us) service_pool.cc:224] Handling call
0709 13:19:24.691277 (+1106736us) spinlock_profiling.cc:243] Waited 375 ms on lock 0x7b540039a654. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:24.712683 (+ 21406us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":861760384}
I20260709 13:19:24.715163 10800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39442 (request call id 1082) took 1299 ms. Trace:
I20260709 13:19:24.715592 10800 rpcz_store.cc:276] 0709 13:19:23.392159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.392312 (+   153us) service_pool.cc:224] Handling call
0709 13:19:24.691799 (+1299487us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b5400294d54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:24.691908 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":497536768}
I20260709 13:19:24.716413 18557 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:24.716575 10947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41020 (request call id 3329) took 1106 ms. Trace:
W20260709 13:19:24.716961 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:24.716929 10947 rpcz_store.cc:276] 0709 13:19:23.584756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.584877 (+   121us) service_pool.cc:224] Handling call
0709 13:19:24.690710 (+1105833us) spinlock_profiling.cc:243] Waited 85.6 ms on lock 0x7b54003d8e54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:24.690826 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":196980992}
I20260709 13:19:24.717844 18622 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:24.718528 18622 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:24.721148 18622 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:24.722427 18563 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:24.723086 18563 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:24.723732 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:24.716945 18557 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:24.712951 18664 consensus_meta.cc:287] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.514s	user 0.011s	sys 0.000s
I20260709 13:19:24.737483 18636 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:24.737957 18636 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:19:24.738317 10797 raft_consensus.cc:1240] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 23. Current term is 24. Ops: []
I20260709 13:19:24.714761 10801 raft_consensus.cc:1275] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 26 index: 13. (index mismatch)
I20260709 13:19:24.732203 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:24.733119 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:24.688035 18739 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:24.735136 18350 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:24.678633 18530 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:24.741673 18664 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:24.759822 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:24.737017 18674 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 25 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:24.776010 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:24.771842 10943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 2924) took 1188 ms. Trace:
I20260709 13:19:24.727449 18557 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:24.770586 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:19:24.773063 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:24.801252 11504 raft_consensus.cc:1275] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 13:19:24.799553 10943 rpcz_store.cc:276] 0709 13:19:23.584037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.587707 (+  3670us) service_pool.cc:224] Handling call
0709 13:19:23.588017 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:23.588218 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:19:23.588451 (+   233us) log.cc:844] Serialized 14 byte log entry
0709 13:19:24.712335 (+1123884us) spinlock_profiling.cc:243] Waited 29.6 ms on lock 0x7b1800370180. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a5523a 00007f9c80ebfa2a
0709 13:19:24.771376 (+ 59041us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:24.771463 (+    87us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:24.771526 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:24.771614 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:24.771771 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1995,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":1034,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":111,"spinlock_wait_cycles":68130560,"thread_start_us":58780,"threads_started":1,"wal-append.queue_time_us":1181168}
I20260709 13:19:24.808945 11508 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 24.
I20260709 13:19:24.814853 18576 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:24.815434 18576 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:24.795655 18596 raft_consensus.cc:2749] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:24.817685 18352 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:24.819491 18589 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:24.825635 10944 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:24.828804 18742 log.cc:927] Time spent T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.062s	user 0.001s	sys 0.002s
I20260709 13:19:24.829452 18261 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:24.747159 18739 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:24.830974 18699 consensus_queue.cc:1059] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 26.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:19:24.831977 18739 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:24.832279 18755 log.cc:927] Time spent T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.061s	user 0.002s	sys 0.000s
I20260709 13:19:24.834017 11477 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:24.842633 10841 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:19:24.843483 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:24.846446 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:24.848439 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:24.851596 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:24.852223 11503 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:24.854544 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:19:24.855640 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:24.855552 10948 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:19:24.853753 11478 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.116.129:39685 timed out after 1.762s (SENT) [suppressed 7 similar messages]
I20260709 13:19:24.840979 18695 raft_consensus.cc:3055] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:19:24.863755 18695 raft_consensus.cc:740] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:24.859112 18715 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:24.865767 18715 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:24.865032 18695 consensus_queue.cc:260] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:24.866110 18715 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:24.830600 18693 consensus_queue.cc:1048] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.036s
I20260709 13:19:24.868675 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:24.869979 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:24.870286 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:24.870900 11512 raft_consensus.cc:2410] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 23 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 13:19:24.871417 11509 raft_consensus.cc:2410] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 23 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 13:19:24.872542 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:24.873268 10945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 2939) took 1121 ms. Trace:
I20260709 13:19:24.873675 10945 rpcz_store.cc:276] 0709 13:19:23.752064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:24.617923 (+865859us) service_pool.cc:224] Handling call
0709 13:19:24.632495 (+ 14572us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:24.632855 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:24.632884 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:24.633355 (+   471us) log.cc:844] Serialized 26 byte log entry
0709 13:19:24.795603 (+162248us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:24.795670 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:24.795774 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:24.873036 (+ 77262us) raft_consensus.cc:1685] finished
0709 13:19:24.873059 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:24.873207 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5839,"raft.queue_time_us":1841,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":11205,"thread_start_us":161800,"threads_started":1,"wal-append.queue_time_us":209696}
W20260709 13:19:24.875299 18749 log.cc:927] Time spent T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.088s	user 0.000s	sys 0.002s
I20260709 13:19:24.875489 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:24.876719 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:24.877380 10945 raft_consensus.cc:2393] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 23: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:24.878059 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:24.878293 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:24.832405 18699 consensus_queue.cc:1048] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.047s
I20260709 13:19:24.878554 11503 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 23.
I20260709 13:19:24.878562 10945 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:24.884716 10841 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:24.883430 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:24.892087 10841 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:24.857544 18422 raft_consensus.cc:2804] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:24.814486 18530 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:24.895574 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:24.890547 18732 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:24.896032 18531 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:24.896122 18422 raft_consensus.cc:697] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 21 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:24.896410 18548 raft_consensus.cc:2749] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:19:24.898844 10875 raft_consensus.cc:1275] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:19:24.906965 10945 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
I20260709 13:19:24.908903 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:24.910497 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:24.911275 11503 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
I20260709 13:19:24.885962 18695 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:19:24.912560 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:24.909963 10841 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:24.916343 10945 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 25.
W20260709 13:19:24.912595 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 58886.4us
I20260709 13:19:24.917659 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:24.918751 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:24.919526 10945 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 22.
I20260709 13:19:24.919476 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:24.920774 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:24.921828 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:24.922349 10945 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 21.
I20260709 13:19:24.922801 11477 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:24.923414 10945 raft_consensus.cc:1275] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:19:24.924750 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:24.924968 10945 raft_consensus.cc:1240] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 13:19:24.857815 18352 consensus_queue.cc:1048] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.053s
I20260709 13:19:24.927325 10945 raft_consensus.cc:1275] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 26 index: 13. (term mismatch)
W20260709 13:19:24.929850 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:24.947837 10945 pending_rounds.cc:85] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 12
I20260709 13:19:24.926842 18693 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:24.926641 18350 raft_consensus.cc:2804] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:19:24.947233 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:24.907418 18465 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:24.907053 18531 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:24.948172 10945 pending_rounds.cc:107] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 25.13
I20260709 13:19:24.950667 18422 consensus_queue.cc:237] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:24.952821 10945 raft_consensus.cc:2889] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:24.916060 18732 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:24.927721 18739 consensus_queue.cc:1048] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 13:19:24.948045 18690 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:24.991245 18690 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:24.991684 10799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 3426) took 1222 ms. Trace:
I20260709 13:19:24.992067 10799 rpcz_store.cc:276] 0709 13:19:23.769532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:24.181287 (+411755us) service_pool.cc:224] Handling call
0709 13:19:24.991469 (+810182us) spinlock_profiling.cc:243] Waited 448 ms on lock 0x7b5400387f54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:24.991623 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1030215040}
I20260709 13:19:24.950027 18693 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:24.993505 18690 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:24.993664 18693 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:24.999794 11509 raft_consensus.cc:1240] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 23. Current term is 24. Ops: []
W20260709 13:19:24.977811 18745 log.cc:927] Time spent T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.212s	user 0.002s	sys 0.000s
I20260709 13:19:25.003756 18610 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:25.008484 18636 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:25.012652 18732 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:24.957041 18531 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:19:24.979183 18747 log.cc:927] Time spent T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.211s	user 0.003s	sys 0.000s
W20260709 13:19:25.017740 18744 log.cc:927] Time spent T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.217s	user 0.000s	sys 0.001s
I20260709 13:19:24.984895 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:24.982239 10798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46758 (request call id 3424) took 1213 ms. Trace:
W20260709 13:19:25.021072 18720 log.cc:927] Time spent T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.646s	user 0.001s	sys 0.001s
I20260709 13:19:25.030675 10949 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 23.
I20260709 13:19:25.032289 10949 raft_consensus.cc:1275] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:19:25.033627 18696 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.033715 11477 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:24.968436 10950 raft_consensus.cc:1275] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:19:25.034268 18696 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:25.034757 11508 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56404 (request call id 1272) took 3215 ms (client timeout 2123 ms). Trace:
I20260709 13:19:25.034087 10795 raft_consensus.cc:1240] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
W20260709 13:19:25.035176 11508 rpcz_store.cc:269] 0709 13:19:21.819036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:21.819165 (+   129us) service_pool.cc:224] Handling call
0709 13:19:25.027455 (+3208290us) spinlock_profiling.cc:243] Waited 2.91 s on lock 0x7b540021e654. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:25.034662 (+  7207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6703330176}
I20260709 13:19:25.036608 18696 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:25.036749 18581 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:19:25.037312 18581 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:25.037741 18581 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:25.037943 10916 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:25.039026 18587 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.039572 18587 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:25.040901 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:25.041707 18587 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:25.042332 18587 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:19:25.042735 18752 log.cc:927] Time spent T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.274s	user 0.002s	sys 0.000s
I20260709 13:19:25.044104 10946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 2923) took 1468 ms. Trace:
I20260709 13:19:25.044461 10946 rpcz_store.cc:276] 0709 13:19:23.582969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.586169 (+  3200us) service_pool.cc:224] Handling call
0709 13:19:23.593098 (+  6929us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:23.600700 (+  7602us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:23.600736 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:23.601169 (+   433us) log.cc:844] Serialized 26 byte log entry
0709 13:19:24.722031 (+1120862us) spinlock_profiling.cc:243] Waited 2.65 ms on lock 0x7b1800283ba0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a550d1 00007f9c80e70a0f
0709 13:19:24.769029 (+ 46998us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:19:24.769075 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:24.769160 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:25.042119 (+272959us) spinlock_profiling.cc:243] Waited 39.8 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:19:25.043876 (+  1757us) raft_consensus.cc:1685] finished
0709 13:19:25.043899 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:25.044042 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7198,"raft.queue_time_us":7521,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":12519,"spinlock_wait_cycles":97617536,"thread_start_us":46777,"threads_started":1,"wal-append.queue_time_us":1159577}
I20260709 13:19:25.048216 11478 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:25.049328 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:25.049666 10916 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:19:25.050915 18545 consensus_queue.cc:1048] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:19:25.051506 18569 consensus_queue.cc:1048] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:19:25.052290 18590 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:19:25.052924 18590 raft_consensus.cc:697] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 27 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:25.052979 10916 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:25.053931 18750 log.cc:927] Time spent T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.284s	user 0.002s	sys 0.000s
I20260709 13:19:25.053874 18690 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:25.019047 10798 rpcz_store.cc:276] 0709 13:19:23.769028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:24.063571 (+294543us) service_pool.cc:224] Handling call
0709 13:19:24.981928 (+918357us) spinlock_profiling.cc:243] Waited 541 ms on lock 0x7b5400387554. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:24.982156 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1244320128}
I20260709 13:19:25.054518 18690 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:25.055078 18690 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:25.057269 18645 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:25.057824 18645 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:25.059976 18645 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:25.063879 18530 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:25.065109 18754 log.cc:927] Time spent T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
W20260709 13:19:25.066615 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:25.067477 10841 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:25.068343 10841 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:25.068593 10950 raft_consensus.cc:1275] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 23 index: 11. (index mismatch)
I20260709 13:19:25.069332 10944 raft_consensus.cc:1275] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:19:25.069518 10867 raft_consensus.cc:1275] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:19:25.070565 18733 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:19:25.086560 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:24.974367 18576 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:25.073699 10944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 2926) took 1482 ms. Trace:
I20260709 13:19:24.980371 18548 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.093868 10841 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:25.100483 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:25.104166 10798 raft_consensus.cc:1240] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 23. Current term is 24. Ops: []
I20260709 13:19:25.105291 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:25.106132 10872 raft_consensus.cc:2410] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 22 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:19:25.101922 18664 consensus_queue.cc:1048] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 13:19:25.108299 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:25.109054 10950 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 19.
I20260709 13:19:25.092994 18645 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.087014 18733 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:25.112103 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:25.112558 18733 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:19:25.053985 18590 consensus_queue.cc:237] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:25.106896 18548 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:25.117025 18548 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:25.078342 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:25.079566 11478 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:25.085647 10792 raft_consensus.cc:1240] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 25. Current term is 26. Ops: []
I20260709 13:19:25.066814 18690 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:25.122401 10766 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:25.137028 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:25.090308 18549 raft_consensus.cc:2804] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:25.091022 10916 leader_election.cc:400] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 election: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 18. Current term is 19.
I20260709 13:19:25.139472 10916 leader_election.cc:403] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:19:25.090909 10944 rpcz_store.cc:276] 0709 13:19:23.591262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.604937 (+ 13675us) service_pool.cc:224] Handling call
0709 13:19:23.605192 (+   255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:24.825141 (+1219949us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b54001601a0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80e2aa1d 00007f9c80e111a4 00007f9c80e93447 00007f9c80eabfe7 00007f9c80ead7de 00007f9c80eb7b49 00007f9c80eb8416 00007f9c80eb4d8f 00007f9c80eb483e 00007f9c85e79419
0709 13:19:25.073456 (+248315us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:25.073648 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":508570496}
I20260709 13:19:25.099164 18557 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:25.093665 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:25.141860 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 48059.2us
I20260709 13:19:25.142676 10841 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:25.094269 11507 raft_consensus.cc:1275] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:19:25.143071 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:25.111349 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:25.143630 10792 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:25.141883 10873 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 23.
I20260709 13:19:25.120035 10842 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:25.010669 18636 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 26 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:25.123029 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 47501us
I20260709 13:19:25.147030 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:25.147089 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:25.147928 10796 raft_consensus.cc:2393] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 22: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:25.147989 10944 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:25.149014 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:19:25.149252 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 58453us
I20260709 13:19:25.123508 10794 raft_consensus.cc:1275] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:19:25.152798 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:25.153725 10798 raft_consensus.cc:2393] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 23: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
W20260709 13:19:25.150892 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 71379.2us
I20260709 13:19:25.157719 11478 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:25.157737 10841 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:25.123463 18483 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.122139 18622 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.155071 18557 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:25.171056 18676 raft_consensus.cc:2749] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:25.171733 18676 raft_consensus.cc:2749] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:25.171252 18622 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:25.112114 18645 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:25.172819 18557 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:25.173926 18762 log.cc:927] Time spent T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.217s	user 0.000s	sys 0.001s
I20260709 13:19:25.155298 18545 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:19:25.174767 18545 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:25.175141 18545 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:25.174193 18622 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:25.164232 18715 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:25.178335 18715 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:25.155601 18734 raft_consensus.cc:2749] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:19:25.180588 11504 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 27.
I20260709 13:19:25.090276 18732 consensus_queue.cc:1048] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
W20260709 13:19:25.186698 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:25.191639 18645 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:25.192008 18532 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:25.192574 18532 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 13:19:25.192687 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 72730.8us
I20260709 13:19:25.192957 18532 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:25.193363 18532 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:19:25.194253 10842 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:25.195057 10795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39442 (request call id 1086) took 1350 ms. Trace:
I20260709 13:19:25.195634 10795 rpcz_store.cc:276] 0709 13:19:23.844473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:24.720987 (+876514us) service_pool.cc:224] Handling call
0709 13:19:25.033866 (+312879us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:25.035734 (+  1868us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:25.194772 (+159038us) spinlock_profiling.cc:243] Waited 294 ms on lock 0x7b5400387f50. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:25.194976 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":675162496}
I20260709 13:19:25.049176 18581 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:25.089651 10793 raft_consensus.cc:2393] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 20: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:25.198606 10794 raft_consensus.cc:1240] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 24. Current term is 25. Ops: [24.18-24.18]
I20260709 13:19:24.965651 10948 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 24.
I20260709 13:19:25.090447 18175 raft_consensus.cc:2749] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:19:25.163297 10797 raft_consensus.cc:1275] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:19:25.094202 18596 consensus_queue.cc:1048] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 13:19:25.089983 18739 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.135792 18696 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.199748 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:25.211716 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:25.189085 18693 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:25.200392 18576 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:19:25.157061 18352 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.203459 10795 raft_consensus.cc:1275] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 23 index: 10. Preceding OpId from leader: term: 25 index: 11. (index mismatch)
I20260709 13:19:25.227852 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:25.207231 10798 raft_consensus.cc:1240] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 22. Current term is 23. Ops: [22.11-22.11]
I20260709 13:19:25.122897 18692 consensus_queue.cc:1048] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.046s
W20260709 13:19:25.209609 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:25.212770 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:25.235076 10794 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 21.
I20260709 13:19:25.214362 18558 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:25.213990 18739 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:25.203903 18734 raft_consensus.cc:2749] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:25.215652 18696 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:25.164824 18379 raft_consensus.cc:2804] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:25.216348 18175 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.200873 18483 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:25.217907 10799 raft_consensus.cc:1240] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:25.164602 18664 consensus_queue.cc:1048] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:19:25.163257 18610 raft_consensus.cc:2749] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 18. Current term is 19.
I20260709 13:19:25.219115 10793 raft_consensus.cc:1240] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:25.201380 10944 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 24.
I20260709 13:19:25.200198 18530 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.229233 10941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 2938) took 1486 ms. Trace:
I20260709 13:19:25.229755 10842 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:25.282401 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 53105.8us
I20260709 13:19:25.165027 18678 consensus_queue.cc:1048] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 13:19:25.228983 10795 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 23.
I20260709 13:19:24.956521 18350 raft_consensus.cc:697] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:25.235984 10792 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
I20260709 13:19:25.265368 10794 raft_consensus.cc:1275] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 13:19:25.273159 18563 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:25.272948 18175 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:25.264384 18576 raft_consensus.cc:697] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 27 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:25.281688 10941 rpcz_store.cc:276] 0709 13:19:23.743104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:24.609463 (+866359us) service_pool.cc:224] Handling call
0709 13:19:24.609784 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:24.610021 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:19:24.758116 (+148095us) log.cc:844] Serialized 14 byte log entry
0709 13:19:25.228814 (+470698us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:25.228886 (+    72us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:19:25.228945 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:25.229029 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:25.229165 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":153223,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":5009,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":5006,"thread_start_us":608383,"threads_started":2}
I20260709 13:19:24.956331 18465 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:25.266291 18379 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:25.311120 10799 raft_consensus.cc:1275] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:19:25.234165 18352 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:25.325603 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:25.274889 18734 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:25.241132 18676 consensus_queue.cc:1048] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 13:19:25.301651 18596 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.302414 18715 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:25.301615 18530 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:25.342922 18734 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:25.339251 10766 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:25.334559 18379 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:25.355378 10766 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:25.350590 18596 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:25.365036 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:25.368270 10944 raft_consensus.cc:1240] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 23. Current term is 24. Ops: []
I20260709 13:19:25.351754 18618 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.370045 18618 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:25.370802 18693 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:25.370914 18734 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:25.357506 18589 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:19:25.368434 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:25.373812 10941 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 23.
I20260709 13:19:25.373819 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:25.374229 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:25.374665 10948 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:25.380358 10944 raft_consensus.cc:1275] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:19:25.358408 18261 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:19:25.383845 18486 consensus_queue.cc:1059] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:19:25.367262 18350 consensus_queue.cc:237] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:25.365542 18690 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:25.394464 10948 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 23.
I20260709 13:19:25.398887 11477 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:25.381750 18589 raft_consensus.cc:697] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:19:25.410574 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 260374us
I20260709 13:19:25.367262 18609 consensus_queue.cc:1059] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.036s
W20260709 13:19:25.411480 18760 log.cc:927] Time spent T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.112s	user 0.001s	sys 0.001s
I20260709 13:19:25.419960 10916 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:25.407281 18569 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:19:25.402398 18261 raft_consensus.cc:697] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:25.428244 10916 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:25.393976 18694 consensus_queue.cc:1048] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 13:19:25.383337 18733 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:25.408066 18581 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:25.418186 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 260474us
I20260709 13:19:25.439500 10841 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:25.443879 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 158902us
I20260709 13:19:25.445065 10842 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:25.387863 18576 consensus_queue.cc:237] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:25.431752 18610 raft_consensus.cc:2804] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:19:25.450253 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 287806us
I20260709 13:19:25.434393 18688 raft_consensus.cc:2804] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:25.459133 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:25.459221 10915 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:25.439087 18569 raft_consensus.cc:697] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 23 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
W20260709 13:19:25.460518 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:25.473719 10799 raft_consensus.cc:1275] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
W20260709 13:19:25.449687 18753 log.cc:927] Time spent T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.686s	user 0.004s	sys 0.000s
I20260709 13:19:25.477963 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:25.473774 10842 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:25.472237 18581 consensus_queue.cc:1048] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.008s
W20260709 13:19:25.472311 18763 log.cc:927] Time spent T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 13:19:25.481729 18465 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:25.473023 18463 raft_consensus.cc:2749] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:25.482930 18463 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:25.461040 18772 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.484184 18772 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:25.486311 18772 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:25.472435 18609 raft_consensus.cc:3055] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:19:25.457379 18696 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:25.487161 18609 raft_consensus.cc:740] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:25.487488 18589 consensus_queue.cc:237] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:25.488085 18609 consensus_queue.cc:260] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:25.490456 10870 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 22.
I20260709 13:19:25.491247 18596 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:25.451505 18483 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:25.492719 18618 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:25.482996 18733 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:25.494938 18261 consensus_queue.cc:237] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:25.495308 18569 consensus_queue.cc:237] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:25.478585 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:25.502292 10766 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:25.504809 10766 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:25.505236 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:25.506143 10870 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 24.
I20260709 13:19:25.509380 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:25.510231 10799 raft_consensus.cc:2468] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 20.
I20260709 13:19:25.515995 10870 raft_consensus.cc:1275] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:19:25.455969 18610 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:25.474017 18688 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:25.518116 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:25.506299 10766 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:25.519677 10915 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:25.524694 10872 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 22.
I20260709 13:19:25.532482 10766 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:19:25.534494 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 60815.9us
I20260709 13:19:25.535068 18483 raft_consensus.cc:2749] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:25.535090 10799 raft_consensus.cc:3060] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:25.536459 18483 consensus_queue.cc:1048] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:19:25.540549 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:25.542666 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:25.543193 10799 raft_consensus.cc:1275] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:19:25.544384 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:25.544507 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:25.545276 10876 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 23.
I20260709 13:19:25.545492 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:25.546033 10871 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:25.546375 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:25.547044 10876 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:19:25.555529 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 116177us
I20260709 13:19:25.555675 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 27 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:25.556958 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:25.557577 11507 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 22.
I20260709 13:19:25.558808 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:25.559242 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:25.558950 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:25.560680 10799 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 20.
I20260709 13:19:25.561199 18422 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:25.561828 18422 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:25.562839 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:25.563030 18558 consensus_queue.cc:1059] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:19:25.563818 18558 raft_consensus.cc:3055] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:25.564152 18558 raft_consensus.cc:740] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:25.564435 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:25.565152 10868 raft_consensus.cc:2393] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 24: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:25.558445 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:25.565056 18558 consensus_queue.cc:260] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:25.567694 10868 raft_consensus.cc:1275] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:19:25.568912 10868 raft_consensus.cc:1240] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 25. Current term is 26. Ops: [25.19-25.19]
I20260709 13:19:25.572403 10876 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 23.
I20260709 13:19:25.573709 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:25.559312 10841 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:25.566548 10871 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 23.
I20260709 13:19:25.570852 11478 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
W20260709 13:19:25.602536 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 45936.6us
I20260709 13:19:25.604128 10841 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:25.512436 18465 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:25.611503 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:25.524057 18610 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:25.533106 18688 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:25.600397 10915 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:25.605969 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:25.579051 18558 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:25.607118 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:19:25.607584 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 75228.7us
I20260709 13:19:25.636596 10876 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:25.638295 10841 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:25.642279 10841 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:25.524951 18352 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:25.514757 18596 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:25.613881 18674 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:19:25.619240 18350 raft_consensus.cc:2749] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:25.538604 18483 consensus_queue.cc:1048] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 13:19:25.525729 18664 consensus_queue.cc:1048] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.110s
I20260709 13:19:25.582971 18733 consensus_queue.cc:1059] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.039s
I20260709 13:19:25.622238 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:19:25.624011 18766 log.cc:927] Time spent T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 13:19:25.623143 18736 raft_consensus.cc:2749] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:19:25.532712 18587 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:25.610237 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:25.581513 18555 raft_consensus.cc:2749] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:19:25.663240 10766 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:25.625717 18609 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:25.671715 10798 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 23.
I20260709 13:19:25.668912 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 1219) took 1374 ms. Trace:
I20260709 13:19:25.611233 18698 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:25.645689 10767 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:25.660187 18422 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:25.680262 18674 raft_consensus.cc:697] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:25.611104 18465 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:25.680066 18395 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.662060 18486 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.660390 18676 raft_consensus.cc:2804] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:25.664909 18696 consensus_queue.cc:1048] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.056s
I20260709 13:19:25.721433 18770 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:25.725405 18532 consensus_meta.cc:287] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.532s	user 0.009s	sys 0.000s
W20260709 13:19:25.730971 18777 log.cc:927] Time spent T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:19:25.731551 18618 raft_consensus.cc:2749] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:19:25.779480 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 116313us
I20260709 13:19:25.822321 18736 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:25.798193 18589 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.798779 18175 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:25.799705 18531 consensus_meta.cc:287] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.785s	user 0.004s	sys 0.008s
I20260709 13:19:25.602816 18732 consensus_queue.cc:1048] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.079s
I20260709 13:19:25.800590 18733 raft_consensus.cc:3055] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 LEADER]: Stepping down as leader of term 25
W20260709 13:19:25.801558 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 159357us
I20260709 13:19:25.802204 18772 raft_consensus.cc:2749] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:25.631942 18592 consensus_queue.cc:260] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:25.665493 18734 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:25.732947 18549 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:25.722371 18379 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:25.611455 18771 raft_consensus.cc:2749] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:25.818015 10876 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 24.
I20260709 13:19:25.821316 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:25.879144 10871 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 23.
I20260709 13:19:25.879690 18379 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:25.880120 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:25.880457 10799 raft_consensus.cc:1240] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:25.881356 10872 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:25.814185 10867 rpcz_store.cc:276] 0709 13:19:24.294690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:24.609647 (+314957us) service_pool.cc:224] Handling call
0709 13:19:25.069230 (+459583us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:25.073546 (+  4316us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:25.668688 (+595142us) spinlock_profiling.cc:243] Waited 459 ms on lock 0x7b54002d4450. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:25.668854 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1056198528}
I20260709 13:19:25.882556 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:25.883311 10867 raft_consensus.cc:2393] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 22: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:25.884012 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:25.885409 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:25.886011 10867 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 27.
I20260709 13:19:25.829869 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:25.830955 10945 raft_consensus.cc:1275] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:19:25.832937 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:25.888078 10944 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 23: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:25.888154 18731 raft_consensus.cc:3055] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:19:25.833925 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:25.889050 10948 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 20.
W20260709 13:19:25.797703 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:25.889590 10872 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 25.
I20260709 13:19:25.890870 18734 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:25.827966 18768 log.cc:927] Time spent T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.216s	user 0.008s	sys 0.000s
I20260709 13:19:25.822505 18698 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:25.891935 18698 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:25.893406 18734 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:25.822717 18695 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:25.854848 18532 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:25.895067 18695 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:25.896977 18532 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:25.897382 18695 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:25.897651 18549 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:25.827245 18465 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:25.827045 18636 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:19:25.861613 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 197381us
I20260709 13:19:25.900134 10916 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:25.826869 18676 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:25.834775 18587 raft_consensus.cc:2749] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:25.900883 18676 raft_consensus.cc:3060] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:25.865629 10841 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:25.864563 18531 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:25.904913 18531 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:25.905488 18531 raft_consensus.cc:2764] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Leader pre-election decision vote started in defunct term 26: won
I20260709 13:19:25.864449 18733 raft_consensus.cc:740] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:25.907241 10870 raft_consensus.cc:2393] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 22: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:25.908872 18592 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:25.881317 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:25.910305 10871 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 22.
I20260709 13:19:25.911158 10871 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
new_leader_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:53438
W20260709 13:19:25.824308 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 178731us
I20260709 13:19:25.913075 18736 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:25.913511 18736 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:25.913401 10767 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:25.834383 18395 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:25.916888 18678 raft_consensus.cc:2749] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:19:25.918035 18530 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:25.918532 18395 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:25.918618 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:25.919579 10867 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 27.
I20260709 13:19:25.920564 18695 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:25.920691 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:25.921038 18545 consensus_meta.cc:287] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.745s	user 0.015s	sys 0.002s
I20260709 13:19:25.911139 18676 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:25.921253 18695 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:25.921528 18545 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:25.923529 18695 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:25.924623 18463 raft_consensus.cc:2749] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:25.930969 18674 consensus_queue.cc:237] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:25.932353 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:25.961719 10873 raft_consensus.cc:2410] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 22 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 9.
I20260709 13:19:25.834699 18622 consensus_queue.cc:1059] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.032s
I20260709 13:19:25.933629 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:25.919623 11478 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:25.940549 10871 raft_consensus.cc:606] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Received request to transfer leadership to TS cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:25.941573 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:25.967219 10872 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 23: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:19:25.967075 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 47539.4us
I20260709 13:19:25.966656 10871 raft_consensus.cc:614] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:25.943259 18676 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:25.945828 18465 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:25.946889 18733 consensus_queue.cc:260] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:25.975704 18581 consensus_queue.cc:1048] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:19:25.976388 18733 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:19:25.977492 18422 consensus_queue.cc:1059] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 24.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 13:19:25.978168 18699 consensus_queue.cc:1048] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:19:25.979393 11478 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:19:25.862859 18589 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:25.981761 18581 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:19:25.982273 18581 raft_consensus.cc:697] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 25 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:25.982990  9682 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:19:25.983108 18589 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:25.985140  9682 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:19:25.948222 18715 consensus_meta.cc:287] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.595s	user 0.011s	sys 0.001s
I20260709 13:19:25.948006 18778 consensus_queue.cc:1048] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:25.888545 18731 raft_consensus.cc:740] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:25.921316 10867 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 26.
I20260709 13:19:25.952224 18739 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:25.953203 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:26.007174 10876 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 20.
W20260709 13:19:25.953986 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 53937.7us
I20260709 13:19:26.008512 10916 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:25.954500 18587 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.954999 18664 raft_consensus.cc:2749] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:25.848712 18770 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:25.838992 18486 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:25.958359 10767 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:19:25.972755 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:25.974460 10871 raft_consensus.cc:1240] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 22. Current term is 23. Ops: [22.11-22.11]
W20260709 13:19:25.999289 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 398957us
I20260709 13:19:25.982134 18698 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:26.005395 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:26.021844 11507 raft_consensus.cc:2410] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 21 because replica has last-logged OpId of term: 19 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 15.
I20260709 13:19:26.022899 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:26.025818 10916 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
W20260709 13:19:26.027982 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 69739.2us
I20260709 13:19:25.973701 18695 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:26.041324 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:26.041603 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:26.042069 10950 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 26.
I20260709 13:19:26.043116 10767 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:26.043169 11510 raft_consensus.cc:1275] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 13:19:26.047004 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:26.048074 11507 raft_consensus.cc:1240] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 22. Current term is 23. Ops: [22.18-22.18]
I20260709 13:19:26.006563 18589 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.048959 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:26.049606 10943 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 21.
I20260709 13:19:26.006609 18715 raft_consensus.cc:2749] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:26.016103 18596 raft_consensus.cc:2749] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:26.058686 10867 raft_consensus.cc:1240] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 25. Current term is 26. Ops: [25.11-25.11]
I20260709 13:19:26.058516 10767 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:26.061385 11505 raft_consensus.cc:1240] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 13:19:26.018301 18692 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:26.061937 10767 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:26.056336 18395 consensus_queue.cc:1059] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 13:19:26.066985 18694 raft_consensus.cc:2804] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:19:26.067180 18736 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:26.068583 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:26.096975 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:26.098312 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:26.069355 10947 raft_consensus.cc:1240] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 23. Current term is 24. Ops: []
I20260709 13:19:26.099931 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:26.100818 10947 raft_consensus.cc:2393] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 24: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:26.101760 10767 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:19:26.075250 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 49548.4us
I20260709 13:19:26.075662 10874 raft_consensus.cc:1240] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:25.978217 18688 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:26.089025 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:26.039539 18622 raft_consensus.cc:2749] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:26.090020 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:26.077123 10766 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:26.071553 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:26.036580 18379 raft_consensus.cc:2749] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:26.112799 10943 raft_consensus.cc:2410] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 23 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:19:26.110205 10875 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 23.
I20260709 13:19:26.118129 10767 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:26.123456 10946 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:26.125303 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:26.113540 10766 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:26.018540 18587 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:26.134714 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:26.136978 10799 raft_consensus.cc:2410] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 23 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:19:26.082171 18690 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.078390 18696 raft_consensus.cc:2749] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:19:26.139987 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:26.152680 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:26.153260 10947 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 26.
I20260709 13:19:26.153532 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:26.140349 10941 raft_consensus.cc:1275] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:19:26.141374 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:26.049247 18732 consensus_queue.cc:1048] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.084s
I20260709 13:19:26.151280 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:26.151623 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:26.122715 18678 raft_consensus.cc:2749] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:26.082171 18695 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
W20260709 13:19:26.168071 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 53881.1us
I20260709 13:19:26.168184 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 26 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:26.130035 18694 raft_consensus.cc:697] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:19:26.139279 18261 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.027s	sys 0.059s
W20260709 13:19:26.173518 18261 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.027s	sys 0.059s
I20260709 13:19:26.155481 10796 raft_consensus.cc:1275] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 13:19:26.156605 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 30 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:26.121624 18530 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:26.129395 18645 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:19:26.178718 18645 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:26.179327 18645 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:19:26.180440 10941 raft_consensus.cc:1275] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 13:19:26.122728 18795 raft_consensus.cc:2804] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:26.183085 18795 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:26.183619 18795 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:26.185252 18531 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.186062 18531 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:26.187352 18422 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:26.188139 18422 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:26.190250 18531 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:26.160702 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 30 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:26.161073 10798 raft_consensus.cc:1275] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:19:26.162022 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:26.056638 18739 consensus_queue.cc:1048] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.080s
I20260709 13:19:26.193114 10947 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:26.077731 18589 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:26.143508 10767 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:26.190658 18422 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:26.213995 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 26 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:26.215445 10797 raft_consensus.cc:1275] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 23 index: 11. (index mismatch)
I20260709 13:19:26.216797 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:26.217214 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:26.214708 10798 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:19:26.217960 10797 raft_consensus.cc:2393] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 20: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:26.220647 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:26.170390 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:26.227571 10798 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 26.
I20260709 13:19:26.152268 18670 raft_consensus.cc:2749] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:19:26.233258 10697 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42956 (request call id 261) took 69466 ms (client timeout 15000 ms). Trace:
I20260709 13:19:26.218304 10946 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 25.
W20260709 13:19:26.231504 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 88066.6us
I20260709 13:19:26.113418 18731 consensus_queue.cc:260] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:26.111635 18692 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:26.155071 18379 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:26.225001 18695 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:26.358961 10842 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:19:26.424505 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 65627.2us
I20260709 13:19:26.159240 18690 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:26.222373 18622 consensus_queue.cc:1059] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.119s
W20260709 13:19:26.417851 18175 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.011s	sys 0.020s
I20260709 13:19:26.290901 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:26.308609 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
W20260709 13:19:26.342706 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.234268 18545 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:26.305655 10947 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 22.
I20260709 13:19:26.420096 18379 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:26.307454 10794 raft_consensus.cc:2393] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 22: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:26.290119 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:19:26.288964 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 119100us
I20260709 13:19:26.378661 10875 raft_consensus.cc:1240] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 25. Current term is 26. Ops: [25.19-25.19]
W20260709 13:19:26.381636 11477 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.000s	sys 0.026s
I20260709 13:19:26.499253 10916 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:26.401255 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:26.364073 18610 consensus_meta.cc:287] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.690s	user 0.012s	sys 0.000s
I20260709 13:19:26.468083 18463 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:19:26.380788 18739 raft_consensus.cc:2749] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:26.354763 18778 consensus_queue.cc:1048] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.073s
I20260709 13:19:26.355762 18715 consensus_queue.cc:1048] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.071s
I20260709 13:19:26.416350 18465 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:26.403702 18422 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:26.480083 11478 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
W20260709 13:19:26.543747 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 65039us
W20260709 13:19:26.233922 10697 rpcz_store.cc:269] 0709 13:18:16.766983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.767516 (+   533us) service_pool.cc:224] Handling call
0709 13:19:26.233145 (+69465629us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:22.271307 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:22.272188 (+   881us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:22.272241 (+    53us) write_op.cc:435] Acquired schema lock
0709 13:19:22.272258 (+    17us) tablet.cc:662] Decoding operations
0709 13:19:22.273093 (+   835us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:22.273210 (+   117us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:22.273234 (+    24us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:19:22.279535 (+  6301us) tablet.cc:701] Row locks acquired
0709 13:19:22.279591 (+    56us) write_op.cc:260] PREPARE: finished
0709 13:19:22.279784 (+   193us) write_op.cc:270] Start()
0709 13:19:22.279938 (+   154us) write_op.cc:276] Timestamp: P: 1783603162279707 usec, L: 0
0709 13:19:22.279972 (+    34us) op_driver.cc:348] REPLICATION: starting
0709 13:19:22.282518 (+  2546us) log.cc:844] Serialized 2094 byte log entry
0709 13:19:22.552342 (+269824us) op_driver.cc:464] REPLICATION: finished
0709 13:19:22.969434 (+417092us) write_op.cc:301] APPLY: starting
0709 13:19:22.988354 (+ 18920us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:23.292551 (+304197us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:23.292691 (+   140us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:23.483238 (+190547us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:23.483552 (+   314us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=10,mrs_lookups=0
0709 13:19:23.483616 (+    64us) write_op.cc:312] APPLY: finished
0709 13:19:23.483979 (+   363us) log.cc:844] Serialized 57 byte log entry
0709 13:19:23.484184 (+   205us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:23.484292 (+   108us) write_op.cc:454] Released schema lock
0709 13:19:23.484357 (+    65us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:23.484804 (+   447us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:23.484832 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4546918,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":275777,"apply.run_cpu_time_us":11454,"apply.run_wall_time_us":557309,"cfile_cache_hit":22,"cfile_cache_hit_bytes":25037,"num_ops":5,"prepare.queue_time_us":468697,"prepare.run_cpu_time_us":25638,"prepare.run_wall_time_us":165732,"raft.queue_time_us":54234,"raft.run_cpu_time_us":4868,"raft.run_wall_time_us":110405,"replication_time_us":272178,"thread_start_us":596586,"threads_started":4,"wal-append.queue_time_us":159527,"wal-append.run_cpu_time_us":9004,"wal-append.run_wall_time_us":2048274}]]}
I20260709 13:19:26.362499 18676 consensus_queue.cc:1048] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.070s
W20260709 13:19:26.492168 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:26.492723 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 116697us
I20260709 13:19:26.380635 18694 consensus_queue.cc:237] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:26.554639 10767 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:26.315796 10872 raft_consensus.cc:1275] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:19:26.369140 18793 consensus_queue.cc:1048] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.069s
W20260709 13:19:26.499616 11477 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.000s	sys 0.027s
W20260709 13:19:26.556953 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 719599us
I20260709 13:19:26.518277 18610 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:26.289206 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 218620us
I20260709 13:19:26.560608 18610 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:26.059693 18581 consensus_queue.cc:237] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:26.563071 18639 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:26.563555 18639 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:26.563931 18639 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:19:26.467273 18693 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.006s	sys 0.034s
W20260709 13:19:26.565511 18693 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.007s	sys 0.035s
I20260709 13:19:26.480574 18486 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:26.488889 18698 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:26.500144 18175 raft_consensus.cc:3055] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:19:26.566720 18175 raft_consensus.cc:740] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:26.493037 18770 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:26.567567 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:26.505769 18379 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:26.561322 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:26.568234 10947 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:26.479239 18563 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.534111 18463 raft_consensus.cc:697] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:26.499455 18557 consensus_queue.cc:1059] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.051s
I20260709 13:19:26.575078 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:26.577978 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:26.578301 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:26.579239 10950 raft_consensus.cc:1275] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 27 index: 19. (term mismatch)
I20260709 13:19:26.543823 18688 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:26.592799 10950 pending_rounds.cc:85] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 18
I20260709 13:19:26.593140 10950 pending_rounds.cc:107] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 26.19
I20260709 13:19:26.593431 10950 raft_consensus.cc:2889] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:26.582301 11477 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:26.524863 18622 raft_consensus.cc:2749] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:26.580268 18486 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.589483 11508 raft_consensus.cc:1275] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:19:26.595077 11505 raft_consensus.cc:1275] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 13:19:26.595688 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:26.600019 10947 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 25.
I20260709 13:19:26.567723 18770 consensus_queue.cc:1048] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.017s
I20260709 13:19:26.590827 18563 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:26.568404 18587 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:26.587291 18609 consensus_meta.cc:287] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.763s	user 0.011s	sys 0.000s
I20260709 13:19:26.576440 18465 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.613998 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:26.581800 18698 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:26.556471 18395 consensus_queue.cc:1048] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 13:19:26.611219 18699 consensus_queue.cc:1048] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.018s
W20260709 13:19:26.631559 18549 consensus_meta.cc:287] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.733s	user 0.012s	sys 0.000s
I20260709 13:19:26.631995 18549 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:26.633061 10874 raft_consensus.cc:1240] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 13:19:26.633225 11512 raft_consensus.cc:2468] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
I20260709 13:19:26.633693 18736 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:26.635397 18695 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:26.637282 18799 log.cc:927] Time spent T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.206s	user 0.000s	sys 0.001s
I20260709 13:19:26.622036 18795 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:26.638175 18792 log.cc:927] Time spent T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
W20260709 13:19:26.639676 18791 log.cc:927] Time spent T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.333s	user 0.001s	sys 0.000s
I20260709 13:19:26.630672 18690 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:26.556864 18422 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:26.642297 18795 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:26.643653 18422 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:26.568302 18448 consensus_queue.cc:1059] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.042s
W20260709 13:19:26.621560 18555 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.041s	sys 0.059s
I20260709 13:19:26.620270 18692 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:26.621403 10875 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:26.646235 10799 raft_consensus.cc:2410] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 24 because replica has last-logged OpId of term: 21 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 9.
W20260709 13:19:26.611590 18637 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.013s	sys 0.030s
I20260709 13:19:26.622684 11477 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:26.673641 10875 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 24.
I20260709 13:19:26.676139 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:19:26.649092 18555 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.041s	sys 0.059s
W20260709 13:19:26.673877 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 51298.3us
W20260709 13:19:26.661391 18733 consensus_meta.cc:287] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.685s	user 0.003s	sys 0.009s
W20260709 13:19:26.677692 10944 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.023s	sys 0.035s
I20260709 13:19:26.635197 18549 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:26.651036 18798 log.cc:927] Time spent T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.278s	user 0.001s	sys 0.000s
I20260709 13:19:26.646215 18545 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:19:26.684535 10944 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.024s	sys 0.035s
I20260709 13:19:26.678411 10875 raft_consensus.cc:3060] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:26.613803 18486 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:26.674541 18637 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.013s	sys 0.033s
I20260709 13:19:26.594249 18693 consensus_queue.cc:1048] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.083s
I20260709 13:19:26.699565 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2868) took 1085 ms. Trace:
I20260709 13:19:26.710835 18486 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:26.677321 18558 raft_consensus.cc:2804] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:26.711105 10873 rpcz_store.cc:276] 0709 13:19:25.614532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:25.962569 (+348037us) service_pool.cc:224] Handling call
0709 13:19:25.962975 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:25.963237 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:19:26.404367 (+441130us) log.cc:844] Serialized 14 byte log entry
0709 13:19:26.699162 (+294795us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:26.699222 (+    60us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:26.699274 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:26.699338 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:26.699502 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":612,"prepare.queue_time_us":630705,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":22698,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":89,"thread_start_us":655106,"threads_started":2}
I20260709 13:19:26.645442 18465 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:26.716650 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:26.717314 10873 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 22.
I20260709 13:19:26.718186 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:26.719756 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:26.608426 18175 consensus_queue.cc:260] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:26.720744 18175 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:19:26.730085 10875 raft_consensus.cc:2468] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 25.
I20260709 13:19:26.731087 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:26.731920 10875 raft_consensus.cc:2410] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 21 because replica has last-logged OpId of term: 19 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 15.
I20260709 13:19:26.718262 11477 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:26.722298 10873 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 24.
I20260709 13:19:26.664315 18639 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:26.648195 18698 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:26.736750 10873 raft_consensus.cc:1275] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:19:26.740701 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:26.740999 10873 raft_consensus.cc:1275] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:19:26.743337 10947 raft_consensus.cc:1275] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:19:26.717854 18558 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:26.715863 18486 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:26.733094 18699 consensus_queue.cc:1048] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.019s
I20260709 13:19:26.758492 18465 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:26.763375 18805 log.cc:927] Time spent T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:19:26.765832 10871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 2740) took 1129 ms. Trace:
I20260709 13:19:26.775359 18486 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:26.776137 10871 rpcz_store.cc:276] 0709 13:19:25.636084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.045299 (+409215us) service_pool.cc:224] Handling call
0709 13:19:26.045677 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:26.046125 (+   448us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:19:26.046165 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:26.060007 (+ 13842us) log.cc:844] Serialized 26 byte log entry
0709 13:19:26.479803 (+419796us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:26.479858 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:26.479931 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:26.765561 (+285630us) raft_consensus.cc:1685] finished
0709 13:19:26.765585 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:26.765763 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3005,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":15816,"thread_start_us":341099,"threads_started":1,"wal-append.queue_time_us":556073}
I20260709 13:19:26.777732 18486 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:26.767278 18798 log.cc:927] Time spent T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:19:26.683102 18463 consensus_queue.cc:237] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 23.10, Last appended by leader: 10, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:26.767889 18695 raft_consensus.cc:2804] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:19:26.727133 18809 consensus_queue.cc:1048] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.057s
I20260709 13:19:26.778471 18569 consensus_queue.cc:1048] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 13:19:26.770457 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.752405 18558 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:19:26.792971 18645 consensus_meta.cc:287] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.613s	user 0.008s	sys 0.004s
I20260709 13:19:26.793464 18645 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:26.755820 18448 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:26.795256 18448 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:26.797597 18448 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:26.799723 10801 raft_consensus.cc:2393] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 28: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:26.806360 18448 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.806946 18645 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 28 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:26.807300 18448 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:26.784268 18695 raft_consensus.cc:697] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 25 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
W20260709 13:19:26.819314 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:26.819762 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 259568us
I20260709 13:19:26.838980 10766 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:19:26.839072 10869 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.832s	user 0.006s	sys 0.009s
W20260709 13:19:26.843441 10869 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.844s	user 0.006s	sys 0.009s
I20260709 13:19:26.850417 18589 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:26.852686 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:19:26.879276 10870 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.008s	sys 0.040s
I20260709 13:19:26.856058 10946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36714 (request call id 1398) took 1026 ms. Trace:
I20260709 13:19:26.880016 10946 rpcz_store.cc:276] 0709 13:19:25.829481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:25.829621 (+   140us) service_pool.cc:224] Handling call
0709 13:19:26.855839 (+1026218us) spinlock_profiling.cc:243] Waited 236 ms on lock 0x7b54003d9354. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:26.855981 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":543315840}
I20260709 13:19:26.881239 10870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2871) took 1262 ms. Trace:
I20260709 13:19:26.857936 18731 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:26.843928 18465 raft_consensus.cc:2749] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:19:26.834208 18596 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.863051 18448 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:26.868801 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:19:26.839288 18592 consensus_meta.cc:287] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.928s	user 0.007s	sys 0.005s
W20260709 13:19:26.875136 11504 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.851s	user 0.018s	sys 0.016s
I20260709 13:19:26.876247 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:26.904448 10875 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:26.878368 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:26.885412 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:26.904482 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:26.885787 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:26.886680 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:26.881644 10870 rpcz_store.cc:276] 0709 13:19:25.618989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:25.965765 (+346776us) service_pool.cc:224] Handling call
0709 13:19:25.966066 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:25.966288 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:19:26.375746 (+409458us) log.cc:844] Serialized 14 byte log entry
0709 13:19:26.880905 (+505159us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:26.880959 (+    54us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:19:26.881005 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:26.881069 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:26.881186 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32036,"prepare.queue_time_us":663208,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":1067,"thread_start_us":880781,"threads_started":2}
W20260709 13:19:26.923194 11504 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.899s	user 0.019s	sys 0.017s
I20260709 13:19:26.926671 10942 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 23.
I20260709 13:19:26.930366 10946 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:26.933327 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:26.874550 18558 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:26.934260 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:26.938822 10943 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Advancing to term 28
I20260709 13:19:26.949663 10870 raft_consensus.cc:1275] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:19:26.950649 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:26.948280 10947 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 19.
I20260709 13:19:26.959990 10766 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:26.943943 10801 raft_consensus.cc:1240] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:26.955504 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:26.967264 10801 raft_consensus.cc:1275] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:19:26.948593 10875 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 23.
I20260709 13:19:26.960739 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:26.973184 10947 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 23.
I20260709 13:19:27.010874 10794 raft_consensus.cc:2393] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 25: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:26.974648 10766 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:27.012753 18637 raft_consensus.cc:2804] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:27.013232 18637 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:26.894124 18589 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:26.945991 18558 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:26.943279 18698 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:26.939785 18639 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:26.946333 18563 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:27.023139 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 48610.9us
I20260709 13:19:26.905331 18596 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:26.947196 18813 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:27.025858 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 504447us
I20260709 13:19:26.947134 18804 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:19:27.050940 10794 raft_consensus.cc:1240] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 25. Current term is 26. Ops: [25.11-25.11]
I20260709 13:19:27.052431 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:27.053117 10794 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:27.057001 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:27.019021 18637 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:27.062264 10916 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:19:27.064715 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:27.065497 10801 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:27.025441 10946 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 24.
I20260709 13:19:27.068071 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:27.068785 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:27.075343 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 27 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:27.078155 10943 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 28.
I20260709 13:19:27.078989 10870 raft_consensus.cc:2393] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 25: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:27.070291 18555 raft_consensus.cc:3055] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:19:27.078404 10793 raft_consensus.cc:1240] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:27.082533 10875 raft_consensus.cc:1275] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 13:19:27.065358 18813 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:27.086971 10916 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:27.090894 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:27.091660 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:27.092061 10841 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:27.039656 18465 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:27.104136 10794 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 22.
I20260709 13:19:27.188252 10794 raft_consensus.cc:1240] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 24. Current term is 25. Ops: [24.18-24.18]
I20260709 13:19:27.072696 18664 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:27.011991 18695 consensus_queue.cc:237] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:27.025408 18589 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:27.083905 18545 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:19:27.112800 18827 log.cc:927] Time spent T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.187s	user 0.000s	sys 0.000s
I20260709 13:19:27.096424 18688 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:19:27.098989 18793 consensus_queue.cc:1059] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 13:19:27.099056 18558 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.153532 18692 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.062186 18379 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:27.161690 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:27.157039 10801 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 24.
I20260709 13:19:27.221848 10873 raft_consensus.cc:2393] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 22: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:27.101861 18778 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:27.085038 10766 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:27.166509 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:27.183827 10946 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 25.
W20260709 13:19:27.225714 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 140788us
I20260709 13:19:27.227787 10946 raft_consensus.cc:3055] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:19:27.228168 10946 raft_consensus.cc:740] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:19:27.233282 11475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.098829 18770 consensus_queue.cc:1059] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 23.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.073s
W20260709 13:19:27.175104 18826 log.cc:927] Time spent T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.213s	user 0.001s	sys 0.006s
I20260709 13:19:27.246047 10948 raft_consensus.cc:3055] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:19:27.246516 10948 raft_consensus.cc:740] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:27.247308 10948 consensus_queue.cc:260] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.11, Last appended by leader: 11, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:27.248412 18828 log.cc:927] Time spent T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:19:27.248711 10948 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:19:27.205345 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 116223us
W20260709 13:19:27.207832 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 461698us
I20260709 13:19:27.177579 18555 raft_consensus.cc:740] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:27.220078 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:27.159562 18801 log.cc:927] Time spent T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.657s	user 0.013s	sys 0.004s
W20260709 13:19:27.191900 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 104994us
I20260709 13:19:27.253391 10798 raft_consensus.cc:2368] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 22. Current term is 23.
I20260709 13:19:27.237818 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 27 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:27.213099 18688 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:27.186705 18465 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:27.264896 18465 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:27.234030 18692 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:27.228955 10946 consensus_queue.cc:260] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.18, Last appended by leader: 18, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:27.250808 10875 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 24.
I20260709 13:19:27.273085 10841 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:27.273087 10946 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:19:27.201745 18831 log.cc:927] Time spent T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.069s	user 0.002s	sys 0.006s
I20260709 13:19:27.202389 18592 raft_consensus.cc:2749] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:27.253919 10916 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:27.214767 18549 raft_consensus.cc:2749] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:19:27.263482 10950 raft_consensus.cc:1240] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:27.243000 18770 raft_consensus.cc:3055] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:19:27.285529 18770 raft_consensus.cc:740] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:27.252729 18395 consensus_queue.cc:1059] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 23.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:19:27.235422 18829 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:27.287266 18834 log.cc:927] Time spent T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 13:19:27.287115 18829 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:27.289330 18829 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:27.290236 10800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3455) took 1147 ms. Trace:
I20260709 13:19:27.240334 18739 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 13:19:27.264626 18688 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:19:27.299459 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:27.303233 10950 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:27.208125 18664 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:27.282615 10916 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:27.284446 18589 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:27.284766 18692 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:27.285156 10870 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:27.292325 10949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 2968) took 1225 ms. Trace:
I20260709 13:19:27.307082 10949 rpcz_store.cc:276] 0709 13:19:26.066827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.074126 (+  7299us) service_pool.cc:224] Handling call
0709 13:19:26.074443 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:26.501656 (+427213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:26.501685 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:26.509059 (+  7374us) log.cc:844] Serialized 26 byte log entry
0709 13:19:26.799643 (+290584us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:26.799680 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:26.799758 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:27.292123 (+492365us) raft_consensus.cc:1685] finished
0709 13:19:27.292146 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:27.292270 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":119782,"raft.queue_time_us":86623,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":3792,"thread_start_us":659474,"threads_started":2,"wal-append.queue_time_us":602857}
I20260709 13:19:27.290620 10800 rpcz_store.cc:276] 0709 13:19:26.142566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.143231 (+   665us) service_pool.cc:224] Handling call
0709 13:19:26.161682 (+ 18451us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:26.487156 (+325474us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:26.487193 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:26.497583 (+ 10390us) log.cc:844] Serialized 26 byte log entry
0709 13:19:26.837793 (+340210us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:26.837868 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:26.837966 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:27.290013 (+452047us) raft_consensus.cc:1685] finished
0709 13:19:27.290033 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:27.290173 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":83337,"raft.queue_time_us":62887,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":59946,"thread_start_us":584414,"threads_started":2,"wal-append.queue_time_us":535947}
I20260709 13:19:27.240904 18778 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:27.235754 18558 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:27.318048 10799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2709) took 1027 ms. Trace:
I20260709 13:19:27.291739 18793 raft_consensus.cc:2804] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:27.321532 18379 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:27.322126 18692 raft_consensus.cc:2804] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:27.325768 18739 raft_consensus.cc:697] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 28 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:27.264166 18690 consensus_queue.cc:1059] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 13:19:27.334303 10799 rpcz_store.cc:276] 0709 13:19:26.290481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.290606 (+   125us) service_pool.cc:224] Handling call
0709 13:19:27.317744 (+1027138us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b540039f8d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:27.317960 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":381516544}
I20260709 13:19:27.280858 18733 consensus_queue.cc:1048] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.055s
I20260709 13:19:27.309242 18596 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:27.337327 18835 log.cc:927] Time spent T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.176s	user 0.001s	sys 0.001s
I20260709 13:19:27.350471 10941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 3368) took 1119 ms. Trace:
I20260709 13:19:27.352545 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:27.348953 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:27.278424 18463 consensus_queue.cc:1048] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 13:19:27.357795 10870 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 24.
I20260709 13:19:27.339058 10950 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 21.
I20260709 13:19:27.363680 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:27.337616 18836 raft_consensus.cc:2804] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:19:27.349890 18843 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.365439 10941 rpcz_store.cc:276] 0709 13:19:26.230907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.231311 (+   404us) service_pool.cc:224] Handling call
0709 13:19:26.231711 (+   400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:26.308504 (+ 76793us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:19:26.362690 (+ 54186us) log.cc:844] Serialized 14 byte log entry
0709 13:19:26.715955 (+353265us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:26.716470 (+   515us) log.cc:844] Serialized 26 byte log entry
0709 13:19:26.716634 (+   164us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:26.716666 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:26.716751 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:27.350145 (+633394us) raft_consensus.cc:1685] finished
0709 13:19:27.350177 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:27.350391 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65268,"prepare.queue_time_us":115626,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":150427,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":16161,"thread_start_us":295852,"threads_started":1,"wal-append.queue_time_us":467544}
I20260709 13:19:27.355232 18793 raft_consensus.cc:697] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:27.318579 18465 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:27.360488 18692 raft_consensus.cc:697] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:27.309178 18555 consensus_queue.cc:260] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:27.388558 18732 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.752s	user 0.004s	sys 0.014s
W20260709 13:19:27.391095 18830 log.cc:927] Time spent T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.212s	user 0.001s	sys 0.001s
I20260709 13:19:27.396369 18555 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:27.404579 18778 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:27.407087 18558 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:27.385176 18843 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:27.409164 18850 log.cc:927] Time spent T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:19:27.411178 18843 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:27.415184 18692 consensus_queue.cc:237] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:27.397236 18739 consensus_queue.cc:237] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:27.444972 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.436857 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2867) took 1865 ms. Trace:
I20260709 13:19:27.459537 10872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2874) took 1833 ms. Trace:
I20260709 13:19:27.436043 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49290 (request call id 1238) took 1438 ms. Trace:
I20260709 13:19:27.480649 10869 rpcz_store.cc:276] 0709 13:19:25.571629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:25.953740 (+382111us) service_pool.cc:224] Handling call
0709 13:19:25.998777 (+ 45037us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:25.999026 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:19:26.879091 (+880065us) log.cc:844] Serialized 14 byte log entry
0709 13:19:27.213526 (+334435us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:27.213669 (+   143us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:19:27.213785 (+   116us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:27.213921 (+   136us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:27.436611 (+222690us) spinlock_profiling.cc:243] Waited 44.7 ms on lock 0x7b5400316fd0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:27.436779 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6489,"prepare.queue_time_us":630718,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":30,"raft.queue_time_us":17620,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":12294,"spinlock_wait_cycles":102760320,"thread_start_us":1195573,"threads_started":2,"wal-append.queue_time_us":396552}
I20260709 13:19:27.483644 10872 rpcz_store.cc:276] 0709 13:19:25.628083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:25.967783 (+339700us) service_pool.cc:224] Handling call
0709 13:19:26.315281 (+347498us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:26.555877 (+240596us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:27.459205 (+903328us) spinlock_profiling.cc:243] Waited 345 ms on lock 0x7b5400340150. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:27.459445 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":792819456}
I20260709 13:19:27.430626 18465 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:27.428488 18175 consensus_meta.cc:287] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.707s	user 0.010s	sys 0.002s
I20260709 13:19:27.495539 11510 raft_consensus.cc:2410] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 26 because replica has last-logged OpId of term: 25 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 18.
I20260709 13:19:27.460853 18379 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.496762 18379 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:27.500743 18379 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:27.451843 18815 log.cc:927] Time spent T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.817s	user 0.000s	sys 0.002s
W20260709 13:19:27.439159 18819 log.cc:927] Time spent T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.675s	user 0.004s	sys 0.001s
I20260709 13:19:27.482060 10867 rpcz_store.cc:276] 0709 13:19:25.997862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.065115 (+ 67253us) service_pool.cc:224] Handling call
0709 13:19:26.065404 (+   289us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:26.065731 (+   327us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:26.065768 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:26.066204 (+   436us) log.cc:844] Serialized 26 byte log entry
0709 13:19:27.098098 (+1031894us) spinlock_profiling.cc:243] Waited 8.26 ms on lock 0x7b18001ade20. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a550d1 00007f9c80e70a0f
0709 13:19:27.252654 (+154556us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:27.252700 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:27.252772 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:27.435633 (+182861us) raft_consensus.cc:1685] finished
0709 13:19:27.435665 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:27.435955 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":634,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":103,"spinlock_wait_cycles":19002752,"thread_start_us":154362,"threads_started":1,"wal-append.queue_time_us":1313204}
I20260709 13:19:27.434047 18804 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:19:27.517742 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 244722us
W20260709 13:19:27.397938 18829 consensus_queue.cc:1175] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" responder_term: 25 status { last_received { term: 23 index: 17 } last_committed_idx: 17 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 24. Current term is 25. Ops: [24.18-24.18]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:19:27.437510 18664 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:27.504469 18732 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.868s	user 0.004s	sys 0.014s
I20260709 13:19:27.519286 10841 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:27.521744 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 209623us
I20260709 13:19:27.614063 10945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 2973) took 1545 ms. Trace:
W20260709 13:19:27.546315 10942 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41020 (request call id 3345) took 2688 ms (client timeout 2149 ms). Trace:
I20260709 13:19:27.615206 18549 raft_consensus.cc:2749] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:19:27.562484 18803 log.cc:927] Time spent T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.931s	user 0.003s	sys 0.000s
W20260709 13:19:27.561347 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:27.616509 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:27.617370 10942 rpcz_store.cc:269] 0709 13:19:24.857672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:24.877996 (+ 20324us) service_pool.cc:224] Handling call
0709 13:19:27.546083 (+2668087us) spinlock_profiling.cc:243] Waited 2.05 s on lock 0x7b54000a96d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:27.546197 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4708690176}
I20260709 13:19:27.614703 10945 rpcz_store.cc:276] 0709 13:19:26.068496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.069195 (+   699us) service_pool.cc:224] Handling call
0709 13:19:26.069602 (+   407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:26.131771 (+ 62169us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:26.131806 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:26.132331 (+   525us) log.cc:844] Serialized 26 byte log entry
0709 13:19:26.508547 (+376216us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:26.508593 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:26.508670 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:27.438463 (+929793us) spinlock_profiling.cc:243] Waited 21.9 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:19:27.597221 (+158758us) raft_consensus.cc:1685] finished
0709 13:19:27.597243 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:27.613988 (+ 16745us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61729,"raft.queue_time_us":86273,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":12530,"spinlock_wait_cycles":52175488,"thread_start_us":332964,"threads_started":1,"wal-append.queue_time_us":551735}
W20260709 13:19:27.525760 10686 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 695ff1b5a68e4111917d909909488d39 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica e01ecd008edd4aa2b6b2b47efbc2ea87 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 20 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:27.619372 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:19:27.615384 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 96188us
I20260709 13:19:27.619882 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:27.620191 10945 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:19:27.620539 10841 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:27.566833 10767 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:27.433542 18793 consensus_queue.cc:237] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:27.622802 10876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 2737) took 1989 ms. Trace:
W20260709 13:19:27.622681 10686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 96993.5us
I20260709 13:19:27.626427 10841 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:27.625545 10876 rpcz_store.cc:276] 0709 13:19:25.633380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.040779 (+407399us) service_pool.cc:224] Handling call
0709 13:19:26.075197 (+ 34418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:26.075523 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:26.075551 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:26.076045 (+   494us) log.cc:844] Serialized 26 byte log entry
0709 13:19:26.474669 (+398624us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:26.474749 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:26.474849 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:27.451100 (+976251us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:19:27.617711 (+166611us) raft_consensus.cc:1685] finished
0709 13:19:27.617749 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:27.622722 (+  4973us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":651,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":36664,"spinlock_wait_cycles":28292992,"thread_start_us":322950,"threads_started":1,"wal-append.queue_time_us":479591}
W20260709 13:19:27.627254 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 60572.4us
I20260709 13:19:27.626107 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:27.628517 10767 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:19:27.545070 18794 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 23 current term: 23 is higher.
I20260709 13:19:27.527339 18804 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:19:27.638870 10942 raft_consensus.cc:1275] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:19:27.618757 18549 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:27.635552 10767 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:27.636950 18618 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:27.634874 18778 raft_consensus.cc:2749] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:27.633209 18645 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:19:27.543692 18637 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:27.662144 18645 raft_consensus.cc:697] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:27.663784 18486 raft_consensus.cc:2749] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:27.663017 18692 raft_consensus.cc:2749] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:19:27.649049 18848 log.cc:927] Time spent T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.265s	user 0.000s	sys 0.001s
W20260709 13:19:27.648592 18821 log.cc:927] Time spent T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.888s	user 0.000s	sys 0.004s
I20260709 13:19:27.630708 18379 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.654644 10767 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:27.644898 10945 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 23.
I20260709 13:19:27.637009 18723 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.651304 18688 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:27.675545 18723 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:27.677220 18688 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 28 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:27.677980 18723 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:27.664716 18778 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:27.678822 18778 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:27.654502 18549 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:27.630223 18733 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:27.672538 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:27.680980 10947 raft_consensus.cc:1240] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 27. Current term is 28. Ops: [27.19-27.19]
I20260709 13:19:27.652566 18618 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:27.696825 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:27.656510 18862 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.710029 11504 raft_consensus.cc:2393] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 25: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
W20260709 13:19:27.702675 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 64565.6us
I20260709 13:19:27.694056 18636 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:19:27.697996 18695 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.692691 18699 consensus_queue.cc:1059] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.057s
I20260709 13:19:27.707926 18618 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:19:27.713338 18731 consensus_meta.cc:287] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.818s	user 0.009s	sys 0.005s
I20260709 13:19:27.817404 10950 raft_consensus.cc:1240] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 21. Current term is 22. Ops: []
W20260709 13:19:27.827991 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 201637us
I20260709 13:19:27.819007 10795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3453) took 1677 ms. Trace:
I20260709 13:19:27.710974 18175 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:27.828688 10795 rpcz_store.cc:276] 0709 13:19:26.142034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.142179 (+   145us) service_pool.cc:224] Handling call
0709 13:19:26.142476 (+   297us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:26.142690 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:26.142715 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:26.143214 (+   499us) log.cc:844] Serialized 26 byte log entry
0709 13:19:26.581296 (+438082us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:26.581341 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:26.581420 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:27.648018 (+1066598us) spinlock_profiling.cc:243] Waited 21.3 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:19:27.811726 (+163708us) raft_consensus.cc:1685] finished
0709 13:19:27.811753 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:27.818930 (+  7177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":48998656,"thread_start_us":406860,"threads_started":1,"wal-append.queue_time_us":572941}
I20260709 13:19:27.828912 10841 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:27.811601 18636 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:27.838658 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 142541us
I20260709 13:19:27.849100 18731 consensus_queue.cc:1048] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.002s
I20260709 13:19:27.852763 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:27.854486 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:27.854885 10797 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 24.
I20260709 13:19:27.855393 10876 raft_consensus.cc:2410] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 23 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 13:19:27.696220 18448 consensus_queue.cc:1048] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.119s
I20260709 13:19:27.856966 10841 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:27.862082 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:27.703439 18862 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:27.698800 18733 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:27.663285 18645 consensus_queue.cc:237] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:27.866937 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:27.876863 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:27.877012 10876 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:19:27.872249 10867 raft_consensus.cc:1275] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:19:27.873667 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:27.874302 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:27.685145 18379 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:27.879572 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:27.883154 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:27.883803 10872 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:27.885753 10796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3461) took 1671 ms. Trace:
I20260709 13:19:27.856617 18636 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:27.886212 10796 rpcz_store.cc:276] 0709 13:19:26.214245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.214433 (+   188us) service_pool.cc:224] Handling call
0709 13:19:26.214896 (+   463us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:26.307603 (+ 92707us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:26.307640 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:26.307703 (+    63us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:26.307754 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:27.885505 (+1577751us) spinlock_profiling.cc:243] Waited 92.5 ms on lock 0x7b5400212ad4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:19:27.885554 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:27.885690 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":212683008}
W20260709 13:19:27.867949 18868 log.cc:927] Time spent T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 13:19:27.860558 18261 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.896422 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:27.873171 18576 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.855693 18692 consensus_queue.cc:1059] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.048s
I20260709 13:19:27.819522 18695 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:27.899950 18867 log.cc:927] Time spent T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.229s	user 0.001s	sys 0.000s
I20260709 13:19:27.901324 18813 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:27.902902 18861 log.cc:927] Time spent T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 13:19:27.875703 18637 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:27.907579 18379 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:27.908034 18836 raft_consensus.cc:697] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 24 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:19:27.909235 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:27.910622 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.938397 10872 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 23.
I20260709 13:19:27.942000 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:27.942677 10942 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 25.
I20260709 13:19:27.858711 18175 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:27.911062 18463 raft_consensus.cc:2749] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:19:27.911119 18732 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.913076 18664 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:19:27.913436 18865 log.cc:927] Time spent T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
I20260709 13:19:27.913645 18699 consensus_queue.cc:1059] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:19:27.915037 18555 raft_consensus.cc:3055] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:19:27.915328 10797 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2708) took 1625 ms (client timeout 1625 ms). Trace:
W20260709 13:19:28.064258 10797 rpcz_store.cc:269] 0709 13:19:26.290021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.290135 (+   114us) service_pool.cc:224] Handling call
0709 13:19:27.915134 (+1624999us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b54002111d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:27.915255 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3150296320}
I20260709 13:19:27.917546 18557 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:28.066699 18557 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:28.067454 18862 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:28.067765 18795 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:28.067997 18862 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.068419 18795 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:28.069175 18557 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:27.917951 18692 raft_consensus.cc:3055] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:19:28.075083 18692 raft_consensus.cc:740] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:27.920305 10876 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 28.
I20260709 13:19:27.944707 11477 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:19:28.076949 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 132363us
I20260709 13:19:27.897372 18261 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:28.068549 18862 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:27.943293 18723 raft_consensus.cc:2749] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:28.085300 10941 raft_consensus.cc:1275] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
W20260709 13:19:28.084659 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 369572us
W20260709 13:19:27.957664 18868 log.cc:927] Time spent T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:19:27.966848 18793 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.966274 18874 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:28.058169 18778 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:28.057040 18688 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:28.064126 18555 raft_consensus.cc:740] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:19:28.084638 18867 log.cc:927] Time spent T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
W20260709 13:19:28.108392 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.905799 18576 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:28.057770 18813 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:28.082535 18610 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:28.087128 18814 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:19:28.119165 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 262285us
I20260709 13:19:28.086329 18557 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:28.131459 10841 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:28.063509 18664 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:28.140292 10841 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:28.140018 10766 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:28.063668 18732 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:28.145992 10801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2718) took 1096 ms. Trace:
I20260709 13:19:28.151086 10801 rpcz_store.cc:276] 0709 13:19:27.049659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:27.222044 (+172385us) service_pool.cc:224] Handling call
0709 13:19:27.222460 (+   416us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:27.223047 (+   587us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 13:19:27.519335 (+296288us) log.cc:844] Serialized 14 byte log entry
0709 13:19:27.637257 (+117922us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:27.637767 (+   510us) log.cc:844] Serialized 26 byte log entry
0709 13:19:27.637969 (+   202us) raft_consensus.cc:1645] Marking committed up to 18
0709 13:19:27.638006 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:27.638079 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:28.145461 (+507382us) raft_consensus.cc:1685] finished
0709 13:19:28.145491 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:28.145905 (+   414us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24718,"prepare.queue_time_us":136145,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":2826,"raft.queue_time_us":875,"raft.run_cpu_time_us":257,"raft.run_wall_time_us":25911,"thread_start_us":388572,"threads_started":2,"wal-append.queue_time_us":161165}
I20260709 13:19:28.129472 18549 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:28.132732 18795 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:28.164443 10944 pending_rounds.cc:85] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 10
I20260709 13:19:28.164904 10944 pending_rounds.cc:107] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 22.11
I20260709 13:19:28.165202 10944 raft_consensus.cc:2889] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:28.169550 18569 raft_consensus.cc:3055] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:19:28.170064 18569 raft_consensus.cc:740] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:28.136683 18610 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:28.141598 18557 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:28.176093 18569 consensus_queue.cc:260] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:28.178092 18175 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:28.178169 10871 raft_consensus.cc:1240] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 22. Current term is 23. Ops: [21.15-22.16]
I20260709 13:19:28.178228 18569 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:28.178896 18175 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:19:28.179194 10871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2881) took 1610 ms. Trace:
I20260709 13:19:28.179301 18175 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:28.179852 18175 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:19:28.179714 10871 rpcz_store.cc:276] 0709 13:19:26.568649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.777739 (+209090us) service_pool.cc:224] Handling call
0709 13:19:28.177717 (+1399978us) spinlock_profiling.cc:243] Waited 32.5 ms on lock 0x7b5800177098. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd90a22 00007f9c86ca9c09 00007f9c85e03ee0 00007f9c85ecd0ab 00007f9c85ecd131 00007f9c85e8efc0 00007f9c85e79929 00007f9c85e793d2 00007f9c7f3f0bd4
0709 13:19:28.177919 (+   202us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:28.178908 (+   989us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:28.179137 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":74752256}
I20260709 13:19:28.182925 18870 consensus_queue.cc:1048] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:28.183729 18870 consensus_queue.cc:1243] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer cb0428cc55984daba1ba63744a5102a2 log is divergent from this leader: its last log entry 21.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
W20260709 13:19:28.185819 18854 log.cc:927] Time spent T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.798s	user 0.002s	sys 0.000s
W20260709 13:19:28.186385 18797 log.cc:927] Time spent T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 1.559s	user 0.001s	sys 0.002s
W20260709 13:19:28.188438 18864 log.cc:927] Time spent T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.483s	user 0.001s	sys 0.000s
I20260709 13:19:28.189319 10868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2864) took 2618 ms. Trace:
I20260709 13:19:28.154098 18877 consensus_queue.cc:1048] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.047s
I20260709 13:19:28.115830 18874 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:28.189755 10868 rpcz_store.cc:276] 0709 13:19:25.570612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:25.612447 (+ 41835us) service_pool.cc:224] Handling call
0709 13:19:25.665703 (+ 53256us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:25.799525 (+133822us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:25.799570 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:25.806206 (+  6636us) log.cc:844] Serialized 26 byte log entry
0709 13:19:26.219996 (+413790us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:19:26.220053 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:26.220152 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:28.185753 (+1965601us) spinlock_profiling.cc:243] Waited 17.5 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:19:28.188979 (+  3226us) raft_consensus.cc:1685] finished
0709 13:19:28.189000 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:28.189236 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":133244,"raft.queue_time_us":134178,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":132,"spinlock_wait_cycles":41173504,"thread_start_us":413315,"threads_started":1,"wal-append.queue_time_us":789958}
I20260709 13:19:28.192013 11506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56410 (request call id 1228) took 1612 ms. Trace:
I20260709 13:19:28.192723 18874 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:28.193615 18733 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:28.193782 11510 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56410 (request call id 1224) took 1616 ms (client timeout 1837 ms). Trace:
I20260709 13:19:28.139513 18836 consensus_queue.cc:237] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:28.194221 11510 rpcz_store.cc:269] 0709 13:19:26.577609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.577734 (+   125us) service_pool.cc:224] Handling call
0709 13:19:28.193537 (+1615803us) spinlock_profiling.cc:243] Waited 902 ms on lock 0x7b54005fee54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:28.193681 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2075726208}
I20260709 13:19:28.133952 18813 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:28.199884 18569 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.200453 18569 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:28.200858 18813 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:28.201835 18695 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:28.119055 18688 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:28.232199 18688 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:28.154675 18812 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 23 current term: 23 is higher.
W20260709 13:19:28.203439 18859 log.cc:927] Time spent T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.500s	user 0.002s	sys 0.000s
I20260709 13:19:28.234990 10794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2717) took 1185 ms. Trace:
I20260709 13:19:28.235505 10794 rpcz_store.cc:276] 0709 13:19:27.049389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:27.189305 (+139916us) service_pool.cc:224] Handling call
0709 13:19:27.189730 (+   425us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:27.203961 (+ 14231us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:27.204082 (+   121us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:27.204972 (+   890us) log.cc:844] Serialized 52 byte log entry
0709 13:19:27.520143 (+315171us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:27.520175 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:27.520223 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:28.202818 (+682595us) spinlock_profiling.cc:243] Waited 4.01 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:19:28.234564 (+ 31746us) raft_consensus.cc:1685] finished
0709 13:19:28.234595 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:28.234909 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3410,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":11865,"spinlock_wait_cycles":9231232,"thread_start_us":251793,"threads_started":1,"wal-append.queue_time_us":479695}
I20260709 13:19:28.204340 18664 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:28.159870 18715 raft_consensus.cc:2764] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 28 FOLLOWER]: Leader pre-election decision vote started in defunct term 27: won
I20260709 13:19:28.137071 18814 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:28.241308 18814 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:28.210166 10943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 3371) took 1120 ms. Trace:
I20260709 13:19:28.211052 18549 consensus_queue.cc:1048] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:28.192407 11506 rpcz_store.cc:276] 0709 13:19:26.579954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.580091 (+   137us) service_pool.cc:224] Handling call
0709 13:19:26.589033 (+  8942us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:26.644484 (+ 55451us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:26.644527 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:26.644573 (+    46us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:19:26.644608 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:28.191773 (+1547165us) spinlock_profiling.cc:243] Waited 55.2 ms on lock 0x7b54005ff5d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:19:28.191827 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:28.191956 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":127079296}
W20260709 13:19:28.174880 18869 log.cc:927] Time spent T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.469s	user 0.005s	sys 0.008s
I20260709 13:19:28.212894 18576 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:28.213611 10874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2884) took 1643 ms. Trace:
I20260709 13:19:28.249254 10874 rpcz_store.cc:276] 0709 13:19:26.570158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.781170 (+211012us) service_pool.cc:224] Handling call
0709 13:19:28.212890 (+1431720us) spinlock_profiling.cc:243] Waited 28.6 ms on lock 0x7b5800177098. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd90a22 00007f9c86ca9c09 00007f9c85e03ee0 00007f9c85ecd0ab 00007f9c85ecd131 00007f9c85e8efc0 00007f9c85e79929 00007f9c85e793d2 00007f9c7f3f0bd4
0709 13:19:28.213106 (+   216us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:28.213297 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:28.213324 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:28.213362 (+    38us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:19:28.213393 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:28.213444 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:28.213555 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":65775872}
I20260709 13:19:28.110766 18793 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:28.244630 10943 rpcz_store.cc:276] 0709 13:19:27.089829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:27.092082 (+  2253us) service_pool.cc:224] Handling call
0709 13:19:27.092409 (+   327us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:27.099207 (+  6798us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:27.099253 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:27.107803 (+  8550us) log.cc:844] Serialized 52 byte log entry
0709 13:19:27.258649 (+150846us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:27.380448 (+121799us) log.cc:844] Serialized 14 byte log entry
0709 13:19:27.903171 (+522723us) spinlock_profiling.cc:243] Waited 1.99 ms on lock 0x7b1800354a00. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7dd42c0a 00007f9c87482219 00007f9c7ddcaa40 00007f9c7ddc54e4 00007f9c7ddc5455 00007f9c80ead060 00007f9c80ebf716 00007f9c80ed0287 00007f9c80ed0208
0709 13:19:27.903374 (+   203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:27.903455 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:28.185176 (+281721us) spinlock_profiling.cc:243] Waited 2.01 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:19:28.209909 (+ 24733us) raft_consensus.cc:1685] finished
0709 13:19:28.209932 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:28.210101 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7225,"prepare.queue_time_us":122091,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":1496,"raft.queue_time_us":534872,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":7983,"spinlock_wait_cycles":9198208,"thread_start_us":73446,"threads_started":1,"wal-append.queue_time_us":190635}
I20260709 13:19:28.188577 18555 consensus_queue.cc:260] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:28.218279 10941 raft_consensus.cc:1275] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:19:28.219434 18531 raft_consensus.cc:2749] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 13:19:28.222318 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.217062 18674 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:19:28.253489 18862 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:28.215353 18622 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:19:28.219755 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.252065 18793 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:28.256970 18804 consensus_meta.cc:287] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.607s	user 0.012s	sys 0.000s
I20260709 13:19:28.265918 18876 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:28.272321 18622 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:28.273986 18804 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:28.274447 18876 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:28.148777 18692 consensus_queue.cc:260] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:28.272073 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:28.276949 10942 raft_consensus.cc:1217] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Deduplicated request from leader. Original: 17.10->[18.11-24.14]   Dedup: 18.11->[20.12-24.14]
I20260709 13:19:28.256367 18732 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:28.293220 10946 raft_consensus.cc:1275] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 13:19:28.266068 18637 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:28.303164 18637 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:28.300621 10941 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:28.300179 10942 pending_rounds.cc:85] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 11
I20260709 13:19:28.306517 10942 pending_rounds.cc:107] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 21.12
I20260709 13:19:28.306897 10942 raft_consensus.cc:2889] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:28.280618 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:28.260391 18639 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.265355 18555 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:28.275689 18674 raft_consensus.cc:697] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:19:28.317219 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 469112us
W20260709 13:19:28.313730 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 173522us
I20260709 13:19:28.290644 18874 consensus_queue.cc:1048] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.025s
I20260709 13:19:28.304167 18618 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:28.328119 10841 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:28.331056 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 246929us
I20260709 13:19:28.314044 18692 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:28.336817 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:28.337736 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:28.339579 10841 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:28.339815 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:28.342232 10946 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 25.
I20260709 13:19:28.358508 10841 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:28.309026 18670 consensus_queue.cc:1048] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.020s
I20260709 13:19:28.364365 10767 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:28.362957 10841 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:28.364821 11477 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:28.368583 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:28.327840 18870 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.312615 18639 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:28.370846 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:28.369915 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:28.328362 18448 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:28.372931 10767 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:28.357971 18699 consensus_queue.cc:1243] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer cb0428cc55984daba1ba63744a5102a2 log is divergent from this leader: its last log entry 22.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 13:19:28.385061 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:28.385747 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.387259 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:28.364987 18733 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:19:28.392550 10943 raft_consensus.cc:2393] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 24: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:28.331385 18531 consensus_queue.cc:1048] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.045s
W20260709 13:19:28.393535 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 130256us
I20260709 13:19:28.358734 18879 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:28.373195 18676 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:28.398571 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:28.399473 10943 raft_consensus.cc:1240] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 13:19:28.399389 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:28.375078 18873 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:28.372215 18645 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:28.405000 10868 raft_consensus.cc:2410] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 21 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:19:28.372570 18878 raft_consensus.cc:2749] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:28.377731 18622 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:28.412303 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 47524.3us
I20260709 13:19:28.416440 11506 raft_consensus.cc:1240] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 13:19:28.396448 18733 raft_consensus.cc:493] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:28.425138 18733 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:28.411972 18645 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:28.429780 18862 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.430400 18862 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:28.379280 18870 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:28.432523 18862 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:28.400789 18880 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:28.436197 18880 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:28.436317 18770 consensus_queue.cc:260] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 25.11, Last appended by leader: 11, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:28.437546 18770 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:19:28.417173 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 1.82944e+06us
I20260709 13:19:28.418936 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:28.440021 11506 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 25.
I20260709 13:19:28.442215 11478 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:28.419773 18699 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:19:28.380978 18448 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:28.414816 18873 raft_consensus.cc:697] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:28.394373 18884 consensus_queue.cc:1048] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.048s
I20260709 13:19:28.403191 18676 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:28.438570 18880 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:28.423055 18886 raft_consensus.cc:2749] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:28.412954 18569 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:28.424077 18874 raft_consensus.cc:2749] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:28.444844 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:19:28.450070 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 77224.4us
I20260709 13:19:28.421011 18557 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:28.472389 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:28.508278 11503 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
I20260709 13:19:28.497731 10841 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:19:28.423060 18636 consensus_meta.cc:287] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.524s	user 0.005s	sys 0.005s
W20260709 13:19:28.503319 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.505523 11506 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
I20260709 13:19:28.505813 10767 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:28.509990 18880 raft_consensus.cc:2749] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:19:28.447266 18637 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:28.398753 18674 consensus_queue.cc:237] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:28.511682 11504 raft_consensus.cc:1275] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:19:28.499809 18676 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:28.446753 18699 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:28.513564 10841 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:28.444473 18804 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 27 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:28.503013 18731 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.443691 18882 consensus_queue.cc:1048] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.101s
I20260709 13:19:28.495679 18695 consensus_queue.cc:1048] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.051s
I20260709 13:19:28.451597 18887 consensus_queue.cc:1059] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.065s
I20260709 13:19:28.407820 18879 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:28.529443 10941 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 20.
I20260709 13:19:28.540880 10948 raft_consensus.cc:1240] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 19. Current term is 20. Ops: []
W20260709 13:19:28.571929 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 52007.4us
W20260709 13:19:28.562804 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.523830 18874 raft_consensus.cc:2749] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:19:28.522922 18531 raft_consensus.cc:2749] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:19:28.578624 10841 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:19:28.533605 18699 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:19:28.595376 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 183222us
I20260709 13:19:28.591910 10841 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:19:28.510757 18636 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:28.524320 18731 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:28.512835 18873 consensus_queue.cc:237] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:28.538578 18876 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:28.583611 18569 raft_consensus.cc:3055] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:28.611281 10766 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:28.524559 18886 consensus_queue.cc:1059] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.067s
W20260709 13:19:28.648664 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 206530us
I20260709 13:19:28.536985 18897 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:19:28.654999 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 63207.6us
I20260709 13:19:28.670244 10841 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:19:28.600332 18862 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 13:19:28.673488 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 140307us
W20260709 13:19:28.683873 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.585150 18888 consensus_queue.cc:1059] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 13:19:28.693764 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:28.640336 18569 raft_consensus.cc:740] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:28.667829 18622 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.677102 18862 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:28.704546 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:28.672555 18898 raft_consensus.cc:2804] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:28.711907 11504 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 27.
I20260709 13:19:28.699571 18463 raft_consensus.cc:2804] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:28.699111 18884 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:28.711004 18894 log.cc:927] Time spent T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.247s	user 0.000s	sys 0.001s
I20260709 13:19:28.718505 18902 raft_consensus.cc:2804] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:28.714967 18862 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:19:28.704082 18907 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:28.702085 18531 consensus_queue.cc:1048] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.029s
I20260709 13:19:28.746979 18907 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:28.715607 18622 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:28.717500 18898 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:28.740156 18636 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 25 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:28.740245 18870 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:28.740695 18879 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:28.745821 18175 consensus_meta.cc:287] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.565s	user 0.005s	sys 0.005s
W20260709 13:19:28.745946 18839 log.cc:927] Time spent T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 1.492s	user 0.004s	sys 0.008s
I20260709 13:19:28.721105 18448 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:28.731005 18889 log.cc:927] Time spent T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.224s	user 0.001s	sys 0.001s
I20260709 13:19:28.735688 18736 raft_consensus.cc:3055] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:19:28.729696 18733 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:28.732782 18884 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:28.751391 18261 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:28.841017 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:28.757120 18175 raft_consensus.cc:515] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:28.843796 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:28.866232 10793 raft_consensus.cc:2393] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 26: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:28.850436 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:28.868059 10797 raft_consensus.cc:2393] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 20: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:28.867882 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:28.836593 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:28.868615 18175 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 26 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:28.869048 10793 raft_consensus.cc:2393] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 24: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:28.721402 18463 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:28.864543 10941 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:28.842916 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:28.848991 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:28.869992 18175 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:28.835625 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 193632us
W20260709 13:19:28.833458 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 127532us
I20260709 13:19:28.848194 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:19:28.847167 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:28.938444 18694 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.297s	user 0.002s	sys 0.021s
W20260709 13:19:28.939050 18694 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.306s	user 0.002s	sys 0.022s
I20260709 13:19:28.940315 18463 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:28.940853 18884 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:28.943529 10800 raft_consensus.cc:1240] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 22. Current term is 23. Ops: [22.11-22.11]
I20260709 13:19:28.943797 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:28.944780 10942 raft_consensus.cc:2468] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 26.
I20260709 13:19:28.947371 18531 consensus_queue.cc:1059] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:28.872036 18549 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.872741 18448 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.948662 10795 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 22.
I20260709 13:19:28.954176 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:28.869189 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:28.750129 18569 consensus_queue.cc:260] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:28.959357 10797 raft_consensus.cc:2376] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 24. Re-sending same reply.
I20260709 13:19:28.748277 18610 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:28.765008 18622 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:28.764832 18676 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:28.960613 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:28.962026 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:28.962527 18676 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:28.751803 18731 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:28.963235 18733 leader_election.cc:290] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:28.963812 10841 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.965122 18692 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:28.879153 18636 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.966114 18692 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:28.966917 18636 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:28.880687 18736 raft_consensus.cc:740] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:28.969659 18692 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:28.786612 18699 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:28.985553 18699 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 27 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:28.834951 10872 raft_consensus.cc:1275] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 28 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 13:19:28.784843 18862 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:28.989643 18862 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 27 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:28.763931 18872 raft_consensus.cc:3055] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:28.990716 18872 raft_consensus.cc:740] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:28.992319 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:28.757457 18902 raft_consensus.cc:697] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:28.841851 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:28.971465 18636 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:28.994752 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:28.995699 10944 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:19:28.869179 10943 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 21.
I20260709 13:19:28.999190 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:28.755398 18898 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:29.000347 10948 raft_consensus.cc:2410] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 26 because replica has last-logged OpId of term: 24 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 17.
I20260709 13:19:29.001104 18463 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:29.003311 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:29.004215 10795 raft_consensus.cc:2393] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 24: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:29.005538 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:29.006855 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:29.007516 10795 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 26.
I20260709 13:19:29.008478 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:29.009800 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:29.010187 18558 raft_consensus.cc:3055] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:19:29.010634 18558 raft_consensus.cc:740] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:28.992482 18872 consensus_queue.cc:260] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:29.011670 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:29.012355 18872 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:29.012054 18558 consensus_queue.cc:260] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:29.012984 10797 raft_consensus.cc:2393] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 24: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:29.013458 18558 raft_consensus.cc:3060] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:29.013973 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:29.015151 18793 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:29.016013 18793 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:29.016702 18618 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:29.019635 18793 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:29.021550 18639 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:29.023147 18645 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:29.023886 18463 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:29.032033 10797 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 20.
I20260709 13:19:29.033965 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 24 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:29.035411 10797 raft_consensus.cc:2410] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 24 because replica has last-logged OpId of term: 21 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 16.
I20260709 13:19:29.036741 18694 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.037495 18694 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:29.038028 18902 consensus_queue.cc:237] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:29.040495  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:29.040964 18694 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:29.043946 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 45430.9us
I20260709 13:19:29.047966 18576 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:29.049263 18576 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:28.749581 18907 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:28.760200 18557 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.874828 18555 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:28.955078 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:28.969417 18736 consensus_queue.cc:260] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:28.881821 18770 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:19:28.970347 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.949299 18175 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:28.951972 10796 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 21.
I20260709 13:19:28.950116 18549 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:28.950883 18448 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:28.975466 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.975696 18731 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:19:28.980986 10870 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54950 (request call id 2744) took 1923 ms (client timeout 1610 ms). Trace:
I20260709 13:19:28.975069 10841 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:19:28.986910 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:28.993108 10941 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 24.
I20260709 13:19:28.986655 10792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3452) took 2844 ms. Trace:
W20260709 13:19:29.057076 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.063900 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:29.063172 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:29.065943 18770 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:29.070169 18731 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:29.073168 18731 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:29.064628 10797 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
I20260709 13:19:29.075162 10797 raft_consensus.cc:1240] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:29.080416 10793 raft_consensus.cc:1240] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 23. Current term is 24. Ops: [23.11-23.11]
I20260709 13:19:29.081526 10947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 1413) took 1820 ms. Trace:
I20260709 13:19:29.082928 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:29.094838 10942 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 18.
W20260709 13:19:29.032044 18837 log.cc:927] Time spent T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 1.793s	user 0.000s	sys 0.004s
I20260709 13:19:29.086557 10766 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:29.105417 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:29.110383 10798 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46766 (request call id 2707) took 2858 ms (client timeout 1918 ms). Trace:
I20260709 13:19:29.061637 10792 rpcz_store.cc:276] 0709 13:19:26.141761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.142081 (+   320us) service_pool.cc:224] Handling call
0709 13:19:26.144368 (+  2287us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:26.857183 (+712815us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:26.857218 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:26.857740 (+   522us) log.cc:844] Serialized 26 byte log entry
0709 13:19:27.171000 (+313260us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:19:27.357575 (+186575us) log.cc:844] Serialized 14 byte log entry
0709 13:19:27.361048 (+  3473us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:27.648381 (+287333us) spinlock_profiling.cc:243] Waited 1.38 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
0709 13:19:28.984732 (+1336351us) spinlock_profiling.cc:243] Waited 3.53 ms on lock 0x7b1800422920. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7dd962a6 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419
0709 13:19:28.986032 (+  1300us) spinlock_profiling.cc:243] Waited 712 ms on lock 0x7b54000bccd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:19:28.986095 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:28.986170 (+    75us) raft_consensus.cc:1685] finished
0709 13:19:28.986180 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:28.986417 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3122,"prepare.queue_time_us":97967,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":7661,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":9600,"spinlock_wait_cycles":1649652224,"thread_start_us":450100,"threads_started":2,"wal-append.queue_time_us":474095,"wal-append.run_cpu_time_us":6370,"wal-append.run_wall_time_us":1612508}
W20260709 13:19:29.111035 10798 rpcz_store.cc:269] 0709 13:19:26.251630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.289864 (+ 38234us) service_pool.cc:224] Handling call
0709 13:19:29.109808 (+2819944us) spinlock_profiling.cc:243] Waited 754 ms on lock 0x7b5400211bd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:29.110097 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1735228800}
W20260709 13:19:29.111526 10686 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 695ff1b5a68e4111917d909909488d39 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica e01ecd008edd4aa2b6b2b47efbc2ea87 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 20 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:29.112784 18555 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:29.114938 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:29.115522 18555 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:29.115847 10796 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:19:29.118526 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 146370us
I20260709 13:19:29.093575 10947 rpcz_store.cc:276] 0709 13:19:27.260761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:27.260922 (+   161us) service_pool.cc:224] Handling call
0709 13:19:27.261262 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:27.686236 (+424974us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:29.080905 (+1394669us) spinlock_profiling.cc:243] Waited 419 ms on lock 0x7b540039a654. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:19:29.081286 (+   381us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":964652288}
W20260709 13:19:29.119683 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.119993 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:29.120792 10947 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Advancing to term 25
I20260709 13:19:29.123121 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:29.123503 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:29.124218 18872 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:19:29.124717 11503 raft_consensus.cc:2410] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 25 because replica has last-logged OpId of term: 24 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 18.
I20260709 13:19:29.124677 18872 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:29.125347 18872 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:19:29.126019 18557 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:29.131490 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:29.132028 11507 raft_consensus.cc:1240] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 13:19:29.132484 11508 raft_consensus.cc:2368] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 25. Current term is 26.
I20260709 13:19:29.132503 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:29.133459 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:29.133579 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:29.134356 11507 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Advancing to term 27
I20260709 13:19:29.135574 11512 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:19:29.060833 10870 rpcz_store.cc:269] 0709 13:19:27.057140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:27.166151 (+109011us) service_pool.cc:224] Handling call
0709 13:19:28.980527 (+1814376us) spinlock_profiling.cc:243] Waited 59.4 ms on lock 0x7b54000983d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:28.980742 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":136583040}
I20260709 13:19:29.134512 11508 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:29.147125 18549 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:29.147794 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.150027 18736 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:29.155452 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:29.156960 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:29.157147 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:29.158222 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:29.158505 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:29.159237 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:29.160044 10876 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:29.160880 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:29.163180 10941 raft_consensus.cc:2393] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 21: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:29.164469 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:29.167253 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:29.169015 18891 log.cc:927] Time spent T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.645s	user 0.000s	sys 0.003s
I20260709 13:19:29.170073 18175 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:29.163097 10950 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 24.
I20260709 13:19:29.172053 10950 pending_rounds.cc:85] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 17
I20260709 13:19:29.172389 10950 pending_rounds.cc:107] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 22.18
I20260709 13:19:29.172425 10800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2720) took 1859 ms. Trace:
I20260709 13:19:29.172916 10950 raft_consensus.cc:2889] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:29.173013 10800 rpcz_store.cc:276] 0709 13:19:27.312900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:27.380068 (+ 67168us) service_pool.cc:224] Handling call
0709 13:19:27.386327 (+  6259us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:28.944135 (+1557808us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:29.172053 (+227918us) spinlock_profiling.cc:243] Waited 1.56 s on lock 0x7b5400385c54. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:19:29.172329 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3579544832}
I20260709 13:19:29.090692 18770 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:29.174714 11505 raft_consensus.cc:2393] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 21: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:29.159543 11511 raft_consensus.cc:2393] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 26: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:29.123610 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:29.188714 11506 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:29.199261 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:29.200210 10944 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 26.
I20260709 13:19:29.147836 11509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56410 (request call id 1222) took 2489 ms. Trace:
I20260709 13:19:29.141831 18576 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:29.219995 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.223368 10793 raft_consensus.cc:1275] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 23 index: 10. Preceding OpId from leader: term: 26 index: 11. (index mismatch)
W20260709 13:19:29.225071 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 138778us
I20260709 13:19:29.134912 18622 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.227797 10947 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 25.
I20260709 13:19:29.233610 11507 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 27.
I20260709 13:19:29.238827 10944 raft_consensus.cc:1217] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Deduplicated request from leader. Original: 20.16->[21.17-21.17]   Dedup: 21.17->[]
I20260709 13:19:29.239430 10944 raft_consensus.cc:1240] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 21. Current term is 22. Ops: [21.17-21.17]
I20260709 13:19:29.240411 10798 raft_consensus.cc:1275] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:19:29.241390 18897 raft_consensus.cc:2749] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:29.241827 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:29.246768 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:29.249820 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:29.250588 11507 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 21.
I20260709 13:19:29.251068 18880 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:29.251624 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:29.251720 18880 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:29.252511 11507 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 22.
I20260709 13:19:29.253144 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:29.253460 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:29.254348 18880 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:29.242684 10798 raft_consensus.cc:2368] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 23. Current term is 24.
I20260709 13:19:29.254678 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:29.271080 11507 raft_consensus.cc:2393] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 27: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:29.261615 11508 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 21.
I20260709 13:19:29.256806 11512 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 24.
I20260709 13:19:29.256346 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:29.256100 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:19:29.273532 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 151334us
I20260709 13:19:29.279326 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:29.271657 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:29.284651 10766 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:29.231853 18448 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:29.290902 10796 raft_consensus.cc:2410] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 21 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:19:29.200831 11509 rpcz_store.cc:276] 0709 13:19:26.572195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.572409 (+   214us) service_pool.cc:224] Handling call
0709 13:19:26.572793 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:26.575385 (+  2592us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:19:28.177122 (+1601737us) spinlock_profiling.cc:243] Waited 4.77 ms on lock 0x7b180031e460. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc5455 00007f9c848ad3c7 00007f9c80ebf9a7 00007f9c80ecf0bf 00007f9c80ecf038
0709 13:19:28.364537 (+187415us) log.cc:844] Serialized 14 byte log entry
0709 13:19:28.665254 (+300717us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:28.665780 (+   526us) log.cc:844] Serialized 26 byte log entry
0709 13:19:28.665922 (+   142us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:19:29.045207 (+379285us) log.cc:844] Serialized 14 byte log entry
0709 13:19:29.047573 (+  2366us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:29.050636 (+  3063us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:29.051463 (+   827us) raft_consensus.cc:1685] finished
0709 13:19:29.051483 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:29.061407 (+  9924us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8723,"prepare.queue_time_us":702610,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":108,"raft.queue_time_us":82761,"raft.run_cpu_time_us":410,"raft.run_wall_time_us":13283,"spinlock_wait_cycles":12481024,"thread_start_us":723685,"threads_started":4,"wal-append.queue_time_us":293673}
I20260709 13:19:29.298525 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:29.299280 10796 raft_consensus.cc:3060] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:19:29.293792 10942 raft_consensus.cc:2468] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 24.
I20260709 13:19:29.295980 11505 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 25.
I20260709 13:19:29.307406 10793 raft_consensus.cc:2393] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 24: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:29.308338 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:29.305728 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:29.313305 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:29.314343 10793 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 18.
I20260709 13:19:29.322850 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:29.290184 18636 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:29.327198 10942 raft_consensus.cc:2368] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 23. Current term is 24.
I20260709 13:19:29.327916 11508 raft_consensus.cc:1240] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 13:19:29.327113 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:29.328583 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:29.332798 10872 raft_consensus.cc:2393] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 27: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:29.350860 10798 raft_consensus.cc:2368] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 24. Current term is 25.
I20260709 13:19:29.355690 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:29.356729 10799 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 23.
I20260709 13:19:29.357709 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:29.310778 18622 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:29.361814 18622 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:29.362164 18873 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.006s	sys 0.016s
W20260709 13:19:29.362628 18873 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.006s	sys 0.017s
I20260709 13:19:29.358709 10799 raft_consensus.cc:2393] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 25: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:29.372370 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:29.374464 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:29.374661 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:29.375347 10942 raft_consensus.cc:2368] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 25. Current term is 26.
I20260709 13:19:29.376153 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:29.376523 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:29.377841 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:29.378449 10942 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Advancing to term 27
I20260709 13:19:29.380648 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:29.382844 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:29.374465 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:29.383788 10944 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 24: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:29.375274 10946 raft_consensus.cc:1275] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 25 index: 11. Preceding OpId from leader: term: 26 index: 11. (term mismatch)
I20260709 13:19:29.387389 10946 pending_rounds.cc:85] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 10
I20260709 13:19:29.387687 10946 pending_rounds.cc:107] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 25.11
I20260709 13:19:29.387931 10946 raft_consensus.cc:2889] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:29.389797 10946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 3377) took 1021 ms. Trace:
I20260709 13:19:29.389935 18557 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:29.391214 18463 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.392038 18463 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:29.374622 18261 raft_consensus.cc:2804] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:29.395402 18261 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:29.395821 10945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36714 (request call id 1419) took 1456 ms. Trace:
I20260709 13:19:29.395897 18261 raft_consensus.cc:3060] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:29.396287 10945 rpcz_store.cc:276] 0709 13:19:27.939727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:27.939878 (+   151us) service_pool.cc:224] Handling call
0709 13:19:27.940236 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:28.213592 (+273356us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:28.213623 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:28.214081 (+   458us) log.cc:844] Serialized 26 byte log entry
0709 13:19:28.373574 (+159493us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:19:28.373621 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:29.395223 (+1021602us) spinlock_profiling.cc:243] Waited 273 ms on lock 0x7b540018ddd4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:19:29.395290 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:29.395359 (+    69us) raft_consensus.cc:1685] finished
0709 13:19:29.395371 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:29.395644 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":895,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":5615,"spinlock_wait_cycles":627849472,"thread_start_us":159181,"threads_started":1,"wal-append.queue_time_us":219099}
I20260709 13:19:29.390163 10946 rpcz_store.cc:276] 0709 13:19:28.368851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:28.394203 (+ 25352us) service_pool.cc:224] Handling call
0709 13:19:28.394748 (+   545us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:29.388467 (+993719us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:29.389531 (+  1064us) spinlock_profiling.cc:243] Waited 740 ms on lock 0x7b5400376754. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:19:29.389739 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1702475520}
I20260709 13:19:29.376722 10799 raft_consensus.cc:3060] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:29.404155 11506 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 23.
I20260709 13:19:29.408354 11509 raft_consensus.cc:1240] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 24. Current term is 25. Ops: []
I20260709 13:19:29.414939 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:19:29.419976 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.419741 11511 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 22.
I20260709 13:19:29.426614 10796 raft_consensus.cc:2468] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 27.
I20260709 13:19:29.432229 10946 raft_consensus.cc:1275] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
W20260709 13:19:29.437275 18778 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.829s	user 0.005s	sys 0.015s
W20260709 13:19:29.437744 18778 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.921s	user 0.005s	sys 0.016s
W20260709 13:19:29.438913 18885 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.891s	user 0.017s	sys 0.025s
W20260709 13:19:29.439350 18885 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.924s	user 0.017s	sys 0.025s
I20260709 13:19:29.440549 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:29.440768 10766 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:29.441522 10800 raft_consensus.cc:2393] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 24: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:29.442345 10766 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:29.444011 10766 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:19:29.445315 18922 log.cc:927] Time spent T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
W20260709 13:19:29.446537 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 66669.3us
I20260709 13:19:29.447080 18814 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.447840 11477 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:29.447715 18814 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:29.448740 18938 log.cc:927] Time spent T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 13:19:29.450239 18814 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:29.447825 10942 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 27.
I20260709 13:19:29.455102 10841 leader_election.cc:400] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 election: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 25. Current term is 26.
I20260709 13:19:29.455566 10841 leader_election.cc:403] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 election: Cancelling election due to peer responding with higher term
I20260709 13:19:29.409756 18463 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:29.461763 18618 raft_consensus.cc:2804] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:29.462327 18618 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:29.462769 18618 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:29.465312 18676 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:29.466231 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:29.466455 18817 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:19:29.467391 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:29.468166 10792 raft_consensus.cc:1240] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 LEADER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:19:29.450968 18887 consensus_queue.cc:1059] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.018s
I20260709 13:19:29.469775 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:29.470571 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:29.485093 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.470454 18448 consensus_queue.cc:1048] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:19:29.457018 18576 raft_consensus.cc:2804] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:19:29.496016 18576 raft_consensus.cc:697] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 26 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:29.497010 18576 consensus_queue.cc:237] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:29.470752 11477 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:19:29.498973 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 75006.4us
I20260709 13:19:29.471971 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:29.499517 18897 raft_consensus.cc:2804] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:19:29.500075 10871 raft_consensus.cc:2368] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for earlier term 26. Current term is 27.
I20260709 13:19:29.499922 10841 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:19:29.500305 18897 raft_consensus.cc:697] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 27 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:29.500864 10841 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:19:29.472939 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:29.501853 10873 raft_consensus.cc:2393] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 21: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:29.464728 18692 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:29.483726 10868 raft_consensus.cc:2410] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 21 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:19:29.483355 18817 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:29.491042 18674 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:19:29.493024 10867 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 25.
I20260709 13:19:29.508137 10876 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 24.
I20260709 13:19:29.503901 11512 raft_consensus.cc:1240] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 22. Current term is 23. Ops: [22.18-22.18]
I20260709 13:19:29.507855 18817 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:29.501547 18897 consensus_queue.cc:237] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:29.519912 18804 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.011s	sys 0.008s
W20260709 13:19:29.520363 18804 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.847s	user 0.011s	sys 0.008s
I20260709 13:19:29.520565 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:29.520829 18878 raft_consensus.cc:3060] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:19:29.521848 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:29.523313 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:29.524636 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:29.525331 11504 raft_consensus.cc:2376] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 22. Re-sending same reply.
I20260709 13:19:29.525485 18885 consensus_queue.cc:1048] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:19:29.526568 10796 raft_consensus.cc:1275] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 13:19:29.527872 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:19:29.530463 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:29.531334 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.433318 18636 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:29.534255 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:29.452807 18872 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:29.513022 18862 raft_consensus.cc:2804] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:29.538699 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:29.515316 18873 raft_consensus.cc:2804] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:19:29.547367 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 24 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:29.516080 18674 raft_consensus.cc:697] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:19:29.549651 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 307216us
I20260709 13:19:29.547838 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:29.551229 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:29.510327 18692 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:29.586681 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:19:29.588850 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.542932 18902 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:29.549266 18862 raft_consensus.cc:697] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:29.611440 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:29.614739 10867 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:19:29.606694 11505 raft_consensus.cc:2468] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 22.
I20260709 13:19:29.545918 18676 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:29.617937 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:29.618284 10876 raft_consensus.cc:2468] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 26.
I20260709 13:19:29.618021 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:29.639816 10946 raft_consensus.cc:2468] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 22.
I20260709 13:19:29.649037 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:19:29.636627 18898 consensus_meta.cc:287] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.538s	user 0.014s	sys 0.000s
I20260709 13:19:29.655397 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:19:29.656374 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.659188 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:29.651752 10943 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.007s	sys 0.020s
W20260709 13:19:29.671497 10943 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.007s	sys 0.021s
I20260709 13:19:29.676108 10799 raft_consensus.cc:2468] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 21.
I20260709 13:19:29.680439 10876 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Advancing to term 28
W20260709 13:19:29.683676 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:29.687578 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.691560 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:29.698465 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:29.702798 10916 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.720072 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:29.721921 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 27 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:29.713478 18674 consensus_queue.cc:237] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:29.734083 18814 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:29.735121 18814 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:29.737771 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:29.738808 11510 raft_consensus.cc:2410] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 25 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 13:19:29.741094 18558 raft_consensus.cc:3055] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 13:19:29.741677 18558 raft_consensus.cc:740] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:29.743496 18636 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:29.747288 18941 log.cc:927] Time spent T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.113s	user 0.000s	sys 0.003s
I20260709 13:19:29.750440 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:29.673695 18902 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:29.768658 18736 consensus_meta.cc:287] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.618s	user 0.005s	sys 0.007s
I20260709 13:19:29.769672 18902 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:29.761874 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:29.764147 18676 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:29.716010 18557 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.681856 18898 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:29.787783 18557 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:29.789727 18898 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:29.722688 10949 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 26.
I20260709 13:19:29.785473 10874 raft_consensus.cc:2393] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 26: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:19:29.793375 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 77599.5us
I20260709 13:19:29.790530 18557 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:29.684758 18862 consensus_queue.cc:237] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:29.783504 18622 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:29.791730 18872 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 28 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:29.810942 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:29.781421 18558 consensus_queue.cc:260] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.19, Last appended by leader: 19, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:29.834656 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:29.841815 10793 raft_consensus.cc:1275] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
W20260709 13:19:29.845507 10950 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.012s	sys 0.012s
W20260709 13:19:29.846328 10950 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.012s	sys 0.012s
I20260709 13:19:29.848783 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:19:29.843338 10797 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.014s	sys 0.007s
W20260709 13:19:29.855655 10797 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.014s	sys 0.007s
I20260709 13:19:29.855513 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:29.829458 18814 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:29.893828 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.876646 10795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46766 (request call id 2726) took 1018 ms. Trace:
I20260709 13:19:29.932314 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:29.858319 10868 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 23.
I20260709 13:19:29.822584 18622 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:29.876924 11478 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:29.953446 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.957302 10949 raft_consensus.cc:1275] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:19:29.932833 10795 rpcz_store.cc:276] 0709 13:19:28.858257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.066008 (+207751us) service_pool.cc:224] Handling call
0709 13:19:29.066431 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:29.066801 (+   370us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:19:29.329137 (+262336us) log.cc:844] Serialized 14 byte log entry
0709 13:19:29.876154 (+547017us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:29.876229 (+    75us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:19:29.876312 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:29.876401 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:29.876570 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10034,"prepare.queue_time_us":205221,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":25,"raft.queue_time_us":26857,"thread_start_us":635701,"threads_started":2}
I20260709 13:19:29.955767 10876 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 28.
I20260709 13:19:29.943569 18558 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:19:29.966121 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:29.966897 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:29.969766 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:29.974723 10868 raft_consensus.cc:2393] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 24: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:29.975395 10876 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 24.
I20260709 13:19:29.952512 18692 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:29.950181 18872 raft_consensus.cc:493] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.952512 18873 raft_consensus.cc:697] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:29.932045 18817 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:29.975363 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:19:29.975382 18933 log.cc:927] Time spent T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.170s	user 0.000s	sys 0.001s
I20260709 13:19:30.023221 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:29.969270 18804 raft_consensus.cc:2749] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:30.030629 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:30.029960 10874 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 24.
I20260709 13:19:30.027756 10943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 3012) took 1353 ms. Trace:
W20260709 13:19:30.044126 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:30.045184 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:30.038482 10915 leader_election.cc:336] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Timed out: RequestConsensusVote RPC to 127.9.116.132:40555 timed out after 1.579s (SENT)
W20260709 13:19:30.047528 10841 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.012s	sys 0.010s
W20260709 13:19:30.052224 10841 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.012s	sys 0.010s
W20260709 13:19:30.052522 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 550283us
W20260709 13:19:30.053074 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 346602us
I20260709 13:19:30.033351 10943 rpcz_store.cc:276] 0709 13:19:28.673998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:28.999275 (+325277us) service_pool.cc:224] Handling call
0709 13:19:28.999782 (+   507us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:29.090966 (+ 91184us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:29.091010 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:29.141003 (+ 49993us) log.cc:844] Serialized 26 byte log entry
0709 13:19:29.672091 (+531088us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:19:29.672156 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:29.672327 (+   171us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:30.022968 (+350641us) raft_consensus.cc:1685] finished
0709 13:19:30.022998 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:30.027658 (+  4660us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57761,"raft.queue_time_us":63189,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":37161,"thread_start_us":525218,"threads_started":1,"wal-append.queue_time_us":795544}
I20260709 13:19:30.058635 10915 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:30.061399 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:30.063359 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:30.067152 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:30.069007 10876 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 23.
I20260709 13:19:30.066636 10875 raft_consensus.cc:2393] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 28: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:19:30.073264 10766 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.116.132:40555 timed out after 2.236s (SENT) [suppressed 15 similar messages]
I20260709 13:19:30.066406 10941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41020 (request call id 3386) took 1362 ms. Trace:
I20260709 13:19:30.085180 10941 rpcz_store.cc:276] 0709 13:19:28.703612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.165520 (+461908us) service_pool.cc:224] Handling call
0709 13:19:29.165891 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:29.166246 (+   355us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:19:29.651147 (+484901us) log.cc:844] Serialized 14 byte log entry
0709 13:19:30.065963 (+414816us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:30.066026 (+    63us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:30.066094 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:30.066181 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:30.066336 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":22047,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":5570,"raft.queue_time_us":10192,"thread_start_us":847303,"threads_started":2}
I20260709 13:19:30.095425 18873 consensus_queue.cc:237] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:30.100090 10874 raft_consensus.cc:1240] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 25. Current term is 26. Ops: [25.11-25.11]
I20260709 13:19:30.102355 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:19:30.103850 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 275301us
I20260709 13:19:30.061345 18872 raft_consensus.cc:515] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:30.108954 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:30.110476 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:30.112056 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:30.113026 10941 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 21.
W20260709 13:19:30.107658 18878 consensus_meta.cc:287] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.585s	user 0.009s	sys 0.005s
I20260709 13:19:30.115381 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:30.115619 10875 raft_consensus.cc:1275] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
W20260709 13:19:30.114290 18929 log.cc:927] Time spent T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.326s	user 0.001s	sys 0.000s
W20260709 13:19:30.123091 18795 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.325s	user 0.016s	sys 0.009s
I20260709 13:19:30.151559 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:30.178261 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:30.151289 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 118655us
W20260709 13:19:30.181674 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:30.152005 18618 consensus_meta.cc:287] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.689s	user 0.013s	sys 0.002s
I20260709 13:19:30.191943 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:19:30.179490 18795 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.455s	user 0.024s	sys 0.010s
I20260709 13:19:30.196254 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:30.153369 18878 raft_consensus.cc:2749] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Leader election lost for term 25. Reason: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 25. Current term is 26.
I20260709 13:19:30.204838 10941 raft_consensus.cc:2393] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 25: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:30.208618 18817 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 25 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:30.213884 18944 log.cc:927] Time spent T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
W20260709 13:19:30.205139 18948 log.cc:927] Time spent T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
W20260709 13:19:30.221889 10915 leader_election.cc:336] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Timed out: RequestConsensusVote RPC to 127.9.116.132:40555 timed out after 1.620s (SENT)
I20260709 13:19:30.203805 18622 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:30.229477 18970 raft_consensus.cc:2749] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:30.266752 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:30.268278 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:30.268615 10946 raft_consensus.cc:1240] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 27. Current term is 28. Ops: [27.19-27.19]
I20260709 13:19:30.231195 10950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41014 (request call id 3016) took 1526 ms. Trace:
I20260709 13:19:30.269824 10950 rpcz_store.cc:276] 0709 13:19:28.704400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.171288 (+466888us) service_pool.cc:224] Handling call
0709 13:19:29.171780 (+   492us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:29.173810 (+  2030us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:29.173844 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:29.174310 (+   466us) log.cc:844] Serialized 26 byte log entry
0709 13:19:29.862571 (+688261us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:19:29.862638 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:29.862744 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:30.230928 (+368184us) raft_consensus.cc:1685] finished
0709 13:19:30.230954 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:30.231115 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":682000,"threads_started":1,"wal-append.queue_time_us":1017472}
W20260709 13:19:30.231740 18894 log.cc:927] Time spent T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.330s	user 0.000s	sys 0.002s
I20260709 13:19:30.234905 18569 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:30.266584 10870 raft_consensus.cc:1275] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
W20260709 13:19:30.236929 18956 log.cc:927] Time spent T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:19:30.237257 18913 raft_consensus.cc:2804] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 13:19:30.237943 18952 log.cc:927] Time spent T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
W20260709 13:19:30.242596 18953 log.cc:927] Time spent T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.187s	user 0.000s	sys 0.001s
I20260709 13:19:30.237985 10797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3482) took 1256 ms. Trace:
W20260709 13:19:30.243468 18950 log.cc:927] Time spent T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
I20260709 13:19:30.247313 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:30.247661 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:30.248438 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:30.248590 10766 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:30.249374 10792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46758 (request call id 3470) took 1388 ms. Trace:
W20260709 13:19:30.278968 10867 consensus_meta.cc:287] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.645s	user 0.011s	sys 0.000s
I20260709 13:19:30.279475 10867 raft_consensus.cc:2468] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 27.
W20260709 13:19:30.259223 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 82743.8us
I20260709 13:19:30.260967 18872 leader_election.cc:290] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:30.264071 18929 log.cc:927] Time spent T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:19:30.264310 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:30.197757 18618 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:30.265503 10686 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 695ff1b5a68e4111917d909909488d39 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica e01ecd008edd4aa2b6b2b47efbc2ea87 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 21 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
W20260709 13:19:30.236035 18957 log.cc:927] Time spent T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:19:30.287328 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:30.275584 10797 rpcz_store.cc:276] 0709 13:19:28.981232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.135796 (+154564us) service_pool.cc:224] Handling call
0709 13:19:29.198643 (+ 62847us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:29.199159 (+   516us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:19:29.873960 (+674801us) log.cc:844] Serialized 14 byte log entry
0709 13:19:30.226205 (+352245us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:30.236993 (+ 10788us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:30.237041 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:30.237779 (+   738us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:30.237926 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":119211,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":26,"raft.queue_time_us":284,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":123,"thread_start_us":906317,"threads_started":2,"wal-append.queue_time_us":274216}
I20260709 13:19:30.288286 10868 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 21.
I20260709 13:19:30.278469 10792 rpcz_store.cc:276] 0709 13:19:28.860673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.112101 (+251428us) service_pool.cc:224] Handling call
0709 13:19:29.112528 (+   427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:29.483996 (+371468us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:30.249075 (+765079us) spinlock_profiling.cc:243] Waited 347 ms on lock 0x7b54003cd554. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:19:30.249311 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":798124928}
I20260709 13:19:30.222500 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:30.282907 10874 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:30.284216 10873 raft_consensus.cc:1240] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 26. Current term is 27. Ops: []
W20260709 13:19:30.271596 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 494719us
I20260709 13:19:30.298818 18618 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:30.303231 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:30.303944 10868 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 23.
W20260709 13:19:30.304404 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 55883.1us
I20260709 13:19:30.307687 10874 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 25.
I20260709 13:19:30.330241 10766 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:30.313110 18618 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:19:30.340117 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:30.347241 10868 raft_consensus.cc:1240] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:30.327776 18576 consensus_queue.cc:1048] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.017s
I20260709 13:19:30.353590 18882 raft_consensus.cc:2804] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:19:30.343976 18618 raft_consensus.cc:697] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:30.357220 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:30.360037 10874 raft_consensus.cc:2468] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 24.
I20260709 13:19:30.357122 10916 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:30.365572 18622 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:19:30.366194 18622 raft_consensus.cc:697] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 24 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:30.367357 18622 consensus_queue.cc:237] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:30.360826 18942 log.cc:927] Time spent T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.488s	user 0.000s	sys 0.003s
I20260709 13:19:30.356510 18882 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:19:30.372407 18951 log.cc:927] Time spent T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.311s	user 0.001s	sys 0.001s
I20260709 13:19:30.372648 18882 raft_consensus.cc:3060] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:30.373376 10916 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:30.375919 18974 raft_consensus.cc:2749] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:30.377084 10944 raft_consensus.cc:1275] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 13:19:30.379168 18898 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:30.379856 18898 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:30.382148 18898 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:30.383378 18618 consensus_queue.cc:237] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:30.384395 18463 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:19:30.384867 18949 log.cc:927] Time spent T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
I20260709 13:19:30.385092 10841 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:30.385360 18692 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:19:30.386243 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:30.385975 18692 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:30.387787 18958 consensus_queue.cc:1059] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:30.388537 18692 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:30.388839 11478 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:30.390031 18943 log.cc:927] Time spent T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.197s	user 0.001s	sys 0.001s
I20260709 13:19:30.385092 18463 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:30.391055 18959 raft_consensus.cc:3055] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:30.391475 18959 raft_consensus.cc:740] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:30.392508 18695 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:19:30.392438 18959 consensus_queue.cc:260] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:30.393006 18463 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:30.389032 18958 raft_consensus.cc:2749] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:19:30.387161 10841 leader_election.cc:400] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 election: Vote denied by peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553) with higher term. Message: Invalid argument: T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 23. Current term is 24.
I20260709 13:19:30.393873 18959 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:30.388863 18913 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:30.394279 10841 leader_election.cc:403] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
I20260709 13:19:30.379884 10869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2904) took 1980 ms. Trace:
I20260709 13:19:30.398085 10841 leader_election.cc:400] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 election: Vote denied by peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553) with higher term. Message: Invalid argument: T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 25. Current term is 26.
I20260709 13:19:30.394829 18913 raft_consensus.cc:3060] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:19:30.400835 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:30.402096 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:30.402989 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:30.403107 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 27 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:30.403231 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:30.403836 10876 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Advancing to term 25
I20260709 13:19:30.404776 18872 raft_consensus.cc:2749] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553) with higher term. Message: Invalid argument: T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 23. Current term is 24.
I20260709 13:19:30.405195 18674 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.405524 18872 consensus_queue.cc:1048] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:19:30.405751 18674 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:30.408119 18674 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:30.393131 18695 raft_consensus.cc:697] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 23 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:30.394070 11478 leader_election.cc:400] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 26 pre-election: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for earlier term 26. Current term is 27.
I20260709 13:19:30.412395 11478 leader_election.cc:403] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 26 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:30.411958 10766 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:30.416846 10876 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 25.
W20260709 13:19:30.411939 18261 consensus_meta.cc:287] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 1.015s	user 0.012s	sys 0.006s
I20260709 13:19:30.425143 18576 raft_consensus.cc:2749] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Leader pre-election lost for term 26. Reason: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for earlier term 26. Current term is 27.
I20260709 13:19:30.427125 18695 consensus_queue.cc:237] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:30.428893 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 55607.2us
I20260709 13:19:30.431150 10766 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:30.431507 10841 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:30.432379 18674 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:19:30.432761 18898 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:30.433184 18898 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:30.433306 18694 consensus_queue.cc:1048] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:19:30.432816 18674 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:30.434583 18674 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:19:30.423740 18261 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:30.441677 18261 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:30.433604 18898 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:30.420372 18817 raft_consensus.cc:2804] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:30.443359 18817 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:30.443631 10915 leader_election.cc:336] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Timed out: RequestConsensusVote RPC to 127.9.116.132:40555 timed out after 1.727s (SENT)
I20260709 13:19:30.443768 18817 raft_consensus.cc:3060] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:30.448146 10842 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:30.451259 18463 raft_consensus.cc:2749] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:30.433604 10916 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:30.482038 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 26 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:30.490490 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:30.493036 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ae8de979e44a2d8c155bffc20a1533" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 26 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:30.473533 18970 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:30.485479 10842 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:30.496711 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:30.496795 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 63283us
I20260709 13:19:30.501480 10868 raft_consensus.cc:1275] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 13:19:30.511852 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:30.505718 10916 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:30.524878 10947 raft_consensus.cc:1275] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 13:19:30.526104 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:30.536377 10942 raft_consensus.cc:1275] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:19:30.506613 18463 raft_consensus.cc:2749] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:30.539529 10841 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:30.497826 18882 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:30.550940 10841 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:30.523684 18736 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.559787 10916 leader_election.cc:400] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 24. Current term is 25.
I20260709 13:19:30.591809 10916 leader_election.cc:403] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:30.566495 10872 raft_consensus.cc:1275] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 12. (term mismatch)
I20260709 13:19:30.398141 10869 rpcz_store.cc:276] 0709 13:19:28.399393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:28.399528 (+   135us) service_pool.cc:224] Handling call
0709 13:19:28.399814 (+   286us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:29.442832 (+1043018us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:29.442888 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:29.443493 (+   605us) log.cc:844] Serialized 26 byte log entry
0709 13:19:29.859187 (+415694us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:29.957989 (+ 98802us) log.cc:844] Serialized 14 byte log entry
0709 13:19:30.374429 (+416440us) spinlock_profiling.cc:243] Waited 2.92 ms on lock 0x7b18004b1540. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7dd42c0a 00007f9c87482219 00007f9c7ddcaa40 00007f9c7ddc54e4 00007f9c7ddc5455 00007f9c80ead060 00007f9c80ebf716 00007f9c80ed0287 00007f9c80ed0208
0709 13:19:30.374630 (+   201us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:30.379270 (+  4640us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b540006c7d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb5e01 00007f9c80eb483e 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6
0709 13:19:30.379343 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:30.379423 (+    80us) raft_consensus.cc:1685] finished
0709 13:19:30.379447 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:30.379636 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1535,"prepare.queue_time_us":98875,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":14759,"raft.queue_time_us":433043,"raft.run_cpu_time_us":262,"raft.run_wall_time_us":16360,"spinlock_wait_cycles":2404146688,"thread_start_us":415210,"threads_started":1,"wal-append.queue_time_us":630553}
I20260709 13:19:30.592433 10872 pending_rounds.cc:85] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 11
I20260709 13:19:30.568964 18882 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 25 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:30.571760 11503 raft_consensus.cc:2410] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 21 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 13:19:30.575750 18674 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:30.575397 10693 catalog_manager.cc:5697] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 16 to 20, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 20 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:30.577975 10873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2914) took 1094 ms. Trace:
W20260709 13:19:30.579382 18978 log.cc:927] Time spent T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
I20260709 13:19:30.606575 10873 rpcz_store.cc:276] 0709 13:19:29.483775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.844862 (+361087us) service_pool.cc:224] Handling call
0709 13:19:30.282151 (+437289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:30.289687 (+  7536us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:30.577771 (+288084us) spinlock_profiling.cc:243] Waited 405 ms on lock 0x7b5400316fd0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb48ad 00007f9c85e79419 00007f9c7f3f0bd4 00007f9c7f3f0b62 00007f9c7f3f0a91 00007f9c7f3f0a0c 00007f9c7f3efcc2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:30.577912 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":930457472}
I20260709 13:19:30.587646 18958 raft_consensus.cc:2749] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:30.592710 10872 pending_rounds.cc:107] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 19.12
I20260709 13:19:30.612202 10872 raft_consensus.cc:2889] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:30.593813 10841 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:19:30.613345 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 73902.1us
I20260709 13:19:30.540021 18795 raft_consensus.cc:2804] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:19:30.602763 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 121373us
I20260709 13:19:30.606410 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:30.591012 18692 raft_consensus.cc:2749] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 13:19:30.616915 18622 raft_consensus.cc:2749] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 24. Current term is 25.
I20260709 13:19:30.618714 10841 leader_election.cc:304] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:30.614277 10916 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:30.639181 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:30.639891 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 104680us
I20260709 13:19:30.617277 18674 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 27 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:30.653774 11503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56416 (request call id 1205) took 1489 ms. Trace:
I20260709 13:19:30.623505 18549 raft_consensus.cc:2749] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:19:30.659581 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:30.620294 18795 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:30.655726 11503 rpcz_store.cc:276] 0709 13:19:29.153861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.252844 (+ 98983us) service_pool.cc:224] Handling call
0709 13:19:30.642974 (+1390130us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b54004290d4. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c80e1e571 00007f9c80eb92fa 00007f9c85e7ab17 00007f9c7f3f56d4 00007f9c7f3f5662 00007f9c7f3f5591 00007f9c7f3f550c 00007f9c7f3f47c2 00007f9c7f1f048c 00007f9c7f1ef8c6 00007f9c7f1ef26f
0709 13:19:30.643165 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3030387584}
I20260709 13:19:30.613898 18576 consensus_queue.cc:1048] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.035s
I20260709 13:19:30.663635 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:30.659051 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:30.660638 10842 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:30.636205 18882 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:30.610276 18817 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:30.648684 18463 raft_consensus.cc:2749] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:19:30.628064 18736 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:30.657238 18557 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:19:30.685870 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:30.668352 18795 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:30.672827 18878 raft_consensus.cc:2749] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:30.692022 18902 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.697105 18622 consensus_queue.cc:1048] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 13:19:30.707381 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 27 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:19:30.700177 10766 leader_election.cc:336] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Timed out: RequestConsensusVote RPC to 127.9.116.132:40555 timed out after 1.505s (SENT)
I20260709 13:19:30.717394 11503 raft_consensus.cc:1275] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
W20260709 13:19:30.705170 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 60647.6us
I20260709 13:19:30.718154 11504 raft_consensus.cc:1275] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 13:19:30.712248 18736 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:30.731400 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:30.701409 18971 consensus_queue.cc:1048] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.048s
I20260709 13:19:30.732976 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:30.734283 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:30.721230 18902 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:30.735563 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:19:30.700874 18986 log.cc:927] Time spent T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:19:30.677929 18974 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.737514 18974 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:30.745262 10795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:30.745594 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:30.745970 10795 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 25.
I20260709 13:19:30.746970 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:30.747072 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:30.747428 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:30.683365 18882 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:30.751060 18882 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:30.735142 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:30.751605 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:19:30.703862 18982 log.cc:927] Time spent T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 13:19:30.704500 18984 log.cc:927] Time spent T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 13:19:30.691524 18557 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:30.757712 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 27 candidate_status { last_received { term: 25 index: 11 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:30.759441 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 27 candidate_status { last_received { term: 25 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:30.762339 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:30.764460 10873 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 25.
I20260709 13:19:30.765852 10693 catalog_manager.cc:5697] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 11 to 13, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 13 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:30.762974 10797 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 21.
I20260709 13:19:30.720675 18817 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:30.768597 10792 raft_consensus.cc:2393] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 27: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
W20260709 13:19:30.780027 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:30.720003 18898 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:30.788370 10693 catalog_manager.cc:5697] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 13 to 14, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 14 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:19:30.760246 18882 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.763875 18557 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:19:30.798589 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 55147.4us
I20260709 13:19:30.767385 18795 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:30.804491 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 43947.3us
I20260709 13:19:30.800047 18882 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:30.807299 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 112202us
I20260709 13:19:30.808310 18878 consensus_queue.cc:1048] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:19:30.809527 18882 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:30.807484 18873 consensus_queue.cc:1059] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: INVALID_TERM, Last received: 21.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 13:19:30.810509 18873 raft_consensus.cc:3055] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:19:30.810941 18873 raft_consensus.cc:740] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:30.811832 18873 consensus_queue.cc:260] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.11, Last appended by leader: 11, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:30.813004 18873 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:19:30.816584 18991 log.cc:927] Time spent T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.101s	user 0.002s	sys 0.000s
I20260709 13:19:30.820506 18974 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:30.763602 18913 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:30.800590 10916 connection.cc:256] RPC call timeout handler was delayed by -1.2190518659999725s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17210000000000003s for a response
W20260709 13:19:30.844861 10916 connection.cc:256] RPC call timeout handler was delayed by -1.0759340289998818s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17520000000000002s for a response
I20260709 13:19:30.824568 18898 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:30.826038 10872 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 21.
I20260709 13:19:30.826321 18795 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:30.827823 11478 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:19:30.831358 10915 leader_election.cc:336] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Timed out: RequestConsensusVote RPC to 127.9.116.132:40555 timed out after 1.889s (SENT)
W20260709 13:19:30.834503 18988 log.cc:927] Time spent T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
I20260709 13:19:30.844419 10941 raft_consensus.cc:1240] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:30.849851 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:30.852619 18862 consensus_queue.cc:1048] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:19:30.852856 18618 consensus_queue.cc:1048] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:19:30.855502 18463 consensus_queue.cc:1059] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:19:30.856489 18913 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 27 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:30.856621 18898 raft_consensus.cc:3055] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:30.857030 18898 raft_consensus.cc:740] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:30.857934 18898 consensus_queue.cc:260] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:30.859179 18898 raft_consensus.cc:3060] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:30.835979 18970 raft_consensus.cc:2749] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:30.864269 10842 leader_election.cc:400] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 election: Vote denied by peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685) with higher term. Message: Invalid argument: T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 23. Current term is 24.
I20260709 13:19:30.865216 10842 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:30.865808 10947 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 21.
I20260709 13:19:30.836418 18902 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:30.872046 10841 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:19:30.883136 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:30.890300 10842 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:30.901727 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 73995.6us
I20260709 13:19:30.901573 18814 consensus_queue.cc:1048] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:19:30.902714 11478 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:19:30.903748 18175 raft_consensus.cc:2804] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:19:30.903792 10945 raft_consensus.cc:1275] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:19:30.904490 18175 raft_consensus.cc:697] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 27 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:30.905444 18175 consensus_queue.cc:237] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:30.909925 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:30.862738 18576 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:19:30.901783 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:30.912381 10946 raft_consensus.cc:1240] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 23. Current term is 24. Ops: []
I20260709 13:19:30.910686 10944 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 21.
I20260709 13:19:30.916781 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:30.931826 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:30.887661 18463 raft_consensus.cc:2749] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:30.935034 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:30.936498 10873 raft_consensus.cc:3060] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:19:30.850219 18674 consensus_queue.cc:1048] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.076s
I20260709 13:19:30.939678 10871 raft_consensus.cc:1275] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:19:30.949982 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:30.951473 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:30.910336 18958 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:30.960980 10868 raft_consensus.cc:2393] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 24: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:30.942706 10870 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 23.
I20260709 13:19:30.954380 10867 raft_consensus.cc:1275] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 13:19:30.961257 10916 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:30.978322 18987 log.cc:927] Time spent T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.274s	user 0.002s	sys 0.001s
I20260709 13:19:30.978433 10796 raft_consensus.cc:1240] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 21. Current term is 22. Ops: [21.17-21.17]
I20260709 13:19:30.975391 18695 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.942139 18872 consensus_queue.cc:1048] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.019s
I20260709 13:19:30.986969 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:30.979460 18695 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:30.978917 18463 consensus_queue.cc:1048] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.004s
I20260709 13:19:30.994827 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:19:30.987063 19000 log.cc:927] Time spent T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 13:19:30.949541 18692 raft_consensus.cc:2749] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:30.993515 18958 raft_consensus.cc:697] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:31.004437 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:31.003823 18622 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:31.020064 18622 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:31.020372 10796 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 25.
I20260709 13:19:31.026386 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:31.003562 18974 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:31.030140 10793 raft_consensus.cc:2393] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 23: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:31.058521 18958 consensus_queue.cc:237] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:31.096503 18985 log.cc:927] Time spent T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.395s	user 0.000s	sys 0.005s
W20260709 13:19:31.102351 18993 log.cc:927] Time spent T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.404s	user 0.005s	sys 0.009s
W20260709 13:19:31.102978 18992 log.cc:927] Time spent T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.366s	user 0.005s	sys 0.000s
W20260709 13:19:31.119520 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 78373.7us
I20260709 13:19:31.120532 10766 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:31.122771 18882 raft_consensus.cc:2749] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:19:31.172878 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 177098us
I20260709 13:19:31.174286 11478 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:31.175350 18558 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:31.175848 18558 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:31.176267 18558 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:19:31.176472 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 294004us
I20260709 13:19:31.192678 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:19:31.195719 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 264210us
I20260709 13:19:31.197742 10941 raft_consensus.cc:1275] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 13:19:31.197983 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:31.198642 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
W20260709 13:19:31.210914 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 250432us
I20260709 13:19:31.211272 10793 raft_consensus.cc:2393] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 21: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:31.212852 10916 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:31.214144 10916 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:31.214340 18814 raft_consensus.cc:2749] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:19:31.214983 10839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:31.214948 18974 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:31.215541 18974 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:31.215926 18974 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:31.216212 18175 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.216955 18175 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:31.218923 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:31.223511 18695 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:31.224197 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 103790us
I20260709 13:19:31.236891 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:31.240844 10794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:31.195225 18622 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:31.231593 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:31.252293 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:31.254242 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:31.255026 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" is_pre_election: true
I20260709 13:19:31.255628 10792 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 22.
I20260709 13:19:31.256966 10766 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:31.261198 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87"
I20260709 13:19:31.256726 10916 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:31.258483 10946 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 28.
I20260709 13:19:31.228924 18558 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:31.266436 10767 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:19:31.286463 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 112292us
I20260709 13:19:31.296051 18379 consensus_queue.cc:1048] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 13:19:31.300165 11478 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:31.303066 18817 raft_consensus.cc:2749] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
/apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:278: Failure
Failed
Bad status: Illegal state: leader_transfer_task size should be 0, but 5
/apache/dev/git/kudu/src/kudu/util/test_util.cc:402: Failure
Failed
Timed out waiting for assertion to pass.
I20260709 13:19:31.306480  9682 tablet_server.cc:179] TabletServer@127.9.116.129:0 shutting down...
I20260709 13:19:31.307912 18558 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:31.308703 11477 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:31.309559 18795 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.310099 18795 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:31.283381 18674 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:31.292963 18814 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 13:19:31.329861 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:31.312932 18795 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:31.337007 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:31.339278 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:31.341113 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:31.347338 10874 raft_consensus.cc:2468] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 26.
I20260709 13:19:31.349856 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 27 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:31.353863 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:31.355379 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:31.355995 10868 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 21.
I20260709 13:19:31.358217 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:31.359247 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:31.347611 10842 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:31.359900 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:31.340050 10875 raft_consensus.cc:2376] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 24. Re-sending same reply.
I20260709 13:19:31.322175 18558 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 13:19:31.371209 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 27 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:31.372587 10916 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:31.324597 18674 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:31.379074 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:31.377327 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:31.397017 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:31.363086 18557 consensus_meta.cc:287] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.554s	user 0.015s	sys 0.011s
W20260709 13:19:31.423693 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 44520.1us
I20260709 13:19:31.293403 18902 consensus_queue.cc:1048] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 13:19:31.413496 11478 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:31.379933 10875 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 19.
I20260709 13:19:31.327394 18695 raft_consensus.cc:2764] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
W20260709 13:19:31.415234 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 117274us
I20260709 13:19:31.323017 18692 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:19:31.439848 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 115292us
W20260709 13:19:31.440689 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 174381us
I20260709 13:19:31.443668 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:31.445115 10870 raft_consensus.cc:1275] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:19:31.377753 18558 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:31.383826 18674 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:31.450695 10871 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 23.
I20260709 13:19:31.446758 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:31.462837 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 115340us
I20260709 13:19:31.447249 10943 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 25.
I20260709 13:19:31.444689 18557 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:31.456348 10916 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:31.414520 18814 raft_consensus.cc:697] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 28 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:31.439007 18970 raft_consensus.cc:2749] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:31.466406 10869 raft_consensus.cc:1275] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
W20260709 13:19:31.462901 10766 leader_election.cc:336] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Timed out: RequestConsensusVote RPC to 127.9.116.132:40555 timed out after 1.861s (SENT)
I20260709 13:19:31.464248 10842 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:31.429092 18898 raft_consensus.cc:2749] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:31.480326 10766 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:31.448276 18175 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:31.467808 19009 log.cc:927] Time spent T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:19:31.459432 18558 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:19:31.494236 10842 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:31.502184 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:31.497175 18959 raft_consensus.cc:2749] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:31.504350 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:31.505019 10868 raft_consensus.cc:2376] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 25. Re-sending same reply.
I20260709 13:19:31.502542 18974 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:31.483435 19004 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.510905 19004 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:31.511312 18814 consensus_queue.cc:237] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:31.491973 18795 raft_consensus.cc:2749] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:31.483361 18261 consensus_queue.cc:1048] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.032s
W20260709 13:19:31.516361 19014 log.cc:927] Time spent T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
I20260709 13:19:31.517252 10916 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:31.498577 18379 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:19:31.514721 11477 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:31.519487 18804 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:19:31.519915 18804 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:31.520257 18804 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:19:31.522111 18618 consensus_queue.cc:1059] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:19:31.481252 18569 raft_consensus.cc:2749] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:19:31.528961 10873 consensus_meta.cc:287] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.574s	user 0.007s	sys 0.007s
I20260709 13:19:31.526319 10767 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:31.528707 10842 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:19:31.547600 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:31.549410 10842 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:31.550093 10767 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:31.550906 10873 raft_consensus.cc:1275] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 13:19:31.468480 18692 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:31.557344 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 57931.2us
I20260709 13:19:31.557633 10767 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:31.558334 10916 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:31.563781 19012 log.cc:927] Time spent T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 13:19:31.564678 18872 raft_consensus.cc:2749] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:31.564968 10842 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:31.566416 10842 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:31.567050 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 104211us
W20260709 13:19:31.587493 19008 log.cc:927] Time spent T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.175s	user 0.000s	sys 0.000s
I20260709 13:19:31.585939 10767 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:31.588428 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:31.589654 18913 raft_consensus.cc:2749] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:19:31.591217 19006 log.cc:927] Time spent T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.263s	user 0.000s	sys 0.000s
I20260709 13:19:31.580583 18970 raft_consensus.cc:2804] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:19:31.581284 18576 consensus_meta.cc:287] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.650s	user 0.008s	sys 0.011s
I20260709 13:19:31.581511 10916 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:31.581789 18692 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:31.531844 18974 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:31.541043 18898 raft_consensus.cc:2749] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:31.557106 18379 raft_consensus.cc:2804] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:19:31.582248 18817 raft_consensus.cc:2749] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:31.582485 18923 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.582999 18618 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:31.550698 19016 raft_consensus.cc:2749] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:31.582023 18549 raft_consensus.cc:2749] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:31.556504 18959 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:31.600281 18576 raft_consensus.cc:2749] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:19:31.592599 18804 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:31.596557 18970 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:31.599004 18873 raft_consensus.cc:2749] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:31.600266 18622 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:31.600508 18379 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:31.617497 18970 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:31.618260 18557 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:31.609283 18959 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:31.620317 18379 raft_consensus.cc:3060] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:31.634158 10916 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586
W20260709 13:19:31.630548 10686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 248327us
I20260709 13:19:31.623389 18959 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:31.609607 18923 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:31.638556 10916 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:19:31.641575 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 260207us
I20260709 13:19:31.663228 10868 raft_consensus.cc:1240] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 24. Current term is 25. Ops: []
I20260709 13:19:31.641546 19016 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:19:31.670377 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:31.671893 10873 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 20.
I20260709 13:19:31.671442 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:31.627311 18557 consensus_queue.cc:1048] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.022s
I20260709 13:19:31.674116 10867 raft_consensus.cc:1240] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 28 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 27. Current term is 28. Ops: [27.17-27.17]
I20260709 13:19:31.674813 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:31.674870 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:31.676045 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:31.676154 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a1fc8ca1b34260908d185a9ebc2bb3" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:31.679785 10876 raft_consensus.cc:2376] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 24. Re-sending same reply.
W20260709 13:19:31.681450 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:31.691495 11478 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:31.669932 18974 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:31.703359 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:31.704315 10867 raft_consensus.cc:2410] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 22 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:19:31.705166 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:31.707005 10916 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:31.671281 19016 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:31.715785 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 27 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:31.695497 18576 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:31.699980 18974 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:31.730330 10869 raft_consensus.cc:2393] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 22: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:19:31.731611 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 146293us
I20260709 13:19:31.732999 10767 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:31.715950 19016 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:19:31.724448 18576 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:31.723218 18814 raft_consensus.cc:2749] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:19:31.753086 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 83729.5us
I20260709 13:19:31.730901 18974 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:19:31.759378 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 52505.4us
I20260709 13:19:31.734735 10874 raft_consensus.cc:2468] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 19.
W20260709 13:19:31.756598 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 158087us
I20260709 13:19:31.767220 10916 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:31.768147 18557 raft_consensus.cc:2749] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:31.768584 10916 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:31.769552 18557 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:19:31.769706 10916 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:31.769994 18557 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:31.770404 18557 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:19:31.770982 10916 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:31.755184 18576 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:31.727488 18674 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:31.775063 18804 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 26 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:31.798321 18622 raft_consensus.cc:2749] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:31.793879 18795 raft_consensus.cc:2749] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:31.831229 18923 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:31.836611 18674 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:31.837836 11478 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:31.842819 10686 catalog_manager.cc:4755] TS e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): ChangeConfig:REMOVE_PEER RPC failed for tablet 695ff1b5a68e4111917d909909488d39: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:31.858091 11478 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:31.847360 18804 raft_consensus.cc:2749] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:19:31.852087 19011 log.cc:927] Time spent T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.313s	user 0.000s	sys 0.001s
I20260709 13:19:31.852039 19017 raft_consensus.cc:3055] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:31.861332 18694 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.870124 10948 raft_consensus.cc:1240] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 23. Current term is 24. Ops: []
W20260709 13:19:31.871213 19027 log.cc:927] Time spent T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:19:31.851910 19036 consensus_queue.cc:1048] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.069s
I20260709 13:19:31.854001 19028 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:19:31.885329 19028 raft_consensus.cc:697] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:31.886736 19028 consensus_queue.cc:237] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:31.893064 19025 log.cc:927] Time spent T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:19:31.897383 18379 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:31.899514 19031 log.cc:927] Time spent T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:19:31.867028 18569 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:31.879053 18694 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:31.913551 18694 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:31.914567 18379 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 25 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:31.928436 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 157509us
I20260709 13:19:31.869982 19017 raft_consensus.cc:740] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:31.929934 10916 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:31.922240 18970 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:31.932974 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 151925us
I20260709 13:19:31.928985 19004 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:31.936806 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 83502.4us
I20260709 13:19:31.936311 19033 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:19:31.942134 19005 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:31.944525 19004 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.953593 18913 consensus_queue.cc:1048] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 13:19:31.950356 18959 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:31.958133 11478 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:31.940459 18558 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:31.951292 19005 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:31.967803 10767 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:31.940129 18576 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:31.945130 18817 consensus_queue.cc:1059] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 26.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.017s
I20260709 13:19:31.980170 10767 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:31.960857 18970 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:31.967377 11478 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:31.966071 18804 raft_consensus.cc:2749] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:31.967267 18974 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:31.954811 19017 consensus_queue.cc:260] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:31.973152 19041 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:19:31.962121 10842 consensus_peers.cc:597] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:31.973721 19005 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:31.974953 18913 raft_consensus.cc:2749] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:19:31.962206 19004 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:31.977442 18558 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 29 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:31.973495 18878 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:32.005228 18804 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:32.015170 10767 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:32.015514 18804 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:32.017549 18804 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:32.006559 18974 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:32.007009 19017 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:32.020098 10767 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:32.021023 18618 raft_consensus.cc:2749] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:19:32.021750 18618 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:19:32.008553 11478 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:31.987311 18817 raft_consensus.cc:3055] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 LEADER]: Stepping down as leader of term 26
W20260709 13:19:31.996861 10915 leader_election.cc:336] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Timed out: RequestConsensusVote RPC to 127.9.116.132:40555 timed out after 1.515s (SENT)
I20260709 13:19:31.999320 18557 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:32.011368 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:32.012688 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:31.996385 18959 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:31.986336 18576 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:32.009518 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:32.009819 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
W20260709 13:19:32.011754 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 49694us
I20260709 13:19:32.026170 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:32.031641 10945 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 22.
I20260709 13:19:32.035166 11506 raft_consensus.cc:2376] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 23. Re-sending same reply.
I20260709 13:19:32.021889 10767 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:32.007128 18692 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:32.041133 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:32.041393 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:32.041132 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:19:32.043061 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 108977us
I20260709 13:19:32.047024 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:32.047124 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:32.048210 11512 raft_consensus.cc:2410] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 23 because replica has last-logged OpId of term: 21 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 16.
I20260709 13:19:32.048444 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:32.050376 11505 raft_consensus.cc:1240] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 27 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 26. Current term is 27. Ops: []
I20260709 13:19:32.052803 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:19:32.055680 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 58842.9us
I20260709 13:19:32.013335 18913 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 13:19:32.012714 19004 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:32.061769 10946 raft_consensus.cc:1240] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:32.026449 19005 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:32.007872 19021 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:19:32.032125 18804 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:32.001672 19033 raft_consensus.cc:2749] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:19:32.073416 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:32.074218 11506 raft_consensus.cc:2393] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 21: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:32.066191 11504 raft_consensus.cc:2393] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 26: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:32.023290 18898 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:32.026741 18817 raft_consensus.cc:740] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:32.068602 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:32.068676 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:32.070343 10950 raft_consensus.cc:2393] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 26: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
W20260709 13:19:32.072275 10915 leader_election.cc:336] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Timed out: RequestConsensusVote RPC to 127.9.116.132:40555 timed out after 1.661s (SENT)
I20260709 13:19:32.072413 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:32.078213 10915 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:32.078593 11512 raft_consensus.cc:2410] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 25 because replica has last-logged OpId of term: 24 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 18.
I20260709 13:19:32.078359 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:19:32.072614 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 50805.5us
I20260709 13:19:32.080119 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:32.081425 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:32.079684 10767 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:32.086246 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:32.075773 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:32.087069 10874 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:32.014318 18878 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:32.089610 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:32.091853 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:32.096948 10949 raft_consensus.cc:2410] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 23 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 13:19:32.092582 11505 raft_consensus.cc:2393] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 22: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:32.088094 10916 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:32.092792 11507 raft_consensus.cc:1275] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:19:32.095220 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:32.099659 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:32.100708 10916 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:32.071851 19021 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Starting leader election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:32.073925 18882 raft_consensus.cc:2749] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 13:19:32.111719 11509 raft_consensus.cc:1275] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 13:19:32.104542 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:32.114636 11506 raft_consensus.cc:2410] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 23 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 13:19:32.010650 19016 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:32.079789 18898 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:32.059481 18569 consensus_queue.cc:1059] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.032s
I20260709 13:19:32.099071 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:32.122205 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:32.123163 11505 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 20.
I20260709 13:19:32.101162 11477 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:32.116828 11503 raft_consensus.cc:1240] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:32.007331 19041 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:32.060299 18692 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:32.128156 19032 raft_consensus.cc:2749] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:32.106093 19038 consensus_queue.cc:1059] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.029s
I20260709 13:19:32.103060 18622 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:32.129490 18814 consensus_queue.cc:1059] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 26.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:19:32.074928 19005 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:32.137645 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:32.110193 19021 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:19:32.179461 10842 consensus_peers.cc:597] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:32.179905 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 121705us
W20260709 13:19:32.180543 10842 leader_election.cc:336] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:32.141254 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:32.183265 11505 raft_consensus.cc:2393] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 26: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:32.184216 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:32.185321 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:19:32.186213 10840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:32.109256 19033 raft_consensus.cc:2804] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:32.088745 18463 consensus_queue.cc:1059] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 13:19:32.187287 18463 raft_consensus.cc:3055] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:19:32.187774 18463 raft_consensus.cc:740] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:32.147784 18557 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:32.149943 19005 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:32.151250 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:32.151999 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:32.152721 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:32.156939 18882 consensus_queue.cc:1048] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:19:32.159694 18898 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:32.121124 18795 raft_consensus.cc:2749] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:32.164518 10767 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:32.157824 11510 raft_consensus.cc:2368] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for earlier term 28. Current term is 29.
I20260709 13:19:32.132432 18695 raft_consensus.cc:2804] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:19:32.167506 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 45520us
I20260709 13:19:32.232270 10916 leader_election.cc:304] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:32.170939 18817 consensus_queue.cc:260] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:32.173332 11507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:32.158619 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:32.233835 11507 raft_consensus.cc:2410] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 for term 25 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 13:19:32.175266 18878 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:32.155333 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:32.177348 19041 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:32.179270 18692 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:32.186334 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:32.242151 11505 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 21.
W20260709 13:19:32.191049 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 119968us
W20260709 13:19:32.243530 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 79138.9us
I20260709 13:19:32.188865 18463 consensus_queue.cc:260] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:32.208647 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:32.210142 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:19:32.213052 10842 consensus_peers.cc:597] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:32.248868 10867 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 28.
W20260709 13:19:32.250958 10842 consensus_peers.cc:597] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:32.144209 10874 raft_consensus.cc:2468] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 24.
I20260709 13:19:32.257861 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:32.258827 10945 raft_consensus.cc:2393] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 22: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:32.260273 18622 raft_consensus.cc:2749] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:32.219252 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:32.261296 10873 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 22.
I20260709 13:19:32.220335 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:32.262195 10871 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:19:32.226318 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:19:32.214820 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 134046us
I20260709 13:19:32.190070 19033 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:32.245163 10869 raft_consensus.cc:1275] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:19:32.263852 10948 raft_consensus.cc:1240] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 23. Current term is 24. Ops: []
I20260709 13:19:32.246621 10916 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:32.238305 10766 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:32.245692 10872 raft_consensus.cc:1275] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 13:19:32.258103 11509 raft_consensus.cc:1240] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 28 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 27. Current term is 28. Ops: []
I20260709 13:19:32.237958 18695 raft_consensus.cc:493] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:32.268980 18695 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:32.264727 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:32.270542 10945 raft_consensus.cc:2410] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 25 because replica has last-logged OpId of term: 23 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 10.
I20260709 13:19:32.272090 19016 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:32.273078 19033 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:32.279675 18804 raft_consensus.cc:3055] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:19:32.280136 18804 raft_consensus.cc:740] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:32.281064 18804 consensus_queue.cc:260] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:32.282080 18804 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:19:32.252425 10842 consensus_peers.cc:597] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:32.292572 10842 consensus_peers.cc:597] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:32.286363 19021 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:32.306234 10842 consensus_peers.cc:597] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:32.307152 10842 consensus_peers.cc:597] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:32.311079 10842 consensus_peers.cc:597] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:32.311899 10842 leader_election.cc:336] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:32.315274 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:32.364563 11509 raft_consensus.cc:1240] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:32.251624 18694 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:19:32.366374 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 53440.5us
I20260709 13:19:32.287632 18463 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:32.312389 10842 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:32.287933 18817 raft_consensus.cc:3060] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:19:32.383697 11508 raft_consensus.cc:2393] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 26: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
W20260709 13:19:32.391019 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 144494us
I20260709 13:19:32.391827 10916 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:32.263593 19017 consensus_queue.cc:1048] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.113s
W20260709 13:19:32.400390 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 172878us
I20260709 13:19:32.296825 19051 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:19:32.399327 10841 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:32.373174 18694 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:32.374611 19045 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:32.405398 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 93547.1us
W20260709 13:19:32.429191 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:32.307013 18902 raft_consensus.cc:2749] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:32.399149 18557 raft_consensus.cc:2749] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:19:32.419898 10766 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:32.426843 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:32.439910 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 48159.9us
W20260709 13:19:32.432909 10915 leader_election.cc:336] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Timed out: RequestConsensusVote RPC to 127.9.116.132:40555 timed out after 1.721s (SENT)
I20260709 13:19:32.433560 10871 raft_consensus.cc:2468] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 29.
I20260709 13:19:32.390223 19005 consensus_queue.cc:1059] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.017s
I20260709 13:19:32.445941 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:32.449069 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:32.440907 10916 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:19:32.431066 19050 log.cc:927] Time spent T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:19:32.458369 10915 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:32.418901 18694 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:32.462345 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 30 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:32.460361 10916 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:19:32.447312 19046 log.cc:927] Time spent T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:19:32.441664 18814 raft_consensus.cc:2749] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:19:32.436725 18959 raft_consensus.cc:2749] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:19:32.465456 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 45659us
I20260709 13:19:32.446936 18549 raft_consensus.cc:493] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:32.469698 10766 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:32.446667 18958 raft_consensus.cc:2749] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:32.470733 19049 raft_consensus.cc:2804] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:32.471534 19038 raft_consensus.cc:2804] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 13:19:32.473677 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:32.492015 11506 raft_consensus.cc:1240] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 25. Current term is 26. Ops: []
I20260709 13:19:32.475209 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:32.457552 18882 raft_consensus.cc:2749] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:32.484831 10943 raft_consensus.cc:1240] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 23. Current term is 24. Ops: []
I20260709 13:19:32.492995 10950 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 24.
W20260709 13:19:32.497232 11476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:32.499732 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:32.508231 10941 raft_consensus.cc:1240] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 23. Current term is 24. Ops: []
I20260709 13:19:32.453092 19045 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:32.514686 10947 raft_consensus.cc:1240] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 25. Current term is 26. Ops: []
I20260709 13:19:32.532437 19033 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:32.533946 10944 raft_consensus.cc:1240] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:32.482607 18695 raft_consensus.cc:515] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:32.536681 19045 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:32.536762 18695 leader_election.cc:290] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:32.513160 19049 raft_consensus.cc:697] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 19 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:32.538635 19049 consensus_queue.cc:237] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:32.543332 19021 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 26 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:32.544026 19021 consensus_queue.cc:1059] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:19:32.545096 19021 raft_consensus.cc:3055] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:19:32.545435 19021 raft_consensus.cc:740] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:32.547202 18970 raft_consensus.cc:3055] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:19:32.547613 18970 raft_consensus.cc:740] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:32.548385 18970 consensus_queue.cc:260] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:32.551148 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 81534.4us
I20260709 13:19:32.552093 10766 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:32.553264 10766 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:32.555058 18618 raft_consensus.cc:2749] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:32.555896 18618 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:32.464454 18622 consensus_queue.cc:1048] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.049s
I20260709 13:19:32.554555 10766 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:32.564283 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:32.512255 18902 raft_consensus.cc:2749] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:32.512557 19016 raft_consensus.cc:2749] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:32.582592 11478 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:32.580904 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:32.493213 18549 raft_consensus.cc:515] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:32.586067 11506 raft_consensus.cc:2410] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 25 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 13:19:32.587471 10766 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:32.589003 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:32.591903 10947 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:32.546231 19021 consensus_queue.cc:260] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:32.607688 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:32.605669 10841 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:19:32.611318 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 151019us
I20260709 13:19:32.563068 19023 consensus_queue.cc:1059] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.028s
I20260709 13:19:32.615772 18959 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:19:32.585594 18882 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:19:32.585620 18923 raft_consensus.cc:2764] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 13:19:32.622925 19034 raft_consensus.cc:2749] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:32.624555 18549 leader_election.cc:290] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:32.626091 10766 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:19:32.647702 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 189375us
I20260709 13:19:32.602944 18576 consensus_queue.cc:1048] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 15, Time since last communication: 0.038s
I20260709 13:19:32.621275 18959 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:32.660868 19017 raft_consensus.cc:2749] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:32.670749 19004 raft_consensus.cc:3055] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:19:32.671298 19004 raft_consensus.cc:740] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:32.672222 19004 consensus_queue.cc:260] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:32.673259 18970 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:19:32.662974 10915 leader_election.cc:336] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Timed out: RequestConsensusVote RPC to 127.9.116.132:40555 timed out after 1.592s (SENT)
I20260709 13:19:32.677145 10915 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:32.661280 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 28 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:32.681494 10947 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 23.
I20260709 13:19:32.635043 19038 raft_consensus.cc:697] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 24 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:19:32.667174 10842 consensus_peers.cc:597] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:32.686777 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 257369us
W20260709 13:19:32.687714 10842 leader_election.cc:336] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:32.691195 10842 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:32.698210 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 72225.7us
W20260709 13:19:32.698693 10915 leader_election.cc:336] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Timed out: RequestConsensusVote RPC to 127.9.116.132:40555 timed out after 1.858s (SENT)
I20260709 13:19:32.699419 10766 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:32.702883 10766 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:19:32.698020 10842 consensus_peers.cc:597] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:32.661345 18959 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:32.661096 19033 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:32.703650 10841 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:19:32.705631 10842 consensus_peers.cc:597] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:32.709738 10766 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:19:32.711066 10915 leader_election.cc:336] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Timed out: RequestConsensusVote RPC to 127.9.116.132:40555 timed out after 2.083s (SENT)
I20260709 13:19:32.722611 10915 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:32.723698 18902 raft_consensus.cc:2749] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:32.724241 10766 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:32.725109 18902 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:32.726217 19028 consensus_queue.cc:1048] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:19:32.731137 10766 leader_election.cc:336] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Timed out: RequestConsensusVote RPC to 127.9.116.132:40555 timed out after 2.001s (SENT)
I20260709 13:19:32.731617 10766 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:32.732416 19028 raft_consensus.cc:2749] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:19:32.732908 19054 log.cc:927] Time spent T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.255s	user 0.003s	sys 0.000s
I20260709 13:19:32.733978 18902 raft_consensus.cc:2749] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:32.734356 19004 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:32.739315 18622 raft_consensus.cc:2749] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:19:32.735141 18974 raft_consensus.cc:3055] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 13:19:32.750964 18974 raft_consensus.cc:740] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:32.751972 18974 consensus_queue.cc:260] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:32.753007 18974 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:19:32.752911 18618 consensus_queue.cc:1059] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:19:32.753877 18618 raft_consensus.cc:3055] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:19:32.754217 18618 raft_consensus.cc:740] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:32.695149 18872 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:32.755053 18618 consensus_queue.cc:260] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:32.756248 18618 raft_consensus.cc:3060] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:32.655212 18694 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:32.755178 18872 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:32.705757 19017 raft_consensus.cc:2749] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:32.701340 19045 raft_consensus.cc:2749] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:32.708029 19034 raft_consensus.cc:2749] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:19:32.761915 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 148413us
I20260709 13:19:32.767848 11477 leader_election.cc:304] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:32.772058 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185480268a684958b8347a4f53af57c5" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 28 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:32.708513 19063 raft_consensus.cc:2749] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:19:32.776268 18175 raft_consensus.cc:2804] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 13:19:32.776389 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 72850.6us
I20260709 13:19:32.788568 18175 raft_consensus.cc:697] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 23 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:32.792881 18872 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:32.793285 19021 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:32.794304 19055 raft_consensus.cc:3055] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:19:32.794836 19055 raft_consensus.cc:740] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:32.775816 18558 consensus_queue.cc:1048] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 13:19:32.796063 19055 consensus_queue.cc:260] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:32.797389 19055 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:19:32.739753 18557 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:32.803891 18175 consensus_queue.cc:237] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:32.809609 11478 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:32.811830 10915 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:32.826282 10915 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:32.794296 18694 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:32.835237 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:32.837939 10868 raft_consensus.cc:3060] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:19:32.842429 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81df256f0c644cfae9fe2d336cbe9bd" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:32.843103 10869 raft_consensus.cc:3060] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:32.823551 18873 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:32.849473 10872 raft_consensus.cc:1240] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 28 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 27. Current term is 28. Ops: [27.17-27.17]
I20260709 13:19:32.823760 19063 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:32.832540 19049 raft_consensus.cc:2749] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:19:32.814297 18569 consensus_queue.cc:1048] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.033s
I20260709 13:19:32.841001 18804 raft_consensus.cc:2804] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 13:19:32.808429 18557 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:32.827792 18923 consensus_queue.cc:1048] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.030s
I20260709 13:19:32.842922 19045 raft_consensus.cc:493] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:32.813506 19038 consensus_queue.cc:237] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:32.848899 18622 consensus_queue.cc:1059] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.017s
W20260709 13:19:32.873018 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 114692us
I20260709 13:19:32.870573 18804 raft_consensus.cc:697] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 29 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:32.868970 19045 raft_consensus.cc:515] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:32.954375 10868 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 26.
W20260709 13:19:32.962772 19067 log.cc:927] Time spent T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 13:19:32.871858 19066 consensus_queue.cc:1059] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.084s
I20260709 13:19:32.965328 19066 raft_consensus.cc:3055] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:19:32.965741 19066 raft_consensus.cc:740] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:32.966744 19066 consensus_queue.cc:260] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:32.968101 19066 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:19:32.975373 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:32.975862 10867 raft_consensus.cc:1240] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 20. Current term is 21. Ops: []
W20260709 13:19:32.982115 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 208605us
W20260709 13:19:32.983059 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:32.990350 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 213211us
I20260709 13:19:32.930265 19069 consensus_queue.cc:1048] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.116s
I20260709 13:19:32.940305 18549 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:32.996376 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:32.947219 18898 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:32.947535 18692 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:33.002701 10916 consensus_peers.cc:597] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:32.976655 18817 consensus_meta.cc:287] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.589s	user 0.011s	sys 0.001s
I20260709 13:19:32.904508 18379 consensus_queue.cc:1059] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.074s
I20260709 13:19:33.006606 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:33.014078 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:33.014925 10941 raft_consensus.cc:2393] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 24: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:33.018617 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:33.019675 10767 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:33.020609 11508 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 24.
W20260709 13:19:33.009529 10916 consensus_peers.cc:597] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:33.028725 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:19:33.029072 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:33.029408 10876 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:33.027874 19048 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:33.027923 10916 leader_election.cc:336] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:19:33.143287 10914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:33.011112 18549 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:33.012326 18898 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:33.142424 10842 leader_election.cc:336] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:19:33.143927 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 117723us
W20260709 13:19:33.182987 10916 leader_election.cc:336] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:19:33.184010 10916 leader_election.cc:336] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:33.184466 10916 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:33.185570 10916 consensus_peers.cc:597] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:33.182583 10842 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:33.020587 18692 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:33.186879 10916 leader_election.cc:336] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:33.139699 18902 raft_consensus.cc:2749] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:19:33.196452 10916 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:33.185561 18463 consensus_meta.cc:287] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.778s	user 0.015s	sys 0.006s
I20260709 13:19:33.136451 18817 raft_consensus.cc:3055] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:33.024381 18804 consensus_queue.cc:237] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.19, Last appended by leader: 19, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:33.203716 10916 leader_election.cc:336] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:33.204341 10916 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:33.150660 18882 consensus_meta.cc:287] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.516s	user 0.013s	sys 0.002s
I20260709 13:19:33.208808 10872 raft_consensus.cc:2410] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 22 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
W20260709 13:19:33.208709 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 72408.9us
W20260709 13:19:33.210922 10842 leader_election.cc:336] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:33.211481 10842 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:33.189034 18622 raft_consensus.cc:2749] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:33.209023 19064 raft_consensus.cc:2749] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:19:33.192559 18618 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:33.205057 18817 raft_consensus.cc:740] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:33.186064 18959 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:33.207135 18814 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:33.236929 10876 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 24.
I20260709 13:19:33.150808 19048 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:33.217329 18882 raft_consensus.cc:2749] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:19:33.244315 10841 leader_election.cc:304] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:19:33.227555 18970 consensus_meta.cc:287] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.539s	user 0.009s	sys 0.002s
W20260709 13:19:33.243382 10842 leader_election.cc:336] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:33.251607 10842 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:33.255571 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:33.256065 10946 raft_consensus.cc:1275] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 16. (term mismatch)
I20260709 13:19:33.256609 10946 pending_rounds.cc:85] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 15
I20260709 13:19:33.256902 10946 pending_rounds.cc:107] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 24.16
I20260709 13:19:33.257234 10946 raft_consensus.cc:2889] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:33.258841 10870 raft_consensus.cc:2393] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 25: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:33.243558 18557 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:33.250120 19059 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:33.364778 10913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:33.229854 18618 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:33.260217 19045 leader_election.cc:290] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:33.376703 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 165109us
W20260709 13:19:33.375885 10869 consensus_meta.cc:287] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.532s	user 0.006s	sys 0.007s
I20260709 13:19:33.393965 10869 raft_consensus.cc:2468] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 23.
I20260709 13:19:33.399679 10916 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:33.288235 19005 raft_consensus.cc:2749] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:19:33.413290 19033 raft_consensus.cc:2804] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Leader election won for term 26
W20260709 13:19:33.441965 11478 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.012s	sys 0.000s
W20260709 13:19:33.442523 11478 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.013s	sys 0.000s
I20260709 13:19:33.380923 19059 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:33.443406 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 620829us
I20260709 13:19:33.371901 18817 consensus_queue.cc:260] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:33.444940 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 195589us
W20260709 13:19:33.441246 19004 consensus_meta.cc:287] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.691s	user 0.010s	sys 0.002s
I20260709 13:19:33.468621 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:33.492828 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:33.493207 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b2d93b69274dbca479bd87d7ea45cb" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
W20260709 13:19:33.496206 19077 log.cc:927] Time spent T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:19:33.496476 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:33.497229 19072 raft_consensus.cc:2749] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:33.498083 19086 raft_consensus.cc:2749] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:19:33.499328 19081 raft_consensus.cc:2749] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:19:33.499780 10874 raft_consensus.cc:1240] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 23. Current term is 24. Ops: []
I20260709 13:19:33.500322 18882 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:33.500849 18882 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:33.501745 19028 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:33.502283 19028 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:33.503324 19076 log.cc:927] Time spent T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.272s	user 0.000s	sys 0.001s
I20260709 13:19:33.504462 19028 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:33.505465 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7595d019b2e74196bc5838962b53798d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:33.508415 18618 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:33.444631 19033 raft_consensus.cc:697] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 26 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:33.509769 19033 consensus_queue.cc:237] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:33.510690 18549 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:33.512485 19085 log.cc:927] Time spent T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:19:33.516068 19048 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:33.516808 10870 raft_consensus.cc:1275] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 13:19:33.518085 10870 raft_consensus.cc:1240] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 24. Current term is 25. Ops: [24.16-24.16]
W20260709 13:19:33.521661 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:33.523015 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:33.526144 11503 raft_consensus.cc:2393] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 26: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:33.533732 10870 raft_consensus.cc:1240] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 29 FOLLOWER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 28. Current term is 29. Ops: []
I20260709 13:19:33.517527 18898 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:33.543395 18882 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:33.533545 18817 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:33.533044 18692 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:33.557225 19078 log.cc:927] Time spent T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.160s	user 0.000s	sys 0.000s
W20260709 13:19:33.560849 10916 leader_election.cc:336] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:33.557250 19089 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:33.559259 18959 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:33.584708 10692 catalog_manager.cc:5697] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 17 to 20. New cstate: current_term: 20 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:33.577973 19089 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:33.595777 19059 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:33.597504 18576 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:19:33.597611 11478 consensus_peers.cc:597] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:33.607847 11478 leader_election.cc:336] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:19:33.607928 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 55231us
I20260709 13:19:33.608393 11478 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:33.608681 10916 leader_election.cc:336] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:33.644699 10916 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:33.646552 19063 raft_consensus.cc:2749] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:19:33.611580 19055 consensus_meta.cc:287] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.814s	user 0.013s	sys 0.001s
W20260709 13:19:33.614307 11478 leader_election.cc:336] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:19:33.626372 19021 consensus_meta.cc:287] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.833s	user 0.013s	sys 0.000s
I20260709 13:19:33.612277 18576 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:33.652769 18576 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:33.615419 18695 raft_consensus.cc:2749] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:19:33.632342 19089 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:33.632551 19049 raft_consensus.cc:3055] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:19:33.654008 19049 raft_consensus.cc:740] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:33.654935 19049 consensus_queue.cc:260] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:33.656412 19049 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:19:33.597589 18974 consensus_meta.cc:287] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.844s	user 0.013s	sys 0.000s
W20260709 13:19:33.653405 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:33.658773 18817 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:33.659336 11478 consensus_peers.cc:597] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:33.659510 18817 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:33.660229 11478 leader_election.cc:336] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:33.660772 11478 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:33.667466 10942 raft_consensus.cc:2410] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 29 because replica has last-logged OpId of term: 27 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 16.
I20260709 13:19:33.668509 18695 raft_consensus.cc:2749] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
W20260709 13:19:33.671006 11478 consensus_peers.cc:597] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:33.671689 11478 consensus_peers.cc:597] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:33.672295 11478 leader_election.cc:336] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:33.672717 11478 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:33.679189 18817 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:33.673452 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 64797us
W20260709 13:19:33.683917 10916 consensus_peers.cc:597] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:33.684624 18569 raft_consensus.cc:2749] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:19:33.684779 10916 leader_election.cc:336] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:33.685436 10916 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:33.686899 10916 consensus_peers.cc:597] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:33.687570 10916 leader_election.cc:336] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:19:33.688948 11478 leader_election.cc:336] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:33.689571 11478 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:33.690267 19049 raft_consensus.cc:2749] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
W20260709 13:19:33.695176 11478 leader_election.cc:336] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:19:33.683925 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 440543us
I20260709 13:19:33.695827 11478 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:33.702977 18558 raft_consensus.cc:2749] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
W20260709 13:19:33.688292 10916 leader_election.cc:336] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:19:33.708000 10916 consensus_peers.cc:597] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:33.711980 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 240270us
I20260709 13:19:33.718727 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:33.720687 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:33.721309 11508 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 23.
I20260709 13:19:33.723453 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:33.723764 18878 raft_consensus.cc:493] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:33.724489 18878 raft_consensus.cc:515] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:33.724479 11508 raft_consensus.cc:2410] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 for term 29 because replica has last-logged OpId of term: 27 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 16.
I20260709 13:19:33.726670 18878 leader_election.cc:290] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:33.728052 18878 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:33.728539 18878 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:33.729091 10872 raft_consensus.cc:1240] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:33.729599 10876 raft_consensus.cc:1240] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 23. Current term is 24. Ops: [21.15-23.16]
I20260709 13:19:33.730674 18878 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:33.727550 10841 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:33.731650 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:33.731945 18872 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:33.732375 18872 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:33.732760 18872 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:33.736968 10869 raft_consensus.cc:1275] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:19:33.736953 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:33.738626 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:33.739239 10946 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:33.740038 10916 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:33.744563 10841 leader_election.cc:304] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:33.749606 10692 catalog_manager.cc:5697] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 18 to 24. New cstate: current_term: 24 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:33.745518 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:33.752154 10841 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:19:33.752640 10692 catalog_manager.cc:5697] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 21 to 24, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 24 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:33.755040 19049 raft_consensus.cc:2749] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:33.755122 18463 raft_consensus.cc:2749] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:19:33.756412 18872 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:33.756799 18463 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:33.757256 18463 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:33.745958 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:33.757733 18463 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:33.758834 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:33.759025 19049 consensus_queue.cc:1059] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 13:19:33.760109 19090 raft_consensus.cc:3055] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 13:19:33.760555 19090 raft_consensus.cc:740] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:33.761503 19090 consensus_queue.cc:260] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.19, Last appended by leader: 19, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:33.761557 10941 raft_consensus.cc:1240] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 26. Current term is 27. Ops: []
I20260709 13:19:33.765415 19090 raft_consensus.cc:3060] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:19:33.765587 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:33.770349 10869 raft_consensus.cc:1275] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:19:33.771703 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:33.754436 10692 catalog_manager.cc:5697] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 9 to 26. New cstate: current_term: 26 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:33.765583 18463 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:33.783062 18463 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:33.784693 18872 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:33.786032 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253bd741d094067902a02bc765ee0fb" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:33.803737 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:33.804968 10947 raft_consensus.cc:2393] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 24: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
W20260709 13:19:33.807832 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:33.809949 10946 raft_consensus.cc:2468] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 25.
I20260709 13:19:33.793030 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:33.797096 19049 consensus_queue.cc:1048] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:19:33.797488 10916 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:33.873693 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 76288.5us
I20260709 13:19:33.797016 19034 consensus_queue.cc:1048] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:19:33.811067 10767 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:33.812106 18974 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:33.907419 11511 raft_consensus.cc:1240] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 29 LEADER]: Rejecting Update request from peer cb0428cc55984daba1ba63744a5102a2 for earlier term 28. Current term is 29. Ops: []
I20260709 13:19:33.788444 10841 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:33.817076 11512 raft_consensus.cc:2468] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 26.
I20260709 13:19:33.828078 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:33.828425 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:33.834051 18576 consensus_queue.cc:1059] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:19:33.834805 10874 raft_consensus.cc:1275] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:19:33.869719 18878 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:33.879688 18175 raft_consensus.cc:2749] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:33.922304 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:33.851218 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:33.852119 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:33.853502 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:19:33.912396 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 101431us
I20260709 13:19:33.911691 11508 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:19:33.915823 10916 consensus_peers.cc:597] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:33.916700 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:33.927127 10870 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 21.
I20260709 13:19:33.835216 18695 consensus_queue.cc:1059] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:19:33.923081 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:33.844334 18817 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:33.868469 18882 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:33.875785 18923 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:33.925374 11505 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 20.
I20260709 13:19:33.917007 18557 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:33.936125 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:33.937973 10916 leader_election.cc:336] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:33.876847 19089 raft_consensus.cc:2749] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:33.954022 10876 raft_consensus.cc:2393] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 24: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:33.921993 19028 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:19:33.951400 10916 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:33.924165 18674 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:33.924571 19033 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:33.919538 19051 consensus_queue.cc:1048] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 13:19:33.939429 18558 consensus_queue.cc:1048] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.022s
I20260709 13:19:33.917168 18974 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:33.963304 19089 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:33.966681 19028 raft_consensus.cc:697] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:33.963441 18695 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:33.963344 18576 raft_consensus.cc:3055] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:19:33.989976 19028 consensus_queue.cc:237] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:33.943823 18817 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:33.991513 18576 raft_consensus.cc:740] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:33.993155 18569 raft_consensus.cc:3055] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 22 LEADER]: Stepping down as leader of term 22
W20260709 13:19:33.999938 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 154427us
I20260709 13:19:34.011737 18379 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:34.011804 19036 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:34.012332 18379 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:34.012465 19036 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:34.014926 19036 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:33.967206 18674 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:34.017427 18674 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:34.019276 18974 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:33.993992 18695 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:34.008786 10867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2964) took 1030 ms. Trace:
I20260709 13:19:34.000942 10869 raft_consensus.cc:1275] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 13:19:33.950672 18882 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:33.959928 18557 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:33.941203 18878 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:33.956498 18923 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:34.030856 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:34.027010 10867 rpcz_store.cc:276] 0709 13:19:32.978946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:32.979074 (+   128us) service_pool.cc:224] Handling call
0709 13:19:32.979384 (+   310us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:32.979747 (+   363us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:32.979772 (+    25us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:33.008796 (+ 29024us) log.cc:844] Serialized 52 byte log entry
0709 13:19:33.596746 (+587950us) spinlock_profiling.cc:243] Waited 4.7 ms on lock 0x7b18001d4d00. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddaae07 00007f9c86c05ab9 00007f9c7ddc769b 00007f9c7ddc593f 00007f9c7ddc7c02 00007f9c80a4f20b 00007f9c80a54d23 00007f9c80a550d1 00007f9c80e70a0f
0709 13:19:33.871442 (+274696us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:33.871501 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:33.871594 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:34.008503 (+136909us) raft_consensus.cc:1685] finished
0709 13:19:34.008525 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:34.008728 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16378,"raft.queue_time_us":51692,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":9552,"spinlock_wait_cycles":10805632,"thread_start_us":274471,"threads_started":1,"wal-append.queue_time_us":917855}
I20260709 13:19:34.029866 10766 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:34.028970 11508 raft_consensus.cc:2468] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 24.
I20260709 13:19:34.056612 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:34.067515 10946 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 20.
I20260709 13:19:34.014695 18379 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:34.006981 19110 log.cc:927] Time spent T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:19:34.022640 18695 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:34.004439 18817 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:34.007037 18569 raft_consensus.cc:740] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:34.092094 11477 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:34.092679 10947 raft_consensus.cc:1275] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:19:34.030580 18618 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:34.108067 10686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 170394us
I20260709 13:19:34.111582 18695 raft_consensus.cc:2804] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:34.112270 18695 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:34.112716 18695 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:34.030584 19034 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:34.119354 19034 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:34.122612 19034 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:33.979591 19033 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:34.070916 18974 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:34.070984 19055 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:34.081892 11509 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 24.
I20260709 13:19:34.098316 18379 raft_consensus.cc:2804] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:34.127591 19066 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:34.093660 19064 consensus_queue.cc:1048] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 13:19:34.006573 18576 consensus_queue.cc:260] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:34.179299 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 111824us
I20260709 13:19:34.181732 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:34.183413 11511 raft_consensus.cc:1275] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
W20260709 13:19:34.184511 10915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:34.184551 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:34.127791 18618 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:34.146653 18379 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:34.177002 18974 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:34.216370 19106 log.cc:927] Time spent T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 13:19:34.219065 18878 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:34.219468 19023 consensus_queue.cc:1048] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:19:34.221622 18882 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:34.226533 18576 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:34.226770 18882 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:34.227310 18882 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:34.229508 18882 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:34.229816 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:34.229996 18882 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:34.230763 18882 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:34.213603 18569 consensus_queue.cc:260] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:34.231660 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:34.231945 18569 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:34.232628 10946 raft_consensus.cc:2410] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 21 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 13:19:34.233222 18882 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:34.234159 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:34.235620 18923 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:34.239239 10875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2974) took 1248 ms. Trace:
I20260709 13:19:34.239670 10875 rpcz_store.cc:276] 0709 13:19:32.990949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.514510 (+523561us) service_pool.cc:224] Handling call
0709 13:19:33.514870 (+   360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:33.718821 (+203951us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:33.718852 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:33.719313 (+   461us) log.cc:844] Serialized 26 byte log entry
0709 13:19:33.848263 (+128950us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:33.906967 (+ 58704us) log.cc:844] Serialized 14 byte log entry
0709 13:19:34.238944 (+331977us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:34.239016 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:34.239053 (+    37us) raft_consensus.cc:1685] finished
0709 13:19:34.239067 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:34.239183 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":537400,"prepare.queue_time_us":58501,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":204759,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":83,"thread_start_us":128531,"threads_started":1,"wal-append.queue_time_us":248935}
I20260709 13:19:34.245430 18463 raft_consensus.cc:2749] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 13:19:34.245615 10842 leader_election.cc:336] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:34.246119 10842 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:34.246198 18463 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:34.251288 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:34.260183 10870 raft_consensus.cc:1240] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Rejecting Update request from peer 12682eb70bc84c76879e04a8628ee586 for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:34.260281 10946 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 28.
I20260709 13:19:34.269943 11477 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:34.178896 19055 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:34.283354 10868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54948 (request call id 2969) took 1303 ms. Trace:
I20260709 13:19:34.214509 18379 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:34.283830 10868 rpcz_store.cc:276] 0709 13:19:32.980370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.400068 (+419698us) service_pool.cc:224] Handling call
0709 13:19:33.416759 (+ 16691us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:33.417045 (+   286us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:19:33.935513 (+518468us) log.cc:844] Serialized 14 byte log entry
0709 13:19:34.282988 (+347475us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:34.283046 (+    58us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:19:34.283096 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:34.283173 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:34.283292 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":157624,"prepare.queue_time_us":262558,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"thread_start_us":687372,"threads_started":2,"wal-append.queue_time_us":284700}
I20260709 13:19:34.272404 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:34.275053 19068 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 13:19:34.186705 19066 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:34.263880 19076 log.cc:927] Time spent T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:19:34.270954 18463 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:34.287029 10869 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 28.
I20260709 13:19:34.288955 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:34.294128 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 505765us
W20260709 13:19:34.298954 10686 catalog_manager.cc:4755] TS e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): ChangeConfig:REMOVE_PEER RPC failed for tablet 695ff1b5a68e4111917d909909488d39: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:34.300400 10950 raft_consensus.cc:1275] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 29 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 13:19:34.302464 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:34.303105 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 57519.5us
W20260709 13:19:34.303999 10842 leader_election.cc:336] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:34.309091 10842 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:34.312140 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 198306us
I20260709 13:19:34.319072 10868 raft_consensus.cc:1275] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 29 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 13:19:34.316705 10841 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:19:34.286979 19068 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:34.322328 10916 leader_election.cc:336] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:34.322889 10916 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:34.321947 10841 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:19:34.331719 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:34.336913 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:34.337667 11508 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 21.
I20260709 13:19:34.332495 10943 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 21.
I20260709 13:19:34.347057 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:34.347820 10873 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 28.
I20260709 13:19:34.315440 19004 raft_consensus.cc:2749] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:19:34.317587 19112 consensus_queue.cc:1048] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.034s
I20260709 13:19:34.370455 19112 consensus_queue.cc:1243] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Peer cb0428cc55984daba1ba63744a5102a2 log is divergent from this leader: its last log entry 28.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20260709 13:19:34.371389 10692 catalog_manager.cc:5697] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 17 to 26. New cstate: current_term: 26 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:19:34.373421 19055 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:34.333793 19115 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:34.327240 18695 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:34.352850 19048 consensus_queue.cc:1059] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 26.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:19:34.355607 19114 raft_consensus.cc:2749] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:34.347074 19033 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:34.356885 18463 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:34.357676 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 44027.6us
I20260709 13:19:34.382000 10767 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:19:34.358736 11477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:34.386456 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:34.386868 10950 pending_rounds.cc:85] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2: Aborting all ops after (but not including) 18
I20260709 13:19:34.387367 10950 pending_rounds.cc:107] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2: Aborting uncommitted NO_OP operation due to leader change: 28.19
I20260709 13:19:34.387938 10950 raft_consensus.cc:2889] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:34.348047 19063 consensus_queue.cc:1048] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.012s
I20260709 13:19:34.326395 19005 raft_consensus.cc:2749] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:34.361904 18557 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:34.340272 18872 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:19:34.368940 19066 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:34.347457 19090 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:34.369176 18618 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:34.372859 10692 catalog_manager.cc:5697] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 10 to 20, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 20 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:34.351081 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:34.377149 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:34.325091 19068 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:19:34.328794 19081 raft_consensus.cc:2804] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:19:34.387316 10945 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 24.
I20260709 13:19:34.395397 11509 raft_consensus.cc:2393] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 21: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:34.396865 19028 raft_consensus.cc:3060] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:34.438228 11477 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:34.435506 10767 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:34.427467 18974 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:34.454375 18878 raft_consensus.cc:3055] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:19:34.454866 18878 raft_consensus.cc:740] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:34.455658 18878 consensus_queue.cc:260] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 26.11, Last appended by leader: 11, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:34.452391 19118 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:34.427032 18872 raft_consensus.cc:697] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:34.433015 19081 raft_consensus.cc:697] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:34.413935 19090 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:34.450836 19112 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 13:19:34.471028 11478 leader_election.cc:336] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:34.426707 19125 consensus_queue.cc:1048] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.052s
I20260709 13:19:34.452076 18817 raft_consensus.cc:2749] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:19:34.456971 19118 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:34.489416 11478 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:34.379318 19123 consensus_queue.cc:1048] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.093s
W20260709 13:19:34.467849 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 279445us
I20260709 13:19:34.495123 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:34.496405 10869 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 25.
I20260709 13:19:34.498651 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:34.504515 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:34.508857 11478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:34.513247 10692 catalog_manager.cc:5697] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 20 to 23. New cstate: current_term: 23 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
W20260709 13:19:34.515295 19133 log.cc:927] Time spent T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.085s	user 0.001s	sys 0.002s
W20260709 13:19:34.516681 10693 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42984 (request call id 248) took 59657 ms (client timeout 15000 ms). Trace:
I20260709 13:19:34.517275 18695 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:34.490372 19112 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:34.519522 19112 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:34.521234 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:34.517136 10693 rpcz_store.cc:269] 0709 13:18:34.859340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.859621 (+   281us) service_pool.cc:224] Handling call
0709 13:19:34.516577 (+59656956us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:31.059296 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:31.060215 (+   919us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:31.060284 (+    69us) write_op.cc:435] Acquired schema lock
0709 13:19:31.060336 (+    52us) tablet.cc:662] Decoding operations
0709 13:19:31.061136 (+   800us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:31.061273 (+   137us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:31.061327 (+    54us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:19:31.061871 (+   544us) tablet.cc:701] Row locks acquired
0709 13:19:31.061930 (+    59us) write_op.cc:260] PREPARE: finished
0709 13:19:31.062143 (+   213us) write_op.cc:270] Start()
0709 13:19:31.062331 (+   188us) write_op.cc:276] Timestamp: P: 1783603171062061 usec, L: 0
0709 13:19:31.062377 (+    46us) op_driver.cc:348] REPLICATION: starting
0709 13:19:31.108362 (+ 45985us) log.cc:844] Serialized 1312 byte log entry
0709 13:19:31.676933 (+568571us) op_driver.cc:464] REPLICATION: finished
0709 13:19:31.889729 (+212796us) write_op.cc:301] APPLY: starting
0709 13:19:31.889930 (+   201us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:31.893582 (+  3652us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:31.893754 (+   172us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:31.904574 (+ 10820us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:31.905010 (+   436us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=6,key_file_lookups=6,delta_file_lookups=6,mrs_lookups=0
0709 13:19:31.905100 (+    90us) write_op.cc:312] APPLY: finished
0709 13:19:31.905492 (+   392us) log.cc:844] Serialized 41 byte log entry
0709 13:19:31.905712 (+   220us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:31.905807 (+    95us) write_op.cc:454] Released schema lock
0709 13:19:31.905871 (+    64us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:31.911212 (+  5341us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:31.911261 (+    49us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":43761,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":193165,"apply.run_cpu_time_us":8504,"apply.run_wall_time_us":16626,"cfile_cache_hit":14,"cfile_cache_hit_bytes":16673,"num_ops":3,"prepare.queue_time_us":174843,"prepare.run_cpu_time_us":8459,"prepare.run_wall_time_us":206250,"raft.queue_time_us":451826,"raft.run_cpu_time_us":54111,"raft.run_wall_time_us":226238,"replication_time_us":579630,"thread_start_us":630176,"threads_started":4,"wal-append.queue_time_us":64691,"wal-append.run_cpu_time_us":48797,"wal-append.run_wall_time_us":1733360}]]}
I20260709 13:19:34.521943 10941 raft_consensus.cc:2468] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 20.
I20260709 13:19:34.468541 19080 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:34.522737 18804 raft_consensus.cc:2749] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:34.529215 19090 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:34.529953 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:34.532658 18817 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:34.533383 18817 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:34.526288 19081 consensus_queue.cc:237] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:34.374191 19115 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:34.535794 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:34.541514 10941 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:34.489265 10916 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:19:34.542496 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 53327.9us
I20260709 13:19:34.488727 18814 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:34.523015 10692 catalog_manager.cc:5697] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 15 to 19, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 19 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:19:34.565753 10692 catalog_manager.cc:5697] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 16 to 25. New cstate: current_term: 25 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:34.613828 10916 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:34.540764 19121 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:34.543313 18873 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:34.558733 19090 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Leader pre-election won for term 29
W20260709 13:19:34.654309 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 323129us
I20260709 13:19:34.661643 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:34.578938 18974 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:34.629415 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 309379us
I20260709 13:19:34.526670 19080 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:34.669163 10842 leader_election.cc:336] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:34.669793 10842 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:34.673255 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 28 candidate_status { last_received { term: 26 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:34.622459 18618 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:34.631945 10916 leader_election.cc:336] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:34.660115 10841 leader_election.cc:304] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:34.675022 10916 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:34.680124 10842 leader_election.cc:336] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:19:34.681818 10842 leader_election.cc:336] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:19:34.685883 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 53976us
W20260709 13:19:34.686625 10916 leader_election.cc:336] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:34.644153 19090 raft_consensus.cc:493] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:34.678484 10841 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:34.697988 10941 raft_consensus.cc:2468] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 21.
I20260709 13:19:34.699811 11477 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:34.661298 19114 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:19:34.627409 10692 catalog_manager.cc:5697] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:34.660634 19034 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:34.624511 18814 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:34.677073 18463 raft_consensus.cc:2749] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 13:19:34.705228 18549 raft_consensus.cc:2804] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:19:34.735489 18804 raft_consensus.cc:2804] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:34.735929 18549 raft_consensus.cc:697] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:34.736135 18804 raft_consensus.cc:697] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 21 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
W20260709 13:19:34.736907 19134 log.cc:927] Time spent T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 13:19:34.736999 18549 consensus_queue.cc:237] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.11, Last appended by leader: 11, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:34.737221 18804 consensus_queue.cc:237] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:34.685870 19005 raft_consensus.cc:2749] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:34.690104 19033 raft_consensus.cc:2749] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:34.700745 19090 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:19:34.656304 18873 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:34.751204 10692 catalog_manager.cc:5697] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 17 to 25. New cstate: current_term: 25 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:34.624207 18872 consensus_queue.cc:237] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:34.647629 19121 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:34.731673 19036 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:34.778667 19036 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:34.779254 10692 catalog_manager.cc:5697] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 14 to 20. New cstate: current_term: 20 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:34.781077 19036 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:34.780987 10692 catalog_manager.cc:5697] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 14 to 22. New cstate: current_term: 22 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:19:34.782542 18817 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:34.782549 10692 catalog_manager.cc:5697] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 14 to 18. New cstate: current_term: 18 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:34.784821 10692 catalog_manager.cc:5697] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 14 to 18. New cstate: current_term: 18 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:34.785306 19068 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:34.786813 11505 raft_consensus.cc:2393] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 29: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:34.789112 19121 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:34.736268 19118 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:34.752427 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 241857us
I20260709 13:19:34.685140 18974 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:34.693975 18618 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:34.804272 19028 raft_consensus.cc:2749] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:34.741958 19034 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:34.812500 18379 consensus_meta.cc:287] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87: Time spent flushing consensus metadata: real 0.528s	user 0.013s	sys 0.000s
I20260709 13:19:34.808320 18817 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:34.858450 11478 leader_election.cc:336] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:34.786366 10692 catalog_manager.cc:5697] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: config changed from index 8 to 13, term changed from 17 to 22, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to <none>, VOTER cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) evicted. New cstate: current_term: 22 committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
W20260709 13:19:34.837574 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 116046us
I20260709 13:19:34.841599 19080 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:34.870249 10767 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:34.872010 19080 raft_consensus.cc:2749] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:34.879835 10767 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:19:34.880849 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 228415us
I20260709 13:19:34.884871 10766 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:34.828775 18557 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:34.892578 10692 catalog_manager.cc:5697] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
W20260709 13:19:34.894994 11478 leader_election.cc:336] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:19:34.902978 11478 leader_election.cc:336] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:34.897827 10767 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:19:34.890628 19141 log.cc:927] Time spent T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 13:19:34.890372 19114 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:34.852516 18379 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:34.919806 10692 catalog_manager.cc:5697] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 15 to 21, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 21 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:34.854575 18817 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:34.919548 19080 raft_consensus.cc:3060] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:19:34.935262 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 239275us
I20260709 13:19:34.908792 19150 raft_consensus.cc:2749] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:19:34.953766 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:34.954684 10942 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 22.
I20260709 13:19:34.905023 18557 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:34.921828 19114 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:19:34.957433 10692 catalog_manager.cc:5697] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 19 to 22. New cstate: current_term: 22 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
W20260709 13:19:34.933856 19145 log.cc:927] Time spent T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.088s	user 0.001s	sys 0.001s
I20260709 13:19:34.928653 19028 raft_consensus.cc:3060] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:19:34.971482 10842 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:34.931444 19122 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:34.973346 10692 catalog_manager.cc:5697] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 15 to 17, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 17 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:19:34.980866 10692 catalog_manager.cc:5697] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 24 to 28. New cstate: current_term: 28 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:35.063549 10692 catalog_manager.cc:5697] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 18 to 22. New cstate: current_term: 22 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:35.065193 10692 catalog_manager.cc:5697] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 18 to 22. New cstate: current_term: 22 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
W20260709 13:19:35.058468 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 127319us
I20260709 13:19:35.074080 10766 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:35.050509 18814 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:35.084362 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 135552us
W20260709 13:19:35.074409 19141 log.cc:927] Time spent T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 13:19:35.091223 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:35.091922 11505 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 24.
I20260709 13:19:35.091782 10692 catalog_manager.cc:5697] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 20 to 25, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 25 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:34.986327 19122 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:35.060701 19145 log.cc:927] Time spent T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
W20260709 13:19:35.063163 19105 log.cc:927] Time spent T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
W20260709 13:19:35.108544 19143 log.cc:927] Time spent T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.203s	user 0.001s	sys 0.000s
I20260709 13:19:35.086464 19148 raft_consensus.cc:2804] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:19:35.116351 19154 log.cc:927] Time spent T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.151s	user 0.000s	sys 0.000s
I20260709 13:19:35.084021 19048 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:19:35.098613 19146 log.cc:927] Time spent T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
I20260709 13:19:35.121373 19122 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:35.121677 19112 consensus_meta.cc:287] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586: Time spent flushing consensus metadata: real 0.602s	user 0.008s	sys 0.002s
I20260709 13:19:35.122682 18557 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:35.119421 19139 log.cc:927] Time spent T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
I20260709 13:19:35.127588 18817 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:35.129029 19148 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:35.170214 19148 raft_consensus.cc:3060] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:35.129127 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:35.168208 19137 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:19:35.131083  9682 thread.cc:527] Waited for 1000ms trying to join with heartbeat (tid 10826)
I20260709 13:19:35.133522 19068 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 29 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:35.135021 18379 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:35.173489 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:35.173375 19137 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:35.138427 19115 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:35.093377 18974 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:35.148113 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:35.068979 18549 raft_consensus.cc:493] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:35.175540 18549 raft_consensus.cc:515] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:35.176172 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 30 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:35.176815 10868 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 29.
I20260709 13:19:35.177846 18549 leader_election.cc:290] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:35.151691 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:35.160177 10841 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:35.106114 18618 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:35.172749 11505 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 26.
I20260709 13:19:35.136219 19034 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:35.123803 19080 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:35.178068 19137 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:35.161268 19004 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:35.184686 19063 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:35.184793 19004 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:35.185278 19063 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:35.185753 10841 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:35.129714 18878 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:19:35.187011 19004 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:35.187454 19063 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:35.187566 19004 raft_consensus.cc:2804] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:35.188038 19004 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:35.188373 19004 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:35.144311 10915 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:19:35.191139 10916 leader_election.cc:336] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:19:35.191282 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 47064.6us
W20260709 13:19:35.192093 10916 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 125 similar messages]
I20260709 13:19:35.192274 19157 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:35.192688 19157 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:35.192629 10915 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:35.193266 19157 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:19:35.193508 10916 leader_election.cc:336] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:35.193807 18974 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:19:35.193976 10916 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:35.194344 18974 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:35.194765 18974 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:35.196645 10871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54950 (request call id 2802) took 1169 ms. Trace:
I20260709 13:19:35.197362 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 30 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:35.197538 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:35.198194 11505 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 22.
I20260709 13:19:35.198342 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:35.150584 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:35.200196 10867 raft_consensus.cc:3060] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:19:35.202823 10915 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:35.203089 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:35.180365 19028 raft_consensus.cc:2749] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:35.201260 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:35.201876 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:35.216523 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:35.216629 18557 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:35.217278 18557 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:35.217584 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:35.206324 19033 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:19:35.219164 19033 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:35.219601 19033 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:19:35.219959 19090 raft_consensus.cc:515] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:35.223038 19090 leader_election.cc:290] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 29 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:35.223367 18557 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:35.223397 10947 raft_consensus.cc:2393] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 in current term 29: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:35.206395 19063 raft_consensus.cc:2749] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:35.207046 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:35.207279 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:35.197048 10871 rpcz_store.cc:276] 0709 13:19:34.027296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:34.029452 (+  2156us) service_pool.cc:224] Handling call
0709 13:19:34.071413 (+ 41961us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:34.248072 (+176659us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:34.248100 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:34.248488 (+   388us) log.cc:844] Serialized 26 byte log entry
0709 13:19:34.353754 (+105266us) spinlock_profiling.cc:243] Waited 1.31 ms on lock 0x7b48001602d8. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c85ddeaa1 00007f9c85dd5b37 00007f9c85dd59ca 00007f9c85ecd26f 00007f9c85eccf37 00007f9c85ee155c 00007f9c85ee1508 00007f9c85ee1488 00007f9c85ee1443 00007f9c85ee06ec 00007f9c84e0aa96
0709 13:19:34.768644 (+414890us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:34.768703 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:34.768776 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:35.196378 (+427602us) raft_consensus.cc:1685] finished
0709 13:19:35.196400 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:35.196586 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":176315,"raft.queue_time_us":192778,"raft.run_cpu_time_us":1227,"raft.run_wall_time_us":84690,"spinlock_wait_cycles":3020544,"thread_start_us":494064,"threads_started":1,"wal-append.queue_time_us":598143}
I20260709 13:19:35.192042 19081 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:35.209785 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:35.243306 10871 raft_consensus.cc:1275] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:19:35.243571 11505 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 23.
I20260709 13:19:35.210095 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5944b8838e584a3890e4e8143f3f2665" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 30 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:35.197750 19112 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:35.211869 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 128952us
W20260709 13:19:35.212952 19152 log.cc:927] Time spent T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.280s	user 0.001s	sys 0.000s
I20260709 13:19:35.162664 19158 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:35.230183 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:35.237587 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:35.247550 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1b662471ff4b9ebda3845b4f224a34" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:19:35.249346 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 62807.1us
I20260709 13:19:35.159878 18463 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:35.217808 18873 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:35.245107 10916 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:35.266572 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:35.272992 10868 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 24.
I20260709 13:19:35.275632 11511 raft_consensus.cc:2393] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 29: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:35.237174 19089 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:35.267765 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 28 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:35.277717 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:35.278631 10916 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:35.156158 19048 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:35.283615 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:35.237643 18557 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:35.308871 18557 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:35.310150 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:35.244789 19028 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:35.289577 10916 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:35.290403 10867 raft_consensus.cc:2468] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 29.
I20260709 13:19:35.291288 19051 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:19:35.276748 19049 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 13:19:35.264213 19064 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:35.265010 19081 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:35.343873 19157 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:35.344954 19081 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:35.244964 19114 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:35.277812 19158 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:35.346925 19028 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:35.348923 19158 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:35.306025 19148 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:35.349274 19028 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:35.350962 19148 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:35.351213 18974 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:35.353317 18974 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:35.280555 19089 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:35.354640 18878 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:35.354980 19089 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:35.311220 18557 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:35.284538 18175 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:35.312209 10841 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:19:35.313481 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:35.316509 10876 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 27.
I20260709 13:19:35.276526 18463 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:35.318568 19172 log.cc:927] Time spent T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:19:35.357172 18175 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:35.329030 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 49434.1us
W20260709 13:19:35.370002 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 57881.9us
I20260709 13:19:35.331384 10872 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:35.333272 11478 leader_election.cc:304] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2
W20260709 13:19:35.341342 10842 leader_election.cc:336] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:35.324324 10767 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:35.318202 19033 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:35.374007 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 84516.9us
W20260709 13:19:35.379522 10916 leader_election.cc:336] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:35.379977 10916 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:35.372349 10841 leader_election.cc:304] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:35.378690 19169 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:35.381053 19169 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:35.355216 18878 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:35.382968 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:35.383513 19169 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:35.384478 18878 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:35.337827 19051 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:19:35.375679 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 51452.8us
I20260709 13:19:35.386408 19051 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:35.381075 19111 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:35.387754 19111 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:35.388072 10767 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:35.389958 10767 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:35.390076 19111 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:35.346993 19114 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:35.337566 19162 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:35.338411 19049 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 29 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:35.393034 10767 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:35.393505 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:35.394345 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 28 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:35.395565 10942 raft_consensus.cc:2410] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 28 because replica has last-logged OpId of term: 26 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 10.
I20260709 13:19:35.396121 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:35.396317 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:35.396692 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:35.396311 11511 raft_consensus.cc:1275] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:19:35.391924 19150 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:35.401927 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:35.402421 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:35.399545 10869 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 20.
I20260709 13:19:35.402988 11509 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:35.404784 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:35.372493 10842 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:35.400168 10767 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:35.405434 11511 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 23.
I20260709 13:19:35.374536 19158 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:19:35.408190 10916 leader_election.cc:336] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:35.393074 19028 raft_consensus.cc:2749] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:35.390924 19080 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:19:35.399435 19123 raft_consensus.cc:2804] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 13:19:35.398934 18557 raft_consensus.cc:2749] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 13:19:35.409636 18175 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:35.415519 10767 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:35.417742 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:35.399734 19049 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 29 FOLLOWER]: Advancing to term 30
I20260709 13:19:35.399155 18814 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:35.436877 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:35.437529 11508 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 21.
I20260709 13:19:35.420948 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414992c8073646798991d2b43fd61f1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:35.414055 10841 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:19:35.427826 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:35.428099 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:35.404971 19114 raft_consensus.cc:2749] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:35.405745 19121 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:35.439893 11503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:35.440615 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 27 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:35.440959 11503 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 22.
I20260709 13:19:35.441376 11512 raft_consensus.cc:2410] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 27 because replica has last-logged OpId of term: 24 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 17.
I20260709 13:19:35.439909 10941 raft_consensus.cc:2393] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 24: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:35.440297 10942 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 21.
I20260709 13:19:35.430272 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:35.430622 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:35.431094 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:35.443145 10874 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 24.
I20260709 13:19:35.443612 10943 raft_consensus.cc:2368] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 26. Current term is 27.
I20260709 13:19:35.431444 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:35.430994 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:35.414644 19158 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:35.446343 11511 raft_consensus.cc:2368] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 24. Current term is 25.
I20260709 13:19:35.432732 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:35.433200 19148 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:19:35.435926 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 94614.8us
I20260709 13:19:35.439852 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:35.432305 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:35.522658 11505 raft_consensus.cc:2393] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 26: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:35.448402 11509 raft_consensus.cc:2468] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 25.
I20260709 13:19:35.408205 18618 consensus_queue.cc:1048] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.029s
W20260709 13:19:35.524806 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 83598.7us
I20260709 13:19:35.402148 19162 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:35.430636 19150 raft_consensus.cc:2749] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:19:35.483106 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 69092us
I20260709 13:19:35.502692 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:35.422887 19115 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
W20260709 13:19:35.452142 10842 leader_election.cc:336] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:35.542330 10842 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:35.504002 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 54061.8us
I20260709 13:19:35.504889 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:35.541435 11478 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
W20260709 13:19:35.506888 19173 log.cc:927] Time spent T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:19:35.358598 19064 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:35.508075 19051 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:35.446563 18814 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:35.515672 10872 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 22.
I20260709 13:19:35.517396 19048 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:35.538050 11508 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Advancing to term 22
I20260709 13:19:35.540105 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 27 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:35.540396 10916 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:35.544358 18175 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:19:35.568773 18175 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:35.569167 18175 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:19:35.545601 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 93515.6us
I20260709 13:19:35.507357 18463 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:35.575886 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:35.447206 19080 raft_consensus.cc:2749] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:19:35.444664 19123 raft_consensus.cc:697] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 29 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:35.452838 19158 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:35.570218 10915 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:19:35.570777 10842 leader_election.cc:336] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:19:35.573388 10916 leader_election.cc:336] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:35.537220 19160 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:35.578255 10944 raft_consensus.cc:2410] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for term 27 because replica has last-logged OpId of term: 24 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 17.
I20260709 13:19:35.445045 19168 raft_consensus.cc:2749] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:35.540179 19055 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:35.543541 19169 raft_consensus.cc:2749] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 13:19:35.603246 10842 leader_election.cc:336] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:35.583891 10950 raft_consensus.cc:1240] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 24. Current term is 25. Ops: []
I20260709 13:19:35.547653 18878 raft_consensus.cc:2749] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:19:35.602843 10949 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 21.
W20260709 13:19:35.607230 10842 leader_election.cc:336] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:35.528514 19148 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:35.579262 19090 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:19:35.615168 10842 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:35.609423 19153 raft_consensus.cc:2804] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:19:35.623153 18814 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:35.623730 19179 log.cc:927] Time spent T 00000000000000000000000000000000 P ae957c8fc66c4b6ba063d0c74cbf5484: Append to log took a long time: real 0.058s	user 0.002s	sys 0.000s
I20260709 13:19:35.548914 19115 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:35.625267 19033 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:35.627569 19064 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:35.632083 11508 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 22.
I20260709 13:19:35.605929 19160 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:35.611442 18695 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:35.638814 10842 leader_election.cc:336] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:35.612932 19157 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:35.646034 19051 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:35.620831 19090 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:35.647495 19090 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:19:35.649638 10766 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:35.651079 19165 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:35.651410 19150 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:35.651619 19165 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:35.652282 19150 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:35.653198 19028 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:35.653774 19028 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:35.654585 18674 raft_consensus.cc:2749] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:19:35.631982 19048 raft_consensus.cc:2749] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:19:35.643455 10842 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:35.620292 19112 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:35.636936 19153 raft_consensus.cc:697] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 25 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:35.642624 19160 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:35.664366 10766 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:35.612133 19055 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:35.662664 18549 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:35.678637 10766 leader_election.cc:400] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 24. Current term is 25.
I20260709 13:19:35.679057 10766 leader_election.cc:403] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:35.683606 19034 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:35.684121 19034 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:35.684464 19034 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:19:35.688539 19176 log.cc:927] Time spent T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
I20260709 13:19:35.673689 19080 consensus_queue.cc:1048] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.019s
I20260709 13:19:35.690398 19080 consensus_queue.cc:1243] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer 12682eb70bc84c76879e04a8628ee586 log is divergent from this leader: its last log entry 23.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 13:19:35.707486 10945 raft_consensus.cc:1240] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 25. Current term is 26. Ops: []
I20260709 13:19:35.712102 19153 consensus_queue.cc:237] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:35.717999 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 79233.7us
W20260709 13:19:35.718740 10842 leader_election.cc:336] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:35.718039 10766 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:35.722893 10766 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:35.701126 18618 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Advancing to term 25
I20260709 13:19:35.721674 19168 raft_consensus.cc:2749] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:35.658777 18695 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:35.742089 19115 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:35.742753 19115 raft_consensus.cc:2749] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:35.744300 19115 raft_consensus.cc:2749] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:19:35.746570 19034 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:35.748775 19034 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 election: Requested vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:35.665341 19004 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:35.700809 18549 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:35.764423 10871 raft_consensus.cc:2393] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 22: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:35.750543 19055 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:35.675143 19123 consensus_queue.cc:237] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.17, Last appended by leader: 17, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:35.770040 18695 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:35.774121 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 51324.9us
I20260709 13:19:35.775199 10766 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:35.776577 10766 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:35.777786 10766 leader_election.cc:400] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 26 pre-election: Vote denied by peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553) with higher term. Message: Invalid argument: T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 26. Current term is 27.
I20260709 13:19:35.777634 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:35.778064 10766 leader_election.cc:403] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 26 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:35.780652 10766 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:35.782436 19115 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Advancing to term 27
I20260709 13:19:35.783809 19080 raft_consensus.cc:2749] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:35.783701 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:35.785953 18549 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:35.783708 19004 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:35.789821 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:35.786895 19034 raft_consensus.cc:2804] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:35.795323 19034 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:35.795748 19034 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:35.800238 19028 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:35.804553 11508 raft_consensus.cc:1240] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 24. Current term is 25. Ops: []
I20260709 13:19:35.805261 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:35.805866 11504 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:35.806671 19115 raft_consensus.cc:2749] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Leader pre-election lost for term 26. Reason: Vote denied by peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553) with higher term. Message: Invalid argument: T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 26. Current term is 27.
I20260709 13:19:35.808389 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:19:35.821162 10842 leader_election.cc:336] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:35.822228 19150 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:35.822460 10842 leader_election.cc:336] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:35.827839 19162 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:35.793170 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:35.829020 10947 raft_consensus.cc:2393] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 21: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:35.794446 19165 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:35.831118 19148 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:35.833359 11505 raft_consensus.cc:1275] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
W20260709 13:19:35.837196 10842 leader_election.cc:336] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:35.839885 19118 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:35.856909 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:35.857566 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:35.858177 10867 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 20.
I20260709 13:19:35.859054 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:35.859505 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:19:35.849499 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 68907.8us
I20260709 13:19:35.859866 11504 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 22.
I20260709 13:19:35.848040 18175 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:35.860090 10867 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:35.860601 10766 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:19:35.863912 10842 leader_election.cc:336] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:35.864766 11505 raft_consensus.cc:1240] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 24. Current term is 25. Ops: [22.16-24.17]
I20260709 13:19:35.868952 10916 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:35.877296 18175 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:35.878067 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:35.878276 10916 leader_election.cc:336] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:35.879328 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:35.879925 11509 raft_consensus.cc:3060] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:19:35.880714 10949 raft_consensus.cc:1275] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 23 index: 11. Preceding OpId from leader: term: 25 index: 12. (index mismatch)
I20260709 13:19:35.888671 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:35.889150 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:35.884519 10916 leader_election.cc:336] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:19:35.890728 10916 leader_election.cc:336] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:35.891168 10916 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:35.892207 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:35.894039 10916 leader_election.cc:336] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:35.864610 10842 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:35.894475 10916 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:35.767603 19158 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:35.897626 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:35.898228 10950 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:35.905972 10867 raft_consensus.cc:2468] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 25.
I20260709 13:19:35.884895 19051 raft_consensus.cc:2804] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:19:35.907815 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 75597.1us
I20260709 13:19:35.912724 11509 raft_consensus.cc:2468] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 27.
W20260709 13:19:35.914570 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 50683.8us
W20260709 13:19:35.915560 10842 leader_election.cc:336] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:35.915985 10842 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:35.916484 10916 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:35.844596 19089 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:35.918467 10915 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:35.921401 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:35.922008 10944 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:35.925133 19063 raft_consensus.cc:2804] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:19:35.925676 19063 raft_consensus.cc:697] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 25 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:19:35.925913 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 65385.7us
I20260709 13:19:35.926966 18817 raft_consensus.cc:2749] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:19:35.870939 19118 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:35.926553 19063 consensus_queue.cc:237] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:35.902256 19153 raft_consensus.cc:2749] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:35.931195 10944 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 22.
I20260709 13:19:35.894438 19188 raft_consensus.cc:2749] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:35.939845 10766 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:35.940502 18878 consensus_queue.cc:1059] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:19:35.941332 19165 consensus_queue.cc:1059] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:19:35.941395 18817 consensus_queue.cc:1059] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 25.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:19:35.941984 10942 raft_consensus.cc:1275] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:19:35.942641 10941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:35.943142 10947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:35.956025 10941 raft_consensus.cc:3060] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:35.944789 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:35.889843 19090 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:35.954329 10950 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 23.
I20260709 13:19:35.910742 18618 raft_consensus.cc:2749] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 24. Current term is 25.
I20260709 13:19:35.914220 19051 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:35.909206 19033 raft_consensus.cc:2749] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:19:35.956653 10948 raft_consensus.cc:2393] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 23: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
I20260709 13:19:35.964020 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:35.965725 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 50194.5us
I20260709 13:19:35.918604 19005 raft_consensus.cc:2749] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:35.969359 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:35.978369 18878 raft_consensus.cc:2749] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:35.979280 18878 raft_consensus.cc:3055] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:19:35.979702 18878 raft_consensus.cc:740] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:35.978394 11504 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:35.972363 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:35.971107 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:35.967048 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:35.930928 18557 raft_consensus.cc:2804] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:19:35.971103 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:35.970253 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:35.973129 10841 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:36.002801 11508 raft_consensus.cc:2393] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 23: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:35.958631 18549 consensus_queue.cc:1048] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.039s
I20260709 13:19:36.015064 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:36.015964 10871 raft_consensus.cc:2368] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 23. Current term is 24.
I20260709 13:19:36.008359 11504 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 23.
I20260709 13:19:36.008543 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:35.999642 19034 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:36.010587 19090 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 28 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:36.011987 19158 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:35.971611 19051 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:35.991088 11505 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 23.
W20260709 13:19:36.025424 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 52373.7us
I20260709 13:19:36.025736 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:36.027158 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:36.029587 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce92c99123994b928d7a775917eb22ed" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:35.981009 19153 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:35.990826 19184 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:36.015996 19129 raft_consensus.cc:2804] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:36.044502 11477 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:19:36.030617 19180 log.cc:927] Time spent T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.275s	user 0.000s	sys 0.001s
I20260709 13:19:36.049041 18175 consensus_queue.cc:1048] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:19:36.052274 10841 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:36.052831 10767 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:36.053480 10841 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:36.054981 19028 raft_consensus.cc:2804] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:36.055521 19028 raft_consensus.cc:697] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:36.040917 10941 raft_consensus.cc:2468] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 25.
I20260709 13:19:36.031929 19169 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:36.027104 18557 raft_consensus.cc:697] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 27 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:36.050535 18175 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:36.063140 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:36.063786 11504 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:36.066999 18557 consensus_queue.cc:237] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:36.072466 19034 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:36.074268 11504 raft_consensus.cc:2468] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 24.
I20260709 13:19:36.078555 10950 raft_consensus.cc:1240] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 25 LEADER]: Rejecting Update request from peer 468a8fd46e824f62b9c2c0d3532a6d26 for earlier term 24. Current term is 25. Ops: []
I20260709 13:19:36.065047 11477 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:36.047538 19129 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:36.033917 19005 consensus_queue.cc:1048] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
W20260709 13:19:36.112149 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 47191.3us
I20260709 13:19:36.065537 19004 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:19:36.065048 19048 raft_consensus.cc:2804] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:36.038610 19153 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:36.115108 18175 raft_consensus.cc:2804] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:19:36.046500 19184 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:36.097324 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 43480.9us
I20260709 13:19:36.130007 18175 raft_consensus.cc:697] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 25 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:36.040192 18878 consensus_queue.cc:260] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:36.118394 19129 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:36.138990 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:36.139793 11504 raft_consensus.cc:2393] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 23: Already voted for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in this term.
I20260709 13:19:36.130885 18175 consensus_queue.cc:237] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:36.142566 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 28 candidate_status { last_received { term: 26 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:36.143188 10874 raft_consensus.cc:3060] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Advancing to term 28
W20260709 13:19:36.151064 10916 leader_election.cc:336] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:19:36.152542 10916 leader_election.cc:336] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:19:36.153796 10916 leader_election.cc:336] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:36.059656 19028 consensus_queue.cc:237] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:36.126811 19004 raft_consensus.cc:697] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:36.154363 10916 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:36.131680 19048 raft_consensus.cc:697] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:36.176841 19198 raft_consensus.cc:2749] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:36.177204 19048 consensus_queue.cc:237] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:36.173575  9682 thread.cc:527] Waited for 2000ms trying to join with heartbeat (tid 10826)
W20260709 13:19:36.186134 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 55969.4us
I20260709 13:19:36.139539 19089 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:36.187024 10767 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:36.194234 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:36.195303 10874 raft_consensus.cc:2468] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 28.
I20260709 13:19:36.136504 18463 consensus_queue.cc:1059] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.049s
I20260709 13:19:36.200634 18463 consensus_queue.cc:1243] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Peer cb0428cc55984daba1ba63744a5102a2 log is divergent from this leader: its last log entry 25.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 13:19:36.198716 10916 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:36.204901 19153 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:36.207722 19184 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:36.187837 19160 consensus_meta.cc:287] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26: Time spent flushing consensus metadata: real 0.507s	user 0.004s	sys 0.005s
I20260709 13:19:36.209766 19160 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:36.211300 10875 raft_consensus.cc:2368] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 23. Current term is 24.
I20260709 13:19:36.212561 19160 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:36.222193 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:36.223001 11504 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 26.
I20260709 13:19:36.225461 10767 leader_election.cc:400] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 23. Current term is 24.
I20260709 13:19:36.225772 10767 leader_election.cc:403] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:19:36.186861 19049 consensus_meta.cc:287] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2: Time spent flushing consensus metadata: real 0.735s	user 0.012s	sys 0.000s
I20260709 13:19:36.225939 10915 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:36.192876 19118 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:36.226588 19184 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:36.227170 19164 raft_consensus.cc:2749] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:36.238803 19187 raft_consensus.cc:3060] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Advancing to term 24
I20260709 13:19:36.228032 18557 raft_consensus.cc:2804] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 13:19:36.239512 19089 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:19:36.248140 10766 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:36.232707 19049 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:36.266295 10766 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:19:36.267323 11478 consensus_peers.cc:597] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:36.268049 11478 leader_election.cc:336] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:19:36.268862 11478 leader_election.cc:336] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:36.269245 11478 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:36.270279 11478 leader_election.cc:336] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:36.255265 19089 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:36.281435 19089 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:19:36.282502 19203 raft_consensus.cc:2749] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:36.272967 19112 raft_consensus.cc:2749] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:36.259210 18557 raft_consensus.cc:697] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 28 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:36.251650 19186 consensus_queue.cc:1059] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.025s
I20260709 13:19:36.285275 18557 consensus_queue.cc:237] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 26.11, Last appended by leader: 11, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:36.289206 10944 raft_consensus.cc:1240] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 30 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 29. Current term is 30. Ops: []
I20260709 13:19:36.252734 19184 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:36.291977 10766 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:36.262102 19051 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:36.296799 10947 raft_consensus.cc:1275] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 29 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
I20260709 13:19:36.247324 19004 consensus_queue.cc:237] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:36.304574 19049 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 30 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:36.308841 18175 consensus_queue.cc:1048] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:19:36.313606 19184 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:36.298631 19186 consensus_queue.cc:1243] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer 12682eb70bc84c76879e04a8628ee586 log is divergent from this leader: its last log entry 23.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 13:19:36.320708 19129 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:36.296365 19164 raft_consensus.cc:2749] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:36.324190 10766 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:36.350518 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:36.351342 10943 raft_consensus.cc:2393] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 22: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:36.352048 18674 raft_consensus.cc:3055] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:19:36.352487 18674 raft_consensus.cc:740] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:36.353339 18674 consensus_queue.cc:260] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:36.354313 18674 raft_consensus.cc:3060] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:36.359611 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:36.360880 11504 raft_consensus.cc:2393] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 22: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:36.375283 10766 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:36.376587 10948 raft_consensus.cc:1240] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 25. Current term is 26. Ops: []
I20260709 13:19:36.339826 19148 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:19:36.384014 19089 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:36.386293 19089 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:36.391808 11478 leader_election.cc:336] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:36.396633 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:36.403550 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 369102us
W20260709 13:19:36.404276 10842 leader_election.cc:336] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:19:36.405275 10842 leader_election.cc:336] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:36.405656 10842 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:36.407174 10874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 26 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:36.411166 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:36.411852 11509 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:36.412345 19148 raft_consensus.cc:697] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:36.413328 19148 consensus_queue.cc:237] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:36.349522 19080 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:36.415045 19080 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:36.415436 19080 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:36.416040 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 30 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:36.418134 18872 raft_consensus.cc:2749] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:36.427772 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:36.428448 11505 raft_consensus.cc:3060] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:19:36.432461 10842 consensus_peers.cc:597] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:36.433535 10842 leader_election.cc:336] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:19:36.434760 10842 leader_election.cc:336] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:36.422933 10841 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:19:36.445992 10874 raft_consensus.cc:1275] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 29 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
I20260709 13:19:36.439976 19005 raft_consensus.cc:2804] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:19:36.451429 18175 consensus_queue.cc:1048] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:19:36.458457 19115 raft_consensus.cc:3055] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:19:36.459328 19115 raft_consensus.cc:740] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:36.469010 18878 raft_consensus.cc:3060] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:36.468375 19051 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:36.477753 19005 raft_consensus.cc:697] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:36.479517 19005 consensus_queue.cc:237] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:36.482518 11509 raft_consensus.cc:2468] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 24.
W20260709 13:19:36.483659 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 108447us
I20260709 13:19:36.488503 11511 raft_consensus.cc:1275] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 13:19:36.495268 10841 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 
I20260709 13:19:36.476153 10842 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:36.490043 11505 raft_consensus.cc:2468] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 27.
W20260709 13:19:36.505530 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 70803us
W20260709 13:19:36.506482 10842 consensus_peers.cc:597] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:36.507187 10842 leader_election.cc:336] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:36.507647 10842 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:36.510138 10915 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:19:36.512763 10842 leader_election.cc:336] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:36.513279 10842 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:36.521523 19129 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:36.526895 18872 raft_consensus.cc:2749] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:36.512132 19150 consensus_queue.cc:1059] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 29.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.018s
I20260709 13:19:36.528795 19153 consensus_queue.cc:1048] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:19:36.486970 19165 raft_consensus.cc:2749] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:36.536168 19169 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:36.538411 19114 raft_consensus.cc:2804] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:19:36.539106 19114 raft_consensus.cc:697] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 27 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:36.539570 19160 raft_consensus.cc:2804] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:19:36.540257 19160 raft_consensus.cc:697] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:36.540050 19114 consensus_queue.cc:237] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:36.541406 19160 consensus_queue.cc:237] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:36.553429 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:19:36.570111 10916 leader_election.cc:336] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:36.525847 19115 consensus_queue.cc:260] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:36.492663 19187 raft_consensus.cc:2749] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 23. Current term is 24.
I20260709 13:19:36.574821 19150 raft_consensus.cc:3055] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 13:19:36.580156 19034 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:36.615284 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 102532us
W20260709 13:19:36.616369 10842 leader_election.cc:336] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:19:36.617434 10842 leader_election.cc:336] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:36.617960 10842 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:36.585984 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:36.586467 10766 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:19:36.571732 10916 leader_election.cc:336] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:19:36.619366 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 47666.8us
I20260709 13:19:36.620959 10942 raft_consensus.cc:2393] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 22: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
W20260709 13:19:36.622833 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 44347.1us
W20260709 13:19:36.620204 10916 consensus_peers.cc:597] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:19:36.624285 10916 leader_election.cc:336] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:36.625648 10916 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:36.632723 10945 raft_consensus.cc:1240] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 25 LEADER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 24. Current term is 25. Ops: []
W20260709 13:19:36.641156 10916 leader_election.cc:336] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:36.641714 10916 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:36.642824 10916 leader_election.cc:336] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:19:36.643862 10916 leader_election.cc:336] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:36.644335 10916 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:36.614881 19150 raft_consensus.cc:740] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
W20260709 13:19:36.657228 10916 leader_election.cc:336] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:36.669906 10916 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:36.672976 18557 raft_consensus.cc:2749] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 13:19:36.622905 19004 raft_consensus.cc:2749] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:36.650044 19184 raft_consensus.cc:2749] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:19:36.688380 11509 raft_consensus.cc:1240] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 24. Current term is 25. Ops: [22.16-24.17]
I20260709 13:19:36.632416 19187 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:19:36.615782 19188 consensus_queue.cc:1059] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: INVALID_TERM, Last received: 25.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.044s
I20260709 13:19:36.637077 18872 raft_consensus.cc:2749] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:36.703519 10945 raft_consensus.cc:1275] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 13:19:36.625125 19034 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:36.707474 19034 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:36.649688 19051 raft_consensus.cc:2749] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:36.713506 18175 consensus_queue.cc:1048] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:19:36.713567 19187 consensus_queue.cc:1059] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:19:36.714598 19187 consensus_queue.cc:1243] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer 12682eb70bc84c76879e04a8628ee586 log is divergent from this leader: its last log entry 23.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
W20260709 13:19:36.723057 10916 leader_election.cc:336] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:19:36.723160 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:36.727468 10916 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:36.675328 19049 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:36.710114 19080 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:36.756729 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:36.767237 18618 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:36.767876 18618 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:36.769682 10945 raft_consensus.cc:1275] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:19:36.770419 18618 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:36.772171 19137 consensus_queue.cc:1048] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:19:36.775580 10868 raft_consensus.cc:1275] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 25 index: 18. (term mismatch)
I20260709 13:19:36.750541 19090 raft_consensus.cc:2749] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:19:36.747023 19150 consensus_queue.cc:260] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 29.15, Last appended by leader: 15, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:36.852151 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:36.867530 10945 raft_consensus.cc:2393] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in current term 23: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:36.788797 18872 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:19:36.858379 10916 leader_election.cc:336] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:36.859369 10868 pending_rounds.cc:85] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting all ops after (but not including) 17
I20260709 13:19:36.871773 10868 pending_rounds.cc:107] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26: Aborting uncommitted NO_OP operation due to leader change: 24.18
I20260709 13:19:36.872184 10868 raft_consensus.cc:2889] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:19:36.874581 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:36.859262 11478 leader_election.cc:336] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:36.862159 10871 raft_consensus.cc:1275] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 13:19:36.864854 11504 raft_consensus.cc:1275] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 13:19:36.865880 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:19:36.883936 10916 leader_election.cc:336] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:36.888507 11478 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:36.890635 11478 leader_election.cc:336] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:36.855811 19168 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:19:36.889613 10916 consensus_peers.cc:597] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:19:36.891777 11478 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:36.900349 19169 raft_consensus.cc:2749] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:36.894646 18695 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:19:36.923254 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 295280us
I20260709 13:19:36.883848 19187 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:36.926537 10871 raft_consensus.cc:1275] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 28 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 26 index: 11. Preceding OpId from leader: term: 28 index: 12. (index mismatch)
I20260709 13:19:36.932577 10767 leader_election.cc:400] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Vote denied by peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) with higher term. Message: Invalid argument: T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 23. Current term is 24.
I20260709 13:19:36.934080 19184 consensus_queue.cc:1048] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:36.942106 10875 raft_consensus.cc:3060] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:19:36.954684 10916 consensus_peers.cc:597] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:36.959410 19080 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:36.960534 10875 raft_consensus.cc:2468] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 22.
I20260709 13:19:36.916707 19150 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 29 FOLLOWER]: Advancing to term 30
I20260709 13:19:36.954988 19115 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:19:36.874221 18872 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:36.925339 18558 raft_consensus.cc:2749] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:19:36.997192 10944 raft_consensus.cc:1275] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:19:36.915951 19168 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:36.924026 18814 consensus_queue.cc:1048] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.053s
W20260709 13:19:37.007205 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 116576us
W20260709 13:19:37.017266 11478 consensus_peers.cc:597] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:36.944000 19187 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:36.994182 19153 consensus_queue.cc:1048] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.033s
I20260709 13:19:37.031872 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:37.035745 11505 raft_consensus.cc:2368] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 23. Current term is 24.
I20260709 13:19:37.029827 19184 consensus_queue.cc:1048] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.009s
I20260709 13:19:37.034453 11511 raft_consensus.cc:1240] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 24. Current term is 25. Ops: [22.16-24.17]
I20260709 13:19:37.043439 10766 leader_election.cc:400] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 election: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 23. Current term is 24.
I20260709 13:19:37.043802 10766 leader_election.cc:403] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
W20260709 13:19:37.048308 10916 consensus_peers.cc:597] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:37.048856 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 51519.5us
W20260709 13:19:37.051034 10916 consensus_peers.cc:597] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:19:37.053876 19080 raft_consensus.cc:3060] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:37.014950 19165 consensus_queue.cc:1048] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.018s
W20260709 13:19:37.092074 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 55542.4us
I20260709 13:19:37.064917 19129 consensus_queue.cc:1059] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.026s
I20260709 13:19:37.116775 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:19:37.185981  9682 thread.cc:527] Waited for 3000ms trying to join with heartbeat (tid 10826)
I20260709 13:19:37.218477 11505 raft_consensus.cc:1275] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 13:19:37.118914 19169 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:37.124112 19129 consensus_queue.cc:1243] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer 12682eb70bc84c76879e04a8628ee586 log is divergent from this leader: its last log entry 23.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 13:19:37.124862 18873 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:37.211320 19165 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:37.241533 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:37.215734 19186 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:37.243733 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:37.247581 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:37.245039 10842 leader_election.cc:336] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:37.248981 10872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9e744bb20c4325969f4fbc01c672e8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:37.249434 10842 leader_election.cc:336] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:37.249765 10842 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:37.221707 19028 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:37.258420 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:37.260905 10764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:37.259157 10842 leader_election.cc:336] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:19:37.263590 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 133692us
W20260709 13:19:37.254487 19219 log.cc:927] Time spent T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.251s	user 0.000s	sys 0.001s
I20260709 13:19:37.260205 19005 raft_consensus.cc:2749] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:37.259948 19168 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:37.239760 18873 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:37.283152 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 65776.6us
I20260709 13:19:37.227486 19169 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:37.252300 19165 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:37.258050 19186 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:37.299461 18873 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:37.265725 19028 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:37.308907 10869 raft_consensus.cc:1275] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:19:37.290712 19165 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:37.291934 19187 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:37.332497 19229 log.cc:927] Time spent T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 13:19:37.334072 10876 raft_consensus.cc:1240] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Rejecting Update request from peer e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 24. Current term is 25. Ops: []
I20260709 13:19:37.340370 19028 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:37.315716 19234 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:37.346004 19234 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:37.348369 19234 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:37.357833 19169 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:37.364014 19186 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:37.377527 19215 log.cc:927] Time spent T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87: Append to log took a long time: real 0.458s	user 0.001s	sys 0.003s
I20260709 13:19:37.379949 19121 raft_consensus.cc:3055] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:19:37.380501 19121 raft_consensus.cc:740] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:37.381350 19121 consensus_queue.cc:260] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:37.386116 19210 log.cc:927] Time spent T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.412s	user 0.001s	sys 0.000s
W20260709 13:19:37.392247 10842 leader_election.cc:336] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:37.392901 19150 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:37.393432 19150 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:37.395807 19150 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:37.401780 10766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:37.411130 10842 consensus_peers.cc:597] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:37.408245 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 168909us
I20260709 13:19:37.382678 19121 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:37.353102 19080 raft_consensus.cc:2749] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555) with higher term. Message: Invalid argument: T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate e01ecd008edd4aa2b6b2b47efbc2ea87 for earlier term 23. Current term is 24.
I20260709 13:19:37.415978 18872 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:37.422185 19230 log.cc:927] Time spent T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
I20260709 13:19:37.423877 18674 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:37.424448 18674 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:37.427459 18674 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:37.430480 10767 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:37.432556 10767 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:37.433212 19246 raft_consensus.cc:2749] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:19:37.434386 18674 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:37.441169 11506 raft_consensus.cc:1275] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:19:37.443575 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 30 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:37.444378 10950 raft_consensus.cc:2393] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 30: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
W20260709 13:19:37.447341 10765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:37.448431 10767 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:37.451676 19246 raft_consensus.cc:2749] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:37.456123 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:37.434592 10916 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:37.456177 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:37.463143 10842 consensus_peers.cc:597] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:37.466158 11511 raft_consensus.cc:1275] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:19:37.469516 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:37.476701 10944 raft_consensus.cc:2393] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 26: Already voted for candidate cb0428cc55984daba1ba63744a5102a2 in this term.
W20260709 13:19:37.477830 10842 consensus_peers.cc:597] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:37.479209 10842 leader_election.cc:336] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:19:37.491948 19231 log.cc:927] Time spent T 00000000000000000000000000000000 P ae957c8fc66c4b6ba063d0c74cbf5484: Append to log took a long time: real 0.253s	user 0.001s	sys 0.001s
W20260709 13:19:37.491274 10842 leader_election.cc:336] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:19:37.494663 10842 leader_election.cc:336] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:19:37.498587 10842 leader_election.cc:336] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:37.478485 19184 raft_consensus.cc:2804] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:37.475876 19164 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:37.503808 19160 consensus_queue.cc:1048] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:37.506500 19165 consensus_queue.cc:1048] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:19:37.520478 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:37.522303 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:37.524273 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 26 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:37.525467 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:37.526618 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 26 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:37.529817 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:37.534994 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 26 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:37.522028 19184 raft_consensus.cc:697] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:37.536857 19184 consensus_queue.cc:237] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:37.526700 18814 consensus_queue.cc:1048] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.012s
W20260709 13:19:37.540009 19224 log.cc:927] Time spent T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.478s	user 0.003s	sys 0.000s
I20260709 13:19:37.518855 19164 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:37.542197 19164 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:37.544762 10949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:37.546056 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:37.551187 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 26 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:37.559160 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:37.531240 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:37.558624 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 58272.8us
I20260709 13:19:37.606737 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:37.609134 11505 raft_consensus.cc:1275] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:19:37.611007 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:37.644040 10766 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:37.649268 10767 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:37.654356 10767 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:37.642920 18674 raft_consensus.cc:2749] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:37.659238 19115 raft_consensus.cc:2749] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:19:37.655117 19187 raft_consensus.cc:2749] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:37.646188 19121 raft_consensus.cc:2749] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:37.658183 10767 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:37.680117 19165 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:37.681561 10767 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:37.685082 19165 raft_consensus.cc:2749] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:19:37.688282 11511 raft_consensus.cc:1275] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:19:37.686661 10767 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:37.704243 10767 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:37.641705 19150 consensus_queue.cc:1048] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.049s
W20260709 13:19:37.719700 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 155112us
I20260709 13:19:37.710393 19165 raft_consensus.cc:2749] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:19:37.719322 19188 raft_consensus.cc:2749] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:19:37.759631 10916 consensus_peers.cc:597] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:37.763302 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 277250us
I20260709 13:19:37.709928 19121 consensus_queue.cc:1048] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.040s
I20260709 13:19:37.738504 19080 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:37.793787 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 76278us
I20260709 13:19:37.795424 19080 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:37.798440 19080 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:37.800387 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 28 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:37.856596 19153 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:37.857273 19153 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:37.859861 19153 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:37.862427 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 28 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:37.866598 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 71598.8us
I20260709 13:19:37.745772 19137 consensus_queue.cc:1048] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.120s
I20260709 13:19:37.866518 19188 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:37.908257 19259 log.cc:927] Time spent T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.159s	user 0.000s	sys 0.000s
W20260709 13:19:37.909379 19257 log.cc:927] Time spent T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
I20260709 13:19:37.868894 18558 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:19:37.896122 10842 consensus_peers.cc:597] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:37.946079 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 51533.3us
W20260709 13:19:37.946774 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 70266.7us
W20260709 13:19:37.967204 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 71160.2us
W20260709 13:19:37.946971 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 54555.9us
I20260709 13:19:37.968987 10767 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:19:38.016705 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 47851.6us
I20260709 13:19:38.003610 18558 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:38.032114 19186 raft_consensus.cc:2749] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:19:37.944835 19188 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:38.035112 19188 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:38.022689 18558 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:38.051718 10942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:38.073549 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 55813.9us
I20260709 13:19:38.087131 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:38.099403 19239 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.099967 19239 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:38.100916 19187 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.101430 19187 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:38.102129 19239 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:38.105204 19245 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.105945 19245 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:38.107551 19187 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:38.109416 19245 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:38.121613 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:38.142084 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:38.154297 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 420109us
W20260709 13:19:38.154995 11478 consensus_peers.cc:597] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:19:38.155624 11478 leader_election.cc:336] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:19:38.156455 11478 consensus_peers.cc:597] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:19:38.160463 11504 raft_consensus.cc:1275] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 13:19:38.163307 10943 raft_consensus.cc:1275] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:19:38.174679 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 31 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:38.175549 10946 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 30.
W20260709 13:19:38.186667  9682 thread.cc:527] Waited for 4000ms trying to join with heartbeat (tid 10826)
I20260709 13:19:38.217738 19187 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.218528 19187 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:38.220955 19187 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:38.236083 10766 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:38.237648 19246 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:38.252043 19246 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:38.254442 19246 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:38.262378 19245 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:38.263146 19245 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:38.265462 19245 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:38.267364 18618 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 13:19:38.267891 18618 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:38.268611 18618 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 30 FOLLOWER]: Advancing to term 31
I20260709 13:19:38.274848 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:38.280690 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:38.281484 10943 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 26.
I20260709 13:19:38.287022 19246 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:38.287792 19246 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:38.290212 19246 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:38.293511 11478 leader_election.cc:336] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:38.332664 11478 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:38.334345 11478 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:38.302527 10841 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:38.303817 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 97704.7us
I20260709 13:19:38.306285 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:38.309294 19188 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:38.316972 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:38.318481 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:38.373265 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9828f212b47c19244738ee2a27a95" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:38.373657 18618 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:38.375661 10916 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.099s	sys 0.239s
W20260709 13:19:38.377316 10916 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.100s	sys 0.239s
W20260709 13:19:38.377422 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 75007.2us
W20260709 13:19:38.377588 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 600395us
I20260709 13:19:38.396452 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:38.354140 18804 raft_consensus.cc:2749] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:38.387439 19228 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 29 FOLLOWER]: Advancing to term 30
I20260709 13:19:38.387701 19111 raft_consensus.cc:2749] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:38.409559 10841 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:38.420751 11478 leader_election.cc:336] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:19:38.404670 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 72720.4us
I20260709 13:19:38.421442 11478 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:38.425364 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:38.425726 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 31 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:38.427708 18175 raft_consensus.cc:2749] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:38.428718 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:38.430387 10841 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:38.424091 10766 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:38.451252 10841 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:38.412216 19080 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.410195 19186 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.458926 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cab289810294719ac9686e40290a6ab" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:38.444460 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:38.467237 10766 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:38.443173 19246 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:19:38.471551 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:38.473711 10686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 68668.2us
I20260709 13:19:38.465610 10695 catalog_manager.cc:5697] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:38.439965 19169 raft_consensus.cc:2749] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:19:38.457971 18878 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:19:38.390939 19188 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:38.458298 19269 raft_consensus.cc:2749] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:38.495071 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 31 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:19:38.503145 10686 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 2b82f18992434ed5ba58b805107060d4 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 468a8fd46e824f62b9c2c0d3532a6d26 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 24 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:38.511473 10767 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:38.514250 19203 raft_consensus.cc:2749] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:19:38.481330 19246 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:38.473610 19165 raft_consensus.cc:2749] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:38.469445 19080 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:38.504774 19157 consensus_queue.cc:1048] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.038s
I20260709 13:19:38.465680 18463 consensus_queue.cc:1048] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 13:19:38.471300 19186 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:38.545769 10695 catalog_manager.cc:5697] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 13 to 15, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 15 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
W20260709 13:19:38.539357 10842 consensus_peers.cc:597] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:19:38.553254 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 95919.5us
W20260709 13:19:38.559724 10692 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42968 (request call id 250) took 14837 ms (client timeout 15000 ms). Trace:
W20260709 13:19:38.567163 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 80709.6us
I20260709 13:19:38.563588 10767 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:38.541837 19246 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:19:38.585880 10767 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:38.588011 10767 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:38.603485 19115 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:19:38.560393 10692 rpcz_store.cc:269] 0709 13:19:23.715801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.716111 (+   310us) service_pool.cc:224] Handling call
0709 13:19:38.553213 (+14837102us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:35.291632 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:35.294356 (+  2724us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:35.294394 (+    38us) write_op.cc:435] Acquired schema lock
0709 13:19:35.294415 (+    21us) tablet.cc:662] Decoding operations
0709 13:19:35.296491 (+  2076us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:35.296580 (+    89us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:35.296614 (+    34us) tablet.cc:685] Acquiring locks for 24 operations
0709 13:19:35.298746 (+  2132us) tablet.cc:701] Row locks acquired
0709 13:19:35.298768 (+    22us) write_op.cc:260] PREPARE: finished
0709 13:19:35.352257 (+ 53489us) write_op.cc:270] Start()
0709 13:19:35.352375 (+   118us) write_op.cc:276] Timestamp: P: 1783603175352205 usec, L: 0
0709 13:19:35.352396 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:19:35.353143 (+   747us) log.cc:844] Serialized 9378 byte log entry
0709 13:19:35.774798 (+421655us) op_driver.cc:464] REPLICATION: finished
0709 13:19:35.971714 (+196916us) write_op.cc:301] APPLY: starting
0709 13:19:35.976430 (+  4716us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:36.383429 (+406999us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:36.383549 (+   120us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:36.793558 (+410009us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:36.903022 (+109464us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=48,key_file_lookups=48,delta_file_lookups=48,mrs_lookups=0
0709 13:19:36.903084 (+    62us) write_op.cc:312] APPLY: finished
0709 13:19:36.903602 (+   518us) log.cc:844] Serialized 212 byte log entry
0709 13:19:37.076959 (+173357us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:37.095085 (+ 18126us) write_op.cc:454] Released schema lock
0709 13:19:37.095161 (+    76us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:37.095641 (+   480us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:37.095672 (+    31us) write_op.cc:454] Released schema lock
0709 13:19:37.452266 (+356594us) spinlock_profiling.cc:243] Waited 23.4 ms on lock 0x7b4400022b4c. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dceec85 00007f9c7dcef6a2 00007f9c80a611db 00007f9c80a5caaf 00007f9c80a5086e 00007f9c80a4fbb6 00007f9c80a4f7d8 00007f9c80a59221 00007f9c80a591d9 00007f9c80a59169
Metrics: {"mutex_wait_us":3067365,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":169766,"apply.run_cpu_time_us":93937,"apply.run_wall_time_us":1144862,"cfile_cache_hit":98,"cfile_cache_hit_bytes":104495,"num_ops":24,"prepare.queue_time_us":42284,"prepare.run_cpu_time_us":19928,"prepare.run_wall_time_us":129881,"raft.queue_time_us":149569,"raft.run_cpu_time_us":42935,"raft.run_wall_time_us":141367,"replication_time_us":422114,"spinlock_wait_cycles":53805056,"thread_start_us":471591,"threads_started":5,"wal-append.queue_time_us":309328,"wal-append.run_cpu_time_us":100228,"wal-append.run_wall_time_us":2553685}]]}
I20260709 13:19:38.591557 19164 raft_consensus.cc:2749] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:19:38.618120 10695 catalog_manager.cc:5697] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 11 to 15, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 15 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
W20260709 13:19:38.617115 10916 consensus_peers.cc:597] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:38.625960 10686 catalog_manager.cc:4755] TS e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): ChangeConfig:REMOVE_PEER RPC failed for tablet 695ff1b5a68e4111917d909909488d39: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:38.631476 19186 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:38.635294 18618 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 31 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:38.600660 19252 raft_consensus.cc:2749] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:38.656009 19188 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:38.658751 10695 catalog_manager.cc:5697] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 16 to 18, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 18 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:38.661286 10695 catalog_manager.cc:5697] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 reported cstate change: config changed from index 13 to 8, term changed from 22 to 15, leader changed from <none> to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132), VOTER cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) added. New cstate: current_term: 15 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:19:38.664980 10916 consensus_peers.cc:597] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:19:38.667680 10916 leader_election.cc:336] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:38.668107 10916 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:38.669189 10916 consensus_peers.cc:597] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:19:38.669915 10916 consensus_peers.cc:597] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:38.621838 19080 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:38.671907 19279 log.cc:927] Time spent T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:19:38.672614 19153 raft_consensus.cc:2749] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:19:38.730433 19228 raft_consensus.cc:2749] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 13:19:38.736717 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 232044us
I20260709 13:19:38.753439 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:38.754438 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:38.754889 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171abdebfc4142cda86de477a5c89a7e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:38.756418 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:38.756625 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:19:38.755920 10842 consensus_peers.cc:597] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:19:38.736717 18878 raft_consensus.cc:2749] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:19:38.765136 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 168738us
I20260709 13:19:38.758443 19148 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.778043 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 31 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:38.779897 18463 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:38.780624 10946 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 30 FOLLOWER]: Advancing to term 31
I20260709 13:19:38.800527 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:38.802484 10767 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:38.806439 10873 raft_consensus.cc:1275] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:19:38.774715 19129 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.811060 18379 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:38.811579 10767 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:38.780685 18463 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:38.779204 19203 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.860509 10946 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 31.
W20260709 13:19:38.870957 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 44797.2us
I20260709 13:19:38.776619 19148 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:38.826484 19080 raft_consensus.cc:2749] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:38.914952 10766 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:38.918059 18379 raft_consensus.cc:2749] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:19:38.918462 19286 log.cc:927] Time spent T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 13:19:38.933705 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 56805.7us
I20260709 13:19:38.938946 18463 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:38.816147 19282 consensus_queue.cc:1048] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.093s
I20260709 13:19:38.946264 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:38.855952 19203 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:38.820958 19129 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:38.956250 10766 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26
W20260709 13:19:38.965741 10842 consensus_peers.cc:597] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:19:38.984705 19129 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:38.986418 19148 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:38.992393 19203 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:39.003314 19090 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:39.004074 19090 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:38.944823 19226 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.006417 19090 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:39.029570 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:19:39.020637 10842 consensus_peers.cc:597] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:19:39.030454 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:39.012398 19121 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 31 FOLLOWER]: Leader election won for term 31
W20260709 13:19:39.033807 19286 log.cc:927] Time spent T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:19:39.017200 19246 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:39.073210 19034 raft_consensus.cc:493] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.076170 19121 raft_consensus.cc:697] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 31 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:39.034053 19226 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:39.098587 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:39.101146 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:39.112759 11506 raft_consensus.cc:2393] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468a8fd46e824f62b9c2c0d3532a6d26 in current term 25: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:39.120999 19226 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:39.123363 19246 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 27 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:39.097276 19034 raft_consensus.cc:515] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:39.126605 19034 leader_election.cc:290] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:39.119042 19121 consensus_queue.cc:237] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 29.15, Last appended by leader: 15, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:39.188478  9682 thread.cc:527] Waited for 5000ms trying to join with heartbeat (tid 10826)
I20260709 13:19:39.203675 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:39.206605 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:39.207757 10876 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 24.
I20260709 13:19:39.203482 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418b3711416f4a3e8fa80068035b5dd8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:39.211829 10937 tablet_service.cc:1558] Processing DeleteTablet for tablet 171abdebfc4142cda86de477a5c89a7e with delete_type TABLET_DATA_TOMBSTONED (TS cb0428cc55984daba1ba63744a5102a2 not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:57812
I20260709 13:19:39.214805 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9094285c302f4de4aab986dc171c11ff" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:39.215668 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:39.221874 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:39.240135 10943 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:19:39.223994 18618 raft_consensus.cc:493] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.249607 18618 raft_consensus.cc:515] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:39.231359 19034 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.251770 19034 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:39.253219 18618 leader_election.cc:290] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:39.254684 19034 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:39.277066 10766 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:39.292621 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:39.293828 10869 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:19:39.293933 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19c20efd6c04443596bb712e3c93c650" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:39.292804 10944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:39.315666 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:39.323159 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741efe79e97646b396f0091dc10c745e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:39.346954 10870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8949b8b2d08942939f64373bccb71936" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:39.354511 10943 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 27.
W20260709 13:19:39.362941 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 390330us
W20260709 13:19:39.367024 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 90329.9us
I20260709 13:19:39.376121 19034 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:39.375751 19310 tablet_replica.cc:333] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:19:39.389392 10869 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 27.
I20260709 13:19:39.416177 10873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:39.417271 10873 raft_consensus.cc:2410] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12682eb70bc84c76879e04a8628ee586 for term 26 because replica has last-logged OpId of term: 24 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 16.
W20260709 13:19:39.419359 11478 leader_election.cc:336] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:39.426759 10766 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
W20260709 13:19:39.437420 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 139207us
I20260709 13:19:39.453910 10766 leader_election.cc:304] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:39.458703 10767 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586
I20260709 13:19:39.463177 19186 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:19:39.463939 19186 raft_consensus.cc:697] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:39.465358 19186 consensus_queue.cc:237] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:39.469688 19246 raft_consensus.cc:2749] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:19:39.471619 11478 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:39.466001 10766 leader_election.cc:304] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:39.473892 19226 raft_consensus.cc:2749] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:19:39.481592 19245 raft_consensus.cc:2749] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:39.490173 19034 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:19:39.490681 19034 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:39.491065 19034 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:19:39.513154 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 70391.1us
W20260709 13:19:39.523057 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 65125.5us
I20260709 13:19:39.524323 10767 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:39.525525 19165 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:19:39.529717 10842 consensus_peers.cc:597] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:19:39.530622 10842 consensus_peers.cc:597] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:19:39.531312 10842 consensus_peers.cc:597] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:19:39.543077 10916 leader_election.cc:336] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:39.545671 10916 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:39.546738 19090 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:39.533869 10696 catalog_manager.cc:5697] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 18 to 19, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 19 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:39.549628 10696 catalog_manager.cc:5697] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 15 to 16, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 16 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:39.564157 10696 catalog_manager.cc:5697] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 14 to 22. New cstate: current_term: 22 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:39.576526 19150 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:39.550419 10916 consensus_peers.cc:597] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:19:39.543146 19034 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:39.577266 19150 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:39.579162 19034 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 26 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:39.579872 19150 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:39.532056 10842 leader_election.cc:336] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:39.596987 10842 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:39.597518 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:39.598215 11508 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:19:39.606623 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 74631.3us
I20260709 13:19:39.612326 19298 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:39.612643 11508 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 26.
I20260709 13:19:39.618920 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:39.621352 10766 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 
I20260709 13:19:39.626919 19252 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:19:39.627514 19252 raft_consensus.cc:697] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 LEADER]: Becoming Leader. State: Replica: e01ecd008edd4aa2b6b2b47efbc2ea87, State: Running, Role: LEADER
I20260709 13:19:39.628551 19252 consensus_queue.cc:237] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:39.635629 10767 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:39.639092 19165 raft_consensus.cc:2749] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:19:39.645285 10695 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42506 (request call id 158) took 62529 ms (client timeout 15000 ms). Trace:
I20260709 13:19:39.649238 19310 raft_consensus.cc:2243] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:39.651345 19310 raft_consensus.cc:2272] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:19:39.645792 10695 rpcz_store.cc:269] 0709 13:18:37.115654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.115899 (+   245us) service_pool.cc:224] Handling call
0709 13:19:39.645052 (+62529153us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:38.928991 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:38.929602 (+   611us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:38.929657 (+    55us) write_op.cc:435] Acquired schema lock
0709 13:19:38.929670 (+    13us) tablet.cc:662] Decoding operations
0709 13:19:38.930332 (+   662us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:38.930387 (+    55us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:38.930419 (+    32us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:19:38.930953 (+   534us) tablet.cc:701] Row locks acquired
0709 13:19:38.930972 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:19:38.931115 (+   143us) write_op.cc:270] Start()
0709 13:19:38.931239 (+   124us) write_op.cc:276] Timestamp: P: 1783603178931069 usec, L: 0
0709 13:19:38.931261 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:19:38.931911 (+   650us) log.cc:844] Serialized 2132 byte log entry
0709 13:19:39.128222 (+196311us) spinlock_profiling.cc:243] Waited 1.23 ms on lock 0x7b18001464a0. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c86c90de1 00007f9c86c9d5b1 00007f9c7ddc83f3 00007f9c7ddca7c1 00007f9c7ddca779 00007f9c7ddca709 00007f9c7ddca6d1 00007f9c7ddc99cd 000000000038f354 000000000038f199 00007f9c7ddab83a
0709 13:19:39.140138 (+ 11916us) op_driver.cc:464] REPLICATION: finished
0709 13:19:39.141503 (+  1365us) write_op.cc:301] APPLY: starting
0709 13:19:39.141586 (+    83us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:39.144735 (+  3149us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:39.144757 (+    22us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:39.312866 (+168109us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:39.313374 (+   508us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=10,mrs_lookups=0
0709 13:19:39.313474 (+   100us) write_op.cc:312] APPLY: finished
0709 13:19:39.314070 (+   596us) log.cc:844] Serialized 57 byte log entry
0709 13:19:39.315974 (+  1904us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:39.316288 (+   314us) write_op.cc:454] Released schema lock
0709 13:19:39.316390 (+   102us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:39.343343 (+ 26953us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:39.343382 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":145894,"reported_tablets":31,"child_traces":[["op",{"apply.queue_time_us":1105,"apply.run_cpu_time_us":11878,"apply.run_wall_time_us":189125,"cfile_cache_hit":22,"cfile_cache_hit_bytes":25037,"num_ops":5,"prepare.queue_time_us":197826,"prepare.run_cpu_time_us":42530,"prepare.run_wall_time_us":182517,"raft.queue_time_us":9878,"raft.run_cpu_time_us":1452,"raft.run_wall_time_us":1452,"replication_time_us":208681,"spinlock_wait_cycles":2827008,"thread_start_us":235114,"threads_started":4,"wal-append.queue_time_us":157661}]]}
I20260709 13:19:39.672127 10696 catalog_manager.cc:5697] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:39.668982 19186 raft_consensus.cc:493] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.676389 19310 ts_tablet_manager.cc:1916] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:19:39.683264 10869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4c0ff1def4960931dda3b52a4a884" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:39.689072 10696 catalog_manager.cc:5697] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 18 to 22. New cstate: current_term: 22 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:39.692205 10696 catalog_manager.cc:5697] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 reported cstate change: config changed from index 8 to 13, term changed from 15 to 18, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to <none>, VOTER cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) evicted. New cstate: current_term: 18 committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:39.699853 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
W20260709 13:19:39.720999 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 79617.5us
I20260709 13:19:39.739051 19186 raft_consensus.cc:515] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:39.746538 10945 raft_consensus.cc:1275] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 31 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 29 index: 15. Preceding OpId from leader: term: 31 index: 16. (index mismatch)
I20260709 13:19:39.748678 19252 consensus_queue.cc:1048] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:19:39.777237 19298 raft_consensus.cc:2749] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:39.778091 10867 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:19:39.765558 11478 consensus_peers.cc:597] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:19:39.797358 10842 consensus_peers.cc:597] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:19:39.802328 10842 consensus_peers.cc:597] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:19:39.815797 19186 leader_election.cc:290] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:39.817730 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:39.821377 10867 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01ecd008edd4aa2b6b2b47efbc2ea87 in term 26.
I20260709 13:19:39.845635 10767 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:39.850361 19252 raft_consensus.cc:493] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.851123 19252 raft_consensus.cc:515] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:39.852146 19034 raft_consensus.cc:2749] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:19:39.856477 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:39.850551 19150 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.860553 19150 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:39.863308 19252 leader_election.cc:290] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:39.878211 10869 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 30 FOLLOWER]: Advancing to term 31
I20260709 13:19:39.899816 19150 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:39.904107 19252 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.904795 19252 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:39.922359 10869 raft_consensus.cc:1275] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 31 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 29 index: 15. Preceding OpId from leader: term: 31 index: 16. (index mismatch)
I20260709 13:19:39.936798 19165 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.938247 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:39.941670 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651c4c65e7f44b1fb4053d3757ac417f" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:39.942898 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "417caa22425d4a25aca2f7d8e9577256" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:39.943620 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73af6c7e9b8e4ffaa65ba65a6199eb61" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:39.944021 10875 raft_consensus.cc:1275] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 13:19:39.948228 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:39.950220 19252 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:39.952963 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e90b642b0d4f06b866170dc0293dc1" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:19:39.958513 11478 consensus_peers.cc:597] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:19:39.969820 19165 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:39.984472 10842 consensus_peers.cc:597] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:19:39.993449 10767 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:39.994441 19188 raft_consensus.cc:2749] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:39.994763 10767 leader_election.cc:304] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:39.995764 19188 raft_consensus.cc:2749] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:39.998291 10767 leader_election.cc:304] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:40.007309 10767 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:40.009702 19199 raft_consensus.cc:2749] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:40.012725 19034 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.013331 19034 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:40.015535 19034 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:40.015890 19186 raft_consensus.cc:2749] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:40.017370 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:40.020588 19187 consensus_queue.cc:1048] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:19:40.020818 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12d33f7a0f8f47e5ad57e854e52ae154" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:40.031874 19165 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:40.003242 19188 consensus_queue.cc:1048] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.029s
I20260709 13:19:40.036603 10766 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:40.039733 19186 raft_consensus.cc:2749] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:40.049649 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:40.062572 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:40.095187 10766 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
W20260709 13:19:40.179514 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 80924.2us
W20260709 13:19:40.192384  9682 thread.cc:527] Waited for 6000ms trying to join with heartbeat (tid 10826)
I20260709 13:19:40.151805 19187 raft_consensus.cc:2749] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:40.188977 19165 raft_consensus.cc:493] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:40.208518 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 113414us
I20260709 13:19:40.209563 19121 raft_consensus.cc:493] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.210337 19121 raft_consensus.cc:515] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:40.221748 19203 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.222677 19203 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:40.223658 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:40.223405 19165 raft_consensus.cc:515] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:40.227631 19203 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:40.227809 10945 raft_consensus.cc:1275] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 13:19:40.232439 19310 ts_tablet_manager.cc:1929] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 17.12
I20260709 13:19:40.233191 19310 log.cc:1199] T 171abdebfc4142cda86de477a5c89a7e P cb0428cc55984daba1ba63744a5102a2: Deleting WAL directory at /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-2-root/wals/171abdebfc4142cda86de477a5c89a7e
I20260709 13:19:40.251694 19165 leader_election.cc:290] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:40.252405 10868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:40.253830 10871 raft_consensus.cc:1275] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 13:19:40.263159 19310 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57812 (request call id 0) took 1052 ms. Trace:
I20260709 13:19:40.265269 19310 rpcz_store.cc:276] 0709 13:19:39.210537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:39.211420 (+   883us) service_pool.cc:224] Handling call
0709 13:19:39.374865 (+163445us) ts_tablet_manager.cc:1151] Deleting tablet 171abdebfc4142cda86de477a5c89a7e
0709 13:19:39.374975 (+   110us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:19:39.912775 (+537800us) tablet_metadata.cc:665] Metadata flushed
0709 13:19:40.232260 (+319485us) tablet_metadata.cc:665] Metadata flushed
0709 13:19:40.263024 (+ 30764us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":228367,"tablet-delete.queue_time_us":142204,"thread_start_us":71037,"threads_started":1}
I20260709 13:19:40.276166 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:40.279016 19329 consensus_queue.cc:1048] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:19:40.288821 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35190606a0c04bc4bac6c8ca7df14ed3" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:40.289546 19121 leader_election.cc:290] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:40.293749 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9354ad0d744d43bbbe2ce992ecbc4e65" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:40.294967 19329 consensus_queue.cc:1048] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:19:40.300791 19329 raft_consensus.cc:493] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 12682eb70bc84c76879e04a8628ee586)
I20260709 13:19:40.301585 19329 raft_consensus.cc:515] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:40.311151 10685 catalog_manager.cc:5028] TS cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): tablet 171abdebfc4142cda86de477a5c89a7e (table test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]) successfully deleted
I20260709 13:19:40.319655 10766 leader_election.cc:304] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:40.323575 19150 raft_consensus.cc:2749] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:40.305896 19329 leader_election.cc:290] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:40.331898 10766 leader_election.cc:304] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:40.333166 19186 raft_consensus.cc:2749] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:19:40.348826 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8a233bc6c44bd2b5d2d44a726b147c" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:40.349661 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:40.353155 11505 raft_consensus.cc:1275] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Refusing update from remote peer e01ecd008edd4aa2b6b2b47efbc2ea87: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 13:19:40.364236 10766 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:40.374557 19333 raft_consensus.cc:2749] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:19:40.380404 11478 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 213 similar messages]
I20260709 13:19:40.369652 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416a6d12184f4fbda69570b95be47851" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:40.391956 10696 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42506 (request call id 159) took 38058 ms (client timeout 15000 ms). Trace:
W20260709 13:19:40.392450 10696 rpcz_store.cc:269] 0709 13:19:02.333757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:02.334018 (+   261us) service_pool.cc:224] Handling call
0709 13:19:40.391874 (+38057856us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:39.822029 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:39.822706 (+   677us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:39.822758 (+    52us) write_op.cc:435] Acquired schema lock
0709 13:19:39.822778 (+    20us) tablet.cc:662] Decoding operations
0709 13:19:39.824672 (+  1894us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:39.824778 (+   106us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:39.824810 (+    32us) tablet.cc:685] Acquiring locks for 6 operations
0709 13:19:39.825440 (+   630us) tablet.cc:701] Row locks acquired
0709 13:19:39.825496 (+    56us) write_op.cc:260] PREPARE: finished
0709 13:19:39.825663 (+   167us) write_op.cc:270] Start()
0709 13:19:39.825809 (+   146us) write_op.cc:276] Timestamp: P: 1783603179825593 usec, L: 0
0709 13:19:39.825832 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:19:39.826756 (+   924us) log.cc:844] Serialized 2418 byte log entry
0709 13:19:40.029286 (+202530us) op_driver.cc:464] REPLICATION: finished
0709 13:19:40.142152 (+112866us) write_op.cc:301] APPLY: starting
0709 13:19:40.142292 (+   140us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:40.215820 (+ 73528us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:40.220244 (+  4424us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:40.307981 (+ 87737us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:40.308224 (+   243us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=12,key_file_lookups=12,delta_file_lookups=12,mrs_lookups=0
0709 13:19:40.308275 (+    51us) write_op.cc:312] APPLY: finished
0709 13:19:40.308534 (+   259us) log.cc:844] Serialized 65 byte log entry
0709 13:19:40.308736 (+   202us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:40.308849 (+   113us) write_op.cc:454] Released schema lock
0709 13:19:40.308894 (+    45us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:40.311395 (+  2501us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:40.342862 (+ 31467us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":40678,"reported_tablets":31,"child_traces":[["op",{"apply.queue_time_us":112517,"apply.run_cpu_time_us":12996,"apply.run_wall_time_us":169244,"cfile_cache_hit":26,"cfile_cache_hit_bytes":29219,"num_ops":6,"prepare.queue_time_us":119732,"prepare.run_cpu_time_us":4259,"prepare.run_wall_time_us":7798,"replication_time_us":203174,"spinlock_wait_cycles":2432,"thread_start_us":175245,"threads_started":2}]]}
W20260709 13:19:40.408210 10842 consensus_peers.cc:597] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:19:40.408262 10692 catalog_manager.cc:5697] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 20 to 21, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 21 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:19:40.427338 10692 catalog_manager.cc:5697] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 18 to 25. New cstate: current_term: 25 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:40.428818 10692 catalog_manager.cc:5697] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 26 to 29. New cstate: current_term: 29 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:40.430194 10692 catalog_manager.cc:5697] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 19 to 25. New cstate: current_term: 25 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:40.433627 10692 catalog_manager.cc:5697] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 21 to 28, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 28 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:19:40.435940 10766 leader_election.cc:304] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:40.420372 10937 tablet_service.cc:1558] Processing DeleteTablet for tablet 171abdebfc4142cda86de477a5c89a7e with delete_type TABLET_DATA_TOMBSTONED (TS cb0428cc55984daba1ba63744a5102a2 not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:57812
I20260709 13:19:40.411295 19186 consensus_queue.cc:1048] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:19:40.440968 19186 consensus_queue.cc:1243] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [LEADER]: Peer 12682eb70bc84c76879e04a8628ee586 log is divergent from this leader: its last log entry 23.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
W20260709 13:19:40.441660 11478 consensus_peers.cc:597] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:19:40.442443 19187 raft_consensus.cc:2749] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:40.443501 10685 catalog_manager.cc:5028] TS cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): tablet 171abdebfc4142cda86de477a5c89a7e (table test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]) successfully deleted
I20260709 13:19:40.435376 10692 catalog_manager.cc:5697] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 18 to 22. New cstate: current_term: 22 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:40.445578 10692 catalog_manager.cc:5697] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 17 to 23. New cstate: current_term: 23 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:40.447132 10692 catalog_manager.cc:5697] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 24 to 27, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 27 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:40.448582 10692 catalog_manager.cc:5697] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 14 to 23, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 23 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:40.450184 10692 catalog_manager.cc:5697] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 15 to 21. New cstate: current_term: 21 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:40.453225 10692 catalog_manager.cc:5697] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 26 to 28. New cstate: current_term: 28 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:19:40.457015 11509 pending_rounds.cc:85] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586: Aborting all ops after (but not including) 15
I20260709 13:19:40.460701 11509 pending_rounds.cc:107] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586: Aborting uncommitted NO_OP operation due to leader change: 23.16
I20260709 13:19:40.461054 11509 raft_consensus.cc:2889] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:40.470014 10692 catalog_manager.cc:5697] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:40.471665 10692 catalog_manager.cc:5697] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 14 to 23. New cstate: current_term: 23 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:19:40.472997 10692 catalog_manager.cc:5697] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 15 to 26. New cstate: current_term: 26 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:40.483772 19186 raft_consensus.cc:493] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.484970 19186 raft_consensus.cc:515] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:40.474270 10692 catalog_manager.cc:5697] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 12 to 22. New cstate: current_term: 22 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:40.492551 19186 leader_election.cc:290] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:40.492007 10692 catalog_manager.cc:5697] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 23 to 25, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 25 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:19:40.493818 10692 catalog_manager.cc:5697] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:19:40.496337 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 26 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:40.496587 10916 consensus_peers.cc:597] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:19:40.499420 10692 catalog_manager.cc:5697] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 17 to 21. New cstate: current_term: 21 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:40.501681 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e87e18e6c9475085e47d6a2fd71a7b" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 26 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:40.502741 10692 catalog_manager.cc:5697] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 17 to 24, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 24 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:40.514015 10692 catalog_manager.cc:5697] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 20 to 23, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 23 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:19:40.515588 10692 catalog_manager.cc:5697] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 21 to 27, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 27 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:40.517490 10767 leader_election.cc:304] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:40.517043 10692 catalog_manager.cc:5697] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 18 to 23, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 23 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:40.519737 10692 catalog_manager.cc:5697] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 16 to 24, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 24 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:19:40.521433 10692 catalog_manager.cc:5697] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 18 to 19, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 19 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:19:40.523458 19187 raft_consensus.cc:2749] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:19:40.526912 11478 consensus_peers.cc:597] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:19:40.523046 10692 catalog_manager.cc:5697] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 14 to 25, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 25 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:40.528012 10692 catalog_manager.cc:5697] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 25 to 27. New cstate: current_term: 27 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:19:40.553833 10842 consensus_peers.cc:597] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:19:40.555190 10692 catalog_manager.cc:5697] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 21 to 26, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 26 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:40.557047 10692 catalog_manager.cc:5697] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 13 to 23, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 23 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:19:40.558535 10692 catalog_manager.cc:5697] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:40.560180 10692 catalog_manager.cc:5697] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 14 to 24. New cstate: current_term: 24 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:40.561719 10692 catalog_manager.cc:5697] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 22 to 25, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 25 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:19:40.572057 10692 catalog_manager.cc:5697] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:40.580713 10692 catalog_manager.cc:5697] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 9 to 24. New cstate: current_term: 24 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:19:40.582144 10692 catalog_manager.cc:5697] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 18 to 19, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 19 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:19:40.583559 10692 catalog_manager.cc:5697] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 18 to 24, leader changed from <none> to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 24 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:40.585496 10692 catalog_manager.cc:5697] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 21 to 24, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 24 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:19:40.587034 10692 catalog_manager.cc:5697] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 17 to 18, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 18 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:19:40.588493 10692 catalog_manager.cc:5697] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 22 to 24, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 24 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:19:40.590003 10692 catalog_manager.cc:5697] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 25 to 29. New cstate: current_term: 29 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:40.609853 10692 catalog_manager.cc:5697] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 16 to 24, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 24 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:40.611716 10692 catalog_manager.cc:5697] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:19:40.623881 10692 catalog_manager.cc:5697] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 11 to 22, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 22 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:40.626507 10692 catalog_manager.cc:5697] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 15 to 20, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 20 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:40.628175 10692 catalog_manager.cc:5697] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 20 to 26, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 26 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
W20260709 13:19:40.630668 10916 consensus_peers.cc:597] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:19:40.633977 10916 consensus_peers.cc:597] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:19:40.695288 10916 consensus_peers.cc:597] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:19:40.718678 19187 raft_consensus.cc:493] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.719401 19187 raft_consensus.cc:515] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:40.721704 19187 leader_election.cc:290] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:40.728052 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:40.729770 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15293e178d84fe686fd66bc4439aba8" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
W20260709 13:19:40.733580 10842 consensus_peers.cc:597] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:19:40.734298 10916 consensus_peers.cc:597] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:19:40.741184 10766 leader_election.cc:304] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:40.742503 19186 raft_consensus.cc:2749] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:19:40.746438 10916 consensus_peers.cc:597] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:19:40.795990 10842 consensus_peers.cc:597] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:19:40.828357 10842 consensus_peers.cc:597] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:19:40.879854 19329 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.880605 19329 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:40.882853 10871 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "695ff1b5a68e4111917d909909488d39"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:34958
I20260709 13:19:40.883589 10871 raft_consensus.cc:606] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 LEADER]: Received request to transfer leadership
I20260709 13:19:40.884089 19329 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:40.887334 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 27 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:40.888243 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a3bcb5441568f4fd9b1c38d719a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 27 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:40.888399 19298 raft_consensus.cc:993] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26: : Instructing follower cb0428cc55984daba1ba63744a5102a2 to start an election
I20260709 13:19:40.890332 19292 raft_consensus.cc:1081] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 LEADER]: Signalling peer cb0428cc55984daba1ba63744a5102a2 to start an election
I20260709 13:19:40.892040 10871 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "695ff1b5a68e4111917d909909488d39"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:34958
I20260709 13:19:40.892099 10950 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "695ff1b5a68e4111917d909909488d39"
dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
 from {username='slave'} at 127.0.0.1:41020
I20260709 13:19:40.892685 10871 raft_consensus.cc:606] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 LEADER]: Received request to transfer leadership
I20260709 13:19:40.892804 10950 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:40.893306 10950 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:40.896384 10766 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
W20260709 13:19:40.897943 10916 consensus_peers.cc:597] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:19:40.900182 19187 raft_consensus.cc:2749] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:19:40.904156 10950 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:40.907032 10950 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:40.908586 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: true dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:40.908965 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: true dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:40.909174 10875 raft_consensus.cc:3055] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:19:40.909556 11509 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:40.909571 10875 raft_consensus.cc:740] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:40.915930 10875 consensus_queue.cc:260] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:40.917770 10916 leader_election.cc:336] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:40.918581 10875 raft_consensus.cc:3060] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:40.938374 19329 raft_consensus.cc:493] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.939079 19329 raft_consensus.cc:515] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:40.939417 11509 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 23.
I20260709 13:19:40.944056 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 28 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:40.949391 10875 raft_consensus.cc:2468] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 23.
I20260709 13:19:40.951117 10916 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:40.950704 10692 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42968 (request call id 251) took 1514 ms. Trace:
I20260709 13:19:40.952701 19329 leader_election.cc:290] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:40.951862 10692 rpcz_store.cc:276] 0709 13:19:39.435461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:39.435771 (+   310us) service_pool.cc:224] Handling call
0709 13:19:40.949650 (+1513879us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:40.703480 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:40.704080 (+   600us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:40.704118 (+    38us) write_op.cc:435] Acquired schema lock
0709 13:19:40.704138 (+    20us) tablet.cc:662] Decoding operations
0709 13:19:40.707607 (+  3469us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:40.707670 (+    63us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:40.707699 (+    29us) tablet.cc:685] Acquiring locks for 44 operations
0709 13:19:40.752456 (+ 44757us) tablet.cc:701] Row locks acquired
0709 13:19:40.752476 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:19:40.752623 (+   147us) write_op.cc:270] Start()
0709 13:19:40.752746 (+   123us) write_op.cc:276] Timestamp: P: 1783603180752574 usec, L: 0
0709 13:19:40.752765 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:19:40.754819 (+  2054us) log.cc:844] Serialized 17252 byte log entry
0709 13:19:40.784759 (+ 29940us) op_driver.cc:464] REPLICATION: finished
0709 13:19:40.785396 (+   637us) write_op.cc:301] APPLY: starting
0709 13:19:40.785480 (+    84us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:40.840546 (+ 55066us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:40.840648 (+   102us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:40.867287 (+ 26639us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:40.867849 (+   562us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=88,key_file_lookups=88,delta_file_lookups=88,mrs_lookups=0
0709 13:19:40.867901 (+    52us) write_op.cc:312] APPLY: finished
0709 13:19:40.868680 (+   779us) log.cc:844] Serialized 372 byte log entry
0709 13:19:40.869629 (+   949us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:40.870133 (+   504us) write_op.cc:454] Released schema lock
0709 13:19:40.870232 (+    99us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:40.939951 (+ 69719us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:40.939985 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":28892,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":236,"apply.run_cpu_time_us":51733,"apply.run_wall_time_us":103725,"cfile_cache_hit":176,"cfile_cache_hit_bytes":184008,"num_ops":44,"prepare.queue_time_us":48190,"prepare.run_cpu_time_us":9443,"prepare.run_wall_time_us":71615,"replication_time_us":31602,"thread_start_us":29981,"threads_started":1}]]}
I20260709 13:19:40.963259 19153 raft_consensus.cc:2804] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:19:40.964074 19153 raft_consensus.cc:697] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 23 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:40.965323 19153 consensus_queue.cc:237] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:40.970024 10686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 53681.1us
I20260709 13:19:40.971820 10691 catalog_manager.cc:5697] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 13 to 24, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 24 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:40.973227 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "747220f0813a4e19a0b4b96bd98d0a2a" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 28 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:40.971958 10875 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "695ff1b5a68e4111917d909909488d39"
dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:34958
I20260709 13:19:40.983594 10875 raft_consensus.cc:606] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Received request to transfer leadership
I20260709 13:19:40.984125 10875 raft_consensus.cc:614] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:40.984349 10766 leader_election.cc:304] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:40.985651 19187 raft_consensus.cc:2749] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:19:40.992365 10691 catalog_manager.cc:5697] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 22 to 25, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 25 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:40.997005 10691 catalog_manager.cc:5697] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 12 to 23. New cstate: current_term: 23 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:40.999256 10691 catalog_manager.cc:5697] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 15 to 21. New cstate: current_term: 21 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:41.003288 10691 catalog_manager.cc:5697] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 19 to 23, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 23 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:41.005650 10691 catalog_manager.cc:5697] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 28 to 29, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 29 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:41.008525 10691 catalog_manager.cc:5697] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 17 to 20, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 20 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:41.012398 10691 catalog_manager.cc:5697] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 reported cstate change: term changed from 17 to 21. New cstate: current_term: 21 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:41.059837 10692 catalog_manager.cc:5697] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 20 to 27, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 27 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:19:41.061470 10692 catalog_manager.cc:5697] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 29 to 31. New cstate: current_term: 31 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:19:41.062752 10692 catalog_manager.cc:5697] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 20 to 22, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 22 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } }
I20260709 13:19:41.064201 10692 catalog_manager.cc:5697] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 24 to 26. New cstate: current_term: 26 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:41.122697 19121 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.123307 19121 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:41.126291 19121 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:41.130399 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 26 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:41.133912 10699 catalog_manager.cc:5697] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 15 to 20, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 20 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:41.135929 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b012e79b82794524a28ea3e94dee1bb2" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 26 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:41.139833 10766 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:41.141300 19121 raft_consensus.cc:2749] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:19:41.136792 10699 catalog_manager.cc:5697] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 22 to 23, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 23 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:19:41.151146 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 62794.6us
I20260709 13:19:41.167925 10691 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42956 (request call id 264) took 8752 ms. Trace:
W20260709 13:19:41.187247 10747 auto_rebalancer.cc:893] Removing replace marker failed: Replica 468a8fd46e824f62b9c2c0d3532a6d26 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 23 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
W20260709 13:19:41.187803 10747 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 13:19:41.188098 10747 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
W20260709 13:19:41.193192  9682 thread.cc:527] Waited for 7000ms trying to join with heartbeat (tid 10826)
I20260709 13:19:41.168566 10691 rpcz_store.cc:276] 0709 13:19:32.415417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:32.444867 (+ 29450us) service_pool.cc:224] Handling call
0709 13:19:41.167828 (+8722961us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:41.019674 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:41.020198 (+   524us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:41.020228 (+    30us) write_op.cc:435] Acquired schema lock
0709 13:19:41.020241 (+    13us) tablet.cc:662] Decoding operations
0709 13:19:41.021062 (+   821us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:41.021114 (+    52us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:41.021137 (+    23us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:19:41.021836 (+   699us) tablet.cc:701] Row locks acquired
0709 13:19:41.021851 (+    15us) write_op.cc:260] PREPARE: finished
0709 13:19:41.021981 (+   130us) write_op.cc:270] Start()
0709 13:19:41.022091 (+   110us) write_op.cc:276] Timestamp: P: 1783603181021937 usec, L: 0
0709 13:19:41.022106 (+    15us) op_driver.cc:348] REPLICATION: starting
0709 13:19:41.022804 (+   698us) log.cc:844] Serialized 3224 byte log entry
0709 13:19:41.025592 (+  2788us) op_driver.cc:464] REPLICATION: finished
0709 13:19:41.026129 (+   537us) write_op.cc:301] APPLY: starting
0709 13:19:41.026220 (+    91us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:41.030713 (+  4493us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:41.030927 (+   214us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:41.036908 (+  5981us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:41.037141 (+   233us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=16,key_file_lookups=16,delta_file_lookups=16,mrs_lookups=0
0709 13:19:41.037193 (+    52us) write_op.cc:312] APPLY: finished
0709 13:19:41.037497 (+   304us) log.cc:844] Serialized 81 byte log entry
0709 13:19:41.040129 (+  2632us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:41.040280 (+   151us) write_op.cc:454] Released schema lock
0709 13:19:41.040329 (+    49us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:41.050987 (+ 10658us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:41.052148 (+  1161us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":131630,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":294,"apply.run_cpu_time_us":11592,"apply.run_wall_time_us":17902,"cfile_cache_hit":32,"cfile_cache_hit_bytes":33456,"num_ops":8,"prepare.queue_time_us":121,"prepare.run_cpu_time_us":3449,"prepare.run_wall_time_us":15346,"replication_time_us":3332}]]}
W20260709 13:19:41.221226 10699 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42506 (request call id 160) took 20228 ms (client timeout 15000 ms). Trace:
W20260709 13:19:41.221784 10699 rpcz_store.cc:269] 0709 13:19:20.992477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:20.993284 (+   807us) service_pool.cc:224] Handling call
0709 13:19:41.221116 (+20227832us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:41.157831 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:41.158400 (+   569us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:41.158436 (+    36us) write_op.cc:435] Acquired schema lock
0709 13:19:41.158455 (+    19us) tablet.cc:662] Decoding operations
0709 13:19:41.158910 (+   455us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:41.158962 (+    52us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:41.158988 (+    26us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:19:41.159348 (+   360us) tablet.cc:701] Row locks acquired
0709 13:19:41.159364 (+    16us) write_op.cc:260] PREPARE: finished
0709 13:19:41.159537 (+   173us) write_op.cc:270] Start()
0709 13:19:41.159646 (+   109us) write_op.cc:276] Timestamp: P: 1783603181159487 usec, L: 0
0709 13:19:41.159669 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:19:41.160650 (+   981us) log.cc:844] Serialized 919 byte log entry
0709 13:19:41.162816 (+  2166us) op_driver.cc:464] REPLICATION: finished
0709 13:19:41.163282 (+   466us) write_op.cc:301] APPLY: starting
0709 13:19:41.163359 (+    77us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:41.164784 (+  1425us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:41.164883 (+    99us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:41.166198 (+  1315us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:41.166379 (+   181us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=4,mrs_lookups=0
0709 13:19:41.166432 (+    53us) write_op.cc:312] APPLY: finished
0709 13:19:41.166666 (+   234us) log.cc:844] Serialized 33 byte log entry
0709 13:19:41.172309 (+  5643us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:41.172527 (+   218us) write_op.cc:454] Released schema lock
0709 13:19:41.172566 (+    39us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:41.173038 (+   472us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:41.173079 (+    41us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":376853,"reported_tablets":31,"child_traces":[["op",{"apply.queue_time_us":260,"apply.run_cpu_time_us":4185,"apply.run_wall_time_us":9659,"cfile_cache_hit":8,"cfile_cache_hit_bytes":8364,"num_ops":2,"prepare.queue_time_us":1512,"prepare.run_cpu_time_us":2847,"prepare.run_wall_time_us":3111,"replication_time_us":2994}]]}
W20260709 13:19:41.231163 10916 consensus_peers.cc:597] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:19:41.243765 10697 catalog_manager.cc:5697] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 23 to 24, leader changed from 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 24 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:41.254187 10697 catalog_manager.cc:5697] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:41.262455  9682 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:19:41.278720  9682 tablet_replica.cc:333] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:19:41.295065 19121 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:19:41.296276 19121 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:41.292186 10916 consensus_peers.cc:597] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:19:41.300038 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:41.313002  9682 raft_consensus.cc:2243] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:41.317538  9682 raft_consensus.cc:2272] T 19c20efd6c04443596bb712e3c93c650 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:41.337474  9682 tablet_replica.cc:333] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:19:41.338513  9682 raft_consensus.cc:2243] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:41.301433 19121 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:41.302687 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4c7fc28a9b461abe203fe66da89aca" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:41.355510  9682 raft_consensus.cc:2272] T 9094285c302f4de4aab986dc171c11ff P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:41.356040 10766 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:41.357774 19186 raft_consensus.cc:2749] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:41.385778  9682 tablet_replica.cc:333] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:19:41.388550  9682 raft_consensus.cc:2243] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:41.389562  9682 raft_consensus.cc:2272] T 5c8a233bc6c44bd2b5d2d44a726b147c P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:41.391367 19121 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.391999 19121 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:41.394938 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:41.395187 19121 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:41.395380 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:41.396965 10766 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:41.398177 19121 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:41.401091  9682 tablet_replica.cc:333] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:19:41.402444  9682 raft_consensus.cc:2243] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:19:41.407280  9682 raft_consensus.cc:2272] T dde624b07ed446e5b7347791196396e2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:41.412711 10937 tablet_service.cc:1558] Processing DeleteTablet for tablet 171abdebfc4142cda86de477a5c89a7e with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 8 (current committed config index is 13)) from {username='slave'} at 127.0.0.1:57812
W20260709 13:19:41.422184 10916 consensus_peers.cc:597] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:41.425184  9682 tablet_replica.cc:333] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:19:41.429628 10697 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42984 (request call id 250) took 9309 ms. Trace:
I20260709 13:19:41.427662  9682 raft_consensus.cc:2243] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 31 LEADER]: Raft consensus shutting down.
I20260709 13:19:41.430109 10697 rpcz_store.cc:276] 0709 13:19:32.127710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:32.153652 (+ 25942us) service_pool.cc:224] Handling call
0709 13:19:41.429563 (+9275911us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:41.284434 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:41.285038 (+   604us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:41.285073 (+    35us) write_op.cc:435] Acquired schema lock
0709 13:19:41.285091 (+    18us) tablet.cc:662] Decoding operations
0709 13:19:41.285506 (+   415us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:41.285563 (+    57us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:41.285591 (+    28us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:19:41.285904 (+   313us) tablet.cc:701] Row locks acquired
0709 13:19:41.285923 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:19:41.286068 (+   145us) write_op.cc:270] Start()
0709 13:19:41.286186 (+   118us) write_op.cc:276] Timestamp: P: 1783603181286021 usec, L: 0
0709 13:19:41.286206 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:19:41.286834 (+   628us) log.cc:844] Serialized 919 byte log entry
0709 13:19:41.288760 (+  1926us) op_driver.cc:464] REPLICATION: finished
0709 13:19:41.290114 (+  1354us) write_op.cc:301] APPLY: starting
0709 13:19:41.290208 (+    94us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:41.296058 (+  5850us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:41.296167 (+   109us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:41.297403 (+  1236us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:41.297568 (+   165us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=4,mrs_lookups=0
0709 13:19:41.297610 (+    42us) write_op.cc:312] APPLY: finished
0709 13:19:41.297812 (+   202us) log.cc:844] Serialized 33 byte log entry
0709 13:19:41.316310 (+ 18498us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:41.316397 (+    87us) write_op.cc:454] Released schema lock
0709 13:19:41.316446 (+    49us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:41.316777 (+   331us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:41.316804 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":314583,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":1132,"apply.run_cpu_time_us":3946,"apply.run_wall_time_us":26623,"cfile_cache_hit":8,"cfile_cache_hit_bytes":8364,"num_ops":2,"prepare.queue_time_us":164,"prepare.run_cpu_time_us":2670,"prepare.run_wall_time_us":2670,"replication_time_us":2425}]]}
I20260709 13:19:41.435976  9682 raft_consensus.cc:2272] T 2778f463396643809ca1f6bef2755afc P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:41.457602 10685 catalog_manager.cc:5028] TS cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): tablet 171abdebfc4142cda86de477a5c89a7e (table test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]) successfully deleted
I20260709 13:19:41.475625  9682 tablet_replica.cc:333] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:19:41.476889  9682 raft_consensus.cc:2243] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:19:41.478271  9682 raft_consensus.cc:2272] T e54874d9d0dc4cb6a82081a61f03a309 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:41.489035 10871 raft_consensus.cc:1275] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:19:41.491979 19195 consensus_queue.cc:1048] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:19:41.492554  9682 tablet_replica.cc:333] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
W20260709 13:19:41.498672 10842 consensus_peers.cc:597] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:19:41.510653  9682 raft_consensus.cc:2243] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:41.511545  9682 raft_consensus.cc:2272] T 418b3711416f4a3e8fa80068035b5dd8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:41.532769  9682 tablet_replica.cc:333] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:19:41.533960  9682 raft_consensus.cc:2243] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:41.534755  9682 raft_consensus.cc:2272] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:41.537984  9682 tablet_replica.cc:333] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:19:41.542284 11509 raft_consensus.cc:1275] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:19:41.544042 19153 consensus_queue.cc:1048] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:19:41.560940  9682 raft_consensus.cc:2243] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:41.561810  9682 raft_consensus.cc:2272] T 35190606a0c04bc4bac6c8ca7df14ed3 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:41.570206  9682 tablet_replica.cc:333] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:19:41.578388 10937 tablet_service.cc:1558] Processing DeleteTablet for tablet 171abdebfc4142cda86de477a5c89a7e with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 8 (current committed config index is 13)) from {username='slave'} at 127.0.0.1:57812
I20260709 13:19:41.585361  9682 raft_consensus.cc:2243] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:19:41.585393 10685 catalog_manager.cc:5028] TS cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): tablet 171abdebfc4142cda86de477a5c89a7e (table test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]) successfully deleted
I20260709 13:19:41.588642  9682 raft_consensus.cc:2272] T 2b82f18992434ed5ba58b805107060d4 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:41.607386  9682 tablet_replica.cc:333] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
W20260709 13:19:41.595283 10916 consensus_peers.cc:597] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:19:41.586472 10698 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42984 (request call id 249) took 40239 ms (client timeout 15000 ms). Trace:
W20260709 13:19:41.617688 10698 rpcz_store.cc:269] 0709 13:19:01.347095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:01.355257 (+  8162us) service_pool.cc:224] Handling call
0709 13:19:41.586287 (+40231030us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":50}
W20260709 13:19:41.619840 10842 consensus_peers.cc:597] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:19:41.621124 19121 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.622182  9682 raft_consensus.cc:2243] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:41.629257  9682 raft_consensus.cc:2272] T b012e79b82794524a28ea3e94dee1bb2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:41.621887 19121 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:41.632853 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:41.638871  9682 tablet_replica.cc:333] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:19:41.641901 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:41.647078 19121 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:41.648808 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695ff1b5a68e4111917d909909488d39" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:41.655272 10697 catalog_manager.cc:5697] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 22 to 23, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 23 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:41.662537 10766 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:41.663722  9682 raft_consensus.cc:2243] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:41.664705  9682 raft_consensus.cc:2272] T 8fd9828f212b47c19244738ee2a27a95 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:41.666277 19121 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:19:41.669241 10842 consensus_peers.cc:597] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:19:41.680620  9682 tablet_replica.cc:333] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:19:41.687791  9682 raft_consensus.cc:2243] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:41.690330  9682 pending_rounds.cc:70] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87: Trying to abort 1 pending ops.
I20260709 13:19:41.693204  9682 pending_rounds.cc:77] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting op as it isn't in flight: id { term: 21 index: 14 } timestamp: 7305638563712983040 op_type: NO_OP noop_request { }
I20260709 13:19:41.693830  9682 raft_consensus.cc:2889] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:19:41.694284  9682 raft_consensus.cc:2272] T 416a6d12184f4fbda69570b95be47851 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:41.707120 10693 catalog_manager.cc:5697] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 16 to 21, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 21 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:41.709889 10693 catalog_manager.cc:5697] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 23 to 25. New cstate: current_term: 25 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:41.757998  9682 tablet_replica.cc:333] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:19:41.759323  9682 raft_consensus.cc:2243] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:41.760457  9682 raft_consensus.cc:2272] T 4dd4c0ff1def4960931dda3b52a4a884 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:41.774462  9682 tablet_replica.cc:333] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:19:41.827507  9682 raft_consensus.cc:2243] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:41.837375  9682 raft_consensus.cc:2272] T 8cab289810294719ac9686e40290a6ab P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:41.844441  9682 tablet_replica.cc:333] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
W20260709 13:19:41.855161 10694 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42956 (request call id 263) took 33110 ms (client timeout 15000 ms). Trace:
W20260709 13:19:41.866724 10694 rpcz_store.cc:269] 0709 13:19:08.745270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:08.745544 (+   274us) service_pool.cc:224] Handling call
0709 13:19:08.972960 (+227416us) spinlock_profiling.cc:243] Waited 2.26 ms on lock 0x7b54002b0110. stack: 00007f9c7dd96343 00007f9c7dd96240 00007f9c7d7251d4 00007f9c87463304 00007f9c87463299 00007f9c7dc50db1 00007f9c7dd90b71 00007f9c7dd910b2 00007f9c86d0aa89 00007f9c86ca2da6 00007f9c86c74088 00007f9c86d7237e 00007f9c852332a4 00007f9c85233232 00007f9c85233161 00007f9c852330dc
0709 13:19:41.854936 (+32881976us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":49,"spinlock_wait_cycles":5206784}
I20260709 13:19:41.867547  9682 raft_consensus.cc:2243] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:19:41.868948  9682 raft_consensus.cc:2272] T 225fa08b9f864105af0cd7a0e7a0d99e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:41.875171 10693 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42506 (request call id 161) took 5164 ms. Trace:
I20260709 13:19:41.878224 10699 catalog_manager.cc:5697] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 24 to 27. New cstate: current_term: 27 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:41.880296  9682 tablet_replica.cc:333] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:19:41.881263  9682 raft_consensus.cc:2243] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:41.882061  9682 raft_consensus.cc:2272] T 9354ad0d744d43bbbe2ce992ecbc4e65 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:41.882601 10699 catalog_manager.cc:5697] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 25 to 27. New cstate: current_term: 27 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:41.893046 10699 catalog_manager.cc:5697] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 21 to 23, leader changed from cb0428cc55984daba1ba63744a5102a2 (127.9.116.131) to e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129). New cstate: current_term: 23 leader_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:41.885563  9682 tablet_replica.cc:333] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
W20260709 13:19:41.895664 10842 consensus_peers.cc:597] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:19:41.896423  9682 raft_consensus.cc:2243] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:41.897388  9682 raft_consensus.cc:2272] T 1d9e744bb20c4325969f4fbc01c672e8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:41.879449 10693 rpcz_store.cc:276] 0709 13:19:36.711234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:36.711465 (+   231us) service_pool.cc:224] Handling call
0709 13:19:41.805025 (+5093560us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:41.805057 (+    32us) write_op.cc:454] Released schema lock
0709 13:19:41.875068 (+ 70011us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:41.785534 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:41.786172 (+   638us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:41.786213 (+    41us) write_op.cc:435] Acquired schema lock
0709 13:19:41.786233 (+    20us) tablet.cc:662] Decoding operations
0709 13:19:41.786697 (+   464us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:41.786767 (+    70us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:41.786828 (+    61us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:19:41.787200 (+   372us) tablet.cc:701] Row locks acquired
0709 13:19:41.787215 (+    15us) write_op.cc:260] PREPARE: finished
0709 13:19:41.787373 (+   158us) write_op.cc:270] Start()
0709 13:19:41.787494 (+   121us) write_op.cc:276] Timestamp: P: 1783603181787323 usec, L: 0
0709 13:19:41.787514 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:19:41.788187 (+   673us) log.cc:844] Serialized 915 byte log entry
0709 13:19:41.790267 (+  2080us) op_driver.cc:464] REPLICATION: finished
0709 13:19:41.790680 (+   413us) write_op.cc:301] APPLY: starting
0709 13:19:41.790766 (+    86us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:41.792271 (+  1505us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:41.792377 (+   106us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:41.793683 (+  1306us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:41.793873 (+   190us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=4,mrs_lookups=0
0709 13:19:41.793930 (+    57us) write_op.cc:312] APPLY: finished
0709 13:19:41.794152 (+   222us) log.cc:844] Serialized 33 byte log entry
0709 13:19:41.794336 (+   184us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:41.794429 (+    93us) write_op.cc:454] Released schema lock
0709 13:19:41.794473 (+    44us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":62354,"reported_tablets":31,"child_traces":[["op",{"apply.queue_time_us":230,"apply.run_cpu_time_us":4065,"apply.run_wall_time_us":4064,"cfile_cache_hit":8,"cfile_cache_hit_bytes":8364,"num_ops":2,"prepare.queue_time_us":129,"prepare.run_cpu_time_us":2947,"prepare.run_wall_time_us":2948,"replication_time_us":2624}]]}
W20260709 13:19:41.902211 10915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 51025us
I20260709 13:19:41.929111  9682 tablet_replica.cc:333] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:19:41.930280  9682 raft_consensus.cc:2243] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 LEADER]: Raft consensus shutting down.
W20260709 13:19:41.935889 10916 consensus_peers.cc:597] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:19:41.945310 10842 consensus_peers.cc:597] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:19:41.946125  9682 raft_consensus.cc:2272] T b4b13050d714473bb01ea0174d634a14 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:41.957867  9682 tablet_replica.cc:333] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:19:41.959925  9682 raft_consensus.cc:2243] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:41.960929  9682 raft_consensus.cc:2272] T 0a574a3bcb5441568f4fd9b1c38d719a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:41.963974  9682 tablet_replica.cc:333] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:19:41.979569  9682 raft_consensus.cc:2243] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:19:41.980931  9682 raft_consensus.cc:2272] T e41482a95776477784d9bd9dacfed231 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:41.984099  9682 tablet_replica.cc:333] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:19:41.985049  9682 raft_consensus.cc:2243] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:19:41.986171  9682 raft_consensus.cc:2272] T bc6987a00cef4dff9545eee242db0982 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:41.989051  9682 tablet_replica.cc:333] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:19:41.996963 19186 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.998618 19186 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:42.000777 19186 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:42.001976  9682 raft_consensus.cc:2243] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:42.002765  9682 raft_consensus.cc:2272] T b761e5168d4e48aeaec47b99f5c7d3d0 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:42.002612 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:42.003114 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba4060a26c42998e9f220a79f28546" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:42.006675  9682 tablet_replica.cc:333] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:19:42.007030 10766 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:42.008172  9682 raft_consensus.cc:2243] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:42.008862 19186 raft_consensus.cc:2749] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:19:42.008988  9682 pending_rounds.cc:70] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87: Trying to abort 1 pending ops.
I20260709 13:19:42.009406  9682 pending_rounds.cc:77] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting op as it isn't in flight: id { term: 22 index: 14 } timestamp: 7305638559683346432 op_type: NO_OP noop_request { }
I20260709 13:19:42.009820  9682 raft_consensus.cc:2889] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:19:42.010215  9682 raft_consensus.cc:2272] T f6e90b642b0d4f06b866170dc0293dc1 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:19:42.010499 10842 consensus_peers.cc:597] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:19:42.013320  9682 tablet_replica.cc:333] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:19:42.016695  9682 raft_consensus.cc:2243] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:19:42.017885  9682 raft_consensus.cc:2272] T e9764b93d3a8407b8cc1ae641620c831 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:19:42.018983 10842 consensus_peers.cc:597] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:19:42.022007  9682 tablet_replica.cc:333] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:19:42.023150  9682 raft_consensus.cc:2243] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 28 LEADER]: Raft consensus shutting down.
W20260709 13:19:42.023586 10842 consensus_peers.cc:597] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:19:42.024338  9682 raft_consensus.cc:2272] T 99ca4d7ec0974268ac5f35c5055da891 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:42.028543  9682 tablet_replica.cc:333] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:19:42.029475  9682 raft_consensus.cc:2243] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:19:42.030838  9682 raft_consensus.cc:2272] T a50b3bb87a48494a8e835471e55a6f74 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:42.034211  9682 tablet_replica.cc:333] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:19:42.035255  9682 raft_consensus.cc:2243] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:42.036036  9682 raft_consensus.cc:2272] T 86ba4060a26c42998e9f220a79f28546 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:42.039222  9682 tablet_replica.cc:333] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:19:42.040266  9682 raft_consensus.cc:2243] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:42.040932  9682 raft_consensus.cc:2272] T 741efe79e97646b396f0091dc10c745e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:19:42.042722 10842 consensus_peers.cc:597] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:19:42.049618  9682 tablet_replica.cc:333] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:19:42.050832  9682 raft_consensus.cc:2243] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:42.051575  9682 pending_rounds.cc:70] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87: Trying to abort 1 pending ops.
I20260709 13:19:42.051882  9682 pending_rounds.cc:77] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting op as it isn't in flight: id { term: 22 index: 13 } timestamp: 7305638514790400000 op_type: NO_OP noop_request { }
I20260709 13:19:42.052310  9682 raft_consensus.cc:2889] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:19:42.052722  9682 raft_consensus.cc:2272] T 32e87e18e6c9475085e47d6a2fd71a7b P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:42.055846  9682 tablet_replica.cc:333] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:19:42.056730  9682 raft_consensus.cc:2243] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:42.057359  9682 raft_consensus.cc:2272] T 747220f0813a4e19a0b4b96bd98d0a2a P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:42.061610  9682 tablet_replica.cc:333] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:19:42.062680  9682 raft_consensus.cc:2243] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:42.063359  9682 pending_rounds.cc:70] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87: Trying to abort 1 pending ops.
I20260709 13:19:42.063679  9682 pending_rounds.cc:77] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting op as it isn't in flight: id { term: 20 index: 12 } timestamp: 7305638554992623616 op_type: NO_OP noop_request { }
I20260709 13:19:42.064093  9682 raft_consensus.cc:2889] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:19:42.064514  9682 raft_consensus.cc:2272] T d32f5e230e464e61b23a41962708d120 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:42.067456  9682 tablet_replica.cc:333] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:19:42.068445  9682 raft_consensus.cc:2243] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:42.069188  9682 pending_rounds.cc:70] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87: Trying to abort 1 pending ops.
I20260709 13:19:42.069548  9682 pending_rounds.cc:77] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting op as it isn't in flight: id { term: 19 index: 14 } timestamp: 7305638480531881984 op_type: NO_OP noop_request { }
I20260709 13:19:42.070007  9682 raft_consensus.cc:2889] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:19:42.070436  9682 raft_consensus.cc:2272] T ce92c99123994b928d7a775917eb22ed P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:42.073611  9682 tablet_replica.cc:333] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:19:42.074573  9682 raft_consensus.cc:2243] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:42.075708  9682 raft_consensus.cc:2272] T 171abdebfc4142cda86de477a5c89a7e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:42.078722  9682 tablet_replica.cc:333] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:19:42.079656  9682 raft_consensus.cc:2243] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:19:42.080760  9682 raft_consensus.cc:2272] T ddb097c9ce1748458d6719bc91f56e1d P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:42.094955  9682 tablet_replica.cc:333] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:19:42.095927  9682 raft_consensus.cc:2243] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:42.096669  9682 raft_consensus.cc:2272] T 12d33f7a0f8f47e5ad57e854e52ae154 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:42.099828  9682 tablet_replica.cc:333] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:19:42.100775  9682 raft_consensus.cc:2243] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:42.101684  9682 raft_consensus.cc:2272] T 651c4c65e7f44b1fb4053d3757ac417f P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:42.104806  9682 tablet_replica.cc:333] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:19:42.105665  9682 raft_consensus.cc:2243] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:42.113745  9682 raft_consensus.cc:2272] T 417caa22425d4a25aca2f7d8e9577256 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:42.116899  9682 tablet_replica.cc:333] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:19:42.123956  9682 raft_consensus.cc:2243] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:42.128880  9682 pending_rounds.cc:70] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87: Trying to abort 1 pending ops.
I20260709 13:19:42.129215  9682 pending_rounds.cc:77] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting op as it isn't in flight: id { term: 20 index: 11 } timestamp: 7305638561653084160 op_type: NO_OP noop_request { }
I20260709 13:19:42.129663  9682 raft_consensus.cc:2889] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:19:42.130074  9682 raft_consensus.cc:2272] T 695ff1b5a68e4111917d909909488d39 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:42.134222  9682 tablet_replica.cc:333] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:19:42.135133  9682 raft_consensus.cc:2243] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:42.135845  9682 raft_consensus.cc:2272] T 9b4c7fc28a9b461abe203fe66da89aca P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:42.138868  9682 tablet_replica.cc:333] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
W20260709 13:19:42.144204 11478 consensus_peers.cc:597] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:19:42.151412  9682 raft_consensus.cc:2243] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:19:42.152729  9682 raft_consensus.cc:2272] T 90ad199246b34499aa32795a8082ffa8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:42.153163 19186 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:42.153855 19186 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:42.155789  9682 tablet_replica.cc:333] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:19:42.156246 19186 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:42.157342 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:42.156745  9682 raft_consensus.cc:2243] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:19:42.158985 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121ef9771004451eb563dbfd77657e0e" candidate_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:42.159173  9682 raft_consensus.cc:2272] T 90ea9e6b562e4600be1cb566739a6a30 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:42.160768 10767 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e01ecd008edd4aa2b6b2b47efbc2ea87; no voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26
I20260709 13:19:42.162047 19186 raft_consensus.cc:2749] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:42.162334  9682 tablet_replica.cc:333] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:19:42.163439  9682 raft_consensus.cc:2243] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:42.164377  9682 pending_rounds.cc:70] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87: Trying to abort 1 pending ops.
I20260709 13:19:42.164651  9682 pending_rounds.cc:77] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87: Aborting op as it isn't in flight: id { term: 23 index: 18 } timestamp: 7305638562746425344 op_type: NO_OP noop_request { }
I20260709 13:19:42.165073  9682 raft_consensus.cc:2889] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:19:42.165463  9682 raft_consensus.cc:2272] T 121ef9771004451eb563dbfd77657e0e P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:42.168490  9682 tablet_replica.cc:333] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:19:42.169269  9682 raft_consensus.cc:2243] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:42.187132  9682 raft_consensus.cc:2272] T f15293e178d84fe686fd66bc4439aba8 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:42.190383  9682 tablet_replica.cc:333] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:19:42.191203  9682 raft_consensus.cc:2243] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:19:42.192325  9682 raft_consensus.cc:2272] T 96558ab13797493e9c168b86fe3b62cd P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:42.195230  9682 tablet_replica.cc:333] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:19:42.195920  9682 raft_consensus.cc:2243] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:19:42.196952  9682 raft_consensus.cc:2272] T 8cf3aebfc54a4652a83388ef9a48b1c2 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:42.212713  9682 tablet_replica.cc:333] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:19:42.213495  9682 raft_consensus.cc:2243] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:19:42.214607  9682 raft_consensus.cc:2272] T fc429d4d563f415d90d6f74da422a740 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:42.217214  9682 tablet_replica.cc:333] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:19:42.217869  9682 raft_consensus.cc:2243] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:42.218456  9682 raft_consensus.cc:2272] T 8949b8b2d08942939f64373bccb71936 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:42.220613  9682 tablet_replica.cc:333] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87: stopping tablet replica
I20260709 13:19:42.221145  9682 raft_consensus.cc:2243] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:19:42.222097  9682 raft_consensus.cc:2272] T d00373866c144f4981da73bd7ae39019 P e01ecd008edd4aa2b6b2b47efbc2ea87 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:19:42.425149 10842 consensus_peers.cc:597] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:19:42.499367 11478 consensus_peers.cc:597] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:19:42.513118 10747 auto_rebalancer.cc:629] Failed to schedule move for tablet 90ea9e6b562e4600be1cb566739a6a30: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:42.538893 19328 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:42.539567 19328 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:42.541708 19328 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:42.546018 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:42.550004 11478 leader_election.cc:336] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:42.550593 11478 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:42.551780 19328 raft_consensus.cc:2749] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:19:42.569705 10950 consensus_queue.cc:237] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
W20260709 13:19:42.601318 10842 consensus_peers.cc:597] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:19:42.619462 10915 consensus_peers.cc:597] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 -> Peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Couldn't send request to peer 12682eb70bc84c76879e04a8628ee586. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8cab289810294719ac9686e40290a6ab. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:42.625890 10876 raft_consensus.cc:1275] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:19:42.627633 19195 consensus_queue.cc:1048] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:19:42.639652 10916 consensus_peers.cc:597] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:42.671718 10876 raft_consensus.cc:2955] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Committing config change with OpId 21.12: config changed from index -1 to 12, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:19:42.670754 19153 raft_consensus.cc:2955] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 21 LEADER]: Committing config change with OpId 21.12: config changed from index -1 to 12, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:19:42.705591 10699 catalog_manager.cc:5697] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: config changed from index -1 to 12, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New cstate: current_term: 21 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:19:42.713940 10876 consensus_queue.cc:237] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:19:42.726689 19345 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:42.727653 19345 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:42.731362 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 32 candidate_status { last_received { term: 31 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:42.740495 10842 leader_election.cc:336] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:42.741106 10842 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:42.741907 19360 raft_consensus.cc:2749] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 13:19:42.742043 19345 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:42.750217 10950 raft_consensus.cc:1275] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:19:42.752772 19353 consensus_queue.cc:1048] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:19:42.754252 10841 consensus_peers.cc:597] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Couldn't send request to peer 12682eb70bc84c76879e04a8628ee586. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b761e5168d4e48aeaec47b99f5c7d3d0. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:42.771660 10842 consensus_peers.cc:597] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:42.778002 19353 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:42.778702 19353 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:42.782159 19353 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:42.783828 19345 raft_consensus.cc:2955] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 LEADER]: Committing config change with OpId 21.15: config changed from index -1 to 15, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
W20260709 13:19:42.785387 10842 leader_election.cc:336] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:42.787042 10948 raft_consensus.cc:2955] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Committing config change with OpId 21.15: config changed from index -1 to 15, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:19:42.791178 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:42.791893 11506 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 26.
I20260709 13:19:42.793054 10841 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:42.794137 19353 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:19:42.794510 19353 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:42.794938 19360 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:42.795522 19360 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:42.797679 19353 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:19:42.835850 10842 leader_election.cc:336] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:42.839588 19360 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:42.843667 11478 consensus_peers.cc:597] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:19:42.842234 19360 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:42.849884 19360 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:42.853986 19353 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:42.859969 10842 leader_election.cc:336] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:42.848398 10697 catalog_manager.cc:5697] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: config changed from index -1 to 15, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New cstate: current_term: 21 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:19:42.863514 19360 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:42.868115 19362 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:42.868732 19362 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:42.870882 19362 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:42.873651 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 32 candidate_status { last_received { term: 31 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:42.874313 10876 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 31.
W20260709 13:19:42.875979 10916 consensus_peers.cc:597] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:19:42.878160 10916 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:42.879823 19362 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 13:19:42.880257 19362 raft_consensus.cc:493] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 31 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:19:42.880651 10916 leader_election.cc:336] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:42.880661 19362 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 31 FOLLOWER]: Advancing to term 32
I20260709 13:19:42.882290 19353 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 27 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:42.883545 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 64338us
I20260709 13:19:42.885133 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:42.886441 10842 consensus_peers.cc:597] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:19:42.888343 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:42.888675 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde624b07ed446e5b7347791196396e2" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:42.889189 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:42.889222 11509 raft_consensus.cc:3060] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:19:42.899833 11509 raft_consensus.cc:2468] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 27.
W20260709 13:19:42.907521 10842 leader_election.cc:336] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:42.912667 19362 raft_consensus.cc:515] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:42.889830 10841 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:42.922176 19371 raft_consensus.cc:2749] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:42.922462 10841 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:42.924530 19371 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:42.928867 19362 leader_election.cc:290] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 32 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:42.930253 19346 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:42.930917 19346 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:42.932333 19362 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:42.932875 19362 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:42.934841 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:42.935029 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2778f463396643809ca1f6bef2755afc" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 32 candidate_status { last_received { term: 31 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:42.935668 10876 raft_consensus.cc:3060] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 31 FOLLOWER]: Advancing to term 32
I20260709 13:19:42.937049 19362 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:42.938616 19346 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:42.938882 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:42.939615 10875 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 24.
W20260709 13:19:42.943112 11478 leader_election.cc:336] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:42.943557 11478 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:42.945489 10876 raft_consensus.cc:2468] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 32.
W20260709 13:19:42.946237 10916 leader_election.cc:336] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 32 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:42.947767 19346 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:42.949859 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:42.950538 10875 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 24.
I20260709 13:19:42.954962 10916 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:42.960073 19344 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:19:42.960541 19344 raft_consensus.cc:493] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:42.960945 19344 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:19:42.962203 10916 leader_election.cc:336] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:42.975128 19344 raft_consensus.cc:515] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:42.967131 19362 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:42.976302 19362 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:42.977437 19344 leader_election.cc:290] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:42.980823 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:42.981972 11509 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 24.
I20260709 13:19:42.975733 10916 leader_election.cc:304] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:42.971550 10875 consensus_queue.cc:237] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:19:42.987506 19362 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:42.995090 19362 raft_consensus.cc:2804] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 13:19:42.995780 19362 raft_consensus.cc:697] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 32 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:43.001441 19362 consensus_queue.cc:237] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 31.16, Last appended by leader: 16, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:42.998551 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54874d9d0dc4cb6a82081a61f03a309" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:43.013597 10876 raft_consensus.cc:3060] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:43.003906 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:43.014273 10871 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 24.
I20260709 13:19:42.974299 10841 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:43.015816 19353 raft_consensus.cc:2804] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:19:43.016343 19353 raft_consensus.cc:697] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:43.017227 19353 consensus_queue.cc:237] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:43.018731 10842 consensus_peers.cc:597] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:43.021311 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 47094.6us
I20260709 13:19:43.039134 10948 raft_consensus.cc:1275] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:19:43.047278 10697 catalog_manager.cc:5697] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 26 to 27, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 27 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:43.051643 19370 consensus_queue.cc:1048] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:19:43.061321 10916 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:43.076208 19195 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:19:43.076651 19195 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:43.077025 19195 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:19:43.078495 10916 leader_election.cc:336] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:19:43.083094 10916 leader_election.cc:336] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:43.084986 19195 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:43.075385 10841 consensus_peers.cc:597] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555): Couldn't send request to peer 12682eb70bc84c76879e04a8628ee586. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d32f5e230e464e61b23a41962708d120. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:43.087080 19370 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:43.087628 19370 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:43.096764 19370 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:43.099165 19195 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:43.099305 10842 leader_election.cc:336] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:43.100158 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:43.100812 11509 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:43.101765 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:43.105719 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b82f18992434ed5ba58b805107060d4" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:43.106266 10871 raft_consensus.cc:3060] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:43.107786 10697 catalog_manager.cc:5697] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 31 to 32, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 32 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:43.108538 11509 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 25.
I20260709 13:19:43.113265 10871 raft_consensus.cc:2468] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 25.
I20260709 13:19:43.121155 10841 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:43.085147 10876 raft_consensus.cc:2468] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 25.
I20260709 13:19:43.122613 19353 raft_consensus.cc:2749] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:19:43.125367 19370 raft_consensus.cc:2955] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 LEADER]: Committing config change with OpId 23.15: config changed from index -1 to 15, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:19:43.128327 10948 raft_consensus.cc:2955] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Committing config change with OpId 23.15: config changed from index -1 to 15, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } } }
I20260709 13:19:43.137074 10916 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:43.137912 19353 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:43.138535 19353 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:43.145838 10842 leader_election.cc:336] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:43.146625 19353 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:43.148305 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:43.150257 10841 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:43.151228 19362 raft_consensus.cc:2804] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:19:43.151779 19362 raft_consensus.cc:697] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 25 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:43.152787 19362 consensus_queue.cc:237] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 25, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:43.158018 10916 leader_election.cc:304] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:43.158936 19353 raft_consensus.cc:2749] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:43.160756 19344 raft_consensus.cc:2804] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:19:43.161319 19344 raft_consensus.cc:697] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 25 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:43.162163 19344 consensus_queue.cc:237] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:43.181165 10697 catalog_manager.cc:5697] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: config changed from index -1 to 15, NON_VOTER 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) added. New cstate: current_term: 23 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:19:43.184890 11478 consensus_peers.cc:597] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:19:43.187526 10916 leader_election.cc:336] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:43.202414 19362 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:43.203042 19362 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:43.205029 19362 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:43.211939 19346 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:43.212589 19346 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:43.215464 19153 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:43.216012 19153 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:43.217833 19153 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:43.220891 11478 leader_election.cc:336] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:43.221668 19346 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:43.223201 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:43.227995 11477 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:43.228873 19328 raft_consensus.cc:2749] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:43.230474 19195 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:43.231047 19195 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:43.233253 19195 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:43.226408 10916 consensus_peers.cc:597] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:19:43.235594 10916 consensus_peers.cc:597] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:19:43.236971 19195 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:43.237534 19195 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:43.239439 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:43.240165 10876 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 28.
I20260709 13:19:43.239755 19195 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:43.244393 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:43.246095 19376 ts_tablet_manager.cc:933] T 8cab289810294719ac9686e40290a6ab P 12682eb70bc84c76879e04a8628ee586: Initiating tablet copy from peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:43.257658 19353 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:43.258289 19353 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:43.260722 19353 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:43.262413 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:43.264355 10842 consensus_peers.cc:597] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:19:43.276268 10842 leader_election.cc:336] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:43.280586 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:43.282171 10842 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:43.283676 19370 raft_consensus.cc:2749] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:19:43.284880 10842 consensus_peers.cc:597] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:19:43.291491 19376 tablet_copy_client.cc:323] T 8cab289810294719ac9686e40290a6ab P 12682eb70bc84c76879e04a8628ee586: tablet copy: Beginning tablet copy session from remote peer at address 127.9.116.131:45553
I20260709 13:19:43.292665 19380 ts_tablet_manager.cc:933] T b761e5168d4e48aeaec47b99f5c7d3d0 P 12682eb70bc84c76879e04a8628ee586: Initiating tablet copy from peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:43.294864 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:43.293749 10959 tablet_copy_service.cc:140] P cb0428cc55984daba1ba63744a5102a2: Received BeginTabletCopySession request for tablet 8cab289810294719ac9686e40290a6ab from peer 12682eb70bc84c76879e04a8628ee586 ({username='slave'} at 127.0.0.1:36714)
I20260709 13:19:43.296329 10959 tablet_copy_service.cc:161] P cb0428cc55984daba1ba63744a5102a2: Beginning new tablet copy session on tablet 8cab289810294719ac9686e40290a6ab from peer 12682eb70bc84c76879e04a8628ee586 at {username='slave'} at 127.0.0.1:36714: session id = 12682eb70bc84c76879e04a8628ee586-8cab289810294719ac9686e40290a6ab
I20260709 13:19:43.308184 10959 tablet_copy_source_session.cc:215] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:19:43.308866 19344 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:43.309460 19344 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:43.312095 19344 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:43.312433 19376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cab289810294719ac9686e40290a6ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:19:43.313089 19380 tablet_copy_client.cc:323] T b761e5168d4e48aeaec47b99f5c7d3d0 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Beginning tablet copy session from remote peer at address 127.9.116.130:37895
I20260709 13:19:43.315141 10886 tablet_copy_service.cc:140] P 468a8fd46e824f62b9c2c0d3532a6d26: Received BeginTabletCopySession request for tablet b761e5168d4e48aeaec47b99f5c7d3d0 from peer 12682eb70bc84c76879e04a8628ee586 ({username='slave'} at 127.0.0.1:49290)
I20260709 13:19:43.315727 10886 tablet_copy_service.cc:161] P 468a8fd46e824f62b9c2c0d3532a6d26: Beginning new tablet copy session on tablet b761e5168d4e48aeaec47b99f5c7d3d0 from peer 12682eb70bc84c76879e04a8628ee586 at {username='slave'} at 127.0.0.1:49290: session id = 12682eb70bc84c76879e04a8628ee586-b761e5168d4e48aeaec47b99f5c7d3d0
I20260709 13:19:43.309824 10916 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:43.321987 19344 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 13:19:43.322518 19344 raft_consensus.cc:493] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:43.322899 19344 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:19:43.324885 10886 tablet_copy_source_session.cc:215] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:19:43.328593 19380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b761e5168d4e48aeaec47b99f5c7d3d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:19:43.330621 19344 raft_consensus.cc:515] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:43.331672 19353 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:43.332207 19353 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:43.332846 19344 leader_election.cc:290] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 29 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:43.334304 19353 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:43.335395 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:43.336009  9682 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 10822)
I20260709 13:19:43.336022 10948 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 27.
I20260709 13:19:43.337442 19353 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:43.337977 19353 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:43.339100 10841 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:43.339998 19370 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:19:43.340417 19370 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:43.340480 19353 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:43.340883 19370 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:19:43.342069 19353 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:43.342958 19371 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:43.342660 19353 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:43.343466 19371 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:43.345304 19353 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:43.349601 19353 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:43.350178 19353 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:43.352470 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:43.353396 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:43.354055 10950 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 23.
I20260709 13:19:43.355633 19353 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:43.356642 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:43.358824 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:43.345448 19371 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:43.359465 10841 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:43.377095 10948 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
I20260709 13:19:43.379103 10841 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:43.378039 10950 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 21.
W20260709 13:19:43.377499 10916 leader_election.cc:336] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:43.336618 10697 catalog_manager.cc:5697] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 24 to 25, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 25 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:19:43.381850 10916 leader_election.cc:336] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:43.382269 10916 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:43.383915 10841 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:43.384881 19371 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:43.395975 19371 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:43.396611 19371 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:43.397462 19376 tablet_copy_client.cc:806] T 8cab289810294719ac9686e40290a6ab P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 0 data blocks...
I20260709 13:19:43.398142 19376 tablet_copy_client.cc:670] T 8cab289810294719ac9686e40290a6ab P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 1 WAL segments...
I20260709 13:19:43.382519 10697 catalog_manager.cc:5697] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 24 to 25, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 25 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:43.386610 19345 raft_consensus.cc:2804] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:43.399129 19345 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:43.399550 19345 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:43.402670 19376 tablet_copy_client.cc:538] T 8cab289810294719ac9686e40290a6ab P 12682eb70bc84c76879e04a8628ee586: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:19:43.387653 19370 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:43.385365 19353 raft_consensus.cc:2804] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:43.406373 19353 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:43.406769 19353 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:43.409477 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb097c9ce1748458d6719bc91f56e1d" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:43.410099 10950 raft_consensus.cc:3060] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:19:43.405171 19370 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 28 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:43.429298 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 47465.9us
W20260709 13:19:43.386099 10842 leader_election.cc:336] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:43.431489 10950 raft_consensus.cc:2468] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 28.
W20260709 13:19:43.432165 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 46925.2us
W20260709 13:19:43.433833 10842 leader_election.cc:336] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:19:43.435463 10842 leader_election.cc:336] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:43.437062 10842 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:43.430153 10916 leader_election.cc:336] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:43.438335 10916 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:43.438651 19384 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
W20260709 13:19:43.439392 10842 leader_election.cc:336] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:43.439253 19384 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:43.436165 19195 raft_consensus.cc:2749] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:19:43.441883 19362 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:43.443562 19370 raft_consensus.cc:2749] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:43.446266 19345 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:43.446900 19344 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:43.449885 19153 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:43.451187 19353 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:43.453181 19371 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:43.474871 10841 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:19:43.471123 10842 leader_election.cc:336] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:43.472879 19153 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:43.472365 19346 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:43.476749 19344 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:43.469379 19380 tablet_copy_client.cc:806] T b761e5168d4e48aeaec47b99f5c7d3d0 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 0 data blocks...
W20260709 13:19:43.484578 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 54441.9us
I20260709 13:19:43.472422 19195 raft_consensus.cc:2749] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:43.498356 19153 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:43.499696 19380 tablet_copy_client.cc:670] T b761e5168d4e48aeaec47b99f5c7d3d0 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 1 WAL segments...
I20260709 13:19:43.500497 19384 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:43.498260 19382 raft_consensus.cc:2804] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 13:19:43.516100 19353 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:43.517439 19345 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:43.520509 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:43.475229 19362 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:43.530159 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225fa08b9f864105af0cd7a0e7a0d99e" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:43.510475 19385 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:43.530838 10950 raft_consensus.cc:3060] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:19:43.530625 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 47356us
I20260709 13:19:43.492684 19346 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:43.522475 19382 raft_consensus.cc:697] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 28 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
W20260709 13:19:43.544867 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 47622.9us
I20260709 13:19:43.545809 19382 consensus_queue.cc:237] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:43.538820 19346 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:43.535116 19195 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:43.547214 19195 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:43.549269 10948 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 25.
I20260709 13:19:43.529330 19371 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:43.542934 19385 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:43.550302 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6987a00cef4dff9545eee242db0982" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:43.541908 10950 raft_consensus.cc:2468] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 22.
I20260709 13:19:43.555191 10945 raft_consensus.cc:3060] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:43.561627 19344 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:43.561681 19362 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:43.568128 19384 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:43.568531 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
W20260709 13:19:43.556900 10842 leader_election.cc:336] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:43.569183 10871 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 22.
I20260709 13:19:43.572570 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:43.573275 11506 raft_consensus.cc:1275] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 13:19:43.573303 10950 raft_consensus.cc:3060] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:43.577556 19387 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:43.577709 19195 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:43.578161 19387 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:43.583293 10945 raft_consensus.cc:2468] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 20.
I20260709 13:19:43.584858 19390 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:43.585410 19390 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:43.587741 19390 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:43.589737 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:43.590094 19388 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:43.590600 19388 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:43.592707 19388 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:43.594342 10842 leader_election.cc:336] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:43.594697 19380 tablet_copy_client.cc:538] T b761e5168d4e48aeaec47b99f5c7d3d0 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:19:43.594944 10842 leader_election.cc:336] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:19:43.595458 10842 leader_election.cc:336] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:43.595551 19387 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:43.595939 10842 leader_election.cc:336] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:19:43.596284 11478 leader_election.cc:336] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:43.568708 19384 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:43.597890 10948 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 23.
I20260709 13:19:43.607407 19393 consensus_queue.cc:1048] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.010s
I20260709 13:19:43.608397 10697 catalog_manager.cc:5697] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 27 to 28, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 28 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
W20260709 13:19:43.610929 11478 leader_election.cc:336] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:43.618148 19376 tablet_bootstrap.cc:492] T 8cab289810294719ac9686e40290a6ab P 12682eb70bc84c76879e04a8628ee586: Bootstrap starting.
I20260709 13:19:43.626489 11477 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:43.631784 10950 raft_consensus.cc:2468] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 24.
W20260709 13:19:43.633682 10842 leader_election.cc:336] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:43.627763 19390 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:19:43.644377 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 65067.3us
I20260709 13:19:43.645366 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:43.643790 19390 raft_consensus.cc:493] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:43.646428 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00373866c144f4981da73bd7ae39019" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:43.647303 10871 raft_consensus.cc:2393] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 24: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:43.646598 19390 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:43.647361 19380 tablet_bootstrap.cc:492] T b761e5168d4e48aeaec47b99f5c7d3d0 P 12682eb70bc84c76879e04a8628ee586: Bootstrap starting.
W20260709 13:19:43.651186 10842 leader_election.cc:336] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:43.653425 19384 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:43.654856 11509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:43.656747 11509 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 23.
I20260709 13:19:43.645392 10841 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:43.658430 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99ca4d7ec0974268ac5f35c5055da891" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:43.659054 10871 raft_consensus.cc:3060] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:19:43.661348 10841 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:43.661751 19384 raft_consensus.cc:2804] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:43.662937 19385 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:43.662273 19345 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:19:43.663424 10841 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:43.664100 19345 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:43.664520 19345 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:19:43.665068 10841 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:43.665696 19385 raft_consensus.cc:2804] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:43.665959 19386 raft_consensus.cc:2804] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:19:43.666251 19385 raft_consensus.cc:697] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:43.666563 19386 raft_consensus.cc:697] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:43.669245 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:43.669891 10945 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 24.
I20260709 13:19:43.671137 19386 consensus_queue.cc:237] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:43.662256 19384 raft_consensus.cc:697] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:43.679419 19384 consensus_queue.cc:237] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:43.683929 10841 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:43.687407 19399 ts_tablet_manager.cc:933] T d32f5e230e464e61b23a41962708d120 P 12682eb70bc84c76879e04a8628ee586: Initiating tablet copy from peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:43.690064 19399 tablet_copy_client.cc:323] T d32f5e230e464e61b23a41962708d120 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Beginning tablet copy session from remote peer at address 127.9.116.130:37895
I20260709 13:19:43.699195 10946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:43.699618 19371 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:19:43.700030 10946 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 23.
I20260709 13:19:43.700335 19371 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:43.700944 19371 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:19:43.706487 10916 leader_election.cc:336] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:43.707079 10916 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:43.708017 19153 raft_consensus.cc:2749] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:19:43.710328 10916 consensus_peers.cc:597] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:19:43.697448 10915 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:43.671356 19385 consensus_queue.cc:237] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:43.716656 10842 consensus_peers.cc:597] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:43.723568 19398 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:43.724335 19398 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:43.725232 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:43.725884 10876 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 24.
I20260709 13:19:43.728078 19153 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:43.728513 19153 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:43.728858 19153 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:43.739073 10886 tablet_copy_service.cc:140] P 468a8fd46e824f62b9c2c0d3532a6d26: Received BeginTabletCopySession request for tablet d32f5e230e464e61b23a41962708d120 from peer 12682eb70bc84c76879e04a8628ee586 ({username='slave'} at 127.0.0.1:49290)
I20260709 13:19:43.742568 19153 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:43.747754 10871 raft_consensus.cc:2468] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 29.
I20260709 13:19:43.754869 10886 tablet_copy_service.cc:161] P 468a8fd46e824f62b9c2c0d3532a6d26: Beginning new tablet copy session on tablet d32f5e230e464e61b23a41962708d120 from peer 12682eb70bc84c76879e04a8628ee586 at {username='slave'} at 127.0.0.1:49290: session id = 12682eb70bc84c76879e04a8628ee586-d32f5e230e464e61b23a41962708d120
W20260709 13:19:43.779575 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 62247.4us
W20260709 13:19:43.784885 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 67899us
I20260709 13:19:43.791544 11478 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:43.796147 19389 raft_consensus.cc:2804] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:43.796622 19389 raft_consensus.cc:493] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:43.796965 19389 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:43.812446 10871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:43.813485 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:43.814419 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" is_pre_election: true
I20260709 13:19:43.815016 10876 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 23.
I20260709 13:19:43.816025 19398 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:43.817413 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:43.812923 19390 raft_consensus.cc:515] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:43.818039 10948 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 18.
I20260709 13:19:43.819005 11506 raft_consensus.cc:2393] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 24: Already voted for candidate 12682eb70bc84c76879e04a8628ee586 in this term.
I20260709 13:19:43.822160 10873 raft_consensus.cc:1275] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 13:19:43.823197 19390 leader_election.cc:290] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:43.828035 19371 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:43.830335 19371 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:43.833355 10867 raft_consensus.cc:1275] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
W20260709 13:19:43.834435 10842 leader_election.cc:336] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:43.835100 10867 raft_consensus.cc:1275] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 32 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 31 index: 16. Preceding OpId from leader: term: 32 index: 17. (index mismatch)
I20260709 13:19:43.835975 10871 raft_consensus.cc:2393] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 25: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
W20260709 13:19:43.830037 10916 leader_election.cc:336] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:19:43.838512 10916 leader_election.cc:336] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:19:43.839052 10916 consensus_peers.cc:597] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:19:43.839457 10916 consensus_peers.cc:597] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:19:43.840250 19153 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:43.841403 10916 leader_election.cc:336] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:43.842166 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cf3aebfc54a4652a83388ef9a48b1c2" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:43.842458 11505 raft_consensus.cc:1275] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 13:19:43.842868 10950 raft_consensus.cc:3060] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:43.843532 11508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4b13050d714473bb01ea0174d634a14" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:43.844120 11508 raft_consensus.cc:3060] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:43.841663 10841 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:43.850996 19394 consensus_queue.cc:1048] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:19:43.851250 10841 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:43.853163 19371 raft_consensus.cc:2804] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:43.853560 19371 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:43.853911 19371 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:19:43.855019 10842 leader_election.cc:336] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:43.855911 10948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ea9e6b562e4600be1cb566739a6a30" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:43.856511 10948 raft_consensus.cc:3060] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:43.856782 11508 raft_consensus.cc:2468] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 24.
I20260709 13:19:43.861953 10950 raft_consensus.cc:2468] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 24.
I20260709 13:19:43.857800 10915 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:43.864017 19394 raft_consensus.cc:2804] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:19:43.865031 19394 raft_consensus.cc:697] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 24 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:19:43.842192 10916 leader_election.cc:336] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:43.865792 19393 raft_consensus.cc:2804] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:43.880540 19393 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:43.880956 19393 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:43.880867 19389 raft_consensus.cc:515] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:43.883526 19405 log.cc:927] Time spent T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 13:19:43.881932 10697 catalog_manager.cc:5697] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 21 to 22, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 22 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:43.824373 19345 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:43.903038 19371 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:43.903725 10875 raft_consensus.cc:2393] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb0428cc55984daba1ba63744a5102a2 in current term 26: Already voted for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in this term.
I20260709 13:19:43.904749 19345 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:43.877537 10886 tablet_copy_source_session.cc:215] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:19:43.865934 19394 consensus_queue.cc:237] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:43.908272 10916 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:43.912034 19389 leader_election.cc:290] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:43.913182 19399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d32f5e230e464e61b23a41962708d120. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:19:43.917277 10945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41482a95776477784d9bd9dacfed231" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:43.917944 10945 raft_consensus.cc:3060] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:19:43.908356 11477 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:43.920066 10916 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:43.920763 10875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9764b93d3a8407b8cc1ae641620c831" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:43.921365 10875 raft_consensus.cc:3060] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:43.921815 19390 raft_consensus.cc:2804] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 13:19:43.921500 10842 leader_election.cc:336] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:43.923979 10916 leader_election.cc:304] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:43.924227 10842 leader_election.cc:336] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:43.922346 19390 raft_consensus.cc:697] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 24 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:43.929129 19390 consensus_queue.cc:237] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:43.931111 19388 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:19:43.931388 19387 raft_consensus.cc:2804] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 13:19:43.931602 19388 raft_consensus.cc:493] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:43.931624 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc429d4d563f415d90d6f74da422a740" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:43.932044 19387 raft_consensus.cc:697] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 29 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:43.932466 10950 raft_consensus.cc:3060] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:43.933619 10916 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:43.932782 19401 raft_consensus.cc:2749] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:43.934758 19344 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:43.935222 19344 raft_consensus.cc:493] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting leader election (detected failure of leader e01ecd008edd4aa2b6b2b47efbc2ea87)
I20260709 13:19:43.935654 19344 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:43.905265 19371 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:43.932245 19388 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:43.932962 19387 consensus_queue.cc:237] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 16, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:43.937994 10945 raft_consensus.cc:2468] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 26.
I20260709 13:19:43.937997 10948 raft_consensus.cc:2468] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 25.
I20260709 13:19:43.943959 10841 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:43.944339 19362 consensus_queue.cc:1048] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:19:43.959599 19345 raft_consensus.cc:2804] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:19:43.960217 19345 raft_consensus.cc:697] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:43.961141 19345 consensus_queue.cc:237] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:43.950423 10875 raft_consensus.cc:2468] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 23.
I20260709 13:19:43.953519 19153 consensus_queue.cc:1048] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:19:43.954242 19195 consensus_queue.cc:1048] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:19:43.972779 10950 raft_consensus.cc:2468] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 24.
W20260709 13:19:43.973242 10916 leader_election.cc:336] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:43.975725 10916 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:43.977931 19153 raft_consensus.cc:2749] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:19:43.979072 10916 leader_election.cc:336] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:43.980298 10916 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:43.981822 19153 raft_consensus.cc:2749] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:19:43.970696 10841 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:43.983256 10916 consensus_peers.cc:597] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:19:43.990315 10916 leader_election.cc:336] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:43.992587 10916 leader_election.cc:304] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 12682eb70bc84c76879e04a8628ee586, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:43.995584 11478 leader_election.cc:336] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:19:43.996577 11478 leader_election.cc:336] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:44.000991 10841 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:44.001233 11478 leader_election.cc:304] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, 468a8fd46e824f62b9c2c0d3532a6d26; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:44.001924 19153 raft_consensus.cc:2749] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:19:44.003225 10916 leader_election.cc:336] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:19:44.007700 10916 consensus_peers.cc:597] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:44.008234 10916 consensus_peers.cc:597] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:44.008674 10916 consensus_peers.cc:597] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:44.013060 19344 raft_consensus.cc:515] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:44.014580 19408 raft_consensus.cc:2804] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:19:44.015270 19408 raft_consensus.cc:697] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 23 LEADER]: Becoming Leader. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Running, Role: LEADER
I20260709 13:19:44.016093 19344 leader_election.cc:290] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:44.016299 19408 consensus_queue.cc:237] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:44.007133 19370 raft_consensus.cc:2804] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:19:44.019503 19370 raft_consensus.cc:697] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:44.020569 19370 consensus_queue.cc:237] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.11, Last appended by leader: 11, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:44.027634 10916 leader_election.cc:336] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:44.003831 19384 raft_consensus.cc:2804] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader election won for term 25
W20260709 13:19:44.033963 10842 consensus_peers.cc:597] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:44.036183 10842 consensus_peers.cc:597] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:19:44.041216 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a50b3bb87a48494a8e835471e55a6f74" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:44.041896 10867 raft_consensus.cc:3060] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:44.060884 19384 raft_consensus.cc:697] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:44.083655 19384 consensus_queue.cc:237] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:44.094542 10867 raft_consensus.cc:2468] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 24.
W20260709 13:19:44.122133 10916 consensus_peers.cc:597] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:19:44.122907 10916 consensus_peers.cc:597] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:19:44.104832 19414 log.cc:927] Time spent T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:19:44.074550 19388 raft_consensus.cc:515] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:44.146180 10699 catalog_manager.cc:5697] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 23 to 24, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 24 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:44.162664 10699 catalog_manager.cc:5697] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 28 to 29, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 29 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:44.170714 10916 leader_election.cc:304] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:19:44.194427 10842 consensus_peers.cc:597] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:19:44.198880 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:40555, user_credentials={real_user=slave}} blocked reactor thread for 192815us
I20260709 13:19:44.210428 19388 leader_election.cc:290] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:44.197628 19195 raft_consensus.cc:2804] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:19:44.218608 10867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ad199246b34499aa32795a8082ffa8" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "468a8fd46e824f62b9c2c0d3532a6d26"
I20260709 13:19:44.219692 10867 raft_consensus.cc:3060] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:19:44.228052 10842 consensus_peers.cc:597] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:44.231343 10916 leader_election.cc:336] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:19:44.232550 10916 consensus_peers.cc:597] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:19:44.233198 19415 log.cc:927] Time spent T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:19:44.239104 19195 raft_consensus.cc:697] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 24 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:44.216038 19393 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:44.242173 19393 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Requested vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:44.248016 10842 leader_election.cc:336] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:44.248512 10697 catalog_manager.cc:5697] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 23 to 24, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 24 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:44.265617 19399 tablet_copy_client.cc:806] T d32f5e230e464e61b23a41962708d120 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 0 data blocks...
I20260709 13:19:44.266366 19399 tablet_copy_client.cc:670] T d32f5e230e464e61b23a41962708d120 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Starting download of 1 WAL segments...
I20260709 13:19:44.268303 10867 raft_consensus.cc:2468] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 25.
I20260709 13:19:44.268088 19195 consensus_queue.cc:237] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:44.284698 10950 raft_consensus.cc:1275] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 13:19:44.286971 10948 raft_consensus.cc:1275] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:19:44.290539 10916 leader_election.cc:304] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: 
W20260709 13:19:44.291661 10842 consensus_peers.cc:597] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:19:44.301728 19394 raft_consensus.cc:2804] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:19:44.302359 19394 raft_consensus.cc:697] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 25 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
I20260709 13:19:44.308880 19394 consensus_queue.cc:237] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:44.313301 10948 raft_consensus.cc:1275] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:19:44.315346 10950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96558ab13797493e9c168b86fe3b62cd" candidate_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2"
I20260709 13:19:44.316293 10950 raft_consensus.cc:3060] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:44.333377 19399 tablet_copy_client.cc:538] T d32f5e230e464e61b23a41962708d120 P 12682eb70bc84c76879e04a8628ee586: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:19:44.336452  9682 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 10822)
W20260709 13:19:44.362859 10842 consensus_peers.cc:597] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:44.368366 10945 raft_consensus.cc:1275] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:19:44.374318 19353 consensus_queue.cc:1048] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:19:44.376636 19353 consensus_queue.cc:1048] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:19:44.384049 11506 raft_consensus.cc:1275] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 13:19:44.382407 19370 consensus_queue.cc:1048] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:19:44.400727 10950 raft_consensus.cc:2468] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468a8fd46e824f62b9c2c0d3532a6d26 in term 19.
W20260709 13:19:44.410411 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 85172.9us
W20260709 13:19:44.411291 10916 leader_election.cc:336] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:44.423504 19153 consensus_queue.cc:1048] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
W20260709 13:19:44.476989 19433 log.cc:927] Time spent T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 13:19:44.478215 19425 log.cc:927] Time spent T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
W20260709 13:19:44.494057 10842 consensus_peers.cc:597] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:19:44.494627 10842 consensus_peers.cc:597] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:44.515496 10950 raft_consensus.cc:1275] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 13:19:44.517879 19376 tablet_bootstrap.cc:492] T 8cab289810294719ac9686e40290a6ab P 12682eb70bc84c76879e04a8628ee586: Bootstrap replayed 1/1 log segments. Stats: ops{read=13 overwritten=1 applied=12 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:19:44.518942 19376 tablet_bootstrap.cc:492] T 8cab289810294719ac9686e40290a6ab P 12682eb70bc84c76879e04a8628ee586: Bootstrap complete.
I20260709 13:19:44.519748 19376 ts_tablet_manager.cc:1403] T 8cab289810294719ac9686e40290a6ab P 12682eb70bc84c76879e04a8628ee586: Time spent bootstrapping tablet: real 0.902s	user 0.047s	sys 0.025s
I20260709 13:19:44.523041 10867 raft_consensus.cc:1275] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 29 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
W20260709 13:19:44.529199 10842 consensus_peers.cc:597] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:19:44.541445 19424 log.cc:927] Time spent T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586: Append to log took a long time: real 0.156s	user 0.000s	sys 0.003s
W20260709 13:19:44.534286 10842 consensus_peers.cc:597] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:44.549065 10842 consensus_peers.cc:597] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:19:44.550575 10916 consensus_peers.cc:597] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:19:44.576090 19376 raft_consensus.cc:359] T 8cab289810294719ac9686e40290a6ab P 12682eb70bc84c76879e04a8628ee586 [term 21 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
W20260709 13:19:44.593412 10842 consensus_peers.cc:597] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:19:44.594025 10842 consensus_peers.cc:597] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:44.599783 10944 raft_consensus.cc:1275] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 13:19:44.600422 10949 raft_consensus.cc:1275] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 23 index: 11. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 13:19:44.600699 10950 raft_consensus.cc:1275] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 13:19:44.604113 19399 tablet_bootstrap.cc:492] T d32f5e230e464e61b23a41962708d120 P 12682eb70bc84c76879e04a8628ee586: Bootstrap starting.
I20260709 13:19:44.606082 10867 raft_consensus.cc:1275] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Refusing update from remote peer 12682eb70bc84c76879e04a8628ee586: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 13:19:44.613807 19408 consensus_queue.cc:1048] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:19:44.583484 19376 raft_consensus.cc:740] T 8cab289810294719ac9686e40290a6ab P 12682eb70bc84c76879e04a8628ee586 [term 21 LEARNER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Initialized, Role: LEARNER
W20260709 13:19:44.620508 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 68552.8us
I20260709 13:19:44.632709 19380 tablet_bootstrap.cc:492] T b761e5168d4e48aeaec47b99f5c7d3d0 P 12682eb70bc84c76879e04a8628ee586: Bootstrap replayed 1/1 log segments. Stats: ops{read=15 overwritten=0 applied=15 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:19:44.660612 19328 consensus_queue.cc:1048] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:19:44.675979 19380 tablet_bootstrap.cc:492] T b761e5168d4e48aeaec47b99f5c7d3d0 P 12682eb70bc84c76879e04a8628ee586: Bootstrap complete.
I20260709 13:19:44.676160 19384 consensus_queue.cc:1048] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:19:44.676882 19380 ts_tablet_manager.cc:1403] T b761e5168d4e48aeaec47b99f5c7d3d0 P 12682eb70bc84c76879e04a8628ee586: Time spent bootstrapping tablet: real 1.030s	user 0.050s	sys 0.017s
I20260709 13:19:44.676342 19344 consensus_queue.cc:1048] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:19:44.677582 10841 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2; no voters: e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:44.683996 19383 raft_consensus.cc:2804] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:44.684595 19383 raft_consensus.cc:697] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Becoming Leader. State: Replica: 468a8fd46e824f62b9c2c0d3532a6d26, State: Running, Role: LEADER
I20260709 13:19:44.685508 19383 consensus_queue.cc:237] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:44.706641 11478 consensus_peers.cc:597] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:44.707154 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 47253.4us
W20260709 13:19:44.707643 10916 consensus_peers.cc:597] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:44.713505 10699 catalog_manager.cc:5697] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 19 to 20, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 20 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
W20260709 13:19:44.722520 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 63650.1us
I20260709 13:19:44.717846 10697 catalog_manager.cc:5697] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 23 to 24, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 24 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:44.720773 10699 catalog_manager.cc:5697] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 23 to 24, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 24 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:44.728799 10697 catalog_manager.cc:5697] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 24 to 25, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 25 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:44.730096 10699 catalog_manager.cc:5697] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 23 to 24, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 24 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:44.731825 10699 catalog_manager.cc:5697] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 25 to 26, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 26 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:44.733879 10699 catalog_manager.cc:5697] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 24 to 25, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 25 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:44.671180 19376 consensus_queue.cc:260] T 8cab289810294719ac9686e40290a6ab P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
W20260709 13:19:44.760428 11478 consensus_peers.cc:597] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:44.726148 19345 consensus_queue.cc:1048] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 13:19:44.727520 19382 consensus_queue.cc:1048] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.025s
W20260709 13:19:44.772220 10916 consensus_peers.cc:597] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:44.779073 10876 raft_consensus.cc:1275] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
W20260709 13:19:44.811328 19442 log.cc:927] Time spent T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:19:44.770560 19397 consensus_queue.cc:1048] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.097s
I20260709 13:19:44.814189 19376 ts_tablet_manager.cc:1434] T 8cab289810294719ac9686e40290a6ab P 12682eb70bc84c76879e04a8628ee586: Time spent starting tablet: real 0.294s	user 0.008s	sys 0.007s
I20260709 13:19:44.814189 19399 tablet_bootstrap.cc:700] T d32f5e230e464e61b23a41962708d120 P 12682eb70bc84c76879e04a8628ee586: Time spent opening tablet: real 0.121s	user 0.001s	sys 0.000s
I20260709 13:19:44.816490 19380 raft_consensus.cc:359] T b761e5168d4e48aeaec47b99f5c7d3d0 P 12682eb70bc84c76879e04a8628ee586 [term 21 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:19:44.817480 19380 raft_consensus.cc:740] T b761e5168d4e48aeaec47b99f5c7d3d0 P 12682eb70bc84c76879e04a8628ee586 [term 21 LEARNER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Initialized, Role: LEARNER
W20260709 13:19:44.819116 11478 consensus_peers.cc:597] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:19:44.845444 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 113076us
W20260709 13:19:44.851498 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 50833.9us
I20260709 13:19:44.852556 11505 raft_consensus.cc:1217] T 8cab289810294719ac9686e40290a6ab P 12682eb70bc84c76879e04a8628ee586 [term 21 LEARNER]: Deduplicated request from leader. Original: 21.11->[21.12-21.12]   Dedup: 21.12->[]
I20260709 13:19:44.818230 19380 consensus_queue.cc:260] T b761e5168d4e48aeaec47b99f5c7d3d0 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:19:44.870772 19438 consensus_queue.cc:1048] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.010s
I20260709 13:19:44.882541 19380 ts_tablet_manager.cc:1434] T b761e5168d4e48aeaec47b99f5c7d3d0 P 12682eb70bc84c76879e04a8628ee586: Time spent starting tablet: real 0.205s	user 0.008s	sys 0.001s
W20260709 13:19:44.892444 10842 consensus_peers.cc:597] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:44.896908 10842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 112472us
I20260709 13:19:44.869764 10959 tablet_copy_service.cc:342] P cb0428cc55984daba1ba63744a5102a2: Request end of tablet copy session 12682eb70bc84c76879e04a8628ee586-8cab289810294719ac9686e40290a6ab received from {username='slave'} at 127.0.0.1:36714
I20260709 13:19:44.961282 10959 tablet_copy_service.cc:434] P cb0428cc55984daba1ba63744a5102a2: ending tablet copy session 12682eb70bc84c76879e04a8628ee586-8cab289810294719ac9686e40290a6ab on tablet 8cab289810294719ac9686e40290a6ab with peer 12682eb70bc84c76879e04a8628ee586
W20260709 13:19:44.969463 10916 consensus_peers.cc:597] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:44.969939 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 107138us
W20260709 13:19:45.035461 10916 consensus_peers.cc:597] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:45.034204 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 72360.9us
W20260709 13:19:45.051852 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 121652us
I20260709 13:19:45.114188 10693 catalog_manager.cc:5697] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: term changed from 22 to 23, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 12682eb70bc84c76879e04a8628ee586 (127.9.116.132). New cstate: current_term: 23 leader_uuid: "12682eb70bc84c76879e04a8628ee586" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:19:45.155141 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 55900.3us
I20260709 13:19:45.195990 10867 raft_consensus.cc:1275] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
W20260709 13:19:45.169751 19463 log.cc:927] Time spent T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2: Append to log took a long time: real 0.073s	user 0.001s	sys 0.001s
I20260709 13:19:45.199201 11505 raft_consensus.cc:1217] T b761e5168d4e48aeaec47b99f5c7d3d0 P 12682eb70bc84c76879e04a8628ee586 [term 21 LEARNER]: Deduplicated request from leader. Original: 21.14->[21.15-21.15]   Dedup: 21.15->[]
I20260709 13:19:45.236217 10886 tablet_copy_service.cc:342] P 468a8fd46e824f62b9c2c0d3532a6d26: Request end of tablet copy session 12682eb70bc84c76879e04a8628ee586-b761e5168d4e48aeaec47b99f5c7d3d0 received from {username='slave'} at 127.0.0.1:49290
I20260709 13:19:45.236744 10886 tablet_copy_service.cc:434] P 468a8fd46e824f62b9c2c0d3532a6d26: ending tablet copy session 12682eb70bc84c76879e04a8628ee586-b761e5168d4e48aeaec47b99f5c7d3d0 on tablet b761e5168d4e48aeaec47b99f5c7d3d0 with peer 12682eb70bc84c76879e04a8628ee586
W20260709 13:19:45.258054 11478 consensus_peers.cc:597] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:19:45.284217 10842 consensus_peers.cc:597] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:19:45.284773 10842 consensus_peers.cc:597] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:19:45.341490 11478 consensus_peers.cc:597] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:19:45.347035 19401 raft_consensus.cc:1064] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2: attempting to promote NON_VOTER 12682eb70bc84c76879e04a8628ee586 to VOTER
W20260709 13:19:45.352620  9682 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 10822)
I20260709 13:19:45.366914 19344 consensus_queue.cc:1048] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:19:45.368590 19401 consensus_queue.cc:237] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 10, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:45.374610 10842 consensus_peers.cc:597] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:45.375309 10842 consensus_peers.cc:597] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:19:45.383123 10842 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111) [suppressed 403 similar messages]
I20260709 13:19:45.384374 10950 raft_consensus.cc:1275] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:19:45.397903 19370 consensus_queue.cc:1048] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:19:45.439643 10699 catalog_manager.cc:5697] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: term changed from 18 to 19, leader changed from e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) to 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130). New cstate: current_term: 19 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:45.477187 11505 raft_consensus.cc:1275] T 8cab289810294719ac9686e40290a6ab P 12682eb70bc84c76879e04a8628ee586 [term 21 LEARNER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:19:45.478202 10871 raft_consensus.cc:1275] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:19:45.480893 19153 consensus_queue.cc:1048] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:19:45.497673 10916 consensus_peers.cc:597] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:45.499083 19153 consensus_queue.cc:1048] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:19:45.542330 19399 tablet_bootstrap.cc:492] T d32f5e230e464e61b23a41962708d120 P 12682eb70bc84c76879e04a8628ee586: Bootstrap replayed 1/1 log segments. Stats: ops{read=15 overwritten=0 applied=15 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:19:45.579241 19399 tablet_bootstrap.cc:492] T d32f5e230e464e61b23a41962708d120 P 12682eb70bc84c76879e04a8628ee586: Bootstrap complete.
I20260709 13:19:45.580052 19399 ts_tablet_manager.cc:1403] T d32f5e230e464e61b23a41962708d120 P 12682eb70bc84c76879e04a8628ee586: Time spent bootstrapping tablet: real 0.976s	user 0.051s	sys 0.011s
I20260709 13:19:45.584753 19399 raft_consensus.cc:359] T d32f5e230e464e61b23a41962708d120 P 12682eb70bc84c76879e04a8628ee586 [term 23 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
I20260709 13:19:45.585608 19399 raft_consensus.cc:740] T d32f5e230e464e61b23a41962708d120 P 12682eb70bc84c76879e04a8628ee586 [term 23 LEARNER]: Becoming Follower/Learner. State: Replica: 12682eb70bc84c76879e04a8628ee586, State: Initialized, Role: LEARNER
I20260709 13:19:45.586349 19399 consensus_queue.cc:260] T d32f5e230e464e61b23a41962708d120 P 12682eb70bc84c76879e04a8628ee586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: NON_VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: true } }
W20260709 13:19:45.586903 10916 consensus_peers.cc:597] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:19:45.588876 19399 ts_tablet_manager.cc:1434] T d32f5e230e464e61b23a41962708d120 P 12682eb70bc84c76879e04a8628ee586: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.002s
I20260709 13:19:45.603363 10886 tablet_copy_service.cc:342] P 468a8fd46e824f62b9c2c0d3532a6d26: Request end of tablet copy session 12682eb70bc84c76879e04a8628ee586-d32f5e230e464e61b23a41962708d120 received from {username='slave'} at 127.0.0.1:49290
I20260709 13:19:45.603830 10886 tablet_copy_service.cc:434] P 468a8fd46e824f62b9c2c0d3532a6d26: ending tablet copy session 12682eb70bc84c76879e04a8628ee586-d32f5e230e464e61b23a41962708d120 on tablet d32f5e230e464e61b23a41962708d120 with peer 12682eb70bc84c76879e04a8628ee586
W20260709 13:19:45.607957 10842 consensus_peers.cc:597] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:19:45.608291 19153 raft_consensus.cc:2955] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 21 LEADER]: Committing config change with OpId 21.13: config changed from index 12 to 13, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:45.613626 11505 raft_consensus.cc:2955] T 8cab289810294719ac9686e40290a6ab P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Committing config change with OpId 21.13: config changed from index 12 to 13, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:45.626061 10876 raft_consensus.cc:2955] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Committing config change with OpId 21.13: config changed from index 12 to 13, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:45.681419 10693 catalog_manager.cc:5697] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: config changed from index 12 to 13, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New cstate: current_term: 21 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:45.703240 19469 raft_consensus.cc:1064] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26: attempting to promote NON_VOTER 12682eb70bc84c76879e04a8628ee586 to VOTER
I20260709 13:19:45.706166 19469 consensus_queue.cc:237] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 13, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:45.756875 11478 consensus_peers.cc:597] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:19:45.811039 10842 consensus_peers.cc:597] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:19:45.832914 10916 consensus_peers.cc:597] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:19:45.843683 10950 raft_consensus.cc:1275] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:19:45.850088 11506 raft_consensus.cc:1275] T b761e5168d4e48aeaec47b99f5c7d3d0 P 12682eb70bc84c76879e04a8628ee586 [term 21 LEARNER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:19:45.852778 19451 consensus_queue.cc:1048] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:19:45.854751 19451 consensus_queue.cc:1048] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.006s
W20260709 13:19:45.874863 10842 consensus_peers.cc:597] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:45.959414 10916 consensus_peers.cc:597] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:19:46.003530 11505 raft_consensus.cc:1217] T d32f5e230e464e61b23a41962708d120 P 12682eb70bc84c76879e04a8628ee586 [term 23 LEARNER]: Deduplicated request from leader. Original: 23.14->[23.15-23.15]   Dedup: 23.15->[]
W20260709 13:19:46.015239 10842 consensus_peers.cc:597] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:19:46.073681 10950 raft_consensus.cc:2955] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Committing config change with OpId 21.16: config changed from index 15 to 16, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:46.069370 19465 raft_consensus.cc:2955] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 LEADER]: Committing config change with OpId 21.16: config changed from index 15 to 16, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
W20260709 13:19:46.099627 10916 consensus_peers.cc:597] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:19:46.107426 11505 raft_consensus.cc:2955] T b761e5168d4e48aeaec47b99f5c7d3d0 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Committing config change with OpId 21.16: config changed from index 15 to 16, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:46.118291 10693 catalog_manager.cc:5697] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: config changed from index 15 to 16, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New cstate: current_term: 21 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
W20260709 13:19:46.145541 10916 consensus_peers.cc:597] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:19:46.226552 10916 consensus_peers.cc:597] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:19:46.231899 10916 consensus_peers.cc:597] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:19:46.273634 10916 consensus_peers.cc:597] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:19:46.353096  9682 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 10822)
W20260709 13:19:46.353154 10916 consensus_peers.cc:597] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:19:46.390517 10916 consensus_peers.cc:597] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:19:46.425446 10842 consensus_peers.cc:597] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:19:46.514609 19465 raft_consensus.cc:1064] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26: attempting to promote NON_VOTER 12682eb70bc84c76879e04a8628ee586 to VOTER
W20260709 13:19:46.549559 10842 consensus_peers.cc:597] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:19:46.546132 19465 consensus_queue.cc:237] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 13, Current term: 23, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:46.570981 10916 consensus_peers.cc:597] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:19:46.586627 10916 consensus_peers.cc:597] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:19:46.619284 11506 raft_consensus.cc:1275] T d32f5e230e464e61b23a41962708d120 P 12682eb70bc84c76879e04a8628ee586 [term 23 LEARNER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:19:46.620728 10950 raft_consensus.cc:1275] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:19:46.623193 19386 consensus_queue.cc:1048] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:19:46.626437 19469 consensus_queue.cc:1048] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.002s
W20260709 13:19:46.630548 10842 consensus_peers.cc:597] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:46.677386 10842 consensus_peers.cc:597] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:19:46.678660 10842 consensus_peers.cc:597] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:19:46.699361 10916 consensus_peers.cc:597] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:19:46.703753 19386 raft_consensus.cc:2955] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 LEADER]: Committing config change with OpId 23.16: config changed from index 15 to 16, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:46.708030 10948 raft_consensus.cc:2955] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Committing config change with OpId 23.16: config changed from index 15 to 16, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
W20260709 13:19:46.779127 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 90077.6us
I20260709 13:19:46.779091 11506 raft_consensus.cc:2955] T d32f5e230e464e61b23a41962708d120 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Committing config change with OpId 23.16: config changed from index 15 to 16, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
W20260709 13:19:46.797458 10916 consensus_peers.cc:597] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:19:46.799510 10842 consensus_peers.cc:597] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:19:46.803375 10697 catalog_manager.cc:5697] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: config changed from index 15 to 16, 12682eb70bc84c76879e04a8628ee586 (127.9.116.132) changed from NON_VOTER to VOTER. New cstate: current_term: 23 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:19:46.816869 10842 consensus_peers.cc:597] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:19:46.870342 10948 consensus_queue.cc:237] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:46.898923 11506 raft_consensus.cc:1275] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:19:46.903105 19387 consensus_queue.cc:1048] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:19:46.935717 10871 consensus_queue.cc:237] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:46.941383 19387 raft_consensus.cc:2955] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 23 LEADER]: Committing config change with OpId 23.14: config changed from index 4 to 14, VOTER 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
W20260709 13:19:46.943291 11478 consensus_peers.cc:597] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:19:46.946748 10948 raft_consensus.cc:1275] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:19:46.947763 11506 raft_consensus.cc:1275] T d32f5e230e464e61b23a41962708d120 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Refusing update from remote peer 468a8fd46e824f62b9c2c0d3532a6d26: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:19:46.958636 11512 raft_consensus.cc:2955] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Committing config change with OpId 23.14: config changed from index 4 to 14, VOTER 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
W20260709 13:19:46.966605 10842 consensus_peers.cc:597] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:19:46.971889 19465 consensus_queue.cc:1048] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:19:46.985041 19386 raft_consensus.cc:2955] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 LEADER]: Committing config change with OpId 23.17: config changed from index 16 to 17, VOTER e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:46.987454 19448 consensus_queue.cc:1048] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:19:47.009370 10950 raft_consensus.cc:2955] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Committing config change with OpId 23.17: config changed from index 16 to 17, VOTER e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:47.023931 11512 raft_consensus.cc:2955] T d32f5e230e464e61b23a41962708d120 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Committing config change with OpId 23.17: config changed from index 16 to 17, VOTER e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
W20260709 13:19:47.029889 10916 consensus_peers.cc:597] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:47.041543 10693 catalog_manager.cc:5697] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 reported cstate change: config changed from index 4 to 14, VOTER 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) evicted. New cstate: current_term: 23 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } } }
I20260709 13:19:47.071169 10686 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d32f5e230e464e61b23a41962708d120 with cas_config_opid_index 16: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:19:47.093262 10699 catalog_manager.cc:5697] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 reported cstate change: config changed from index 16 to 17, VOTER e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129) evicted. New cstate: current_term: 23 leader_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:47.099233 10863 tablet_service.cc:1558] Processing DeleteTablet for tablet 695ff1b5a68e4111917d909909488d39 with delete_type TABLET_DATA_TOMBSTONED (TS 468a8fd46e824f62b9c2c0d3532a6d26 not found in new config with opid_index 14) from {username='slave'} at 127.0.0.1:34958
I20260709 13:19:47.128307 19500 tablet_replica.cc:333] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
W20260709 13:19:47.129670 10842 consensus_peers.cc:597] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:19:47.138042 10685 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 695ff1b5a68e4111917d909909488d39 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 11)
W20260709 13:19:47.156106 10842 consensus_peers.cc:597] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:19:47.157979 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 48126.6us
W20260709 13:19:47.164281 10842 consensus_peers.cc:597] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:19:47.164817 10842 consensus_peers.cc:597] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:19:47.165256 10842 consensus_peers.cc:597] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:19:47.168648 19500 raft_consensus.cc:2243] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:47.169495 19500 raft_consensus.cc:2272] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:47.172461 19500 ts_tablet_manager.cc:1916] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 13:19:47.180421 10916 consensus_peers.cc:597] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:19:47.191918 11478 consensus_peers.cc:597] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:19:47.206986 11478 consensus_peers.cc:597] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:19:47.226823 19500 ts_tablet_manager.cc:1929] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 23.13
I20260709 13:19:47.227406 19500 log.cc:1199] T 695ff1b5a68e4111917d909909488d39 P 468a8fd46e824f62b9c2c0d3532a6d26: Deleting WAL directory at /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0/minicluster-data/ts-1-root/wals/695ff1b5a68e4111917d909909488d39
I20260709 13:19:47.235069 10686 catalog_manager.cc:5028] TS 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): tablet 695ff1b5a68e4111917d909909488d39 (table test-workload [id=2cc9dc95268049cfa138bef9f37fcbe7]) successfully deleted
W20260709 13:19:47.238000 10686 catalog_manager.cc:4755] TS e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet d32f5e230e464e61b23a41962708d120: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:19:47.258882 10916 consensus_peers.cc:597] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:19:47.270730 10842 consensus_peers.cc:597] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:19:47.325830 10842 consensus_peers.cc:597] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:19:47.353776  9682 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 10822)
W20260709 13:19:47.413043 10842 consensus_peers.cc:597] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:19:47.430436 10916 consensus_peers.cc:597] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:19:47.447212 10842 consensus_peers.cc:597] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:19:47.471369 10916 consensus_peers.cc:597] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:19:47.475101 11478 consensus_peers.cc:597] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:19:47.485311 10842 consensus_peers.cc:597] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:19:47.593197 10842 consensus_peers.cc:597] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:19:47.667639 10842 consensus_peers.cc:597] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:19:47.744552 11478 consensus_peers.cc:597] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:19:47.793730 10916 consensus_peers.cc:597] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:19:47.830721 10842 consensus_peers.cc:597] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:19:48.080585 10916 consensus_peers.cc:597] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:19:48.100577 10842 consensus_peers.cc:597] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:19:48.264737 10842 consensus_peers.cc:597] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:19:48.313596 10842 consensus_peers.cc:597] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:19:48.354322  9682 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 10822)
W20260709 13:19:48.381819 11478 consensus_peers.cc:597] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:19:48.451614 10686 catalog_manager.cc:4755] TS e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet d32f5e230e464e61b23a41962708d120: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:19:48.583261 10916 consensus_peers.cc:597] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:19:48.615936 10916 consensus_peers.cc:597] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:19:48.627213 10916 consensus_peers.cc:597] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:19:48.724723 10916 consensus_peers.cc:597] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:19:48.763386 10916 consensus_peers.cc:597] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:19:48.994064 10916 consensus_peers.cc:597] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:19:48.994711 10916 consensus_peers.cc:597] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:19:48.995244 10916 consensus_peers.cc:597] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:19:49.064451 10842 consensus_peers.cc:597] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:19:49.071475 10842 consensus_peers.cc:597] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:19:49.075139 10842 consensus_peers.cc:597] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:19:49.071553 10916 consensus_peers.cc:597] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:19:49.097564 10916 consensus_peers.cc:597] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:19:49.116999 10842 consensus_peers.cc:597] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:19:49.131265 10842 consensus_peers.cc:597] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:19:49.146159 10916 consensus_peers.cc:597] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:19:49.155386 10842 consensus_peers.cc:597] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:19:49.312438 10916 consensus_peers.cc:597] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:19:49.339010 10842 consensus_peers.cc:597] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:19:49.392467  9682 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 10822)
W20260709 13:19:49.496825 10916 consensus_peers.cc:597] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:19:49.504159 10916 consensus_peers.cc:597] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:19:49.570760 10842 consensus_peers.cc:597] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:19:49.587857 10916 consensus_peers.cc:597] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:19:49.599560 10842 consensus_peers.cc:597] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:19:49.602571 11478 consensus_peers.cc:597] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:19:49.639504 10686 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 50627.6us
W20260709 13:19:49.650758 10842 consensus_peers.cc:597] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:19:49.659276 10686 catalog_manager.cc:4755] TS e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet d32f5e230e464e61b23a41962708d120: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:19:49.666231 11478 consensus_peers.cc:597] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:19:49.667537 11478 consensus_peers.cc:597] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:19:49.733939 10842 consensus_peers.cc:597] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:19:49.769721 10842 consensus_peers.cc:597] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:19:49.791301 10842 consensus_peers.cc:597] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:19:49.804291 11478 consensus_peers.cc:597] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:19:49.804978 10916 consensus_peers.cc:597] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:19:49.811174 10842 consensus_peers.cc:597] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:19:49.971098 10842 consensus_peers.cc:597] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:19:49.973130 10916 consensus_peers.cc:597] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:19:50.023492 10842 consensus_peers.cc:597] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:19:50.048519 10842 consensus_peers.cc:597] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:19:50.066994 10842 consensus_peers.cc:597] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:19:50.156085 11478 consensus_peers.cc:597] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:19:50.211730 10842 consensus_peers.cc:597] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:19:50.257193 10842 consensus_peers.cc:597] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:19:50.383472 10916 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111) [suppressed 493 similar messages]
W20260709 13:19:50.393623  9682 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 10822)
W20260709 13:19:50.543993 10916 consensus_peers.cc:597] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:19:50.672660 10842 consensus_peers.cc:597] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:19:50.673491 10842 consensus_peers.cc:597] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:19:50.759621 10916 consensus_peers.cc:597] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:19:50.766376 10842 consensus_peers.cc:597] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:19:50.959239 10916 consensus_peers.cc:597] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:19:50.970918 11478 consensus_peers.cc:597] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:19:51.115844 10916 consensus_peers.cc:597] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:19:51.204849 10916 consensus_peers.cc:597] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:19:51.277390 10916 consensus_peers.cc:597] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:19:51.325970 10916 consensus_peers.cc:597] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:19:51.370223 10842 consensus_peers.cc:597] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:19:51.375350 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 62618.3us
W20260709 13:19:51.394143  9682 thread.cc:527] Waited for 9000ms trying to join with diag-logger (tid 10822)
W20260709 13:19:51.427160 10842 consensus_peers.cc:597] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:19:51.425282 10916 consensus_peers.cc:597] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:19:51.428275 10916 consensus_peers.cc:597] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:19:51.434672 10842 consensus_peers.cc:597] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:19:51.466655 10916 consensus_peers.cc:597] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:19:51.550738 10842 consensus_peers.cc:597] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:19:51.664762 10916 consensus_peers.cc:597] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:19:51.673976 10842 consensus_peers.cc:597] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:19:51.685599 10842 consensus_peers.cc:597] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:19:51.687728 10916 consensus_peers.cc:597] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:19:51.718710 10916 consensus_peers.cc:597] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:19:51.765574 10686 catalog_manager.cc:4755] TS e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet d32f5e230e464e61b23a41962708d120: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:19:51.783190 10916 consensus_peers.cc:597] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:19:51.788223 10842 consensus_peers.cc:597] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:19:51.806231 10916 consensus_peers.cc:597] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:19:51.820502 10916 consensus_peers.cc:597] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:19:51.920516 10916 consensus_peers.cc:597] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:19:51.976991 10842 consensus_peers.cc:597] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:19:52.086987 11478 consensus_peers.cc:597] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:19:52.093152 10842 consensus_peers.cc:597] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:19:52.165809 10842 consensus_peers.cc:597] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:19:52.177237 11478 consensus_peers.cc:597] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:19:52.206167 10916 consensus_peers.cc:597] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:19:52.224558 10842 consensus_peers.cc:597] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:19:52.227519 10842 consensus_peers.cc:597] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:19:52.238299 10842 consensus_peers.cc:597] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:19:52.348258 11478 consensus_peers.cc:597] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:19:52.394631  9682 thread.cc:527] Waited for 10000ms trying to join with diag-logger (tid 10822)
W20260709 13:19:52.436013 10916 consensus_peers.cc:597] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:19:52.447520 10842 consensus_peers.cc:597] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:19:52.448341 10842 consensus_peers.cc:597] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:19:52.448911 10842 consensus_peers.cc:597] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:19:52.475375 11478 consensus_peers.cc:597] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:19:52.527235 10842 consensus_peers.cc:597] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:19:52.573449 10842 consensus_peers.cc:597] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:19:52.593261 10842 consensus_peers.cc:597] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:19:52.608500 11478 consensus_peers.cc:597] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:19:52.795861 10916 consensus_peers.cc:597] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:19:52.912755 10842 consensus_peers.cc:597] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:19:53.319744 10842 consensus_peers.cc:597] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:19:53.320276 10916 consensus_peers.cc:597] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:19:53.367298 10842 consensus_peers.cc:597] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:19:53.383664 10916 consensus_peers.cc:597] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:19:53.384266 11478 consensus_peers.cc:597] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:19:53.395056 10916 consensus_peers.cc:597] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:19:53.395195  9682 thread.cc:527] Waited for 11000ms trying to join with diag-logger (tid 10822)
W20260709 13:19:53.397661 10842 consensus_peers.cc:597] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:19:53.543469 10916 consensus_peers.cc:597] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:19:53.677417 10916 consensus_peers.cc:597] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:19:53.678020 10916 consensus_peers.cc:597] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:19:53.685117 10842 consensus_peers.cc:597] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:19:53.734683 10842 consensus_peers.cc:597] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:19:53.738456 10916 consensus_peers.cc:597] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:19:53.764115 10916 consensus_peers.cc:597] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:19:53.799257 10916 consensus_peers.cc:597] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:19:54.018157 10916 consensus_peers.cc:597] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:19:54.115172 10842 consensus_peers.cc:597] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:19:54.115664 10842 consensus_peers.cc:597] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:19:54.131723 10916 consensus_peers.cc:597] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:19:54.257405 11478 consensus_peers.cc:597] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:19:54.292762 10842 consensus_peers.cc:597] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:19:54.325012 10916 consensus_peers.cc:597] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:19:54.360064 10688 maintenance_manager.cc:419] P ae957c8fc66c4b6ba063d0c74cbf5484: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.033352
W20260709 13:19:54.370165 10916 consensus_peers.cc:597] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:19:54.398618  9682 thread.cc:527] Waited for 12000ms trying to join with diag-logger (tid 10822)
W20260709 13:19:54.406872 10842 consensus_peers.cc:597] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:19:54.447783 10842 consensus_peers.cc:597] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:19:54.468509 10842 consensus_peers.cc:597] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:19:54.477754 10916 consensus_peers.cc:597] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:19:54.483678 10842 consensus_peers.cc:597] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:19:54.484594 10687 maintenance_manager.cc:643] P ae957c8fc66c4b6ba063d0c74cbf5484: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.123s	user 0.040s	sys 0.010s Metrics: {"bytes_written":6989,"delete_count":0,"lbm_write_time_us":1724,"lbm_writes_lt_1ms":6,"reinsert_count":0,"thread_start_us":24981,"threads_started":1,"update_count":282,"wal-append.queue_time_us":21763}
I20260709 13:19:54.492074 10688 maintenance_manager.cc:419] P ae957c8fc66c4b6ba063d0c74cbf5484: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.350198
W20260709 13:19:54.513357 10916 consensus_peers.cc:597] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:19:54.513912 10916 consensus_peers.cc:597] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:19:54.572031 10842 consensus_peers.cc:597] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:19:54.621538 10842 consensus_peers.cc:597] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:19:54.697122 10842 consensus_peers.cc:597] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:19:54.707026 10916 consensus_peers.cc:597] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:19:54.715549 11478 consensus_peers.cc:597] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:19:54.753813 10842 consensus_peers.cc:597] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:19:54.766395 11478 consensus_peers.cc:597] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:19:54.776721 10842 consensus_peers.cc:597] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:19:54.823475 10842 consensus_peers.cc:597] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:19:54.861420 11478 consensus_peers.cc:597] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:19:54.864358 10687 maintenance_manager.cc:643] P ae957c8fc66c4b6ba063d0c74cbf5484: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.371s	user 0.091s	sys 0.015s Metrics: {"cfile_cache_miss":9,"cfile_cache_miss_bytes":28405,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":11538,"dirs.run_cpu_time_us":595,"dirs.run_wall_time_us":4549,"lbm_read_time_us":2061,"lbm_reads_lt_1ms":25,"lbm_write_time_us":1812,"lbm_writes_lt_1ms":12,"peak_mem_usage":308833,"reinsert_count":0,"thread_start_us":52298,"threads_started":1,"update_count":303}
I20260709 13:19:54.867048 10688 maintenance_manager.cc:419] P ae957c8fc66c4b6ba063d0c74cbf5484: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260709 13:19:54.867925 10687 maintenance_manager.cc:643] P ae957c8fc66c4b6ba063d0c74cbf5484: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {}
W20260709 13:19:54.883008 11478 consensus_peers.cc:597] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:19:55.121913 10842 consensus_peers.cc:597] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:19:55.154347 10842 consensus_peers.cc:597] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:19:55.200210 10916 consensus_peers.cc:597] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:19:55.251838 10842 consensus_peers.cc:597] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:19:55.286768 10842 consensus_peers.cc:597] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:19:55.325781 10916 consensus_peers.cc:597] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:19:55.392141 10842 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111) [suppressed 478 similar messages]
W20260709 13:19:55.399185  9682 thread.cc:527] Waited for 13000ms trying to join with diag-logger (tid 10822)
W20260709 13:19:55.639305 10842 consensus_peers.cc:597] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:19:55.756328 10842 consensus_peers.cc:597] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:19:55.856427 10916 consensus_peers.cc:597] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:19:55.881439 10842 consensus_peers.cc:597] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:19:55.951577 10842 consensus_peers.cc:597] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:19:55.953815 10686 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 47274.9us
W20260709 13:19:55.960907 10916 consensus_peers.cc:597] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:19:55.973088 10686 catalog_manager.cc:4755] TS e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet d32f5e230e464e61b23a41962708d120: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:19:55.994117 11478 consensus_peers.cc:597] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:19:56.055079 10916 consensus_peers.cc:597] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:19:56.081092 10916 consensus_peers.cc:597] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:19:56.093613 10842 consensus_peers.cc:597] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:19:56.115801 10916 consensus_peers.cc:597] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:19:56.133311 10916 consensus_peers.cc:597] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:19:56.163648 10916 consensus_peers.cc:597] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:19:56.285923 10916 consensus_peers.cc:597] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:19:56.312916 10842 consensus_peers.cc:597] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:19:56.400418  9682 thread.cc:527] Waited for 14000ms trying to join with diag-logger (tid 10822)
W20260709 13:19:56.480177 10916 consensus_peers.cc:597] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:19:56.522300 10916 consensus_peers.cc:597] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:19:56.699194 10842 consensus_peers.cc:597] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:19:56.710047 10842 consensus_peers.cc:597] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:19:56.727250 11478 consensus_peers.cc:597] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:19:56.743637 10916 consensus_peers.cc:597] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:19:56.754750  9682 tablet_server.cc:196] TabletServer@127.9.116.129:0 shutdown complete.
W20260709 13:19:56.773198 10842 consensus_peers.cc:597] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:19:56.821333 10916 consensus_peers.cc:597] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:19:56.822113 10842 consensus_peers.cc:597] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:19:56.822620 10842 consensus_peers.cc:597] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:19:56.880407 10916 consensus_peers.cc:597] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:19:56.959331 10842 consensus_peers.cc:597] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:19:56.974489 10842 consensus_peers.cc:597] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:19:57.019099 10842 consensus_peers.cc:597] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:19:57.026736 10916 consensus_peers.cc:597] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:19:57.051399 11478 consensus_peers.cc:597] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:19:57.054042 10916 consensus_peers.cc:597] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:19:57.153589 10842 consensus_peers.cc:597] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:19:57.188886 11478 consensus_peers.cc:597] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:19:57.245313  9682 tablet_server.cc:179] TabletServer@127.9.116.130:0 shutting down...
W20260709 13:19:57.248136 11478 consensus_peers.cc:597] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:19:57.253299 10842 consensus_peers.cc:597] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:19:57.257843 10916 consensus_peers.cc:597] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:19:57.273401 10916 consensus_peers.cc:597] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:57.277293 10916 consensus_peers.cc:597] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:57.293504 10842 consensus_peers.cc:597] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:19:57.333784 11478 consensus_peers.cc:597] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:57.341485 11478 consensus_peers.cc:597] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:19:57.344967 10916 consensus_peers.cc:597] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:19:57.359299 10916 consensus_peers.cc:597] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:57.382553 10916 consensus_peers.cc:597] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:57.394877 10916 consensus_peers.cc:597] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:57.400739 10842 consensus_peers.cc:597] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:19:57.424852 11478 consensus_peers.cc:597] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:57.428076 11478 consensus_peers.cc:597] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:57.429931 11478 consensus_peers.cc:597] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:57.440606 11478 consensus_peers.cc:597] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:57.452113 10842 consensus_peers.cc:597] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:19:57.457886 10842 consensus_peers.cc:597] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:19:57.471571 10916 consensus_peers.cc:597] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:57.474063 11478 consensus_peers.cc:597] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:57.511834 10916 consensus_peers.cc:597] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:57.519330 11478 consensus_peers.cc:597] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:57.548390  9682 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:19:57.553570  9682 tablet_replica.cc:333] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:19:57.554750  9682 raft_consensus.cc:2243] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:19:57.556242  9682 raft_consensus.cc:2272] T 5c8a233bc6c44bd2b5d2d44a726b147c P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:57.564493  9682 tablet_replica.cc:333] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:19:57.569489  9682 raft_consensus.cc:2243] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:57.575279  9682 raft_consensus.cc:2272] T 2778f463396643809ca1f6bef2755afc P 468a8fd46e824f62b9c2c0d3532a6d26 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:57.578362  9682 tablet_replica.cc:333] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:19:57.579424  9682 raft_consensus.cc:2243] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 LEADER]: Raft consensus shutting down.
W20260709 13:19:57.593909 10916 consensus_peers.cc:597] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:57.604174  9682 raft_consensus.cc:2272] T dde624b07ed446e5b7347791196396e2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:57.607091  9682 tablet_replica.cc:333] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:19:57.607926  9682 raft_consensus.cc:2243] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:19:57.609001  9682 raft_consensus.cc:2272] T 9094285c302f4de4aab986dc171c11ff P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:19:57.619699 10842 consensus_peers.cc:597] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:19:57.620652 10916 consensus_peers.cc:597] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:57.630244 10916 consensus_peers.cc:597] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:57.631146 10916 consensus_peers.cc:597] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:57.632967 10916 consensus_peers.cc:597] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:57.627466  9682 tablet_replica.cc:333] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:19:57.638716  9682 raft_consensus.cc:2243] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:19:57.640069  9682 raft_consensus.cc:2272] T 418b3711416f4a3e8fa80068035b5dd8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 13:19:57.642235 11478 consensus_peers.cc:597] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:57.652027  9682 tablet_replica.cc:333] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:19:57.653035  9682 raft_consensus.cc:2243] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:57.653805  9682 raft_consensus.cc:2272] T e54874d9d0dc4cb6a82081a61f03a309 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 13:19:57.662305 10916 consensus_peers.cc:597] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:57.662634  9682 tablet_replica.cc:333] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:19:57.667670  9682 raft_consensus.cc:2243] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:57.668433  9682 raft_consensus.cc:2272] T ad1b662471ff4b9ebda3845b4f224a34 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:57.671223  9682 tablet_replica.cc:333] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:19:57.672008  9682 raft_consensus.cc:2243] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:19:57.673125  9682 raft_consensus.cc:2272] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:57.676882  9682 tablet_replica.cc:333] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
W20260709 13:19:57.682406 10916 consensus_peers.cc:597] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:19:57.686316 11478 consensus_peers.cc:597] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:57.729652  9682 raft_consensus.cc:2243] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:57.730684  9682 raft_consensus.cc:2272] T 2b82f18992434ed5ba58b805107060d4 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 13:19:57.737115 10916 consensus_peers.cc:597] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:57.738880  9682 tablet_replica.cc:333] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:19:57.753906  9682 raft_consensus.cc:2243] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:19:57.755159  9682 raft_consensus.cc:2272] T 4dd4c0ff1def4960931dda3b52a4a884 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:57.758126  9682 tablet_replica.cc:333] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
W20260709 13:19:57.768262 10842 consensus_peers.cc:597] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:19:57.792018  9682 raft_consensus.cc:2243] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:19:57.793641  9682 raft_consensus.cc:2272] T b012e79b82794524a28ea3e94dee1bb2 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:57.796792  9682 tablet_replica.cc:333] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:19:57.798226  9682 raft_consensus.cc:2243] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:19:57.799474  9682 raft_consensus.cc:2272] T 8fd9828f212b47c19244738ee2a27a95 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:57.808040  9682 tablet_replica.cc:333] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:19:57.811024  9682 raft_consensus.cc:2243] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:57.812435  9682 raft_consensus.cc:2272] T 8cab289810294719ac9686e40290a6ab P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:57.817549  9682 tablet_replica.cc:333] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:19:57.818416  9682 raft_consensus.cc:2243] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:57.819551  9682 raft_consensus.cc:2272] T 7595d019b2e74196bc5838962b53798d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:57.822808  9682 tablet_replica.cc:333] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:19:57.823741  9682 raft_consensus.cc:2243] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:19:57.824837  9682 raft_consensus.cc:2272] T 225fa08b9f864105af0cd7a0e7a0d99e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:57.829049  9682 tablet_replica.cc:333] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:19:57.830494  9682 raft_consensus.cc:2243] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:57.831223  9682 raft_consensus.cc:2272] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:57.835310  9682 tablet_replica.cc:333] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:19:57.836117  9682 raft_consensus.cc:2243] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:57.839135  9682 raft_consensus.cc:2272] T b253bd741d094067902a02bc765ee0fb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:57.842327  9682 tablet_replica.cc:333] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:19:57.843212  9682 raft_consensus.cc:2243] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:57.843894  9682 raft_consensus.cc:2272] T 35190606a0c04bc4bac6c8ca7df14ed3 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:19:57.845664 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 52137.8us
I20260709 13:19:57.846659  9682 tablet_replica.cc:333] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:19:57.847633  9682 raft_consensus.cc:2243] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:57.848491  9682 raft_consensus.cc:2272] T 9354ad0d744d43bbbe2ce992ecbc4e65 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:57.851226  9682 tablet_replica.cc:333] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
W20260709 13:19:57.854550 10916 consensus_peers.cc:597] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:57.855505  9682 raft_consensus.cc:2243] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:19:57.857494  9682 raft_consensus.cc:2272] T 1d9e744bb20c4325969f4fbc01c672e8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:57.867558  9682 tablet_replica.cc:333] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:19:57.868440  9682 raft_consensus.cc:2243] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:57.869421  9682 raft_consensus.cc:2272] T 51ae8de979e44a2d8c155bffc20a1533 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:57.873457  9682 tablet_replica.cc:333] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:19:57.874279  9682 raft_consensus.cc:2243] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:19:57.898537  9682 raft_consensus.cc:2272] T e41482a95776477784d9bd9dacfed231 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:57.903633  9682 tablet_replica.cc:333] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:19:57.904501  9682 raft_consensus.cc:2243] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:57.905436  9682 raft_consensus.cc:2272] T 5944b8838e584a3890e4e8143f3f2665 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:57.908272  9682 tablet_replica.cc:333] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:19:57.909356  9682 raft_consensus.cc:2243] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:57.910122  9682 raft_consensus.cc:2272] T 414992c8073646798991d2b43fd61f1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:57.912935  9682 tablet_replica.cc:333] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:19:57.914500  9682 raft_consensus.cc:2243] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:19:57.915518  9682 raft_consensus.cc:2272] T bc6987a00cef4dff9545eee242db0982 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:57.918310  9682 tablet_replica.cc:333] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:19:57.919145  9682 raft_consensus.cc:2243] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:19:57.920538  9682 raft_consensus.cc:2272] T f6e90b642b0d4f06b866170dc0293dc1 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:57.923156  9682 tablet_replica.cc:333] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:19:57.923985  9682 raft_consensus.cc:2243] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:19:57.925164  9682 raft_consensus.cc:2272] T b761e5168d4e48aeaec47b99f5c7d3d0 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:57.927664  9682 tablet_replica.cc:333] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:19:57.928434  9682 raft_consensus.cc:2243] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:57.929025  9682 raft_consensus.cc:2272] T 32e87e18e6c9475085e47d6a2fd71a7b P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:57.931675  9682 tablet_replica.cc:333] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:19:57.932430  9682 raft_consensus.cc:2243] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:57.934363  9682 raft_consensus.cc:2272] T 651c4c65e7f44b1fb4053d3757ac417f P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:57.936905  9682 tablet_replica.cc:333] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:19:57.937824  9682 raft_consensus.cc:2243] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:57.939061  9682 raft_consensus.cc:2272] T 417caa22425d4a25aca2f7d8e9577256 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:57.941978  9682 tablet_replica.cc:333] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:19:57.942771  9682 raft_consensus.cc:2243] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:57.943495  9682 raft_consensus.cc:2272] T 90ad199246b34499aa32795a8082ffa8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:57.946034  9682 tablet_replica.cc:333] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:19:57.947021  9682 raft_consensus.cc:2243] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:19:57.948310  9682 raft_consensus.cc:2272] T d32f5e230e464e61b23a41962708d120 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:57.997035  9682 tablet_replica.cc:333] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:19:57.997915  9682 raft_consensus.cc:2243] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:19:57.999203  9682 raft_consensus.cc:2272] T ce92c99123994b928d7a775917eb22ed P 468a8fd46e824f62b9c2c0d3532a6d26 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:58.015308  9682 tablet_replica.cc:333] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:19:58.016222  9682 raft_consensus.cc:2243] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 28 FOLLOWER]: Raft consensus shutting down.
W20260709 13:19:58.016299 10916 consensus_peers.cc:597] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:19:58.024914  9682 raft_consensus.cc:2272] T 747220f0813a4e19a0b4b96bd98d0a2a P 468a8fd46e824f62b9c2c0d3532a6d26 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:58.046595  9682 tablet_replica.cc:333] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:19:58.047580  9682 raft_consensus.cc:2243] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:19:58.049386  9682 raft_consensus.cc:2272] T 90ea9e6b562e4600be1cb566739a6a30 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:58.095820  9682 tablet_replica.cc:333] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:19:58.096658  9682 raft_consensus.cc:2243] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:19:58.097913  9682 raft_consensus.cc:2272] T 121ef9771004451eb563dbfd77657e0e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:58.112182  9682 tablet_replica.cc:333] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:19:58.113011  9682 raft_consensus.cc:2243] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:58.113708  9682 raft_consensus.cc:2272] T f15293e178d84fe686fd66bc4439aba8 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:58.116694  9682 tablet_replica.cc:333] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:19:58.117532  9682 raft_consensus.cc:2243] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:19:58.118754  9682 raft_consensus.cc:2272] T 96558ab13797493e9c168b86fe3b62cd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:58.121618  9682 tablet_replica.cc:333] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:19:58.122435  9682 raft_consensus.cc:2243] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:19:58.123962  9682 raft_consensus.cc:2272] T d00373866c144f4981da73bd7ae39019 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:58.126969  9682 tablet_replica.cc:333] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:19:58.127799  9682 raft_consensus.cc:2243] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:58.128804  9682 raft_consensus.cc:2272] T a50b3bb87a48494a8e835471e55a6f74 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:58.131800  9682 tablet_replica.cc:333] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:19:58.141259  9682 raft_consensus.cc:2243] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:58.142284  9682 raft_consensus.cc:2272] T 76b2d93b69274dbca479bd87d7ea45cb P 468a8fd46e824f62b9c2c0d3532a6d26 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:58.145059  9682 tablet_replica.cc:333] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:19:58.145928  9682 raft_consensus.cc:2243] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:58.146658  9682 raft_consensus.cc:2272] T e9764b93d3a8407b8cc1ae641620c831 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:58.160215  9682 tablet_replica.cc:333] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:19:58.161047  9682 raft_consensus.cc:2243] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:58.161792  9682 raft_consensus.cc:2272] T 99ca4d7ec0974268ac5f35c5055da891 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:58.167644  9682 tablet_replica.cc:333] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:19:58.168648  9682 raft_consensus.cc:2243] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:19:58.170243  9682 raft_consensus.cc:2272] T fc429d4d563f415d90d6f74da422a740 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:19:58.171661 11478 consensus_peers.cc:597] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:58.173432  9682 tablet_replica.cc:333] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:19:58.174279  9682 raft_consensus.cc:2243] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:19:58.175419  9682 raft_consensus.cc:2272] T 8949b8b2d08942939f64373bccb71936 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:58.178017  9682 tablet_replica.cc:333] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:19:58.178764  9682 raft_consensus.cc:2243] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:19:58.179972  9682 raft_consensus.cc:2272] T 171abdebfc4142cda86de477a5c89a7e P 468a8fd46e824f62b9c2c0d3532a6d26 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:58.182868  9682 tablet_replica.cc:333] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:19:58.183665  9682 raft_consensus.cc:2243] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:58.184414  9682 raft_consensus.cc:2272] T 185480268a684958b8347a4f53af57c5 P 468a8fd46e824f62b9c2c0d3532a6d26 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:58.187173  9682 tablet_replica.cc:333] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:19:58.187898  9682 raft_consensus.cc:2243] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:58.188797  9682 raft_consensus.cc:2272] T b81df256f0c644cfae9fe2d336cbe9bd P 468a8fd46e824f62b9c2c0d3532a6d26 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:58.199982  9682 tablet_replica.cc:333] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26: stopping tablet replica
I20260709 13:19:58.208317  9682 raft_consensus.cc:2243] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 28 LEADER]: Raft consensus shutting down.
I20260709 13:19:58.209678  9682 raft_consensus.cc:2272] T ddb097c9ce1748458d6719bc91f56e1d P 468a8fd46e824f62b9c2c0d3532a6d26 [term 28 FOLLOWER]: Raft consensus is shut down!
W20260709 13:19:58.217609 10916 consensus_peers.cc:597] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:19:58.423060 10916 consensus_peers.cc:597] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:19:58.435978 10916 connection.cc:573] client connection to 127.9.116.130:37895 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260709 13:19:58.514967 10916 consensus_peers.cc:597] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:19:58.534183 10916 consensus_peers.cc:597] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:19:58.576247 10916 consensus_peers.cc:597] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:19:58.584204 11478 consensus_peers.cc:597] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:19:58.593302 10916 consensus_peers.cc:597] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:19:58.656261 19548 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:58.656953 19548 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:58.659773 19548 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:58.673966 11478 leader_election.cc:336] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:19:58.674923 11478 leader_election.cc:336] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:19:58.675473 11478 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:58.680634 19548 raft_consensus.cc:2749] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:19:58.682574 10916 consensus_peers.cc:597] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:19:58.740362 10916 consensus_peers.cc:597] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:19:58.791489 19530 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:58.792071 19530 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:58.794147 19530 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:58.817044 10916 leader_election.cc:336] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:19:58.817903 10916 leader_election.cc:336] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:19:58.818457 10916 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:58.819427 19530 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:19:58.858060 10897 debug-util.cc:398] Leaking SignalData structure 0x7b080039e840 after lost signal to thread 9685
W20260709 13:19:58.859328 10897 debug-util.cc:398] Leaking SignalData structure 0x7b080035c560 after lost signal to thread 10733
W20260709 13:19:58.860852 10897 debug-util.cc:398] Leaking SignalData structure 0x7b0800324dc0 after lost signal to thread 10974
W20260709 13:19:58.862107 10897 debug-util.cc:398] Leaking SignalData structure 0x7b0800364920 after lost signal to thread 11536
I20260709 13:19:58.871659 19530 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:58.872265 19530 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:58.874439 19530 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:58.882594 10916 leader_election.cc:336] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
W20260709 13:19:58.883608 10916 leader_election.cc:336] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:58.884743 10916 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:58.891243 19530 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:19:58.909381 10916 consensus_peers.cc:597] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:19:58.928807 19548 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:58.929476 19548 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:58.932217 19548 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:58.940651 11478 leader_election.cc:336] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:19:58.941437 11478 leader_election.cc:336] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:19:58.941951 11478 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:58.942885 19548 raft_consensus.cc:2749] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:19:58.951218 10916 consensus_peers.cc:597] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:19:59.019496 19548 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:59.020044 19548 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:59.022521 19548 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:59.024785 19530 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:59.025326 19530 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:59.027184 19530 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:59.032650 19543 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:59.033206 19543 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:59.035409 19543 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:59.037880 11478 leader_election.cc:336] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:59.038753 19548 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:59.039371 19548 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:59.042074 10916 leader_election.cc:336] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:19:59.043078 10916 leader_election.cc:336] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:19:59.043524 10916 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:59.044276 19530 raft_consensus.cc:2749] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:19:59.047302 11478 leader_election.cc:336] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:19:59.047838 11478 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:59.048713 19527 raft_consensus.cc:2749] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:59.042178 19548 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:59.051471 19548 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:59.052049 19548 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:59.054234 19548 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:59.056059 10916 leader_election.cc:336] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:19:59.056905 10916 leader_election.cc:336] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:19:59.059073 10916 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:59.061726 11478 leader_election.cc:336] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:59.062295 19530 raft_consensus.cc:2749] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:19:59.066421 11478 leader_election.cc:336] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:19:59.070246 11478 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:59.071204 19530 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:59.072228 19530 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:59.074064 19530 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:59.074891 19548 raft_consensus.cc:2749] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:59.076463 19530 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:59.077016 19530 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:59.078889 19530 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:59.080786 19548 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:59.081323 19548 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:59.085136 19548 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:19:59.085449 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
W20260709 13:19:59.093647 11478 leader_election.cc:336] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:19:59.094607 11478 leader_election.cc:336] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:19:59.095031 11478 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:59.099081 10916 leader_election.cc:336] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:19:59.100870 19548 raft_consensus.cc:2749] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:19:59.105044 11478 leader_election.cc:336] T d32f5e230e464e61b23a41962708d120 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:19:59.105482 11478 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:19:59.108083 19548 raft_consensus.cc:2749] T d32f5e230e464e61b23a41962708d120 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:19:59.113615 10916 leader_election.cc:336] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
W20260709 13:19:59.118363 10916 leader_election.cc:336] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:59.118875 10916 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:59.119616 19530 raft_consensus.cc:2749] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:19:59.120639 10916 leader_election.cc:336] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:59.121088 10916 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:59.121798 19530 raft_consensus.cc:2749] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:19:59.217028 19530 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:59.217573 19530 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:59.219583 19530 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:59.220253 19527 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:59.220471 19543 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:59.220834 19527 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:59.220935 19543 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:59.222707 19527 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:59.222762 19543 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:59.226069 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:59.226755 11506 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 23.
I20260709 13:19:59.229056 10915 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:59.230074 19530 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:59.230415 19530 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:59.230722 19530 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:19:59.230649 11478 leader_election.cc:336] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
W20260709 13:19:59.232379 11478 leader_election.cc:336] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:59.232833 11478 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:59.233402 10916 leader_election.cc:336] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:19:59.234464 19527 raft_consensus.cc:2749] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:59.234928 19543 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:59.235430 19543 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:59.237423 19543 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:59.246311 19530 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:59.248499 19530 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 election: Requested vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
W20260709 13:19:59.249729 10916 leader_election.cc:336] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:59.250270 10916 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:59.250823 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d32f5e230e464e61b23a41962708d120" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586"
I20260709 13:19:59.251434 11506 raft_consensus.cc:3060] T d32f5e230e464e61b23a41962708d120 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:59.256098 19530 raft_consensus.cc:2749] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:59.262959 19527 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:59.263489 19527 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:59.264007 11506 raft_consensus.cc:2468] T d32f5e230e464e61b23a41962708d120 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 24.
I20260709 13:19:59.265429 19527 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:59.266333 10915 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 
I20260709 13:19:59.267254 19530 raft_consensus.cc:2804] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:19:59.267707 19530 raft_consensus.cc:697] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 24 LEADER]: Becoming Leader. State: Replica: cb0428cc55984daba1ba63744a5102a2, State: Running, Role: LEADER
W20260709 13:19:59.270005 11478 leader_election.cc:336] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:59.271203 19530 consensus_queue.cc:237] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:19:59.271834 11478 leader_election.cc:336] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:19:59.272320 11478 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:59.273052 19527 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:19:59.275210 10916 leader_election.cc:336] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:19:59.290838 19543 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:19:59.291499 10916 leader_election.cc:336] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:59.291431 19543 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:19:59.292351 10916 consensus_peers.cc:597] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:19:59.293026 10916 leader_election.cc:336] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:19:59.293448 10916 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:59.293831 19543 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:59.294346 19553 raft_consensus.cc:2749] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:19:59.294440 10916 leader_election.cc:336] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:19:59.311741 10693 catalog_manager.cc:5697] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 reported cstate change: term changed from 23 to 24, leader changed from 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130) to cb0428cc55984daba1ba63744a5102a2 (127.9.116.131). New cstate: current_term: 24 leader_uuid: "cb0428cc55984daba1ba63744a5102a2" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:19:59.322959 10916 consensus_peers.cc:597] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:19:59.323589 10916 leader_election.cc:336] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:19:59.324625 10916 leader_election.cc:336] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:19:59.325090 10916 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:59.325960 19530 raft_consensus.cc:2749] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 13:19:59.369994 11478 consensus_peers.cc:597] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:19:59.377637 10916 consensus_peers.cc:597] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:19:59.434473 19527 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:59.435066 19527 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:59.439101 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:19:59.450892 19527 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:19:59.452171 11478 leader_election.cc:336] T b761e5168d4e48aeaec47b99f5c7d3d0 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:59.454362 11478 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:19:59.454847  9682 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 10897)
W20260709 13:19:59.456122 11478 leader_election.cc:336] T b761e5168d4e48aeaec47b99f5c7d3d0 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:19:59.456594 19527 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:19:59.461436 11478 consensus_peers.cc:597] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:19:59.477914 19553 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:59.478569 19553 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:59.481387 19553 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:19:59.482399 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:19:59.483144 11506 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 21.
I20260709 13:19:59.484038 19553 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:59.484695 19553 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:59.487497 19553 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:59.496910 19553 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:59.497692 19553 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:59.501919 19553 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:59.505314 11478 consensus_peers.cc:597] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:19:59.507531 19527 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:59.508147 19527 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:19:59.510179 19527 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:59.511565 19543 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:59.513042 19543 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:59.509543 19553 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:19:59.515231 19553 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:59.516366 19543 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:19:59.523149 19553 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:59.521371 10916 leader_election.cc:336] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:19:59.534389 10916 leader_election.cc:336] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
W20260709 13:19:59.519145 11478 leader_election.cc:336] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:19:59.536511 11478 leader_election.cc:336] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:19:59.536952 11478 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:59.537561 19527 raft_consensus.cc:2749] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:59.539757 10916 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:59.541571 19543 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:19:59.541877 10916 leader_election.cc:336] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:19:59.544534 19543 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:19:59.544678 10916 leader_election.cc:336] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:59.545133 10916 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:59.545082 19543 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:19:59.546933 19553 raft_consensus.cc:2749] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:59.548859 19543 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:19:59.589200 10916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:39685, user_credentials={real_user=slave}} blocked reactor thread for 44343us
W20260709 13:19:59.591560 10916 leader_election.cc:336] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:19:59.604202 10916 leader_election.cc:336] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:19:59.609989 10916 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:59.613004 19543 raft_consensus.cc:2749] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:19:59.626848 10916 leader_election.cc:336] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
W20260709 13:19:59.647616 10916 leader_election.cc:336] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:19:59.648888 10916 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:59.660923 19553 raft_consensus.cc:2749] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:19:59.661324 10916 leader_election.cc:336] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:19:59.674997 10916 leader_election.cc:336] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:19:59.683456 10916 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:59.685304 19560 raft_consensus.cc:2749] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:19:59.689678 10916 leader_election.cc:336] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
W20260709 13:19:59.707151 10916 leader_election.cc:336] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:19:59.708123 11478 consensus_peers.cc:597] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:19:59.707736 10916 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:19:59.710124 19560 raft_consensus.cc:2749] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:19:59.716128 10916 consensus_peers.cc:597] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:19:59.751219 11478 consensus_peers.cc:597] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:19:59.765841 10916 consensus_peers.cc:597] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:19:59.792371 10916 consensus_peers.cc:597] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:19:59.809018 10916 consensus_peers.cc:597] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:19:59.816886 10916 consensus_peers.cc:597] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:59.857462 11505 raft_consensus.cc:1275] T d32f5e230e464e61b23a41962708d120 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Refusing update from remote peer cb0428cc55984daba1ba63744a5102a2: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 13:19:59.859529 19560 consensus_queue.cc:1048] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 13:19:59.954150 10916 consensus_peers.cc:597] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:19:59.979851 10916 consensus_peers.cc:597] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:00.009125 10916 consensus_peers.cc:597] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:00.027912 11478 consensus_peers.cc:597] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:00.030121 11478 consensus_peers.cc:597] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:00.059507 10916 consensus_peers.cc:597] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:00.082248 11478 consensus_peers.cc:597] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:00.088017 10916 consensus_peers.cc:597] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:00.107592 11478 consensus_peers.cc:597] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:00.113696 11478 consensus_peers.cc:597] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:00.119534 10916 consensus_peers.cc:597] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:00.127419 10916 consensus_peers.cc:597] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:00.133366 10916 consensus_peers.cc:597] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:00.141661 10916 consensus_peers.cc:597] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:00.172246 10916 consensus_peers.cc:597] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:00.173430 11478 consensus_peers.cc:597] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:00.176306 11478 consensus_peers.cc:597] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:00.205544 10916 consensus_peers.cc:597] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:00.208909 10916 consensus_peers.cc:597] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:00.288595 11478 consensus_peers.cc:597] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:00.303874 10916 consensus_peers.cc:597] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:00.327926 10916 consensus_peers.cc:597] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:00.361083 10916 consensus_peers.cc:597] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:00.404170 11478 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111) [suppressed 568 similar messages]
I20260709 13:20:00.407330 19561 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:00.408380 19561 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:20:00.411384 11478 consensus_peers.cc:597] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:00.410684 19561 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:20:00.416656 11478 consensus_peers.cc:597] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:00.419557 11478 leader_election.cc:336] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:20:00.425493 11478 leader_election.cc:336] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:00.426052 11478 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:00.426919 19561 raft_consensus.cc:2749] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:20:00.456641  9682 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 10897)
W20260709 13:20:00.456943 10916 consensus_peers.cc:597] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:00.509784 10916 consensus_peers.cc:597] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:20:00.541105 19560 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:00.541836 19560 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:20:00.544219 19560 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:20:00.554296 10916 leader_election.cc:336] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:20:00.555120 10916 leader_election.cc:336] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:00.555601 10916 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:00.557139 19553 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:00.684573 19561 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:00.685406 19561 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:20:00.698581 19561 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:20:00.704407 11478 leader_election.cc:336] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:20:00.710237 11478 leader_election.cc:336] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:00.710690 11478 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:00.711766 19561 raft_consensus.cc:2749] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:00.724290 19560 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:00.725069 19560 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:20:00.727517 19560 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:20:00.773948 10916 leader_election.cc:336] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:00.779698 19560 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:00.780802 19560 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:20:00.783177 19560 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:20:00.784251 10916 leader_election.cc:336] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:20:00.784793 10916 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:00.785594 19560 raft_consensus.cc:2749] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:20:00.787047 10916 consensus_peers.cc:597] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:00.808530 10916 consensus_peers.cc:597] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:00.846686 10916 consensus_peers.cc:597] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:00.847357 10916 leader_election.cc:336] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:20:00.862424 19548 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:00.863214 19548 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:20:00.867347 19548 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:20:00.874567 11478 leader_election.cc:336] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:20:00.875355 11478 leader_election.cc:336] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:00.875851 11478 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:00.876574 19548 raft_consensus.cc:2749] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:20:00.877377 10916 leader_election.cc:336] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:00.877933 10916 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:00.880138 19560 raft_consensus.cc:2749] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:00.882015 19560 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:00.882689 19560 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:20:00.884788 19560 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:20:00.886210 19553 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:00.886881 19553 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:20:00.889040 19553 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:20:00.901800 19548 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:00.902679 19548 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:20:00.905069 19548 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:20:00.920385 11478 leader_election.cc:336] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:20:00.921204 11478 leader_election.cc:336] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:00.921597 11478 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:20:00.925681 10916 leader_election.cc:336] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:00.927310 19561 raft_consensus.cc:2749] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:20:00.928669 10916 leader_election.cc:336] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
W20260709 13:20:00.936856 10916 leader_election.cc:336] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:20:00.945554 10916 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:00.946432 19560 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:20:00.953758 10916 leader_election.cc:336] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:20:00.954910 10916 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:00.955754 19560 raft_consensus.cc:2749] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:20:00.967231 19548 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:00.968022 19548 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:20:00.970613 19548 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:20:00.977279 19560 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:00.978030 19560 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:20:00.978775 11478 leader_election.cc:336] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:20:00.979470 11478 leader_election.cc:336] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:00.979861 11478 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:00.980073 19560 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:20:00.980763 19548 raft_consensus.cc:2749] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:20:01.013688 10916 consensus_peers.cc:597] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:01.015494 10916 consensus_peers.cc:597] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:01.035642 10916 leader_election.cc:336] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:20:01.041050 10916 leader_election.cc:336] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:01.041657 10916 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:01.042562 19553 raft_consensus.cc:2749] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:01.120610 19553 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:01.121273 19553 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:20:01.123525 19553 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:20:01.148180 19553 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:01.148950 19553 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:20:01.151769 19553 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:20:01.153263 11478 consensus_peers.cc:597] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:01.153887 10916 leader_election.cc:336] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:20:01.156172 10916 leader_election.cc:336] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:01.156698 10916 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:01.169842 19553 raft_consensus.cc:2749] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:20:01.173032 19560 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:01.173735 19560 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:20:01.176586 19560 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:20:01.186048 10916 leader_election.cc:336] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:20:01.187392 10916 leader_election.cc:336] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:01.187861 10916 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:01.188714 19560 raft_consensus.cc:2749] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:20:01.191788 10916 leader_election.cc:336] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:20:01.192488 10916 consensus_peers.cc:597] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:01.193351 10916 leader_election.cc:336] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:01.193943 10916 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:01.196470 19560 raft_consensus.cc:2749] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 13:20:01.265332 10916 consensus_peers.cc:597] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:20:01.281785 19567 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:01.282624 19567 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:20:01.284982 19567 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:20:01.293553 11478 leader_election.cc:336] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:20:01.294860 11478 leader_election.cc:336] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:01.295351 11478 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:01.296164 19567 raft_consensus.cc:2749] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:01.297943 19567 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:01.298619 19567 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:20:01.300918 19567 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:20:01.311376 11478 leader_election.cc:336] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:20:01.312563 11478 leader_election.cc:336] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:01.321852 11478 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:20:01.325979 10916 consensus_peers.cc:597] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:20:01.327494 19567 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:20:01.332279 10916 consensus_peers.cc:597] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:20:01.376432 19568 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:01.377151 19568 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:20:01.379530 19568 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:20:01.390946 10916 leader_election.cc:336] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:20:01.391736 10916 leader_election.cc:336] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:01.392187 10916 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:01.392973 19568 raft_consensus.cc:2749] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:01.438241 19568 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:01.438982 19568 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:20:01.441380 19568 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:20:01.451470 10916 leader_election.cc:336] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:20:01.455210 10916 leader_election.cc:336] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:01.455744 10916 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:01.456504 19568 raft_consensus.cc:2749] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:20:01.457213  9682 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 10897)
I20260709 13:20:01.586874 19567 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:01.587760 19567 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:20:01.590039 19567 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:20:01.602591 11478 leader_election.cc:336] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
W20260709 13:20:01.604040 11478 leader_election.cc:336] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:20:01.604987 11478 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:01.607190 19561 raft_consensus.cc:2749] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:20:01.653941 10916 consensus_peers.cc:597] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:01.668968 10916 consensus_peers.cc:597] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:20:01.684202 19569 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:01.685042 19569 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:20:01.690366 11506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "cb0428cc55984daba1ba63744a5102a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "12682eb70bc84c76879e04a8628ee586" is_pre_election: true
I20260709 13:20:01.691272 11506 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0428cc55984daba1ba63744a5102a2 in term 21.
I20260709 13:20:01.689654 19569 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), 12682eb70bc84c76879e04a8628ee586 (127.9.116.132:40555)
I20260709 13:20:01.704058 19569 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:01.704799 19569 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:20:01.705914 10916 leader_election.cc:336] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:20:01.706768 10916 leader_election.cc:336] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:01.707175 19569 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:20:01.707194 10916 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:01.708683 19553 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:20:01.719605 10916 leader_election.cc:336] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:20:01.724006 19569 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
W20260709 13:20:01.724664 10916 leader_election.cc:336] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:01.724758 19569 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:20:01.728652 19569 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:20:01.727074 10916 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:01.739557 19553 raft_consensus.cc:2749] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:20:01.743216 10916 consensus_peers.cc:597] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:01.761052 10916 leader_election.cc:336] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:20:01.761901 10916 leader_election.cc:336] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:01.762326 10916 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:01.763154 19553 raft_consensus.cc:2749] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:20:01.781497 10916 consensus_peers.cc:597] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:01.824618 11478 consensus_peers.cc:597] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:20:01.863143 19553 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:01.863811 19553 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:20:01.866181 19553 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:20:01.872768 10916 leader_election.cc:336] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:20:01.875463 10916 leader_election.cc:336] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:01.876909 10916 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:01.877735 19553 raft_consensus.cc:2749] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:20:01.896319 11478 consensus_peers.cc:597] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:01.950518 11478 consensus_peers.cc:597] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:02.093899 11478 consensus_peers.cc:597] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:02.108026 11478 consensus_peers.cc:597] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:02.110101 10916 consensus_peers.cc:597] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:02.169858 11478 consensus_peers.cc:597] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:02.183493 19548 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:02.184242 19548 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:20:02.186841 19548 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:20:02.208091 11478 leader_election.cc:336] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
W20260709 13:20:02.215179 11478 leader_election.cc:336] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:20:02.218741 11478 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:02.224702 19548 raft_consensus.cc:2749] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:20:02.318238 10916 consensus_peers.cc:597] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:02.324278 10916 consensus_peers.cc:597] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:02.325217 10916 consensus_peers.cc:597] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:02.345973 10916 consensus_peers.cc:597] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:02.358173 10916 consensus_peers.cc:597] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:02.387755 11478 consensus_peers.cc:597] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:02.437305 11478 consensus_peers.cc:597] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:02.467988 10916 consensus_peers.cc:597] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:02.490619 10916 consensus_peers.cc:597] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:02.506322 11478 consensus_peers.cc:597] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:20:02.525072 11478 consensus_peers.cc:597] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:02.564872 10916 consensus_peers.cc:597] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:02.603117 19548 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:02.603775 19548 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:20:02.605991 19548 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:20:02.613370 11478 leader_election.cc:336] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:20:02.614198 11478 leader_election.cc:336] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:02.614674 11478 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:02.615505 19548 raft_consensus.cc:2749] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:20:02.629650 10916 consensus_peers.cc:597] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:02.671996 10916 consensus_peers.cc:597] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:02.700529 10916 consensus_peers.cc:597] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:02.757906 19569 raft_consensus.cc:493] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:02.758486 19569 raft_consensus.cc:515] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:20:02.760741 19569 leader_election.cc:290] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:20:02.762343 10916 consensus_peers.cc:597] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:02.774053 11478 consensus_peers.cc:597] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:02.776948 10916 consensus_peers.cc:597] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:02.782760 10916 consensus_peers.cc:597] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:02.785835 11478 consensus_peers.cc:597] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:02.791340 10916 leader_election.cc:336] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:20:02.797032 10916 leader_election.cc:336] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:02.799561 10916 leader_election.cc:304] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:02.802995 19553 raft_consensus.cc:2749] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:20:02.811915 11478 consensus_peers.cc:597] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:02.825124 10916 consensus_peers.cc:597] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:02.841830 11478 consensus_peers.cc:597] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:02.843993 10916 consensus_peers.cc:597] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:02.869866 10916 consensus_peers.cc:597] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:20:02.875247 19553 raft_consensus.cc:493] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:02.875813 19553 raft_consensus.cc:515] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:20:02.877784 19553 leader_election.cc:290] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:20:03.073840 19569 raft_consensus.cc:493] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:03.074373 19569 raft_consensus.cc:515] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:20:03.076220 19569 leader_election.cc:290] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:20:03.102870 19571 raft_consensus.cc:493] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:03.103484 19571 raft_consensus.cc:515] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:20:03.105562 19571 leader_election.cc:290] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:20:03.111744 11478 leader_election.cc:336] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:20:03.112658 11478 leader_election.cc:336] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:03.113102 11478 leader_election.cc:304] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:03.114055 19571 raft_consensus.cc:2749] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:20:03.129576 10916 consensus_peers.cc:597] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:03.131666 11477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:45553, user_credentials={real_user=slave}} blocked reactor thread for 52874.7us
W20260709 13:20:03.164075 10916 leader_election.cc:336] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:20:03.181525 10916 leader_election.cc:336] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:03.182075 10916 leader_election.cc:304] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:03.182920 19569 raft_consensus.cc:2749] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:20:03.205024 11478 consensus_peers.cc:597] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:03.207635 19575 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:03.208280 19575 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:20:03.210494 19575 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:20:03.215139 10916 leader_election.cc:336] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:20:03.217360 11478 leader_election.cc:336] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:20:03.219319 11478 leader_election.cc:336] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:03.219790 11478 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:03.220554 19575 raft_consensus.cc:2749] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:20:03.232205 10916 leader_election.cc:336] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:03.233307 10916 leader_election.cc:304] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
W20260709 13:20:03.234432 10916 consensus_peers.cc:597] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:03.234531 19553 raft_consensus.cc:2749] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:20:03.235002 10916 consensus_peers.cc:597] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:03.247208 10916 consensus_peers.cc:597] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:03.251073 10916 consensus_peers.cc:597] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:03.264850 10916 consensus_peers.cc:597] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:20:03.329447 19569 raft_consensus.cc:493] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:03.329983 19569 raft_consensus.cc:515] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:20:03.332124 19569 leader_election.cc:290] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:20:03.337406 10916 consensus_peers.cc:597] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:03.337944 10916 leader_election.cc:336] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:20:03.340519 10916 leader_election.cc:336] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:03.340998 10916 leader_election.cc:304] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:03.341776 19569 raft_consensus.cc:2749] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:03.375358 19569 raft_consensus.cc:493] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:03.376005 19569 raft_consensus.cc:515] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:20:03.378412 19569 leader_election.cc:290] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:20:03.386296 10916 leader_election.cc:336] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:20:03.388861 10916 leader_election.cc:336] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:03.390465 10916 leader_election.cc:304] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:03.391283 19569 raft_consensus.cc:2749] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:20:03.526247 10916 consensus_peers.cc:597] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:20:03.537318 11478 consensus_peers.cc:597] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:20:03.587031 19572 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:03.587738 19572 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:20:03.593573 10943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b761e5168d4e48aeaec47b99f5c7d3d0" candidate_uuid: "12682eb70bc84c76879e04a8628ee586" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cb0428cc55984daba1ba63744a5102a2" is_pre_election: true
I20260709 13:20:03.594244 10943 raft_consensus.cc:2468] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12682eb70bc84c76879e04a8628ee586 in term 21.
W20260709 13:20:03.602116 11478 leader_election.cc:336] T b761e5168d4e48aeaec47b99f5c7d3d0 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:20:03.602622 19572 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:20:03.605378 19569 raft_consensus.cc:493] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:03.605986 19569 raft_consensus.cc:515] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:20:03.606483 11478 leader_election.cc:336] T b761e5168d4e48aeaec47b99f5c7d3d0 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:03.607031 11478 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586, cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:03.608335 19569 leader_election.cc:290] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:20:03.608371 19572 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:20:03.615393 10916 leader_election.cc:336] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:20:03.617120 10916 consensus_peers.cc:597] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:03.617810 10916 leader_election.cc:336] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:03.618229 10916 leader_election.cc:304] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:03.618996 19569 raft_consensus.cc:2749] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:03.702737 19572 raft_consensus.cc:493] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:03.703341 19572 raft_consensus.cc:515] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:20:03.705524 19572 leader_election.cc:290] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:20:03.712786 11478 leader_election.cc:336] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:20:03.713549 19569 raft_consensus.cc:493] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:03.714110 19569 raft_consensus.cc:515] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:20:03.716194 19569 leader_election.cc:290] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:20:03.718320 11478 leader_election.cc:336] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:03.718813 11478 leader_election.cc:304] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:03.720116 19575 raft_consensus.cc:2749] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:20:03.722915 10916 leader_election.cc:336] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:20:03.724772 10916 leader_election.cc:336] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:03.725196 10916 leader_election.cc:304] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:03.725919 19553 raft_consensus.cc:2749] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:03.764391 19553 raft_consensus.cc:493] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:03.764876 19553 raft_consensus.cc:515] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:20:03.766819 19553 leader_election.cc:290] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:20:03.768599 10916 consensus_peers.cc:597] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:03.785383 10916 leader_election.cc:336] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:20:03.787813 10916 leader_election.cc:336] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:03.788365 10916 leader_election.cc:304] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:03.789146 19553 raft_consensus.cc:2749] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:20:03.829257 19553 raft_consensus.cc:493] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:03.829806 19553 raft_consensus.cc:515] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:20:03.831923 19553 leader_election.cc:290] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:20:03.839116 10916 leader_election.cc:336] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:20:03.839996 10916 leader_election.cc:336] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:03.840479 10916 leader_election.cc:304] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:03.841228 19553 raft_consensus.cc:2749] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:03.843209 19553 raft_consensus.cc:493] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:03.843727 19553 raft_consensus.cc:515] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:20:03.845898 19553 leader_election.cc:290] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:20:03.853581 10916 leader_election.cc:336] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:20:03.854615 10916 leader_election.cc:336] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:03.855089 10916 leader_election.cc:304] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:03.855764 19553 raft_consensus.cc:2749] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:20:03.871433 10916 consensus_peers.cc:597] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:03.907538 10916 consensus_peers.cc:597] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:20:03.941088  9682 tablet_server.cc:196] TabletServer@127.9.116.130:0 shutdown complete.
I20260709 13:20:03.957548 19572 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:03.958364 19572 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:20:03.959528 10916 consensus_peers.cc:597] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:20:03.961117 19572 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:20:03.966841 19553 raft_consensus.cc:493] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:03.967391 19553 raft_consensus.cc:515] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
W20260709 13:20:03.968545 10730 debug-util.cc:398] Leaking SignalData structure 0x7b0800399720 after lost signal to thread 9685
I20260709 13:20:03.969341 19553 leader_election.cc:290] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:20:03.969697 10730 debug-util.cc:398] Leaking SignalData structure 0x7b0800038c20 after lost signal to thread 10733
W20260709 13:20:03.971318 10730 debug-util.cc:398] Leaking SignalData structure 0x7b0800355a60 after lost signal to thread 10974
W20260709 13:20:03.973278 10730 debug-util.cc:398] Leaking SignalData structure 0x7b0800334400 after lost signal to thread 11536
W20260709 13:20:03.968427 11478 leader_election.cc:336] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:20:03.977767 10916 leader_election.cc:336] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:20:03.979483 10916 leader_election.cc:336] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:03.979888 10916 leader_election.cc:304] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:03.980556 19553 raft_consensus.cc:2749] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:20:03.982374 11478 leader_election.cc:336] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:03.982923 11478 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:03.986128 19572 raft_consensus.cc:2749] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:20:04.004985 10916 consensus_peers.cc:597] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:20:04.100596 19572 raft_consensus.cc:493] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:04.101142 19572 raft_consensus.cc:515] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:20:04.108423 19572 leader_election.cc:290] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:20:04.117280 11478 leader_election.cc:336] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:20:04.124312 11478 leader_election.cc:336] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:04.125065 11478 leader_election.cc:304] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:04.125866 19572 raft_consensus.cc:2749] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:04.187036 19572 raft_consensus.cc:493] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:04.187661 19572 raft_consensus.cc:515] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:20:04.190089 19572 leader_election.cc:290] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:20:04.202395 11478 leader_election.cc:336] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:20:04.202911 10686 catalog_manager.cc:4755] TS e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet d32f5e230e464e61b23a41962708d120: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:20:04.203289 11478 leader_election.cc:336] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:04.203828 11478 leader_election.cc:304] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:04.205238 19572 raft_consensus.cc:2749] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:20:04.281538 10916 consensus_peers.cc:597] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:04.282140 11478 consensus_peers.cc:597] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:20:04.287459 19572 raft_consensus.cc:493] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:04.288141 19572 raft_consensus.cc:515] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:20:04.301344 19572 leader_election.cc:290] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:20:04.330662 11478 leader_election.cc:336] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
W20260709 13:20:04.335978 11478 leader_election.cc:336] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:20:04.340169 11478 leader_election.cc:304] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:04.341043 19575 raft_consensus.cc:2749] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:20:04.345542 10916 consensus_peers.cc:597] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:04.366111 11478 consensus_peers.cc:597] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:20:04.373580  9682 tablet_server.cc:179] TabletServer@127.9.116.131:0 shutting down...
W20260709 13:20:04.455749 11477 consensus_peers.cc:597] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 -> Peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Couldn't send request to peer cb0428cc55984daba1ba63744a5102a2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:04.471485 11477 consensus_peers.cc:597] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 -> Peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Couldn't send request to peer cb0428cc55984daba1ba63744a5102a2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:04.473430 11477 consensus_peers.cc:597] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 -> Peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Couldn't send request to peer cb0428cc55984daba1ba63744a5102a2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:04.485159 11478 consensus_peers.cc:597] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:20:04.510551 19553 raft_consensus.cc:493] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:04.511152 19553 raft_consensus.cc:515] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:20:04.513392 19553 leader_election.cc:290] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:20:04.522244 19553 raft_consensus.cc:493] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:04.522847 19553 raft_consensus.cc:515] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:20:04.523675  9682 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:20:04.524847 19553 leader_election.cc:290] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:20:04.525957  9682 tablet_replica.cc:333] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:20:04.527791  9682 raft_consensus.cc:2243] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:04.530895  9682 raft_consensus.cc:2272] T 19c20efd6c04443596bb712e3c93c650 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:04.534220  9682 tablet_replica.cc:333] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:20:04.539394  9682 raft_consensus.cc:2243] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Raft consensus shutting down.
W20260709 13:20:04.542965 10916 leader_election.cc:336] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:20:04.548328 11477 consensus_peers.cc:597] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 -> Peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Couldn't send request to peer cb0428cc55984daba1ba63744a5102a2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:04.549018 10916 leader_election.cc:336] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:04.549465 10916 leader_election.cc:304] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:04.550231 19569 raft_consensus.cc:2749] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:04.552114  9682 raft_consensus.cc:2272] T 9094285c302f4de4aab986dc171c11ff P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:04.564020 10916 leader_election.cc:336] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
W20260709 13:20:04.564755 10916 leader_election.cc:336] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:20:04.565150 10916 leader_election.cc:304] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:04.566087  9682 tablet_replica.cc:333] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:20:04.565965 19569 raft_consensus.cc:2749] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:04.567224  9682 raft_consensus.cc:2243] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 32 LEADER]: Raft consensus shutting down.
W20260709 13:20:04.567416 10916 consensus_peers.cc:597] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:20:04.569322  9682 raft_consensus.cc:2272] T 2778f463396643809ca1f6bef2755afc P cb0428cc55984daba1ba63744a5102a2 [term 32 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:04.575508 11478 consensus_peers.cc:597] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:20:04.585330  9682 tablet_replica.cc:333] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:20:04.588637  9682 raft_consensus.cc:2243] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:20:04.592697  9682 raft_consensus.cc:2272] T e54874d9d0dc4cb6a82081a61f03a309 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:04.596071  9682 tablet_replica.cc:333] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:20:04.597039  9682 raft_consensus.cc:2243] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:04.598623  9682 raft_consensus.cc:2272] T ad1b662471ff4b9ebda3845b4f224a34 P cb0428cc55984daba1ba63744a5102a2 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:04.601557  9682 tablet_replica.cc:333] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:20:04.602608  9682 raft_consensus.cc:2243] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:04.603808  9682 raft_consensus.cc:2272] T 418b3711416f4a3e8fa80068035b5dd8 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:04.606760  9682 tablet_replica.cc:333] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:20:04.607816  9682 raft_consensus.cc:2243] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:20:04.609741 19569 raft_consensus.cc:493] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:04.610236 19569 raft_consensus.cc:515] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } }
I20260709 13:20:04.612349 19569 leader_election.cc:290] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:20:04.613186  9682 raft_consensus.cc:2272] T 2b82f18992434ed5ba58b805107060d4 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:04.616163  9682 tablet_replica.cc:333] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:20:04.617115  9682 raft_consensus.cc:2243] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:04.618403  9682 raft_consensus.cc:2272] T 416a6d12184f4fbda69570b95be47851 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:04.620740 10916 leader_election.cc:336] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:20:04.621377  9682 tablet_replica.cc:333] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
W20260709 13:20:04.621548 10916 leader_election.cc:336] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:04.622046 10916 leader_election.cc:304] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb0428cc55984daba1ba63744a5102a2; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:04.622857 19569 raft_consensus.cc:2749] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:04.622622  9682 raft_consensus.cc:2243] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:20:04.626442  9682 raft_consensus.cc:2272] T 8cab289810294719ac9686e40290a6ab P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:04.640707 11477 consensus_peers.cc:597] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 -> Peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Couldn't send request to peer cb0428cc55984daba1ba63744a5102a2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:04.641284  9682 tablet_replica.cc:333] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:20:04.642292  9682 raft_consensus.cc:2243] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:04.643074  9682 raft_consensus.cc:2272] T 7595d019b2e74196bc5838962b53798d P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:04.649086 11478 consensus_peers.cc:597] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:20:04.650604  9682 tablet_replica.cc:333] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:20:04.651858  9682 raft_consensus.cc:2243] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:20:04.657039  9682 raft_consensus.cc:2272] T b4b13050d714473bb01ea0174d634a14 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:04.660547  9682 tablet_replica.cc:333] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:20:04.661450  9682 raft_consensus.cc:2243] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:04.662503  9682 raft_consensus.cc:2272] T 225fa08b9f864105af0cd7a0e7a0d99e P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:04.665652  9682 tablet_replica.cc:333] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:20:04.666527  9682 raft_consensus.cc:2243] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:20:04.667923  9682 raft_consensus.cc:2272] T 35190606a0c04bc4bac6c8ca7df14ed3 P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:04.675809 11477 consensus_peers.cc:597] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 -> Peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Couldn't send request to peer cb0428cc55984daba1ba63744a5102a2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:04.683045  9682 tablet_replica.cc:333] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:20:04.686087  9682 raft_consensus.cc:2243] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:20:04.687744  9682 raft_consensus.cc:2272] T 9354ad0d744d43bbbe2ce992ecbc4e65 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:04.695583 11478 consensus_peers.cc:597] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:20:04.697880  9682 tablet_replica.cc:333] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:20:04.701198  9682 raft_consensus.cc:2243] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Raft consensus shutting down.
W20260709 13:20:04.709127 11477 consensus_peers.cc:597] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 -> Peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Couldn't send request to peer cb0428cc55984daba1ba63744a5102a2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:04.711908  9682 raft_consensus.cc:2272] T 51ae8de979e44a2d8c155bffc20a1533 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:04.716208  9682 tablet_replica.cc:333] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:20:04.722613  9682 raft_consensus.cc:2243] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:20:04.724224  9682 raft_consensus.cc:2272] T 90ad199246b34499aa32795a8082ffa8 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:04.729135  9682 tablet_replica.cc:333] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:20:04.732393  9682 raft_consensus.cc:2243] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:04.733734  9682 raft_consensus.cc:2272] T 37a1fc8ca1b34260908d185a9ebc2bb3 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:04.737927  9682 tablet_replica.cc:333] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:20:04.741607  9682 raft_consensus.cc:2243] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:04.743827  9682 raft_consensus.cc:2272] T 8fd9828f212b47c19244738ee2a27a95 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:04.752605  9682 tablet_replica.cc:333] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:20:04.753530  9682 raft_consensus.cc:2243] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:04.754662  9682 raft_consensus.cc:2272] T b012e79b82794524a28ea3e94dee1bb2 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:04.757541  9682 tablet_replica.cc:333] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:20:04.763343  9682 raft_consensus.cc:2243] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:04.764165  9682 raft_consensus.cc:2272] T 5944b8838e584a3890e4e8143f3f2665 P cb0428cc55984daba1ba63744a5102a2 [term 29 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:04.774856 11477 consensus_peers.cc:597] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 -> Peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Couldn't send request to peer cb0428cc55984daba1ba63744a5102a2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:04.779511  9682 tablet_replica.cc:333] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:20:04.780572  9682 raft_consensus.cc:2243] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:04.781765  9682 raft_consensus.cc:2272] T b253bd741d094067902a02bc765ee0fb P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:04.794766 11477 consensus_peers.cc:597] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 -> Peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Couldn't send request to peer cb0428cc55984daba1ba63744a5102a2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:04.796088  9682 tablet_replica.cc:333] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:20:04.806052  9682 raft_consensus.cc:2243] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:20:04.807523  9682 raft_consensus.cc:2272] T 0a574a3bcb5441568f4fd9b1c38d719a P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:04.810645  9682 tablet_replica.cc:333] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:20:04.811506  9682 raft_consensus.cc:2243] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 28 LEADER]: Raft consensus shutting down.
I20260709 13:20:04.813119  9682 raft_consensus.cc:2272] T 747220f0813a4e19a0b4b96bd98d0a2a P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:04.822695  9682 tablet_replica.cc:333] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:20:04.825815  9682 raft_consensus.cc:2243] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:20:04.830111  9682 raft_consensus.cc:2272] T d32f5e230e464e61b23a41962708d120 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:04.840801  9682 tablet_replica.cc:333] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:20:04.841845  9682 raft_consensus.cc:2243] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:04.843130  9682 raft_consensus.cc:2272] T ce92c99123994b928d7a775917eb22ed P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:04.846117  9682 tablet_replica.cc:333] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:20:04.851013  9682 raft_consensus.cc:2243] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:04.851795  9682 raft_consensus.cc:2272] T b81df256f0c644cfae9fe2d336cbe9bd P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:04.866823 11477 consensus_peers.cc:597] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 -> Peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Couldn't send request to peer cb0428cc55984daba1ba63744a5102a2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:04.871987  9682 tablet_replica.cc:333] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
W20260709 13:20:04.873787 11478 consensus_peers.cc:597] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:04.874708 11477 consensus_peers.cc:597] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 -> Peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Couldn't send request to peer cb0428cc55984daba1ba63744a5102a2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:04.875629  9682 raft_consensus.cc:2243] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:04.876459  9682 raft_consensus.cc:2272] T 185480268a684958b8347a4f53af57c5 P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:04.881449  9682 tablet_replica.cc:333] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
W20260709 13:20:04.884660 11478 consensus_peers.cc:597] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:20:04.885670  9682 raft_consensus.cc:2243] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:20:04.887005  9682 raft_consensus.cc:2272] T 76b2d93b69274dbca479bd87d7ea45cb P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:04.897022  9682 tablet_replica.cc:333] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
W20260709 13:20:04.907054 11477 consensus_peers.cc:597] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 -> Peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Couldn't send request to peer cb0428cc55984daba1ba63744a5102a2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:04.907924  9682 raft_consensus.cc:2243] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 29 LEADER]: Raft consensus shutting down.
I20260709 13:20:04.909777  9682 raft_consensus.cc:2272] T 99ca4d7ec0974268ac5f35c5055da891 P cb0428cc55984daba1ba63744a5102a2 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:04.916288  9682 tablet_replica.cc:333] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:20:04.917846  9682 raft_consensus.cc:2243] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:20:04.919204  9682 raft_consensus.cc:2272] T a50b3bb87a48494a8e835471e55a6f74 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:04.922055  9682 tablet_replica.cc:333] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
W20260709 13:20:04.922678 10916 consensus_peers.cc:597] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:20:04.923004  9682 raft_consensus.cc:2243] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:20:04.924397  9682 raft_consensus.cc:2272] T 86ba4060a26c42998e9f220a79f28546 P cb0428cc55984daba1ba63744a5102a2 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:04.927230  9682 tablet_replica.cc:333] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:20:04.928058  9682 raft_consensus.cc:2243] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:04.928958  9682 raft_consensus.cc:2272] T 741efe79e97646b396f0091dc10c745e P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:04.931622  9682 tablet_replica.cc:333] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:20:04.932375  9682 raft_consensus.cc:2243] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:04.934365  9682 raft_consensus.cc:2272] T b761e5168d4e48aeaec47b99f5c7d3d0 P cb0428cc55984daba1ba63744a5102a2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:04.938911  9682 tablet_replica.cc:333] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:20:04.939764  9682 raft_consensus.cc:2243] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:20:04.941490  9682 raft_consensus.cc:2272] T 32e87e18e6c9475085e47d6a2fd71a7b P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:04.946162  9682 tablet_replica.cc:333] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:20:04.947120  9682 raft_consensus.cc:2243] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:20:04.948359  9682 raft_consensus.cc:2272] T 414992c8073646798991d2b43fd61f1d P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:04.951337  9682 tablet_replica.cc:333] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:20:04.952135  9682 raft_consensus.cc:2243] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:04.952992  9682 raft_consensus.cc:2272] T bc6987a00cef4dff9545eee242db0982 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:04.955674  9682 tablet_replica.cc:333] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:20:04.956485  9682 raft_consensus.cc:2243] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:04.957234  9682 raft_consensus.cc:2272] T d00373866c144f4981da73bd7ae39019 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:04.959949  9682 tablet_replica.cc:333] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:20:04.960731  9682 raft_consensus.cc:2243] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:04.961385  9682 raft_consensus.cc:2272] T 90ea9e6b562e4600be1cb566739a6a30 P cb0428cc55984daba1ba63744a5102a2 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:04.964296  9682 tablet_replica.cc:333] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:20:04.965046  9682 raft_consensus.cc:2243] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:04.965885  9682 raft_consensus.cc:2272] T e41482a95776477784d9bd9dacfed231 P cb0428cc55984daba1ba63744a5102a2 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:04.970763  9682 tablet_replica.cc:333] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:20:04.971588  9682 raft_consensus.cc:2243] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:04.972774  9682 raft_consensus.cc:2272] T ddb097c9ce1748458d6719bc91f56e1d P cb0428cc55984daba1ba63744a5102a2 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:04.984936  9682 tablet_replica.cc:333] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:20:04.985745  9682 raft_consensus.cc:2243] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Raft consensus shutting down.
W20260709 13:20:04.986339 10916 consensus_peers.cc:597] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:20:04.987138  9682 raft_consensus.cc:2272] T 12d33f7a0f8f47e5ad57e854e52ae154 P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:04.990218  9682 tablet_replica.cc:333] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:20:04.991091  9682 raft_consensus.cc:2243] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:20:04.992508  9682 raft_consensus.cc:2272] T 651c4c65e7f44b1fb4053d3757ac417f P cb0428cc55984daba1ba63744a5102a2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:05.007532  9682 tablet_replica.cc:333] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:20:05.008379  9682 raft_consensus.cc:2243] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:20:05.009738  9682 raft_consensus.cc:2272] T 417caa22425d4a25aca2f7d8e9577256 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:05.014757  9682 tablet_replica.cc:333] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:20:05.015590  9682 raft_consensus.cc:2243] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:20:05.017560  9682 raft_consensus.cc:2272] T 695ff1b5a68e4111917d909909488d39 P cb0428cc55984daba1ba63744a5102a2 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:05.020540  9682 tablet_replica.cc:333] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:20:05.021235  9682 raft_consensus.cc:2243] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:20:05.055416  9682 raft_consensus.cc:2272] T 9b4c7fc28a9b461abe203fe66da89aca P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:05.062196  9682 tablet_replica.cc:333] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
W20260709 13:20:05.066453 11478 consensus_peers.cc:597] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:20:05.066926  9682 raft_consensus.cc:2243] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:05.095712  9682 raft_consensus.cc:2272] T fc429d4d563f415d90d6f74da422a740 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:05.110977  9682 tablet_replica.cc:333] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:20:05.131192  9682 raft_consensus.cc:2243] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:05.132653  9682 raft_consensus.cc:2272] T 96558ab13797493e9c168b86fe3b62cd P cb0428cc55984daba1ba63744a5102a2 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:05.133595 11478 consensus_peers.cc:597] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:20:05.141834  9682 tablet_replica.cc:333] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:20:05.142544  9682 raft_consensus.cc:2243] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:05.143172  9682 raft_consensus.cc:2272] T 8cf3aebfc54a4652a83388ef9a48b1c2 P cb0428cc55984daba1ba63744a5102a2 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:05.145433  9682 tablet_replica.cc:333] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2: stopping tablet replica
I20260709 13:20:05.146255  9682 raft_consensus.cc:2243] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:05.147055  9682 raft_consensus.cc:2272] T 8949b8b2d08942939f64373bccb71936 P cb0428cc55984daba1ba63744a5102a2 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:05.161473 11478 consensus_peers.cc:597] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:05.233026 11478 consensus_peers.cc:597] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:05.326529 11478 consensus_peers.cc:597] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:05.410028 11478 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111) [suppressed 577 similar messages]
W20260709 13:20:05.507242 11478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:37895, user_credentials={real_user=slave}} blocked reactor thread for 73786.6us
W20260709 13:20:05.579361 11478 consensus_peers.cc:597] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:05.579996 11478 consensus_peers.cc:597] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 -> Peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Couldn't send request to peer 468a8fd46e824f62b9c2c0d3532a6d26. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:20:05.660584 19581 raft_consensus.cc:493] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:05.661183 19581 raft_consensus.cc:515] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:20:05.664775 19581 leader_election.cc:290] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:20:05.687359 11478 leader_election.cc:336] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:20:05.693708 11478 leader_election.cc:336] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:05.694293 11478 leader_election.cc:304] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:05.695564 19582 raft_consensus.cc:2749] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:20:05.702252 19581 raft_consensus.cc:493] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:05.702937 19581 raft_consensus.cc:515] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:20:05.705133 19581 leader_election.cc:290] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
W20260709 13:20:05.721832 11478 leader_election.cc:336] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
W20260709 13:20:05.727038 11478 leader_election.cc:336] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:20:05.727648 11478 leader_election.cc:304] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:05.734889 19581 raft_consensus.cc:2749] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:05.874322 19581 raft_consensus.cc:493] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:20:05.875008 19581 raft_consensus.cc:515] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:20:05.884563 19582 raft_consensus.cc:493] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:20:05.885150 19582 raft_consensus.cc:515] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
W20260709 13:20:05.887347 11478 leader_election.cc:336] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:05.888199 19582 leader_election.cc:290] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:20:05.899086 11477 leader_election.cc:336] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Network error: Client connection negotiation failed: client connection to 127.9.116.131:45553: connect: Connection refused (error 111)
I20260709 13:20:05.904799 11477 leader_election.cc:304] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:20:05.905825 19582 raft_consensus.cc:2749] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:20:05.900267 19581 leader_election.cc:290] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:20:05.906957 11478 leader_election.cc:336] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:20:05.910418 11478 leader_election.cc:336] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:05.910993 11478 leader_election.cc:304] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:05.911933 19581 raft_consensus.cc:493] T d32f5e230e464e61b23a41962708d120 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:20:05.912609 19581 raft_consensus.cc:515] T d32f5e230e464e61b23a41962708d120 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:20:05.915112 19581 leader_election.cc:290] T d32f5e230e464e61b23a41962708d120 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:20:05.915592 11477 leader_election.cc:336] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Network error: Client connection negotiation failed: client connection to 127.9.116.131:45553: connect: Connection refused (error 111)
I20260709 13:20:05.916189 19582 raft_consensus.cc:2749] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:20:05.933466 11477 leader_election.cc:336] T d32f5e230e464e61b23a41962708d120 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Network error: Client connection negotiation failed: client connection to 127.9.116.131:45553: connect: Connection refused (error 111)
W20260709 13:20:05.943601 11478 leader_election.cc:336] T d32f5e230e464e61b23a41962708d120 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:05.944249 11478 leader_election.cc:304] T d32f5e230e464e61b23a41962708d120 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:20:05.945147 19582 raft_consensus.cc:2749] T d32f5e230e464e61b23a41962708d120 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:20:05.980736 11478 consensus_peers.cc:597] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 -> Peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Couldn't send request to peer e01ecd008edd4aa2b6b2b47efbc2ea87. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:20:06.058399 19581 raft_consensus.cc:493] T 8cab289810294719ac9686e40290a6ab P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:20:06.059068 19581 raft_consensus.cc:515] T 8cab289810294719ac9686e40290a6ab P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } attrs { replace: true } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:20:06.062031 19581 leader_election.cc:290] T 8cab289810294719ac9686e40290a6ab P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:20:06.062655 19581 raft_consensus.cc:493] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:20:06.063346 19581 raft_consensus.cc:515] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:20:06.065583 19581 leader_election.cc:290] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:20:06.086733 11477 leader_election.cc:336] T 8cab289810294719ac9686e40290a6ab P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Network error: Client connection negotiation failed: client connection to 127.9.116.131:45553: connect: Connection refused (error 111)
W20260709 13:20:06.088025 11477 leader_election.cc:336] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Network error: Client connection negotiation failed: client connection to 127.9.116.131:45553: connect: Connection refused (error 111)
W20260709 13:20:06.088465 11478 leader_election.cc:336] T 8cab289810294719ac9686e40290a6ab P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:20:06.090588 11478 leader_election.cc:304] T 8cab289810294719ac9686e40290a6ab P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:06.091677 19581 raft_consensus.cc:2749] T 8cab289810294719ac9686e40290a6ab P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:20:06.092624 11478 leader_election.cc:336] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:20:06.093124 11478 leader_election.cc:304] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:06.093922 19581 raft_consensus.cc:2749] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:20:06.095665 11478 leader_election.cc:336] T 8cab289810294719ac9686e40290a6ab P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:06.131749 19581 raft_consensus.cc:493] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:20:06.132305 19581 raft_consensus.cc:515] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:20:06.135542 19582 raft_consensus.cc:493] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:20:06.136085 19582 raft_consensus.cc:515] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:20:06.138123 19582 leader_election.cc:290] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:20:06.140875 19581 leader_election.cc:290] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:20:06.144416 11477 leader_election.cc:336] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Network error: Client connection negotiation failed: client connection to 127.9.116.131:45553: connect: Connection refused (error 111)
W20260709 13:20:06.146466 11477 leader_election.cc:336] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Network error: Client connection negotiation failed: client connection to 127.9.116.131:45553: connect: Connection refused (error 111)
W20260709 13:20:06.156657 11478 leader_election.cc:336] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:20:06.157549 11478 leader_election.cc:304] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:06.158396 19581 raft_consensus.cc:2749] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:20:06.159538 11478 leader_election.cc:336] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:20:06.160001 11478 leader_election.cc:304] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:06.160801 19581 raft_consensus.cc:2749] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:06.266119 19581 raft_consensus.cc:493] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:20:06.266657 19581 raft_consensus.cc:515] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:20:06.268848 19581 leader_election.cc:290] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:20:06.271343 19581 raft_consensus.cc:493] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:20:06.271927 19581 raft_consensus.cc:515] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:20:06.274156 19581 leader_election.cc:290] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:20:06.278342 11477 leader_election.cc:336] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Network error: Client connection negotiation failed: client connection to 127.9.116.131:45553: connect: Connection refused (error 111)
W20260709 13:20:06.278970 11477 leader_election.cc:336] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Network error: Client connection negotiation failed: client connection to 127.9.116.131:45553: connect: Connection refused (error 111)
W20260709 13:20:06.284555 11478 leader_election.cc:336] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:20:06.285084 11478 leader_election.cc:304] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:06.285959 19581 raft_consensus.cc:2749] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:20:06.297973 11478 leader_election.cc:336] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:20:06.298738 11478 leader_election.cc:304] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:06.300799 19581 raft_consensus.cc:2749] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:20:06.315125  9682 tablet_server.cc:196] TabletServer@127.9.116.131:0 shutdown complete.
I20260709 13:20:06.335198 19581 raft_consensus.cc:493] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:20:06.335858 19581 raft_consensus.cc:515] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:20:06.342051 19581 leader_election.cc:290] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:20:06.344691 11477 leader_election.cc:336] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Network error: Client connection negotiation failed: client connection to 127.9.116.131:45553: connect: Connection refused (error 111)
W20260709 13:20:06.348843 11478 leader_election.cc:336] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:20:06.349417 11478 leader_election.cc:304] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: cb0428cc55984daba1ba63744a5102a2, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:06.350418 19581 raft_consensus.cc:2749] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:06.443758 19581 raft_consensus.cc:493] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader cb0428cc55984daba1ba63744a5102a2)
I20260709 13:20:06.444461 19581 raft_consensus.cc:515] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:20:06.459465 19581 leader_election.cc:290] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
I20260709 13:20:06.462656  9682 tablet_server.cc:179] TabletServer@127.9.116.132:0 shutting down...
W20260709 13:20:06.464382 11478 leader_election.cc:336] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
W20260709 13:20:06.468279 11477 leader_election.cc:336] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Network error: Client connection negotiation failed: client connection to 127.9.116.131:45553: connect: Connection refused (error 111)
I20260709 13:20:06.468844 11477 leader_election.cc:304] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:20:06.470109 19581 raft_consensus.cc:2749] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:20:06.491266 19581 raft_consensus.cc:493] T b761e5168d4e48aeaec47b99f5c7d3d0 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:06.491848 19581 raft_consensus.cc:515] T b761e5168d4e48aeaec47b99f5c7d3d0 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "cb0428cc55984daba1ba63744a5102a2" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45553 } attrs { replace: true } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:20:06.494649 19581 leader_election.cc:290] T b761e5168d4e48aeaec47b99f5c7d3d0 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895), cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553)
W20260709 13:20:06.514902 11477 leader_election.cc:336] T b761e5168d4e48aeaec47b99f5c7d3d0 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): Network error: Client connection negotiation failed: client connection to 127.9.116.131:45553: connect: Connection refused (error 111)
W20260709 13:20:06.517414 11533 debug-util.cc:398] Leaking SignalData structure 0x7b0800084700 after lost signal to thread 9685
W20260709 13:20:06.518522 11533 debug-util.cc:398] Leaking SignalData structure 0x7b08002dd9c0 after lost signal to thread 10733
W20260709 13:20:06.519944 11533 debug-util.cc:398] Leaking SignalData structure 0x7b08002d9a80 after lost signal to thread 11536
W20260709 13:20:06.531999 11478 leader_election.cc:336] T b761e5168d4e48aeaec47b99f5c7d3d0 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:06.532646 11478 leader_election.cc:304] T b761e5168d4e48aeaec47b99f5c7d3d0 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, cb0428cc55984daba1ba63744a5102a2
I20260709 13:20:06.533396 19581 raft_consensus.cc:2749] T b761e5168d4e48aeaec47b99f5c7d3d0 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:20:06.533828 11478 leader_election.cc:336] T b761e5168d4e48aeaec47b99f5c7d3d0 P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:20:06.553407  9682 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:20:06.554526  9682 tablet_replica.cc:333] T b761e5168d4e48aeaec47b99f5c7d3d0 P 12682eb70bc84c76879e04a8628ee586: stopping tablet replica
I20260709 13:20:06.555663  9682 raft_consensus.cc:2243] T b761e5168d4e48aeaec47b99f5c7d3d0 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:06.556666  9682 raft_consensus.cc:2272] T b761e5168d4e48aeaec47b99f5c7d3d0 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.567889  9682 tablet_replica.cc:333] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586: stopping tablet replica
I20260709 13:20:06.572563  9682 raft_consensus.cc:2243] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:06.573588  9682 raft_consensus.cc:2272] T 171abdebfc4142cda86de477a5c89a7e P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.576635  9682 tablet_replica.cc:333] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586: stopping tablet replica
I20260709 13:20:06.577445  9682 raft_consensus.cc:2243] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:06.578512  9682 raft_consensus.cc:2272] T 2b82f18992434ed5ba58b805107060d4 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.581955  9682 tablet_replica.cc:333] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586: stopping tablet replica
I20260709 13:20:06.582916  9682 raft_consensus.cc:2243] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:06.584115  9682 raft_consensus.cc:2272] T 695ff1b5a68e4111917d909909488d39 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.586764  9682 tablet_replica.cc:333] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586: stopping tablet replica
I20260709 13:20:06.587595  9682 raft_consensus.cc:2243] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:20:06.588752  9682 raft_consensus.cc:2272] T b253bd741d094067902a02bc765ee0fb P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.591519  9682 tablet_replica.cc:333] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586: stopping tablet replica
I20260709 13:20:06.592290  9682 raft_consensus.cc:2243] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:06.593043  9682 raft_consensus.cc:2272] T 414992c8073646798991d2b43fd61f1d P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.597007  9682 tablet_replica.cc:333] T 8cab289810294719ac9686e40290a6ab P 12682eb70bc84c76879e04a8628ee586: stopping tablet replica
I20260709 13:20:06.597931  9682 raft_consensus.cc:2243] T 8cab289810294719ac9686e40290a6ab P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:06.598743  9682 raft_consensus.cc:2272] T 8cab289810294719ac9686e40290a6ab P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.601774  9682 tablet_replica.cc:333] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586: stopping tablet replica
I20260709 13:20:06.602602  9682 raft_consensus.cc:2243] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:20:06.603703  9682 raft_consensus.cc:2272] T 37a1fc8ca1b34260908d185a9ebc2bb3 P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.606438  9682 tablet_replica.cc:333] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586: stopping tablet replica
I20260709 13:20:06.607219  9682 raft_consensus.cc:2243] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:20:06.608320  9682 raft_consensus.cc:2272] T 19c20efd6c04443596bb712e3c93c650 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.610828  9682 tablet_replica.cc:333] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586: stopping tablet replica
I20260709 13:20:06.611559  9682 raft_consensus.cc:2243] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:06.612409  9682 raft_consensus.cc:2272] T 4dd4c0ff1def4960931dda3b52a4a884 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.615329  9682 tablet_replica.cc:333] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586: stopping tablet replica
I20260709 13:20:06.616055  9682 raft_consensus.cc:2243] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:06.616647  9682 raft_consensus.cc:2272] T 0a574a3bcb5441568f4fd9b1c38d719a P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.619405  9682 tablet_replica.cc:333] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586: stopping tablet replica
I20260709 13:20:06.620155  9682 raft_consensus.cc:2243] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:20:06.621340  9682 raft_consensus.cc:2272] T 741efe79e97646b396f0091dc10c745e P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.624073  9682 tablet_replica.cc:333] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586: stopping tablet replica
I20260709 13:20:06.624912  9682 raft_consensus.cc:2243] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:20:06.626055  9682 raft_consensus.cc:2272] T 185480268a684958b8347a4f53af57c5 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.628743  9682 tablet_replica.cc:333] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586: stopping tablet replica
I20260709 13:20:06.629559  9682 raft_consensus.cc:2243] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:06.630259  9682 raft_consensus.cc:2272] T 76b2d93b69274dbca479bd87d7ea45cb P 12682eb70bc84c76879e04a8628ee586 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.631904 19591 raft_consensus.cc:493] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 468a8fd46e824f62b9c2c0d3532a6d26)
I20260709 13:20:06.632486 19591 raft_consensus.cc:515] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e01ecd008edd4aa2b6b2b47efbc2ea87" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 39685 } } peers { permanent_uuid: "468a8fd46e824f62b9c2c0d3532a6d26" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37895 } } peers { permanent_uuid: "12682eb70bc84c76879e04a8628ee586" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 40555 } attrs { promote: false } }
I20260709 13:20:06.633008  9682 tablet_replica.cc:333] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586: stopping tablet replica
I20260709 13:20:06.633888  9682 raft_consensus.cc:2243] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:20:06.635362  9682 raft_consensus.cc:2272] T 8cf3aebfc54a4652a83388ef9a48b1c2 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.638204  9682 tablet_replica.cc:333] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586: stopping tablet replica
I20260709 13:20:06.639060  9682 raft_consensus.cc:2243] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:20:06.640369  9682 raft_consensus.cc:2272] T f15293e178d84fe686fd66bc4439aba8 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.643132  9682 tablet_replica.cc:333] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586: stopping tablet replica
I20260709 13:20:06.643942  9682 raft_consensus.cc:2243] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 29 LEADER]: Raft consensus shutting down.
I20260709 13:20:06.644999  9682 raft_consensus.cc:2272] T 5944b8838e584a3890e4e8143f3f2665 P 12682eb70bc84c76879e04a8628ee586 [term 29 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:06.645349 11478 leader_election.cc:336] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:20:06.646713 11478 leader_election.cc:336] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895): Network error: Client connection negotiation failed: client connection to 127.9.116.130:37895: connect: Connection refused (error 111)
I20260709 13:20:06.647289 11478 leader_election.cc:304] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12682eb70bc84c76879e04a8628ee586; no voters: 468a8fd46e824f62b9c2c0d3532a6d26, e01ecd008edd4aa2b6b2b47efbc2ea87
I20260709 13:20:06.645982 19591 leader_election.cc:290] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685), 468a8fd46e824f62b9c2c0d3532a6d26 (127.9.116.130:37895)
I20260709 13:20:06.648195 19581 raft_consensus.cc:2749] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:06.647851  9682 tablet_replica.cc:333] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586: stopping tablet replica
I20260709 13:20:06.649561  9682 raft_consensus.cc:2243] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:20:06.650650  9682 raft_consensus.cc:2272] T 416a6d12184f4fbda69570b95be47851 P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.653385  9682 tablet_replica.cc:333] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586: stopping tablet replica
I20260709 13:20:06.654109  9682 raft_consensus.cc:2243] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:20:06.655253  9682 raft_consensus.cc:2272] T e9764b93d3a8407b8cc1ae641620c831 P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.657829  9682 tablet_replica.cc:333] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586: stopping tablet replica
I20260709 13:20:06.658547  9682 raft_consensus.cc:2243] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:06.659308  9682 raft_consensus.cc:2272] T 5c8a233bc6c44bd2b5d2d44a726b147c P 12682eb70bc84c76879e04a8628ee586 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.661885  9682 tablet_replica.cc:333] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586: stopping tablet replica
I20260709 13:20:06.662626  9682 raft_consensus.cc:2243] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 29 LEADER]: Raft consensus shutting down.
I20260709 13:20:06.663776  9682 raft_consensus.cc:2272] T ad1b662471ff4b9ebda3845b4f224a34 P 12682eb70bc84c76879e04a8628ee586 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.666411  9682 tablet_replica.cc:333] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586: stopping tablet replica
I20260709 13:20:06.667171  9682 raft_consensus.cc:2243] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:06.667860  9682 raft_consensus.cc:2272] T 121ef9771004451eb563dbfd77657e0e P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.670603  9682 tablet_replica.cc:333] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586: stopping tablet replica
I20260709 13:20:06.671402  9682 raft_consensus.cc:2243] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:06.672259  9682 raft_consensus.cc:2272] T 1d9e744bb20c4325969f4fbc01c672e8 P 12682eb70bc84c76879e04a8628ee586 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.674806  9682 tablet_replica.cc:333] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586: stopping tablet replica
I20260709 13:20:06.675488  9682 raft_consensus.cc:2243] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:20:06.676560  9682 raft_consensus.cc:2272] T 51ae8de979e44a2d8c155bffc20a1533 P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.679515  9682 tablet_replica.cc:333] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586: stopping tablet replica
I20260709 13:20:06.680310  9682 raft_consensus.cc:2243] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:06.680948  9682 raft_consensus.cc:2272] T 9b4c7fc28a9b461abe203fe66da89aca P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.683706  9682 tablet_replica.cc:333] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586: stopping tablet replica
I20260709 13:20:06.684505  9682 raft_consensus.cc:2243] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:20:06.685665  9682 raft_consensus.cc:2272] T 7595d019b2e74196bc5838962b53798d P 12682eb70bc84c76879e04a8628ee586 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.688476  9682 tablet_replica.cc:333] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586: stopping tablet replica
I20260709 13:20:06.689234  9682 raft_consensus.cc:2243] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:06.689844  9682 raft_consensus.cc:2272] T f6e90b642b0d4f06b866170dc0293dc1 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.692400  9682 tablet_replica.cc:333] T d32f5e230e464e61b23a41962708d120 P 12682eb70bc84c76879e04a8628ee586: stopping tablet replica
I20260709 13:20:06.693233  9682 raft_consensus.cc:2243] T d32f5e230e464e61b23a41962708d120 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:06.694366  9682 raft_consensus.cc:2272] T d32f5e230e464e61b23a41962708d120 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.697396  9682 tablet_replica.cc:333] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586: stopping tablet replica
I20260709 13:20:06.698156  9682 raft_consensus.cc:2243] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:20:06.699306  9682 raft_consensus.cc:2272] T b81df256f0c644cfae9fe2d336cbe9bd P 12682eb70bc84c76879e04a8628ee586 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.701956  9682 tablet_replica.cc:333] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586: stopping tablet replica
I20260709 13:20:06.702677  9682 raft_consensus.cc:2243] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:06.703547  9682 raft_consensus.cc:2272] T 86ba4060a26c42998e9f220a79f28546 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.706288  9682 tablet_replica.cc:333] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586: stopping tablet replica
I20260709 13:20:06.707032  9682 raft_consensus.cc:2243] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:06.707648  9682 raft_consensus.cc:2272] T 73af6c7e9b8e4ffaa65ba65a6199eb61 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.710381  9682 tablet_replica.cc:333] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586: stopping tablet replica
I20260709 13:20:06.711148  9682 raft_consensus.cc:2243] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:06.711825  9682 raft_consensus.cc:2272] T b4b13050d714473bb01ea0174d634a14 P 12682eb70bc84c76879e04a8628ee586 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.714609  9682 tablet_replica.cc:333] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586: stopping tablet replica
I20260709 13:20:06.715284  9682 raft_consensus.cc:2243] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:06.716063  9682 raft_consensus.cc:2272] T 12d33f7a0f8f47e5ad57e854e52ae154 P 12682eb70bc84c76879e04a8628ee586 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.718581  9682 tablet_replica.cc:333] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586: stopping tablet replica
I20260709 13:20:06.719244  9682 raft_consensus.cc:2243] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:06.720070  9682 raft_consensus.cc:2272] T dde624b07ed446e5b7347791196396e2 P 12682eb70bc84c76879e04a8628ee586 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:08.000514  9682 tablet_server.cc:196] TabletServer@127.9.116.132:0 shutdown complete.
I20260709 13:20:08.067816  9682 master.cc:562] Master@127.9.116.190:36007 shutting down...
W20260709 13:20:08.104875 10747 auto_rebalancer.cc:873] Removing replace marker failed: Catalog manager is not initialized. State: Closing
W20260709 13:20:08.105206 10747 auto_rebalancer.cc:901] Could not move replica: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:20:08.105453 10747 auto_rebalancer.cc:330] scheduled replica move failed to complete: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:20:10.217825 10730 debug-util.cc:398] Leaking SignalData structure 0x7b08002c5ac0 after lost signal to thread 9685
W20260709 13:20:10.218570 10730 debug-util.cc:398] Leaking SignalData structure 0x7b0800291cc0 after lost signal to thread 10733
W20260709 13:20:11.299427 10685 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.116.131:45553: connect: Connection refused (error 111) [suppressed 104 similar messages]
W20260709 13:20:11.303297 10685 catalog_manager.cc:4755] TS cb0428cc55984daba1ba63744a5102a2 (127.9.116.131:45553): ChangeConfig:REMOVE_PEER RPC failed for tablet 2b82f18992434ed5ba58b805107060d4: Network error: Client connection negotiation failed: client connection to 127.9.116.131:45553: connect: Connection refused (error 111)
W20260709 13:20:20.614661 10686 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
W20260709 13:20:20.618942 10686 catalog_manager.cc:4755] TS e01ecd008edd4aa2b6b2b47efbc2ea87 (127.9.116.129:39685): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet d32f5e230e464e61b23a41962708d120: Network error: Client connection negotiation failed: client connection to 127.9.116.129:39685: connect: Connection refused (error 111)
I20260709 13:20:21.729140  9682 raft_consensus.cc:2243] T 00000000000000000000000000000000 P ae957c8fc66c4b6ba063d0c74cbf5484 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:21.730341  9682 raft_consensus.cc:2272] T 00000000000000000000000000000000 P ae957c8fc66c4b6ba063d0c74cbf5484 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:21.730728  9682 tablet_replica.cc:333] T 00000000000000000000000000000000 P ae957c8fc66c4b6ba063d0c74cbf5484: stopping tablet replica
I20260709 13:20:21.907677  9682 master.cc:584] Master@127.9.116.190:36007 shutdown complete.
I20260709 13:20:21.959816  9682 test_util.cc:182] -----------------------------------------------
I20260709 13:20:21.960032  9682 test_util.cc:183] Had failures, leaving test files at /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602916530764-9682-0
[  FAILED  ] LeaderRebalancerTest.AddTserver (282272 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 13:20:21.963613  9682 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.9.116.190:44649
I20260709 13:20:21.964736  9682 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:20:21.970882 19598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:20:21.971246 19599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:20:21.971654 19601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:20:21.972548  9682 server_base.cc:1061] running on GCE node
I20260709 13:20:21.975354  9682 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:20:21.975550  9682 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:20:21.975744  9682 hybrid_clock.cc:648] HybridClock initialized: now 1783603221975710 us; error 0 us; skew 500 ppm
I20260709 13:20:21.976275  9682 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:20:21.978732  9682 webserver.cc:533] Webserver started at http://127.9.116.190:38055/ using document root <none> and password file <none>
I20260709 13:20:21.979413  9682 fs_manager.cc:362] Metadata directory not provided
I20260709 13:20:21.979619  9682 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:20:21.979914  9682 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:20:21.981258  9682 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602916530764-9682-0/minicluster-data/master-0-root/instance:
uuid: "e027addf51a94fb8b711872407dc97b2"
format_stamp: "Formatted at 2026-07-09 13:20:21 on dist-test-slave-1t6m"
I20260709 13:20:21.987929  9682 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.005s	sys 0.004s
I20260709 13:20:21.992682 19606 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:20:21.993492  9682 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:20:21.993868  9682 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602916530764-9682-0/minicluster-data/master-0-root
uuid: "e027addf51a94fb8b711872407dc97b2"
format_stamp: "Formatted at 2026-07-09 13:20:21 on dist-test-slave-1t6m"
I20260709 13:20:21.994221  9682 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602916530764-9682-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602916530764-9682-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602916530764-9682-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:20:22.017683  9682 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:20:22.019127  9682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:20:22.058748  9682 rpc_server.cc:307] RPC server started. Bound to: 127.9.116.190:44649
I20260709 13:20:22.058889 19657 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.116.190:44649 every 8 connection(s)
I20260709 13:20:22.063067 19658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:22.074507 19658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e027addf51a94fb8b711872407dc97b2: Bootstrap starting.
I20260709 13:20:22.079614 19658 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e027addf51a94fb8b711872407dc97b2: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:22.084179 19658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e027addf51a94fb8b711872407dc97b2: No bootstrap required, opened a new log
I20260709 13:20:22.086521 19658 raft_consensus.cc:359] T 00000000000000000000000000000000 P e027addf51a94fb8b711872407dc97b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e027addf51a94fb8b711872407dc97b2" member_type: VOTER }
I20260709 13:20:22.087030 19658 raft_consensus.cc:385] T 00000000000000000000000000000000 P e027addf51a94fb8b711872407dc97b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:22.087283 19658 raft_consensus.cc:740] T 00000000000000000000000000000000 P e027addf51a94fb8b711872407dc97b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e027addf51a94fb8b711872407dc97b2, State: Initialized, Role: FOLLOWER
I20260709 13:20:22.087944 19658 consensus_queue.cc:260] T 00000000000000000000000000000000 P e027addf51a94fb8b711872407dc97b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e027addf51a94fb8b711872407dc97b2" member_type: VOTER }
I20260709 13:20:22.088454 19658 raft_consensus.cc:399] T 00000000000000000000000000000000 P e027addf51a94fb8b711872407dc97b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:20:22.088699 19658 raft_consensus.cc:493] T 00000000000000000000000000000000 P e027addf51a94fb8b711872407dc97b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:20:22.088986 19658 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e027addf51a94fb8b711872407dc97b2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:22.093693 19658 raft_consensus.cc:515] T 00000000000000000000000000000000 P e027addf51a94fb8b711872407dc97b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e027addf51a94fb8b711872407dc97b2" member_type: VOTER }
I20260709 13:20:22.094201 19658 leader_election.cc:304] T 00000000000000000000000000000000 P e027addf51a94fb8b711872407dc97b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e027addf51a94fb8b711872407dc97b2; no voters: 
I20260709 13:20:22.095497 19658 leader_election.cc:290] T 00000000000000000000000000000000 P e027addf51a94fb8b711872407dc97b2 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:20:22.096026 19661 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e027addf51a94fb8b711872407dc97b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:22.097584 19661 raft_consensus.cc:697] T 00000000000000000000000000000000 P e027addf51a94fb8b711872407dc97b2 [term 1 LEADER]: Becoming Leader. State: Replica: e027addf51a94fb8b711872407dc97b2, State: Running, Role: LEADER
I20260709 13:20:22.098429 19661 consensus_queue.cc:237] T 00000000000000000000000000000000 P e027addf51a94fb8b711872407dc97b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e027addf51a94fb8b711872407dc97b2" member_type: VOTER }
I20260709 13:20:22.098695 19658 sys_catalog.cc:565] T 00000000000000000000000000000000 P e027addf51a94fb8b711872407dc97b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:20:22.109719 19663 sys_catalog.cc:455] T 00000000000000000000000000000000 P e027addf51a94fb8b711872407dc97b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e027addf51a94fb8b711872407dc97b2. Latest consensus state: current_term: 1 leader_uuid: "e027addf51a94fb8b711872407dc97b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e027addf51a94fb8b711872407dc97b2" member_type: VOTER } }
I20260709 13:20:22.109637 19662 sys_catalog.cc:455] T 00000000000000000000000000000000 P e027addf51a94fb8b711872407dc97b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e027addf51a94fb8b711872407dc97b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e027addf51a94fb8b711872407dc97b2" member_type: VOTER } }
I20260709 13:20:22.110405 19663 sys_catalog.cc:458] T 00000000000000000000000000000000 P e027addf51a94fb8b711872407dc97b2 [sys.catalog]: This master's current role is: LEADER
I20260709 13:20:22.110605 19662 sys_catalog.cc:458] T 00000000000000000000000000000000 P e027addf51a94fb8b711872407dc97b2 [sys.catalog]: This master's current role is: LEADER
I20260709 13:20:22.112680 19672 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:20:22.120431 19672 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:20:22.121493  9682 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:20:22.130117 19672 catalog_manager.cc:1383] Generated new cluster ID: d1ade8eb38c6415fbe78740484eb9ec5
I20260709 13:20:22.130424 19672 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:20:22.165489 19672 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:20:22.166860 19672 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:20:22.180891 19672 catalog_manager.cc:6070] T 00000000000000000000000000000000 P e027addf51a94fb8b711872407dc97b2: Generated new TSK 0
I20260709 13:20:22.181511 19672 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:20:22.188158  9682 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:20:22.194546 19679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:20:22.195741 19680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:20:22.198429 19682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:20:22.199208  9682 server_base.cc:1061] running on GCE node
I20260709 13:20:22.201928  9682 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:20:22.202159  9682 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:20:22.202317  9682 hybrid_clock.cc:648] HybridClock initialized: now 1783603222202302 us; error 0 us; skew 500 ppm
I20260709 13:20:22.202972  9682 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:20:22.205348  9682 webserver.cc:533] Webserver started at http://127.9.116.129:44883/ using document root <none> and password file <none>
I20260709 13:20:22.205850  9682 fs_manager.cc:362] Metadata directory not provided
I20260709 13:20:22.206051  9682 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:20:22.206332  9682 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:20:22.207585  9682 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602916530764-9682-0/minicluster-data/ts-0-root/instance:
uuid: "8df38c4c53c44e94b36c1e8992476d7c"
format_stamp: "Formatted at 2026-07-09 13:20:22 on dist-test-slave-1t6m"
I20260709 13:20:22.212409  9682 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.004s
I20260709 13:20:22.215934 19687 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:20:22.216670  9682 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 13:20:22.217012  9682 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602916530764-9682-0/minicluster-data/ts-0-root
uuid: "8df38c4c53c44e94b36c1e8992476d7c"
format_stamp: "Formatted at 2026-07-09 13:20:22 on dist-test-slave-1t6m"
I20260709 13:20:22.217315  9682 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602916530764-9682-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602916530764-9682-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602916530764-9682-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:20:22.228405  9682 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:20:22.229673  9682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:20:22.231223  9682 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:20:22.233678  9682 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:20:22.233924  9682 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:20:22.234191  9682 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:20:22.234392  9682 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:20:22.277498  9682 rpc_server.cc:307] RPC server started. Bound to: 127.9.116.129:41855
I20260709 13:20:22.277626 19749 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.116.129:41855 every 8 connection(s)
I20260709 13:20:22.282982  9682 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:20:22.290632 19754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:20:22.291569 19755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:20:22.295313  9682 server_base.cc:1061] running on GCE node
I20260709 13:20:22.296576 19750 heartbeater.cc:344] Connected to a master server at 127.9.116.190:44649
I20260709 13:20:22.297137 19750 heartbeater.cc:461] Registering TS with master...
W20260709 13:20:22.297081 19757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:20:22.298405 19750 heartbeater.cc:507] Master 127.9.116.190:44649 requested a full tablet report, sending...
I20260709 13:20:22.298596  9682 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:20:22.298986  9682 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:20:22.299161  9682 hybrid_clock.cc:648] HybridClock initialized: now 1783603222299145 us; error 0 us; skew 500 ppm
I20260709 13:20:22.299808  9682 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:20:22.301096 19623 ts_manager.cc:194] Registered new tserver with Master: 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:22.302887  9682 webserver.cc:533] Webserver started at http://127.9.116.130:35921/ using document root <none> and password file <none>
I20260709 13:20:22.303577  9682 fs_manager.cc:362] Metadata directory not provided
I20260709 13:20:22.303820  9682 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:20:22.303834 19623 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60438
I20260709 13:20:22.304293  9682 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:20:22.305827  9682 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602916530764-9682-0/minicluster-data/ts-1-root/instance:
uuid: "e90ffed3ba174ababeb21d9415d4d918"
format_stamp: "Formatted at 2026-07-09 13:20:22 on dist-test-slave-1t6m"
I20260709 13:20:22.310724  9682 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.004s
I20260709 13:20:22.314527 19762 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:20:22.315447  9682 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:20:22.315822  9682 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602916530764-9682-0/minicluster-data/ts-1-root
uuid: "e90ffed3ba174ababeb21d9415d4d918"
format_stamp: "Formatted at 2026-07-09 13:20:22 on dist-test-slave-1t6m"
I20260709 13:20:22.316133  9682 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602916530764-9682-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602916530764-9682-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602916530764-9682-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:20:22.362414  9682 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:20:22.363775  9682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:20:22.365350  9682 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:20:22.367710  9682 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:20:22.367949  9682 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:20:22.368204  9682 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:20:22.368376  9682 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:20:22.411311  9682 rpc_server.cc:307] RPC server started. Bound to: 127.9.116.130:43319
I20260709 13:20:22.411409 19824 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.116.130:43319 every 8 connection(s)
I20260709 13:20:22.416559  9682 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:20:22.424789 19828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:20:22.425808 19829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:20:22.430037 19831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:20:22.430094  9682 server_base.cc:1061] running on GCE node
I20260709 13:20:22.430935 19825 heartbeater.cc:344] Connected to a master server at 127.9.116.190:44649
I20260709 13:20:22.431375 19825 heartbeater.cc:461] Registering TS with master...
I20260709 13:20:22.431499  9682 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:20:22.431813  9682 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:20:22.431991  9682 hybrid_clock.cc:648] HybridClock initialized: now 1783603222431956 us; error 0 us; skew 500 ppm
I20260709 13:20:22.432300 19825 heartbeater.cc:507] Master 127.9.116.190:44649 requested a full tablet report, sending...
I20260709 13:20:22.432705  9682 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:20:22.434991 19623 ts_manager.cc:194] Registered new tserver with Master: e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:22.435734  9682 webserver.cc:533] Webserver started at http://127.9.116.131:40859/ using document root <none> and password file <none>
I20260709 13:20:22.436439  9682 fs_manager.cc:362] Metadata directory not provided
I20260709 13:20:22.436688  9682 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:20:22.437003  9682 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:20:22.437013 19623 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60454
I20260709 13:20:22.438943  9682 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602916530764-9682-0/minicluster-data/ts-2-root/instance:
uuid: "b80b3a62d7d549a9a5c42def46769ac4"
format_stamp: "Formatted at 2026-07-09 13:20:22 on dist-test-slave-1t6m"
I20260709 13:20:22.443697  9682 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:20:22.447626 19836 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:20:22.448462  9682 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:20:22.448810  9682 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602916530764-9682-0/minicluster-data/ts-2-root
uuid: "b80b3a62d7d549a9a5c42def46769ac4"
format_stamp: "Formatted at 2026-07-09 13:20:22 on dist-test-slave-1t6m"
I20260709 13:20:22.449110  9682 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602916530764-9682-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602916530764-9682-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602916530764-9682-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:20:22.470846  9682 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:20:22.472189  9682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:20:22.473779  9682 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:20:22.476228  9682 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:20:22.476425  9682 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:20:22.476747  9682 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:20:22.476915  9682 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:20:22.520862  9682 rpc_server.cc:307] RPC server started. Bound to: 127.9.116.131:45249
I20260709 13:20:22.520963 19898 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.116.131:45249 every 8 connection(s)
I20260709 13:20:22.526062  9682 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:20:22.534353 19902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:20:22.535734 19903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:20:22.537447 19905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:20:22.538211  9682 server_base.cc:1061] running on GCE node
I20260709 13:20:22.539309 19899 heartbeater.cc:344] Connected to a master server at 127.9.116.190:44649
I20260709 13:20:22.539378  9682 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:20:22.539839  9682 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:20:22.539844 19899 heartbeater.cc:461] Registering TS with master...
I20260709 13:20:22.540124  9682 hybrid_clock.cc:648] HybridClock initialized: now 1783603222540105 us; error 0 us; skew 500 ppm
I20260709 13:20:22.540886 19899 heartbeater.cc:507] Master 127.9.116.190:44649 requested a full tablet report, sending...
I20260709 13:20:22.540933  9682 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:20:22.543223 19623 ts_manager.cc:194] Registered new tserver with Master: b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:22.543995  9682 webserver.cc:533] Webserver started at http://127.9.116.132:44229/ using document root <none> and password file <none>
I20260709 13:20:22.544705  9682 fs_manager.cc:362] Metadata directory not provided
I20260709 13:20:22.544754 19623 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60458
I20260709 13:20:22.544942  9682 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:20:22.545378  9682 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:20:22.546737  9682 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602916530764-9682-0/minicluster-data/ts-3-root/instance:
uuid: "c62716bf66c24330860573535b8b9430"
format_stamp: "Formatted at 2026-07-09 13:20:22 on dist-test-slave-1t6m"
I20260709 13:20:22.551530  9682 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:20:22.555279 19910 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:20:22.556051  9682 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 13:20:22.556399  9682 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602916530764-9682-0/minicluster-data/ts-3-root
uuid: "c62716bf66c24330860573535b8b9430"
format_stamp: "Formatted at 2026-07-09 13:20:22 on dist-test-slave-1t6m"
I20260709 13:20:22.556692  9682 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602916530764-9682-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602916530764-9682-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602916530764-9682-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:20:22.575551  9682 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:20:22.576924  9682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:20:22.578500  9682 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:20:22.580932  9682 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:20:22.581192  9682 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:20:22.581460  9682 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:20:22.581638  9682 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:20:22.625351  9682 rpc_server.cc:307] RPC server started. Bound to: 127.9.116.132:42643
I20260709 13:20:22.625468 19972 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.116.132:42643 every 8 connection(s)
I20260709 13:20:22.640403 19973 heartbeater.cc:344] Connected to a master server at 127.9.116.190:44649
I20260709 13:20:22.640959 19973 heartbeater.cc:461] Registering TS with master...
I20260709 13:20:22.641837 19973 heartbeater.cc:507] Master 127.9.116.190:44649 requested a full tablet report, sending...
I20260709 13:20:22.643944 19623 ts_manager.cc:194] Registered new tserver with Master: c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:22.644461  9682 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.015854114s
I20260709 13:20:22.645423 19623 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60470
I20260709 13:20:22.672916 19623 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60486:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 13:20:22.917400 19790 tablet_service.cc:1511] Processing CreateTablet for tablet 6a066b96902f41dbb1511eccf26820ce (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:20:22.919730 19790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a066b96902f41dbb1511eccf26820ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:22.919847 19789 tablet_service.cc:1511] Processing CreateTablet for tablet 180230b77b7640b29d055ae8f4358ffe (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:20:22.920331 19788 tablet_service.cc:1511] Processing CreateTablet for tablet 3cad7a5f46f9407a99ea6f2cc5a8322c (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:20:22.920331 19786 tablet_service.cc:1511] Processing CreateTablet for tablet e8cfec59c63741869c636df6356546d0 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:20:22.921828 19787 tablet_service.cc:1511] Processing CreateTablet for tablet af881b097b4245b1be6072579197cb6b (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:20:22.923605 19786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8cfec59c63741869c636df6356546d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:22.925416 19788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3cad7a5f46f9407a99ea6f2cc5a8322c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:22.927677 19789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 180230b77b7640b29d055ae8f4358ffe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:22.931417 19787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af881b097b4245b1be6072579197cb6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:22.932588 19785 tablet_service.cc:1511] Processing CreateTablet for tablet 5d9f7b34cb064bbe8a6938e4f92d57c5 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:20:22.934106 19785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d9f7b34cb064bbe8a6938e4f92d57c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:22.933952 19784 tablet_service.cc:1511] Processing CreateTablet for tablet f820e93ffdcd428ea4adfca88589a850 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:20:22.935462 19784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f820e93ffdcd428ea4adfca88589a850. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:22.951857 19783 tablet_service.cc:1511] Processing CreateTablet for tablet d762eace063c4e04821c749d21702c3f (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:20:22.953420 19783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d762eace063c4e04821c749d21702c3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:22.953491 19715 tablet_service.cc:1511] Processing CreateTablet for tablet 6a066b96902f41dbb1511eccf26820ce (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:20:22.955977 19714 tablet_service.cc:1511] Processing CreateTablet for tablet 180230b77b7640b29d055ae8f4358ffe (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:20:22.958379 19713 tablet_service.cc:1511] Processing CreateTablet for tablet 9b2e4bf3fb5c456dab400b096f7bd729 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:20:22.960813 19712 tablet_service.cc:1511] Processing CreateTablet for tablet af881b097b4245b1be6072579197cb6b (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:20:22.963339 19711 tablet_service.cc:1511] Processing CreateTablet for tablet e8cfec59c63741869c636df6356546d0 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:20:22.965700 19710 tablet_service.cc:1511] Processing CreateTablet for tablet 0f3d7bd0ea32401fa8cb8ab3292a7808 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:20:22.968016 19709 tablet_service.cc:1511] Processing CreateTablet for tablet f820e93ffdcd428ea4adfca88589a850 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:20:22.968720 19782 tablet_service.cc:1511] Processing CreateTablet for tablet 19a59035bda747fdbc59b7e6914101dd (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:20:22.970156 19782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19a59035bda747fdbc59b7e6914101dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:22.970651 19708 tablet_service.cc:1511] Processing CreateTablet for tablet d762eace063c4e04821c749d21702c3f (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:20:22.973642 19707 tablet_service.cc:1511] Processing CreateTablet for tablet ac3bca5bb7644a8fbc78df43ef3331a0 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:20:22.977511 19715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a066b96902f41dbb1511eccf26820ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:22.982363 19781 tablet_service.cc:1511] Processing CreateTablet for tablet 2a411802fdaa42e7863236f0af595fe2 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:20:22.983270 19707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac3bca5bb7644a8fbc78df43ef3331a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:22.984089 19781 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a411802fdaa42e7863236f0af595fe2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:22.987257 19710 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f3d7bd0ea32401fa8cb8ab3292a7808. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:22.991225 19714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 180230b77b7640b29d055ae8f4358ffe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:22.993211 19786 tablet_service.cc:1511] Processing CreateTablet for tablet 2eef64a7d31b4e1e899aee0857cb0e07 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:20:22.994521 19786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2eef64a7d31b4e1e899aee0857cb0e07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:22.995272 19709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f820e93ffdcd428ea4adfca88589a850. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:22.999257 19712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af881b097b4245b1be6072579197cb6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.001348 19708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d762eace063c4e04821c749d21702c3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.002025 19711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8cfec59c63741869c636df6356546d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.003119 19713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b2e4bf3fb5c456dab400b096f7bd729. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.014750 19706 tablet_service.cc:1511] Processing CreateTablet for tablet 2a411802fdaa42e7863236f0af595fe2 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:20:23.016336 19706 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a411802fdaa42e7863236f0af595fe2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.017452 19787 tablet_service.cc:1511] Processing CreateTablet for tablet 984fa8de2ccb49359c2b475e4adfc1ce (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:20:23.018904 19787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 984fa8de2ccb49359c2b475e4adfc1ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.032055 19785 tablet_service.cc:1511] Processing CreateTablet for tablet 85d493223fe04c939b6effd840955727 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:20:23.034060 19785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85d493223fe04c939b6effd840955727. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.048035 19786 tablet_service.cc:1511] Processing CreateTablet for tablet 353a546cf8c54671971c239502ef165a (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:20:23.049518 19786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 353a546cf8c54671971c239502ef165a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.049753 19938 tablet_service.cc:1511] Processing CreateTablet for tablet 180230b77b7640b29d055ae8f4358ffe (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:20:23.052134 19937 tablet_service.cc:1511] Processing CreateTablet for tablet 9b2e4bf3fb5c456dab400b096f7bd729 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:20:23.053659 19937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b2e4bf3fb5c456dab400b096f7bd729. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.060004 19932 tablet_service.cc:1511] Processing CreateTablet for tablet d762eace063c4e04821c749d21702c3f (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:20:23.061538 19932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d762eace063c4e04821c749d21702c3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.065526 19934 tablet_service.cc:1511] Processing CreateTablet for tablet 0f3d7bd0ea32401fa8cb8ab3292a7808 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:20:23.065622 19707 tablet_service.cc:1511] Processing CreateTablet for tablet 2eef64a7d31b4e1e899aee0857cb0e07 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:20:23.067220 19934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f3d7bd0ea32401fa8cb8ab3292a7808. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.068137 19707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2eef64a7d31b4e1e899aee0857cb0e07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.072134 19935 tablet_service.cc:1511] Processing CreateTablet for tablet e8cfec59c63741869c636df6356546d0 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:20:23.073834 19935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8cfec59c63741869c636df6356546d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.076248 19936 tablet_service.cc:1511] Processing CreateTablet for tablet 3cad7a5f46f9407a99ea6f2cc5a8322c (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:20:23.078059 19936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3cad7a5f46f9407a99ea6f2cc5a8322c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.080240 19931 tablet_service.cc:1511] Processing CreateTablet for tablet ac3bca5bb7644a8fbc78df43ef3331a0 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:20:23.081804 19931 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac3bca5bb7644a8fbc78df43ef3331a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.058521 19933 tablet_service.cc:1511] Processing CreateTablet for tablet 5d9f7b34cb064bbe8a6938e4f92d57c5 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:20:23.084079 19933 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d9f7b34cb064bbe8a6938e4f92d57c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.087177 19938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 180230b77b7640b29d055ae8f4358ffe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.092002 19786 tablet_service.cc:1511] Processing CreateTablet for tablet 65e7a5222efe43e49e1fbb3a3d7cf99c (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:20:23.092608 19930 tablet_service.cc:1511] Processing CreateTablet for tablet 19a59035bda747fdbc59b7e6914101dd (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:20:23.093379 19786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65e7a5222efe43e49e1fbb3a3d7cf99c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.094058 19930 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19a59035bda747fdbc59b7e6914101dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.095999 19929 tablet_service.cc:1511] Processing CreateTablet for tablet 2eef64a7d31b4e1e899aee0857cb0e07 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:20:23.097388 19929 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2eef64a7d31b4e1e899aee0857cb0e07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.103330 19713 tablet_service.cc:1511] Processing CreateTablet for tablet e804b7d8fd7e44f59516fb299bea7a2c (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
W20260709 13:20:23.108618 19610 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.9.116.129:41855, user_credentials={real_user=slave}} blocked reactor thread for 46443.1us
I20260709 13:20:23.109285 19993 tablet_bootstrap.cc:492] T f820e93ffdcd428ea4adfca88589a850 P e90ffed3ba174ababeb21d9415d4d918: Bootstrap starting.
I20260709 13:20:23.112339 19784 tablet_service.cc:1511] Processing CreateTablet for tablet ad38a1519fb748588f5a322589274004 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
W20260709 13:20:23.114106 19670 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:20:23.125931 19789 tablet_service.cc:1511] Processing CreateTablet for tablet 9b9137146e3644a6b0026778530fa8a6 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:20:23.148797 19784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad38a1519fb748588f5a322589274004. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:20:23.154929 19609 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.9.116.130:43319, user_credentials={real_user=slave}} blocked reactor thread for 58647.2us
I20260709 13:20:23.156209 19709 tablet_service.cc:1511] Processing CreateTablet for tablet 984fa8de2ccb49359c2b475e4adfc1ce (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:20:23.161616 19863 tablet_service.cc:1511] Processing CreateTablet for tablet 9b2e4bf3fb5c456dab400b096f7bd729 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:20:23.163138 19863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b2e4bf3fb5c456dab400b096f7bd729. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.164016 19862 tablet_service.cc:1511] Processing CreateTablet for tablet 3cad7a5f46f9407a99ea6f2cc5a8322c (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:20:23.165441 19862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3cad7a5f46f9407a99ea6f2cc5a8322c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.167994 19860 tablet_service.cc:1511] Processing CreateTablet for tablet 0f3d7bd0ea32401fa8cb8ab3292a7808 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:20:23.169509 19860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f3d7bd0ea32401fa8cb8ab3292a7808. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.169215 19858 tablet_service.cc:1511] Processing CreateTablet for tablet f820e93ffdcd428ea4adfca88589a850 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:20:23.170609 19858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f820e93ffdcd428ea4adfca88589a850. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.139097 19713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e804b7d8fd7e44f59516fb299bea7a2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.179951 19864 tablet_service.cc:1511] Processing CreateTablet for tablet 6a066b96902f41dbb1511eccf26820ce (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:20:23.180203 19933 tablet_service.cc:1511] Processing CreateTablet for tablet e804b7d8fd7e44f59516fb299bea7a2c (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:20:23.167994 19859 tablet_service.cc:1511] Processing CreateTablet for tablet 5d9f7b34cb064bbe8a6938e4f92d57c5 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:20:23.178241 19861 tablet_service.cc:1511] Processing CreateTablet for tablet af881b097b4245b1be6072579197cb6b (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:20:23.179961 19857 tablet_service.cc:1511] Processing CreateTablet for tablet ac3bca5bb7644a8fbc78df43ef3331a0 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:20:23.184770 19856 tablet_service.cc:1511] Processing CreateTablet for tablet 19a59035bda747fdbc59b7e6914101dd (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:20:23.186688 19855 tablet_service.cc:1511] Processing CreateTablet for tablet 2a411802fdaa42e7863236f0af595fe2 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:20:23.201233 19994 tablet_bootstrap.cc:492] T 6a066b96902f41dbb1511eccf26820ce P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:23.202172 19864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a066b96902f41dbb1511eccf26820ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.246856 19937 tablet_service.cc:1511] Processing CreateTablet for tablet 353a546cf8c54671971c239502ef165a (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:20:23.248098 19995 tablet_bootstrap.cc:492] T 9b2e4bf3fb5c456dab400b096f7bd729 P c62716bf66c24330860573535b8b9430: Bootstrap starting.
I20260709 13:20:23.251375 19861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af881b097b4245b1be6072579197cb6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.254979 19859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d9f7b34cb064bbe8a6938e4f92d57c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.255775 19857 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac3bca5bb7644a8fbc78df43ef3331a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.216010 19709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 984fa8de2ccb49359c2b475e4adfc1ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.218087 19781 tablet_service.cc:1511] Processing CreateTablet for tablet 5b9511148896441a948b772784f62a3d (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:20:23.218319 19782 tablet_service.cc:1511] Processing CreateTablet for tablet a323eb80c5084159803e7d4ea5d9c410 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:20:23.259568 19855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a411802fdaa42e7863236f0af595fe2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.263376 19995 tablet_bootstrap.cc:654] T 9b2e4bf3fb5c456dab400b096f7bd729 P c62716bf66c24330860573535b8b9430: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:23.264992 19856 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19a59035bda747fdbc59b7e6914101dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.225987 19930 tablet_service.cc:1511] Processing CreateTablet for tablet 85d493223fe04c939b6effd840955727 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:20:23.226624 19714 tablet_service.cc:1511] Processing CreateTablet for tablet 353a546cf8c54671971c239502ef165a (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:20:23.227586 19790 tablet_service.cc:1511] Processing CreateTablet for tablet a36c11f368f34bf0aa4925a099ac1f91 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:20:23.228904 19788 tablet_service.cc:1511] Processing CreateTablet for tablet 7e3a9d11df99411d8f0edb0df6f14d87 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:20:23.229018 19715 tablet_service.cc:1511] Processing CreateTablet for tablet ad38a1519fb748588f5a322589274004 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:20:23.230321 19789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b9137146e3644a6b0026778530fa8a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.231415 19711 tablet_service.cc:1511] Processing CreateTablet for tablet 9b9137146e3644a6b0026778530fa8a6 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:20:23.227094 19706 tablet_service.cc:1511] Processing CreateTablet for tablet 86964b8a61ce494f9096c4fefdc6efbf (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:20:23.232743 19787 tablet_service.cc:1511] Processing CreateTablet for tablet d1da95785dbf4ef4bde6f5e6ab35b444 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:20:23.283411 19993 tablet_bootstrap.cc:700] T f820e93ffdcd428ea4adfca88589a850 P e90ffed3ba174ababeb21d9415d4d918: Time spent opening tablet: real 0.110s	user 0.001s	sys 0.000s
I20260709 13:20:23.241320 19710 tablet_service.cc:1511] Processing CreateTablet for tablet 5b9511148896441a948b772784f62a3d (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:20:23.236311 19712 tablet_service.cc:1511] Processing CreateTablet for tablet 19e83b82855d4129bb8525e6ff05316b (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:20:23.283545 19787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1da95785dbf4ef4bde6f5e6ab35b444. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.299022 19790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a36c11f368f34bf0aa4925a099ac1f91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.300282 19993 tablet_bootstrap.cc:654] T f820e93ffdcd428ea4adfca88589a850 P e90ffed3ba174ababeb21d9415d4d918: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:23.302333 19994 tablet_bootstrap.cc:654] T 6a066b96902f41dbb1511eccf26820ce P 8df38c4c53c44e94b36c1e8992476d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:23.302496 19781 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b9511148896441a948b772784f62a3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.307137 19750 heartbeater.cc:499] Master 127.9.116.190:44649 was elected leader, sending a full tablet report...
I20260709 13:20:23.311039 19784 tablet_service.cc:1511] Processing CreateTablet for tablet 0c709292a0304b3c84a777427a754bb5 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:20:23.312568 19784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c709292a0304b3c84a777427a754bb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.312352 19938 tablet_service.cc:1511] Processing CreateTablet for tablet 86964b8a61ce494f9096c4fefdc6efbf (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:20:23.316670 19863 tablet_service.cc:1511] Processing CreateTablet for tablet a36c11f368f34bf0aa4925a099ac1f91 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:20:23.317768 19788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e3a9d11df99411d8f0edb0df6f14d87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.318116 19863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a36c11f368f34bf0aa4925a099ac1f91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.286149 19937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 353a546cf8c54671971c239502ef165a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.283564 19714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 353a546cf8c54671971c239502ef165a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.323184 19782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a323eb80c5084159803e7d4ea5d9c410. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.332882 19707 tablet_service.cc:1511] Processing CreateTablet for tablet 0587017774214f24bdfa4a3da36e367e (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:20:23.335201 19783 tablet_service.cc:1511] Processing CreateTablet for tablet e9369d9b7f5649be932967d24689a936 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:20:23.338312 19708 tablet_service.cc:1511] Processing CreateTablet for tablet a323eb80c5084159803e7d4ea5d9c410 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:20:23.340799 19858 tablet_service.cc:1511] Processing CreateTablet for tablet e804b7d8fd7e44f59516fb299bea7a2c (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:20:23.344475 19932 tablet_service.cc:1511] Processing CreateTablet for tablet 9b9137146e3644a6b0026778530fa8a6 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:20:23.340927 19935 tablet_service.cc:1511] Processing CreateTablet for tablet 65e7a5222efe43e49e1fbb3a3d7cf99c (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:20:23.358116 19783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9369d9b7f5649be932967d24689a936. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.345644 19929 tablet_service.cc:1511] Processing CreateTablet for tablet 19e83b82855d4129bb8525e6ff05316b (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:20:23.360327 19858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e804b7d8fd7e44f59516fb299bea7a2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.366403 19861 tablet_service.cc:1511] Processing CreateTablet for tablet 85d493223fe04c939b6effd840955727 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:20:23.367857 19861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85d493223fe04c939b6effd840955727. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.371582 19938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86964b8a61ce494f9096c4fefdc6efbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.347227 19934 tablet_service.cc:1511] Processing CreateTablet for tablet e9369d9b7f5649be932967d24689a936 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:20:23.376992 19932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b9137146e3644a6b0026778530fa8a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.377653 19929 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19e83b82855d4129bb8525e6ff05316b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.380616 19935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65e7a5222efe43e49e1fbb3a3d7cf99c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.388976 19995 tablet_bootstrap.cc:492] T 9b2e4bf3fb5c456dab400b096f7bd729 P c62716bf66c24330860573535b8b9430: No bootstrap required, opened a new log
I20260709 13:20:23.389643 19995 ts_tablet_manager.cc:1403] T 9b2e4bf3fb5c456dab400b096f7bd729 P c62716bf66c24330860573535b8b9430: Time spent bootstrapping tablet: real 0.142s	user 0.013s	sys 0.007s
I20260709 13:20:23.380110 19710 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b9511148896441a948b772784f62a3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.391067 19936 tablet_service.cc:1511] Processing CreateTablet for tablet a323eb80c5084159803e7d4ea5d9c410 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:20:23.399039 19931 tablet_service.cc:1511] Processing CreateTablet for tablet 0587017774214f24bdfa4a3da36e367e (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:20:23.401280 19862 tablet_service.cc:1511] Processing CreateTablet for tablet 984fa8de2ccb49359c2b475e4adfc1ce (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:20:23.402595 19862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 984fa8de2ccb49359c2b475e4adfc1ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.405956 19713 tablet_service.cc:1511] Processing CreateTablet for tablet 7e3a9d11df99411d8f0edb0df6f14d87 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:20:23.408166 19933 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e804b7d8fd7e44f59516fb299bea7a2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.408784 19706 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86964b8a61ce494f9096c4fefdc6efbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.411079 19786 tablet_service.cc:1511] Processing CreateTablet for tablet 8bdf042a063d4596970f49730ffa04b4 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:20:23.394335 19995 raft_consensus.cc:359] T 9b2e4bf3fb5c456dab400b096f7bd729 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:23.420030 19855 tablet_service.cc:1511] Processing CreateTablet for tablet 86964b8a61ce494f9096c4fefdc6efbf (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:20:23.422676 19993 tablet_bootstrap.cc:492] T f820e93ffdcd428ea4adfca88589a850 P e90ffed3ba174ababeb21d9415d4d918: No bootstrap required, opened a new log
I20260709 13:20:23.424932 19997 tablet_bootstrap.cc:492] T af881b097b4245b1be6072579197cb6b P b80b3a62d7d549a9a5c42def46769ac4: Bootstrap starting.
I20260709 13:20:23.431001 19786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bdf042a063d4596970f49730ffa04b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.424665 19995 raft_consensus.cc:385] T 9b2e4bf3fb5c456dab400b096f7bd729 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:23.442652 19825 heartbeater.cc:499] Master 127.9.116.190:44649 was elected leader, sending a full tablet report...
I20260709 13:20:23.447009 19993 ts_tablet_manager.cc:1403] T f820e93ffdcd428ea4adfca88589a850 P e90ffed3ba174ababeb21d9415d4d918: Time spent bootstrapping tablet: real 0.331s	user 0.017s	sys 0.010s
I20260709 13:20:23.460597 19993 raft_consensus.cc:359] T f820e93ffdcd428ea4adfca88589a850 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:23.461404 19993 raft_consensus.cc:385] T f820e93ffdcd428ea4adfca88589a850 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:23.461757 19993 raft_consensus.cc:740] T f820e93ffdcd428ea4adfca88589a850 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Initialized, Role: FOLLOWER
I20260709 13:20:23.441886 19995 raft_consensus.cc:740] T 9b2e4bf3fb5c456dab400b096f7bd729 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62716bf66c24330860573535b8b9430, State: Initialized, Role: FOLLOWER
I20260709 13:20:23.487409 19857 tablet_service.cc:1511] Processing CreateTablet for tablet 19e83b82855d4129bb8525e6ff05316b (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:20:23.487426 19995 consensus_queue.cc:260] T 9b2e4bf3fb5c456dab400b096f7bd729 P c62716bf66c24330860573535b8b9430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:23.471166 19856 tablet_service.cc:1511] Processing CreateTablet for tablet 65e7a5222efe43e49e1fbb3a3d7cf99c (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:20:23.474977 19855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86964b8a61ce494f9096c4fefdc6efbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.495118 19856 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65e7a5222efe43e49e1fbb3a3d7cf99c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.476589 19860 tablet_service.cc:1511] Processing CreateTablet for tablet ad38a1519fb748588f5a322589274004 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:20:23.465526 19712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19e83b82855d4129bb8525e6ff05316b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.464460 19931 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0587017774214f24bdfa4a3da36e367e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.491166 19863 tablet_service.cc:1511] Processing CreateTablet for tablet e9369d9b7f5649be932967d24689a936 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:20:23.513902 19864 tablet_service.cc:1511] Processing CreateTablet for tablet 5b9511148896441a948b772784f62a3d (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:20:23.515141 19713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e3a9d11df99411d8f0edb0df6f14d87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.462661 19993 consensus_queue.cc:260] T f820e93ffdcd428ea4adfca88589a850 P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:23.486447 19709 tablet_service.cc:1511] Processing CreateTablet for tablet d1da95785dbf4ef4bde6f5e6ab35b444 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:20:23.528426 19930 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85d493223fe04c939b6effd840955727. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.532955 19861 tablet_service.cc:1511] Processing CreateTablet for tablet 0587017774214f24bdfa4a3da36e367e (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:20:23.540097 19787 tablet_service.cc:1511] Processing CreateTablet for tablet 63868398f3604f529519d926d2831f4e (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:20:23.541098 19857 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19e83b82855d4129bb8525e6ff05316b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.541828 19936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a323eb80c5084159803e7d4ea5d9c410. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.541806 19859 tablet_service.cc:1511] Processing CreateTablet for tablet 7e3a9d11df99411d8f0edb0df6f14d87 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:20:23.553330 19994 tablet_bootstrap.cc:492] T 6a066b96902f41dbb1511eccf26820ce P 8df38c4c53c44e94b36c1e8992476d7c: No bootstrap required, opened a new log
I20260709 13:20:23.552374 19997 tablet_bootstrap.cc:654] T af881b097b4245b1be6072579197cb6b P b80b3a62d7d549a9a5c42def46769ac4: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:23.560571 19783 tablet_service.cc:1511] Processing CreateTablet for tablet f012387a39df446bbd9dbea32dc03202 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:20:23.573843 19994 ts_tablet_manager.cc:1403] T 6a066b96902f41dbb1511eccf26820ce P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.374s	user 0.013s	sys 0.031s
I20260709 13:20:23.574874 19995 ts_tablet_manager.cc:1434] T 9b2e4bf3fb5c456dab400b096f7bd729 P c62716bf66c24330860573535b8b9430: Time spent starting tablet: real 0.185s	user 0.026s	sys 0.052s
I20260709 13:20:23.547504 19899 heartbeater.cc:499] Master 127.9.116.190:44649 was elected leader, sending a full tablet report...
I20260709 13:20:23.575656 19973 heartbeater.cc:499] Master 127.9.116.190:44649 was elected leader, sending a full tablet report...
I20260709 13:20:23.576623 19995 tablet_bootstrap.cc:492] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P c62716bf66c24330860573535b8b9430: Bootstrap starting.
I20260709 13:20:23.578948 19994 raft_consensus.cc:359] T 6a066b96902f41dbb1511eccf26820ce P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:23.580233 19864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b9511148896441a948b772784f62a3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.614984 19994 raft_consensus.cc:385] T 6a066b96902f41dbb1511eccf26820ce P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:23.615397 19994 raft_consensus.cc:740] T 6a066b96902f41dbb1511eccf26820ce P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:23.581883 19787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63868398f3604f529519d926d2831f4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.589864 19709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1da95785dbf4ef4bde6f5e6ab35b444. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.616520 19783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f012387a39df446bbd9dbea32dc03202. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.618810 19708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a323eb80c5084159803e7d4ea5d9c410. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.601465 19993 ts_tablet_manager.cc:1434] T f820e93ffdcd428ea4adfca88589a850 P e90ffed3ba174ababeb21d9415d4d918: Time spent starting tablet: real 0.154s	user 0.009s	sys 0.001s
I20260709 13:20:23.623298 19711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b9137146e3644a6b0026778530fa8a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.604898 19710 tablet_service.cc:1511] Processing CreateTablet for tablet 7e3099f1888c4ed18c30523bf9ec94f5 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:20:23.624256 19993 tablet_bootstrap.cc:492] T e8cfec59c63741869c636df6356546d0 P e90ffed3ba174ababeb21d9415d4d918: Bootstrap starting.
I20260709 13:20:23.625013 19710 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e3099f1888c4ed18c30523bf9ec94f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.610549 19712 tablet_service.cc:1511] Processing CreateTablet for tablet 63868398f3604f529519d926d2831f4e (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:20:23.611478 19934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9369d9b7f5649be932967d24689a936. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.611739 19706 tablet_service.cc:1511] Processing CreateTablet for tablet 8bdf042a063d4596970f49730ffa04b4 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:20:23.615554 19863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9369d9b7f5649be932967d24689a936. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.629079 19715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad38a1519fb748588f5a322589274004. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:20:23.638715 19974 tablet.cc:2406] T 9b2e4bf3fb5c456dab400b096f7bd729 P c62716bf66c24330860573535b8b9430: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:20:23.641144 19706 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bdf042a063d4596970f49730ffa04b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.643739 19858 tablet_service.cc:1511] Processing CreateTablet for tablet 7e3099f1888c4ed18c30523bf9ec94f5 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:20:23.652356 19861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0587017774214f24bdfa4a3da36e367e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.657112 20000 raft_consensus.cc:493] T 9b2e4bf3fb5c456dab400b096f7bd729 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:23.657723 20000 raft_consensus.cc:515] T 9b2e4bf3fb5c456dab400b096f7bd729 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:23.651553 19994 consensus_queue.cc:260] T 6a066b96902f41dbb1511eccf26820ce P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:23.663199 20000 leader_election.cc:290] T 9b2e4bf3fb5c456dab400b096f7bd729 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:23.681789 19997 tablet_bootstrap.cc:492] T af881b097b4245b1be6072579197cb6b P b80b3a62d7d549a9a5c42def46769ac4: No bootstrap required, opened a new log
I20260709 13:20:23.682394 19997 ts_tablet_manager.cc:1403] T af881b097b4245b1be6072579197cb6b P b80b3a62d7d549a9a5c42def46769ac4: Time spent bootstrapping tablet: real 0.268s	user 0.019s	sys 0.017s
I20260709 13:20:23.683884 19932 tablet_service.cc:1511] Processing CreateTablet for tablet d1da95785dbf4ef4bde6f5e6ab35b444 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:20:23.685321 19932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1da95785dbf4ef4bde6f5e6ab35b444. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.686285 19997 raft_consensus.cc:359] T af881b097b4245b1be6072579197cb6b P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:23.687157 19997 raft_consensus.cc:385] T af881b097b4245b1be6072579197cb6b P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:23.687475 19997 raft_consensus.cc:740] T af881b097b4245b1be6072579197cb6b P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Initialized, Role: FOLLOWER
I20260709 13:20:23.654474 19929 tablet_service.cc:1511] Processing CreateTablet for tablet a36c11f368f34bf0aa4925a099ac1f91 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:20:23.692524 19859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e3a9d11df99411d8f0edb0df6f14d87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.686839 19707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0587017774214f24bdfa4a3da36e367e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.650169 19855 tablet_service.cc:1511] Processing CreateTablet for tablet 0c709292a0304b3c84a777427a754bb5 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:20:23.696174 19929 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a36c11f368f34bf0aa4925a099ac1f91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.717335 19997 consensus_queue.cc:260] T af881b097b4245b1be6072579197cb6b P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:23.725826 19937 tablet_service.cc:1511] Processing CreateTablet for tablet 7e3099f1888c4ed18c30523bf9ec94f5 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:20:23.726918 19712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63868398f3604f529519d926d2831f4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.737604 19860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad38a1519fb748588f5a322589274004. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.744715 19938 tablet_service.cc:1511] Processing CreateTablet for tablet 0c709292a0304b3c84a777427a754bb5 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:20:23.746058 19995 tablet_bootstrap.cc:654] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P c62716bf66c24330860573535b8b9430: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:23.746186 19938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c709292a0304b3c84a777427a754bb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.747327 19993 tablet_bootstrap.cc:654] T e8cfec59c63741869c636df6356546d0 P e90ffed3ba174ababeb21d9415d4d918: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:23.747109 19714 tablet_service.cc:1511] Processing CreateTablet for tablet 0413f22bd6d7447ca630b75be496555f (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:20:23.748703 19714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0413f22bd6d7447ca630b75be496555f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.751271 19937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e3099f1888c4ed18c30523bf9ec94f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.753872 19862 tablet_service.cc:1511] Processing CreateTablet for tablet 8bdf042a063d4596970f49730ffa04b4 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:20:23.756742 19786 tablet_service.cc:1511] Processing CreateTablet for tablet c393f9d18341477c97fac9a4af7e3adf (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:20:23.758281 19786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c393f9d18341477c97fac9a4af7e3adf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.770681 19788 tablet_service.cc:1511] Processing CreateTablet for tablet e3ca37daad8341ad99cbba926f37232e (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:20:23.772166 19788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3ca37daad8341ad99cbba926f37232e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.776342 19856 tablet_service.cc:1511] Processing CreateTablet for tablet 0413f22bd6d7447ca630b75be496555f (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:20:23.795717 19787 tablet_service.cc:1511] Processing CreateTablet for tablet 1fa8a0db61be45c3aa42edb11db4c5ff (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:20:23.797224 19787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fa8a0db61be45c3aa42edb11db4c5ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.797446 20004 raft_consensus.cc:493] T 6a066b96902f41dbb1511eccf26820ce P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:23.798022 20004 raft_consensus.cc:515] T 6a066b96902f41dbb1511eccf26820ce P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:23.801724 19863 tablet_service.cc:1511] Processing CreateTablet for tablet f012387a39df446bbd9dbea32dc03202 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:20:23.840711 20004 leader_election.cc:290] T 6a066b96902f41dbb1511eccf26820ce P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:23.809630 19855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c709292a0304b3c84a777427a754bb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.861532 19874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b2e4bf3fb5c456dab400b096f7bd729" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:23.861855 19931 tablet_service.cc:1511] Processing CreateTablet for tablet 63868398f3604f529519d926d2831f4e (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:20:23.863050 19997 ts_tablet_manager.cc:1434] T af881b097b4245b1be6072579197cb6b P b80b3a62d7d549a9a5c42def46769ac4: Time spent starting tablet: real 0.180s	user 0.031s	sys 0.030s
I20260709 13:20:23.865916 19997 tablet_bootstrap.cc:492] T 9b2e4bf3fb5c456dab400b096f7bd729 P b80b3a62d7d549a9a5c42def46769ac4: Bootstrap starting.
W20260709 13:20:23.871963 19913 leader_election.cc:343] T 9b2e4bf3fb5c456dab400b096f7bd729 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:23.859931 19715 tablet_service.cc:1511] Processing CreateTablet for tablet e3ca37daad8341ad99cbba926f37232e (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:20:23.873886 19715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3ca37daad8341ad99cbba926f37232e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.927547 19856 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0413f22bd6d7447ca630b75be496555f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.935003 20001 raft_consensus.cc:493] T f820e93ffdcd428ea4adfca88589a850 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:23.937937 20001 raft_consensus.cc:515] T f820e93ffdcd428ea4adfca88589a850 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:23.938167 19784 tablet_service.cc:1511] Processing CreateTablet for tablet 563940ad96444ce59e393ea303cc3352 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:20:23.939769 19784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 563940ad96444ce59e393ea303cc3352. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.949843 19933 tablet_service.cc:1511] Processing CreateTablet for tablet 0413f22bd6d7447ca630b75be496555f (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:20:23.951676 20001 leader_election.cc:290] T f820e93ffdcd428ea4adfca88589a850 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:23.906356 19706 tablet_service.cc:1511] Processing CreateTablet for tablet 3aaced67782041fa9960c745301a96e7 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:20:23.967631 19782 tablet_service.cc:1511] Processing CreateTablet for tablet d5f8619b1f9f46adaa41774af856d9b1 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:20:23.969105 19782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5f8619b1f9f46adaa41774af856d9b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.963876 19931 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63868398f3604f529519d926d2831f4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.944809 19708 tablet_service.cc:1511] Processing CreateTablet for tablet c393f9d18341477c97fac9a4af7e3adf (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:20:23.982462 19706 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3aaced67782041fa9960c745301a96e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:23.963311 19994 ts_tablet_manager.cc:1434] T 6a066b96902f41dbb1511eccf26820ce P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.386s	user 0.020s	sys 0.014s
I20260709 13:20:24.002967 19858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e3099f1888c4ed18c30523bf9ec94f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.028121 19993 tablet_bootstrap.cc:492] T e8cfec59c63741869c636df6356546d0 P e90ffed3ba174ababeb21d9415d4d918: No bootstrap required, opened a new log
I20260709 13:20:24.028635 20009 raft_consensus.cc:493] T af881b097b4245b1be6072579197cb6b P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:24.039033 20009 raft_consensus.cc:515] T af881b097b4245b1be6072579197cb6b P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:24.045061 19708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c393f9d18341477c97fac9a4af7e3adf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.045892 19859 tablet_service.cc:1511] Processing CreateTablet for tablet c393f9d18341477c97fac9a4af7e3adf (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:20:24.035719 19933 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0413f22bd6d7447ca630b75be496555f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.058612 19993 ts_tablet_manager.cc:1403] T e8cfec59c63741869c636df6356546d0 P e90ffed3ba174ababeb21d9415d4d918: Time spent bootstrapping tablet: real 0.433s	user 0.028s	sys 0.040s
I20260709 13:20:24.032716 19711 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38406 (request call id 16) took 1020 ms. Trace:
I20260709 13:20:24.080958 19713 tablet_service.cc:1511] Processing CreateTablet for tablet d5f8619b1f9f46adaa41774af856d9b1 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:20:24.082242 19709 tablet_service.cc:1511] Processing CreateTablet for tablet 563940ad96444ce59e393ea303cc3352 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:20:24.088120 19725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b2e4bf3fb5c456dab400b096f7bd729" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:24.092378 19935 tablet_service.cc:1511] Processing CreateTablet for tablet 3aaced67782041fa9960c745301a96e7 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:20:24.066365 19860 tablet_service.cc:1511] Processing CreateTablet for tablet 3aaced67782041fa9960c745301a96e7 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:20:24.093806 19935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3aaced67782041fa9960c745301a96e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.095775 19707 tablet_service.cc:1511] Processing CreateTablet for tablet 7a5476f5d68d496eaedce93e80bff5e4 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:20:24.097074 19707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a5476f5d68d496eaedce93e80bff5e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.099534 19864 tablet_service.cc:1511] Processing CreateTablet for tablet 1fa8a0db61be45c3aa42edb11db4c5ff (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:20:24.100124 19861 tablet_service.cc:1511] Processing CreateTablet for tablet 563940ad96444ce59e393ea303cc3352 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:20:24.099577 19934 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:57472 (request call id 17) took 1005 ms. Trace:
I20260709 13:20:24.101815 19934 rpcz_store.cc:276] 0709 13:20:23.094524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:23.346152 (+251628us) service_pool.cc:224] Handling call
0709 13:20:23.392614 (+ 46462us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:20:23.392735 (+   121us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:20:23.847164 (+454429us) tablet_metadata.cc:665] Metadata flushed
0709 13:20:24.099050 (+251886us) tablet_replica.cc:206] Creating consensus instance
0709 13:20:24.099517 (+   467us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:24.102582 19930 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:57472 (request call id 11) took 1010 ms. Trace:
I20260709 13:20:24.103492 19709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 563940ad96444ce59e393ea303cc3352. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.103848 19934 tablet_service.cc:1511] Processing CreateTablet for tablet d5f8619b1f9f46adaa41774af856d9b1 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:20:24.104278 19932 tablet_service.cc:1511] Processing CreateTablet for tablet f012387a39df446bbd9dbea32dc03202 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:20:24.103012 19930 rpcz_store.cc:276] 0709 13:20:23.092533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:23.225001 (+132468us) service_pool.cc:224] Handling call
0709 13:20:23.345139 (+120138us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:20:23.345246 (+   107us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:20:24.043808 (+698562us) tablet_metadata.cc:665] Metadata flushed
0709 13:20:24.093107 (+ 49299us) tablet_replica.cc:206] Creating consensus instance
0709 13:20:24.102523 (+  9416us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:24.084663 19863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f012387a39df446bbd9dbea32dc03202. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:20:24.112649 19914 leader_election.cc:343] T 9b2e4bf3fb5c456dab400b096f7bd729 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:24.107995 19994 tablet_bootstrap.cc:492] T ac3bca5bb7644a8fbc78df43ef3331a0 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:24.061017 20009 leader_election.cc:290] T af881b097b4245b1be6072579197cb6b P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:24.113198 19936 tablet_service.cc:1511] Processing CreateTablet for tablet e3ca37daad8341ad99cbba926f37232e (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:20:24.083931 19711 rpcz_store.cc:276] 0709 13:20:23.012464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:23.230476 (+218012us) service_pool.cc:224] Handling call
0709 13:20:23.285850 (+ 55374us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:20:23.285951 (+   101us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:20:23.821729 (+535778us) tablet_metadata.cc:665] Metadata flushed
0709 13:20:23.982994 (+161265us) tablet_replica.cc:206] Creating consensus instance
0709 13:20:24.024967 (+ 41973us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:24.118057 19914 leader_election.cc:304] T 9b2e4bf3fb5c456dab400b096f7bd729 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4
W20260709 13:20:24.119253 19670 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:20:24.122967 20000 raft_consensus.cc:2749] T 9b2e4bf3fb5c456dab400b096f7bd729 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:24.127897 19934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5f8619b1f9f46adaa41774af856d9b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.128657 19936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3ca37daad8341ad99cbba926f37232e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.142355 19993 raft_consensus.cc:359] T e8cfec59c63741869c636df6356546d0 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:24.143319 19713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5f8619b1f9f46adaa41774af856d9b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.143927 19930 tablet_service.cc:1511] Processing CreateTablet for tablet eb7bc0eba7e94b9eb1eeec9bbd1a6e97 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:20:24.145895 19725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f820e93ffdcd428ea4adfca88589a850" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:24.152127 19995 tablet_bootstrap.cc:492] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P c62716bf66c24330860573535b8b9430: No bootstrap required, opened a new log
I20260709 13:20:24.152757 19995 ts_tablet_manager.cc:1403] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P c62716bf66c24330860573535b8b9430: Time spent bootstrapping tablet: real 0.576s	user 0.031s	sys 0.023s
I20260709 13:20:24.153712 19994 tablet_bootstrap.cc:654] T ac3bca5bb7644a8fbc78df43ef3331a0 P 8df38c4c53c44e94b36c1e8992476d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:24.143303 19993 raft_consensus.cc:385] T e8cfec59c63741869c636df6356546d0 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:24.155404 19993 raft_consensus.cc:740] T e8cfec59c63741869c636df6356546d0 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Initialized, Role: FOLLOWER
I20260709 13:20:24.156242 19993 consensus_queue.cc:260] T e8cfec59c63741869c636df6356546d0 P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
W20260709 13:20:24.157766 19766 leader_election.cc:343] T f820e93ffdcd428ea4adfca88589a850 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:24.170238 19993 ts_tablet_manager.cc:1434] T e8cfec59c63741869c636df6356546d0 P e90ffed3ba174ababeb21d9415d4d918: Time spent starting tablet: real 0.096s	user 0.005s	sys 0.001s
I20260709 13:20:24.180291 19994 tablet_bootstrap.cc:492] T ac3bca5bb7644a8fbc78df43ef3331a0 P 8df38c4c53c44e94b36c1e8992476d7c: No bootstrap required, opened a new log
I20260709 13:20:24.180958 19994 ts_tablet_manager.cc:1403] T ac3bca5bb7644a8fbc78df43ef3331a0 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.117s	user 0.000s	sys 0.012s
I20260709 13:20:24.184288 19994 raft_consensus.cc:359] T ac3bca5bb7644a8fbc78df43ef3331a0 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:24.185072 19994 raft_consensus.cc:385] T ac3bca5bb7644a8fbc78df43ef3331a0 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:24.185393 19994 raft_consensus.cc:740] T ac3bca5bb7644a8fbc78df43ef3331a0 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:24.098493 19937 tablet_service.cc:1511] Processing CreateTablet for tablet 1fa8a0db61be45c3aa42edb11db4c5ff (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:20:24.204957 19938 tablet_service.cc:1511] Processing CreateTablet for tablet 7a5476f5d68d496eaedce93e80bff5e4 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:20:24.202366 19930 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb7bc0eba7e94b9eb1eeec9bbd1a6e97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.213181 19932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f012387a39df446bbd9dbea32dc03202. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.213073 19800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a066b96902f41dbb1511eccf26820ce" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:24.216135 19993 tablet_bootstrap.cc:492] T af881b097b4245b1be6072579197cb6b P e90ffed3ba174ababeb21d9415d4d918: Bootstrap starting.
I20260709 13:20:24.216938 19937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fa8a0db61be45c3aa42edb11db4c5ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.220085 19712 tablet_service.cc:1511] Processing CreateTablet for tablet 1cceb7f75d7040c2b98049590928b891 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:20:24.221463 19712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1cceb7f75d7040c2b98049590928b891. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.221554 19787 tablet_service.cc:1511] Processing CreateTablet for tablet dc5c36b41652443a8130592df8a66eb9 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:20:24.202350 19859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c393f9d18341477c97fac9a4af7e3adf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.223869 19929 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:57472 (request call id 20) took 1034 ms. Trace:
I20260709 13:20:24.224876 19860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3aaced67782041fa9960c745301a96e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.225535 19938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a5476f5d68d496eaedce93e80bff5e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.226475 19857 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:40862 (request call id 16) took 1040 ms. Trace:
I20260709 13:20:24.226899 19857 rpcz_store.cc:276] 0709 13:20:23.186030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:23.486427 (+300397us) service_pool.cc:224] Handling call
0709 13:20:23.488347 (+  1920us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:20:23.488450 (+   103us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:20:23.958427 (+469977us) tablet_metadata.cc:665] Metadata flushed
0709 13:20:24.225969 (+267542us) tablet_replica.cc:206] Creating consensus instance
0709 13:20:24.226414 (+   445us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:24.239682 19874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f820e93ffdcd428ea4adfca88589a850" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:24.225548 19995 raft_consensus.cc:359] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:24.240510 19995 raft_consensus.cc:385] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:24.240844 19995 raft_consensus.cc:740] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62716bf66c24330860573535b8b9430, State: Initialized, Role: FOLLOWER
W20260709 13:20:24.241531 19765 leader_election.cc:343] T f820e93ffdcd428ea4adfca88589a850 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:24.241658 19995 consensus_queue.cc:260] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P c62716bf66c24330860573535b8b9430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:24.242117 19765 leader_election.cc:304] T f820e93ffdcd428ea4adfca88589a850 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e90ffed3ba174ababeb21d9415d4d918; no voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4
I20260709 13:20:24.242947 20001 raft_consensus.cc:2749] T f820e93ffdcd428ea4adfca88589a850 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:24.243819 19861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 563940ad96444ce59e393ea303cc3352. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.244127 19782 tablet_service.cc:1511] Processing CreateTablet for tablet eb7bc0eba7e94b9eb1eeec9bbd1a6e97 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:20:24.249454 19787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc5c36b41652443a8130592df8a66eb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.224345 19929 rpcz_store.cc:276] 0709 13:20:23.189909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:23.653501 (+463592us) service_pool.cc:224] Handling call
0709 13:20:23.691534 (+ 38033us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:20:23.691670 (+   136us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:20:24.123882 (+432212us) tablet_metadata.cc:665] Metadata flushed
0709 13:20:24.214635 (+ 90753us) tablet_replica.cc:206] Creating consensus instance
0709 13:20:24.223808 (+  9173us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:24.255792 19929 tablet_service.cc:1511] Processing CreateTablet for tablet 1cceb7f75d7040c2b98049590928b891 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:20:24.257282 19929 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1cceb7f75d7040c2b98049590928b891. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.264045 19995 ts_tablet_manager.cc:1434] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P c62716bf66c24330860573535b8b9430: Time spent starting tablet: real 0.111s	user 0.004s	sys 0.004s
I20260709 13:20:24.264991 19997 tablet_bootstrap.cc:654] T 9b2e4bf3fb5c456dab400b096f7bd729 P b80b3a62d7d549a9a5c42def46769ac4: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:24.273170 19858 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:40862 (request call id 22) took 1051 ms. Trace:
W20260709 13:20:24.273940 19690 leader_election.cc:343] T 6a066b96902f41dbb1511eccf26820ce P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:24.273674 19858 rpcz_store.cc:276] 0709 13:20:23.221828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:23.642727 (+420899us) service_pool.cc:224] Handling call
0709 13:20:23.644739 (+  2012us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:20:23.644851 (+   112us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:20:24.265565 (+620714us) tablet_metadata.cc:665] Metadata flushed
0709 13:20:24.272650 (+  7085us) tablet_replica.cc:206] Creating consensus instance
0709 13:20:24.273103 (+   453us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:24.265215 19995 tablet_bootstrap.cc:492] T 19a59035bda747fdbc59b7e6914101dd P c62716bf66c24330860573535b8b9430: Bootstrap starting.
I20260709 13:20:24.281204 19864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fa8a0db61be45c3aa42edb11db4c5ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.282346 19786 tablet_service.cc:1511] Processing CreateTablet for tablet 1cceb7f75d7040c2b98049590928b891 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:20:24.284184 19786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1cceb7f75d7040c2b98049590928b891. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.285522 19713 tablet_service.cc:1511] Processing CreateTablet for tablet dc5c36b41652443a8130592df8a66eb9 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:20:24.290124 19862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bdf042a063d4596970f49730ffa04b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.218711 19994 consensus_queue.cc:260] T ac3bca5bb7644a8fbc78df43ef3331a0 P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:24.323750 19713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc5c36b41652443a8130592df8a66eb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.326354 19782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb7bc0eba7e94b9eb1eeec9bbd1a6e97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.329325 19994 ts_tablet_manager.cc:1434] T ac3bca5bb7644a8fbc78df43ef3331a0 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.148s	user 0.009s	sys 0.000s
I20260709 13:20:24.330502 19995 tablet_bootstrap.cc:654] T 19a59035bda747fdbc59b7e6914101dd P c62716bf66c24330860573535b8b9430: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:24.383103 19800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af881b097b4245b1be6072579197cb6b" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:24.332551 19994 tablet_bootstrap.cc:492] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:24.341174 19997 tablet_bootstrap.cc:492] T 9b2e4bf3fb5c456dab400b096f7bd729 P b80b3a62d7d549a9a5c42def46769ac4: No bootstrap required, opened a new log
I20260709 13:20:24.385005 19997 ts_tablet_manager.cc:1403] T 9b2e4bf3fb5c456dab400b096f7bd729 P b80b3a62d7d549a9a5c42def46769ac4: Time spent bootstrapping tablet: real 0.519s	user 0.026s	sys 0.016s
I20260709 13:20:24.390349 19715 tablet_service.cc:1511] Processing CreateTablet for tablet 0a9c6b64c8e04c4283d24ae6290a52ea (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:20:24.391906 19715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a9c6b64c8e04c4283d24ae6290a52ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.345844 19993 tablet_bootstrap.cc:654] T af881b097b4245b1be6072579197cb6b P e90ffed3ba174ababeb21d9415d4d918: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:24.347793 19858 tablet_service.cc:1511] Processing CreateTablet for tablet 7a5476f5d68d496eaedce93e80bff5e4 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:20:24.350577 19714 tablet_service.cc:1511] Processing CreateTablet for tablet ece79531bbd749d3b9300fe2fbeef9af (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:20:24.364650 19857 tablet_service.cc:1511] Processing CreateTablet for tablet dc5c36b41652443a8130592df8a66eb9 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:20:24.365880 19859 tablet_service.cc:1511] Processing CreateTablet for tablet eb7bc0eba7e94b9eb1eeec9bbd1a6e97 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:20:24.367241 19725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af881b097b4245b1be6072579197cb6b" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:24.368435 19874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a066b96902f41dbb1511eccf26820ce" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:24.371344 19861 tablet_service.cc:1511] Processing CreateTablet for tablet ece79531bbd749d3b9300fe2fbeef9af (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:20:24.388275 19997 raft_consensus.cc:359] T 9b2e4bf3fb5c456dab400b096f7bd729 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:24.349419 19933 tablet_service.cc:1511] Processing CreateTablet for tablet ece79531bbd749d3b9300fe2fbeef9af (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
W20260709 13:20:24.399085 19690 leader_election.cc:343] T 6a066b96902f41dbb1511eccf26820ce P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:24.399755 19690 leader_election.cc:304] T 6a066b96902f41dbb1511eccf26820ce P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c; no voters: b80b3a62d7d549a9a5c42def46769ac4, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:24.400992 20004 raft_consensus.cc:2749] T 6a066b96902f41dbb1511eccf26820ce P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:20:24.402921 19840 leader_election.cc:343] T af881b097b4245b1be6072579197cb6b P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:24.398907 19858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a5476f5d68d496eaedce93e80bff5e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.404057 19857 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc5c36b41652443a8130592df8a66eb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.405586 19994 tablet_bootstrap.cc:654] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:24.406644 19859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb7bc0eba7e94b9eb1eeec9bbd1a6e97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.399223 19997 raft_consensus.cc:385] T 9b2e4bf3fb5c456dab400b096f7bd729 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:24.399962 19933 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ece79531bbd749d3b9300fe2fbeef9af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.410981 19997 raft_consensus.cc:740] T 9b2e4bf3fb5c456dab400b096f7bd729 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Initialized, Role: FOLLOWER
I20260709 13:20:24.392256 19784 tablet_service.cc:1511] Processing CreateTablet for tablet 0a9c6b64c8e04c4283d24ae6290a52ea (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:20:24.351771 19788 tablet_service.cc:1511] Processing CreateTablet for tablet b832efa5a2e34b3894e33629110e6fef (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:20:24.412716 19784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a9c6b64c8e04c4283d24ae6290a52ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.413327 19788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b832efa5a2e34b3894e33629110e6fef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.403380 19714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ece79531bbd749d3b9300fe2fbeef9af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.419173 19855 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:40862 (request call id 23) took 1079 ms. Trace:
I20260709 13:20:24.419664 19855 rpcz_store.cc:276] 0709 13:20:23.339747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:23.649167 (+309420us) service_pool.cc:224] Handling call
0709 13:20:23.700263 (+ 51096us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:20:23.700450 (+   187us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:20:24.346875 (+646425us) tablet_metadata.cc:665] Metadata flushed
0709 13:20:24.418608 (+ 71733us) tablet_replica.cc:206] Creating consensus instance
0709 13:20:24.419107 (+   499us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:24.422938 19997 consensus_queue.cc:260] T 9b2e4bf3fb5c456dab400b096f7bd729 P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:24.425422 19997 ts_tablet_manager.cc:1434] T 9b2e4bf3fb5c456dab400b096f7bd729 P b80b3a62d7d549a9a5c42def46769ac4: Time spent starting tablet: real 0.040s	user 0.001s	sys 0.005s
I20260709 13:20:24.426568 19997 tablet_bootstrap.cc:492] T f820e93ffdcd428ea4adfca88589a850 P b80b3a62d7d549a9a5c42def46769ac4: Bootstrap starting.
W20260709 13:20:24.435510 19839 leader_election.cc:343] T af881b097b4245b1be6072579197cb6b P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:24.436064 19839 leader_election.cc:304] T af881b097b4245b1be6072579197cb6b P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4; no voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:24.436878 20009 raft_consensus.cc:2749] T af881b097b4245b1be6072579197cb6b P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:24.434808 19709 tablet_service.cc:1511] Processing CreateTablet for tablet 8abfa1aa35c44699960a8166fa79c41d (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:20:24.440485 20001 raft_consensus.cc:493] T e8cfec59c63741869c636df6356546d0 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:24.441042 20001 raft_consensus.cc:515] T e8cfec59c63741869c636df6356546d0 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:24.441439 19709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8abfa1aa35c44699960a8166fa79c41d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.443286 19861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ece79531bbd749d3b9300fe2fbeef9af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.443643 20001 leader_election.cc:290] T e8cfec59c63741869c636df6356546d0 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:24.445200 19855 tablet_service.cc:1511] Processing CreateTablet for tablet 0a9c6b64c8e04c4283d24ae6290a52ea (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:20:24.446580 19855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a9c6b64c8e04c4283d24ae6290a52ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.461041 19929 tablet_service.cc:1511] Processing CreateTablet for tablet 8abfa1aa35c44699960a8166fa79c41d (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:20:24.462478 19929 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8abfa1aa35c44699960a8166fa79c41d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.459861 19725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8cfec59c63741869c636df6356546d0" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
W20260709 13:20:24.468916 19766 leader_election.cc:343] T e8cfec59c63741869c636df6356546d0 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:24.477828 19856 tablet_service.cc:1511] Processing CreateTablet for tablet b832efa5a2e34b3894e33629110e6fef (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:20:24.479483 19934 tablet_service.cc:1511] Processing CreateTablet for tablet b832efa5a2e34b3894e33629110e6fef (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:20:24.480921 19934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b832efa5a2e34b3894e33629110e6fef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.489665 19995 tablet_bootstrap.cc:492] T 19a59035bda747fdbc59b7e6914101dd P c62716bf66c24330860573535b8b9430: No bootstrap required, opened a new log
I20260709 13:20:24.490263 19995 ts_tablet_manager.cc:1403] T 19a59035bda747fdbc59b7e6914101dd P c62716bf66c24330860573535b8b9430: Time spent bootstrapping tablet: real 0.225s	user 0.016s	sys 0.000s
I20260709 13:20:24.491952 19782 tablet_service.cc:1511] Processing CreateTablet for tablet 8abfa1aa35c44699960a8166fa79c41d (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:20:24.495358 19782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8abfa1aa35c44699960a8166fa79c41d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.494289 19993 tablet_bootstrap.cc:492] T af881b097b4245b1be6072579197cb6b P e90ffed3ba174ababeb21d9415d4d918: No bootstrap required, opened a new log
I20260709 13:20:24.493546 19995 raft_consensus.cc:359] T 19a59035bda747fdbc59b7e6914101dd P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:24.503921 19856 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b832efa5a2e34b3894e33629110e6fef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.502698 19995 raft_consensus.cc:385] T 19a59035bda747fdbc59b7e6914101dd P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:24.504719 19995 raft_consensus.cc:740] T 19a59035bda747fdbc59b7e6914101dd P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62716bf66c24330860573535b8b9430, State: Initialized, Role: FOLLOWER
I20260709 13:20:24.505605 19995 consensus_queue.cc:260] T 19a59035bda747fdbc59b7e6914101dd P c62716bf66c24330860573535b8b9430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:24.505728 19860 tablet_service.cc:1511] Processing CreateTablet for tablet bc04a038495940c1bbeda4a2e6f17b82 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:20:24.507232 19860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc04a038495940c1bbeda4a2e6f17b82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.510984 19862 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:40862 (request call id 24) took 1085 ms. Trace:
I20260709 13:20:24.511502 19862 rpcz_store.cc:276] 0709 13:20:23.426091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:23.752887 (+326796us) service_pool.cc:224] Handling call
0709 13:20:23.772629 (+ 19742us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:20:23.772764 (+   135us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:20:24.382408 (+609644us) tablet_metadata.cc:665] Metadata flushed
0709 13:20:24.510384 (+127976us) tablet_replica.cc:206] Creating consensus instance
0709 13:20:24.510914 (+   530us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:24.515674 19995 ts_tablet_manager.cc:1434] T 19a59035bda747fdbc59b7e6914101dd P c62716bf66c24330860573535b8b9430: Time spent starting tablet: real 0.025s	user 0.009s	sys 0.000s
I20260709 13:20:24.525946 19997 tablet_bootstrap.cc:654] T f820e93ffdcd428ea4adfca88589a850 P b80b3a62d7d549a9a5c42def46769ac4: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:24.527112 19948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8cfec59c63741869c636df6356546d0" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
W20260709 13:20:24.529047 19765 leader_election.cc:343] T e8cfec59c63741869c636df6356546d0 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c62716bf66c24330860573535b8b9430 (127.9.116.132:42643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:24.529693 19765 leader_election.cc:304] T e8cfec59c63741869c636df6356546d0 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e90ffed3ba174ababeb21d9415d4d918; no voters: 8df38c4c53c44e94b36c1e8992476d7c, c62716bf66c24330860573535b8b9430
I20260709 13:20:24.532096 20001 raft_consensus.cc:2749] T e8cfec59c63741869c636df6356546d0 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:24.541077 19994 tablet_bootstrap.cc:492] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c: No bootstrap required, opened a new log
I20260709 13:20:24.541913 19994 ts_tablet_manager.cc:1403] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.210s	user 0.040s	sys 0.017s
I20260709 13:20:24.542956 19995 tablet_bootstrap.cc:492] T 180230b77b7640b29d055ae8f4358ffe P c62716bf66c24330860573535b8b9430: Bootstrap starting.
I20260709 13:20:24.533587 19993 ts_tablet_manager.cc:1403] T af881b097b4245b1be6072579197cb6b P e90ffed3ba174ababeb21d9415d4d918: Time spent bootstrapping tablet: real 0.318s	user 0.013s	sys 0.014s
I20260709 13:20:24.546600 19994 raft_consensus.cc:359] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:24.547840 19994 raft_consensus.cc:385] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:24.547981 19929 tablet_service.cc:1511] Processing CreateTablet for tablet bc04a038495940c1bbeda4a2e6f17b82 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:20:24.548350 19994 raft_consensus.cc:740] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:24.549443 19929 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc04a038495940c1bbeda4a2e6f17b82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.549683 19994 consensus_queue.cc:260] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:24.552821 19994 ts_tablet_manager.cc:1434] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.011s	user 0.000s	sys 0.009s
I20260709 13:20:24.553937 19994 tablet_bootstrap.cc:492] T f820e93ffdcd428ea4adfca88589a850 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:24.563104 19994 tablet_bootstrap.cc:654] T f820e93ffdcd428ea4adfca88589a850 P 8df38c4c53c44e94b36c1e8992476d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:24.571312 19995 tablet_bootstrap.cc:654] T 180230b77b7640b29d055ae8f4358ffe P c62716bf66c24330860573535b8b9430: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:24.573735 19714 tablet_service.cc:1511] Processing CreateTablet for tablet bc04a038495940c1bbeda4a2e6f17b82 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:20:24.575186 19788 tablet_service.cc:1511] Processing CreateTablet for tablet 87364a1289ed4aa1b9b2a7a9951ed776 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:20:24.577095 19788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87364a1289ed4aa1b9b2a7a9951ed776. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.578874 19857 tablet_service.cc:1511] Processing CreateTablet for tablet 87364a1289ed4aa1b9b2a7a9951ed776 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:20:24.580732 19857 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87364a1289ed4aa1b9b2a7a9951ed776. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.575219 19714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc04a038495940c1bbeda4a2e6f17b82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.587411 20004 raft_consensus.cc:493] T ac3bca5bb7644a8fbc78df43ef3331a0 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:24.588008 20004 raft_consensus.cc:515] T ac3bca5bb7644a8fbc78df43ef3331a0 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:24.590420 20004 leader_election.cc:290] T ac3bca5bb7644a8fbc78df43ef3331a0 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:24.596616 20004 raft_consensus.cc:493] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:24.588532 19997 tablet_bootstrap.cc:492] T f820e93ffdcd428ea4adfca88589a850 P b80b3a62d7d549a9a5c42def46769ac4: No bootstrap required, opened a new log
I20260709 13:20:24.598106 19997 ts_tablet_manager.cc:1403] T f820e93ffdcd428ea4adfca88589a850 P b80b3a62d7d549a9a5c42def46769ac4: Time spent bootstrapping tablet: real 0.172s	user 0.023s	sys 0.024s
I20260709 13:20:24.594187 19874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3bca5bb7644a8fbc78df43ef3331a0" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
W20260709 13:20:24.602252 19690 leader_election.cc:343] T ac3bca5bb7644a8fbc78df43ef3331a0 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:24.602761 19948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3bca5bb7644a8fbc78df43ef3331a0" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:24.597631 20004 raft_consensus.cc:515] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
W20260709 13:20:24.607187 19690 leader_election.cc:343] T ac3bca5bb7644a8fbc78df43ef3331a0 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c62716bf66c24330860573535b8b9430 (127.9.116.132:42643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:24.607662 19690 leader_election.cc:304] T ac3bca5bb7644a8fbc78df43ef3331a0 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c; no voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430
I20260709 13:20:24.610699 19874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b2e4bf3fb5c456dab400b096f7bd729" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:24.626541 19715 tablet_service.cc:1511] Processing CreateTablet for tablet cb33acfa7a9f46ab95b0e77e1394f258 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:20:24.631703 19874 raft_consensus.cc:2468] T 9b2e4bf3fb5c456dab400b096f7bd729 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 0.
I20260709 13:20:24.632619 19715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb33acfa7a9f46ab95b0e77e1394f258. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.633230 19690 leader_election.cc:304] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4; no voters: 
I20260709 13:20:24.610378 19933 tablet_service.cc:1511] Processing CreateTablet for tablet 87364a1289ed4aa1b9b2a7a9951ed776 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:20:24.610924 19997 raft_consensus.cc:359] T f820e93ffdcd428ea4adfca88589a850 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:24.635989 19997 raft_consensus.cc:385] T f820e93ffdcd428ea4adfca88589a850 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:24.636158 19933 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87364a1289ed4aa1b9b2a7a9951ed776. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.616729 19784 tablet_service.cc:1511] Processing CreateTablet for tablet cb33acfa7a9f46ab95b0e77e1394f258 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:20:24.638880 19997 raft_consensus.cc:740] T f820e93ffdcd428ea4adfca88589a850 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Initialized, Role: FOLLOWER
I20260709 13:20:24.637640 19937 tablet_service.cc:1511] Processing CreateTablet for tablet bd80f9f8533b4cd3b5861ec11917a03d (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:20:24.609525 20004 leader_election.cc:290] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:24.623683 19861 tablet_service.cc:1511] Processing CreateTablet for tablet cb33acfa7a9f46ab95b0e77e1394f258 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:20:24.644022 19787 tablet_service.cc:1511] Processing CreateTablet for tablet bd80f9f8533b4cd3b5861ec11917a03d (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:20:24.644518 19861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb33acfa7a9f46ab95b0e77e1394f258. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.614598 19948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b2e4bf3fb5c456dab400b096f7bd729" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:24.645354 19948 raft_consensus.cc:2468] T 9b2e4bf3fb5c456dab400b096f7bd729 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 0.
I20260709 13:20:24.650697 19787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd80f9f8533b4cd3b5861ec11917a03d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.651186 19858 tablet_service.cc:1511] Processing CreateTablet for tablet 9f60d4b859264720a20f5670c3c71a6f (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:20:24.651825 19930 tablet_service.cc:1511] Processing CreateTablet for tablet 9f60d4b859264720a20f5670c3c71a6f (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:20:24.652477 19858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f60d4b859264720a20f5670c3c71a6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.647930 20004 raft_consensus.cc:2749] T ac3bca5bb7644a8fbc78df43ef3331a0 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:24.653331 19997 consensus_queue.cc:260] T f820e93ffdcd428ea4adfca88589a850 P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:24.655315 19937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd80f9f8533b4cd3b5861ec11917a03d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.655992 19930 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f60d4b859264720a20f5670c3c71a6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.655367 20031 raft_consensus.cc:2804] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:24.656884 20031 raft_consensus.cc:493] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:24.657243 20031 raft_consensus.cc:3060] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:24.662345 19784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb33acfa7a9f46ab95b0e77e1394f258. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.664558 19997 ts_tablet_manager.cc:1434] T f820e93ffdcd428ea4adfca88589a850 P b80b3a62d7d549a9a5c42def46769ac4: Time spent starting tablet: real 0.066s	user 0.010s	sys 0.002s
I20260709 13:20:24.676002 19997 tablet_bootstrap.cc:492] T 3cad7a5f46f9407a99ea6f2cc5a8322c P b80b3a62d7d549a9a5c42def46769ac4: Bootstrap starting.
I20260709 13:20:24.672228 19993 raft_consensus.cc:359] T af881b097b4245b1be6072579197cb6b P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:24.683540 19993 raft_consensus.cc:385] T af881b097b4245b1be6072579197cb6b P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:24.684453 19993 raft_consensus.cc:740] T af881b097b4245b1be6072579197cb6b P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Initialized, Role: FOLLOWER
I20260709 13:20:24.687716 19993 consensus_queue.cc:260] T af881b097b4245b1be6072579197cb6b P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:24.683838 19995 tablet_bootstrap.cc:492] T 180230b77b7640b29d055ae8f4358ffe P c62716bf66c24330860573535b8b9430: No bootstrap required, opened a new log
I20260709 13:20:24.695425 19995 ts_tablet_manager.cc:1403] T 180230b77b7640b29d055ae8f4358ffe P c62716bf66c24330860573535b8b9430: Time spent bootstrapping tablet: real 0.163s	user 0.032s	sys 0.018s
I20260709 13:20:24.693471 19997 tablet_bootstrap.cc:654] T 3cad7a5f46f9407a99ea6f2cc5a8322c P b80b3a62d7d549a9a5c42def46769ac4: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:24.705384 19788 tablet_service.cc:1511] Processing CreateTablet for tablet 29f92a880ce04259ae0ee9c5288949ff (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:20:24.709223 19788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29f92a880ce04259ae0ee9c5288949ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.712939 20000 raft_consensus.cc:493] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:24.713515 20000 raft_consensus.cc:515] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:24.724102 19855 tablet_service.cc:1511] Processing CreateTablet for tablet 29f92a880ce04259ae0ee9c5288949ff (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:20:24.725530 19855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29f92a880ce04259ae0ee9c5288949ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.725857 19929 tablet_service.cc:1511] Processing CreateTablet for tablet 29f92a880ce04259ae0ee9c5288949ff (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:20:24.727183 19929 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29f92a880ce04259ae0ee9c5288949ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.728246 19995 raft_consensus.cc:359] T 180230b77b7640b29d055ae8f4358ffe P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:24.729048 19995 raft_consensus.cc:385] T 180230b77b7640b29d055ae8f4358ffe P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:24.729349 19995 raft_consensus.cc:740] T 180230b77b7640b29d055ae8f4358ffe P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62716bf66c24330860573535b8b9430, State: Initialized, Role: FOLLOWER
I20260709 13:20:24.730052 19995 consensus_queue.cc:260] T 180230b77b7640b29d055ae8f4358ffe P c62716bf66c24330860573535b8b9430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:24.732836 19859 tablet_service.cc:1511] Processing CreateTablet for tablet 59fc86c7e9f648e3b3f51e936e2bcfec (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:20:24.734210 19859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59fc86c7e9f648e3b3f51e936e2bcfec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.734421 19787 tablet_service.cc:1511] Processing CreateTablet for tablet 59fc86c7e9f648e3b3f51e936e2bcfec (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:20:24.735772 19782 tablet_service.cc:1511] Processing CreateTablet for tablet 6551866a22dd4aaf8ee318b9d91b3a02 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
W20260709 13:20:24.719576 19690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45249, user_credentials={real_user=slave}} blocked reactor thread for 86464.3us
I20260709 13:20:24.747362 19934 tablet_service.cc:1511] Processing CreateTablet for tablet 8fd58e41ecee4779b3e72df3aea4463a (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:20:24.737821 19860 tablet_service.cc:1511] Processing CreateTablet for tablet 8fd58e41ecee4779b3e72df3aea4463a (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:20:24.748781 19934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fd58e41ecee4779b3e72df3aea4463a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.749626 19860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fd58e41ecee4779b3e72df3aea4463a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.749794 19709 tablet_service.cc:1511] Processing CreateTablet for tablet 9f60d4b859264720a20f5670c3c71a6f (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:20:24.747355 19930 tablet_service.cc:1511] Processing CreateTablet for tablet 6551866a22dd4aaf8ee318b9d91b3a02 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:20:24.752398 19930 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6551866a22dd4aaf8ee318b9d91b3a02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.754424 19782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6551866a22dd4aaf8ee318b9d91b3a02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.754966 20009 raft_consensus.cc:493] T f820e93ffdcd428ea4adfca88589a850 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:24.755481 20009 raft_consensus.cc:515] T f820e93ffdcd428ea4adfca88589a850 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:24.756399 20000 leader_election.cc:290] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:24.757875 20009 leader_election.cc:290] T f820e93ffdcd428ea4adfca88589a850 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:24.755277 20031 raft_consensus.cc:515] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:24.759236 19800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f820e93ffdcd428ea4adfca88589a850" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:24.759579 19725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f820e93ffdcd428ea4adfca88589a850" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:24.759918 19800 raft_consensus.cc:2468] T f820e93ffdcd428ea4adfca88589a850 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 0.
I20260709 13:20:24.760740 20031 leader_election.cc:290] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Requested vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
W20260709 13:20:24.761075 19840 leader_election.cc:343] T f820e93ffdcd428ea4adfca88589a850 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:24.762184 19839 leader_election.cc:304] T f820e93ffdcd428ea4adfca88589a850 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, e90ffed3ba174ababeb21d9415d4d918; no voters: 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:24.762420 19994 tablet_bootstrap.cc:492] T f820e93ffdcd428ea4adfca88589a850 P 8df38c4c53c44e94b36c1e8992476d7c: No bootstrap required, opened a new log
I20260709 13:20:24.763579 19994 ts_tablet_manager.cc:1403] T f820e93ffdcd428ea4adfca88589a850 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.210s	user 0.013s	sys 0.010s
I20260709 13:20:24.751106 19709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f60d4b859264720a20f5670c3c71a6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.748623 19714 tablet_service.cc:1511] Processing CreateTablet for tablet bd80f9f8533b4cd3b5861ec11917a03d (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:20:24.768061 19714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd80f9f8533b4cd3b5861ec11917a03d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.770974 20009 raft_consensus.cc:2804] T f820e93ffdcd428ea4adfca88589a850 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:24.771117 19787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59fc86c7e9f648e3b3f51e936e2bcfec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.771485 20009 raft_consensus.cc:493] T f820e93ffdcd428ea4adfca88589a850 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:24.771905 20009 raft_consensus.cc:3060] T f820e93ffdcd428ea4adfca88589a850 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:24.778903 19994 raft_consensus.cc:359] T f820e93ffdcd428ea4adfca88589a850 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:24.779748 19994 raft_consensus.cc:385] T f820e93ffdcd428ea4adfca88589a850 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:24.780045 19994 raft_consensus.cc:740] T f820e93ffdcd428ea4adfca88589a850 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:24.781332 19713 tablet_service.cc:1511] Processing CreateTablet for tablet 59fc86c7e9f648e3b3f51e936e2bcfec (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:20:24.782706 19713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59fc86c7e9f648e3b3f51e936e2bcfec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.789170 19993 ts_tablet_manager.cc:1434] T af881b097b4245b1be6072579197cb6b P e90ffed3ba174ababeb21d9415d4d918: Time spent starting tablet: real 0.244s	user 0.005s	sys 0.004s
I20260709 13:20:24.790418 19993 tablet_bootstrap.cc:492] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P e90ffed3ba174ababeb21d9415d4d918: Bootstrap starting.
I20260709 13:20:24.799844 19715 tablet_service.cc:1511] Processing CreateTablet for tablet 6551866a22dd4aaf8ee318b9d91b3a02 (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:20:24.801513 19715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6551866a22dd4aaf8ee318b9d91b3a02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.803077 19994 consensus_queue.cc:260] T f820e93ffdcd428ea4adfca88589a850 P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:24.804492 20000 raft_consensus.cc:493] T 19a59035bda747fdbc59b7e6914101dd P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:24.805016 20000 raft_consensus.cc:515] T 19a59035bda747fdbc59b7e6914101dd P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:24.805676 19994 ts_tablet_manager.cc:1434] T f820e93ffdcd428ea4adfca88589a850 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.042s	user 0.003s	sys 0.003s
I20260709 13:20:24.806181 19712 tablet_service.cc:1511] Processing CreateTablet for tablet 8fd58e41ecee4779b3e72df3aea4463a (DEFAULT_TABLE table=test-workload [id=0ace7c704ec8470687eec4a3c1f5345f]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:20:24.807186 20000 leader_election.cc:290] T 19a59035bda747fdbc59b7e6914101dd P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:24.807160 19800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9f7b34cb064bbe8a6938e4f92d57c5" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:24.808247 19874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b2e4bf3fb5c456dab400b096f7bd729" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
I20260709 13:20:24.808835 19874 raft_consensus.cc:3060] T 9b2e4bf3fb5c456dab400b096f7bd729 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:24.809367 19873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9f7b34cb064bbe8a6938e4f92d57c5" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:24.813728 19800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19a59035bda747fdbc59b7e6914101dd" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:24.814133 19712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fd58e41ecee4779b3e72df3aea4463a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:24.818495 19994 tablet_bootstrap.cc:492] T af881b097b4245b1be6072579197cb6b P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:24.823906 19948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b2e4bf3fb5c456dab400b096f7bd729" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:24.827179 19948 raft_consensus.cc:3060] T 9b2e4bf3fb5c456dab400b096f7bd729 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:24.828047 19873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19a59035bda747fdbc59b7e6914101dd" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:24.833597 20001 raft_consensus.cc:493] T af881b097b4245b1be6072579197cb6b P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:24.834182 20001 raft_consensus.cc:515] T af881b097b4245b1be6072579197cb6b P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
W20260709 13:20:24.834908 19913 leader_election.cc:343] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:20:24.835711 19913 leader_election.cc:343] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:24.836133 19913 leader_election.cc:304] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c62716bf66c24330860573535b8b9430; no voters: b80b3a62d7d549a9a5c42def46769ac4, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:24.836521 20001 leader_election.cc:290] T af881b097b4245b1be6072579197cb6b P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
W20260709 13:20:24.837335 19913 leader_election.cc:343] T 19a59035bda747fdbc59b7e6914101dd P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:24.837463 19725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af881b097b4245b1be6072579197cb6b" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
W20260709 13:20:24.838013 19913 leader_election.cc:343] T 19a59035bda747fdbc59b7e6914101dd P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:24.838117 19873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af881b097b4245b1be6072579197cb6b" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:24.838804 19873 raft_consensus.cc:2468] T af881b097b4245b1be6072579197cb6b P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 0.
I20260709 13:20:24.834275 20009 raft_consensus.cc:515] T f820e93ffdcd428ea4adfca88589a850 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
W20260709 13:20:24.839226 19766 leader_election.cc:343] T af881b097b4245b1be6072579197cb6b P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:24.840133 19765 leader_election.cc:304] T af881b097b4245b1be6072579197cb6b P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, e90ffed3ba174ababeb21d9415d4d918; no voters: 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:24.841362 20001 raft_consensus.cc:2804] T af881b097b4245b1be6072579197cb6b P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:24.841521 20009 leader_election.cc:290] T f820e93ffdcd428ea4adfca88589a850 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 election: Requested vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:24.841822 20001 raft_consensus.cc:493] T af881b097b4245b1be6072579197cb6b P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:24.842213 20001 raft_consensus.cc:3060] T af881b097b4245b1be6072579197cb6b P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:24.843132 20035 raft_consensus.cc:2749] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:24.847052 19800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f820e93ffdcd428ea4adfca88589a850" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
I20260709 13:20:24.847615 19725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f820e93ffdcd428ea4adfca88589a850" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
I20260709 13:20:24.847720 19800 raft_consensus.cc:3060] T f820e93ffdcd428ea4adfca88589a850 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:24.848521 19725 raft_consensus.cc:3060] T f820e93ffdcd428ea4adfca88589a850 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:24.838442 19913 leader_election.cc:304] T 19a59035bda747fdbc59b7e6914101dd P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c62716bf66c24330860573535b8b9430; no voters: b80b3a62d7d549a9a5c42def46769ac4, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:24.852028 19874 raft_consensus.cc:2468] T 9b2e4bf3fb5c456dab400b096f7bd729 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 1.
I20260709 13:20:24.852703 20035 raft_consensus.cc:2749] T 19a59035bda747fdbc59b7e6914101dd P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:24.857234 19800 raft_consensus.cc:2468] T f820e93ffdcd428ea4adfca88589a850 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 1.
I20260709 13:20:24.858238 19948 raft_consensus.cc:2468] T 9b2e4bf3fb5c456dab400b096f7bd729 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 1.
I20260709 13:20:24.863983 19839 leader_election.cc:304] T f820e93ffdcd428ea4adfca88589a850 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, e90ffed3ba174ababeb21d9415d4d918; no voters: 
I20260709 13:20:24.861622 20001 raft_consensus.cc:515] T af881b097b4245b1be6072579197cb6b P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:24.864514 19690 leader_election.cc:304] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4; no voters: 
I20260709 13:20:24.864953 20009 raft_consensus.cc:2804] T f820e93ffdcd428ea4adfca88589a850 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:24.866181 20031 raft_consensus.cc:2804] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:24.866761 20031 raft_consensus.cc:697] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 LEADER]: Becoming Leader. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Running, Role: LEADER
I20260709 13:20:24.867530 20001 leader_election.cc:290] T af881b097b4245b1be6072579197cb6b P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 election: Requested vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:24.867580 19874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af881b097b4245b1be6072579197cb6b" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
I20260709 13:20:24.871701 19724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af881b097b4245b1be6072579197cb6b" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
I20260709 13:20:24.867756 20031 consensus_queue.cc:237] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
W20260709 13:20:24.873406 19766 leader_election.cc:343] T af881b097b4245b1be6072579197cb6b P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:24.873974 19995 ts_tablet_manager.cc:1434] T 180230b77b7640b29d055ae8f4358ffe P c62716bf66c24330860573535b8b9430: Time spent starting tablet: real 0.177s	user 0.008s	sys 0.007s
I20260709 13:20:24.875406 19995 tablet_bootstrap.cc:492] T e8cfec59c63741869c636df6356546d0 P c62716bf66c24330860573535b8b9430: Bootstrap starting.
I20260709 13:20:24.869262 19874 raft_consensus.cc:3060] T af881b097b4245b1be6072579197cb6b P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:24.893455 19725 raft_consensus.cc:2468] T f820e93ffdcd428ea4adfca88589a850 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 1.
I20260709 13:20:24.897194 19622 catalog_manager.cc:5697] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c reported cstate change: term changed from 0 to 1, leader changed from <none> to 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129). New cstate: current_term: 1 leader_uuid: "8df38c4c53c44e94b36c1e8992476d7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:24.905563 19874 raft_consensus.cc:2468] T af881b097b4245b1be6072579197cb6b P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 1.
I20260709 13:20:24.907317 19765 leader_election.cc:304] T af881b097b4245b1be6072579197cb6b P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, e90ffed3ba174ababeb21d9415d4d918; no voters: 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:24.909020 20001 raft_consensus.cc:2804] T af881b097b4245b1be6072579197cb6b P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:24.920557 20009 raft_consensus.cc:697] T f820e93ffdcd428ea4adfca88589a850 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 LEADER]: Becoming Leader. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Running, Role: LEADER
I20260709 13:20:24.922840 20009 consensus_queue.cc:237] T f820e93ffdcd428ea4adfca88589a850 P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:24.925665 19993 tablet_bootstrap.cc:654] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P e90ffed3ba174ababeb21d9415d4d918: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:24.930925 19994 tablet_bootstrap.cc:654] T af881b097b4245b1be6072579197cb6b P 8df38c4c53c44e94b36c1e8992476d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:24.943498 19997 tablet_bootstrap.cc:492] T 3cad7a5f46f9407a99ea6f2cc5a8322c P b80b3a62d7d549a9a5c42def46769ac4: No bootstrap required, opened a new log
I20260709 13:20:24.944182 19997 ts_tablet_manager.cc:1403] T 3cad7a5f46f9407a99ea6f2cc5a8322c P b80b3a62d7d549a9a5c42def46769ac4: Time spent bootstrapping tablet: real 0.268s	user 0.029s	sys 0.028s
I20260709 13:20:24.946358 20001 raft_consensus.cc:697] T af881b097b4245b1be6072579197cb6b P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Becoming Leader. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Running, Role: LEADER
I20260709 13:20:24.954499 19997 raft_consensus.cc:359] T 3cad7a5f46f9407a99ea6f2cc5a8322c P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:24.957747 19997 raft_consensus.cc:385] T 3cad7a5f46f9407a99ea6f2cc5a8322c P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:24.958214 19997 raft_consensus.cc:740] T 3cad7a5f46f9407a99ea6f2cc5a8322c P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Initialized, Role: FOLLOWER
I20260709 13:20:24.959448 19995 tablet_bootstrap.cc:654] T e8cfec59c63741869c636df6356546d0 P c62716bf66c24330860573535b8b9430: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:24.959648 19997 consensus_queue.cc:260] T 3cad7a5f46f9407a99ea6f2cc5a8322c P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:24.962931 19997 ts_tablet_manager.cc:1434] T 3cad7a5f46f9407a99ea6f2cc5a8322c P b80b3a62d7d549a9a5c42def46769ac4: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.005s
I20260709 13:20:24.963891 19997 tablet_bootstrap.cc:492] T 2a411802fdaa42e7863236f0af595fe2 P b80b3a62d7d549a9a5c42def46769ac4: Bootstrap starting.
I20260709 13:20:24.963073 19621 catalog_manager.cc:5697] T f820e93ffdcd428ea4adfca88589a850 P b80b3a62d7d549a9a5c42def46769ac4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131). New cstate: current_term: 1 leader_uuid: "b80b3a62d7d549a9a5c42def46769ac4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:24.967890 20001 consensus_queue.cc:237] T af881b097b4245b1be6072579197cb6b P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:24.978543 19997 tablet_bootstrap.cc:654] T 2a411802fdaa42e7863236f0af595fe2 P b80b3a62d7d549a9a5c42def46769ac4: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:24.981315 19994 tablet_bootstrap.cc:492] T af881b097b4245b1be6072579197cb6b P 8df38c4c53c44e94b36c1e8992476d7c: No bootstrap required, opened a new log
I20260709 13:20:24.982117 19994 ts_tablet_manager.cc:1403] T af881b097b4245b1be6072579197cb6b P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.164s	user 0.009s	sys 0.013s
I20260709 13:20:24.985181 19993 tablet_bootstrap.cc:492] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P e90ffed3ba174ababeb21d9415d4d918: No bootstrap required, opened a new log
I20260709 13:20:24.985747 19993 ts_tablet_manager.cc:1403] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P e90ffed3ba174ababeb21d9415d4d918: Time spent bootstrapping tablet: real 0.196s	user 0.020s	sys 0.008s
I20260709 13:20:24.985448 19994 raft_consensus.cc:359] T af881b097b4245b1be6072579197cb6b P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:24.986191 19994 raft_consensus.cc:385] T af881b097b4245b1be6072579197cb6b P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:24.986531 19994 raft_consensus.cc:740] T af881b097b4245b1be6072579197cb6b P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:24.987457 19994 consensus_queue.cc:260] T af881b097b4245b1be6072579197cb6b P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:24.989014 19993 raft_consensus.cc:359] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:24.989842 19994 ts_tablet_manager.cc:1434] T af881b097b4245b1be6072579197cb6b P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.007s	user 0.000s	sys 0.005s
I20260709 13:20:24.991124 19994 tablet_bootstrap.cc:492] T e8cfec59c63741869c636df6356546d0 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:24.989843 19993 raft_consensus.cc:385] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:24.995275 19993 raft_consensus.cc:740] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Initialized, Role: FOLLOWER
I20260709 13:20:25.000703 19623 catalog_manager.cc:5697] T af881b097b4245b1be6072579197cb6b P e90ffed3ba174ababeb21d9415d4d918 reported cstate change: term changed from 0 to 1, leader changed from <none> to e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "e90ffed3ba174ababeb21d9415d4d918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:24.997103 19993 consensus_queue.cc:260] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:25.003615 19995 tablet_bootstrap.cc:492] T e8cfec59c63741869c636df6356546d0 P c62716bf66c24330860573535b8b9430: No bootstrap required, opened a new log
I20260709 13:20:25.004484 19995 ts_tablet_manager.cc:1403] T e8cfec59c63741869c636df6356546d0 P c62716bf66c24330860573535b8b9430: Time spent bootstrapping tablet: real 0.129s	user 0.004s	sys 0.022s
I20260709 13:20:25.004151 19993 ts_tablet_manager.cc:1434] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P e90ffed3ba174ababeb21d9415d4d918: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.000s
I20260709 13:20:25.006753 19993 tablet_bootstrap.cc:492] T 2eef64a7d31b4e1e899aee0857cb0e07 P e90ffed3ba174ababeb21d9415d4d918: Bootstrap starting.
I20260709 13:20:25.007674 19994 tablet_bootstrap.cc:654] T e8cfec59c63741869c636df6356546d0 P 8df38c4c53c44e94b36c1e8992476d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:25.009863 19995 raft_consensus.cc:359] T e8cfec59c63741869c636df6356546d0 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:25.012468 19995 raft_consensus.cc:385] T e8cfec59c63741869c636df6356546d0 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:25.012970 19995 raft_consensus.cc:740] T e8cfec59c63741869c636df6356546d0 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62716bf66c24330860573535b8b9430, State: Initialized, Role: FOLLOWER
I20260709 13:20:25.014420 19993 tablet_bootstrap.cc:654] T 2eef64a7d31b4e1e899aee0857cb0e07 P e90ffed3ba174ababeb21d9415d4d918: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:25.014236 19995 consensus_queue.cc:260] T e8cfec59c63741869c636df6356546d0 P c62716bf66c24330860573535b8b9430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:25.017678 19995 ts_tablet_manager.cc:1434] T e8cfec59c63741869c636df6356546d0 P c62716bf66c24330860573535b8b9430: Time spent starting tablet: real 0.013s	user 0.010s	sys 0.000s
I20260709 13:20:25.019233 19995 tablet_bootstrap.cc:492] T d762eace063c4e04821c749d21702c3f P c62716bf66c24330860573535b8b9430: Bootstrap starting.
I20260709 13:20:25.023967 19997 tablet_bootstrap.cc:492] T 2a411802fdaa42e7863236f0af595fe2 P b80b3a62d7d549a9a5c42def46769ac4: No bootstrap required, opened a new log
I20260709 13:20:25.024852 19997 ts_tablet_manager.cc:1403] T 2a411802fdaa42e7863236f0af595fe2 P b80b3a62d7d549a9a5c42def46769ac4: Time spent bootstrapping tablet: real 0.061s	user 0.014s	sys 0.005s
I20260709 13:20:25.029866 19995 tablet_bootstrap.cc:654] T d762eace063c4e04821c749d21702c3f P c62716bf66c24330860573535b8b9430: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:25.029385 19997 raft_consensus.cc:359] T 2a411802fdaa42e7863236f0af595fe2 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:25.033278 19997 raft_consensus.cc:385] T 2a411802fdaa42e7863236f0af595fe2 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:25.033700 19997 raft_consensus.cc:740] T 2a411802fdaa42e7863236f0af595fe2 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Initialized, Role: FOLLOWER
I20260709 13:20:25.034668 19997 consensus_queue.cc:260] T 2a411802fdaa42e7863236f0af595fe2 P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:25.037403 19997 ts_tablet_manager.cc:1434] T 2a411802fdaa42e7863236f0af595fe2 P b80b3a62d7d549a9a5c42def46769ac4: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:20:25.038511 19997 tablet_bootstrap.cc:492] T 85d493223fe04c939b6effd840955727 P b80b3a62d7d549a9a5c42def46769ac4: Bootstrap starting.
I20260709 13:20:25.039408 19993 tablet_bootstrap.cc:492] T 2eef64a7d31b4e1e899aee0857cb0e07 P e90ffed3ba174ababeb21d9415d4d918: No bootstrap required, opened a new log
I20260709 13:20:25.039950 19993 ts_tablet_manager.cc:1403] T 2eef64a7d31b4e1e899aee0857cb0e07 P e90ffed3ba174ababeb21d9415d4d918: Time spent bootstrapping tablet: real 0.033s	user 0.012s	sys 0.014s
I20260709 13:20:25.043332 19993 raft_consensus.cc:359] T 2eef64a7d31b4e1e899aee0857cb0e07 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:25.044143 19993 raft_consensus.cc:385] T 2eef64a7d31b4e1e899aee0857cb0e07 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:25.044438 19993 raft_consensus.cc:740] T 2eef64a7d31b4e1e899aee0857cb0e07 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Initialized, Role: FOLLOWER
I20260709 13:20:25.045338 19993 consensus_queue.cc:260] T 2eef64a7d31b4e1e899aee0857cb0e07 P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:25.048007 19993 ts_tablet_manager.cc:1434] T 2eef64a7d31b4e1e899aee0857cb0e07 P e90ffed3ba174ababeb21d9415d4d918: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:20:25.049373 19993 tablet_bootstrap.cc:492] T 353a546cf8c54671971c239502ef165a P e90ffed3ba174ababeb21d9415d4d918: Bootstrap starting.
I20260709 13:20:25.054360 19997 tablet_bootstrap.cc:654] T 85d493223fe04c939b6effd840955727 P b80b3a62d7d549a9a5c42def46769ac4: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:25.056396 19993 tablet_bootstrap.cc:654] T 353a546cf8c54671971c239502ef165a P e90ffed3ba174ababeb21d9415d4d918: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:25.056216 20031 raft_consensus.cc:493] T af881b097b4245b1be6072579197cb6b P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:25.057253 20031 raft_consensus.cc:515] T af881b097b4245b1be6072579197cb6b P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:25.062422 20035 raft_consensus.cc:493] T 180230b77b7640b29d055ae8f4358ffe P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:25.063445 20035 raft_consensus.cc:515] T 180230b77b7640b29d055ae8f4358ffe P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:25.068136 19800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af881b097b4245b1be6072579197cb6b" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:25.070483 19725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "180230b77b7640b29d055ae8f4358ffe" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
W20260709 13:20:25.074110 19914 leader_election.cc:343] T 180230b77b7640b29d055ae8f4358ffe P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:25.078377 19874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af881b097b4245b1be6072579197cb6b" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:25.079834 19874 raft_consensus.cc:2393] T af881b097b4245b1be6072579197cb6b P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8df38c4c53c44e94b36c1e8992476d7c in current term 1: Already voted for candidate e90ffed3ba174ababeb21d9415d4d918 in this term.
I20260709 13:20:25.082368 19690 leader_election.cc:304] T af881b097b4245b1be6072579197cb6b P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c; no voters: b80b3a62d7d549a9a5c42def46769ac4, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:25.085633 19800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "180230b77b7640b29d055ae8f4358ffe" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
W20260709 13:20:25.089533 19913 leader_election.cc:343] T 180230b77b7640b29d055ae8f4358ffe P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:25.090304 19913 leader_election.cc:304] T 180230b77b7640b29d055ae8f4358ffe P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:25.091555 20000 raft_consensus.cc:2749] T 180230b77b7640b29d055ae8f4358ffe P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:25.092130 20035 leader_election.cc:290] T 180230b77b7640b29d055ae8f4358ffe P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:25.093240 20033 raft_consensus.cc:3060] T af881b097b4245b1be6072579197cb6b P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:25.097568 19994 tablet_bootstrap.cc:492] T e8cfec59c63741869c636df6356546d0 P 8df38c4c53c44e94b36c1e8992476d7c: No bootstrap required, opened a new log
I20260709 13:20:25.098330 19994 ts_tablet_manager.cc:1403] T e8cfec59c63741869c636df6356546d0 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.107s	user 0.005s	sys 0.009s
I20260709 13:20:25.076086 20031 leader_election.cc:290] T af881b097b4245b1be6072579197cb6b P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:25.102952 19994 raft_consensus.cc:359] T e8cfec59c63741869c636df6356546d0 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:25.103997 19994 raft_consensus.cc:385] T e8cfec59c63741869c636df6356546d0 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:25.104386 19994 raft_consensus.cc:740] T e8cfec59c63741869c636df6356546d0 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:25.107926 20033 raft_consensus.cc:2749] T af881b097b4245b1be6072579197cb6b P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:25.108098 19994 consensus_queue.cc:260] T e8cfec59c63741869c636df6356546d0 P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:25.108734 19995 tablet_bootstrap.cc:492] T d762eace063c4e04821c749d21702c3f P c62716bf66c24330860573535b8b9430: No bootstrap required, opened a new log
I20260709 13:20:25.111596 19995 ts_tablet_manager.cc:1403] T d762eace063c4e04821c749d21702c3f P c62716bf66c24330860573535b8b9430: Time spent bootstrapping tablet: real 0.093s	user 0.046s	sys 0.035s
I20260709 13:20:25.113168 19994 ts_tablet_manager.cc:1434] T e8cfec59c63741869c636df6356546d0 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.001s
I20260709 13:20:25.109303 19993 tablet_bootstrap.cc:492] T 353a546cf8c54671971c239502ef165a P e90ffed3ba174ababeb21d9415d4d918: No bootstrap required, opened a new log
I20260709 13:20:25.113942 19993 ts_tablet_manager.cc:1403] T 353a546cf8c54671971c239502ef165a P e90ffed3ba174ababeb21d9415d4d918: Time spent bootstrapping tablet: real 0.065s	user 0.038s	sys 0.016s
I20260709 13:20:25.114708 19994 tablet_bootstrap.cc:492] T 180230b77b7640b29d055ae8f4358ffe P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:25.117901 19993 raft_consensus.cc:359] T 353a546cf8c54671971c239502ef165a P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:25.120890 19995 raft_consensus.cc:359] T d762eace063c4e04821c749d21702c3f P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
W20260709 13:20:25.121861 19670 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:20:25.122136 19995 raft_consensus.cc:385] T d762eace063c4e04821c749d21702c3f P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:25.122567 19995 raft_consensus.cc:740] T d762eace063c4e04821c749d21702c3f P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62716bf66c24330860573535b8b9430, State: Initialized, Role: FOLLOWER
I20260709 13:20:25.123656 19994 tablet_bootstrap.cc:654] T 180230b77b7640b29d055ae8f4358ffe P 8df38c4c53c44e94b36c1e8992476d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:25.123469 19995 consensus_queue.cc:260] T d762eace063c4e04821c749d21702c3f P c62716bf66c24330860573535b8b9430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:25.126389 19995 ts_tablet_manager.cc:1434] T d762eace063c4e04821c749d21702c3f P c62716bf66c24330860573535b8b9430: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 13:20:25.121884 19993 raft_consensus.cc:385] T 353a546cf8c54671971c239502ef165a P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:25.127231 19993 raft_consensus.cc:740] T 353a546cf8c54671971c239502ef165a P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Initialized, Role: FOLLOWER
I20260709 13:20:25.128527 19993 consensus_queue.cc:260] T 353a546cf8c54671971c239502ef165a P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:25.132485 19993 ts_tablet_manager.cc:1434] T 353a546cf8c54671971c239502ef165a P e90ffed3ba174ababeb21d9415d4d918: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.003s
I20260709 13:20:25.134239 19993 tablet_bootstrap.cc:492] T 180230b77b7640b29d055ae8f4358ffe P e90ffed3ba174ababeb21d9415d4d918: Bootstrap starting.
I20260709 13:20:25.138984 20009 raft_consensus.cc:493] T 2a411802fdaa42e7863236f0af595fe2 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:25.140090 19995 tablet_bootstrap.cc:492] T 2eef64a7d31b4e1e899aee0857cb0e07 P c62716bf66c24330860573535b8b9430: Bootstrap starting.
I20260709 13:20:25.139766 20009 raft_consensus.cc:515] T 2a411802fdaa42e7863236f0af595fe2 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:25.144742 19725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a411802fdaa42e7863236f0af595fe2" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:25.146952 20009 leader_election.cc:290] T 2a411802fdaa42e7863236f0af595fe2 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
W20260709 13:20:25.147871 19840 leader_election.cc:343] T 2a411802fdaa42e7863236f0af595fe2 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:25.150565 19800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a411802fdaa42e7863236f0af595fe2" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
W20260709 13:20:25.155117 19839 leader_election.cc:343] T 2a411802fdaa42e7863236f0af595fe2 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:25.155659 19839 leader_election.cc:304] T 2a411802fdaa42e7863236f0af595fe2 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4; no voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:25.156503 20009 raft_consensus.cc:2749] T 2a411802fdaa42e7863236f0af595fe2 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:25.157706 19993 tablet_bootstrap.cc:654] T 180230b77b7640b29d055ae8f4358ffe P e90ffed3ba174ababeb21d9415d4d918: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:25.163416 19995 tablet_bootstrap.cc:654] T 2eef64a7d31b4e1e899aee0857cb0e07 P c62716bf66c24330860573535b8b9430: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:25.175709 19995 tablet_bootstrap.cc:492] T 2eef64a7d31b4e1e899aee0857cb0e07 P c62716bf66c24330860573535b8b9430: No bootstrap required, opened a new log
I20260709 13:20:25.176337 19995 ts_tablet_manager.cc:1403] T 2eef64a7d31b4e1e899aee0857cb0e07 P c62716bf66c24330860573535b8b9430: Time spent bootstrapping tablet: real 0.037s	user 0.021s	sys 0.000s
I20260709 13:20:25.177646 19993 tablet_bootstrap.cc:492] T 180230b77b7640b29d055ae8f4358ffe P e90ffed3ba174ababeb21d9415d4d918: No bootstrap required, opened a new log
I20260709 13:20:25.178294 19993 ts_tablet_manager.cc:1403] T 180230b77b7640b29d055ae8f4358ffe P e90ffed3ba174ababeb21d9415d4d918: Time spent bootstrapping tablet: real 0.044s	user 0.017s	sys 0.004s
I20260709 13:20:25.178913 19994 tablet_bootstrap.cc:492] T 180230b77b7640b29d055ae8f4358ffe P 8df38c4c53c44e94b36c1e8992476d7c: No bootstrap required, opened a new log
I20260709 13:20:25.179728 19994 ts_tablet_manager.cc:1403] T 180230b77b7640b29d055ae8f4358ffe P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.065s	user 0.047s	sys 0.016s
I20260709 13:20:25.179276 19995 raft_consensus.cc:359] T 2eef64a7d31b4e1e899aee0857cb0e07 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:25.180202 19995 raft_consensus.cc:385] T 2eef64a7d31b4e1e899aee0857cb0e07 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:25.180529 19995 raft_consensus.cc:740] T 2eef64a7d31b4e1e899aee0857cb0e07 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62716bf66c24330860573535b8b9430, State: Initialized, Role: FOLLOWER
I20260709 13:20:25.181234 19997 tablet_bootstrap.cc:492] T 85d493223fe04c939b6effd840955727 P b80b3a62d7d549a9a5c42def46769ac4: No bootstrap required, opened a new log
I20260709 13:20:25.181322 19995 consensus_queue.cc:260] T 2eef64a7d31b4e1e899aee0857cb0e07 P c62716bf66c24330860573535b8b9430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:25.181968 19997 ts_tablet_manager.cc:1403] T 85d493223fe04c939b6effd840955727 P b80b3a62d7d549a9a5c42def46769ac4: Time spent bootstrapping tablet: real 0.144s	user 0.031s	sys 0.031s
I20260709 13:20:25.181982 19993 raft_consensus.cc:359] T 180230b77b7640b29d055ae8f4358ffe P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:25.183043 19993 raft_consensus.cc:385] T 180230b77b7640b29d055ae8f4358ffe P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:25.183409 19993 raft_consensus.cc:740] T 180230b77b7640b29d055ae8f4358ffe P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Initialized, Role: FOLLOWER
I20260709 13:20:25.183876 19995 ts_tablet_manager.cc:1434] T 2eef64a7d31b4e1e899aee0857cb0e07 P c62716bf66c24330860573535b8b9430: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:20:25.184989 19995 tablet_bootstrap.cc:492] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P c62716bf66c24330860573535b8b9430: Bootstrap starting.
I20260709 13:20:25.184350 19993 consensus_queue.cc:260] T 180230b77b7640b29d055ae8f4358ffe P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:25.187018 19993 ts_tablet_manager.cc:1434] T 180230b77b7640b29d055ae8f4358ffe P e90ffed3ba174ababeb21d9415d4d918: Time spent starting tablet: real 0.008s	user 0.011s	sys 0.000s
I20260709 13:20:25.188230 19993 tablet_bootstrap.cc:492] T d762eace063c4e04821c749d21702c3f P e90ffed3ba174ababeb21d9415d4d918: Bootstrap starting.
I20260709 13:20:25.191725 19995 tablet_bootstrap.cc:654] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P c62716bf66c24330860573535b8b9430: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:25.195338 19993 tablet_bootstrap.cc:654] T d762eace063c4e04821c749d21702c3f P e90ffed3ba174ababeb21d9415d4d918: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:25.201640 19997 raft_consensus.cc:359] T 85d493223fe04c939b6effd840955727 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:25.202597 19997 raft_consensus.cc:385] T 85d493223fe04c939b6effd840955727 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:25.202988 19997 raft_consensus.cc:740] T 85d493223fe04c939b6effd840955727 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Initialized, Role: FOLLOWER
I20260709 13:20:25.203948 19997 consensus_queue.cc:260] T 85d493223fe04c939b6effd840955727 P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:25.206624 19997 ts_tablet_manager.cc:1434] T 85d493223fe04c939b6effd840955727 P b80b3a62d7d549a9a5c42def46769ac4: Time spent starting tablet: real 0.024s	user 0.002s	sys 0.002s
I20260709 13:20:25.208053 19997 tablet_bootstrap.cc:492] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P b80b3a62d7d549a9a5c42def46769ac4: Bootstrap starting.
I20260709 13:20:25.208017 19994 raft_consensus.cc:359] T 180230b77b7640b29d055ae8f4358ffe P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:25.209133 19994 raft_consensus.cc:385] T 180230b77b7640b29d055ae8f4358ffe P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:25.209523 19994 raft_consensus.cc:740] T 180230b77b7640b29d055ae8f4358ffe P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:25.210597 19994 consensus_queue.cc:260] T 180230b77b7640b29d055ae8f4358ffe P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:25.212276 20009 raft_consensus.cc:493] T 3cad7a5f46f9407a99ea6f2cc5a8322c P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:25.212791 20009 raft_consensus.cc:515] T 3cad7a5f46f9407a99ea6f2cc5a8322c P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:25.213804 19994 ts_tablet_manager.cc:1434] T 180230b77b7640b29d055ae8f4358ffe P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.001s
I20260709 13:20:25.215124 19994 tablet_bootstrap.cc:492] T 2a411802fdaa42e7863236f0af595fe2 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:25.215261 20009 leader_election.cc:290] T 3cad7a5f46f9407a99ea6f2cc5a8322c P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:25.218755 19800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cad7a5f46f9407a99ea6f2cc5a8322c" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:25.223076 20001 raft_consensus.cc:493] T 180230b77b7640b29d055ae8f4358ffe P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:25.223918 20001 raft_consensus.cc:515] T 180230b77b7640b29d055ae8f4358ffe P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:25.227607 19948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "180230b77b7640b29d055ae8f4358ffe" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:25.228296 19948 raft_consensus.cc:2468] T 180230b77b7640b29d055ae8f4358ffe P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 0.
I20260709 13:20:25.229663 19765 leader_election.cc:304] T 180230b77b7640b29d055ae8f4358ffe P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c62716bf66c24330860573535b8b9430, e90ffed3ba174ababeb21d9415d4d918; no voters: 
I20260709 13:20:25.230042 20001 leader_election.cc:290] T 180230b77b7640b29d055ae8f4358ffe P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:25.230545 20040 raft_consensus.cc:2804] T 180230b77b7640b29d055ae8f4358ffe P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:25.230994 20040 raft_consensus.cc:493] T 180230b77b7640b29d055ae8f4358ffe P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:25.231320 20040 raft_consensus.cc:3060] T 180230b77b7640b29d055ae8f4358ffe P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:25.231303 19725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "180230b77b7640b29d055ae8f4358ffe" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:25.232199 19725 raft_consensus.cc:2468] T 180230b77b7640b29d055ae8f4358ffe P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 0.
I20260709 13:20:25.234891 19993 tablet_bootstrap.cc:492] T d762eace063c4e04821c749d21702c3f P e90ffed3ba174ababeb21d9415d4d918: No bootstrap required, opened a new log
I20260709 13:20:25.235404 19993 ts_tablet_manager.cc:1403] T d762eace063c4e04821c749d21702c3f P e90ffed3ba174ababeb21d9415d4d918: Time spent bootstrapping tablet: real 0.047s	user 0.012s	sys 0.016s
I20260709 13:20:25.238631 19997 tablet_bootstrap.cc:654] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P b80b3a62d7d549a9a5c42def46769ac4: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:25.239039 20001 raft_consensus.cc:493] T 2eef64a7d31b4e1e899aee0857cb0e07 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:25.239660 20001 raft_consensus.cc:515] T 2eef64a7d31b4e1e899aee0857cb0e07 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:25.238387 19993 raft_consensus.cc:359] T d762eace063c4e04821c749d21702c3f P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:25.242223 19993 raft_consensus.cc:385] T d762eace063c4e04821c749d21702c3f P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:25.242515 19993 raft_consensus.cc:740] T d762eace063c4e04821c749d21702c3f P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Initialized, Role: FOLLOWER
I20260709 13:20:25.242640 19948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eef64a7d31b4e1e899aee0857cb0e07" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:25.243312 19948 raft_consensus.cc:2468] T 2eef64a7d31b4e1e899aee0857cb0e07 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 0.
I20260709 13:20:25.243388 19993 consensus_queue.cc:260] T d762eace063c4e04821c749d21702c3f P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:25.244730 19765 leader_election.cc:304] T 2eef64a7d31b4e1e899aee0857cb0e07 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c62716bf66c24330860573535b8b9430, e90ffed3ba174ababeb21d9415d4d918; no voters: 
I20260709 13:20:25.246327 19994 tablet_bootstrap.cc:654] T 2a411802fdaa42e7863236f0af595fe2 P 8df38c4c53c44e94b36c1e8992476d7c: Neither blocks nor log segments found. Creating new log.
W20260709 13:20:25.247179 19839 leader_election.cc:343] T 3cad7a5f46f9407a99ea6f2cc5a8322c P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:25.250705 19725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eef64a7d31b4e1e899aee0857cb0e07" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:25.251677 20001 leader_election.cc:290] T 2eef64a7d31b4e1e899aee0857cb0e07 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:25.261133 20001 raft_consensus.cc:493] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:25.259146 19993 ts_tablet_manager.cc:1434] T d762eace063c4e04821c749d21702c3f P e90ffed3ba174ababeb21d9415d4d918: Time spent starting tablet: real 0.023s	user 0.021s	sys 0.000s
I20260709 13:20:25.263916 19994 tablet_bootstrap.cc:492] T 2a411802fdaa42e7863236f0af595fe2 P 8df38c4c53c44e94b36c1e8992476d7c: No bootstrap required, opened a new log
I20260709 13:20:25.264510 19994 ts_tablet_manager.cc:1403] T 2a411802fdaa42e7863236f0af595fe2 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.050s	user 0.016s	sys 0.001s
I20260709 13:20:25.263979 19993 tablet_bootstrap.cc:492] T 6a066b96902f41dbb1511eccf26820ce P e90ffed3ba174ababeb21d9415d4d918: Bootstrap starting.
I20260709 13:20:25.263908 20040 raft_consensus.cc:515] T 180230b77b7640b29d055ae8f4358ffe P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:25.268637 19948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cad7a5f46f9407a99ea6f2cc5a8322c" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:25.259330 20064 raft_consensus.cc:2804] T 2eef64a7d31b4e1e899aee0857cb0e07 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:25.270579 20064 raft_consensus.cc:493] T 2eef64a7d31b4e1e899aee0857cb0e07 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:25.271049 20064 raft_consensus.cc:3060] T 2eef64a7d31b4e1e899aee0857cb0e07 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:25.271582 19994 raft_consensus.cc:359] T 2a411802fdaa42e7863236f0af595fe2 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:25.272431 19994 raft_consensus.cc:385] T 2a411802fdaa42e7863236f0af595fe2 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:20:25.272665 19839 leader_election.cc:343] T 3cad7a5f46f9407a99ea6f2cc5a8322c P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c62716bf66c24330860573535b8b9430 (127.9.116.132:42643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:25.272759 19994 raft_consensus.cc:740] T 2a411802fdaa42e7863236f0af595fe2 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:25.273451 19839 leader_election.cc:304] T 3cad7a5f46f9407a99ea6f2cc5a8322c P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4; no voters: c62716bf66c24330860573535b8b9430, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:25.274108 19993 tablet_bootstrap.cc:654] T 6a066b96902f41dbb1511eccf26820ce P e90ffed3ba174ababeb21d9415d4d918: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:25.275883 20009 raft_consensus.cc:2749] T 3cad7a5f46f9407a99ea6f2cc5a8322c P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:25.258816 19997 tablet_bootstrap.cc:492] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P b80b3a62d7d549a9a5c42def46769ac4: No bootstrap required, opened a new log
I20260709 13:20:25.261677 20001 raft_consensus.cc:515] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:25.276966 19997 ts_tablet_manager.cc:1403] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P b80b3a62d7d549a9a5c42def46769ac4: Time spent bootstrapping tablet: real 0.069s	user 0.022s	sys 0.007s
I20260709 13:20:25.273892 19994 consensus_queue.cc:260] T 2a411802fdaa42e7863236f0af595fe2 P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:25.279084 19995 tablet_bootstrap.cc:492] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P c62716bf66c24330860573535b8b9430: No bootstrap required, opened a new log
I20260709 13:20:25.279639 19995 ts_tablet_manager.cc:1403] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P c62716bf66c24330860573535b8b9430: Time spent bootstrapping tablet: real 0.095s	user 0.011s	sys 0.009s
I20260709 13:20:25.281001 19997 raft_consensus.cc:359] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:25.283708 20040 leader_election.cc:290] T 180230b77b7640b29d055ae8f4358ffe P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 election: Requested vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
W20260709 13:20:25.283789 19766 leader_election.cc:343] T 2eef64a7d31b4e1e899aee0857cb0e07 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:25.282038 19997 raft_consensus.cc:385] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:25.285727 19725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "180230b77b7640b29d055ae8f4358ffe" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
I20260709 13:20:25.285773 19995 raft_consensus.cc:359] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:25.287264 20035 raft_consensus.cc:493] T e8cfec59c63741869c636df6356546d0 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:25.287858 20001 leader_election.cc:290] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:25.287973 20035 raft_consensus.cc:515] T e8cfec59c63741869c636df6356546d0 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:25.289739 20064 raft_consensus.cc:515] T 2eef64a7d31b4e1e899aee0857cb0e07 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:25.286680 19725 raft_consensus.cc:3060] T 180230b77b7640b29d055ae8f4358ffe P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:25.291661 19724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8cfec59c63741869c636df6356546d0" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:25.292297 19724 raft_consensus.cc:2468] T e8cfec59c63741869c636df6356546d0 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c62716bf66c24330860573535b8b9430 in term 0.
I20260709 13:20:25.293591 19914 leader_election.cc:304] T e8cfec59c63741869c636df6356546d0 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, c62716bf66c24330860573535b8b9430; no voters: 
I20260709 13:20:25.286676 19995 raft_consensus.cc:385] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:25.294255 19995 raft_consensus.cc:740] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62716bf66c24330860573535b8b9430, State: Initialized, Role: FOLLOWER
I20260709 13:20:25.286696 19994 ts_tablet_manager.cc:1434] T 2a411802fdaa42e7863236f0af595fe2 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.000s
I20260709 13:20:25.294474 20000 raft_consensus.cc:2804] T e8cfec59c63741869c636df6356546d0 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:25.285777 19997 raft_consensus.cc:740] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Initialized, Role: FOLLOWER
I20260709 13:20:25.295035 19995 consensus_queue.cc:260] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P c62716bf66c24330860573535b8b9430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:25.295059 20000 raft_consensus.cc:493] T e8cfec59c63741869c636df6356546d0 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:25.296707 20000 raft_consensus.cc:3060] T e8cfec59c63741869c636df6356546d0 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:25.297138 19997 consensus_queue.cc:260] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:25.298164 20064 leader_election.cc:290] T 2eef64a7d31b4e1e899aee0857cb0e07 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 election: Requested vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:25.301254 19997 ts_tablet_manager.cc:1434] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P b80b3a62d7d549a9a5c42def46769ac4: Time spent starting tablet: real 0.024s	user 0.003s	sys 0.006s
I20260709 13:20:25.301813 20035 leader_election.cc:290] T e8cfec59c63741869c636df6356546d0 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:25.302490 19997 tablet_bootstrap.cc:492] T 19a59035bda747fdbc59b7e6914101dd P b80b3a62d7d549a9a5c42def46769ac4: Bootstrap starting.
I20260709 13:20:25.301314 19994 tablet_bootstrap.cc:492] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:25.305311 19995 ts_tablet_manager.cc:1434] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P c62716bf66c24330860573535b8b9430: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 13:20:25.306223 19995 tablet_bootstrap.cc:492] T 3cad7a5f46f9407a99ea6f2cc5a8322c P c62716bf66c24330860573535b8b9430: Bootstrap starting.
I20260709 13:20:25.306497 20009 raft_consensus.cc:493] T 85d493223fe04c939b6effd840955727 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:25.294989 20001 raft_consensus.cc:493] T 353a546cf8c54671971c239502ef165a P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:25.307036 20009 raft_consensus.cc:515] T 85d493223fe04c939b6effd840955727 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:25.307351 20001 raft_consensus.cc:515] T 353a546cf8c54671971c239502ef165a P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:25.309250 20009 leader_election.cc:290] T 85d493223fe04c939b6effd840955727 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:25.309559 20001 leader_election.cc:290] T 353a546cf8c54671971c239502ef165a P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:25.313060 19994 tablet_bootstrap.cc:654] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P 8df38c4c53c44e94b36c1e8992476d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:25.319964 19995 tablet_bootstrap.cc:654] T 3cad7a5f46f9407a99ea6f2cc5a8322c P c62716bf66c24330860573535b8b9430: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:25.320644 19800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d493223fe04c939b6effd840955727" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:25.320916 19948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d493223fe04c939b6effd840955727" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:25.324245 19997 tablet_bootstrap.cc:654] T 19a59035bda747fdbc59b7e6914101dd P b80b3a62d7d549a9a5c42def46769ac4: Neither blocks nor log segments found. Creating new log.
W20260709 13:20:25.324734 19839 leader_election.cc:343] T 85d493223fe04c939b6effd840955727 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c62716bf66c24330860573535b8b9430 (127.9.116.132:42643): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:20:25.325573 19839 leader_election.cc:343] T 85d493223fe04c939b6effd840955727 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:25.326195 19839 leader_election.cc:304] T 85d493223fe04c939b6effd840955727 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4; no voters: c62716bf66c24330860573535b8b9430, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:25.327095 20009 raft_consensus.cc:2749] T 85d493223fe04c939b6effd840955727 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:25.329026 19724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eef64a7d31b4e1e899aee0857cb0e07" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
I20260709 13:20:25.330439 19993 tablet_bootstrap.cc:492] T 6a066b96902f41dbb1511eccf26820ce P e90ffed3ba174ababeb21d9415d4d918: No bootstrap required, opened a new log
I20260709 13:20:25.330937 19993 ts_tablet_manager.cc:1403] T 6a066b96902f41dbb1511eccf26820ce P e90ffed3ba174ababeb21d9415d4d918: Time spent bootstrapping tablet: real 0.067s	user 0.011s	sys 0.014s
I20260709 13:20:25.333864 19993 raft_consensus.cc:359] T 6a066b96902f41dbb1511eccf26820ce P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:25.334566 19993 raft_consensus.cc:385] T 6a066b96902f41dbb1511eccf26820ce P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:25.334290 20009 consensus_queue.cc:1048] T f820e93ffdcd428ea4adfca88589a850 P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:20:25.334896 19993 raft_consensus.cc:740] T 6a066b96902f41dbb1511eccf26820ce P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Initialized, Role: FOLLOWER
I20260709 13:20:25.335726 19993 consensus_queue.cc:260] T 6a066b96902f41dbb1511eccf26820ce P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:25.337920 19993 ts_tablet_manager.cc:1434] T 6a066b96902f41dbb1511eccf26820ce P e90ffed3ba174ababeb21d9415d4d918: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:20:25.338963 19993 tablet_bootstrap.cc:492] T 2a411802fdaa42e7863236f0af595fe2 P e90ffed3ba174ababeb21d9415d4d918: Bootstrap starting.
I20260709 13:20:25.339109 19997 tablet_bootstrap.cc:492] T 19a59035bda747fdbc59b7e6914101dd P b80b3a62d7d549a9a5c42def46769ac4: No bootstrap required, opened a new log
I20260709 13:20:25.339679 19997 ts_tablet_manager.cc:1403] T 19a59035bda747fdbc59b7e6914101dd P b80b3a62d7d549a9a5c42def46769ac4: Time spent bootstrapping tablet: real 0.037s	user 0.009s	sys 0.008s
I20260709 13:20:25.342278 20000 raft_consensus.cc:515] T e8cfec59c63741869c636df6356546d0 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:25.342661 19997 raft_consensus.cc:359] T 19a59035bda747fdbc59b7e6914101dd P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:25.344936 19723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353a546cf8c54671971c239502ef165a" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:25.349205 19997 raft_consensus.cc:385] T 19a59035bda747fdbc59b7e6914101dd P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:25.349535 19997 raft_consensus.cc:740] T 19a59035bda747fdbc59b7e6914101dd P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Initialized, Role: FOLLOWER
I20260709 13:20:25.349937 20001 raft_consensus.cc:493] T d762eace063c4e04821c749d21702c3f P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:25.350419 20001 raft_consensus.cc:515] T d762eace063c4e04821c749d21702c3f P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:25.350286 19997 consensus_queue.cc:260] T 19a59035bda747fdbc59b7e6914101dd P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:25.346239 19993 tablet_bootstrap.cc:654] T 2a411802fdaa42e7863236f0af595fe2 P e90ffed3ba174ababeb21d9415d4d918: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:25.324331 19725 raft_consensus.cc:2468] T 180230b77b7640b29d055ae8f4358ffe P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 1.
I20260709 13:20:25.352664 20009 raft_consensus.cc:493] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:25.352741 20001 leader_election.cc:290] T d762eace063c4e04821c749d21702c3f P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:25.353211 20009 raft_consensus.cc:515] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:25.353777 19997 ts_tablet_manager.cc:1434] T 19a59035bda747fdbc59b7e6914101dd P b80b3a62d7d549a9a5c42def46769ac4: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:20:25.355887 19723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8cfec59c63741869c636df6356546d0" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
I20260709 13:20:25.356519 19723 raft_consensus.cc:3060] T e8cfec59c63741869c636df6356546d0 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:20:25.347319 19765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:42643, user_credentials={real_user=slave}} blocked reactor thread for 102696us
W20260709 13:20:25.346271 19766 leader_election.cc:343] T 2eef64a7d31b4e1e899aee0857cb0e07 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:25.360177 19947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "180230b77b7640b29d055ae8f4358ffe" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:25.360884 19947 raft_consensus.cc:3060] T 180230b77b7640b29d055ae8f4358ffe P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:25.361145 20033 consensus_queue.cc:1048] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:20:25.361316 19766 leader_election.cc:343] T 353a546cf8c54671971c239502ef165a P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:25.362957 19766 leader_election.cc:304] T 180230b77b7640b29d055ae8f4358ffe P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918; no voters: 
I20260709 13:20:25.363845 19722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d762eace063c4e04821c749d21702c3f" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:25.372886 20001 raft_consensus.cc:2804] T 180230b77b7640b29d055ae8f4358ffe P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:25.373548 20001 raft_consensus.cc:697] T 180230b77b7640b29d055ae8f4358ffe P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Becoming Leader. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Running, Role: LEADER
I20260709 13:20:25.374490 20001 consensus_queue.cc:237] T 180230b77b7640b29d055ae8f4358ffe P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
W20260709 13:20:25.373051 19766 leader_election.cc:343] T d762eace063c4e04821c749d21702c3f P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:25.368304 19948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9f7b34cb064bbe8a6938e4f92d57c5" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:25.370129 19945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eef64a7d31b4e1e899aee0857cb0e07" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:25.380214 19942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f3d7bd0ea32401fa8cb8ab3292a7808" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:25.380563 20009 leader_election.cc:290] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:25.380678 19948 raft_consensus.cc:2468] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 0.
I20260709 13:20:25.380990 19942 raft_consensus.cc:2468] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 0.
I20260709 13:20:25.377943 19943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d762eace063c4e04821c749d21702c3f" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:25.382365 19943 raft_consensus.cc:2468] T d762eace063c4e04821c749d21702c3f P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 0.
I20260709 13:20:25.382469 19947 raft_consensus.cc:2468] T 180230b77b7640b29d055ae8f4358ffe P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 1.
I20260709 13:20:25.382773 19799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8cfec59c63741869c636df6356546d0" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
I20260709 13:20:25.383503 19799 raft_consensus.cc:3060] T e8cfec59c63741869c636df6356546d0 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:25.378528 19800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8cfec59c63741869c636df6356546d0" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:25.380064 19722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f3d7bd0ea32401fa8cb8ab3292a7808" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:25.380761 19945 raft_consensus.cc:3060] T 2eef64a7d31b4e1e899aee0857cb0e07 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:25.350657 20000 leader_election.cc:290] T e8cfec59c63741869c636df6356546d0 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 election: Requested vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:25.370762 19944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353a546cf8c54671971c239502ef165a" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:25.386979 19839 leader_election.cc:304] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430; no voters: 
W20260709 13:20:25.387804 19840 leader_election.cc:343] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:25.396636 19945 raft_consensus.cc:2468] T 2eef64a7d31b4e1e899aee0857cb0e07 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 1.
I20260709 13:20:25.396786 19765 leader_election.cc:304] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c62716bf66c24330860573535b8b9430, e90ffed3ba174ababeb21d9415d4d918; no voters: 
I20260709 13:20:25.396023 20009 raft_consensus.cc:2804] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:25.364894 19723 raft_consensus.cc:2468] T e8cfec59c63741869c636df6356546d0 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c62716bf66c24330860573535b8b9430 in term 1.
I20260709 13:20:25.394429 19799 raft_consensus.cc:2468] T e8cfec59c63741869c636df6356546d0 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c62716bf66c24330860573535b8b9430 in term 1.
I20260709 13:20:25.397809 19874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9f7b34cb064bbe8a6938e4f92d57c5" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:25.398375 19997 tablet_bootstrap.cc:492] T a36c11f368f34bf0aa4925a099ac1f91 P b80b3a62d7d549a9a5c42def46769ac4: Bootstrap starting.
I20260709 13:20:25.410764 20009 raft_consensus.cc:493] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:25.412402 20033 raft_consensus.cc:493] T 2a411802fdaa42e7863236f0af595fe2 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:25.412635 20009 raft_consensus.cc:3060] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:25.412972 20033 raft_consensus.cc:515] T 2a411802fdaa42e7863236f0af595fe2 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:25.406396 19995 tablet_bootstrap.cc:492] T 3cad7a5f46f9407a99ea6f2cc5a8322c P c62716bf66c24330860573535b8b9430: No bootstrap required, opened a new log
I20260709 13:20:25.413115 19914 leader_election.cc:304] T e8cfec59c63741869c636df6356546d0 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, c62716bf66c24330860573535b8b9430; no voters: 
I20260709 13:20:25.415236 20033 leader_election.cc:290] T 2a411802fdaa42e7863236f0af595fe2 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:25.406925 19994 tablet_bootstrap.cc:492] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P 8df38c4c53c44e94b36c1e8992476d7c: No bootstrap required, opened a new log
I20260709 13:20:25.416174 20064 raft_consensus.cc:2804] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:25.416074 19765 leader_election.cc:304] T d762eace063c4e04821c749d21702c3f P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c62716bf66c24330860573535b8b9430, e90ffed3ba174ababeb21d9415d4d918; no voters: 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:25.414654 20000 raft_consensus.cc:2804] T e8cfec59c63741869c636df6356546d0 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:25.416711 20064 raft_consensus.cc:493] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:25.418694 20000 raft_consensus.cc:697] T e8cfec59c63741869c636df6356546d0 P c62716bf66c24330860573535b8b9430 [term 1 LEADER]: Becoming Leader. State: Replica: c62716bf66c24330860573535b8b9430, State: Running, Role: LEADER
I20260709 13:20:25.417586 19799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a411802fdaa42e7863236f0af595fe2" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:25.419914 20000 consensus_queue.cc:237] T e8cfec59c63741869c636df6356546d0 P c62716bf66c24330860573535b8b9430 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:25.417922 19874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a411802fdaa42e7863236f0af595fe2" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:25.421131 19874 raft_consensus.cc:2468] T 2a411802fdaa42e7863236f0af595fe2 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 0.
I20260709 13:20:25.422045 20009 raft_consensus.cc:515] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:25.424588 19997 tablet_bootstrap.cc:654] T a36c11f368f34bf0aa4925a099ac1f91 P b80b3a62d7d549a9a5c42def46769ac4: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:25.422544 19690 leader_election.cc:304] T 2a411802fdaa42e7863236f0af595fe2 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4; no voters: 
W20260709 13:20:25.430435 19690 leader_election.cc:343] T 2a411802fdaa42e7863236f0af595fe2 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:25.431005 20033 raft_consensus.cc:2804] T 2a411802fdaa42e7863236f0af595fe2 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:25.432124 20033 raft_consensus.cc:493] T 2a411802fdaa42e7863236f0af595fe2 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:25.432477 20033 raft_consensus.cc:3060] T 2a411802fdaa42e7863236f0af595fe2 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:20:25.417989 19765 leader_election.cc:343] T 353a546cf8c54671971c239502ef165a P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c62716bf66c24330860573535b8b9430 (127.9.116.132:42643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:25.418300 20065 raft_consensus.cc:2804] T d762eace063c4e04821c749d21702c3f P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:25.430984 19946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f3d7bd0ea32401fa8cb8ab3292a7808" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:25.435209 19765 leader_election.cc:304] T 353a546cf8c54671971c239502ef165a P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e90ffed3ba174ababeb21d9415d4d918; no voters: 8df38c4c53c44e94b36c1e8992476d7c, c62716bf66c24330860573535b8b9430
I20260709 13:20:25.435976 19946 raft_consensus.cc:3060] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:25.436709 20040 raft_consensus.cc:2749] T 353a546cf8c54671971c239502ef165a P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:25.435238 20065 raft_consensus.cc:493] T d762eace063c4e04821c749d21702c3f P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:25.438369 20065 raft_consensus.cc:3060] T d762eace063c4e04821c749d21702c3f P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:25.429415 20064 raft_consensus.cc:3060] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:25.423442 19993 tablet_bootstrap.cc:492] T 2a411802fdaa42e7863236f0af595fe2 P e90ffed3ba174ababeb21d9415d4d918: No bootstrap required, opened a new log
I20260709 13:20:25.447580 19993 ts_tablet_manager.cc:1403] T 2a411802fdaa42e7863236f0af595fe2 P e90ffed3ba174ababeb21d9415d4d918: Time spent bootstrapping tablet: real 0.109s	user 0.010s	sys 0.008s
I20260709 13:20:25.450701 20083 consensus_queue.cc:1048] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:20:25.459530 19997 tablet_bootstrap.cc:492] T a36c11f368f34bf0aa4925a099ac1f91 P b80b3a62d7d549a9a5c42def46769ac4: No bootstrap required, opened a new log
I20260709 13:20:25.460048 19997 ts_tablet_manager.cc:1403] T a36c11f368f34bf0aa4925a099ac1f91 P b80b3a62d7d549a9a5c42def46769ac4: Time spent bootstrapping tablet: real 0.062s	user 0.019s	sys 0.004s
I20260709 13:20:25.429347 19995 ts_tablet_manager.cc:1403] T 3cad7a5f46f9407a99ea6f2cc5a8322c P c62716bf66c24330860573535b8b9430: Time spent bootstrapping tablet: real 0.123s	user 0.023s	sys 0.010s
W20260709 13:20:25.460503 19765 leader_election.cc:343] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:25.461649 19765 leader_election.cc:304] T 2eef64a7d31b4e1e899aee0857cb0e07 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c62716bf66c24330860573535b8b9430, e90ffed3ba174ababeb21d9415d4d918; no voters: 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:25.462972 19997 raft_consensus.cc:359] T a36c11f368f34bf0aa4925a099ac1f91 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:25.463730 19997 raft_consensus.cc:385] T a36c11f368f34bf0aa4925a099ac1f91 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:25.464002 19997 raft_consensus.cc:740] T a36c11f368f34bf0aa4925a099ac1f91 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Initialized, Role: FOLLOWER
I20260709 13:20:25.464768 19997 consensus_queue.cc:260] T a36c11f368f34bf0aa4925a099ac1f91 P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:25.468978 19997 ts_tablet_manager.cc:1434] T a36c11f368f34bf0aa4925a099ac1f91 P b80b3a62d7d549a9a5c42def46769ac4: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:20:25.469676 20001 raft_consensus.cc:2804] T 2eef64a7d31b4e1e899aee0857cb0e07 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:25.470247 20001 raft_consensus.cc:697] T 2eef64a7d31b4e1e899aee0857cb0e07 P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Becoming Leader. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Running, Role: LEADER
I20260709 13:20:25.469425 19623 catalog_manager.cc:5697] T 180230b77b7640b29d055ae8f4358ffe P e90ffed3ba174ababeb21d9415d4d918 reported cstate change: term changed from 0 to 1, leader changed from <none> to e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "e90ffed3ba174ababeb21d9415d4d918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:25.471140 20001 consensus_queue.cc:237] T 2eef64a7d31b4e1e899aee0857cb0e07 P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:25.416072 19994 ts_tablet_manager.cc:1403] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.115s	user 0.018s	sys 0.019s
I20260709 13:20:25.481240 19997 tablet_bootstrap.cc:492] T ac3bca5bb7644a8fbc78df43ef3331a0 P b80b3a62d7d549a9a5c42def46769ac4: Bootstrap starting.
I20260709 13:20:25.484181 19995 raft_consensus.cc:359] T 3cad7a5f46f9407a99ea6f2cc5a8322c P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:25.485046 19995 raft_consensus.cc:385] T 3cad7a5f46f9407a99ea6f2cc5a8322c P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:25.485368 19995 raft_consensus.cc:740] T 3cad7a5f46f9407a99ea6f2cc5a8322c P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62716bf66c24330860573535b8b9430, State: Initialized, Role: FOLLOWER
I20260709 13:20:25.488492 19997 tablet_bootstrap.cc:654] T ac3bca5bb7644a8fbc78df43ef3331a0 P b80b3a62d7d549a9a5c42def46769ac4: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:25.488719 19946 raft_consensus.cc:2468] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 1.
I20260709 13:20:25.494481 19839 leader_election.cc:304] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430; no voters: 
I20260709 13:20:25.496261 20037 raft_consensus.cc:2804] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:25.497023 20037 raft_consensus.cc:697] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 LEADER]: Becoming Leader. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Running, Role: LEADER
I20260709 13:20:25.498284 20037 consensus_queue.cc:237] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:25.500669 19993 raft_consensus.cc:359] T 2a411802fdaa42e7863236f0af595fe2 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:25.501907 19993 raft_consensus.cc:385] T 2a411802fdaa42e7863236f0af595fe2 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:25.502297 19993 raft_consensus.cc:740] T 2a411802fdaa42e7863236f0af595fe2 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Initialized, Role: FOLLOWER
I20260709 13:20:25.504067 19997 tablet_bootstrap.cc:492] T ac3bca5bb7644a8fbc78df43ef3331a0 P b80b3a62d7d549a9a5c42def46769ac4: No bootstrap required, opened a new log
I20260709 13:20:25.477757 19994 raft_consensus.cc:359] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:25.504642 19994 raft_consensus.cc:385] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:25.504671 19997 ts_tablet_manager.cc:1403] T ac3bca5bb7644a8fbc78df43ef3331a0 P b80b3a62d7d549a9a5c42def46769ac4: Time spent bootstrapping tablet: real 0.024s	user 0.018s	sys 0.000s
I20260709 13:20:25.505158 19994 raft_consensus.cc:740] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:25.506546 19994 consensus_queue.cc:260] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:25.511588 19994 ts_tablet_manager.cc:1434] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.036s	user 0.007s	sys 0.000s
I20260709 13:20:25.517304 19994 tablet_bootstrap.cc:492] T d762eace063c4e04821c749d21702c3f P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:25.519721 20040 consensus_queue.cc:1048] T af881b097b4245b1be6072579197cb6b P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:20:25.509163 20009 leader_election.cc:290] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 election: Requested vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:25.509372 19723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f3d7bd0ea32401fa8cb8ab3292a7808" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
I20260709 13:20:25.527911 19723 raft_consensus.cc:3060] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:25.509873 20079 consensus_queue.cc:1048] T f820e93ffdcd428ea4adfca88589a850 P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.022s
I20260709 13:20:25.537128 19994 tablet_bootstrap.cc:654] T d762eace063c4e04821c749d21702c3f P 8df38c4c53c44e94b36c1e8992476d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:25.541188 19623 catalog_manager.cc:5697] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P b80b3a62d7d549a9a5c42def46769ac4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131). New cstate: current_term: 1 leader_uuid: "b80b3a62d7d549a9a5c42def46769ac4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:25.490854 19995 consensus_queue.cc:260] T 3cad7a5f46f9407a99ea6f2cc5a8322c P c62716bf66c24330860573535b8b9430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:25.547389 19995 ts_tablet_manager.cc:1434] T 3cad7a5f46f9407a99ea6f2cc5a8322c P c62716bf66c24330860573535b8b9430: Time spent starting tablet: real 0.087s	user 0.007s	sys 0.000s
I20260709 13:20:25.548511 19995 tablet_bootstrap.cc:492] T ac3bca5bb7644a8fbc78df43ef3331a0 P c62716bf66c24330860573535b8b9430: Bootstrap starting.
I20260709 13:20:25.552573 19994 tablet_bootstrap.cc:492] T d762eace063c4e04821c749d21702c3f P 8df38c4c53c44e94b36c1e8992476d7c: No bootstrap required, opened a new log
I20260709 13:20:25.553305 19994 ts_tablet_manager.cc:1403] T d762eace063c4e04821c749d21702c3f P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.036s	user 0.011s	sys 0.002s
I20260709 13:20:25.559985 20065 raft_consensus.cc:515] T d762eace063c4e04821c749d21702c3f P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:25.561914 19994 raft_consensus.cc:359] T d762eace063c4e04821c749d21702c3f P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:25.562860 19994 raft_consensus.cc:385] T d762eace063c4e04821c749d21702c3f P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:25.563244 19994 raft_consensus.cc:740] T d762eace063c4e04821c749d21702c3f P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:25.564282 19994 consensus_queue.cc:260] T d762eace063c4e04821c749d21702c3f P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:25.565125 19621 catalog_manager.cc:5697] T e8cfec59c63741869c636df6356546d0 P c62716bf66c24330860573535b8b9430 reported cstate change: term changed from 0 to 1, leader changed from <none> to c62716bf66c24330860573535b8b9430 (127.9.116.132). New cstate: current_term: 1 leader_uuid: "c62716bf66c24330860573535b8b9430" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:25.566272 19994 ts_tablet_manager.cc:1434] T d762eace063c4e04821c749d21702c3f P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:20:25.566129 19993 consensus_queue.cc:260] T 2a411802fdaa42e7863236f0af595fe2 P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:25.567502 19994 tablet_bootstrap.cc:492] T 2eef64a7d31b4e1e899aee0857cb0e07 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:25.568095 19946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d762eace063c4e04821c749d21702c3f" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:25.568816 19946 raft_consensus.cc:3060] T d762eace063c4e04821c749d21702c3f P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:25.571256 19622 catalog_manager.cc:5697] T 2eef64a7d31b4e1e899aee0857cb0e07 P e90ffed3ba174ababeb21d9415d4d918 reported cstate change: term changed from 0 to 1, leader changed from <none> to e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "e90ffed3ba174ababeb21d9415d4d918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:25.574983 19994 tablet_bootstrap.cc:654] T 2eef64a7d31b4e1e899aee0857cb0e07 P 8df38c4c53c44e94b36c1e8992476d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:25.576558 19723 raft_consensus.cc:2468] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 1.
I20260709 13:20:25.578343 19946 raft_consensus.cc:2468] T d762eace063c4e04821c749d21702c3f P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 1.
I20260709 13:20:25.580992 20064 raft_consensus.cc:515] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:25.589373 19993 ts_tablet_manager.cc:1434] T 2a411802fdaa42e7863236f0af595fe2 P e90ffed3ba174ababeb21d9415d4d918: Time spent starting tablet: real 0.141s	user 0.007s	sys 0.000s
I20260709 13:20:25.590493 19993 tablet_bootstrap.cc:492] T 19a59035bda747fdbc59b7e6914101dd P e90ffed3ba174ababeb21d9415d4d918: Bootstrap starting.
I20260709 13:20:25.596876 19874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9f7b34cb064bbe8a6938e4f92d57c5" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
I20260709 13:20:25.598526 20040 raft_consensus.cc:493] T 6a066b96902f41dbb1511eccf26820ce P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:25.598814 19765 leader_election.cc:304] T d762eace063c4e04821c749d21702c3f P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c62716bf66c24330860573535b8b9430, e90ffed3ba174ababeb21d9415d4d918; no voters: 
I20260709 13:20:25.513016 20033 raft_consensus.cc:515] T 2a411802fdaa42e7863236f0af595fe2 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:25.599861 19995 tablet_bootstrap.cc:654] T ac3bca5bb7644a8fbc78df43ef3331a0 P c62716bf66c24330860573535b8b9430: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:25.599184 20040 raft_consensus.cc:515] T 6a066b96902f41dbb1511eccf26820ce P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:25.508526 19997 raft_consensus.cc:359] T ac3bca5bb7644a8fbc78df43ef3331a0 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:25.609771 20033 leader_election.cc:290] T 2a411802fdaa42e7863236f0af595fe2 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Requested vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:25.605960 19993 tablet_bootstrap.cc:654] T 19a59035bda747fdbc59b7e6914101dd P e90ffed3ba174ababeb21d9415d4d918: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:25.611467 19874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a411802fdaa42e7863236f0af595fe2" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
I20260709 13:20:25.611806 20040 leader_election.cc:290] T 6a066b96902f41dbb1511eccf26820ce P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:25.600593 20064 leader_election.cc:290] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 election: Requested vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:25.609830 19997 raft_consensus.cc:385] T ac3bca5bb7644a8fbc78df43ef3331a0 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:25.612178 19874 raft_consensus.cc:3060] T 2a411802fdaa42e7863236f0af595fe2 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:25.612452 19997 raft_consensus.cc:740] T ac3bca5bb7644a8fbc78df43ef3331a0 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Initialized, Role: FOLLOWER
I20260709 13:20:25.613329 19997 consensus_queue.cc:260] T ac3bca5bb7644a8fbc78df43ef3331a0 P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:25.615746 19997 ts_tablet_manager.cc:1434] T ac3bca5bb7644a8fbc78df43ef3331a0 P b80b3a62d7d549a9a5c42def46769ac4: Time spent starting tablet: real 0.111s	user 0.007s	sys 0.000s
I20260709 13:20:25.616312 19723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a066b96902f41dbb1511eccf26820ce" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:25.616945 19997 tablet_bootstrap.cc:492] T 6a066b96902f41dbb1511eccf26820ce P b80b3a62d7d549a9a5c42def46769ac4: Bootstrap starting.
I20260709 13:20:25.617079 19723 raft_consensus.cc:2468] T 6a066b96902f41dbb1511eccf26820ce P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 0.
I20260709 13:20:25.619063 19766 leader_election.cc:304] T 6a066b96902f41dbb1511eccf26820ce P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918; no voters: 
I20260709 13:20:25.620867 19874 raft_consensus.cc:2468] T 2a411802fdaa42e7863236f0af595fe2 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 1.
I20260709 13:20:25.622432 19690 leader_election.cc:304] T 2a411802fdaa42e7863236f0af595fe2 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4; no voters: 
I20260709 13:20:25.623378 20033 raft_consensus.cc:2804] T 2a411802fdaa42e7863236f0af595fe2 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:25.623878 20033 raft_consensus.cc:697] T 2a411802fdaa42e7863236f0af595fe2 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 LEADER]: Becoming Leader. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Running, Role: LEADER
I20260709 13:20:25.624694 20033 consensus_queue.cc:237] T 2a411802fdaa42e7863236f0af595fe2 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:25.626026 20064 raft_consensus.cc:2804] T 6a066b96902f41dbb1511eccf26820ce P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:25.626530 20064 raft_consensus.cc:493] T 6a066b96902f41dbb1511eccf26820ce P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:25.626981 20064 raft_consensus.cc:3060] T 6a066b96902f41dbb1511eccf26820ce P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:25.631347 19799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a411802fdaa42e7863236f0af595fe2" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
I20260709 13:20:25.635923 19799 raft_consensus.cc:3060] T 2a411802fdaa42e7863236f0af595fe2 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:25.639839 19993 tablet_bootstrap.cc:492] T 19a59035bda747fdbc59b7e6914101dd P e90ffed3ba174ababeb21d9415d4d918: No bootstrap required, opened a new log
I20260709 13:20:25.640383 19993 ts_tablet_manager.cc:1403] T 19a59035bda747fdbc59b7e6914101dd P e90ffed3ba174ababeb21d9415d4d918: Time spent bootstrapping tablet: real 0.050s	user 0.022s	sys 0.009s
I20260709 13:20:25.643406 19993 raft_consensus.cc:359] T 19a59035bda747fdbc59b7e6914101dd P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:25.644162 19993 raft_consensus.cc:385] T 19a59035bda747fdbc59b7e6914101dd P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:25.644440 19993 raft_consensus.cc:740] T 19a59035bda747fdbc59b7e6914101dd P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Initialized, Role: FOLLOWER
I20260709 13:20:25.645095 19993 consensus_queue.cc:260] T 19a59035bda747fdbc59b7e6914101dd P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:25.647625 19993 ts_tablet_manager.cc:1434] T 19a59035bda747fdbc59b7e6914101dd P e90ffed3ba174ababeb21d9415d4d918: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:20:25.649024 19993 tablet_bootstrap.cc:492] T 3cad7a5f46f9407a99ea6f2cc5a8322c P e90ffed3ba174ababeb21d9415d4d918: Bootstrap starting.
I20260709 13:20:25.648911 19799 raft_consensus.cc:2468] T 2a411802fdaa42e7863236f0af595fe2 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 1.
I20260709 13:20:25.649765 20065 leader_election.cc:290] T d762eace063c4e04821c749d21702c3f P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 election: Requested vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:25.651453 19723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d762eace063c4e04821c749d21702c3f" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
I20260709 13:20:25.652127 19723 raft_consensus.cc:3060] T d762eace063c4e04821c749d21702c3f P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:20:25.652684 19765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.132:42643, user_credentials={real_user=slave}} blocked reactor thread for 54101.1us
I20260709 13:20:25.655596 19993 tablet_bootstrap.cc:654] T 3cad7a5f46f9407a99ea6f2cc5a8322c P e90ffed3ba174ababeb21d9415d4d918: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:25.652156 20065 raft_consensus.cc:2804] T d762eace063c4e04821c749d21702c3f P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:25.656150 20065 raft_consensus.cc:697] T d762eace063c4e04821c749d21702c3f P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Becoming Leader. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Running, Role: LEADER
I20260709 13:20:25.607738 19994 tablet_bootstrap.cc:492] T 2eef64a7d31b4e1e899aee0857cb0e07 P 8df38c4c53c44e94b36c1e8992476d7c: No bootstrap required, opened a new log
I20260709 13:20:25.659164 19995 tablet_bootstrap.cc:492] T ac3bca5bb7644a8fbc78df43ef3331a0 P c62716bf66c24330860573535b8b9430: No bootstrap required, opened a new log
I20260709 13:20:25.659494 19994 ts_tablet_manager.cc:1403] T 2eef64a7d31b4e1e899aee0857cb0e07 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.092s	user 0.021s	sys 0.015s
I20260709 13:20:25.659631 19995 ts_tablet_manager.cc:1403] T ac3bca5bb7644a8fbc78df43ef3331a0 P c62716bf66c24330860573535b8b9430: Time spent bootstrapping tablet: real 0.111s	user 0.010s	sys 0.009s
I20260709 13:20:25.659967 20065 consensus_queue.cc:237] T d762eace063c4e04821c749d21702c3f P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:25.664333 19946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9f7b34cb064bbe8a6938e4f92d57c5" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:25.664494 19995 raft_consensus.cc:359] T ac3bca5bb7644a8fbc78df43ef3331a0 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:25.665011 19946 raft_consensus.cc:3060] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:25.665169 19995 raft_consensus.cc:385] T ac3bca5bb7644a8fbc78df43ef3331a0 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:25.665498 19995 raft_consensus.cc:740] T ac3bca5bb7644a8fbc78df43ef3331a0 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62716bf66c24330860573535b8b9430, State: Initialized, Role: FOLLOWER
I20260709 13:20:25.668767 19723 raft_consensus.cc:2468] T d762eace063c4e04821c749d21702c3f P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 1.
I20260709 13:20:25.668885 19995 consensus_queue.cc:260] T ac3bca5bb7644a8fbc78df43ef3331a0 P c62716bf66c24330860573535b8b9430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:25.673687 19946 raft_consensus.cc:2468] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 1.
I20260709 13:20:25.674616 20009 raft_consensus.cc:493] T 19a59035bda747fdbc59b7e6914101dd P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:25.675458 20009 raft_consensus.cc:515] T 19a59035bda747fdbc59b7e6914101dd P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
W20260709 13:20:25.676746 19765 leader_election.cc:343] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:25.678826 19997 tablet_bootstrap.cc:654] T 6a066b96902f41dbb1511eccf26820ce P b80b3a62d7d549a9a5c42def46769ac4: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:25.678552 19874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a066b96902f41dbb1511eccf26820ce" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:25.680859 19995 ts_tablet_manager.cc:1434] T ac3bca5bb7644a8fbc78df43ef3331a0 P c62716bf66c24330860573535b8b9430: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.003s
I20260709 13:20:25.662585 19994 raft_consensus.cc:359] T 2eef64a7d31b4e1e899aee0857cb0e07 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:25.682030 19994 raft_consensus.cc:385] T 2eef64a7d31b4e1e899aee0857cb0e07 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:25.682361 19994 raft_consensus.cc:740] T 2eef64a7d31b4e1e899aee0857cb0e07 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:25.684072 19995 tablet_bootstrap.cc:492] T 9b9137146e3644a6b0026778530fa8a6 P c62716bf66c24330860573535b8b9430: Bootstrap starting.
I20260709 13:20:25.684365 19994 consensus_queue.cc:260] T 2eef64a7d31b4e1e899aee0857cb0e07 P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:25.683354 20000 raft_consensus.cc:493] T 3cad7a5f46f9407a99ea6f2cc5a8322c P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:25.686067 20000 raft_consensus.cc:515] T 3cad7a5f46f9407a99ea6f2cc5a8322c P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:25.687232 20040 consensus_queue.cc:1048] T af881b097b4245b1be6072579197cb6b P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:25.687538 19799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19a59035bda747fdbc59b7e6914101dd" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:25.688252 20000 leader_election.cc:290] T 3cad7a5f46f9407a99ea6f2cc5a8322c P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:25.688529 19799 raft_consensus.cc:2468] T 19a59035bda747fdbc59b7e6914101dd P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 0.
I20260709 13:20:25.689167 20009 leader_election.cc:290] T 19a59035bda747fdbc59b7e6914101dd P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:25.693193 19946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19a59035bda747fdbc59b7e6914101dd" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:25.693380 19839 leader_election.cc:304] T 19a59035bda747fdbc59b7e6914101dd P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, e90ffed3ba174ababeb21d9415d4d918; no voters: 
I20260709 13:20:25.693966 19946 raft_consensus.cc:2468] T 19a59035bda747fdbc59b7e6914101dd P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 0.
I20260709 13:20:25.694329 20009 raft_consensus.cc:2804] T 19a59035bda747fdbc59b7e6914101dd P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:25.694764 20009 raft_consensus.cc:493] T 19a59035bda747fdbc59b7e6914101dd P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:25.695154 20009 raft_consensus.cc:3060] T 19a59035bda747fdbc59b7e6914101dd P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:25.694051 19874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cad7a5f46f9407a99ea6f2cc5a8322c" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:25.696890 19874 raft_consensus.cc:2468] T 3cad7a5f46f9407a99ea6f2cc5a8322c P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c62716bf66c24330860573535b8b9430 in term 0.
I20260709 13:20:25.698407 19913 leader_election.cc:304] T 3cad7a5f46f9407a99ea6f2cc5a8322c P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430; no voters: 
I20260709 13:20:25.699571 19621 catalog_manager.cc:5697] T 2a411802fdaa42e7863236f0af595fe2 P 8df38c4c53c44e94b36c1e8992476d7c reported cstate change: term changed from 0 to 1, leader changed from <none> to 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129). New cstate: current_term: 1 leader_uuid: "8df38c4c53c44e94b36c1e8992476d7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:25.700990 20000 raft_consensus.cc:2804] T 3cad7a5f46f9407a99ea6f2cc5a8322c P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:25.706360 20000 raft_consensus.cc:493] T 3cad7a5f46f9407a99ea6f2cc5a8322c P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:25.707937 20035 raft_consensus.cc:493] T ac3bca5bb7644a8fbc78df43ef3331a0 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:25.710214 20035 raft_consensus.cc:515] T ac3bca5bb7644a8fbc78df43ef3331a0 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:25.712026 20064 raft_consensus.cc:515] T 6a066b96902f41dbb1511eccf26820ce P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:25.708659 19765 leader_election.cc:304] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c62716bf66c24330860573535b8b9430, e90ffed3ba174ababeb21d9415d4d918; no voters: b80b3a62d7d549a9a5c42def46769ac4
I20260709 13:20:25.714236 20064 leader_election.cc:290] T 6a066b96902f41dbb1511eccf26820ce P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 election: Requested vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:25.714167 19874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3bca5bb7644a8fbc78df43ef3331a0" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:25.714823 20035 leader_election.cc:290] T ac3bca5bb7644a8fbc78df43ef3331a0 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:25.721910 19994 ts_tablet_manager.cc:1434] T 2eef64a7d31b4e1e899aee0857cb0e07 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.062s	user 0.007s	sys 0.000s
I20260709 13:20:25.714901 20000 raft_consensus.cc:3060] T 3cad7a5f46f9407a99ea6f2cc5a8322c P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:25.716457 19724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a066b96902f41dbb1511eccf26820ce" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
I20260709 13:20:25.724191 19724 raft_consensus.cc:3060] T 6a066b96902f41dbb1511eccf26820ce P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:25.724517 19994 tablet_bootstrap.cc:492] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:25.722337 19995 tablet_bootstrap.cc:654] T 9b9137146e3644a6b0026778530fa8a6 P c62716bf66c24330860573535b8b9430: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:25.715425 19723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3bca5bb7644a8fbc78df43ef3331a0" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:25.718156 19872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a066b96902f41dbb1511eccf26820ce" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
I20260709 13:20:25.727773 19723 raft_consensus.cc:2468] T ac3bca5bb7644a8fbc78df43ef3331a0 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c62716bf66c24330860573535b8b9430 in term 0.
W20260709 13:20:25.715719 19765 leader_election.cc:343] T 6a066b96902f41dbb1511eccf26820ce P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:25.722759 19874 raft_consensus.cc:2468] T ac3bca5bb7644a8fbc78df43ef3331a0 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c62716bf66c24330860573535b8b9430 in term 0.
I20260709 13:20:25.730520 19993 tablet_bootstrap.cc:492] T 3cad7a5f46f9407a99ea6f2cc5a8322c P e90ffed3ba174ababeb21d9415d4d918: No bootstrap required, opened a new log
I20260709 13:20:25.732239 19993 ts_tablet_manager.cc:1403] T 3cad7a5f46f9407a99ea6f2cc5a8322c P e90ffed3ba174ababeb21d9415d4d918: Time spent bootstrapping tablet: real 0.083s	user 0.014s	sys 0.030s
I20260709 13:20:25.730509 19914 leader_election.cc:304] T ac3bca5bb7644a8fbc78df43ef3331a0 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, c62716bf66c24330860573535b8b9430; no voters: 
I20260709 13:20:25.734172 20035 raft_consensus.cc:2804] T ac3bca5bb7644a8fbc78df43ef3331a0 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:25.734602 20035 raft_consensus.cc:493] T ac3bca5bb7644a8fbc78df43ef3331a0 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:25.735028 20035 raft_consensus.cc:3060] T ac3bca5bb7644a8fbc78df43ef3331a0 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:25.735364 19997 tablet_bootstrap.cc:492] T 6a066b96902f41dbb1511eccf26820ce P b80b3a62d7d549a9a5c42def46769ac4: No bootstrap required, opened a new log
I20260709 13:20:25.740813 19997 ts_tablet_manager.cc:1403] T 6a066b96902f41dbb1511eccf26820ce P b80b3a62d7d549a9a5c42def46769ac4: Time spent bootstrapping tablet: real 0.124s	user 0.016s	sys 0.000s
I20260709 13:20:25.728238 20065 raft_consensus.cc:2804] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:25.742089 20065 raft_consensus.cc:697] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Becoming Leader. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Running, Role: LEADER
I20260709 13:20:25.743017 20065 consensus_queue.cc:237] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
W20260709 13:20:25.735999 19765 leader_election.cc:343] T 6a066b96902f41dbb1511eccf26820ce P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:25.736002 19799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cad7a5f46f9407a99ea6f2cc5a8322c" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:25.743664 19997 raft_consensus.cc:359] T 6a066b96902f41dbb1511eccf26820ce P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:25.744504 19994 tablet_bootstrap.cc:654] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:25.766510 19997 raft_consensus.cc:385] T 6a066b96902f41dbb1511eccf26820ce P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:25.767163 19997 raft_consensus.cc:740] T 6a066b96902f41dbb1511eccf26820ce P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Initialized, Role: FOLLOWER
I20260709 13:20:25.766180 19993 raft_consensus.cc:359] T 3cad7a5f46f9407a99ea6f2cc5a8322c P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:25.770411 19993 raft_consensus.cc:385] T 3cad7a5f46f9407a99ea6f2cc5a8322c P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:25.770653 19993 raft_consensus.cc:740] T 3cad7a5f46f9407a99ea6f2cc5a8322c P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Initialized, Role: FOLLOWER
I20260709 13:20:25.772025 19993 consensus_queue.cc:260] T 3cad7a5f46f9407a99ea6f2cc5a8322c P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
W20260709 13:20:25.747385 19913 leader_election.cc:343] T 3cad7a5f46f9407a99ea6f2cc5a8322c P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:25.769912 20000 raft_consensus.cc:515] T 3cad7a5f46f9407a99ea6f2cc5a8322c P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:25.721014 19620 catalog_manager.cc:5697] T d762eace063c4e04821c749d21702c3f P e90ffed3ba174ababeb21d9415d4d918 reported cstate change: term changed from 0 to 1, leader changed from <none> to e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "e90ffed3ba174ababeb21d9415d4d918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:25.775312 20000 leader_election.cc:290] T 3cad7a5f46f9407a99ea6f2cc5a8322c P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 election: Requested vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:25.776345 19993 ts_tablet_manager.cc:1434] T 3cad7a5f46f9407a99ea6f2cc5a8322c P e90ffed3ba174ababeb21d9415d4d918: Time spent starting tablet: real 0.044s	user 0.009s	sys 0.007s
I20260709 13:20:25.776512 19799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cad7a5f46f9407a99ea6f2cc5a8322c" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
I20260709 13:20:25.777428 19993 tablet_bootstrap.cc:492] T 984fa8de2ccb49359c2b475e4adfc1ce P e90ffed3ba174ababeb21d9415d4d918: Bootstrap starting.
I20260709 13:20:25.778165 19874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cad7a5f46f9407a99ea6f2cc5a8322c" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
I20260709 13:20:25.778815 19874 raft_consensus.cc:3060] T 3cad7a5f46f9407a99ea6f2cc5a8322c P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:25.777393 19799 raft_consensus.cc:3060] T 3cad7a5f46f9407a99ea6f2cc5a8322c P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:25.785413 20035 raft_consensus.cc:515] T ac3bca5bb7644a8fbc78df43ef3331a0 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:25.788275 19724 raft_consensus.cc:2468] T 6a066b96902f41dbb1511eccf26820ce P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 1.
I20260709 13:20:25.789249 19997 consensus_queue.cc:260] T 6a066b96902f41dbb1511eccf26820ce P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:25.790019 19766 leader_election.cc:304] T 6a066b96902f41dbb1511eccf26820ce P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918; no voters: b80b3a62d7d549a9a5c42def46769ac4
I20260709 13:20:25.791115 20064 raft_consensus.cc:2804] T 6a066b96902f41dbb1511eccf26820ce P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:25.791611 20064 raft_consensus.cc:697] T 6a066b96902f41dbb1511eccf26820ce P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Becoming Leader. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Running, Role: LEADER
I20260709 13:20:25.792006 19872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3bca5bb7644a8fbc78df43ef3331a0" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
I20260709 13:20:25.792472 20064 consensus_queue.cc:237] T 6a066b96902f41dbb1511eccf26820ce P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:25.792934 19872 raft_consensus.cc:3060] T ac3bca5bb7644a8fbc78df43ef3331a0 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:25.807358 19799 raft_consensus.cc:2468] T 3cad7a5f46f9407a99ea6f2cc5a8322c P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c62716bf66c24330860573535b8b9430 in term 1.
I20260709 13:20:25.810088 19913 leader_election.cc:304] T 3cad7a5f46f9407a99ea6f2cc5a8322c P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c62716bf66c24330860573535b8b9430, e90ffed3ba174ababeb21d9415d4d918; no voters: 
I20260709 13:20:25.811220 20000 raft_consensus.cc:2804] T 3cad7a5f46f9407a99ea6f2cc5a8322c P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:25.811968 19997 ts_tablet_manager.cc:1434] T 6a066b96902f41dbb1511eccf26820ce P b80b3a62d7d549a9a5c42def46769ac4: Time spent starting tablet: real 0.071s	user 0.011s	sys 0.006s
I20260709 13:20:25.813438 19997 tablet_bootstrap.cc:492] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P b80b3a62d7d549a9a5c42def46769ac4: Bootstrap starting.
I20260709 13:20:25.826153 19872 raft_consensus.cc:2468] T ac3bca5bb7644a8fbc78df43ef3331a0 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c62716bf66c24330860573535b8b9430 in term 1.
I20260709 13:20:25.828016 19913 leader_election.cc:304] T ac3bca5bb7644a8fbc78df43ef3331a0 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430; no voters: 
I20260709 13:20:25.804906 19874 raft_consensus.cc:2468] T 3cad7a5f46f9407a99ea6f2cc5a8322c P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c62716bf66c24330860573535b8b9430 in term 1.
I20260709 13:20:25.836800 20000 raft_consensus.cc:697] T 3cad7a5f46f9407a99ea6f2cc5a8322c P c62716bf66c24330860573535b8b9430 [term 1 LEADER]: Becoming Leader. State: Replica: c62716bf66c24330860573535b8b9430, State: Running, Role: LEADER
I20260709 13:20:25.841166 20000 consensus_queue.cc:237] T 3cad7a5f46f9407a99ea6f2cc5a8322c P c62716bf66c24330860573535b8b9430 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:25.849110 20105 raft_consensus.cc:2804] T ac3bca5bb7644a8fbc78df43ef3331a0 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:25.849854 20037 raft_consensus.cc:493] T a36c11f368f34bf0aa4925a099ac1f91 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:25.849936 20105 raft_consensus.cc:697] T ac3bca5bb7644a8fbc78df43ef3331a0 P c62716bf66c24330860573535b8b9430 [term 1 LEADER]: Becoming Leader. State: Replica: c62716bf66c24330860573535b8b9430, State: Running, Role: LEADER
I20260709 13:20:25.850540 20037 raft_consensus.cc:515] T a36c11f368f34bf0aa4925a099ac1f91 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:25.851606 20105 consensus_queue.cc:237] T ac3bca5bb7644a8fbc78df43ef3331a0 P c62716bf66c24330860573535b8b9430 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:25.855571 19799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a36c11f368f34bf0aa4925a099ac1f91" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:25.856364 20037 leader_election.cc:290] T a36c11f368f34bf0aa4925a099ac1f91 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
W20260709 13:20:25.857461 19839 leader_election.cc:343] T a36c11f368f34bf0aa4925a099ac1f91 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:25.859879 19994 tablet_bootstrap.cc:492] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c: No bootstrap required, opened a new log
I20260709 13:20:25.860421 19994 ts_tablet_manager.cc:1403] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.136s	user 0.024s	sys 0.029s
I20260709 13:20:25.862171 19995 tablet_bootstrap.cc:492] T 9b9137146e3644a6b0026778530fa8a6 P c62716bf66c24330860573535b8b9430: No bootstrap required, opened a new log
I20260709 13:20:25.862663 19995 ts_tablet_manager.cc:1403] T 9b9137146e3644a6b0026778530fa8a6 P c62716bf66c24330860573535b8b9430: Time spent bootstrapping tablet: real 0.181s	user 0.018s	sys 0.013s
I20260709 13:20:25.867027 20009 raft_consensus.cc:515] T 19a59035bda747fdbc59b7e6914101dd P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:25.870343 19724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3bca5bb7644a8fbc78df43ef3331a0" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
I20260709 13:20:25.871008 19724 raft_consensus.cc:3060] T ac3bca5bb7644a8fbc78df43ef3331a0 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:25.874004 19997 tablet_bootstrap.cc:654] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P b80b3a62d7d549a9a5c42def46769ac4: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:25.867051 20035 leader_election.cc:290] T ac3bca5bb7644a8fbc78df43ef3331a0 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 election: Requested vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:25.868332 19995 raft_consensus.cc:359] T 9b9137146e3644a6b0026778530fa8a6 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:25.876425 19995 raft_consensus.cc:385] T 9b9137146e3644a6b0026778530fa8a6 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:25.877013 19995 raft_consensus.cc:740] T 9b9137146e3644a6b0026778530fa8a6 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62716bf66c24330860573535b8b9430, State: Initialized, Role: FOLLOWER
I20260709 13:20:25.877993 19995 consensus_queue.cc:260] T 9b9137146e3644a6b0026778530fa8a6 P c62716bf66c24330860573535b8b9430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:25.879038 20035 consensus_queue.cc:1048] T e8cfec59c63741869c636df6356546d0 P c62716bf66c24330860573535b8b9430 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:20:25.880188 19946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a36c11f368f34bf0aa4925a099ac1f91" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:25.880997 19995 ts_tablet_manager.cc:1434] T 9b9137146e3644a6b0026778530fa8a6 P c62716bf66c24330860573535b8b9430: Time spent starting tablet: real 0.018s	user 0.004s	sys 0.004s
I20260709 13:20:25.863349 19994 raft_consensus.cc:359] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
W20260709 13:20:25.882083 19839 leader_election.cc:343] T a36c11f368f34bf0aa4925a099ac1f91 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c62716bf66c24330860573535b8b9430 (127.9.116.132:42643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:25.882681 19994 raft_consensus.cc:385] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:25.882315 19799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19a59035bda747fdbc59b7e6914101dd" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
I20260709 13:20:25.882761 19839 leader_election.cc:304] T a36c11f368f34bf0aa4925a099ac1f91 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4; no voters: c62716bf66c24330860573535b8b9430, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:25.883394 19799 raft_consensus.cc:3060] T 19a59035bda747fdbc59b7e6914101dd P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:25.883646 20037 raft_consensus.cc:2749] T a36c11f368f34bf0aa4925a099ac1f91 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:25.886065 19946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19a59035bda747fdbc59b7e6914101dd" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:25.886718 19946 raft_consensus.cc:3060] T 19a59035bda747fdbc59b7e6914101dd P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:25.882390 19995 tablet_bootstrap.cc:492] T 19e83b82855d4129bb8525e6ff05316b P c62716bf66c24330860573535b8b9430: Bootstrap starting.
I20260709 13:20:25.883467 20009 leader_election.cc:290] T 19a59035bda747fdbc59b7e6914101dd P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 election: Requested vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:25.886089 19993 tablet_bootstrap.cc:654] T 984fa8de2ccb49359c2b475e4adfc1ce P e90ffed3ba174ababeb21d9415d4d918: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:25.882268 19724 raft_consensus.cc:2468] T ac3bca5bb7644a8fbc78df43ef3331a0 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c62716bf66c24330860573535b8b9430 in term 1.
I20260709 13:20:25.891130 19994 raft_consensus.cc:740] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:25.893023 20064 consensus_queue.cc:1048] T 180230b77b7640b29d055ae8f4358ffe P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:25.899820 19994 consensus_queue.cc:260] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:25.904317 19620 catalog_manager.cc:5697] T 3cad7a5f46f9407a99ea6f2cc5a8322c P c62716bf66c24330860573535b8b9430 reported cstate change: term changed from 0 to 1, leader changed from <none> to c62716bf66c24330860573535b8b9430 (127.9.116.132). New cstate: current_term: 1 leader_uuid: "c62716bf66c24330860573535b8b9430" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:25.913592 19994 ts_tablet_manager.cc:1434] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.053s	user 0.007s	sys 0.000s
I20260709 13:20:25.914642 19994 tablet_bootstrap.cc:492] T 984fa8de2ccb49359c2b475e4adfc1ce P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:25.916000 19993 tablet_bootstrap.cc:492] T 984fa8de2ccb49359c2b475e4adfc1ce P e90ffed3ba174ababeb21d9415d4d918: No bootstrap required, opened a new log
I20260709 13:20:25.917060 19993 ts_tablet_manager.cc:1403] T 984fa8de2ccb49359c2b475e4adfc1ce P e90ffed3ba174ababeb21d9415d4d918: Time spent bootstrapping tablet: real 0.140s	user 0.012s	sys 0.001s
I20260709 13:20:25.920742 19799 raft_consensus.cc:2468] T 19a59035bda747fdbc59b7e6914101dd P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 1.
I20260709 13:20:25.921195 19623 catalog_manager.cc:5697] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P e90ffed3ba174ababeb21d9415d4d918 reported cstate change: term changed from 0 to 1, leader changed from <none> to e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "e90ffed3ba174ababeb21d9415d4d918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:25.922271 19839 leader_election.cc:304] T 19a59035bda747fdbc59b7e6914101dd P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, e90ffed3ba174ababeb21d9415d4d918; no voters: 
I20260709 13:20:25.924968 19995 tablet_bootstrap.cc:654] T 19e83b82855d4129bb8525e6ff05316b P c62716bf66c24330860573535b8b9430: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:25.924822 19993 raft_consensus.cc:359] T 984fa8de2ccb49359c2b475e4adfc1ce P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:25.927052 19993 raft_consensus.cc:385] T 984fa8de2ccb49359c2b475e4adfc1ce P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:25.927948 19993 raft_consensus.cc:740] T 984fa8de2ccb49359c2b475e4adfc1ce P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Initialized, Role: FOLLOWER
I20260709 13:20:25.928882 20009 raft_consensus.cc:2804] T 19a59035bda747fdbc59b7e6914101dd P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:25.928833 19993 consensus_queue.cc:260] T 984fa8de2ccb49359c2b475e4adfc1ce P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:25.901496 19946 raft_consensus.cc:2468] T 19a59035bda747fdbc59b7e6914101dd P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 1.
I20260709 13:20:25.929457 20009 raft_consensus.cc:697] T 19a59035bda747fdbc59b7e6914101dd P b80b3a62d7d549a9a5c42def46769ac4 [term 1 LEADER]: Becoming Leader. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Running, Role: LEADER
I20260709 13:20:25.931434 19993 ts_tablet_manager.cc:1434] T 984fa8de2ccb49359c2b475e4adfc1ce P e90ffed3ba174ababeb21d9415d4d918: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.002s
I20260709 13:20:25.930626 20009 consensus_queue.cc:237] T 19a59035bda747fdbc59b7e6914101dd P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:25.932512 19993 tablet_bootstrap.cc:492] T ad38a1519fb748588f5a322589274004 P e90ffed3ba174ababeb21d9415d4d918: Bootstrap starting.
I20260709 13:20:25.936841 19994 tablet_bootstrap.cc:654] T 984fa8de2ccb49359c2b475e4adfc1ce P 8df38c4c53c44e94b36c1e8992476d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:25.939227 19993 tablet_bootstrap.cc:654] T ad38a1519fb748588f5a322589274004 P e90ffed3ba174ababeb21d9415d4d918: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:25.941442 20035 consensus_queue.cc:1048] T e8cfec59c63741869c636df6356546d0 P c62716bf66c24330860573535b8b9430 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:20:25.926987 19623 catalog_manager.cc:5697] T 6a066b96902f41dbb1511eccf26820ce P e90ffed3ba174ababeb21d9415d4d918 reported cstate change: term changed from 0 to 1, leader changed from <none> to e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "e90ffed3ba174ababeb21d9415d4d918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:25.957190 19997 tablet_bootstrap.cc:492] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P b80b3a62d7d549a9a5c42def46769ac4: No bootstrap required, opened a new log
I20260709 13:20:25.957782 19997 ts_tablet_manager.cc:1403] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P b80b3a62d7d549a9a5c42def46769ac4: Time spent bootstrapping tablet: real 0.145s	user 0.013s	sys 0.000s
I20260709 13:20:25.961076 19997 raft_consensus.cc:359] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:25.961818 19997 raft_consensus.cc:385] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:25.962128 19997 raft_consensus.cc:740] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Initialized, Role: FOLLOWER
I20260709 13:20:25.963068 19997 consensus_queue.cc:260] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:25.965705 19997 ts_tablet_manager.cc:1434] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P b80b3a62d7d549a9a5c42def46769ac4: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 13:20:25.966770 19997 tablet_bootstrap.cc:492] T e804b7d8fd7e44f59516fb299bea7a2c P b80b3a62d7d549a9a5c42def46769ac4: Bootstrap starting.
I20260709 13:20:25.967998 20064 consensus_queue.cc:1048] T 180230b77b7640b29d055ae8f4358ffe P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:25.973158 19997 tablet_bootstrap.cc:654] T e804b7d8fd7e44f59516fb299bea7a2c P b80b3a62d7d549a9a5c42def46769ac4: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:25.982908 19995 tablet_bootstrap.cc:492] T 19e83b82855d4129bb8525e6ff05316b P c62716bf66c24330860573535b8b9430: No bootstrap required, opened a new log
I20260709 13:20:25.991926 20037 raft_consensus.cc:493] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:25.992551 20037 raft_consensus.cc:515] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:25.997182 19723 raft_consensus.cc:3060] T 2eef64a7d31b4e1e899aee0857cb0e07 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:25.997926 19800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9f7b34cb064bbe8a6938e4f92d57c5" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:26.002002 19994 tablet_bootstrap.cc:492] T 984fa8de2ccb49359c2b475e4adfc1ce P 8df38c4c53c44e94b36c1e8992476d7c: No bootstrap required, opened a new log
I20260709 13:20:26.002597 19994 ts_tablet_manager.cc:1403] T 984fa8de2ccb49359c2b475e4adfc1ce P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.088s	user 0.009s	sys 0.002s
I20260709 13:20:26.004802 20037 leader_election.cc:290] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:26.006031 19994 raft_consensus.cc:359] T 984fa8de2ccb49359c2b475e4adfc1ce P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:26.006932 19994 raft_consensus.cc:385] T 984fa8de2ccb49359c2b475e4adfc1ce P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:26.007272 19994 raft_consensus.cc:740] T 984fa8de2ccb49359c2b475e4adfc1ce P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:26.007483 19945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9f7b34cb064bbe8a6938e4f92d57c5" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:26.008535 19945 raft_consensus.cc:2393] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b80b3a62d7d549a9a5c42def46769ac4 in current term 1: Already voted for candidate e90ffed3ba174ababeb21d9415d4d918 in this term.
I20260709 13:20:26.008177 19994 consensus_queue.cc:260] T 984fa8de2ccb49359c2b475e4adfc1ce P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:26.010370 19839 leader_election.cc:304] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4; no voters: c62716bf66c24330860573535b8b9430, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:26.010684 19994 ts_tablet_manager.cc:1434] T 984fa8de2ccb49359c2b475e4adfc1ce P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:20:26.011837 19994 tablet_bootstrap.cc:492] T 5b9511148896441a948b772784f62a3d P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:26.011765 20037 raft_consensus.cc:3060] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:26.018533 19994 tablet_bootstrap.cc:654] T 5b9511148896441a948b772784f62a3d P 8df38c4c53c44e94b36c1e8992476d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:26.020416 19995 ts_tablet_manager.cc:1403] T 19e83b82855d4129bb8525e6ff05316b P c62716bf66c24330860573535b8b9430: Time spent bootstrapping tablet: real 0.138s	user 0.008s	sys 0.005s
I20260709 13:20:26.024581 20037 raft_consensus.cc:2749] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:26.024926 19623 catalog_manager.cc:5697] T ac3bca5bb7644a8fbc78df43ef3331a0 P c62716bf66c24330860573535b8b9430 reported cstate change: term changed from 0 to 1, leader changed from <none> to c62716bf66c24330860573535b8b9430 (127.9.116.132). New cstate: current_term: 1 leader_uuid: "c62716bf66c24330860573535b8b9430" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:26.031174 20065 consensus_queue.cc:1048] T 2eef64a7d31b4e1e899aee0857cb0e07 P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:20:26.038897 20065 consensus_queue.cc:1048] T 2eef64a7d31b4e1e899aee0857cb0e07 P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:26.042521 19994 tablet_bootstrap.cc:492] T 5b9511148896441a948b772784f62a3d P 8df38c4c53c44e94b36c1e8992476d7c: No bootstrap required, opened a new log
I20260709 13:20:26.031509 19995 raft_consensus.cc:359] T 19e83b82855d4129bb8525e6ff05316b P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:26.044533 19994 ts_tablet_manager.cc:1403] T 5b9511148896441a948b772784f62a3d P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.033s	user 0.026s	sys 0.000s
I20260709 13:20:26.044570 19995 raft_consensus.cc:385] T 19e83b82855d4129bb8525e6ff05316b P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:26.045604 19995 raft_consensus.cc:740] T 19e83b82855d4129bb8525e6ff05316b P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62716bf66c24330860573535b8b9430, State: Initialized, Role: FOLLOWER
I20260709 13:20:26.044286 20037 raft_consensus.cc:493] T 6a066b96902f41dbb1511eccf26820ce P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:26.052338 19995 consensus_queue.cc:260] T 19e83b82855d4129bb8525e6ff05316b P c62716bf66c24330860573535b8b9430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:26.063719 19623 catalog_manager.cc:5697] T 19a59035bda747fdbc59b7e6914101dd P b80b3a62d7d549a9a5c42def46769ac4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131). New cstate: current_term: 1 leader_uuid: "b80b3a62d7d549a9a5c42def46769ac4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:26.070237 19997 tablet_bootstrap.cc:492] T e804b7d8fd7e44f59516fb299bea7a2c P b80b3a62d7d549a9a5c42def46769ac4: No bootstrap required, opened a new log
I20260709 13:20:26.070863 19997 ts_tablet_manager.cc:1403] T e804b7d8fd7e44f59516fb299bea7a2c P b80b3a62d7d549a9a5c42def46769ac4: Time spent bootstrapping tablet: real 0.104s	user 0.021s	sys 0.014s
I20260709 13:20:26.072662 20031 consensus_queue.cc:1048] T 2a411802fdaa42e7863236f0af595fe2 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:26.077729 19997 raft_consensus.cc:359] T e804b7d8fd7e44f59516fb299bea7a2c P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:26.078451 19997 raft_consensus.cc:385] T e804b7d8fd7e44f59516fb299bea7a2c P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:26.078724 19997 raft_consensus.cc:740] T e804b7d8fd7e44f59516fb299bea7a2c P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Initialized, Role: FOLLOWER
I20260709 13:20:26.079454 19997 consensus_queue.cc:260] T e804b7d8fd7e44f59516fb299bea7a2c P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:26.082316 19995 ts_tablet_manager.cc:1434] T 19e83b82855d4129bb8525e6ff05316b P c62716bf66c24330860573535b8b9430: Time spent starting tablet: real 0.061s	user 0.009s	sys 0.000s
I20260709 13:20:26.083407 19995 tablet_bootstrap.cc:492] T 353a546cf8c54671971c239502ef165a P c62716bf66c24330860573535b8b9430: Bootstrap starting.
I20260709 13:20:26.081472 20009 consensus_queue.cc:1048] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:20:26.089470 19997 ts_tablet_manager.cc:1434] T e804b7d8fd7e44f59516fb299bea7a2c P b80b3a62d7d549a9a5c42def46769ac4: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:20:26.090723 19997 tablet_bootstrap.cc:492] T 86964b8a61ce494f9096c4fefdc6efbf P b80b3a62d7d549a9a5c42def46769ac4: Bootstrap starting.
I20260709 13:20:26.053303 20037 raft_consensus.cc:515] T 6a066b96902f41dbb1511eccf26820ce P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:26.101488 20065 raft_consensus.cc:493] T 984fa8de2ccb49359c2b475e4adfc1ce P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:26.106410 20065 raft_consensus.cc:515] T 984fa8de2ccb49359c2b475e4adfc1ce P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:26.108585 19800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a066b96902f41dbb1511eccf26820ce" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:26.109740 19872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984fa8de2ccb49359c2b475e4adfc1ce" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:26.109787 19722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984fa8de2ccb49359c2b475e4adfc1ce" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:26.110857 19722 raft_consensus.cc:2468] T 984fa8de2ccb49359c2b475e4adfc1ce P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 0.
W20260709 13:20:26.111493 19765 leader_election.cc:343] T 984fa8de2ccb49359c2b475e4adfc1ce P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:26.112529 19995 tablet_bootstrap.cc:654] T 353a546cf8c54671971c239502ef165a P c62716bf66c24330860573535b8b9430: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:26.112795 19766 leader_election.cc:304] T 984fa8de2ccb49359c2b475e4adfc1ce P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918; no voters: b80b3a62d7d549a9a5c42def46769ac4
I20260709 13:20:26.114216 20040 raft_consensus.cc:2804] T 984fa8de2ccb49359c2b475e4adfc1ce P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:26.114957 20040 raft_consensus.cc:493] T 984fa8de2ccb49359c2b475e4adfc1ce P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:26.115432 20040 raft_consensus.cc:3060] T 984fa8de2ccb49359c2b475e4adfc1ce P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:26.120800 19722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a066b96902f41dbb1511eccf26820ce" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:26.122035 19722 raft_consensus.cc:2393] T 6a066b96902f41dbb1511eccf26820ce P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b80b3a62d7d549a9a5c42def46769ac4 in current term 1: Already voted for candidate e90ffed3ba174ababeb21d9415d4d918 in this term.
I20260709 13:20:26.123526 19840 leader_election.cc:304] T 6a066b96902f41dbb1511eccf26820ce P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4; no voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:26.108662 20065 leader_election.cc:290] T 984fa8de2ccb49359c2b475e4adfc1ce P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:26.124436 20009 raft_consensus.cc:3060] T 6a066b96902f41dbb1511eccf26820ce P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:26.105195 19997 tablet_bootstrap.cc:654] T 86964b8a61ce494f9096c4fefdc6efbf P b80b3a62d7d549a9a5c42def46769ac4: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:26.114293 19994 raft_consensus.cc:359] T 5b9511148896441a948b772784f62a3d P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:26.128295 19994 raft_consensus.cc:385] T 5b9511148896441a948b772784f62a3d P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:26.128732 19994 raft_consensus.cc:740] T 5b9511148896441a948b772784f62a3d P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:26.110859 20037 leader_election.cc:290] T 6a066b96902f41dbb1511eccf26820ce P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:26.134867 19994 consensus_queue.cc:260] T 5b9511148896441a948b772784f62a3d P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:26.141762 19993 tablet_bootstrap.cc:492] T ad38a1519fb748588f5a322589274004 P e90ffed3ba174ababeb21d9415d4d918: No bootstrap required, opened a new log
I20260709 13:20:26.142356 19993 ts_tablet_manager.cc:1403] T ad38a1519fb748588f5a322589274004 P e90ffed3ba174ababeb21d9415d4d918: Time spent bootstrapping tablet: real 0.210s	user 0.007s	sys 0.014s
I20260709 13:20:26.150978 19994 ts_tablet_manager.cc:1434] T 5b9511148896441a948b772784f62a3d P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.106s	user 0.012s	sys 0.000s
I20260709 13:20:26.146489 20040 raft_consensus.cc:515] T 984fa8de2ccb49359c2b475e4adfc1ce P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:26.160972 20031 consensus_queue.cc:1048] T 2a411802fdaa42e7863236f0af595fe2 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.022s
I20260709 13:20:26.161098 20080 consensus_queue.cc:1048] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20260709 13:20:26.171322 19722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984fa8de2ccb49359c2b475e4adfc1ce" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
I20260709 13:20:26.187418 19722 raft_consensus.cc:3060] T 984fa8de2ccb49359c2b475e4adfc1ce P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:26.173007 20009 raft_consensus.cc:2749] T 6a066b96902f41dbb1511eccf26820ce P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:26.174098 19872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984fa8de2ccb49359c2b475e4adfc1ce" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
I20260709 13:20:26.175997 20083 raft_consensus.cc:493] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:26.191236 20083 raft_consensus.cc:515] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:26.173280 20040 leader_election.cc:290] T 984fa8de2ccb49359c2b475e4adfc1ce P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 election: Requested vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:26.193490 20105 raft_consensus.cc:493] T 9b9137146e3644a6b0026778530fa8a6 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:26.194041 20105 raft_consensus.cc:515] T 9b9137146e3644a6b0026778530fa8a6 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:26.195936 19946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e804b7d8fd7e44f59516fb299bea7a2c" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:26.196287 20105 leader_election.cc:290] T 9b9137146e3644a6b0026778530fa8a6 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:26.197311 19872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e804b7d8fd7e44f59516fb299bea7a2c" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
W20260709 13:20:26.197824 19690 leader_election.cc:343] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c62716bf66c24330860573535b8b9430 (127.9.116.132:42643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:26.198257 19872 raft_consensus.cc:2468] T e804b7d8fd7e44f59516fb299bea7a2c P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 0.
I20260709 13:20:26.199684 19724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9137146e3644a6b0026778530fa8a6" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:26.200657 19800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9137146e3644a6b0026778530fa8a6" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
W20260709 13:20:26.201996 19914 leader_election.cc:343] T 9b9137146e3644a6b0026778530fa8a6 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:26.202248 19690 leader_election.cc:304] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4; no voters: c62716bf66c24330860573535b8b9430
W20260709 13:20:26.199888 19765 leader_election.cc:343] T 984fa8de2ccb49359c2b475e4adfc1ce P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:26.203506 20031 raft_consensus.cc:2804] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:26.203882 20031 raft_consensus.cc:493] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:26.204213 20031 raft_consensus.cc:3060] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:26.204706 20064 consensus_queue.cc:1048] T d762eace063c4e04821c749d21702c3f P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:20:26.205214 20065 consensus_queue.cc:1048] T 6a066b96902f41dbb1511eccf26820ce P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:20:26.208149 19913 leader_election.cc:343] T 9b9137146e3644a6b0026778530fa8a6 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:26.209386 19913 leader_election.cc:304] T 9b9137146e3644a6b0026778530fa8a6 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:26.214869 20105 raft_consensus.cc:2749] T 9b9137146e3644a6b0026778530fa8a6 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:26.226634 20064 consensus_queue.cc:1048] T 6a066b96902f41dbb1511eccf26820ce P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:20:26.159487 19993 raft_consensus.cc:359] T ad38a1519fb748588f5a322589274004 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:26.272447 19993 raft_consensus.cc:385] T ad38a1519fb748588f5a322589274004 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:26.272706 19993 raft_consensus.cc:740] T ad38a1519fb748588f5a322589274004 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Initialized, Role: FOLLOWER
I20260709 13:20:26.273360 19993 consensus_queue.cc:260] T ad38a1519fb748588f5a322589274004 P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:26.153885 19994 tablet_bootstrap.cc:492] T 86964b8a61ce494f9096c4fefdc6efbf P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:26.275218 20064 consensus_queue.cc:1048] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:26.193529 20083 leader_election.cc:290] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:26.181825 19997 tablet_bootstrap.cc:492] T 86964b8a61ce494f9096c4fefdc6efbf P b80b3a62d7d549a9a5c42def46769ac4: No bootstrap required, opened a new log
W20260709 13:20:26.250947 19670 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:20:26.180022 19995 tablet_bootstrap.cc:492] T 353a546cf8c54671971c239502ef165a P c62716bf66c24330860573535b8b9430: No bootstrap required, opened a new log
I20260709 13:20:26.249985 20105 consensus_queue.cc:1048] T ac3bca5bb7644a8fbc78df43ef3331a0 P c62716bf66c24330860573535b8b9430 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.031s
I20260709 13:20:26.280378 19995 ts_tablet_manager.cc:1403] T 353a546cf8c54671971c239502ef165a P c62716bf66c24330860573535b8b9430: Time spent bootstrapping tablet: real 0.197s	user 0.013s	sys 0.000s
I20260709 13:20:26.237461 19722 raft_consensus.cc:2468] T 984fa8de2ccb49359c2b475e4adfc1ce P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 1.
I20260709 13:20:26.282531 19766 leader_election.cc:304] T 984fa8de2ccb49359c2b475e4adfc1ce P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918; no voters: b80b3a62d7d549a9a5c42def46769ac4
I20260709 13:20:26.283388 20065 raft_consensus.cc:2804] T 984fa8de2ccb49359c2b475e4adfc1ce P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:26.283939 20065 raft_consensus.cc:697] T 984fa8de2ccb49359c2b475e4adfc1ce P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Becoming Leader. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Running, Role: LEADER
I20260709 13:20:26.284222 19997 ts_tablet_manager.cc:1403] T 86964b8a61ce494f9096c4fefdc6efbf P b80b3a62d7d549a9a5c42def46769ac4: Time spent bootstrapping tablet: real 0.194s	user 0.016s	sys 0.008s
I20260709 13:20:26.284803 20065 consensus_queue.cc:237] T 984fa8de2ccb49359c2b475e4adfc1ce P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:26.285815 19994 tablet_bootstrap.cc:654] T 86964b8a61ce494f9096c4fefdc6efbf P 8df38c4c53c44e94b36c1e8992476d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:26.264683 20040 consensus_queue.cc:1048] T d762eace063c4e04821c749d21702c3f P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260709 13:20:26.287968 19997 raft_consensus.cc:359] T 86964b8a61ce494f9096c4fefdc6efbf P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:26.288314 19995 raft_consensus.cc:359] T 353a546cf8c54671971c239502ef165a P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:26.289009 19997 raft_consensus.cc:385] T 86964b8a61ce494f9096c4fefdc6efbf P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:26.289211 19995 raft_consensus.cc:385] T 353a546cf8c54671971c239502ef165a P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:26.289480 19997 raft_consensus.cc:740] T 86964b8a61ce494f9096c4fefdc6efbf P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Initialized, Role: FOLLOWER
I20260709 13:20:26.289659 19995 raft_consensus.cc:740] T 353a546cf8c54671971c239502ef165a P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62716bf66c24330860573535b8b9430, State: Initialized, Role: FOLLOWER
I20260709 13:20:26.290823 19995 consensus_queue.cc:260] T 353a546cf8c54671971c239502ef165a P c62716bf66c24330860573535b8b9430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:26.293082 19993 ts_tablet_manager.cc:1434] T ad38a1519fb748588f5a322589274004 P e90ffed3ba174ababeb21d9415d4d918: Time spent starting tablet: real 0.150s	user 0.006s	sys 0.003s
I20260709 13:20:26.294488 19993 tablet_bootstrap.cc:492] T 85d493223fe04c939b6effd840955727 P e90ffed3ba174ababeb21d9415d4d918: Bootstrap starting.
I20260709 13:20:26.301010 20105 consensus_queue.cc:1048] T ac3bca5bb7644a8fbc78df43ef3331a0 P c62716bf66c24330860573535b8b9430 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:26.302160 20108 raft_consensus.cc:493] T 19e83b82855d4129bb8525e6ff05316b P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:26.302266 20040 consensus_queue.cc:1048] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:20:26.302812 20108 raft_consensus.cc:515] T 19e83b82855d4129bb8525e6ff05316b P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:26.305181 20108 leader_election.cc:290] T 19e83b82855d4129bb8525e6ff05316b P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:26.307348 19873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19e83b82855d4129bb8525e6ff05316b" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
W20260709 13:20:26.309087 19913 leader_election.cc:343] T 19e83b82855d4129bb8525e6ff05316b P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:26.309937 20064 raft_consensus.cc:493] T ad38a1519fb748588f5a322589274004 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:26.310629 20064 raft_consensus.cc:515] T ad38a1519fb748588f5a322589274004 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:26.312582 19722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19e83b82855d4129bb8525e6ff05316b" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
W20260709 13:20:26.314042 19914 leader_election.cc:343] T 19e83b82855d4129bb8525e6ff05316b P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:26.314505 19914 leader_election.cc:304] T 19e83b82855d4129bb8525e6ff05316b P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4
I20260709 13:20:26.314553 19723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad38a1519fb748588f5a322589274004" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:26.315832 20000 raft_consensus.cc:2749] T 19e83b82855d4129bb8525e6ff05316b P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:20:26.316687 19766 leader_election.cc:343] T ad38a1519fb748588f5a322589274004 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:26.324339 19995 ts_tablet_manager.cc:1434] T 353a546cf8c54671971c239502ef165a P c62716bf66c24330860573535b8b9430: Time spent starting tablet: real 0.044s	user 0.007s	sys 0.000s
I20260709 13:20:26.325330 19995 tablet_bootstrap.cc:492] T 86964b8a61ce494f9096c4fefdc6efbf P c62716bf66c24330860573535b8b9430: Bootstrap starting.
I20260709 13:20:26.327528 19997 consensus_queue.cc:260] T 86964b8a61ce494f9096c4fefdc6efbf P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:26.226773 20031 raft_consensus.cc:515] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:26.333839 20031 leader_election.cc:290] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Requested vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:26.333762 19874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e804b7d8fd7e44f59516fb299bea7a2c" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
I20260709 13:20:26.334582 19874 raft_consensus.cc:3060] T e804b7d8fd7e44f59516fb299bea7a2c P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:26.339877 19946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e804b7d8fd7e44f59516fb299bea7a2c" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430"
W20260709 13:20:26.341219 19690 leader_election.cc:343] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c62716bf66c24330860573535b8b9430 (127.9.116.132:42643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:26.343122 20080 consensus_queue.cc:1048] T 19a59035bda747fdbc59b7e6914101dd P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:20:26.346124 20000 consensus_queue.cc:1048] T 3cad7a5f46f9407a99ea6f2cc5a8322c P c62716bf66c24330860573535b8b9430 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:26.347823 19993 tablet_bootstrap.cc:654] T 85d493223fe04c939b6effd840955727 P e90ffed3ba174ababeb21d9415d4d918: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:26.350081 19873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad38a1519fb748588f5a322589274004" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
W20260709 13:20:26.352608 19765 leader_election.cc:343] T ad38a1519fb748588f5a322589274004 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:26.353264 19765 leader_election.cc:304] T ad38a1519fb748588f5a322589274004 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e90ffed3ba174ababeb21d9415d4d918; no voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4
I20260709 13:20:26.354226 20040 raft_consensus.cc:2749] T ad38a1519fb748588f5a322589274004 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:26.355154 20064 leader_election.cc:290] T ad38a1519fb748588f5a322589274004 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:26.346449 19997 ts_tablet_manager.cc:1434] T 86964b8a61ce494f9096c4fefdc6efbf P b80b3a62d7d549a9a5c42def46769ac4: Time spent starting tablet: real 0.062s	user 0.007s	sys 0.001s
I20260709 13:20:26.357200 19997 tablet_bootstrap.cc:492] T 984fa8de2ccb49359c2b475e4adfc1ce P b80b3a62d7d549a9a5c42def46769ac4: Bootstrap starting.
I20260709 13:20:26.360406 19994 tablet_bootstrap.cc:492] T 86964b8a61ce494f9096c4fefdc6efbf P 8df38c4c53c44e94b36c1e8992476d7c: No bootstrap required, opened a new log
I20260709 13:20:26.360980 19994 ts_tablet_manager.cc:1403] T 86964b8a61ce494f9096c4fefdc6efbf P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.208s	user 0.008s	sys 0.002s
I20260709 13:20:26.360163 20105 consensus_queue.cc:1048] T 3cad7a5f46f9407a99ea6f2cc5a8322c P c62716bf66c24330860573535b8b9430 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:20:26.363802 19994 raft_consensus.cc:359] T 86964b8a61ce494f9096c4fefdc6efbf P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:26.364557 19994 raft_consensus.cc:385] T 86964b8a61ce494f9096c4fefdc6efbf P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:26.364816 19994 raft_consensus.cc:740] T 86964b8a61ce494f9096c4fefdc6efbf P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:26.371222 19994 consensus_queue.cc:260] T 86964b8a61ce494f9096c4fefdc6efbf P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:26.372749 19995 tablet_bootstrap.cc:654] T 86964b8a61ce494f9096c4fefdc6efbf P c62716bf66c24330860573535b8b9430: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:26.375542 19997 tablet_bootstrap.cc:654] T 984fa8de2ccb49359c2b475e4adfc1ce P b80b3a62d7d549a9a5c42def46769ac4: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:26.376662 19994 ts_tablet_manager.cc:1434] T 86964b8a61ce494f9096c4fefdc6efbf P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
I20260709 13:20:26.377682 19994 tablet_bootstrap.cc:492] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:26.377532 19622 catalog_manager.cc:5697] T 984fa8de2ccb49359c2b475e4adfc1ce P e90ffed3ba174ababeb21d9415d4d918 reported cstate change: term changed from 0 to 1, leader changed from <none> to e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "e90ffed3ba174ababeb21d9415d4d918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:26.388996 20080 consensus_queue.cc:1048] T 19a59035bda747fdbc59b7e6914101dd P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:20:26.404862 19995 tablet_bootstrap.cc:492] T 86964b8a61ce494f9096c4fefdc6efbf P c62716bf66c24330860573535b8b9430: No bootstrap required, opened a new log
I20260709 13:20:26.405555 19995 ts_tablet_manager.cc:1403] T 86964b8a61ce494f9096c4fefdc6efbf P c62716bf66c24330860573535b8b9430: Time spent bootstrapping tablet: real 0.080s	user 0.011s	sys 0.008s
I20260709 13:20:26.407233 19874 raft_consensus.cc:2468] T e804b7d8fd7e44f59516fb299bea7a2c P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 1.
I20260709 13:20:26.424110 19993 tablet_bootstrap.cc:492] T 85d493223fe04c939b6effd840955727 P e90ffed3ba174ababeb21d9415d4d918: No bootstrap required, opened a new log
I20260709 13:20:26.424726 19993 ts_tablet_manager.cc:1403] T 85d493223fe04c939b6effd840955727 P e90ffed3ba174ababeb21d9415d4d918: Time spent bootstrapping tablet: real 0.131s	user 0.015s	sys 0.012s
I20260709 13:20:26.425537 19690 leader_election.cc:304] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4; no voters: c62716bf66c24330860573535b8b9430
I20260709 13:20:26.426978 20083 raft_consensus.cc:2804] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:26.427456 20083 raft_consensus.cc:697] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c [term 1 LEADER]: Becoming Leader. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Running, Role: LEADER
I20260709 13:20:26.427577 19993 raft_consensus.cc:359] T 85d493223fe04c939b6effd840955727 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:26.428324 19993 raft_consensus.cc:385] T 85d493223fe04c939b6effd840955727 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:26.428730 19993 raft_consensus.cc:740] T 85d493223fe04c939b6effd840955727 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Initialized, Role: FOLLOWER
I20260709 13:20:26.428354 20083 consensus_queue.cc:237] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:26.429445 19993 consensus_queue.cc:260] T 85d493223fe04c939b6effd840955727 P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:26.431272 20132 raft_consensus.cc:493] T 86964b8a61ce494f9096c4fefdc6efbf P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:26.431816 20132 raft_consensus.cc:515] T 86964b8a61ce494f9096c4fefdc6efbf P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:26.434041 20132 leader_election.cc:290] T 86964b8a61ce494f9096c4fefdc6efbf P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:26.437772 19871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86964b8a61ce494f9096c4fefdc6efbf" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:26.438303 19993 ts_tablet_manager.cc:1434] T 85d493223fe04c939b6effd840955727 P e90ffed3ba174ababeb21d9415d4d918: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:20:26.439476 19993 tablet_bootstrap.cc:492] T 65e7a5222efe43e49e1fbb3a3d7cf99c P e90ffed3ba174ababeb21d9415d4d918: Bootstrap starting.
I20260709 13:20:26.439692 19871 raft_consensus.cc:2468] T 86964b8a61ce494f9096c4fefdc6efbf P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 0.
I20260709 13:20:26.441987 19945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86964b8a61ce494f9096c4fefdc6efbf" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:26.444350 19690 leader_election.cc:304] T 86964b8a61ce494f9096c4fefdc6efbf P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4; no voters: 
I20260709 13:20:26.445391 20031 raft_consensus.cc:2804] T 86964b8a61ce494f9096c4fefdc6efbf P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:26.445858 20031 raft_consensus.cc:493] T 86964b8a61ce494f9096c4fefdc6efbf P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:26.446259 20031 raft_consensus.cc:3060] T 86964b8a61ce494f9096c4fefdc6efbf P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:26.446833 19997 tablet_bootstrap.cc:492] T 984fa8de2ccb49359c2b475e4adfc1ce P b80b3a62d7d549a9a5c42def46769ac4: No bootstrap required, opened a new log
I20260709 13:20:26.447346 19997 ts_tablet_manager.cc:1403] T 984fa8de2ccb49359c2b475e4adfc1ce P b80b3a62d7d549a9a5c42def46769ac4: Time spent bootstrapping tablet: real 0.090s	user 0.017s	sys 0.004s
I20260709 13:20:26.450388 19997 raft_consensus.cc:359] T 984fa8de2ccb49359c2b475e4adfc1ce P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:26.451107 19997 raft_consensus.cc:385] T 984fa8de2ccb49359c2b475e4adfc1ce P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:26.451382 19997 raft_consensus.cc:740] T 984fa8de2ccb49359c2b475e4adfc1ce P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Initialized, Role: FOLLOWER
I20260709 13:20:26.452039 19997 consensus_queue.cc:260] T 984fa8de2ccb49359c2b475e4adfc1ce P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:26.454116 19997 ts_tablet_manager.cc:1434] T 984fa8de2ccb49359c2b475e4adfc1ce P b80b3a62d7d549a9a5c42def46769ac4: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:20:26.455031 19997 tablet_bootstrap.cc:492] T 65e7a5222efe43e49e1fbb3a3d7cf99c P b80b3a62d7d549a9a5c42def46769ac4: Bootstrap starting.
I20260709 13:20:26.424295 19995 raft_consensus.cc:359] T 86964b8a61ce494f9096c4fefdc6efbf P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:26.460887 19995 raft_consensus.cc:385] T 86964b8a61ce494f9096c4fefdc6efbf P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:26.461103 19997 tablet_bootstrap.cc:654] T 65e7a5222efe43e49e1fbb3a3d7cf99c P b80b3a62d7d549a9a5c42def46769ac4: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:26.461146 19995 raft_consensus.cc:740] T 86964b8a61ce494f9096c4fefdc6efbf P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62716bf66c24330860573535b8b9430, State: Initialized, Role: FOLLOWER
I20260709 13:20:26.462736 19995 consensus_queue.cc:260] T 86964b8a61ce494f9096c4fefdc6efbf P c62716bf66c24330860573535b8b9430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:26.463712 19945 raft_consensus.cc:2468] T 86964b8a61ce494f9096c4fefdc6efbf P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 0.
I20260709 13:20:26.464982 19994 tablet_bootstrap.cc:654] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:26.466022 19995 ts_tablet_manager.cc:1434] T 86964b8a61ce494f9096c4fefdc6efbf P c62716bf66c24330860573535b8b9430: Time spent starting tablet: real 0.060s	user 0.007s	sys 0.000s
I20260709 13:20:26.468364 19995 tablet_bootstrap.cc:492] T 0587017774214f24bdfa4a3da36e367e P c62716bf66c24330860573535b8b9430: Bootstrap starting.
I20260709 13:20:26.482395 20132 raft_consensus.cc:493] T 5b9511148896441a948b772784f62a3d P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:26.483767 20132 raft_consensus.cc:515] T 5b9511148896441a948b772784f62a3d P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:26.504370 19800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9511148896441a948b772784f62a3d" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
W20260709 13:20:26.506896 19690 leader_election.cc:343] T 5b9511148896441a948b772784f62a3d P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:26.486732 20132 leader_election.cc:290] T 5b9511148896441a948b772784f62a3d P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:26.511090 20031 raft_consensus.cc:515] T 86964b8a61ce494f9096c4fefdc6efbf P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:26.516813 20031 leader_election.cc:290] T 86964b8a61ce494f9096c4fefdc6efbf P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Requested vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:26.517865 19945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86964b8a61ce494f9096c4fefdc6efbf" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:26.518518 19945 raft_consensus.cc:3060] T 86964b8a61ce494f9096c4fefdc6efbf P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:26.519758 19873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86964b8a61ce494f9096c4fefdc6efbf" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
I20260709 13:20:26.520395 19873 raft_consensus.cc:3060] T 86964b8a61ce494f9096c4fefdc6efbf P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:26.521317 19619 catalog_manager.cc:5697] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c reported cstate change: term changed from 0 to 1, leader changed from <none> to 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129). New cstate: current_term: 1 leader_uuid: "8df38c4c53c44e94b36c1e8992476d7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:26.522454 19995 tablet_bootstrap.cc:654] T 0587017774214f24bdfa4a3da36e367e P c62716bf66c24330860573535b8b9430: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:26.522740 19994 tablet_bootstrap.cc:492] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c: No bootstrap required, opened a new log
I20260709 13:20:26.523280 19994 ts_tablet_manager.cc:1403] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.146s	user 0.021s	sys 0.018s
I20260709 13:20:26.523161 19874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9511148896441a948b772784f62a3d" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
W20260709 13:20:26.524609 19690 leader_election.cc:343] T 5b9511148896441a948b772784f62a3d P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:26.525048 19690 leader_election.cc:304] T 5b9511148896441a948b772784f62a3d P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c; no voters: b80b3a62d7d549a9a5c42def46769ac4, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:26.525311 19993 tablet_bootstrap.cc:654] T 65e7a5222efe43e49e1fbb3a3d7cf99c P e90ffed3ba174ababeb21d9415d4d918: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:26.526245 19994 raft_consensus.cc:359] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:26.526970 19994 raft_consensus.cc:385] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:26.527251 19994 raft_consensus.cc:740] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:26.527974 19994 consensus_queue.cc:260] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:26.528944 20083 raft_consensus.cc:2749] T 5b9511148896441a948b772784f62a3d P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:26.531138 19994 ts_tablet_manager.cc:1434] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:20:26.532210 19994 tablet_bootstrap.cc:492] T 353a546cf8c54671971c239502ef165a P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:26.536775 19945 raft_consensus.cc:2468] T 86964b8a61ce494f9096c4fefdc6efbf P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 1.
I20260709 13:20:26.538167 19994 tablet_bootstrap.cc:654] T 353a546cf8c54671971c239502ef165a P 8df38c4c53c44e94b36c1e8992476d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:26.539743 19873 raft_consensus.cc:2468] T 86964b8a61ce494f9096c4fefdc6efbf P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 1.
I20260709 13:20:26.541610 19690 leader_election.cc:304] T 86964b8a61ce494f9096c4fefdc6efbf P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4; no voters: 
I20260709 13:20:26.542560 20031 raft_consensus.cc:2804] T 86964b8a61ce494f9096c4fefdc6efbf P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:26.543150 19997 tablet_bootstrap.cc:492] T 65e7a5222efe43e49e1fbb3a3d7cf99c P b80b3a62d7d549a9a5c42def46769ac4: No bootstrap required, opened a new log
I20260709 13:20:26.543668 19997 ts_tablet_manager.cc:1403] T 65e7a5222efe43e49e1fbb3a3d7cf99c P b80b3a62d7d549a9a5c42def46769ac4: Time spent bootstrapping tablet: real 0.089s	user 0.034s	sys 0.021s
I20260709 13:20:26.543334 20031 raft_consensus.cc:697] T 86964b8a61ce494f9096c4fefdc6efbf P 8df38c4c53c44e94b36c1e8992476d7c [term 1 LEADER]: Becoming Leader. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Running, Role: LEADER
I20260709 13:20:26.547070 19997 raft_consensus.cc:359] T 65e7a5222efe43e49e1fbb3a3d7cf99c P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:26.547247 20031 consensus_queue.cc:237] T 86964b8a61ce494f9096c4fefdc6efbf P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:26.548111 19997 raft_consensus.cc:385] T 65e7a5222efe43e49e1fbb3a3d7cf99c P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:26.548796 19997 raft_consensus.cc:740] T 65e7a5222efe43e49e1fbb3a3d7cf99c P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Initialized, Role: FOLLOWER
I20260709 13:20:26.549777 19997 consensus_queue.cc:260] T 65e7a5222efe43e49e1fbb3a3d7cf99c P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:26.555097 20064 raft_consensus.cc:493] T 85d493223fe04c939b6effd840955727 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:26.555683 20064 raft_consensus.cc:515] T 85d493223fe04c939b6effd840955727 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:26.557449 19995 tablet_bootstrap.cc:492] T 0587017774214f24bdfa4a3da36e367e P c62716bf66c24330860573535b8b9430: No bootstrap required, opened a new log
I20260709 13:20:26.557715 20064 leader_election.cc:290] T 85d493223fe04c939b6effd840955727 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:26.558161 19995 ts_tablet_manager.cc:1403] T 0587017774214f24bdfa4a3da36e367e P c62716bf66c24330860573535b8b9430: Time spent bootstrapping tablet: real 0.091s	user 0.014s	sys 0.003s
I20260709 13:20:26.559674 19945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d493223fe04c939b6effd840955727" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:26.559674 19873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d493223fe04c939b6effd840955727" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:26.560855 19873 raft_consensus.cc:2468] T 85d493223fe04c939b6effd840955727 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 0.
W20260709 13:20:26.561460 19765 leader_election.cc:343] T 85d493223fe04c939b6effd840955727 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c62716bf66c24330860573535b8b9430 (127.9.116.132:42643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:26.561661 19993 tablet_bootstrap.cc:492] T 65e7a5222efe43e49e1fbb3a3d7cf99c P e90ffed3ba174ababeb21d9415d4d918: No bootstrap required, opened a new log
I20260709 13:20:26.562366 19993 ts_tablet_manager.cc:1403] T 65e7a5222efe43e49e1fbb3a3d7cf99c P e90ffed3ba174ababeb21d9415d4d918: Time spent bootstrapping tablet: real 0.123s	user 0.013s	sys 0.004s
I20260709 13:20:26.562275 19765 leader_election.cc:304] T 85d493223fe04c939b6effd840955727 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, e90ffed3ba174ababeb21d9415d4d918; no voters: c62716bf66c24330860573535b8b9430
I20260709 13:20:26.563953 20040 raft_consensus.cc:2804] T 85d493223fe04c939b6effd840955727 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:26.564401 20040 raft_consensus.cc:493] T 85d493223fe04c939b6effd840955727 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:26.564757 20040 raft_consensus.cc:3060] T 85d493223fe04c939b6effd840955727 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:26.567606 19993 raft_consensus.cc:359] T 65e7a5222efe43e49e1fbb3a3d7cf99c P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:26.568588 19993 raft_consensus.cc:385] T 65e7a5222efe43e49e1fbb3a3d7cf99c P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:26.568885 19993 raft_consensus.cc:740] T 65e7a5222efe43e49e1fbb3a3d7cf99c P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Initialized, Role: FOLLOWER
I20260709 13:20:26.573100 20040 raft_consensus.cc:515] T 85d493223fe04c939b6effd840955727 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:26.574527 19995 raft_consensus.cc:359] T 0587017774214f24bdfa4a3da36e367e P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:26.563934 19997 ts_tablet_manager.cc:1434] T 65e7a5222efe43e49e1fbb3a3d7cf99c P b80b3a62d7d549a9a5c42def46769ac4: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 13:20:26.575290 19995 raft_consensus.cc:385] T 0587017774214f24bdfa4a3da36e367e P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:26.575623 19995 raft_consensus.cc:740] T 0587017774214f24bdfa4a3da36e367e P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62716bf66c24330860573535b8b9430, State: Initialized, Role: FOLLOWER
I20260709 13:20:26.575941 19621 catalog_manager.cc:5697] T 86964b8a61ce494f9096c4fefdc6efbf P 8df38c4c53c44e94b36c1e8992476d7c reported cstate change: term changed from 0 to 1, leader changed from <none> to 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129). New cstate: current_term: 1 leader_uuid: "8df38c4c53c44e94b36c1e8992476d7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:26.578826 19994 tablet_bootstrap.cc:492] T 353a546cf8c54671971c239502ef165a P 8df38c4c53c44e94b36c1e8992476d7c: No bootstrap required, opened a new log
I20260709 13:20:26.577595 19993 consensus_queue.cc:260] T 65e7a5222efe43e49e1fbb3a3d7cf99c P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:26.579309 19994 ts_tablet_manager.cc:1403] T 353a546cf8c54671971c239502ef165a P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.047s	user 0.014s	sys 0.009s
I20260709 13:20:26.582160 19994 raft_consensus.cc:359] T 353a546cf8c54671971c239502ef165a P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:26.582814 19994 raft_consensus.cc:385] T 353a546cf8c54671971c239502ef165a P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:26.583101 19994 raft_consensus.cc:740] T 353a546cf8c54671971c239502ef165a P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:26.578519 19995 consensus_queue.cc:260] T 0587017774214f24bdfa4a3da36e367e P c62716bf66c24330860573535b8b9430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:26.583858 19994 consensus_queue.cc:260] T 353a546cf8c54671971c239502ef165a P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:26.584440 19997 tablet_bootstrap.cc:492] T e9369d9b7f5649be932967d24689a936 P b80b3a62d7d549a9a5c42def46769ac4: Bootstrap starting.
I20260709 13:20:26.585001 19993 ts_tablet_manager.cc:1434] T 65e7a5222efe43e49e1fbb3a3d7cf99c P e90ffed3ba174ababeb21d9415d4d918: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.001s
I20260709 13:20:26.585960 19993 tablet_bootstrap.cc:492] T d1da95785dbf4ef4bde6f5e6ab35b444 P e90ffed3ba174ababeb21d9415d4d918: Bootstrap starting.
I20260709 13:20:26.586000 19994 ts_tablet_manager.cc:1434] T 353a546cf8c54671971c239502ef165a P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:20:26.587100 19994 tablet_bootstrap.cc:492] T a323eb80c5084159803e7d4ea5d9c410 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:26.588174 19873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d493223fe04c939b6effd840955727" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
I20260709 13:20:26.588790 19873 raft_consensus.cc:3060] T 85d493223fe04c939b6effd840955727 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:26.591490 19995 ts_tablet_manager.cc:1434] T 0587017774214f24bdfa4a3da36e367e P c62716bf66c24330860573535b8b9430: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.000s
I20260709 13:20:26.592537 19995 tablet_bootstrap.cc:492] T e804b7d8fd7e44f59516fb299bea7a2c P c62716bf66c24330860573535b8b9430: Bootstrap starting.
I20260709 13:20:26.593616 19994 tablet_bootstrap.cc:654] T a323eb80c5084159803e7d4ea5d9c410 P 8df38c4c53c44e94b36c1e8992476d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:26.607249 19993 tablet_bootstrap.cc:654] T d1da95785dbf4ef4bde6f5e6ab35b444 P e90ffed3ba174ababeb21d9415d4d918: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:26.611353 19873 raft_consensus.cc:2468] T 85d493223fe04c939b6effd840955727 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 1.
I20260709 13:20:26.614202 19945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d493223fe04c939b6effd840955727" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:26.617537 19995 tablet_bootstrap.cc:654] T e804b7d8fd7e44f59516fb299bea7a2c P c62716bf66c24330860573535b8b9430: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:26.619429 19765 leader_election.cc:304] T 85d493223fe04c939b6effd840955727 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, e90ffed3ba174ababeb21d9415d4d918; no voters: 
W20260709 13:20:26.620503 19765 leader_election.cc:343] T 85d493223fe04c939b6effd840955727 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c62716bf66c24330860573535b8b9430 (127.9.116.132:42643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:26.621352 20065 raft_consensus.cc:2804] T 85d493223fe04c939b6effd840955727 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:26.621901 20065 raft_consensus.cc:697] T 85d493223fe04c939b6effd840955727 P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Becoming Leader. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Running, Role: LEADER
I20260709 13:20:26.623013 20065 consensus_queue.cc:237] T 85d493223fe04c939b6effd840955727 P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:26.628342 19993 tablet_bootstrap.cc:492] T d1da95785dbf4ef4bde6f5e6ab35b444 P e90ffed3ba174ababeb21d9415d4d918: No bootstrap required, opened a new log
I20260709 13:20:26.628752 19993 ts_tablet_manager.cc:1403] T d1da95785dbf4ef4bde6f5e6ab35b444 P e90ffed3ba174ababeb21d9415d4d918: Time spent bootstrapping tablet: real 0.043s	user 0.009s	sys 0.009s
I20260709 13:20:26.629794 20000 raft_consensus.cc:493] T 353a546cf8c54671971c239502ef165a P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:26.617846 20040 leader_election.cc:290] T 85d493223fe04c939b6effd840955727 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 election: Requested vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:26.630338 20000 raft_consensus.cc:515] T 353a546cf8c54671971c239502ef165a P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:26.631601 19993 raft_consensus.cc:359] T d1da95785dbf4ef4bde6f5e6ab35b444 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:26.632292 19993 raft_consensus.cc:385] T d1da95785dbf4ef4bde6f5e6ab35b444 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:26.632558 19993 raft_consensus.cc:740] T d1da95785dbf4ef4bde6f5e6ab35b444 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Initialized, Role: FOLLOWER
I20260709 13:20:26.632755 20000 leader_election.cc:290] T 353a546cf8c54671971c239502ef165a P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:26.633285 19993 consensus_queue.cc:260] T d1da95785dbf4ef4bde6f5e6ab35b444 P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:26.634725 19722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353a546cf8c54671971c239502ef165a" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:26.635195 19799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353a546cf8c54671971c239502ef165a" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:26.635907 19722 raft_consensus.cc:2468] T 353a546cf8c54671971c239502ef165a P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c62716bf66c24330860573535b8b9430 in term 0.
I20260709 13:20:26.635914 19799 raft_consensus.cc:2468] T 353a546cf8c54671971c239502ef165a P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c62716bf66c24330860573535b8b9430 in term 0.
I20260709 13:20:26.637274 19914 leader_election.cc:304] T 353a546cf8c54671971c239502ef165a P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, c62716bf66c24330860573535b8b9430; no voters: 
I20260709 13:20:26.639326 20000 raft_consensus.cc:2804] T 353a546cf8c54671971c239502ef165a P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:26.639863 20000 raft_consensus.cc:493] T 353a546cf8c54671971c239502ef165a P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:26.640302 20000 raft_consensus.cc:3060] T 353a546cf8c54671971c239502ef165a P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:26.645219 19993 ts_tablet_manager.cc:1434] T d1da95785dbf4ef4bde6f5e6ab35b444 P e90ffed3ba174ababeb21d9415d4d918: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.001s
I20260709 13:20:26.645444 19621 catalog_manager.cc:5697] T 85d493223fe04c939b6effd840955727 P e90ffed3ba174ababeb21d9415d4d918 reported cstate change: term changed from 0 to 1, leader changed from <none> to e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "e90ffed3ba174ababeb21d9415d4d918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:26.649503 20000 raft_consensus.cc:515] T 353a546cf8c54671971c239502ef165a P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:26.649861 19997 tablet_bootstrap.cc:654] T e9369d9b7f5649be932967d24689a936 P b80b3a62d7d549a9a5c42def46769ac4: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:26.654690 19995 tablet_bootstrap.cc:492] T e804b7d8fd7e44f59516fb299bea7a2c P c62716bf66c24330860573535b8b9430: No bootstrap required, opened a new log
I20260709 13:20:26.655228 19995 ts_tablet_manager.cc:1403] T e804b7d8fd7e44f59516fb299bea7a2c P c62716bf66c24330860573535b8b9430: Time spent bootstrapping tablet: real 0.063s	user 0.005s	sys 0.020s
I20260709 13:20:26.655383 19722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353a546cf8c54671971c239502ef165a" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
I20260709 13:20:26.656045 19722 raft_consensus.cc:3060] T 353a546cf8c54671971c239502ef165a P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:26.656973 19799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353a546cf8c54671971c239502ef165a" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
I20260709 13:20:26.657569 19799 raft_consensus.cc:3060] T 353a546cf8c54671971c239502ef165a P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:26.657910 19995 raft_consensus.cc:359] T e804b7d8fd7e44f59516fb299bea7a2c P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:26.658815 19995 raft_consensus.cc:385] T e804b7d8fd7e44f59516fb299bea7a2c P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:26.659117 19995 raft_consensus.cc:740] T e804b7d8fd7e44f59516fb299bea7a2c P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62716bf66c24330860573535b8b9430, State: Initialized, Role: FOLLOWER
I20260709 13:20:26.660320 19993 tablet_bootstrap.cc:492] T a36c11f368f34bf0aa4925a099ac1f91 P e90ffed3ba174ababeb21d9415d4d918: Bootstrap starting.
I20260709 13:20:26.659948 19995 consensus_queue.cc:260] T e804b7d8fd7e44f59516fb299bea7a2c P c62716bf66c24330860573535b8b9430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:26.662323 19995 ts_tablet_manager.cc:1434] T e804b7d8fd7e44f59516fb299bea7a2c P c62716bf66c24330860573535b8b9430: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:20:26.663525 19995 tablet_bootstrap.cc:492] T d1da95785dbf4ef4bde6f5e6ab35b444 P c62716bf66c24330860573535b8b9430: Bootstrap starting.
I20260709 13:20:26.666113 19799 raft_consensus.cc:2468] T 353a546cf8c54671971c239502ef165a P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c62716bf66c24330860573535b8b9430 in term 1.
I20260709 13:20:26.667222 19993 tablet_bootstrap.cc:654] T a36c11f368f34bf0aa4925a099ac1f91 P e90ffed3ba174ababeb21d9415d4d918: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:26.667970 19913 leader_election.cc:304] T 353a546cf8c54671971c239502ef165a P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c62716bf66c24330860573535b8b9430, e90ffed3ba174ababeb21d9415d4d918; no voters: 
I20260709 13:20:26.669552 19995 tablet_bootstrap.cc:654] T d1da95785dbf4ef4bde6f5e6ab35b444 P c62716bf66c24330860573535b8b9430: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:26.670974 20035 raft_consensus.cc:2804] T 353a546cf8c54671971c239502ef165a P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:26.671469 20035 raft_consensus.cc:697] T 353a546cf8c54671971c239502ef165a P c62716bf66c24330860573535b8b9430 [term 1 LEADER]: Becoming Leader. State: Replica: c62716bf66c24330860573535b8b9430, State: Running, Role: LEADER
I20260709 13:20:26.672272 20035 consensus_queue.cc:237] T 353a546cf8c54671971c239502ef165a P c62716bf66c24330860573535b8b9430 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:26.674937 19995 tablet_bootstrap.cc:492] T d1da95785dbf4ef4bde6f5e6ab35b444 P c62716bf66c24330860573535b8b9430: No bootstrap required, opened a new log
I20260709 13:20:26.675434 19995 ts_tablet_manager.cc:1403] T d1da95785dbf4ef4bde6f5e6ab35b444 P c62716bf66c24330860573535b8b9430: Time spent bootstrapping tablet: real 0.012s	user 0.010s	sys 0.000s
I20260709 13:20:26.678083 19995 raft_consensus.cc:359] T d1da95785dbf4ef4bde6f5e6ab35b444 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:26.666667 20000 leader_election.cc:290] T 353a546cf8c54671971c239502ef165a P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 election: Requested vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:26.680733 19994 tablet_bootstrap.cc:492] T a323eb80c5084159803e7d4ea5d9c410 P 8df38c4c53c44e94b36c1e8992476d7c: No bootstrap required, opened a new log
I20260709 13:20:26.681298 19994 ts_tablet_manager.cc:1403] T a323eb80c5084159803e7d4ea5d9c410 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.094s	user 0.011s	sys 0.035s
I20260709 13:20:26.681066 19722 raft_consensus.cc:2468] T 353a546cf8c54671971c239502ef165a P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c62716bf66c24330860573535b8b9430 in term 1.
I20260709 13:20:26.686975 19994 raft_consensus.cc:359] T a323eb80c5084159803e7d4ea5d9c410 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:26.687716 19994 raft_consensus.cc:385] T a323eb80c5084159803e7d4ea5d9c410 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:26.688000 19994 raft_consensus.cc:740] T a323eb80c5084159803e7d4ea5d9c410 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:26.688823 19994 consensus_queue.cc:260] T a323eb80c5084159803e7d4ea5d9c410 P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:26.691296 19994 ts_tablet_manager.cc:1434] T a323eb80c5084159803e7d4ea5d9c410 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 13:20:26.679206 19995 raft_consensus.cc:385] T d1da95785dbf4ef4bde6f5e6ab35b444 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:26.691803 19995 raft_consensus.cc:740] T d1da95785dbf4ef4bde6f5e6ab35b444 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62716bf66c24330860573535b8b9430, State: Initialized, Role: FOLLOWER
I20260709 13:20:26.693162 19994 tablet_bootstrap.cc:492] T ad38a1519fb748588f5a322589274004 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:26.692265 20000 raft_consensus.cc:493] T 0587017774214f24bdfa4a3da36e367e P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:26.696100 20000 raft_consensus.cc:515] T 0587017774214f24bdfa4a3da36e367e P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:26.696735 19995 consensus_queue.cc:260] T d1da95785dbf4ef4bde6f5e6ab35b444 P c62716bf66c24330860573535b8b9430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:26.698977 19995 ts_tablet_manager.cc:1434] T d1da95785dbf4ef4bde6f5e6ab35b444 P c62716bf66c24330860573535b8b9430: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.001s
I20260709 13:20:26.699918 19995 tablet_bootstrap.cc:492] T a323eb80c5084159803e7d4ea5d9c410 P c62716bf66c24330860573535b8b9430: Bootstrap starting.
I20260709 13:20:26.701922 20064 consensus_queue.cc:1048] T 984fa8de2ccb49359c2b475e4adfc1ce P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:20:26.707201 20000 leader_election.cc:290] T 0587017774214f24bdfa4a3da36e367e P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:26.709882 19873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0587017774214f24bdfa4a3da36e367e" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:26.709532 19997 tablet_bootstrap.cc:492] T e9369d9b7f5649be932967d24689a936 P b80b3a62d7d549a9a5c42def46769ac4: No bootstrap required, opened a new log
I20260709 13:20:26.711392 19997 ts_tablet_manager.cc:1403] T e9369d9b7f5649be932967d24689a936 P b80b3a62d7d549a9a5c42def46769ac4: Time spent bootstrapping tablet: real 0.127s	user 0.015s	sys 0.000s
I20260709 13:20:26.711905 20064 raft_consensus.cc:493] T d1da95785dbf4ef4bde6f5e6ab35b444 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:26.712481 19913 leader_election.cc:343] T 0587017774214f24bdfa4a3da36e367e P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:26.712574 19724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0587017774214f24bdfa4a3da36e367e" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
W20260709 13:20:26.714176 19914 leader_election.cc:343] T 0587017774214f24bdfa4a3da36e367e P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:26.714619 19914 leader_election.cc:304] T 0587017774214f24bdfa4a3da36e367e P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4
I20260709 13:20:26.715075 19997 raft_consensus.cc:359] T e9369d9b7f5649be932967d24689a936 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:26.712595 20064 raft_consensus.cc:515] T d1da95785dbf4ef4bde6f5e6ab35b444 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:26.715785 19997 raft_consensus.cc:385] T e9369d9b7f5649be932967d24689a936 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:26.718176 19997 raft_consensus.cc:740] T e9369d9b7f5649be932967d24689a936 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Initialized, Role: FOLLOWER
I20260709 13:20:26.719486 20064 leader_election.cc:290] T d1da95785dbf4ef4bde6f5e6ab35b444 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:26.720117 20080 raft_consensus.cc:493] T 984fa8de2ccb49359c2b475e4adfc1ce P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:26.720264 19945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1da95785dbf4ef4bde6f5e6ab35b444" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:26.719528 19997 consensus_queue.cc:260] T e9369d9b7f5649be932967d24689a936 P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:26.720651 20080 raft_consensus.cc:515] T 984fa8de2ccb49359c2b475e4adfc1ce P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:26.720949 19945 raft_consensus.cc:2468] T d1da95785dbf4ef4bde6f5e6ab35b444 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 0.
I20260709 13:20:26.723387 19765 leader_election.cc:304] T d1da95785dbf4ef4bde6f5e6ab35b444 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c62716bf66c24330860573535b8b9430, e90ffed3ba174ababeb21d9415d4d918; no voters: 
I20260709 13:20:26.723640 20080 leader_election.cc:290] T 984fa8de2ccb49359c2b475e4adfc1ce P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:26.724169 20000 raft_consensus.cc:2749] T 0587017774214f24bdfa4a3da36e367e P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:26.724026 19724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1da95785dbf4ef4bde6f5e6ab35b444" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:26.716737 19622 catalog_manager.cc:5697] T 353a546cf8c54671971c239502ef165a P c62716bf66c24330860573535b8b9430 reported cstate change: term changed from 0 to 1, leader changed from <none> to c62716bf66c24330860573535b8b9430 (127.9.116.132). New cstate: current_term: 1 leader_uuid: "c62716bf66c24330860573535b8b9430" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:26.732096 19994 tablet_bootstrap.cc:654] T ad38a1519fb748588f5a322589274004 P 8df38c4c53c44e94b36c1e8992476d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:26.724923 19723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984fa8de2ccb49359c2b475e4adfc1ce" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:26.733310 19799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984fa8de2ccb49359c2b475e4adfc1ce" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:26.736330 20080 raft_consensus.cc:493] T e9369d9b7f5649be932967d24689a936 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:26.736829 20080 raft_consensus.cc:515] T e9369d9b7f5649be932967d24689a936 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:26.738914 20080 leader_election.cc:290] T e9369d9b7f5649be932967d24689a936 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:26.741175 19945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9369d9b7f5649be932967d24689a936" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:26.742532 19799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9369d9b7f5649be932967d24689a936" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
W20260709 13:20:26.744812 19839 leader_election.cc:343] T e9369d9b7f5649be932967d24689a936 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c62716bf66c24330860573535b8b9430 (127.9.116.132:42643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:26.746594 19995 tablet_bootstrap.cc:654] T a323eb80c5084159803e7d4ea5d9c410 P c62716bf66c24330860573535b8b9430: Neither blocks nor log segments found. Creating new log.
W20260709 13:20:26.748332 19839 leader_election.cc:343] T e9369d9b7f5649be932967d24689a936 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:26.748998 19839 leader_election.cc:304] T e9369d9b7f5649be932967d24689a936 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4; no voters: c62716bf66c24330860573535b8b9430, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:26.750682 19840 leader_election.cc:304] T 984fa8de2ccb49359c2b475e4adfc1ce P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4; no voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:26.745464 20064 raft_consensus.cc:2804] T d1da95785dbf4ef4bde6f5e6ab35b444 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:26.751638 20064 raft_consensus.cc:493] T d1da95785dbf4ef4bde6f5e6ab35b444 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:26.752118 20064 raft_consensus.cc:3060] T d1da95785dbf4ef4bde6f5e6ab35b444 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:26.752272 20037 raft_consensus.cc:2749] T e9369d9b7f5649be932967d24689a936 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:26.753060 20037 raft_consensus.cc:2749] T 984fa8de2ccb49359c2b475e4adfc1ce P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:26.756160 19997 ts_tablet_manager.cc:1434] T e9369d9b7f5649be932967d24689a936 P b80b3a62d7d549a9a5c42def46769ac4: Time spent starting tablet: real 0.044s	user 0.008s	sys 0.000s
W20260709 13:20:26.756474 19766 leader_election.cc:343] T d1da95785dbf4ef4bde6f5e6ab35b444 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:26.767515 19997 tablet_bootstrap.cc:492] T 7e3a9d11df99411d8f0edb0df6f14d87 P b80b3a62d7d549a9a5c42def46769ac4: Bootstrap starting.
I20260709 13:20:26.774538 19873 raft_consensus.cc:3060] T 984fa8de2ccb49359c2b475e4adfc1ce P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:26.775789 19997 tablet_bootstrap.cc:654] T 7e3a9d11df99411d8f0edb0df6f14d87 P b80b3a62d7d549a9a5c42def46769ac4: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:26.803238 20040 consensus_queue.cc:1048] T 984fa8de2ccb49359c2b475e4adfc1ce P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:20:26.799589 20064 raft_consensus.cc:515] T d1da95785dbf4ef4bde6f5e6ab35b444 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:26.829666 20000 raft_consensus.cc:493] T e804b7d8fd7e44f59516fb299bea7a2c P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:26.830268 20000 raft_consensus.cc:515] T e804b7d8fd7e44f59516fb299bea7a2c P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:26.834378 19994 tablet_bootstrap.cc:492] T ad38a1519fb748588f5a322589274004 P 8df38c4c53c44e94b36c1e8992476d7c: No bootstrap required, opened a new log
I20260709 13:20:26.835239 19994 ts_tablet_manager.cc:1403] T ad38a1519fb748588f5a322589274004 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.142s	user 0.042s	sys 0.040s
I20260709 13:20:26.832463 20000 leader_election.cc:290] T e804b7d8fd7e44f59516fb299bea7a2c P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:26.830446 20064 leader_election.cc:290] T d1da95785dbf4ef4bde6f5e6ab35b444 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 election: Requested vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:26.837261 19993 tablet_bootstrap.cc:492] T a36c11f368f34bf0aa4925a099ac1f91 P e90ffed3ba174ababeb21d9415d4d918: No bootstrap required, opened a new log
I20260709 13:20:26.839250 19993 ts_tablet_manager.cc:1403] T a36c11f368f34bf0aa4925a099ac1f91 P e90ffed3ba174ababeb21d9415d4d918: Time spent bootstrapping tablet: real 0.179s	user 0.024s	sys 0.049s
I20260709 13:20:26.843350 19722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1da95785dbf4ef4bde6f5e6ab35b444" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
I20260709 13:20:26.841559 19945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1da95785dbf4ef4bde6f5e6ab35b444" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:26.844388 19945 raft_consensus.cc:3060] T d1da95785dbf4ef4bde6f5e6ab35b444 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:26.844677 19724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e804b7d8fd7e44f59516fb299bea7a2c" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
W20260709 13:20:26.854175 19766 leader_election.cc:343] T d1da95785dbf4ef4bde6f5e6ab35b444 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:26.857571 19997 tablet_bootstrap.cc:492] T 7e3a9d11df99411d8f0edb0df6f14d87 P b80b3a62d7d549a9a5c42def46769ac4: No bootstrap required, opened a new log
I20260709 13:20:26.858208 19997 ts_tablet_manager.cc:1403] T 7e3a9d11df99411d8f0edb0df6f14d87 P b80b3a62d7d549a9a5c42def46769ac4: Time spent bootstrapping tablet: real 0.091s	user 0.018s	sys 0.000s
I20260709 13:20:26.862124 20132 raft_consensus.cc:493] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:26.862877 20132 raft_consensus.cc:515] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:26.863544 19945 raft_consensus.cc:2468] T d1da95785dbf4ef4bde6f5e6ab35b444 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 1.
I20260709 13:20:26.864919 20132 leader_election.cc:290] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:26.865157 19765 leader_election.cc:304] T d1da95785dbf4ef4bde6f5e6ab35b444 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c62716bf66c24330860573535b8b9430, e90ffed3ba174ababeb21d9415d4d918; no voters: 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:26.865371 19993 raft_consensus.cc:359] T a36c11f368f34bf0aa4925a099ac1f91 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:26.866061 19993 raft_consensus.cc:385] T a36c11f368f34bf0aa4925a099ac1f91 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:26.866364 19993 raft_consensus.cc:740] T a36c11f368f34bf0aa4925a099ac1f91 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Initialized, Role: FOLLOWER
I20260709 13:20:26.867340 20065 raft_consensus.cc:2804] T d1da95785dbf4ef4bde6f5e6ab35b444 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:26.867903 20065 raft_consensus.cc:697] T d1da95785dbf4ef4bde6f5e6ab35b444 P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Becoming Leader. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Running, Role: LEADER
I20260709 13:20:26.867624 19997 raft_consensus.cc:359] T 7e3a9d11df99411d8f0edb0df6f14d87 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:26.868515 19997 raft_consensus.cc:385] T 7e3a9d11df99411d8f0edb0df6f14d87 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:26.868415 19993 consensus_queue.cc:260] T a36c11f368f34bf0aa4925a099ac1f91 P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:26.868875 19997 raft_consensus.cc:740] T 7e3a9d11df99411d8f0edb0df6f14d87 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Initialized, Role: FOLLOWER
I20260709 13:20:26.868832 20065 consensus_queue.cc:237] T d1da95785dbf4ef4bde6f5e6ab35b444 P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:26.870584 19993 ts_tablet_manager.cc:1434] T a36c11f368f34bf0aa4925a099ac1f91 P e90ffed3ba174ababeb21d9415d4d918: Time spent starting tablet: real 0.031s	user 0.004s	sys 0.000s
I20260709 13:20:26.871591 19993 tablet_bootstrap.cc:492] T 5b9511148896441a948b772784f62a3d P e90ffed3ba174ababeb21d9415d4d918: Bootstrap starting.
I20260709 13:20:26.873057 19997 consensus_queue.cc:260] T 7e3a9d11df99411d8f0edb0df6f14d87 P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:26.874225 19874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19e83b82855d4129bb8525e6ff05316b" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:26.867592 19994 raft_consensus.cc:359] T ad38a1519fb748588f5a322589274004 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:26.878258 19994 raft_consensus.cc:385] T ad38a1519fb748588f5a322589274004 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:26.878592 19994 raft_consensus.cc:740] T ad38a1519fb748588f5a322589274004 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:26.878736 19993 tablet_bootstrap.cc:654] T 5b9511148896441a948b772784f62a3d P e90ffed3ba174ababeb21d9415d4d918: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:26.875164 19873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e804b7d8fd7e44f59516fb299bea7a2c" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:26.879750 19873 raft_consensus.cc:2393] T e804b7d8fd7e44f59516fb299bea7a2c P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c62716bf66c24330860573535b8b9430 in current term 1: Already voted for candidate 8df38c4c53c44e94b36c1e8992476d7c in this term.
I20260709 13:20:26.886479 19913 leader_election.cc:304] T e804b7d8fd7e44f59516fb299bea7a2c P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4
I20260709 13:20:26.887326 20000 raft_consensus.cc:3060] T e804b7d8fd7e44f59516fb299bea7a2c P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:26.879410 19994 consensus_queue.cc:260] T ad38a1519fb748588f5a322589274004 P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:26.888854 19994 ts_tablet_manager.cc:1434] T ad38a1519fb748588f5a322589274004 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.053s	user 0.006s	sys 0.000s
I20260709 13:20:26.893342 19994 tablet_bootstrap.cc:492] T 8bdf042a063d4596970f49730ffa04b4 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:26.880749 19945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19e83b82855d4129bb8525e6ff05316b" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:26.899523 19945 raft_consensus.cc:2468] T 19e83b82855d4129bb8525e6ff05316b P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 0.
I20260709 13:20:26.895812 20161 raft_consensus.cc:493] T 65e7a5222efe43e49e1fbb3a3d7cf99c P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:26.900959 20161 raft_consensus.cc:515] T 65e7a5222efe43e49e1fbb3a3d7cf99c P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:26.884192 19997 ts_tablet_manager.cc:1434] T 7e3a9d11df99411d8f0edb0df6f14d87 P b80b3a62d7d549a9a5c42def46769ac4: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260709 13:20:26.904023 19997 tablet_bootstrap.cc:492] T ad38a1519fb748588f5a322589274004 P b80b3a62d7d549a9a5c42def46769ac4: Bootstrap starting.
W20260709 13:20:26.881798 19690 leader_election.cc:343] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:26.908053 19690 leader_election.cc:304] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, c62716bf66c24330860573535b8b9430; no voters: b80b3a62d7d549a9a5c42def46769ac4
I20260709 13:20:26.909029 20000 raft_consensus.cc:2749] T e804b7d8fd7e44f59516fb299bea7a2c P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:26.911777 20132 raft_consensus.cc:2804] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:26.912220 20132 raft_consensus.cc:493] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:26.912544 20132 raft_consensus.cc:3060] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:26.914692 20161 leader_election.cc:290] T 65e7a5222efe43e49e1fbb3a3d7cf99c P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:26.916337 19871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e7a5222efe43e49e1fbb3a3d7cf99c" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:26.917025 19871 raft_consensus.cc:2468] T 65e7a5222efe43e49e1fbb3a3d7cf99c P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 0.
I20260709 13:20:26.917177 19945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e7a5222efe43e49e1fbb3a3d7cf99c" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:26.921092 19765 leader_election.cc:304] T 65e7a5222efe43e49e1fbb3a3d7cf99c P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, e90ffed3ba174ababeb21d9415d4d918; no voters: 
W20260709 13:20:26.922067 19765 leader_election.cc:343] T 65e7a5222efe43e49e1fbb3a3d7cf99c P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c62716bf66c24330860573535b8b9430 (127.9.116.132:42643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:26.923106 20161 raft_consensus.cc:2804] T 65e7a5222efe43e49e1fbb3a3d7cf99c P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:26.923568 20161 raft_consensus.cc:493] T 65e7a5222efe43e49e1fbb3a3d7cf99c P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:26.923928 20161 raft_consensus.cc:3060] T 65e7a5222efe43e49e1fbb3a3d7cf99c P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:26.924957 19997 tablet_bootstrap.cc:654] T ad38a1519fb748588f5a322589274004 P b80b3a62d7d549a9a5c42def46769ac4: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:26.916556 19995 tablet_bootstrap.cc:492] T a323eb80c5084159803e7d4ea5d9c410 P c62716bf66c24330860573535b8b9430: No bootstrap required, opened a new log
I20260709 13:20:26.931296 19995 ts_tablet_manager.cc:1403] T a323eb80c5084159803e7d4ea5d9c410 P c62716bf66c24330860573535b8b9430: Time spent bootstrapping tablet: real 0.232s	user 0.016s	sys 0.010s
I20260709 13:20:26.934199 19995 raft_consensus.cc:359] T a323eb80c5084159803e7d4ea5d9c410 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:26.936220 19997 tablet_bootstrap.cc:492] T ad38a1519fb748588f5a322589274004 P b80b3a62d7d549a9a5c42def46769ac4: No bootstrap required, opened a new log
I20260709 13:20:26.945448 19995 raft_consensus.cc:385] T a323eb80c5084159803e7d4ea5d9c410 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:26.945740 19997 ts_tablet_manager.cc:1403] T ad38a1519fb748588f5a322589274004 P b80b3a62d7d549a9a5c42def46769ac4: Time spent bootstrapping tablet: real 0.042s	user 0.015s	sys 0.000s
I20260709 13:20:26.949071 19997 raft_consensus.cc:359] T ad38a1519fb748588f5a322589274004 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:26.949802 19997 raft_consensus.cc:385] T ad38a1519fb748588f5a322589274004 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:26.950138 19997 raft_consensus.cc:740] T ad38a1519fb748588f5a322589274004 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Initialized, Role: FOLLOWER
I20260709 13:20:26.940325 19994 tablet_bootstrap.cc:654] T 8bdf042a063d4596970f49730ffa04b4 P 8df38c4c53c44e94b36c1e8992476d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:26.953351 20161 raft_consensus.cc:515] T 65e7a5222efe43e49e1fbb3a3d7cf99c P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:26.953672 19997 consensus_queue.cc:260] T ad38a1519fb748588f5a322589274004 P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:26.945809 19995 raft_consensus.cc:740] T a323eb80c5084159803e7d4ea5d9c410 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62716bf66c24330860573535b8b9430, State: Initialized, Role: FOLLOWER
I20260709 13:20:26.955592 19995 consensus_queue.cc:260] T a323eb80c5084159803e7d4ea5d9c410 P c62716bf66c24330860573535b8b9430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:26.956398 19873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e7a5222efe43e49e1fbb3a3d7cf99c" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
I20260709 13:20:26.956858 19997 ts_tablet_manager.cc:1434] T ad38a1519fb748588f5a322589274004 P b80b3a62d7d549a9a5c42def46769ac4: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:20:26.957058 19873 raft_consensus.cc:3060] T 65e7a5222efe43e49e1fbb3a3d7cf99c P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:26.958227 19997 tablet_bootstrap.cc:492] T 0587017774214f24bdfa4a3da36e367e P b80b3a62d7d549a9a5c42def46769ac4: Bootstrap starting.
I20260709 13:20:26.963408 19995 ts_tablet_manager.cc:1434] T a323eb80c5084159803e7d4ea5d9c410 P c62716bf66c24330860573535b8b9430: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.001s
I20260709 13:20:26.965126 19995 tablet_bootstrap.cc:492] T 65e7a5222efe43e49e1fbb3a3d7cf99c P c62716bf66c24330860573535b8b9430: Bootstrap starting.
I20260709 13:20:26.969139 19873 raft_consensus.cc:2468] T 65e7a5222efe43e49e1fbb3a3d7cf99c P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 1.
I20260709 13:20:26.970479 19765 leader_election.cc:304] T 65e7a5222efe43e49e1fbb3a3d7cf99c P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, e90ffed3ba174ababeb21d9415d4d918; no voters: 
I20260709 13:20:26.971385 20040 raft_consensus.cc:2804] T 65e7a5222efe43e49e1fbb3a3d7cf99c P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:26.971855 20040 raft_consensus.cc:697] T 65e7a5222efe43e49e1fbb3a3d7cf99c P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Becoming Leader. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Running, Role: LEADER
I20260709 13:20:26.949872 20132 raft_consensus.cc:515] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:26.972645 20040 consensus_queue.cc:237] T 65e7a5222efe43e49e1fbb3a3d7cf99c P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:26.974731 20132 leader_election.cc:290] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Requested vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:26.976761 19945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19e83b82855d4129bb8525e6ff05316b" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:26.977046 19873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19e83b82855d4129bb8525e6ff05316b" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
I20260709 13:20:26.977445 19945 raft_consensus.cc:3060] T 19e83b82855d4129bb8525e6ff05316b P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:20:26.978438 19690 leader_election.cc:343] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:26.980151 19946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e7a5222efe43e49e1fbb3a3d7cf99c" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430"
W20260709 13:20:26.981631 19765 leader_election.cc:343] T 65e7a5222efe43e49e1fbb3a3d7cf99c P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c62716bf66c24330860573535b8b9430 (127.9.116.132:42643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:26.982249 20161 leader_election.cc:290] T 65e7a5222efe43e49e1fbb3a3d7cf99c P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 election: Requested vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:26.984081 19997 tablet_bootstrap.cc:654] T 0587017774214f24bdfa4a3da36e367e P b80b3a62d7d549a9a5c42def46769ac4: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:26.986513 20083 consensus_queue.cc:1048] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:26.987182 19945 raft_consensus.cc:2468] T 19e83b82855d4129bb8525e6ff05316b P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 1.
I20260709 13:20:26.988729 19690 leader_election.cc:304] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, c62716bf66c24330860573535b8b9430; no voters: b80b3a62d7d549a9a5c42def46769ac4
I20260709 13:20:26.990669 20083 raft_consensus.cc:2804] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:26.991158 20083 raft_consensus.cc:697] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c [term 1 LEADER]: Becoming Leader. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Running, Role: LEADER
I20260709 13:20:26.991439 19994 tablet_bootstrap.cc:492] T 8bdf042a063d4596970f49730ffa04b4 P 8df38c4c53c44e94b36c1e8992476d7c: No bootstrap required, opened a new log
I20260709 13:20:26.991922 19994 ts_tablet_manager.cc:1403] T 8bdf042a063d4596970f49730ffa04b4 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.099s	user 0.007s	sys 0.003s
I20260709 13:20:26.991971 20083 consensus_queue.cc:237] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:26.997726 19994 raft_consensus.cc:359] T 8bdf042a063d4596970f49730ffa04b4 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:26.998454 19994 raft_consensus.cc:385] T 8bdf042a063d4596970f49730ffa04b4 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:26.998737 19994 raft_consensus.cc:740] T 8bdf042a063d4596970f49730ffa04b4 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:27.003150 19994 consensus_queue.cc:260] T 8bdf042a063d4596970f49730ffa04b4 P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:27.003809 20180 consensus_queue.cc:1048] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:20:27.004603 19997 tablet_bootstrap.cc:492] T 0587017774214f24bdfa4a3da36e367e P b80b3a62d7d549a9a5c42def46769ac4: No bootstrap required, opened a new log
I20260709 13:20:27.005074 19997 ts_tablet_manager.cc:1403] T 0587017774214f24bdfa4a3da36e367e P b80b3a62d7d549a9a5c42def46769ac4: Time spent bootstrapping tablet: real 0.047s	user 0.016s	sys 0.000s
I20260709 13:20:27.009579 19620 catalog_manager.cc:5697] T d1da95785dbf4ef4bde6f5e6ab35b444 P e90ffed3ba174ababeb21d9415d4d918 reported cstate change: term changed from 0 to 1, leader changed from <none> to e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "e90ffed3ba174ababeb21d9415d4d918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:27.017020 19997 raft_consensus.cc:359] T 0587017774214f24bdfa4a3da36e367e P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:27.017652 19997 raft_consensus.cc:385] T 0587017774214f24bdfa4a3da36e367e P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:27.017964 19997 raft_consensus.cc:740] T 0587017774214f24bdfa4a3da36e367e P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Initialized, Role: FOLLOWER
I20260709 13:20:27.023213 19997 consensus_queue.cc:260] T 0587017774214f24bdfa4a3da36e367e P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:27.025364 19997 ts_tablet_manager.cc:1434] T 0587017774214f24bdfa4a3da36e367e P b80b3a62d7d549a9a5c42def46769ac4: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.000s
I20260709 13:20:27.026414 19997 tablet_bootstrap.cc:492] T 5b9511148896441a948b772784f62a3d P b80b3a62d7d549a9a5c42def46769ac4: Bootstrap starting.
I20260709 13:20:27.032725 19997 tablet_bootstrap.cc:654] T 5b9511148896441a948b772784f62a3d P b80b3a62d7d549a9a5c42def46769ac4: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:27.038913 19995 tablet_bootstrap.cc:654] T 65e7a5222efe43e49e1fbb3a3d7cf99c P c62716bf66c24330860573535b8b9430: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:27.042906 19993 tablet_bootstrap.cc:492] T 5b9511148896441a948b772784f62a3d P e90ffed3ba174ababeb21d9415d4d918: No bootstrap required, opened a new log
I20260709 13:20:27.043390 19993 ts_tablet_manager.cc:1403] T 5b9511148896441a948b772784f62a3d P e90ffed3ba174ababeb21d9415d4d918: Time spent bootstrapping tablet: real 0.172s	user 0.018s	sys 0.043s
I20260709 13:20:27.046097 19993 raft_consensus.cc:359] T 5b9511148896441a948b772784f62a3d P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:27.046824 19993 raft_consensus.cc:385] T 5b9511148896441a948b772784f62a3d P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:27.047089 19993 raft_consensus.cc:740] T 5b9511148896441a948b772784f62a3d P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Initialized, Role: FOLLOWER
I20260709 13:20:27.047762 19993 consensus_queue.cc:260] T 5b9511148896441a948b772784f62a3d P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:27.049815 19993 ts_tablet_manager.cc:1434] T 5b9511148896441a948b772784f62a3d P e90ffed3ba174ababeb21d9415d4d918: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.003s
I20260709 13:20:27.049412 19622 catalog_manager.cc:5697] T 65e7a5222efe43e49e1fbb3a3d7cf99c P e90ffed3ba174ababeb21d9415d4d918 reported cstate change: term changed from 0 to 1, leader changed from <none> to e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "e90ffed3ba174ababeb21d9415d4d918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:27.052605 19994 ts_tablet_manager.cc:1434] T 8bdf042a063d4596970f49730ffa04b4 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.060s	user 0.007s	sys 0.000s
I20260709 13:20:27.038110 19620 catalog_manager.cc:5697] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c reported cstate change: term changed from 0 to 1, leader changed from <none> to 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129). New cstate: current_term: 1 leader_uuid: "8df38c4c53c44e94b36c1e8992476d7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:27.054045 19993 tablet_bootstrap.cc:492] T 9b9137146e3644a6b0026778530fa8a6 P e90ffed3ba174ababeb21d9415d4d918: Bootstrap starting.
I20260709 13:20:27.055306 19994 tablet_bootstrap.cc:492] T 0587017774214f24bdfa4a3da36e367e P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:27.061129 19993 tablet_bootstrap.cc:654] T 9b9137146e3644a6b0026778530fa8a6 P e90ffed3ba174ababeb21d9415d4d918: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:27.095726 19997 tablet_bootstrap.cc:492] T 5b9511148896441a948b772784f62a3d P b80b3a62d7d549a9a5c42def46769ac4: No bootstrap required, opened a new log
I20260709 13:20:27.096257 19997 ts_tablet_manager.cc:1403] T 5b9511148896441a948b772784f62a3d P b80b3a62d7d549a9a5c42def46769ac4: Time spent bootstrapping tablet: real 0.070s	user 0.032s	sys 0.036s
I20260709 13:20:27.097512 19994 tablet_bootstrap.cc:654] T 0587017774214f24bdfa4a3da36e367e P 8df38c4c53c44e94b36c1e8992476d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:27.099284 19997 raft_consensus.cc:359] T 5b9511148896441a948b772784f62a3d P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:27.100939 19997 raft_consensus.cc:385] T 5b9511148896441a948b772784f62a3d P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:27.101241 19997 raft_consensus.cc:740] T 5b9511148896441a948b772784f62a3d P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Initialized, Role: FOLLOWER
I20260709 13:20:27.101579 20000 consensus_queue.cc:1048] T 353a546cf8c54671971c239502ef165a P c62716bf66c24330860573535b8b9430 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:27.102200 19997 consensus_queue.cc:260] T 5b9511148896441a948b772784f62a3d P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:27.111567 19993 tablet_bootstrap.cc:492] T 9b9137146e3644a6b0026778530fa8a6 P e90ffed3ba174ababeb21d9415d4d918: No bootstrap required, opened a new log
I20260709 13:20:27.112164 19993 ts_tablet_manager.cc:1403] T 9b9137146e3644a6b0026778530fa8a6 P e90ffed3ba174ababeb21d9415d4d918: Time spent bootstrapping tablet: real 0.062s	user 0.021s	sys 0.023s
I20260709 13:20:27.112828 20181 consensus_queue.cc:1048] T 86964b8a61ce494f9096c4fefdc6efbf P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:27.121388 20040 consensus_queue.cc:1048] T 85d493223fe04c939b6effd840955727 P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:20:27.126262 19997 ts_tablet_manager.cc:1434] T 5b9511148896441a948b772784f62a3d P b80b3a62d7d549a9a5c42def46769ac4: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.000s
I20260709 13:20:27.128265 19997 tablet_bootstrap.cc:492] T 19e83b82855d4129bb8525e6ff05316b P b80b3a62d7d549a9a5c42def46769ac4: Bootstrap starting.
I20260709 13:20:27.137013 19993 raft_consensus.cc:359] T 9b9137146e3644a6b0026778530fa8a6 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:27.137936 19993 raft_consensus.cc:385] T 9b9137146e3644a6b0026778530fa8a6 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:27.138281 19993 raft_consensus.cc:740] T 9b9137146e3644a6b0026778530fa8a6 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Initialized, Role: FOLLOWER
I20260709 13:20:27.138687 19995 tablet_bootstrap.cc:492] T 65e7a5222efe43e49e1fbb3a3d7cf99c P c62716bf66c24330860573535b8b9430: No bootstrap required, opened a new log
I20260709 13:20:27.139274 19995 ts_tablet_manager.cc:1403] T 65e7a5222efe43e49e1fbb3a3d7cf99c P c62716bf66c24330860573535b8b9430: Time spent bootstrapping tablet: real 0.174s	user 0.009s	sys 0.002s
I20260709 13:20:27.142076 19995 raft_consensus.cc:359] T 65e7a5222efe43e49e1fbb3a3d7cf99c P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:27.142771 19995 raft_consensus.cc:385] T 65e7a5222efe43e49e1fbb3a3d7cf99c P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:27.143065 19995 raft_consensus.cc:740] T 65e7a5222efe43e49e1fbb3a3d7cf99c P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62716bf66c24330860573535b8b9430, State: Initialized, Role: FOLLOWER
I20260709 13:20:27.143332 19997 tablet_bootstrap.cc:654] T 19e83b82855d4129bb8525e6ff05316b P b80b3a62d7d549a9a5c42def46769ac4: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:27.144037 19995 consensus_queue.cc:260] T 65e7a5222efe43e49e1fbb3a3d7cf99c P c62716bf66c24330860573535b8b9430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:27.146653 19995 ts_tablet_manager.cc:1434] T 65e7a5222efe43e49e1fbb3a3d7cf99c P c62716bf66c24330860573535b8b9430: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.002s
I20260709 13:20:27.148015 19995 tablet_bootstrap.cc:492] T 7e3099f1888c4ed18c30523bf9ec94f5 P c62716bf66c24330860573535b8b9430: Bootstrap starting.
I20260709 13:20:27.148439 20083 raft_consensus.cc:493] T a323eb80c5084159803e7d4ea5d9c410 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:27.148981 20083 raft_consensus.cc:515] T a323eb80c5084159803e7d4ea5d9c410 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
W20260709 13:20:27.152725 19765 consensus_peers.cc:597] T 85d493223fe04c939b6effd840955727 P e90ffed3ba174ababeb21d9415d4d918 -> Peer c62716bf66c24330860573535b8b9430 (127.9.116.132:42643): Couldn't send request to peer c62716bf66c24330860573535b8b9430. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:27.153707 20083 leader_election.cc:290] T a323eb80c5084159803e7d4ea5d9c410 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:27.156683 19945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a323eb80c5084159803e7d4ea5d9c410" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:27.157409 19945 raft_consensus.cc:2468] T a323eb80c5084159803e7d4ea5d9c410 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 0.
I20260709 13:20:27.159968 19799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a323eb80c5084159803e7d4ea5d9c410" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:27.163846 19995 tablet_bootstrap.cc:654] T 7e3099f1888c4ed18c30523bf9ec94f5 P c62716bf66c24330860573535b8b9430: Neither blocks nor log segments found. Creating new log.
W20260709 13:20:27.167626 19690 leader_election.cc:343] T a323eb80c5084159803e7d4ea5d9c410 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:27.168423 19690 leader_election.cc:304] T a323eb80c5084159803e7d4ea5d9c410 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, c62716bf66c24330860573535b8b9430; no voters: e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:27.169829 20031 raft_consensus.cc:2804] T a323eb80c5084159803e7d4ea5d9c410 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:27.170205 20031 raft_consensus.cc:493] T a323eb80c5084159803e7d4ea5d9c410 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:27.170532 20031 raft_consensus.cc:3060] T a323eb80c5084159803e7d4ea5d9c410 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:27.174659 20083 consensus_queue.cc:1048] T 86964b8a61ce494f9096c4fefdc6efbf P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:20:27.178867 20040 raft_consensus.cc:493] T a36c11f368f34bf0aa4925a099ac1f91 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:27.179456 20040 raft_consensus.cc:515] T a36c11f368f34bf0aa4925a099ac1f91 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:27.181016 20000 consensus_queue.cc:1048] T 353a546cf8c54671971c239502ef165a P c62716bf66c24330860573535b8b9430 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:27.183423 20040 leader_election.cc:290] T a36c11f368f34bf0aa4925a099ac1f91 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:27.184954 19873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a36c11f368f34bf0aa4925a099ac1f91" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:27.185689 19873 raft_consensus.cc:2468] T a36c11f368f34bf0aa4925a099ac1f91 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 0.
I20260709 13:20:27.187026 19765 leader_election.cc:304] T a36c11f368f34bf0aa4925a099ac1f91 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, e90ffed3ba174ababeb21d9415d4d918; no voters: 
I20260709 13:20:27.187399 19946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a36c11f368f34bf0aa4925a099ac1f91" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
W20260709 13:20:27.189325 19765 leader_election.cc:343] T a36c11f368f34bf0aa4925a099ac1f91 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c62716bf66c24330860573535b8b9430 (127.9.116.132:42643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:27.203644 19993 consensus_queue.cc:260] T 9b9137146e3644a6b0026778530fa8a6 P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:27.190373 20040 raft_consensus.cc:2804] T a36c11f368f34bf0aa4925a099ac1f91 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:27.206377 20000 raft_consensus.cc:493] T 65e7a5222efe43e49e1fbb3a3d7cf99c P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:27.208650 20000 raft_consensus.cc:515] T 65e7a5222efe43e49e1fbb3a3d7cf99c P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:27.191097 20161 raft_consensus.cc:493] T 5b9511148896441a948b772784f62a3d P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:27.211573 20161 raft_consensus.cc:515] T 5b9511148896441a948b772784f62a3d P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:27.213310 19873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e7a5222efe43e49e1fbb3a3d7cf99c" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:27.213632 20161 leader_election.cc:290] T 5b9511148896441a948b772784f62a3d P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:27.214159 19873 raft_consensus.cc:2393] T 65e7a5222efe43e49e1fbb3a3d7cf99c P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c62716bf66c24330860573535b8b9430 in current term 1: Already voted for candidate e90ffed3ba174ababeb21d9415d4d918 in this term.
I20260709 13:20:27.214550 20009 raft_consensus.cc:493] T 7e3a9d11df99411d8f0edb0df6f14d87 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:27.215055 20009 raft_consensus.cc:515] T 7e3a9d11df99411d8f0edb0df6f14d87 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:27.217157 20009 leader_election.cc:290] T 7e3a9d11df99411d8f0edb0df6f14d87 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:27.218261 19724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9511148896441a948b772784f62a3d" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:27.218986 19724 raft_consensus.cc:2468] T 5b9511148896441a948b772784f62a3d P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 0.
I20260709 13:20:27.219781 19722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3a9d11df99411d8f0edb0df6f14d87" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:27.220713 19799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3a9d11df99411d8f0edb0df6f14d87" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:27.221740 19766 leader_election.cc:304] T 5b9511148896441a948b772784f62a3d P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918; no voters: 
W20260709 13:20:27.222196 19839 leader_election.cc:343] T 7e3a9d11df99411d8f0edb0df6f14d87 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:27.222849 20173 raft_consensus.cc:2804] T 5b9511148896441a948b772784f62a3d P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:27.223277 20173 raft_consensus.cc:493] T 5b9511148896441a948b772784f62a3d P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:27.223596 20173 raft_consensus.cc:3060] T 5b9511148896441a948b772784f62a3d P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:27.210712 20000 leader_election.cc:290] T 65e7a5222efe43e49e1fbb3a3d7cf99c P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
W20260709 13:20:27.227228 19840 leader_election.cc:343] T 7e3a9d11df99411d8f0edb0df6f14d87 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:27.227764 19840 leader_election.cc:304] T 7e3a9d11df99411d8f0edb0df6f14d87 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4; no voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:27.238385 20037 raft_consensus.cc:2749] T 7e3a9d11df99411d8f0edb0df6f14d87 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:27.238917 20009 raft_consensus.cc:493] T ad38a1519fb748588f5a322589274004 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:27.266873 20009 raft_consensus.cc:515] T ad38a1519fb748588f5a322589274004 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:27.269052 20009 leader_election.cc:290] T ad38a1519fb748588f5a322589274004 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:27.270996 20181 raft_consensus.cc:493] T ad38a1519fb748588f5a322589274004 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:27.271744 20181 raft_consensus.cc:515] T ad38a1519fb748588f5a322589274004 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:27.240171 20080 raft_consensus.cc:493] T 0587017774214f24bdfa4a3da36e367e P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:27.272493 19723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad38a1519fb748588f5a322589274004" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:27.272766 20080 raft_consensus.cc:515] T 0587017774214f24bdfa4a3da36e367e P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:27.273195 19723 raft_consensus.cc:2468] T ad38a1519fb748588f5a322589274004 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 0.
I20260709 13:20:27.274003 20181 leader_election.cc:290] T ad38a1519fb748588f5a322589274004 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:27.234710 20173 raft_consensus.cc:515] T 5b9511148896441a948b772784f62a3d P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:27.274693 19840 leader_election.cc:304] T ad38a1519fb748588f5a322589274004 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4; no voters: 
I20260709 13:20:27.247924 20031 raft_consensus.cc:515] T a323eb80c5084159803e7d4ea5d9c410 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:27.275619 20009 raft_consensus.cc:2804] T ad38a1519fb748588f5a322589274004 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:27.276039 20009 raft_consensus.cc:493] T ad38a1519fb748588f5a322589274004 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:27.276408 20009 raft_consensus.cc:3060] T ad38a1519fb748588f5a322589274004 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:27.276544 19945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0587017774214f24bdfa4a3da36e367e" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:27.274878 20080 leader_election.cc:290] T 0587017774214f24bdfa4a3da36e367e P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:27.276885 19723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0587017774214f24bdfa4a3da36e367e" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:27.260064 19997 tablet_bootstrap.cc:492] T 19e83b82855d4129bb8525e6ff05316b P b80b3a62d7d549a9a5c42def46769ac4: No bootstrap required, opened a new log
I20260709 13:20:27.278079 19997 ts_tablet_manager.cc:1403] T 19e83b82855d4129bb8525e6ff05316b P b80b3a62d7d549a9a5c42def46769ac4: Time spent bootstrapping tablet: real 0.150s	user 0.045s	sys 0.032s
W20260709 13:20:27.278688 19840 leader_election.cc:343] T 0587017774214f24bdfa4a3da36e367e P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:27.278971 19797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad38a1519fb748588f5a322589274004" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:27.239578 20040 raft_consensus.cc:493] T a36c11f368f34bf0aa4925a099ac1f91 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:27.279714 19797 raft_consensus.cc:2468] T ad38a1519fb748588f5a322589274004 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 0.
I20260709 13:20:27.272547 19798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad38a1519fb748588f5a322589274004" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:27.280633 19798 raft_consensus.cc:2468] T ad38a1519fb748588f5a322589274004 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 0.
I20260709 13:20:27.281153 19997 raft_consensus.cc:359] T 19e83b82855d4129bb8525e6ff05316b P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:27.281888 19997 raft_consensus.cc:385] T 19e83b82855d4129bb8525e6ff05316b P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:27.282143 19997 raft_consensus.cc:740] T 19e83b82855d4129bb8525e6ff05316b P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Initialized, Role: FOLLOWER
I20260709 13:20:27.277303 19945 raft_consensus.cc:2468] T 0587017774214f24bdfa4a3da36e367e P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 0.
I20260709 13:20:27.283188 19800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e7a5222efe43e49e1fbb3a3d7cf99c" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:27.283779 19839 leader_election.cc:304] T 0587017774214f24bdfa4a3da36e367e P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:27.284962 20080 raft_consensus.cc:2804] T 0587017774214f24bdfa4a3da36e367e P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:27.284752 19997 consensus_queue.cc:260] T 19e83b82855d4129bb8525e6ff05316b P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:27.285377 20080 raft_consensus.cc:493] T 0587017774214f24bdfa4a3da36e367e P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:27.285775 20080 raft_consensus.cc:3060] T 0587017774214f24bdfa4a3da36e367e P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:27.287204 19993 ts_tablet_manager.cc:1434] T 9b9137146e3644a6b0026778530fa8a6 P e90ffed3ba174ababeb21d9415d4d918: Time spent starting tablet: real 0.175s	user 0.007s	sys 0.002s
I20260709 13:20:27.288308 19993 tablet_bootstrap.cc:492] T e9369d9b7f5649be932967d24689a936 P e90ffed3ba174ababeb21d9415d4d918: Bootstrap starting.
I20260709 13:20:27.261528 20083 raft_consensus.cc:493] T 8bdf042a063d4596970f49730ffa04b4 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:27.277287 20031 leader_election.cc:290] T a323eb80c5084159803e7d4ea5d9c410 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Requested vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:27.263984 19873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9511148896441a948b772784f62a3d" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:27.263494 19994 tablet_bootstrap.cc:492] T 0587017774214f24bdfa4a3da36e367e P 8df38c4c53c44e94b36c1e8992476d7c: No bootstrap required, opened a new log
I20260709 13:20:27.290721 19871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad38a1519fb748588f5a322589274004" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:27.291258 19690 leader_election.cc:304] T ad38a1519fb748588f5a322589274004 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918; no voters: 
I20260709 13:20:27.295260 19800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a323eb80c5084159803e7d4ea5d9c410" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
I20260709 13:20:27.296031 19945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a323eb80c5084159803e7d4ea5d9c410" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:27.279762 20040 raft_consensus.cc:3060] T a36c11f368f34bf0aa4925a099ac1f91 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:27.301558 19997 ts_tablet_manager.cc:1434] T 19e83b82855d4129bb8525e6ff05316b P b80b3a62d7d549a9a5c42def46769ac4: Time spent starting tablet: real 0.023s	user 0.002s	sys 0.005s
I20260709 13:20:27.303339 19913 leader_election.cc:304] T 65e7a5222efe43e49e1fbb3a3d7cf99c P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c62716bf66c24330860573535b8b9430; no voters: b80b3a62d7d549a9a5c42def46769ac4, e90ffed3ba174ababeb21d9415d4d918
W20260709 13:20:27.308321 19766 consensus_peers.cc:597] T d1da95785dbf4ef4bde6f5e6ab35b444 P e90ffed3ba174ababeb21d9415d4d918 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:27.274150 19995 tablet_bootstrap.cc:492] T 7e3099f1888c4ed18c30523bf9ec94f5 P c62716bf66c24330860573535b8b9430: No bootstrap required, opened a new log
I20260709 13:20:27.311867 20009 raft_consensus.cc:515] T ad38a1519fb748588f5a322589274004 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:27.315272 20080 raft_consensus.cc:515] T 0587017774214f24bdfa4a3da36e367e P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:27.318840 20083 raft_consensus.cc:515] T 8bdf042a063d4596970f49730ffa04b4 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:27.329685 20083 leader_election.cc:290] T 8bdf042a063d4596970f49730ffa04b4 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:27.322374 19994 ts_tablet_manager.cc:1403] T 0587017774214f24bdfa4a3da36e367e P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.267s	user 0.042s	sys 0.038s
I20260709 13:20:27.325656 19995 ts_tablet_manager.cc:1403] T 7e3099f1888c4ed18c30523bf9ec94f5 P c62716bf66c24330860573535b8b9430: Time spent bootstrapping tablet: real 0.178s	user 0.015s	sys 0.025s
I20260709 13:20:27.329155 20080 leader_election.cc:290] T 0587017774214f24bdfa4a3da36e367e P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 election: Requested vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:27.321264 19873 raft_consensus.cc:2468] T 5b9511148896441a948b772784f62a3d P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 0.
W20260709 13:20:27.321223 19670 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:20:27.333113 19995 raft_consensus.cc:359] T 7e3099f1888c4ed18c30523bf9ec94f5 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:27.333817 19995 raft_consensus.cc:385] T 7e3099f1888c4ed18c30523bf9ec94f5 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:27.334074 19995 raft_consensus.cc:740] T 7e3099f1888c4ed18c30523bf9ec94f5 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62716bf66c24330860573535b8b9430, State: Initialized, Role: FOLLOWER
I20260709 13:20:27.334138 19994 raft_consensus.cc:359] T 0587017774214f24bdfa4a3da36e367e P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:27.334859 19994 raft_consensus.cc:385] T 0587017774214f24bdfa4a3da36e367e P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:27.335155 19994 raft_consensus.cc:740] T 0587017774214f24bdfa4a3da36e367e P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:27.325923 19945 raft_consensus.cc:3060] T a323eb80c5084159803e7d4ea5d9c410 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:27.326812 20035 raft_consensus.cc:3060] T 65e7a5222efe43e49e1fbb3a3d7cf99c P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:27.335880 19994 consensus_queue.cc:260] T 0587017774214f24bdfa4a3da36e367e P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:27.337985 19994 ts_tablet_manager.cc:1434] T 0587017774214f24bdfa4a3da36e367e P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 13:20:27.322803 19993 tablet_bootstrap.cc:654] T e9369d9b7f5649be932967d24689a936 P e90ffed3ba174ababeb21d9415d4d918: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:27.334968 19995 consensus_queue.cc:260] T 7e3099f1888c4ed18c30523bf9ec94f5 P c62716bf66c24330860573535b8b9430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:27.339270 19994 tablet_bootstrap.cc:492] T 7e3a9d11df99411d8f0edb0df6f14d87 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:27.326442 19997 tablet_bootstrap.cc:492] T 7e3099f1888c4ed18c30523bf9ec94f5 P b80b3a62d7d549a9a5c42def46769ac4: Bootstrap starting.
I20260709 13:20:27.323139 20181 raft_consensus.cc:2804] T ad38a1519fb748588f5a322589274004 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:27.341745 20181 raft_consensus.cc:493] T ad38a1519fb748588f5a322589274004 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:27.343954 20181 raft_consensus.cc:3060] T ad38a1519fb748588f5a322589274004 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:27.344089 20173 leader_election.cc:290] T 5b9511148896441a948b772784f62a3d P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 election: Requested vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
W20260709 13:20:27.341792 19690 leader_election.cc:343] T a323eb80c5084159803e7d4ea5d9c410 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:27.349504 19871 raft_consensus.cc:2393] T ad38a1519fb748588f5a322589274004 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8df38c4c53c44e94b36c1e8992476d7c in current term 1: Already voted for candidate b80b3a62d7d549a9a5c42def46769ac4 in this term.
I20260709 13:20:27.350471 19946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0587017774214f24bdfa4a3da36e367e" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:27.351199 19946 raft_consensus.cc:3060] T 0587017774214f24bdfa4a3da36e367e P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:27.351135 19945 raft_consensus.cc:2468] T a323eb80c5084159803e7d4ea5d9c410 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 1.
I20260709 13:20:27.351991 19800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bdf042a063d4596970f49730ffa04b4" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:27.352902 19871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bdf042a063d4596970f49730ffa04b4" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
W20260709 13:20:27.353618 19690 leader_election.cc:343] T 8bdf042a063d4596970f49730ffa04b4 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:27.354352 19690 leader_election.cc:304] T a323eb80c5084159803e7d4ea5d9c410 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, c62716bf66c24330860573535b8b9430; no voters: e90ffed3ba174ababeb21d9415d4d918
W20260709 13:20:27.356019 19690 leader_election.cc:343] T 8bdf042a063d4596970f49730ffa04b4 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:27.360381 19995 ts_tablet_manager.cc:1434] T 7e3099f1888c4ed18c30523bf9ec94f5 P c62716bf66c24330860573535b8b9430: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.000s
I20260709 13:20:27.361009 19723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0587017774214f24bdfa4a3da36e367e" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
I20260709 13:20:27.361569 19995 tablet_bootstrap.cc:492] T e9369d9b7f5649be932967d24689a936 P c62716bf66c24330860573535b8b9430: Bootstrap starting.
I20260709 13:20:27.363313 20181 raft_consensus.cc:515] T ad38a1519fb748588f5a322589274004 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:27.358556 20083 raft_consensus.cc:2804] T a323eb80c5084159803e7d4ea5d9c410 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:27.365033 20083 raft_consensus.cc:697] T a323eb80c5084159803e7d4ea5d9c410 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 LEADER]: Becoming Leader. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Running, Role: LEADER
I20260709 13:20:27.365927 20083 consensus_queue.cc:237] T a323eb80c5084159803e7d4ea5d9c410 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:27.368185 19995 tablet_bootstrap.cc:654] T e9369d9b7f5649be932967d24689a936 P c62716bf66c24330860573535b8b9430: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:27.368937 20181 leader_election.cc:290] T ad38a1519fb748588f5a322589274004 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Requested vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:27.361635 19723 raft_consensus.cc:3060] T 0587017774214f24bdfa4a3da36e367e P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:27.379868 19871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9511148896441a948b772784f62a3d" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
I20260709 13:20:27.380548 19871 raft_consensus.cc:3060] T 5b9511148896441a948b772784f62a3d P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:27.377523 20040 raft_consensus.cc:515] T a36c11f368f34bf0aa4925a099ac1f91 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:27.366681 20035 raft_consensus.cc:2749] T 65e7a5222efe43e49e1fbb3a3d7cf99c P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:27.388872 20009 leader_election.cc:290] T ad38a1519fb748588f5a322589274004 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 election: Requested vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:27.389464 19799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad38a1519fb748588f5a322589274004" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
I20260709 13:20:27.403820 19997 tablet_bootstrap.cc:654] T 7e3099f1888c4ed18c30523bf9ec94f5 P b80b3a62d7d549a9a5c42def46769ac4: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:27.392424 19722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9511148896441a948b772784f62a3d" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
I20260709 13:20:27.404644 19873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a36c11f368f34bf0aa4925a099ac1f91" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
I20260709 13:20:27.404999 19722 raft_consensus.cc:3060] T 5b9511148896441a948b772784f62a3d P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:27.405453 19873 raft_consensus.cc:3060] T a36c11f368f34bf0aa4925a099ac1f91 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:27.389748 20040 leader_election.cc:290] T a36c11f368f34bf0aa4925a099ac1f91 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 election: Requested vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:27.403766 19799 raft_consensus.cc:3060] T ad38a1519fb748588f5a322589274004 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:27.420210 19945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a36c11f368f34bf0aa4925a099ac1f91" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:27.382529 19946 raft_consensus.cc:2468] T 0587017774214f24bdfa4a3da36e367e P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 1.
W20260709 13:20:27.422925 19765 leader_election.cc:343] T a36c11f368f34bf0aa4925a099ac1f91 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c62716bf66c24330860573535b8b9430 (127.9.116.132:42643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:27.424881 19839 leader_election.cc:304] T 0587017774214f24bdfa4a3da36e367e P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430; no voters: 
I20260709 13:20:27.426301 19994 tablet_bootstrap.cc:654] T 7e3a9d11df99411d8f0edb0df6f14d87 P 8df38c4c53c44e94b36c1e8992476d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:27.425957 20009 raft_consensus.cc:2804] T 0587017774214f24bdfa4a3da36e367e P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:27.428160 20009 raft_consensus.cc:697] T 0587017774214f24bdfa4a3da36e367e P b80b3a62d7d549a9a5c42def46769ac4 [term 1 LEADER]: Becoming Leader. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Running, Role: LEADER
I20260709 13:20:27.429047 20009 consensus_queue.cc:237] T 0587017774214f24bdfa4a3da36e367e P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:27.430074 19723 raft_consensus.cc:2468] T 0587017774214f24bdfa4a3da36e367e P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 1.
I20260709 13:20:27.359719 19690 leader_election.cc:304] T 8bdf042a063d4596970f49730ffa04b4 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c; no voters: b80b3a62d7d549a9a5c42def46769ac4, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:27.439496 20181 raft_consensus.cc:2749] T 8bdf042a063d4596970f49730ffa04b4 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:27.442292 19873 raft_consensus.cc:2468] T a36c11f368f34bf0aa4925a099ac1f91 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 1.
W20260709 13:20:27.443085 19690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45249, user_credentials={real_user=slave}} blocked reactor thread for 87077.8us
I20260709 13:20:27.458572 19872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad38a1519fb748588f5a322589274004" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
I20260709 13:20:27.459138 19874 raft_consensus.cc:3060] T 19e83b82855d4129bb8525e6ff05316b P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:27.459452 19872 raft_consensus.cc:2393] T ad38a1519fb748588f5a322589274004 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8df38c4c53c44e94b36c1e8992476d7c in current term 1: Already voted for candidate b80b3a62d7d549a9a5c42def46769ac4 in this term.
I20260709 13:20:27.459949 19799 raft_consensus.cc:2468] T ad38a1519fb748588f5a322589274004 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 1.
I20260709 13:20:27.461048 19800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad38a1519fb748588f5a322589274004" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
I20260709 13:20:27.461284 19839 leader_election.cc:304] T ad38a1519fb748588f5a322589274004 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, e90ffed3ba174ababeb21d9415d4d918; no voters: 
I20260709 13:20:27.461889 19800 raft_consensus.cc:2393] T ad38a1519fb748588f5a322589274004 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8df38c4c53c44e94b36c1e8992476d7c in current term 1: Already voted for candidate b80b3a62d7d549a9a5c42def46769ac4 in this term.
I20260709 13:20:27.462222 20080 raft_consensus.cc:2804] T ad38a1519fb748588f5a322589274004 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:27.462638 19871 raft_consensus.cc:2468] T 5b9511148896441a948b772784f62a3d P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 1.
I20260709 13:20:27.463641 20080 raft_consensus.cc:697] T ad38a1519fb748588f5a322589274004 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 LEADER]: Becoming Leader. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Running, Role: LEADER
I20260709 13:20:27.464587 20080 consensus_queue.cc:237] T ad38a1519fb748588f5a322589274004 P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:27.466637 19722 raft_consensus.cc:2468] T 5b9511148896441a948b772784f62a3d P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 1.
I20260709 13:20:27.467528 19725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad38a1519fb748588f5a322589274004" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
I20260709 13:20:27.467792 19995 tablet_bootstrap.cc:492] T e9369d9b7f5649be932967d24689a936 P c62716bf66c24330860573535b8b9430: No bootstrap required, opened a new log
I20260709 13:20:27.468425 19995 ts_tablet_manager.cc:1403] T e9369d9b7f5649be932967d24689a936 P c62716bf66c24330860573535b8b9430: Time spent bootstrapping tablet: real 0.107s	user 0.033s	sys 0.041s
I20260709 13:20:27.469296 19725 raft_consensus.cc:2393] T ad38a1519fb748588f5a322589274004 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b80b3a62d7d549a9a5c42def46769ac4 in current term 1: Already voted for candidate 8df38c4c53c44e94b36c1e8992476d7c in this term.
I20260709 13:20:27.470669 19766 leader_election.cc:304] T 5b9511148896441a948b772784f62a3d P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918; no voters: 
I20260709 13:20:27.471449 19995 raft_consensus.cc:359] T e9369d9b7f5649be932967d24689a936 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:27.471976 20173 raft_consensus.cc:2804] T 5b9511148896441a948b772784f62a3d P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:27.472177 19995 raft_consensus.cc:385] T e9369d9b7f5649be932967d24689a936 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:27.472568 19995 raft_consensus.cc:740] T e9369d9b7f5649be932967d24689a936 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62716bf66c24330860573535b8b9430, State: Initialized, Role: FOLLOWER
I20260709 13:20:27.472584 20173 raft_consensus.cc:697] T 5b9511148896441a948b772784f62a3d P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Becoming Leader. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Running, Role: LEADER
I20260709 13:20:27.473311 19995 consensus_queue.cc:260] T e9369d9b7f5649be932967d24689a936 P c62716bf66c24330860573535b8b9430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:27.473662 20173 consensus_queue.cc:237] T 5b9511148896441a948b772784f62a3d P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:27.475379 19995 ts_tablet_manager.cc:1434] T e9369d9b7f5649be932967d24689a936 P c62716bf66c24330860573535b8b9430: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:20:27.476351 19995 tablet_bootstrap.cc:492] T 85d493223fe04c939b6effd840955727 P c62716bf66c24330860573535b8b9430: Bootstrap starting.
I20260709 13:20:27.488389 19690 leader_election.cc:304] T ad38a1519fb748588f5a322589274004 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c; no voters: b80b3a62d7d549a9a5c42def46769ac4, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:27.489074 20161 consensus_queue.cc:1048] T 65e7a5222efe43e49e1fbb3a3d7cf99c P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:27.491321 20181 raft_consensus.cc:2749] T ad38a1519fb748588f5a322589274004 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:20:27.493157 20065 consensus_queue.cc:1048] T d1da95785dbf4ef4bde6f5e6ab35b444 P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:27.495091 19995 tablet_bootstrap.cc:654] T 85d493223fe04c939b6effd840955727 P c62716bf66c24330860573535b8b9430: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:27.495249 20065 consensus_queue.cc:1048] T 65e7a5222efe43e49e1fbb3a3d7cf99c P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:27.487668 19619 catalog_manager.cc:5697] T a323eb80c5084159803e7d4ea5d9c410 P 8df38c4c53c44e94b36c1e8992476d7c reported cstate change: term changed from 0 to 1, leader changed from <none> to 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129). New cstate: current_term: 1 leader_uuid: "8df38c4c53c44e94b36c1e8992476d7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:27.506700 19765 leader_election.cc:304] T a36c11f368f34bf0aa4925a099ac1f91 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, e90ffed3ba174ababeb21d9415d4d918; no voters: c62716bf66c24330860573535b8b9430
I20260709 13:20:27.507719 20161 raft_consensus.cc:2804] T a36c11f368f34bf0aa4925a099ac1f91 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:27.508306 19997 tablet_bootstrap.cc:492] T 7e3099f1888c4ed18c30523bf9ec94f5 P b80b3a62d7d549a9a5c42def46769ac4: No bootstrap required, opened a new log
I20260709 13:20:27.508473 20161 raft_consensus.cc:697] T a36c11f368f34bf0aa4925a099ac1f91 P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Becoming Leader. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Running, Role: LEADER
I20260709 13:20:27.508834 19997 ts_tablet_manager.cc:1403] T 7e3099f1888c4ed18c30523bf9ec94f5 P b80b3a62d7d549a9a5c42def46769ac4: Time spent bootstrapping tablet: real 0.184s	user 0.019s	sys 0.009s
I20260709 13:20:27.509640 20161 consensus_queue.cc:237] T a36c11f368f34bf0aa4925a099ac1f91 P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:27.518714 19993 tablet_bootstrap.cc:492] T e9369d9b7f5649be932967d24689a936 P e90ffed3ba174ababeb21d9415d4d918: No bootstrap required, opened a new log
I20260709 13:20:27.520298 19993 ts_tablet_manager.cc:1403] T e9369d9b7f5649be932967d24689a936 P e90ffed3ba174ababeb21d9415d4d918: Time spent bootstrapping tablet: real 0.232s	user 0.021s	sys 0.003s
I20260709 13:20:27.522969 19997 raft_consensus.cc:359] T 7e3099f1888c4ed18c30523bf9ec94f5 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:27.523752 19997 raft_consensus.cc:385] T 7e3099f1888c4ed18c30523bf9ec94f5 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:27.524091 19997 raft_consensus.cc:740] T 7e3099f1888c4ed18c30523bf9ec94f5 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Initialized, Role: FOLLOWER
I20260709 13:20:27.525799 19997 consensus_queue.cc:260] T 7e3099f1888c4ed18c30523bf9ec94f5 P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:27.505193 20181 consensus_queue.cc:1048] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:20:27.549858 20037 raft_consensus.cc:493] T 7e3099f1888c4ed18c30523bf9ec94f5 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:27.550714 20037 raft_consensus.cc:515] T 7e3099f1888c4ed18c30523bf9ec94f5 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:27.557976 20037 leader_election.cc:290] T 7e3099f1888c4ed18c30523bf9ec94f5 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:27.561009 19944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3099f1888c4ed18c30523bf9ec94f5" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:27.562034 19944 raft_consensus.cc:2468] T 7e3099f1888c4ed18c30523bf9ec94f5 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 0.
I20260709 13:20:27.566825 19839 leader_election.cc:304] T 7e3099f1888c4ed18c30523bf9ec94f5 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430; no voters: 
I20260709 13:20:27.571041 20037 raft_consensus.cc:2804] T 7e3099f1888c4ed18c30523bf9ec94f5 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:27.575033 20037 raft_consensus.cc:493] T 7e3099f1888c4ed18c30523bf9ec94f5 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:27.576221 20037 raft_consensus.cc:3060] T 7e3099f1888c4ed18c30523bf9ec94f5 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:27.581586 20009 raft_consensus.cc:493] T 19e83b82855d4129bb8525e6ff05316b P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:27.582398 20009 raft_consensus.cc:515] T 19e83b82855d4129bb8525e6ff05316b P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:27.581780 19993 raft_consensus.cc:359] T e9369d9b7f5649be932967d24689a936 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:27.585199 19993 raft_consensus.cc:385] T e9369d9b7f5649be932967d24689a936 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:27.541858 20083 consensus_queue.cc:1048] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.044s
I20260709 13:20:27.586695 19994 tablet_bootstrap.cc:492] T 7e3a9d11df99411d8f0edb0df6f14d87 P 8df38c4c53c44e94b36c1e8992476d7c: No bootstrap required, opened a new log
I20260709 13:20:27.594697 19993 raft_consensus.cc:740] T e9369d9b7f5649be932967d24689a936 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Initialized, Role: FOLLOWER
I20260709 13:20:27.589872 19997 ts_tablet_manager.cc:1434] T 7e3099f1888c4ed18c30523bf9ec94f5 P b80b3a62d7d549a9a5c42def46769ac4: Time spent starting tablet: real 0.081s	user 0.008s	sys 0.000s
I20260709 13:20:27.595187 19994 ts_tablet_manager.cc:1403] T 7e3a9d11df99411d8f0edb0df6f14d87 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.256s	user 0.015s	sys 0.001s
I20260709 13:20:27.596184 19997 tablet_bootstrap.cc:492] T c393f9d18341477c97fac9a4af7e3adf P b80b3a62d7d549a9a5c42def46769ac4: Bootstrap starting.
I20260709 13:20:27.591768 20065 raft_consensus.cc:493] T 9b9137146e3644a6b0026778530fa8a6 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:27.602458 19993 consensus_queue.cc:260] T e9369d9b7f5649be932967d24689a936 P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:27.608351 19723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3099f1888c4ed18c30523bf9ec94f5" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:27.608796 19993 ts_tablet_manager.cc:1434] T e9369d9b7f5649be932967d24689a936 P e90ffed3ba174ababeb21d9415d4d918: Time spent starting tablet: real 0.088s	user 0.009s	sys 0.000s
W20260709 13:20:27.609980 19840 leader_election.cc:343] T 7e3099f1888c4ed18c30523bf9ec94f5 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:27.610210 19993 tablet_bootstrap.cc:492] T a323eb80c5084159803e7d4ea5d9c410 P e90ffed3ba174ababeb21d9415d4d918: Bootstrap starting.
I20260709 13:20:27.611804 19994 raft_consensus.cc:359] T 7e3a9d11df99411d8f0edb0df6f14d87 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:27.612547 19994 raft_consensus.cc:385] T 7e3a9d11df99411d8f0edb0df6f14d87 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:27.612854 19994 raft_consensus.cc:740] T 7e3a9d11df99411d8f0edb0df6f14d87 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:27.617375 19995 tablet_bootstrap.cc:492] T 85d493223fe04c939b6effd840955727 P c62716bf66c24330860573535b8b9430: No bootstrap required, opened a new log
I20260709 13:20:27.617909 19995 ts_tablet_manager.cc:1403] T 85d493223fe04c939b6effd840955727 P c62716bf66c24330860573535b8b9430: Time spent bootstrapping tablet: real 0.142s	user 0.008s	sys 0.016s
I20260709 13:20:27.618630 19947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19e83b82855d4129bb8525e6ff05316b" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:27.619455 19947 raft_consensus.cc:2468] T 19e83b82855d4129bb8525e6ff05316b P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 1.
I20260709 13:20:27.604571 20065 raft_consensus.cc:515] T 9b9137146e3644a6b0026778530fa8a6 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:27.624256 19997 tablet_bootstrap.cc:654] T c393f9d18341477c97fac9a4af7e3adf P b80b3a62d7d549a9a5c42def46769ac4: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:27.624687 20009 leader_election.cc:290] T 19e83b82855d4129bb8525e6ff05316b P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:27.627079 19723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19e83b82855d4129bb8525e6ff05316b" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:27.627562 19839 leader_election.cc:304] T 19e83b82855d4129bb8525e6ff05316b P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430; no voters: 
I20260709 13:20:27.628644 20065 leader_election.cc:290] T 9b9137146e3644a6b0026778530fa8a6 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:27.630252 19723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9137146e3644a6b0026778530fa8a6" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
W20260709 13:20:27.632004 19766 leader_election.cc:343] T 9b9137146e3644a6b0026778530fa8a6 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:27.635545 19995 raft_consensus.cc:359] T 85d493223fe04c939b6effd840955727 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:27.636343 19995 raft_consensus.cc:385] T 85d493223fe04c939b6effd840955727 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:27.636618 19995 raft_consensus.cc:740] T 85d493223fe04c939b6effd840955727 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62716bf66c24330860573535b8b9430, State: Initialized, Role: FOLLOWER
I20260709 13:20:27.615947 19994 consensus_queue.cc:260] T 7e3a9d11df99411d8f0edb0df6f14d87 P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:27.639235 19994 ts_tablet_manager.cc:1434] T 7e3a9d11df99411d8f0edb0df6f14d87 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.044s	user 0.007s	sys 0.000s
I20260709 13:20:27.640218 19994 tablet_bootstrap.cc:492] T d1da95785dbf4ef4bde6f5e6ab35b444 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:27.641098 19995 consensus_queue.cc:260] T 85d493223fe04c939b6effd840955727 P c62716bf66c24330860573535b8b9430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:27.650983 19995 ts_tablet_manager.cc:1434] T 85d493223fe04c939b6effd840955727 P c62716bf66c24330860573535b8b9430: Time spent starting tablet: real 0.033s	user 0.008s	sys 0.000s
I20260709 13:20:27.652010 19995 tablet_bootstrap.cc:492] T 0c709292a0304b3c84a777427a754bb5 P c62716bf66c24330860573535b8b9430: Bootstrap starting.
I20260709 13:20:27.658767 19995 tablet_bootstrap.cc:654] T 0c709292a0304b3c84a777427a754bb5 P c62716bf66c24330860573535b8b9430: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:27.659089 20080 raft_consensus.cc:2804] T 19e83b82855d4129bb8525e6ff05316b P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:20:27.662958 19994 tablet_bootstrap.cc:654] T d1da95785dbf4ef4bde6f5e6ab35b444 P 8df38c4c53c44e94b36c1e8992476d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:27.661725 20080 raft_consensus.cc:493] T 19e83b82855d4129bb8525e6ff05316b P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:27.671294 20080 raft_consensus.cc:3060] T 19e83b82855d4129bb8525e6ff05316b P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:27.674222 19946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9137146e3644a6b0026778530fa8a6" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:27.675138 19946 raft_consensus.cc:2468] T 9b9137146e3644a6b0026778530fa8a6 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 0.
I20260709 13:20:27.685473 19993 tablet_bootstrap.cc:654] T a323eb80c5084159803e7d4ea5d9c410 P e90ffed3ba174ababeb21d9415d4d918: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:27.697678 19765 leader_election.cc:304] T 9b9137146e3644a6b0026778530fa8a6 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c62716bf66c24330860573535b8b9430, e90ffed3ba174ababeb21d9415d4d918; no voters: 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:27.699127 20173 raft_consensus.cc:2804] T 9b9137146e3644a6b0026778530fa8a6 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:27.699589 20173 raft_consensus.cc:493] T 9b9137146e3644a6b0026778530fa8a6 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:27.700003 20173 raft_consensus.cc:3060] T 9b9137146e3644a6b0026778530fa8a6 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:27.714094 19995 tablet_bootstrap.cc:492] T 0c709292a0304b3c84a777427a754bb5 P c62716bf66c24330860573535b8b9430: No bootstrap required, opened a new log
I20260709 13:20:27.715008 19995 ts_tablet_manager.cc:1403] T 0c709292a0304b3c84a777427a754bb5 P c62716bf66c24330860573535b8b9430: Time spent bootstrapping tablet: real 0.063s	user 0.018s	sys 0.007s
I20260709 13:20:27.720654 19621 catalog_manager.cc:5697] T 5b9511148896441a948b772784f62a3d P e90ffed3ba174ababeb21d9415d4d918 reported cstate change: term changed from 0 to 1, leader changed from <none> to e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "e90ffed3ba174ababeb21d9415d4d918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:27.723619 19995 raft_consensus.cc:359] T 0c709292a0304b3c84a777427a754bb5 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:27.724722 19995 raft_consensus.cc:385] T 0c709292a0304b3c84a777427a754bb5 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:27.725077 19995 raft_consensus.cc:740] T 0c709292a0304b3c84a777427a754bb5 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62716bf66c24330860573535b8b9430, State: Initialized, Role: FOLLOWER
I20260709 13:20:27.727303 19621 catalog_manager.cc:5697] T a36c11f368f34bf0aa4925a099ac1f91 P e90ffed3ba174ababeb21d9415d4d918 reported cstate change: term changed from 0 to 1, leader changed from <none> to e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "e90ffed3ba174ababeb21d9415d4d918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:27.734277 19995 consensus_queue.cc:260] T 0c709292a0304b3c84a777427a754bb5 P c62716bf66c24330860573535b8b9430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:27.738215 20173 raft_consensus.cc:515] T 9b9137146e3644a6b0026778530fa8a6 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:27.740037 19994 tablet_bootstrap.cc:492] T d1da95785dbf4ef4bde6f5e6ab35b444 P 8df38c4c53c44e94b36c1e8992476d7c: No bootstrap required, opened a new log
I20260709 13:20:27.740279 19995 ts_tablet_manager.cc:1434] T 0c709292a0304b3c84a777427a754bb5 P c62716bf66c24330860573535b8b9430: Time spent starting tablet: real 0.025s	user 0.005s	sys 0.002s
I20260709 13:20:27.740593 19994 ts_tablet_manager.cc:1403] T d1da95785dbf4ef4bde6f5e6ab35b444 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.101s	user 0.013s	sys 0.008s
I20260709 13:20:27.741338 19995 tablet_bootstrap.cc:492] T a36c11f368f34bf0aa4925a099ac1f91 P c62716bf66c24330860573535b8b9430: Bootstrap starting.
I20260709 13:20:27.743511 19994 raft_consensus.cc:359] T d1da95785dbf4ef4bde6f5e6ab35b444 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:27.744225 19994 raft_consensus.cc:385] T d1da95785dbf4ef4bde6f5e6ab35b444 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:27.744519 19994 raft_consensus.cc:740] T d1da95785dbf4ef4bde6f5e6ab35b444 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:27.745450 20173 leader_election.cc:290] T 9b9137146e3644a6b0026778530fa8a6 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 election: Requested vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:27.748238 19995 tablet_bootstrap.cc:654] T a36c11f368f34bf0aa4925a099ac1f91 P c62716bf66c24330860573535b8b9430: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:27.748555 19994 consensus_queue.cc:260] T d1da95785dbf4ef4bde6f5e6ab35b444 P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:27.750224 19994 ts_tablet_manager.cc:1434] T d1da95785dbf4ef4bde6f5e6ab35b444 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:20:27.751338 19994 tablet_bootstrap.cc:492] T 9b9137146e3644a6b0026778530fa8a6 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:27.755026 19944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9137146e3644a6b0026778530fa8a6" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:27.755477 19723 raft_consensus.cc:3060] T d1da95785dbf4ef4bde6f5e6ab35b444 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:27.754336 20037 raft_consensus.cc:515] T 7e3099f1888c4ed18c30523bf9ec94f5 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:27.755764 19944 raft_consensus.cc:3060] T 9b9137146e3644a6b0026778530fa8a6 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:27.757079 19997 tablet_bootstrap.cc:492] T c393f9d18341477c97fac9a4af7e3adf P b80b3a62d7d549a9a5c42def46769ac4: No bootstrap required, opened a new log
I20260709 13:20:27.757637 19997 ts_tablet_manager.cc:1403] T c393f9d18341477c97fac9a4af7e3adf P b80b3a62d7d549a9a5c42def46769ac4: Time spent bootstrapping tablet: real 0.162s	user 0.021s	sys 0.008s
I20260709 13:20:27.758611 20037 leader_election.cc:290] T 7e3099f1888c4ed18c30523bf9ec94f5 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 election: Requested vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:27.759187 19725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9137146e3644a6b0026778530fa8a6" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
I20260709 13:20:27.761215 19997 raft_consensus.cc:359] T c393f9d18341477c97fac9a4af7e3adf P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:27.762532 19946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3099f1888c4ed18c30523bf9ec94f5" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:27.763273 19946 raft_consensus.cc:3060] T 7e3099f1888c4ed18c30523bf9ec94f5 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:27.733778 20080 raft_consensus.cc:515] T 19e83b82855d4129bb8525e6ff05316b P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:27.766891 19871 raft_consensus.cc:1240] T 19e83b82855d4129bb8525e6ff05316b P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Rejecting Update request from peer 8df38c4c53c44e94b36c1e8992476d7c for earlier term 1. Current term is 2. Ops: []
W20260709 13:20:27.767863 19766 leader_election.cc:343] T 9b9137146e3644a6b0026778530fa8a6 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:27.769460 19722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3099f1888c4ed18c30523bf9ec94f5" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
I20260709 13:20:27.762630 19997 raft_consensus.cc:385] T c393f9d18341477c97fac9a4af7e3adf P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:27.770112 19997 raft_consensus.cc:740] T c393f9d18341477c97fac9a4af7e3adf P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Initialized, Role: FOLLOWER
I20260709 13:20:27.770947 20031 consensus_queue.cc:1059] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:20:27.773527 20031 raft_consensus.cc:3055] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:20:27.773976 20031 raft_consensus.cc:740] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Running, Role: LEADER
I20260709 13:20:27.774566 19946 raft_consensus.cc:2468] T 7e3099f1888c4ed18c30523bf9ec94f5 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 1.
I20260709 13:20:27.775146 20031 consensus_queue.cc:260] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:27.776175 19997 consensus_queue.cc:260] T c393f9d18341477c97fac9a4af7e3adf P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:27.776921 20031 raft_consensus.cc:3060] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:27.778330 19997 ts_tablet_manager.cc:1434] T c393f9d18341477c97fac9a4af7e3adf P b80b3a62d7d549a9a5c42def46769ac4: Time spent starting tablet: real 0.020s	user 0.009s	sys 0.000s
I20260709 13:20:27.779274 19997 tablet_bootstrap.cc:492] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4: Bootstrap starting.
W20260709 13:20:27.772028 19840 leader_election.cc:343] T 7e3099f1888c4ed18c30523bf9ec94f5 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:27.782400 19839 leader_election.cc:304] T 7e3099f1888c4ed18c30523bf9ec94f5 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:27.783694 20037 raft_consensus.cc:2804] T 7e3099f1888c4ed18c30523bf9ec94f5 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:27.784365 20037 raft_consensus.cc:697] T 7e3099f1888c4ed18c30523bf9ec94f5 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 LEADER]: Becoming Leader. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Running, Role: LEADER
I20260709 13:20:27.785351 20037 consensus_queue.cc:237] T 7e3099f1888c4ed18c30523bf9ec94f5 P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:27.772250 19944 raft_consensus.cc:2468] T 9b9137146e3644a6b0026778530fa8a6 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 1.
I20260709 13:20:27.795444 19994 tablet_bootstrap.cc:654] T 9b9137146e3644a6b0026778530fa8a6 P 8df38c4c53c44e94b36c1e8992476d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:27.800999 19765 leader_election.cc:304] T 9b9137146e3644a6b0026778530fa8a6 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c62716bf66c24330860573535b8b9430, e90ffed3ba174ababeb21d9415d4d918; no voters: 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:27.803504 20209 raft_consensus.cc:2804] T 9b9137146e3644a6b0026778530fa8a6 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:27.804052 20209 raft_consensus.cc:697] T 9b9137146e3644a6b0026778530fa8a6 P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Becoming Leader. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Running, Role: LEADER
I20260709 13:20:27.804924 20209 consensus_queue.cc:237] T 9b9137146e3644a6b0026778530fa8a6 P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:27.807125 19622 catalog_manager.cc:5697] T 0587017774214f24bdfa4a3da36e367e P b80b3a62d7d549a9a5c42def46769ac4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131). New cstate: current_term: 1 leader_uuid: "b80b3a62d7d549a9a5c42def46769ac4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:27.818168 19997 tablet_bootstrap.cc:654] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:27.820057 20083 raft_consensus.cc:493] T 7e3a9d11df99411d8f0edb0df6f14d87 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:27.822445 19722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19e83b82855d4129bb8525e6ff05316b" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
I20260709 13:20:27.824676 19944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19e83b82855d4129bb8525e6ff05316b" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:27.825627 20198 raft_consensus.cc:493] T 0c709292a0304b3c84a777427a754bb5 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:27.840651 20083 raft_consensus.cc:515] T 7e3a9d11df99411d8f0edb0df6f14d87 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:27.841008 20198 raft_consensus.cc:515] T 0c709292a0304b3c84a777427a754bb5 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:27.843137 20083 leader_election.cc:290] T 7e3a9d11df99411d8f0edb0df6f14d87 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:27.843236 20198 leader_election.cc:290] T 0c709292a0304b3c84a777427a754bb5 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:27.845162 19800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c709292a0304b3c84a777427a754bb5" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
W20260709 13:20:27.846896 19913 leader_election.cc:343] T 0c709292a0304b3c84a777427a754bb5 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:27.848537 19871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c709292a0304b3c84a777427a754bb5" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
W20260709 13:20:27.849764 19913 leader_election.cc:343] T 0c709292a0304b3c84a777427a754bb5 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:27.850221 19913 leader_election.cc:304] T 0c709292a0304b3c84a777427a754bb5 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c62716bf66c24330860573535b8b9430; no voters: b80b3a62d7d549a9a5c42def46769ac4, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:27.851436 20198 raft_consensus.cc:493] T e9369d9b7f5649be932967d24689a936 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:27.851692 19995 tablet_bootstrap.cc:492] T a36c11f368f34bf0aa4925a099ac1f91 P c62716bf66c24330860573535b8b9430: No bootstrap required, opened a new log
I20260709 13:20:27.852209 19995 ts_tablet_manager.cc:1403] T a36c11f368f34bf0aa4925a099ac1f91 P c62716bf66c24330860573535b8b9430: Time spent bootstrapping tablet: real 0.111s	user 0.043s	sys 0.039s
I20260709 13:20:27.851958 20198 raft_consensus.cc:515] T e9369d9b7f5649be932967d24689a936 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:27.854172 20198 leader_election.cc:290] T e9369d9b7f5649be932967d24689a936 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:27.851436 20000 raft_consensus.cc:2749] T 0c709292a0304b3c84a777427a754bb5 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:27.855202 19994 tablet_bootstrap.cc:492] T 9b9137146e3644a6b0026778530fa8a6 P 8df38c4c53c44e94b36c1e8992476d7c: No bootstrap required, opened a new log
I20260709 13:20:27.855185 19995 raft_consensus.cc:359] T a36c11f368f34bf0aa4925a099ac1f91 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:27.855863 19994 ts_tablet_manager.cc:1403] T 9b9137146e3644a6b0026778530fa8a6 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.105s	user 0.011s	sys 0.003s
I20260709 13:20:27.855993 19995 raft_consensus.cc:385] T a36c11f368f34bf0aa4925a099ac1f91 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:27.856355 19995 raft_consensus.cc:740] T a36c11f368f34bf0aa4925a099ac1f91 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62716bf66c24330860573535b8b9430, State: Initialized, Role: FOLLOWER
I20260709 13:20:27.857030 19995 consensus_queue.cc:260] T a36c11f368f34bf0aa4925a099ac1f91 P c62716bf66c24330860573535b8b9430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:27.858027 19800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9369d9b7f5649be932967d24689a936" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:27.858834 19800 raft_consensus.cc:2468] T e9369d9b7f5649be932967d24689a936 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c62716bf66c24330860573535b8b9430 in term 0.
I20260709 13:20:27.859280 19995 ts_tablet_manager.cc:1434] T a36c11f368f34bf0aa4925a099ac1f91 P c62716bf66c24330860573535b8b9430: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:20:27.860010 19994 raft_consensus.cc:359] T 9b9137146e3644a6b0026778530fa8a6 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:27.860245 19995 tablet_bootstrap.cc:492] T 0413f22bd6d7447ca630b75be496555f P c62716bf66c24330860573535b8b9430: Bootstrap starting.
I20260709 13:20:27.860357 19913 leader_election.cc:304] T e9369d9b7f5649be932967d24689a936 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c62716bf66c24330860573535b8b9430, e90ffed3ba174ababeb21d9415d4d918; no voters: 
I20260709 13:20:27.861725 20000 raft_consensus.cc:2804] T e9369d9b7f5649be932967d24689a936 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:27.861791 19994 raft_consensus.cc:385] T 9b9137146e3644a6b0026778530fa8a6 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:27.862195 20000 raft_consensus.cc:493] T e9369d9b7f5649be932967d24689a936 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:27.862360 19994 raft_consensus.cc:740] T 9b9137146e3644a6b0026778530fa8a6 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:27.862901 20000 raft_consensus.cc:3060] T e9369d9b7f5649be932967d24689a936 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:27.863372 19994 consensus_queue.cc:260] T 9b9137146e3644a6b0026778530fa8a6 P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:27.867641 19995 tablet_bootstrap.cc:654] T 0413f22bd6d7447ca630b75be496555f P c62716bf66c24330860573535b8b9430: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:27.869146 19994 ts_tablet_manager.cc:1434] T 9b9137146e3644a6b0026778530fa8a6 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.013s	user 0.003s	sys 0.004s
I20260709 13:20:27.869899 20212 consensus_queue.cc:1048] T 5b9511148896441a948b772784f62a3d P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:20:27.872093 19994 tablet_bootstrap.cc:492] T 7e3099f1888c4ed18c30523bf9ec94f5 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:27.829457 19993 tablet_bootstrap.cc:492] T a323eb80c5084159803e7d4ea5d9c410 P e90ffed3ba174ababeb21d9415d4d918: No bootstrap required, opened a new log
I20260709 13:20:27.829516 20080 leader_election.cc:290] T 19e83b82855d4129bb8525e6ff05316b P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 election: Requested vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
W20260709 13:20:27.838397 19690 consensus_peers.cc:597] T a323eb80c5084159803e7d4ea5d9c410 P 8df38c4c53c44e94b36c1e8992476d7c -> Peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Couldn't send request to peer e90ffed3ba174ababeb21d9415d4d918. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:27.877642 19871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3a9d11df99411d8f0edb0df6f14d87" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:27.878450 19871 raft_consensus.cc:2468] T 7e3a9d11df99411d8f0edb0df6f14d87 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 0.
I20260709 13:20:27.879750 19995 tablet_bootstrap.cc:492] T 0413f22bd6d7447ca630b75be496555f P c62716bf66c24330860573535b8b9430: No bootstrap required, opened a new log
I20260709 13:20:27.880266 19995 ts_tablet_manager.cc:1403] T 0413f22bd6d7447ca630b75be496555f P c62716bf66c24330860573535b8b9430: Time spent bootstrapping tablet: real 0.020s	user 0.014s	sys 0.003s
I20260709 13:20:27.882314 19800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3a9d11df99411d8f0edb0df6f14d87" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:27.883128 19995 raft_consensus.cc:359] T 0413f22bd6d7447ca630b75be496555f P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:27.883855 19995 raft_consensus.cc:385] T 0413f22bd6d7447ca630b75be496555f P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:27.884184 19995 raft_consensus.cc:740] T 0413f22bd6d7447ca630b75be496555f P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62716bf66c24330860573535b8b9430, State: Initialized, Role: FOLLOWER
W20260709 13:20:27.883973 19690 leader_election.cc:343] T 7e3a9d11df99411d8f0edb0df6f14d87 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:27.885143 19995 consensus_queue.cc:260] T 0413f22bd6d7447ca630b75be496555f P c62716bf66c24330860573535b8b9430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:27.885829 19690 leader_election.cc:304] T 7e3a9d11df99411d8f0edb0df6f14d87 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4; no voters: e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:27.886663 20083 raft_consensus.cc:2804] T 7e3a9d11df99411d8f0edb0df6f14d87 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:27.887049 20083 raft_consensus.cc:493] T 7e3a9d11df99411d8f0edb0df6f14d87 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:27.887377 20083 raft_consensus.cc:3060] T 7e3a9d11df99411d8f0edb0df6f14d87 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:27.887396 19995 ts_tablet_manager.cc:1434] T 0413f22bd6d7447ca630b75be496555f P c62716bf66c24330860573535b8b9430: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:20:27.888614 19994 tablet_bootstrap.cc:654] T 7e3099f1888c4ed18c30523bf9ec94f5 P 8df38c4c53c44e94b36c1e8992476d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:27.888610 19995 tablet_bootstrap.cc:492] T e3ca37daad8341ad99cbba926f37232e P c62716bf66c24330860573535b8b9430: Bootstrap starting.
I20260709 13:20:27.896667 19995 tablet_bootstrap.cc:654] T e3ca37daad8341ad99cbba926f37232e P c62716bf66c24330860573535b8b9430: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:27.899004 20083 raft_consensus.cc:515] T 7e3a9d11df99411d8f0edb0df6f14d87 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:27.902966 19800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3a9d11df99411d8f0edb0df6f14d87" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
W20260709 13:20:27.905272 19690 leader_election.cc:343] T 7e3a9d11df99411d8f0edb0df6f14d87 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:27.906401 19997 tablet_bootstrap.cc:492] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4: No bootstrap required, opened a new log
I20260709 13:20:27.907292 19997 ts_tablet_manager.cc:1403] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4: Time spent bootstrapping tablet: real 0.128s	user 0.023s	sys 0.011s
I20260709 13:20:27.907500 19993 ts_tablet_manager.cc:1403] T a323eb80c5084159803e7d4ea5d9c410 P e90ffed3ba174ababeb21d9415d4d918: Time spent bootstrapping tablet: real 0.298s	user 0.019s	sys 0.011s
I20260709 13:20:27.908388 19995 tablet_bootstrap.cc:492] T e3ca37daad8341ad99cbba926f37232e P c62716bf66c24330860573535b8b9430: No bootstrap required, opened a new log
I20260709 13:20:27.908903 19995 ts_tablet_manager.cc:1403] T e3ca37daad8341ad99cbba926f37232e P c62716bf66c24330860573535b8b9430: Time spent bootstrapping tablet: real 0.021s	user 0.012s	sys 0.004s
I20260709 13:20:27.910857 19993 raft_consensus.cc:359] T a323eb80c5084159803e7d4ea5d9c410 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:27.911180 19997 raft_consensus.cc:359] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:27.911640 19995 raft_consensus.cc:359] T e3ca37daad8341ad99cbba926f37232e P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:27.912321 19871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9369d9b7f5649be932967d24689a936" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:27.915988 20080 consensus_queue.cc:1048] T 0587017774214f24bdfa4a3da36e367e P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:20:27.920715 20173 consensus_queue.cc:1048] T 5b9511148896441a948b772784f62a3d P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:20:27.922595 20009 consensus_queue.cc:1048] T ad38a1519fb748588f5a322589274004 P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:20:27.923276 19997 raft_consensus.cc:385] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:27.931538 19997 raft_consensus.cc:740] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Initialized, Role: FOLLOWER
I20260709 13:20:27.932350 19997 consensus_queue.cc:260] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:27.923422 19995 raft_consensus.cc:385] T e3ca37daad8341ad99cbba926f37232e P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:27.933885 19995 raft_consensus.cc:740] T e3ca37daad8341ad99cbba926f37232e P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62716bf66c24330860573535b8b9430, State: Initialized, Role: FOLLOWER
I20260709 13:20:27.934628 19995 consensus_queue.cc:260] T e3ca37daad8341ad99cbba926f37232e P c62716bf66c24330860573535b8b9430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:27.923691 19871 raft_consensus.cc:2468] T e9369d9b7f5649be932967d24689a936 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c62716bf66c24330860573535b8b9430 in term 0.
I20260709 13:20:27.937772 19995 ts_tablet_manager.cc:1434] T e3ca37daad8341ad99cbba926f37232e P c62716bf66c24330860573535b8b9430: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.000s
I20260709 13:20:27.938894 19995 tablet_bootstrap.cc:492] T 3aaced67782041fa9960c745301a96e7 P c62716bf66c24330860573535b8b9430: Bootstrap starting.
I20260709 13:20:27.944713 19871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3a9d11df99411d8f0edb0df6f14d87" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
I20260709 13:20:27.945416 19871 raft_consensus.cc:3060] T 7e3a9d11df99411d8f0edb0df6f14d87 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:27.944341 19619 catalog_manager.cc:5697] T 9b9137146e3644a6b0026778530fa8a6 P e90ffed3ba174ababeb21d9415d4d918 reported cstate change: term changed from 0 to 1, leader changed from <none> to e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "e90ffed3ba174ababeb21d9415d4d918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:27.944432 19997 ts_tablet_manager.cc:1434] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4: Time spent starting tablet: real 0.037s	user 0.002s	sys 0.004s
I20260709 13:20:27.948316 19997 tablet_bootstrap.cc:492] T 0413f22bd6d7447ca630b75be496555f P b80b3a62d7d549a9a5c42def46769ac4: Bootstrap starting.
I20260709 13:20:27.923256 19993 raft_consensus.cc:385] T a323eb80c5084159803e7d4ea5d9c410 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:27.950306 19993 raft_consensus.cc:740] T a323eb80c5084159803e7d4ea5d9c410 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Initialized, Role: FOLLOWER
I20260709 13:20:27.952652 19995 tablet_bootstrap.cc:654] T 3aaced67782041fa9960c745301a96e7 P c62716bf66c24330860573535b8b9430: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:27.871980 20000 raft_consensus.cc:515] T e9369d9b7f5649be932967d24689a936 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:27.954998 19993 consensus_queue.cc:260] T a323eb80c5084159803e7d4ea5d9c410 P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:27.956003 20198 raft_consensus.cc:493] T 85d493223fe04c939b6effd840955727 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:27.958740 20198 raft_consensus.cc:515] T 85d493223fe04c939b6effd840955727 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:27.957094 19994 tablet_bootstrap.cc:492] T 7e3099f1888c4ed18c30523bf9ec94f5 P 8df38c4c53c44e94b36c1e8992476d7c: No bootstrap required, opened a new log
I20260709 13:20:27.960186 19994 ts_tablet_manager.cc:1403] T 7e3099f1888c4ed18c30523bf9ec94f5 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.088s	user 0.034s	sys 0.023s
I20260709 13:20:27.925030 20083 leader_election.cc:290] T 7e3a9d11df99411d8f0edb0df6f14d87 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Requested vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:27.964126 19993 ts_tablet_manager.cc:1434] T a323eb80c5084159803e7d4ea5d9c410 P e90ffed3ba174ababeb21d9415d4d918: Time spent starting tablet: real 0.056s	user 0.007s	sys 0.001s
I20260709 13:20:27.967999 19993 tablet_bootstrap.cc:492] T 0c709292a0304b3c84a777427a754bb5 P e90ffed3ba174ababeb21d9415d4d918: Bootstrap starting.
I20260709 13:20:27.993692 20198 leader_election.cc:290] T 85d493223fe04c939b6effd840955727 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:27.995393 19799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d493223fe04c939b6effd840955727" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:27.985450 19945 raft_consensus.cc:3060] T 85d493223fe04c939b6effd840955727 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:28.001309 19994 raft_consensus.cc:359] T 7e3099f1888c4ed18c30523bf9ec94f5 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:28.004051 19994 raft_consensus.cc:385] T 7e3099f1888c4ed18c30523bf9ec94f5 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:28.004457 19994 raft_consensus.cc:740] T 7e3099f1888c4ed18c30523bf9ec94f5 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:28.002534 19995 tablet_bootstrap.cc:492] T 3aaced67782041fa9960c745301a96e7 P c62716bf66c24330860573535b8b9430: No bootstrap required, opened a new log
I20260709 13:20:28.005635 19995 ts_tablet_manager.cc:1403] T 3aaced67782041fa9960c745301a96e7 P c62716bf66c24330860573535b8b9430: Time spent bootstrapping tablet: real 0.067s	user 0.017s	sys 0.013s
I20260709 13:20:28.005267 19994 consensus_queue.cc:260] T 7e3099f1888c4ed18c30523bf9ec94f5 P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:28.009253 19994 ts_tablet_manager.cc:1434] T 7e3099f1888c4ed18c30523bf9ec94f5 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.049s	user 0.004s	sys 0.003s
I20260709 13:20:28.007397 20132 raft_consensus.cc:493] T d1da95785dbf4ef4bde6f5e6ab35b444 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e90ffed3ba174ababeb21d9415d4d918)
I20260709 13:20:28.020391 20132 raft_consensus.cc:515] T d1da95785dbf4ef4bde6f5e6ab35b444 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:28.025925 19994 tablet_bootstrap.cc:492] T 63868398f3604f529519d926d2831f4e P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:28.008184 20009 consensus_queue.cc:1048] T ad38a1519fb748588f5a322589274004 P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20260709 13:20:28.035197 19946 raft_consensus.cc:3060] T a36c11f368f34bf0aa4925a099ac1f91 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:28.037614 19993 tablet_bootstrap.cc:654] T 0c709292a0304b3c84a777427a754bb5 P e90ffed3ba174ababeb21d9415d4d918: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:28.042867 20181 consensus_queue.cc:1048] T a323eb80c5084159803e7d4ea5d9c410 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:20:28.027004 19871 raft_consensus.cc:2468] T 7e3a9d11df99411d8f0edb0df6f14d87 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 1.
I20260709 13:20:28.045037 19995 raft_consensus.cc:359] T 3aaced67782041fa9960c745301a96e7 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:28.045990 19995 raft_consensus.cc:385] T 3aaced67782041fa9960c745301a96e7 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:28.046299 19995 raft_consensus.cc:740] T 3aaced67782041fa9960c745301a96e7 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62716bf66c24330860573535b8b9430, State: Initialized, Role: FOLLOWER
I20260709 13:20:28.047055 19995 consensus_queue.cc:260] T 3aaced67782041fa9960c745301a96e7 P c62716bf66c24330860573535b8b9430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:28.052644 19690 leader_election.cc:304] T 7e3a9d11df99411d8f0edb0df6f14d87 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4; no voters: e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:28.057753 19995 ts_tablet_manager.cc:1434] T 3aaced67782041fa9960c745301a96e7 P c62716bf66c24330860573535b8b9430: Time spent starting tablet: real 0.052s	user 0.004s	sys 0.004s
I20260709 13:20:28.060624 19995 tablet_bootstrap.cc:492] T 63868398f3604f529519d926d2831f4e P c62716bf66c24330860573535b8b9430: Bootstrap starting.
I20260709 13:20:28.075446 20173 consensus_queue.cc:1048] T a36c11f368f34bf0aa4925a099ac1f91 P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:20:28.063910 20181 raft_consensus.cc:2804] T 7e3a9d11df99411d8f0edb0df6f14d87 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:28.079003 20181 raft_consensus.cc:697] T 7e3a9d11df99411d8f0edb0df6f14d87 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 LEADER]: Becoming Leader. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Running, Role: LEADER
I20260709 13:20:28.030993 19722 raft_consensus.cc:2468] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 2.
I20260709 13:20:28.091861 19995 tablet_bootstrap.cc:654] T 63868398f3604f529519d926d2831f4e P c62716bf66c24330860573535b8b9430: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:28.094669 19994 tablet_bootstrap.cc:654] T 63868398f3604f529519d926d2831f4e P 8df38c4c53c44e94b36c1e8992476d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:28.103271 19871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d493223fe04c939b6effd840955727" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:28.118264 20009 consensus_queue.cc:1048] T 0587017774214f24bdfa4a3da36e367e P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:20:28.120666 19997 tablet_bootstrap.cc:654] T 0413f22bd6d7447ca630b75be496555f P b80b3a62d7d549a9a5c42def46769ac4: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:28.122661 20083 raft_consensus.cc:493] T 7e3099f1888c4ed18c30523bf9ec94f5 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:28.123229 20083 raft_consensus.cc:515] T 7e3099f1888c4ed18c30523bf9ec94f5 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:28.125385 20083 leader_election.cc:290] T 7e3099f1888c4ed18c30523bf9ec94f5 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:28.129899 19993 tablet_bootstrap.cc:492] T 0c709292a0304b3c84a777427a754bb5 P e90ffed3ba174ababeb21d9415d4d918: No bootstrap required, opened a new log
I20260709 13:20:28.130471 19993 ts_tablet_manager.cc:1403] T 0c709292a0304b3c84a777427a754bb5 P e90ffed3ba174ababeb21d9415d4d918: Time spent bootstrapping tablet: real 0.163s	user 0.012s	sys 0.003s
W20260709 13:20:28.132052 20226 log.cc:927] Time spent T 9b9137146e3644a6b0026778530fa8a6 P e90ffed3ba174ababeb21d9415d4d918: Append to log took a long time: real 0.111s	user 0.001s	sys 0.001s
I20260709 13:20:28.133874 20181 consensus_queue.cc:237] T 7e3a9d11df99411d8f0edb0df6f14d87 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:28.140801 20132 leader_election.cc:290] T d1da95785dbf4ef4bde6f5e6ab35b444 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:28.143317 19874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3099f1888c4ed18c30523bf9ec94f5" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:28.107118 20000 leader_election.cc:290] T e9369d9b7f5649be932967d24689a936 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 election: Requested vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:28.140076 19993 raft_consensus.cc:359] T 0c709292a0304b3c84a777427a754bb5 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:28.145777 19943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1da95785dbf4ef4bde6f5e6ab35b444" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:28.148325 19800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9369d9b7f5649be932967d24689a936" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
I20260709 13:20:28.149060 19800 raft_consensus.cc:3060] T e9369d9b7f5649be932967d24689a936 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:28.150179 19941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3099f1888c4ed18c30523bf9ec94f5" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:28.151103 19941 raft_consensus.cc:2393] T 7e3099f1888c4ed18c30523bf9ec94f5 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8df38c4c53c44e94b36c1e8992476d7c in current term 1: Already voted for candidate b80b3a62d7d549a9a5c42def46769ac4 in this term.
I20260709 13:20:28.152243 19799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1da95785dbf4ef4bde6f5e6ab35b444" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:28.154672 19840 leader_election.cc:304] T 19e83b82855d4129bb8525e6ff05316b P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4; no voters: c62716bf66c24330860573535b8b9430
I20260709 13:20:28.158915 20009 raft_consensus.cc:2804] T 19e83b82855d4129bb8525e6ff05316b P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:20:28.159471 20009 raft_consensus.cc:697] T 19e83b82855d4129bb8525e6ff05316b P b80b3a62d7d549a9a5c42def46769ac4 [term 2 LEADER]: Becoming Leader. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Running, Role: LEADER
I20260709 13:20:28.160353 20009 consensus_queue.cc:237] T 19e83b82855d4129bb8525e6ff05316b P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:28.161446 19873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9369d9b7f5649be932967d24689a936" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
I20260709 13:20:28.162179 19873 raft_consensus.cc:3060] T e9369d9b7f5649be932967d24689a936 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:28.163837 19690 leader_election.cc:304] T 7e3099f1888c4ed18c30523bf9ec94f5 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c; no voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430
I20260709 13:20:28.147970 19993 raft_consensus.cc:385] T 0c709292a0304b3c84a777427a754bb5 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:28.165481 19993 raft_consensus.cc:740] T 0c709292a0304b3c84a777427a754bb5 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Initialized, Role: FOLLOWER
I20260709 13:20:28.167300 20031 raft_consensus.cc:3060] T 7e3099f1888c4ed18c30523bf9ec94f5 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:28.171970 19873 raft_consensus.cc:2468] T e9369d9b7f5649be932967d24689a936 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c62716bf66c24330860573535b8b9430 in term 1.
I20260709 13:20:28.172804 19993 consensus_queue.cc:260] T 0c709292a0304b3c84a777427a754bb5 P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:28.173465 19913 leader_election.cc:304] T 85d493223fe04c939b6effd840955727 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c62716bf66c24330860573535b8b9430; no voters: b80b3a62d7d549a9a5c42def46769ac4, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:28.177687 19993 ts_tablet_manager.cc:1434] T 0c709292a0304b3c84a777427a754bb5 P e90ffed3ba174ababeb21d9415d4d918: Time spent starting tablet: real 0.047s	user 0.010s	sys 0.000s
I20260709 13:20:28.177157 20031 raft_consensus.cc:2749] T 7e3099f1888c4ed18c30523bf9ec94f5 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:28.179668 19993 tablet_bootstrap.cc:492] T 8bdf042a063d4596970f49730ffa04b4 P e90ffed3ba174ababeb21d9415d4d918: Bootstrap starting.
I20260709 13:20:28.177713 19622 catalog_manager.cc:5697] T ad38a1519fb748588f5a322589274004 P b80b3a62d7d549a9a5c42def46769ac4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131). New cstate: current_term: 1 leader_uuid: "b80b3a62d7d549a9a5c42def46769ac4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:28.183609 19622 catalog_manager.cc:5697] T 7e3099f1888c4ed18c30523bf9ec94f5 P b80b3a62d7d549a9a5c42def46769ac4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131). New cstate: current_term: 1 leader_uuid: "b80b3a62d7d549a9a5c42def46769ac4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:28.188405 19690 leader_election.cc:304] T d1da95785dbf4ef4bde6f5e6ab35b444 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c; no voters: c62716bf66c24330860573535b8b9430, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:28.191210 20083 raft_consensus.cc:2749] T d1da95785dbf4ef4bde6f5e6ab35b444 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:20:28.202355 20000 raft_consensus.cc:493] T e3ca37daad8341ad99cbba926f37232e P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:28.207494 20000 raft_consensus.cc:515] T e3ca37daad8341ad99cbba926f37232e P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:28.210439 20000 leader_election.cc:290] T e3ca37daad8341ad99cbba926f37232e P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:28.212903 20173 consensus_queue.cc:1048] T a36c11f368f34bf0aa4925a099ac1f91 P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:28.208897 19995 tablet_bootstrap.cc:492] T 63868398f3604f529519d926d2831f4e P c62716bf66c24330860573535b8b9430: No bootstrap required, opened a new log
I20260709 13:20:28.217341 20132 raft_consensus.cc:493] T 9b9137146e3644a6b0026778530fa8a6 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:28.217978 20132 raft_consensus.cc:515] T 9b9137146e3644a6b0026778530fa8a6 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:28.220543 20132 leader_election.cc:290] T 9b9137146e3644a6b0026778530fa8a6 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:28.221942 19722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3ca37daad8341ad99cbba926f37232e" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:28.223839 19995 ts_tablet_manager.cc:1403] T 63868398f3604f529519d926d2831f4e P c62716bf66c24330860573535b8b9430: Time spent bootstrapping tablet: real 0.164s	user 0.031s	sys 0.008s
I20260709 13:20:28.225601 19799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9137146e3644a6b0026778530fa8a6" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:28.226856 19995 raft_consensus.cc:359] T 63868398f3604f529519d926d2831f4e P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:28.227583 19995 raft_consensus.cc:385] T 63868398f3604f529519d926d2831f4e P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:28.227881 19995 raft_consensus.cc:740] T 63868398f3604f529519d926d2831f4e P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62716bf66c24330860573535b8b9430, State: Initialized, Role: FOLLOWER
I20260709 13:20:28.228518 19995 consensus_queue.cc:260] T 63868398f3604f529519d926d2831f4e P c62716bf66c24330860573535b8b9430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
W20260709 13:20:28.232203 19914 leader_election.cc:343] T e3ca37daad8341ad99cbba926f37232e P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:28.236047 19799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3ca37daad8341ad99cbba926f37232e" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:28.244668 19995 ts_tablet_manager.cc:1434] T 63868398f3604f529519d926d2831f4e P c62716bf66c24330860573535b8b9430: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.001s
I20260709 13:20:28.245738 20037 raft_consensus.cc:493] T c393f9d18341477c97fac9a4af7e3adf P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:28.246279 20037 raft_consensus.cc:515] T c393f9d18341477c97fac9a4af7e3adf P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:28.249926 20037 leader_election.cc:290] T c393f9d18341477c97fac9a4af7e3adf P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:28.251986 19799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c393f9d18341477c97fac9a4af7e3adf" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:28.252069 19721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c393f9d18341477c97fac9a4af7e3adf" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
W20260709 13:20:28.253578 19839 leader_election.cc:343] T c393f9d18341477c97fac9a4af7e3adf P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:20:28.254122 19840 leader_election.cc:343] T c393f9d18341477c97fac9a4af7e3adf P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:28.254935 19840 leader_election.cc:304] T c393f9d18341477c97fac9a4af7e3adf P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4; no voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:28.200284 19800 raft_consensus.cc:2468] T e9369d9b7f5649be932967d24689a936 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c62716bf66c24330860573535b8b9430 in term 1.
I20260709 13:20:28.257020 19946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9137146e3644a6b0026778530fa8a6" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:28.257911 19946 raft_consensus.cc:2393] T 9b9137146e3644a6b0026778530fa8a6 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8df38c4c53c44e94b36c1e8992476d7c in current term 1: Already voted for candidate e90ffed3ba174ababeb21d9415d4d918 in this term.
I20260709 13:20:28.259934 20237 raft_consensus.cc:2749] T 85d493223fe04c939b6effd840955727 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:28.263302 19690 leader_election.cc:304] T 9b9137146e3644a6b0026778530fa8a6 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c; no voters: c62716bf66c24330860573535b8b9430, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:28.264015 19994 tablet_bootstrap.cc:492] T 63868398f3604f529519d926d2831f4e P 8df38c4c53c44e94b36c1e8992476d7c: No bootstrap required, opened a new log
I20260709 13:20:28.264597 19994 ts_tablet_manager.cc:1403] T 63868398f3604f529519d926d2831f4e P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.239s	user 0.010s	sys 0.011s
I20260709 13:20:28.265012 20132 raft_consensus.cc:3060] T 9b9137146e3644a6b0026778530fa8a6 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:28.270073 20173 raft_consensus.cc:493] T a323eb80c5084159803e7d4ea5d9c410 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:28.270843 20173 raft_consensus.cc:515] T a323eb80c5084159803e7d4ea5d9c410 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
W20260709 13:20:28.273042 19913 leader_election.cc:343] T e3ca37daad8341ad99cbba926f37232e P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:28.273450 20237 raft_consensus.cc:493] T 0413f22bd6d7447ca630b75be496555f P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:28.273567 20173 leader_election.cc:290] T a323eb80c5084159803e7d4ea5d9c410 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:28.273828 19913 leader_election.cc:304] T e3ca37daad8341ad99cbba926f37232e P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:28.274076 20237 raft_consensus.cc:515] T 0413f22bd6d7447ca630b75be496555f P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:28.265465 20009 raft_consensus.cc:2749] T c393f9d18341477c97fac9a4af7e3adf P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:28.275950 19993 tablet_bootstrap.cc:654] T 8bdf042a063d4596970f49730ffa04b4 P e90ffed3ba174ababeb21d9415d4d918: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:28.245750 19995 tablet_bootstrap.cc:492] T f012387a39df446bbd9dbea32dc03202 P c62716bf66c24330860573535b8b9430: Bootstrap starting.
I20260709 13:20:28.277779 19725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a323eb80c5084159803e7d4ea5d9c410" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:28.280383 20000 raft_consensus.cc:2749] T e3ca37daad8341ad99cbba926f37232e P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:28.281695 19913 leader_election.cc:304] T e9369d9b7f5649be932967d24689a936 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c62716bf66c24330860573535b8b9430, e90ffed3ba174ababeb21d9415d4d918; no voters: 
I20260709 13:20:28.279785 20080 raft_consensus.cc:493] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:28.287355 20080 raft_consensus.cc:515] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:28.288019 20132 raft_consensus.cc:2749] T 9b9137146e3644a6b0026778530fa8a6 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:28.288322 20237 leader_election.cc:290] T 0413f22bd6d7447ca630b75be496555f P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:28.289609 20080 leader_election.cc:290] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:28.290096 19725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0413f22bd6d7447ca630b75be496555f" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:28.279232 19946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a323eb80c5084159803e7d4ea5d9c410" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:28.291800 20237 raft_consensus.cc:2804] T e9369d9b7f5649be932967d24689a936 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:20:28.291977 19914 leader_election.cc:343] T 0413f22bd6d7447ca630b75be496555f P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:28.292510 20237 raft_consensus.cc:697] T e9369d9b7f5649be932967d24689a936 P c62716bf66c24330860573535b8b9430 [term 1 LEADER]: Becoming Leader. State: Replica: c62716bf66c24330860573535b8b9430, State: Running, Role: LEADER
I20260709 13:20:28.294544 20237 consensus_queue.cc:237] T e9369d9b7f5649be932967d24689a936 P c62716bf66c24330860573535b8b9430 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:28.296929 20080 consensus_queue.cc:1048] T 7e3099f1888c4ed18c30523bf9ec94f5 P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:28.298897 19765 leader_election.cc:304] T a323eb80c5084159803e7d4ea5d9c410 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e90ffed3ba174ababeb21d9415d4d918; no voters: 8df38c4c53c44e94b36c1e8992476d7c, c62716bf66c24330860573535b8b9430
I20260709 13:20:28.301271 19800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563940ad96444ce59e393ea303cc3352" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:28.301517 19871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0413f22bd6d7447ca630b75be496555f" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:28.302949 20040 raft_consensus.cc:2749] T a323eb80c5084159803e7d4ea5d9c410 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:28.302757 19621 catalog_manager.cc:5697] T 7e3a9d11df99411d8f0edb0df6f14d87 P 8df38c4c53c44e94b36c1e8992476d7c reported cstate change: term changed from 0 to 1, leader changed from <none> to 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129). New cstate: current_term: 1 leader_uuid: "8df38c4c53c44e94b36c1e8992476d7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: UNKNOWN } } }
W20260709 13:20:28.304544 19839 leader_election.cc:343] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:28.306100 19993 tablet_bootstrap.cc:492] T 8bdf042a063d4596970f49730ffa04b4 P e90ffed3ba174ababeb21d9415d4d918: No bootstrap required, opened a new log
I20260709 13:20:28.284997 19995 tablet_bootstrap.cc:654] T f012387a39df446bbd9dbea32dc03202 P c62716bf66c24330860573535b8b9430: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:28.285542 19997 tablet_bootstrap.cc:492] T 0413f22bd6d7447ca630b75be496555f P b80b3a62d7d549a9a5c42def46769ac4: No bootstrap required, opened a new log
I20260709 13:20:28.307194 19997 ts_tablet_manager.cc:1403] T 0413f22bd6d7447ca630b75be496555f P b80b3a62d7d549a9a5c42def46769ac4: Time spent bootstrapping tablet: real 0.359s	user 0.009s	sys 0.012s
I20260709 13:20:28.286413 19994 raft_consensus.cc:359] T 63868398f3604f529519d926d2831f4e P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:28.290306 19721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563940ad96444ce59e393ea303cc3352" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
W20260709 13:20:28.311580 19840 leader_election.cc:343] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:28.312079 19840 leader_election.cc:304] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4; no voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:28.313989 20080 raft_consensus.cc:2749] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:28.314904 19994 raft_consensus.cc:385] T 63868398f3604f529519d926d2831f4e P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:28.315274 19994 raft_consensus.cc:740] T 63868398f3604f529519d926d2831f4e P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:28.316218 19997 raft_consensus.cc:359] T 0413f22bd6d7447ca630b75be496555f P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:28.316926 19997 raft_consensus.cc:385] T 0413f22bd6d7447ca630b75be496555f P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:28.317209 19997 raft_consensus.cc:740] T 0413f22bd6d7447ca630b75be496555f P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Initialized, Role: FOLLOWER
I20260709 13:20:28.317401 19994 consensus_queue.cc:260] T 63868398f3604f529519d926d2831f4e P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:28.318043 19997 consensus_queue.cc:260] T 0413f22bd6d7447ca630b75be496555f P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:28.322566 19994 ts_tablet_manager.cc:1434] T 63868398f3604f529519d926d2831f4e P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.058s	user 0.008s	sys 0.000s
I20260709 13:20:28.331990 19994 tablet_bootstrap.cc:492] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:28.329384 20237 raft_consensus.cc:493] T 3aaced67782041fa9960c745301a96e7 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:28.332824 20237 raft_consensus.cc:515] T 3aaced67782041fa9960c745301a96e7 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:28.336102 20080 consensus_queue.cc:1048] T 7e3099f1888c4ed18c30523bf9ec94f5 P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:20:28.341637 20132 raft_consensus.cc:493] T 63868398f3604f529519d926d2831f4e P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:28.342422 20132 raft_consensus.cc:515] T 63868398f3604f529519d926d2831f4e P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:28.346760 20132 leader_election.cc:290] T 63868398f3604f529519d926d2831f4e P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
W20260709 13:20:28.328802 19913 leader_election.cc:343] T 0413f22bd6d7447ca630b75be496555f P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:28.337343 20237 leader_election.cc:290] T 3aaced67782041fa9960c745301a96e7 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:28.338354 19721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaced67782041fa9960c745301a96e7" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
W20260709 13:20:28.349318 19914 leader_election.cc:343] T 3aaced67782041fa9960c745301a96e7 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:28.349323 19800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63868398f3604f529519d926d2831f4e" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
W20260709 13:20:28.352116 19690 leader_election.cc:343] T 63868398f3604f529519d926d2831f4e P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:28.351437 19913 leader_election.cc:304] T 0413f22bd6d7447ca630b75be496555f P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4
I20260709 13:20:28.353199 20237 raft_consensus.cc:2749] T 0413f22bd6d7447ca630b75be496555f P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:28.346913 19946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63868398f3604f529519d926d2831f4e" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:28.330271 19997 ts_tablet_manager.cc:1434] T 0413f22bd6d7447ca630b75be496555f P b80b3a62d7d549a9a5c42def46769ac4: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.002s
I20260709 13:20:28.354040 19946 raft_consensus.cc:2468] T 63868398f3604f529519d926d2831f4e P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 0.
I20260709 13:20:28.355382 19690 leader_election.cc:304] T 63868398f3604f529519d926d2831f4e P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, c62716bf66c24330860573535b8b9430; no voters: e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:28.356920 20132 raft_consensus.cc:2804] T 63868398f3604f529519d926d2831f4e P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:28.357491 20132 raft_consensus.cc:493] T 63868398f3604f529519d926d2831f4e P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:28.358024 20132 raft_consensus.cc:3060] T 63868398f3604f529519d926d2831f4e P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:28.369014 19871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaced67782041fa9960c745301a96e7" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:28.371534 19997 tablet_bootstrap.cc:492] T 0c709292a0304b3c84a777427a754bb5 P b80b3a62d7d549a9a5c42def46769ac4: Bootstrap starting.
I20260709 13:20:28.330731 19993 ts_tablet_manager.cc:1403] T 8bdf042a063d4596970f49730ffa04b4 P e90ffed3ba174ababeb21d9415d4d918: Time spent bootstrapping tablet: real 0.151s	user 0.012s	sys 0.000s
I20260709 13:20:28.387840 19993 raft_consensus.cc:359] T 8bdf042a063d4596970f49730ffa04b4 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
W20260709 13:20:28.372536 19913 leader_election.cc:343] T 3aaced67782041fa9960c745301a96e7 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:28.392584 19913 leader_election.cc:304] T 3aaced67782041fa9960c745301a96e7 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4
I20260709 13:20:28.393548 20237 raft_consensus.cc:2749] T 3aaced67782041fa9960c745301a96e7 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:28.404410 19994 tablet_bootstrap.cc:654] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:28.403932 19622 catalog_manager.cc:5697] T 19e83b82855d4129bb8525e6ff05316b P b80b3a62d7d549a9a5c42def46769ac4 reported cstate change: term changed from 1 to 2, leader changed from 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129) to b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131). New cstate: current_term: 2 leader_uuid: "b80b3a62d7d549a9a5c42def46769ac4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:28.406836 19997 tablet_bootstrap.cc:654] T 0c709292a0304b3c84a777427a754bb5 P b80b3a62d7d549a9a5c42def46769ac4: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:28.409729 20040 consensus_queue.cc:1048] T 9b9137146e3644a6b0026778530fa8a6 P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:20:28.388857 19993 raft_consensus.cc:385] T 8bdf042a063d4596970f49730ffa04b4 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:28.433243 19993 raft_consensus.cc:740] T 8bdf042a063d4596970f49730ffa04b4 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Initialized, Role: FOLLOWER
I20260709 13:20:28.434603 19993 consensus_queue.cc:260] T 8bdf042a063d4596970f49730ffa04b4 P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:28.435817 19997 tablet_bootstrap.cc:492] T 0c709292a0304b3c84a777427a754bb5 P b80b3a62d7d549a9a5c42def46769ac4: No bootstrap required, opened a new log
I20260709 13:20:28.436218 19993 ts_tablet_manager.cc:1434] T 8bdf042a063d4596970f49730ffa04b4 P e90ffed3ba174ababeb21d9415d4d918: Time spent starting tablet: real 0.052s	user 0.007s	sys 0.000s
I20260709 13:20:28.436329 19997 ts_tablet_manager.cc:1403] T 0c709292a0304b3c84a777427a754bb5 P b80b3a62d7d549a9a5c42def46769ac4: Time spent bootstrapping tablet: real 0.065s	user 0.009s	sys 0.004s
I20260709 13:20:28.439267 19997 raft_consensus.cc:359] T 0c709292a0304b3c84a777427a754bb5 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:28.439883 19800 raft_consensus.cc:3060] T a323eb80c5084159803e7d4ea5d9c410 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:28.439960 19997 raft_consensus.cc:385] T 0c709292a0304b3c84a777427a754bb5 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:28.444422 20040 consensus_queue.cc:1048] T 9b9137146e3644a6b0026778530fa8a6 P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:20:28.445011 19997 raft_consensus.cc:740] T 0c709292a0304b3c84a777427a754bb5 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Initialized, Role: FOLLOWER
I20260709 13:20:28.446803 19997 consensus_queue.cc:260] T 0c709292a0304b3c84a777427a754bb5 P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:28.446209 19620 catalog_manager.cc:5697] T e9369d9b7f5649be932967d24689a936 P c62716bf66c24330860573535b8b9430 reported cstate change: term changed from 0 to 1, leader changed from <none> to c62716bf66c24330860573535b8b9430 (127.9.116.132). New cstate: current_term: 1 leader_uuid: "c62716bf66c24330860573535b8b9430" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:28.451436 20040 raft_consensus.cc:493] T 0c709292a0304b3c84a777427a754bb5 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:28.452294 19997 ts_tablet_manager.cc:1434] T 0c709292a0304b3c84a777427a754bb5 P b80b3a62d7d549a9a5c42def46769ac4: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:20:28.453336 19997 tablet_bootstrap.cc:492] T f012387a39df446bbd9dbea32dc03202 P b80b3a62d7d549a9a5c42def46769ac4: Bootstrap starting.
I20260709 13:20:28.432971 19995 tablet_bootstrap.cc:492] T f012387a39df446bbd9dbea32dc03202 P c62716bf66c24330860573535b8b9430: No bootstrap required, opened a new log
I20260709 13:20:28.451989 20040 raft_consensus.cc:515] T 0c709292a0304b3c84a777427a754bb5 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:28.454823 19995 ts_tablet_manager.cc:1403] T f012387a39df446bbd9dbea32dc03202 P c62716bf66c24330860573535b8b9430: Time spent bootstrapping tablet: real 0.209s	user 0.044s	sys 0.057s
I20260709 13:20:28.460579 19871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c709292a0304b3c84a777427a754bb5" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:28.461362 19871 raft_consensus.cc:2468] T 0c709292a0304b3c84a777427a754bb5 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 0.
I20260709 13:20:28.461681 19995 raft_consensus.cc:359] T f012387a39df446bbd9dbea32dc03202 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:28.462433 19995 raft_consensus.cc:385] T f012387a39df446bbd9dbea32dc03202 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:28.462954 19765 leader_election.cc:304] T 0c709292a0304b3c84a777427a754bb5 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, e90ffed3ba174ababeb21d9415d4d918; no voters: 
I20260709 13:20:28.464013 20065 raft_consensus.cc:2804] T 0c709292a0304b3c84a777427a754bb5 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:28.462702 19995 raft_consensus.cc:740] T f012387a39df446bbd9dbea32dc03202 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62716bf66c24330860573535b8b9430, State: Initialized, Role: FOLLOWER
I20260709 13:20:28.464403 20065 raft_consensus.cc:493] T 0c709292a0304b3c84a777427a754bb5 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:28.465029 19997 tablet_bootstrap.cc:654] T f012387a39df446bbd9dbea32dc03202 P b80b3a62d7d549a9a5c42def46769ac4: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:28.464974 20065 raft_consensus.cc:3060] T 0c709292a0304b3c84a777427a754bb5 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:28.465224 19995 consensus_queue.cc:260] T f012387a39df446bbd9dbea32dc03202 P c62716bf66c24330860573535b8b9430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:28.468106 19995 ts_tablet_manager.cc:1434] T f012387a39df446bbd9dbea32dc03202 P c62716bf66c24330860573535b8b9430: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:20:28.469125 19995 tablet_bootstrap.cc:492] T d5f8619b1f9f46adaa41774af856d9b1 P c62716bf66c24330860573535b8b9430: Bootstrap starting.
I20260709 13:20:28.472600 20040 leader_election.cc:290] T 0c709292a0304b3c84a777427a754bb5 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:28.485407 19995 tablet_bootstrap.cc:654] T d5f8619b1f9f46adaa41774af856d9b1 P c62716bf66c24330860573535b8b9430: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:28.472900 19993 tablet_bootstrap.cc:492] T 7e3a9d11df99411d8f0edb0df6f14d87 P e90ffed3ba174ababeb21d9415d4d918: Bootstrap starting.
I20260709 13:20:28.474740 19946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c709292a0304b3c84a777427a754bb5" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:28.496497 19946 raft_consensus.cc:2468] T 0c709292a0304b3c84a777427a754bb5 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 0.
I20260709 13:20:28.421754 20132 raft_consensus.cc:515] T 63868398f3604f529519d926d2831f4e P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:28.503005 19997 tablet_bootstrap.cc:492] T f012387a39df446bbd9dbea32dc03202 P b80b3a62d7d549a9a5c42def46769ac4: No bootstrap required, opened a new log
I20260709 13:20:28.503563 19997 ts_tablet_manager.cc:1403] T f012387a39df446bbd9dbea32dc03202 P b80b3a62d7d549a9a5c42def46769ac4: Time spent bootstrapping tablet: real 0.050s	user 0.013s	sys 0.001s
I20260709 13:20:28.505511 19995 tablet_bootstrap.cc:492] T d5f8619b1f9f46adaa41774af856d9b1 P c62716bf66c24330860573535b8b9430: No bootstrap required, opened a new log
I20260709 13:20:28.506052 19995 ts_tablet_manager.cc:1403] T d5f8619b1f9f46adaa41774af856d9b1 P c62716bf66c24330860573535b8b9430: Time spent bootstrapping tablet: real 0.037s	user 0.011s	sys 0.005s
I20260709 13:20:28.506175 19946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63868398f3604f529519d926d2831f4e" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:28.506996 19946 raft_consensus.cc:3060] T 63868398f3604f529519d926d2831f4e P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:28.506289 19997 raft_consensus.cc:359] T f012387a39df446bbd9dbea32dc03202 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:28.511112 19997 raft_consensus.cc:385] T f012387a39df446bbd9dbea32dc03202 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:28.511449 19997 raft_consensus.cc:740] T f012387a39df446bbd9dbea32dc03202 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Initialized, Role: FOLLOWER
I20260709 13:20:28.512142 19997 consensus_queue.cc:260] T f012387a39df446bbd9dbea32dc03202 P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:28.515117 20132 leader_election.cc:290] T 63868398f3604f529519d926d2831f4e P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Requested vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:28.517082 19800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63868398f3604f529519d926d2831f4e" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
I20260709 13:20:28.517159 19997 ts_tablet_manager.cc:1434] T f012387a39df446bbd9dbea32dc03202 P b80b3a62d7d549a9a5c42def46769ac4: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:20:28.520097 19997 tablet_bootstrap.cc:492] T 1fa8a0db61be45c3aa42edb11db4c5ff P b80b3a62d7d549a9a5c42def46769ac4: Bootstrap starting.
W20260709 13:20:28.520350 19690 leader_election.cc:343] T 63868398f3604f529519d926d2831f4e P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:28.522931 19995 raft_consensus.cc:359] T d5f8619b1f9f46adaa41774af856d9b1 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:28.523676 19995 raft_consensus.cc:385] T d5f8619b1f9f46adaa41774af856d9b1 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:28.523936 19995 raft_consensus.cc:740] T d5f8619b1f9f46adaa41774af856d9b1 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62716bf66c24330860573535b8b9430, State: Initialized, Role: FOLLOWER
I20260709 13:20:28.476534 20065 raft_consensus.cc:515] T 0c709292a0304b3c84a777427a754bb5 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:28.524590 19995 consensus_queue.cc:260] T d5f8619b1f9f46adaa41774af856d9b1 P c62716bf66c24330860573535b8b9430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:28.526072 20065 leader_election.cc:290] T 0c709292a0304b3c84a777427a754bb5 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 election: Requested vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:28.526835 19995 ts_tablet_manager.cc:1434] T d5f8619b1f9f46adaa41774af856d9b1 P c62716bf66c24330860573535b8b9430: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 13:20:28.527364 19871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c709292a0304b3c84a777427a754bb5" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
I20260709 13:20:28.527856 19995 tablet_bootstrap.cc:492] T 1cceb7f75d7040c2b98049590928b891 P c62716bf66c24330860573535b8b9430: Bootstrap starting.
I20260709 13:20:28.530983 19942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c709292a0304b3c84a777427a754bb5" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:28.531786 19942 raft_consensus.cc:3060] T 0c709292a0304b3c84a777427a754bb5 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:28.535435 19997 tablet_bootstrap.cc:654] T 1fa8a0db61be45c3aa42edb11db4c5ff P b80b3a62d7d549a9a5c42def46769ac4: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:28.527959 19871 raft_consensus.cc:3060] T 0c709292a0304b3c84a777427a754bb5 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:28.537521 19995 tablet_bootstrap.cc:654] T 1cceb7f75d7040c2b98049590928b891 P c62716bf66c24330860573535b8b9430: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:28.548959 19997 tablet_bootstrap.cc:492] T 1fa8a0db61be45c3aa42edb11db4c5ff P b80b3a62d7d549a9a5c42def46769ac4: No bootstrap required, opened a new log
I20260709 13:20:28.549476 19997 ts_tablet_manager.cc:1403] T 1fa8a0db61be45c3aa42edb11db4c5ff P b80b3a62d7d549a9a5c42def46769ac4: Time spent bootstrapping tablet: real 0.030s	user 0.007s	sys 0.003s
I20260709 13:20:28.549768 19993 tablet_bootstrap.cc:654] T 7e3a9d11df99411d8f0edb0df6f14d87 P e90ffed3ba174ababeb21d9415d4d918: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:28.552263 19997 raft_consensus.cc:359] T 1fa8a0db61be45c3aa42edb11db4c5ff P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:28.553012 19997 raft_consensus.cc:385] T 1fa8a0db61be45c3aa42edb11db4c5ff P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:28.553279 19997 raft_consensus.cc:740] T 1fa8a0db61be45c3aa42edb11db4c5ff P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Initialized, Role: FOLLOWER
I20260709 13:20:28.553889 19946 raft_consensus.cc:2468] T 63868398f3604f529519d926d2831f4e P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 1.
I20260709 13:20:28.558676 19871 raft_consensus.cc:2468] T 0c709292a0304b3c84a777427a754bb5 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 1.
I20260709 13:20:28.558956 19690 leader_election.cc:304] T 63868398f3604f529519d926d2831f4e P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, c62716bf66c24330860573535b8b9430; no voters: e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:28.560278 20132 raft_consensus.cc:2804] T 63868398f3604f529519d926d2831f4e P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:28.560534 19995 tablet_bootstrap.cc:492] T 1cceb7f75d7040c2b98049590928b891 P c62716bf66c24330860573535b8b9430: No bootstrap required, opened a new log
I20260709 13:20:28.561056 20132 raft_consensus.cc:697] T 63868398f3604f529519d926d2831f4e P 8df38c4c53c44e94b36c1e8992476d7c [term 1 LEADER]: Becoming Leader. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Running, Role: LEADER
I20260709 13:20:28.561528 19995 ts_tablet_manager.cc:1403] T 1cceb7f75d7040c2b98049590928b891 P c62716bf66c24330860573535b8b9430: Time spent bootstrapping tablet: real 0.034s	user 0.014s	sys 0.001s
I20260709 13:20:28.561988 20132 consensus_queue.cc:237] T 63868398f3604f529519d926d2831f4e P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:28.564476 19995 raft_consensus.cc:359] T 1cceb7f75d7040c2b98049590928b891 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:28.567021 19765 leader_election.cc:304] T 0c709292a0304b3c84a777427a754bb5 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, e90ffed3ba174ababeb21d9415d4d918; no voters: 
I20260709 13:20:28.570226 20212 raft_consensus.cc:2804] T 0c709292a0304b3c84a777427a754bb5 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:28.570906 20212 raft_consensus.cc:697] T 0c709292a0304b3c84a777427a754bb5 P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Becoming Leader. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Running, Role: LEADER
I20260709 13:20:28.573158 20212 consensus_queue.cc:237] T 0c709292a0304b3c84a777427a754bb5 P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:28.565219 19995 raft_consensus.cc:385] T 1cceb7f75d7040c2b98049590928b891 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:28.575134 19995 raft_consensus.cc:740] T 1cceb7f75d7040c2b98049590928b891 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62716bf66c24330860573535b8b9430, State: Initialized, Role: FOLLOWER
I20260709 13:20:28.575853 19995 consensus_queue.cc:260] T 1cceb7f75d7040c2b98049590928b891 P c62716bf66c24330860573535b8b9430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
W20260709 13:20:28.577917 19690 consensus_peers.cc:597] T 7e3a9d11df99411d8f0edb0df6f14d87 P 8df38c4c53c44e94b36c1e8992476d7c -> Peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Couldn't send request to peer e90ffed3ba174ababeb21d9415d4d918. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:28.578734 19995 ts_tablet_manager.cc:1434] T 1cceb7f75d7040c2b98049590928b891 P c62716bf66c24330860573535b8b9430: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.001s
I20260709 13:20:28.579437 19993 tablet_bootstrap.cc:492] T 7e3a9d11df99411d8f0edb0df6f14d87 P e90ffed3ba174ababeb21d9415d4d918: No bootstrap required, opened a new log
I20260709 13:20:28.579916 19995 tablet_bootstrap.cc:492] T 7a5476f5d68d496eaedce93e80bff5e4 P c62716bf66c24330860573535b8b9430: Bootstrap starting.
I20260709 13:20:28.580461 19946 raft_consensus.cc:3060] T 19e83b82855d4129bb8525e6ff05316b P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:28.553992 19997 consensus_queue.cc:260] T 1fa8a0db61be45c3aa42edb11db4c5ff P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:28.581615 19721 raft_consensus.cc:1275] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Refusing update from remote peer b80b3a62d7d549a9a5c42def46769ac4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:28.557557 19994 tablet_bootstrap.cc:492] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c: No bootstrap required, opened a new log
I20260709 13:20:28.583191 19994 ts_tablet_manager.cc:1403] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.252s	user 0.028s	sys 0.028s
I20260709 13:20:28.583213 20079 consensus_queue.cc:1048] T 19e83b82855d4129bb8525e6ff05316b P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:20:28.585776 19995 tablet_bootstrap.cc:654] T 7a5476f5d68d496eaedce93e80bff5e4 P c62716bf66c24330860573535b8b9430: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:28.586052 19994 raft_consensus.cc:359] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:28.586745 19994 raft_consensus.cc:385] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:28.587069 19994 raft_consensus.cc:740] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
W20260709 13:20:28.587400 19670 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:20:28.587730 19994 consensus_queue.cc:260] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:28.579952 19993 ts_tablet_manager.cc:1403] T 7e3a9d11df99411d8f0edb0df6f14d87 P e90ffed3ba174ababeb21d9415d4d918: Time spent bootstrapping tablet: real 0.107s	user 0.013s	sys 0.000s
I20260709 13:20:28.591984 19997 ts_tablet_manager.cc:1434] T 1fa8a0db61be45c3aa42edb11db4c5ff P b80b3a62d7d549a9a5c42def46769ac4: Time spent starting tablet: real 0.042s	user 0.007s	sys 0.001s
I20260709 13:20:28.593024 19997 tablet_bootstrap.cc:492] T 3aaced67782041fa9960c745301a96e7 P b80b3a62d7d549a9a5c42def46769ac4: Bootstrap starting.
I20260709 13:20:28.593729 19993 raft_consensus.cc:359] T 7e3a9d11df99411d8f0edb0df6f14d87 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:28.594436 19993 raft_consensus.cc:385] T 7e3a9d11df99411d8f0edb0df6f14d87 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:28.594689 19993 raft_consensus.cc:740] T 7e3a9d11df99411d8f0edb0df6f14d87 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Initialized, Role: FOLLOWER
I20260709 13:20:28.595538 19993 consensus_queue.cc:260] T 7e3a9d11df99411d8f0edb0df6f14d87 P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:28.597777 19993 ts_tablet_manager.cc:1434] T 7e3a9d11df99411d8f0edb0df6f14d87 P e90ffed3ba174ababeb21d9415d4d918: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:20:28.598744 19993 tablet_bootstrap.cc:492] T 63868398f3604f529519d926d2831f4e P e90ffed3ba174ababeb21d9415d4d918: Bootstrap starting.
I20260709 13:20:28.599910 19997 tablet_bootstrap.cc:654] T 3aaced67782041fa9960c745301a96e7 P b80b3a62d7d549a9a5c42def46769ac4: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:28.602120 19620 catalog_manager.cc:5697] T 63868398f3604f529519d926d2831f4e P 8df38c4c53c44e94b36c1e8992476d7c reported cstate change: term changed from 0 to 1, leader changed from <none> to 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129). New cstate: current_term: 1 leader_uuid: "8df38c4c53c44e94b36c1e8992476d7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:28.603773 19995 tablet_bootstrap.cc:492] T 7a5476f5d68d496eaedce93e80bff5e4 P c62716bf66c24330860573535b8b9430: No bootstrap required, opened a new log
I20260709 13:20:28.604255 19995 ts_tablet_manager.cc:1403] T 7a5476f5d68d496eaedce93e80bff5e4 P c62716bf66c24330860573535b8b9430: Time spent bootstrapping tablet: real 0.025s	user 0.007s	sys 0.004s
I20260709 13:20:28.609977 19942 raft_consensus.cc:2468] T 0c709292a0304b3c84a777427a754bb5 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 1.
I20260709 13:20:28.611436 19994 ts_tablet_manager.cc:1434] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.001s
I20260709 13:20:28.615466 20237 raft_consensus.cc:493] T f012387a39df446bbd9dbea32dc03202 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:28.615411 19995 raft_consensus.cc:359] T 7a5476f5d68d496eaedce93e80bff5e4 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:28.616338 19995 raft_consensus.cc:385] T 7a5476f5d68d496eaedce93e80bff5e4 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:28.616636 19995 raft_consensus.cc:740] T 7a5476f5d68d496eaedce93e80bff5e4 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62716bf66c24330860573535b8b9430, State: Initialized, Role: FOLLOWER
I20260709 13:20:28.616189 20237 raft_consensus.cc:515] T f012387a39df446bbd9dbea32dc03202 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:28.621065 19871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f012387a39df446bbd9dbea32dc03202" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:28.621300 19799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f012387a39df446bbd9dbea32dc03202" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:28.621870 19871 raft_consensus.cc:2468] T f012387a39df446bbd9dbea32dc03202 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c62716bf66c24330860573535b8b9430 in term 0.
I20260709 13:20:28.622514 20237 leader_election.cc:290] T f012387a39df446bbd9dbea32dc03202 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:28.617424 19995 consensus_queue.cc:260] T 7a5476f5d68d496eaedce93e80bff5e4 P c62716bf66c24330860573535b8b9430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
W20260709 13:20:28.623709 19913 leader_election.cc:343] T f012387a39df446bbd9dbea32dc03202 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:28.610033 19946 raft_consensus.cc:1275] T 19e83b82855d4129bb8525e6ff05316b P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Refusing update from remote peer b80b3a62d7d549a9a5c42def46769ac4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:28.623703 20079 raft_consensus.cc:493] T 0413f22bd6d7447ca630b75be496555f P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:28.626531 19913 leader_election.cc:304] T f012387a39df446bbd9dbea32dc03202 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430; no voters: e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:28.626977 19994 tablet_bootstrap.cc:492] T c393f9d18341477c97fac9a4af7e3adf P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:28.626618 20079 raft_consensus.cc:515] T 0413f22bd6d7447ca630b75be496555f P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:28.627467 20237 raft_consensus.cc:2804] T f012387a39df446bbd9dbea32dc03202 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:28.627857 20237 raft_consensus.cc:493] T f012387a39df446bbd9dbea32dc03202 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:28.628188 20237 raft_consensus.cc:3060] T f012387a39df446bbd9dbea32dc03202 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:28.630033 20083 raft_consensus.cc:493] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:28.630297 20080 consensus_queue.cc:1048] T 19e83b82855d4129bb8525e6ff05316b P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:20:28.630609 20083 raft_consensus.cc:515] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:28.632771 20083 leader_election.cc:290] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:28.633561 19721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0413f22bd6d7447ca630b75be496555f" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:28.633955 19994 tablet_bootstrap.cc:654] T c393f9d18341477c97fac9a4af7e3adf P 8df38c4c53c44e94b36c1e8992476d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:28.634425 19946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0413f22bd6d7447ca630b75be496555f" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
W20260709 13:20:28.635216 19840 leader_election.cc:343] T 0413f22bd6d7447ca630b75be496555f P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:28.636678 20237 raft_consensus.cc:515] T f012387a39df446bbd9dbea32dc03202 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:28.644935 20237 leader_election.cc:290] T f012387a39df446bbd9dbea32dc03202 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 election: Requested vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:28.636873 19800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5f8619b1f9f46adaa41774af856d9b1" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:28.636911 19944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5f8619b1f9f46adaa41774af856d9b1" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:28.637485 19995 ts_tablet_manager.cc:1434] T 7a5476f5d68d496eaedce93e80bff5e4 P c62716bf66c24330860573535b8b9430: Time spent starting tablet: real 0.033s	user 0.010s	sys 0.001s
I20260709 13:20:28.646133 19944 raft_consensus.cc:2468] T d5f8619b1f9f46adaa41774af856d9b1 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 0.
W20260709 13:20:28.646539 19690 leader_election.cc:343] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:28.647078 19995 tablet_bootstrap.cc:492] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P c62716bf66c24330860573535b8b9430: Bootstrap starting.
I20260709 13:20:28.646996 19799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f012387a39df446bbd9dbea32dc03202" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
I20260709 13:20:28.647503 19946 raft_consensus.cc:2468] T 0413f22bd6d7447ca630b75be496555f P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 0.
I20260709 13:20:28.648478 19871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f012387a39df446bbd9dbea32dc03202" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
I20260709 13:20:28.649158 19871 raft_consensus.cc:3060] T f012387a39df446bbd9dbea32dc03202 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:28.637714 20079 leader_election.cc:290] T 0413f22bd6d7447ca630b75be496555f P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:28.652248 19690 leader_election.cc:304] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, c62716bf66c24330860573535b8b9430; no voters: e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:28.653107 19839 leader_election.cc:304] T 0413f22bd6d7447ca630b75be496555f P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:28.654136 20181 raft_consensus.cc:2804] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:28.654819 20080 raft_consensus.cc:2804] T 0413f22bd6d7447ca630b75be496555f P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:28.655213 20080 raft_consensus.cc:493] T 0413f22bd6d7447ca630b75be496555f P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:28.655526 20080 raft_consensus.cc:3060] T 0413f22bd6d7447ca630b75be496555f P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:28.654577 20181 raft_consensus.cc:493] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:28.657310 20181 raft_consensus.cc:3060] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:28.657950 19995 tablet_bootstrap.cc:654] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P c62716bf66c24330860573535b8b9430: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:28.641422 19994 tablet_bootstrap.cc:492] T c393f9d18341477c97fac9a4af7e3adf P 8df38c4c53c44e94b36c1e8992476d7c: No bootstrap required, opened a new log
I20260709 13:20:28.667057 19994 ts_tablet_manager.cc:1403] T c393f9d18341477c97fac9a4af7e3adf P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.055s	user 0.013s	sys 0.000s
W20260709 13:20:28.673524 19913 leader_election.cc:343] T f012387a39df446bbd9dbea32dc03202 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:28.622701 19993 tablet_bootstrap.cc:654] T 63868398f3604f529519d926d2831f4e P e90ffed3ba174ababeb21d9415d4d918: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:28.665817 19623 catalog_manager.cc:5697] T 0c709292a0304b3c84a777427a754bb5 P e90ffed3ba174ababeb21d9415d4d918 reported cstate change: term changed from 0 to 1, leader changed from <none> to e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "e90ffed3ba174ababeb21d9415d4d918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:28.680226 20181 raft_consensus.cc:515] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:28.687575 19944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5f8619b1f9f46adaa41774af856d9b1" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:28.688611 19944 raft_consensus.cc:3060] T d5f8619b1f9f46adaa41774af856d9b1 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:28.689924 19799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5f8619b1f9f46adaa41774af856d9b1" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
W20260709 13:20:28.691753 19690 leader_election.cc:343] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:28.680620 19994 raft_consensus.cc:359] T c393f9d18341477c97fac9a4af7e3adf P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:28.692971 19994 raft_consensus.cc:385] T c393f9d18341477c97fac9a4af7e3adf P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:28.693246 19994 raft_consensus.cc:740] T c393f9d18341477c97fac9a4af7e3adf P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:28.693888 20212 raft_consensus.cc:493] T 8bdf042a063d4596970f49730ffa04b4 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:28.692369 19871 raft_consensus.cc:2468] T f012387a39df446bbd9dbea32dc03202 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c62716bf66c24330860573535b8b9430 in term 1.
I20260709 13:20:28.695940 20212 raft_consensus.cc:515] T 8bdf042a063d4596970f49730ffa04b4 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:28.697094 19913 leader_election.cc:304] T f012387a39df446bbd9dbea32dc03202 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430; no voters: e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:28.698431 20212 leader_election.cc:290] T 8bdf042a063d4596970f49730ffa04b4 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:28.698626 20237 raft_consensus.cc:2804] T f012387a39df446bbd9dbea32dc03202 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:28.699158 20237 raft_consensus.cc:697] T f012387a39df446bbd9dbea32dc03202 P c62716bf66c24330860573535b8b9430 [term 1 LEADER]: Becoming Leader. State: Replica: c62716bf66c24330860573535b8b9430, State: Running, Role: LEADER
I20260709 13:20:28.694916 19994 consensus_queue.cc:260] T c393f9d18341477c97fac9a4af7e3adf P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:28.706310 19994 ts_tablet_manager.cc:1434] T c393f9d18341477c97fac9a4af7e3adf P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.039s	user 0.008s	sys 0.000s
I20260709 13:20:28.695163 20181 leader_election.cc:290] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Requested vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:28.709303 19994 tablet_bootstrap.cc:492] T 0413f22bd6d7447ca630b75be496555f P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:28.714833 19721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bdf042a063d4596970f49730ffa04b4" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:28.715605 19721 raft_consensus.cc:2468] T 8bdf042a063d4596970f49730ffa04b4 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 0.
I20260709 13:20:28.707177 20237 consensus_queue.cc:237] T f012387a39df446bbd9dbea32dc03202 P c62716bf66c24330860573535b8b9430 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:28.717271 19766 leader_election.cc:304] T 8bdf042a063d4596970f49730ffa04b4 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918; no voters: 
I20260709 13:20:28.704313 19874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bdf042a063d4596970f49730ffa04b4" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:28.723642 20083 consensus_queue.cc:1048] T 7e3a9d11df99411d8f0edb0df6f14d87 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:28.725723 20173 raft_consensus.cc:2804] T 8bdf042a063d4596970f49730ffa04b4 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:28.726212 20173 raft_consensus.cc:493] T 8bdf042a063d4596970f49730ffa04b4 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:28.726572 20173 raft_consensus.cc:3060] T 8bdf042a063d4596970f49730ffa04b4 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:20:28.726224 19765 leader_election.cc:343] T 8bdf042a063d4596970f49730ffa04b4 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:28.735152 20000 consensus_queue.cc:1048] T e9369d9b7f5649be932967d24689a936 P c62716bf66c24330860573535b8b9430 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:20:28.747702 19995 tablet_bootstrap.cc:492] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P c62716bf66c24330860573535b8b9430: No bootstrap required, opened a new log
I20260709 13:20:28.749918 19995 ts_tablet_manager.cc:1403] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P c62716bf66c24330860573535b8b9430: Time spent bootstrapping tablet: real 0.103s	user 0.012s	sys 0.016s
I20260709 13:20:28.758145 19997 tablet_bootstrap.cc:492] T 3aaced67782041fa9960c745301a96e7 P b80b3a62d7d549a9a5c42def46769ac4: No bootstrap required, opened a new log
I20260709 13:20:28.758884 19997 ts_tablet_manager.cc:1403] T 3aaced67782041fa9960c745301a96e7 P b80b3a62d7d549a9a5c42def46769ac4: Time spent bootstrapping tablet: real 0.166s	user 0.016s	sys 0.008s
I20260709 13:20:28.763728 19994 tablet_bootstrap.cc:654] T 0413f22bd6d7447ca630b75be496555f P 8df38c4c53c44e94b36c1e8992476d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:28.767546 19997 raft_consensus.cc:359] T 3aaced67782041fa9960c745301a96e7 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:28.768561 19997 raft_consensus.cc:385] T 3aaced67782041fa9960c745301a96e7 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:28.768901 19997 raft_consensus.cc:740] T 3aaced67782041fa9960c745301a96e7 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Initialized, Role: FOLLOWER
I20260709 13:20:28.772058 20173 raft_consensus.cc:515] T 8bdf042a063d4596970f49730ffa04b4 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:28.779965 19874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bdf042a063d4596970f49730ffa04b4" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
I20260709 13:20:28.780093 19623 catalog_manager.cc:5697] T f012387a39df446bbd9dbea32dc03202 P c62716bf66c24330860573535b8b9430 reported cstate change: term changed from 0 to 1, leader changed from <none> to c62716bf66c24330860573535b8b9430 (127.9.116.132). New cstate: current_term: 1 leader_uuid: "c62716bf66c24330860573535b8b9430" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: HEALTHY } } }
W20260709 13:20:28.781519 19765 leader_election.cc:343] T 8bdf042a063d4596970f49730ffa04b4 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:28.784742 19997 consensus_queue.cc:260] T 3aaced67782041fa9960c745301a96e7 P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:28.785684 19995 raft_consensus.cc:359] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:28.786244 19995 raft_consensus.cc:385] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:28.786456 19995 raft_consensus.cc:740] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62716bf66c24330860573535b8b9430, State: Initialized, Role: FOLLOWER
I20260709 13:20:28.787159 19995 consensus_queue.cc:260] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P c62716bf66c24330860573535b8b9430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:28.782109 20173 leader_election.cc:290] T 8bdf042a063d4596970f49730ffa04b4 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 election: Requested vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:28.793289 19997 ts_tablet_manager.cc:1434] T 3aaced67782041fa9960c745301a96e7 P b80b3a62d7d549a9a5c42def46769ac4: Time spent starting tablet: real 0.034s	user 0.005s	sys 0.002s
I20260709 13:20:28.794384 19997 tablet_bootstrap.cc:492] T 8bdf042a063d4596970f49730ffa04b4 P b80b3a62d7d549a9a5c42def46769ac4: Bootstrap starting.
I20260709 13:20:28.785564 19944 raft_consensus.cc:2468] T d5f8619b1f9f46adaa41774af856d9b1 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 1.
I20260709 13:20:28.802233 19690 leader_election.cc:304] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, c62716bf66c24330860573535b8b9430; no voters: e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:28.803551 20083 raft_consensus.cc:2804] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:28.804183 20083 raft_consensus.cc:697] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 LEADER]: Becoming Leader. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Running, Role: LEADER
I20260709 13:20:28.805287 20083 consensus_queue.cc:237] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:28.807672 20132 raft_consensus.cc:493] T c393f9d18341477c97fac9a4af7e3adf P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:28.808406 20132 raft_consensus.cc:515] T c393f9d18341477c97fac9a4af7e3adf P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:28.822504 20000 consensus_queue.cc:1048] T e9369d9b7f5649be932967d24689a936 P c62716bf66c24330860573535b8b9430 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:20:28.826815 20037 raft_consensus.cc:493] T 3aaced67782041fa9960c745301a96e7 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:28.829988 20037 raft_consensus.cc:515] T 3aaced67782041fa9960c745301a96e7 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:28.837198 20037 leader_election.cc:290] T 3aaced67782041fa9960c745301a96e7 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:28.838577 19725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaced67782041fa9960c745301a96e7" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
W20260709 13:20:28.840423 19840 leader_election.cc:343] T 3aaced67782041fa9960c745301a96e7 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:28.842420 19993 tablet_bootstrap.cc:492] T 63868398f3604f529519d926d2831f4e P e90ffed3ba174ababeb21d9415d4d918: No bootstrap required, opened a new log
I20260709 13:20:28.843652 19993 ts_tablet_manager.cc:1403] T 63868398f3604f529519d926d2831f4e P e90ffed3ba174ababeb21d9415d4d918: Time spent bootstrapping tablet: real 0.245s	user 0.014s	sys 0.009s
I20260709 13:20:28.844929 19995 ts_tablet_manager.cc:1434] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P c62716bf66c24330860573535b8b9430: Time spent starting tablet: real 0.095s	user 0.006s	sys 0.005s
I20260709 13:20:28.856283 19995 tablet_bootstrap.cc:492] T 1fa8a0db61be45c3aa42edb11db4c5ff P c62716bf66c24330860573535b8b9430: Bootstrap starting.
I20260709 13:20:28.844694 20080 raft_consensus.cc:515] T 0413f22bd6d7447ca630b75be496555f P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:28.791548 19721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bdf042a063d4596970f49730ffa04b4" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
I20260709 13:20:28.861855 19721 raft_consensus.cc:3060] T 8bdf042a063d4596970f49730ffa04b4 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:28.862488 20132 leader_election.cc:290] T c393f9d18341477c97fac9a4af7e3adf P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:28.861543 19993 raft_consensus.cc:359] T 63868398f3604f529519d926d2831f4e P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:28.866590 19993 raft_consensus.cc:385] T 63868398f3604f529519d926d2831f4e P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:28.867887 19993 raft_consensus.cc:740] T 63868398f3604f529519d926d2831f4e P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Initialized, Role: FOLLOWER
I20260709 13:20:28.868093 20040 raft_consensus.cc:493] T 7e3a9d11df99411d8f0edb0df6f14d87 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:28.869921 20040 raft_consensus.cc:515] T 7e3a9d11df99411d8f0edb0df6f14d87 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:28.872407 20040 leader_election.cc:290] T 7e3a9d11df99411d8f0edb0df6f14d87 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:28.868902 19725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0413f22bd6d7447ca630b75be496555f" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
I20260709 13:20:28.873210 19722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3a9d11df99411d8f0edb0df6f14d87" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:28.873951 19993 consensus_queue.cc:260] T 63868398f3604f529519d926d2831f4e P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
W20260709 13:20:28.875175 19840 leader_election.cc:343] T 0413f22bd6d7447ca630b75be496555f P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:28.869405 20080 leader_election.cc:290] T 0413f22bd6d7447ca630b75be496555f P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 election: Requested vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:28.880462 19874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3a9d11df99411d8f0edb0df6f14d87" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:28.881772 19997 tablet_bootstrap.cc:654] T 8bdf042a063d4596970f49730ffa04b4 P b80b3a62d7d549a9a5c42def46769ac4: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:28.891918 19944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaced67782041fa9960c745301a96e7" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:28.897183 19993 ts_tablet_manager.cc:1434] T 63868398f3604f529519d926d2831f4e P e90ffed3ba174ababeb21d9415d4d918: Time spent starting tablet: real 0.048s	user 0.007s	sys 0.000s
I20260709 13:20:28.892773 19946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0413f22bd6d7447ca630b75be496555f" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430"
W20260709 13:20:28.898685 19690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:42643, user_credentials={real_user=slave}} blocked reactor thread for 55756.3us
I20260709 13:20:28.899928 19993 tablet_bootstrap.cc:492] T f012387a39df446bbd9dbea32dc03202 P e90ffed3ba174ababeb21d9415d4d918: Bootstrap starting.
I20260709 13:20:28.902139 19798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c393f9d18341477c97fac9a4af7e3adf" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:28.902361 19873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c393f9d18341477c97fac9a4af7e3adf" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:28.903018 19873 raft_consensus.cc:2468] T c393f9d18341477c97fac9a4af7e3adf P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 0.
I20260709 13:20:28.903367 20000 raft_consensus.cc:493] T 7a5476f5d68d496eaedce93e80bff5e4 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:28.903851 20000 raft_consensus.cc:515] T 7a5476f5d68d496eaedce93e80bff5e4 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:28.906149 20000 leader_election.cc:290] T 7a5476f5d68d496eaedce93e80bff5e4 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:28.907612 19721 raft_consensus.cc:2468] T 8bdf042a063d4596970f49730ffa04b4 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 1.
I20260709 13:20:28.908366 19995 tablet_bootstrap.cc:654] T 1fa8a0db61be45c3aa42edb11db4c5ff P c62716bf66c24330860573535b8b9430: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:28.908818 19946 raft_consensus.cc:3060] T 0413f22bd6d7447ca630b75be496555f P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:28.908918 19766 leader_election.cc:304] T 8bdf042a063d4596970f49730ffa04b4 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918; no voters: b80b3a62d7d549a9a5c42def46769ac4
I20260709 13:20:28.910316 20040 raft_consensus.cc:2804] T 8bdf042a063d4596970f49730ffa04b4 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:28.910435 19994 tablet_bootstrap.cc:492] T 0413f22bd6d7447ca630b75be496555f P 8df38c4c53c44e94b36c1e8992476d7c: No bootstrap required, opened a new log
I20260709 13:20:28.911033 19994 ts_tablet_manager.cc:1403] T 0413f22bd6d7447ca630b75be496555f P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.202s	user 0.036s	sys 0.034s
I20260709 13:20:28.911084 20040 raft_consensus.cc:697] T 8bdf042a063d4596970f49730ffa04b4 P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Becoming Leader. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Running, Role: LEADER
I20260709 13:20:28.912146 20040 consensus_queue.cc:237] T 8bdf042a063d4596970f49730ffa04b4 P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:28.909083 19690 leader_election.cc:304] T c393f9d18341477c97fac9a4af7e3adf P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4; no voters: 
I20260709 13:20:28.892669 19944 raft_consensus.cc:2468] T 3aaced67782041fa9960c745301a96e7 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 0.
I20260709 13:20:28.915777 19994 raft_consensus.cc:359] T 0413f22bd6d7447ca630b75be496555f P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:28.916477 19994 raft_consensus.cc:385] T 0413f22bd6d7447ca630b75be496555f P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:28.916743 19994 raft_consensus.cc:740] T 0413f22bd6d7447ca630b75be496555f P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
W20260709 13:20:28.917829 19690 leader_election.cc:343] T c393f9d18341477c97fac9a4af7e3adf P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:28.917485 19994 consensus_queue.cc:260] T 0413f22bd6d7447ca630b75be496555f P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:28.917135 20132 raft_consensus.cc:2804] T c393f9d18341477c97fac9a4af7e3adf P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:28.921277 20132 raft_consensus.cc:493] T c393f9d18341477c97fac9a4af7e3adf P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:28.921624 20132 raft_consensus.cc:3060] T c393f9d18341477c97fac9a4af7e3adf P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:28.922232 19839 leader_election.cc:304] T 3aaced67782041fa9960c745301a96e7 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:28.923492 20037 raft_consensus.cc:2804] T 3aaced67782041fa9960c745301a96e7 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:28.923877 20037 raft_consensus.cc:493] T 3aaced67782041fa9960c745301a96e7 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:28.925499 19994 ts_tablet_manager.cc:1434] T 0413f22bd6d7447ca630b75be496555f P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.002s
I20260709 13:20:28.908833 19722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5476f5d68d496eaedce93e80bff5e4" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:28.924233 20037 raft_consensus.cc:3060] T 3aaced67782041fa9960c745301a96e7 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:20:28.936959 19914 leader_election.cc:343] T 7a5476f5d68d496eaedce93e80bff5e4 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:28.933907 19874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5476f5d68d496eaedce93e80bff5e4" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:28.949223 19623 catalog_manager.cc:5697] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c reported cstate change: term changed from 0 to 1, leader changed from <none> to 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129). New cstate: current_term: 1 leader_uuid: "8df38c4c53c44e94b36c1e8992476d7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: UNKNOWN } } }
W20260709 13:20:28.955013 19913 leader_election.cc:343] T 7a5476f5d68d496eaedce93e80bff5e4 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:28.955597 19913 leader_election.cc:304] T 7a5476f5d68d496eaedce93e80bff5e4 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4
I20260709 13:20:28.958271 19997 tablet_bootstrap.cc:492] T 8bdf042a063d4596970f49730ffa04b4 P b80b3a62d7d549a9a5c42def46769ac4: No bootstrap required, opened a new log
I20260709 13:20:28.958886 19997 ts_tablet_manager.cc:1403] T 8bdf042a063d4596970f49730ffa04b4 P b80b3a62d7d549a9a5c42def46769ac4: Time spent bootstrapping tablet: real 0.165s	user 0.012s	sys 0.012s
I20260709 13:20:28.959334 20000 raft_consensus.cc:2749] T 7a5476f5d68d496eaedce93e80bff5e4 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:28.961550 19997 raft_consensus.cc:359] T 8bdf042a063d4596970f49730ffa04b4 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:28.962278 19997 raft_consensus.cc:385] T 8bdf042a063d4596970f49730ffa04b4 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:28.962531 19997 raft_consensus.cc:740] T 8bdf042a063d4596970f49730ffa04b4 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Initialized, Role: FOLLOWER
I20260709 13:20:28.968631 19993 tablet_bootstrap.cc:654] T f012387a39df446bbd9dbea32dc03202 P e90ffed3ba174ababeb21d9415d4d918: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:28.972615 19994 tablet_bootstrap.cc:492] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:28.974362 19765 leader_election.cc:304] T 7e3a9d11df99411d8f0edb0df6f14d87 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e90ffed3ba174ababeb21d9415d4d918; no voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4
I20260709 13:20:28.975700 20065 raft_consensus.cc:2749] T 7e3a9d11df99411d8f0edb0df6f14d87 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:28.978910 19997 consensus_queue.cc:260] T 8bdf042a063d4596970f49730ffa04b4 P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:28.984372 20132 raft_consensus.cc:515] T c393f9d18341477c97fac9a4af7e3adf P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:28.987056 20132 leader_election.cc:290] T c393f9d18341477c97fac9a4af7e3adf P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Requested vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:28.988404 19946 raft_consensus.cc:2468] T 0413f22bd6d7447ca630b75be496555f P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 1.
I20260709 13:20:28.981946 20037 raft_consensus.cc:515] T 3aaced67782041fa9960c745301a96e7 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:28.989646 19997 ts_tablet_manager.cc:1434] T 8bdf042a063d4596970f49730ffa04b4 P b80b3a62d7d549a9a5c42def46769ac4: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.000s
I20260709 13:20:28.991218 20040 raft_consensus.cc:493] T 63868398f3604f529519d926d2831f4e P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:28.991839 20040 raft_consensus.cc:515] T 63868398f3604f529519d926d2831f4e P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:28.993806 19839 leader_election.cc:304] T 0413f22bd6d7447ca630b75be496555f P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:28.994185 19721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaced67782041fa9960c745301a96e7" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
I20260709 13:20:28.994498 20040 leader_election.cc:290] T 63868398f3604f529519d926d2831f4e P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:28.994679 20080 raft_consensus.cc:2804] T 0413f22bd6d7447ca630b75be496555f P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:28.995204 20080 raft_consensus.cc:697] T 0413f22bd6d7447ca630b75be496555f P b80b3a62d7d549a9a5c42def46769ac4 [term 1 LEADER]: Becoming Leader. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Running, Role: LEADER
I20260709 13:20:28.996089 20080 consensus_queue.cc:237] T 0413f22bd6d7447ca630b75be496555f P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:28.997987 19799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c393f9d18341477c97fac9a4af7e3adf" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
W20260709 13:20:28.999701 19840 leader_election.cc:343] T 3aaced67782041fa9960c745301a96e7 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:29.000217 19994 tablet_bootstrap.cc:654] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:29.001149 19997 tablet_bootstrap.cc:492] T b832efa5a2e34b3894e33629110e6fef P b80b3a62d7d549a9a5c42def46769ac4: Bootstrap starting.
I20260709 13:20:29.001394 19946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaced67782041fa9960c745301a96e7" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:29.002038 19946 raft_consensus.cc:3060] T 3aaced67782041fa9960c745301a96e7 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:20:29.002895 19690 leader_election.cc:343] T c393f9d18341477c97fac9a4af7e3adf P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:29.006999 19944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63868398f3604f529519d926d2831f4e" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:29.007869 19944 raft_consensus.cc:2393] T 63868398f3604f529519d926d2831f4e P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e90ffed3ba174ababeb21d9415d4d918 in current term 1: Already voted for candidate 8df38c4c53c44e94b36c1e8992476d7c in this term.
I20260709 13:20:29.008209 19995 tablet_bootstrap.cc:492] T 1fa8a0db61be45c3aa42edb11db4c5ff P c62716bf66c24330860573535b8b9430: No bootstrap required, opened a new log
I20260709 13:20:29.008355 19874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c393f9d18341477c97fac9a4af7e3adf" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
I20260709 13:20:29.008700 19995 ts_tablet_manager.cc:1403] T 1fa8a0db61be45c3aa42edb11db4c5ff P c62716bf66c24330860573535b8b9430: Time spent bootstrapping tablet: real 0.153s	user 0.021s	sys 0.014s
I20260709 13:20:29.008852 19946 raft_consensus.cc:2468] T 3aaced67782041fa9960c745301a96e7 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 1.
I20260709 13:20:29.009023 19874 raft_consensus.cc:3060] T c393f9d18341477c97fac9a4af7e3adf P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:29.010020 19839 leader_election.cc:304] T 3aaced67782041fa9960c745301a96e7 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:29.010923 20009 raft_consensus.cc:2804] T 3aaced67782041fa9960c745301a96e7 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:29.011075 19721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63868398f3604f529519d926d2831f4e" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:29.011416 20009 raft_consensus.cc:697] T 3aaced67782041fa9960c745301a96e7 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 LEADER]: Becoming Leader. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Running, Role: LEADER
I20260709 13:20:29.011648 19995 raft_consensus.cc:359] T 1fa8a0db61be45c3aa42edb11db4c5ff P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:29.012317 20009 consensus_queue.cc:237] T 3aaced67782041fa9960c745301a96e7 P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:29.013345 19766 leader_election.cc:304] T 63868398f3604f529519d926d2831f4e P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e90ffed3ba174ababeb21d9415d4d918; no voters: 8df38c4c53c44e94b36c1e8992476d7c, c62716bf66c24330860573535b8b9430
I20260709 13:20:29.012320 19995 raft_consensus.cc:385] T 1fa8a0db61be45c3aa42edb11db4c5ff P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:29.014424 19995 raft_consensus.cc:740] T 1fa8a0db61be45c3aa42edb11db4c5ff P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62716bf66c24330860573535b8b9430, State: Initialized, Role: FOLLOWER
I20260709 13:20:29.001504 19622 catalog_manager.cc:5697] T 8bdf042a063d4596970f49730ffa04b4 P e90ffed3ba174ababeb21d9415d4d918 reported cstate change: term changed from 0 to 1, leader changed from <none> to e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "e90ffed3ba174ababeb21d9415d4d918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:29.015095 19995 consensus_queue.cc:260] T 1fa8a0db61be45c3aa42edb11db4c5ff P c62716bf66c24330860573535b8b9430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:29.015928 20173 raft_consensus.cc:3060] T 63868398f3604f529519d926d2831f4e P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:28.991261 20037 leader_election.cc:290] T 3aaced67782041fa9960c745301a96e7 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 election: Requested vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:29.021727 19799 raft_consensus.cc:3060] T 7e3a9d11df99411d8f0edb0df6f14d87 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:29.031721 19995 ts_tablet_manager.cc:1434] T 1fa8a0db61be45c3aa42edb11db4c5ff P c62716bf66c24330860573535b8b9430: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 13:20:29.033097 19995 tablet_bootstrap.cc:492] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430: Bootstrap starting.
I20260709 13:20:29.048892 20037 raft_consensus.cc:493] T 1fa8a0db61be45c3aa42edb11db4c5ff P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:29.050233 20037 raft_consensus.cc:515] T 1fa8a0db61be45c3aa42edb11db4c5ff P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:29.050809 20000 raft_consensus.cc:493] T 1cceb7f75d7040c2b98049590928b891 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:29.051496 20000 raft_consensus.cc:515] T 1cceb7f75d7040c2b98049590928b891 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:29.052258 20037 leader_election.cc:290] T 1fa8a0db61be45c3aa42edb11db4c5ff P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:29.055183 19721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cceb7f75d7040c2b98049590928b891" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:29.056720 19800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa8a0db61be45c3aa42edb11db4c5ff" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
W20260709 13:20:29.059974 19914 leader_election.cc:343] T 1cceb7f75d7040c2b98049590928b891 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:20:29.060153 19839 leader_election.cc:343] T 1fa8a0db61be45c3aa42edb11db4c5ff P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:29.063624 19800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cceb7f75d7040c2b98049590928b891" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:29.061425 20000 leader_election.cc:290] T 1cceb7f75d7040c2b98049590928b891 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
W20260709 13:20:29.066243 19913 leader_election.cc:343] T 1cceb7f75d7040c2b98049590928b891 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:29.067444 19946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa8a0db61be45c3aa42edb11db4c5ff" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:29.069814 19946 raft_consensus.cc:2468] T 1fa8a0db61be45c3aa42edb11db4c5ff P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 0.
I20260709 13:20:29.066726 19874 raft_consensus.cc:2468] T c393f9d18341477c97fac9a4af7e3adf P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 1.
I20260709 13:20:29.077464 19997 tablet_bootstrap.cc:654] T b832efa5a2e34b3894e33629110e6fef P b80b3a62d7d549a9a5c42def46769ac4: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:29.066896 19913 leader_election.cc:304] T 1cceb7f75d7040c2b98049590928b891 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:29.084375 20000 raft_consensus.cc:2749] T 1cceb7f75d7040c2b98049590928b891 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:29.087034 20173 raft_consensus.cc:2749] T 63868398f3604f529519d926d2831f4e P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:29.087186 19839 leader_election.cc:304] T 1fa8a0db61be45c3aa42edb11db4c5ff P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430; no voters: e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:29.088907 20037 raft_consensus.cc:2804] T 1fa8a0db61be45c3aa42edb11db4c5ff P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:29.089303 20037 raft_consensus.cc:493] T 1fa8a0db61be45c3aa42edb11db4c5ff P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:29.089628 20037 raft_consensus.cc:3060] T 1fa8a0db61be45c3aa42edb11db4c5ff P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:29.089864 19993 tablet_bootstrap.cc:492] T f012387a39df446bbd9dbea32dc03202 P e90ffed3ba174ababeb21d9415d4d918: No bootstrap required, opened a new log
I20260709 13:20:29.090364 19993 ts_tablet_manager.cc:1403] T f012387a39df446bbd9dbea32dc03202 P e90ffed3ba174ababeb21d9415d4d918: Time spent bootstrapping tablet: real 0.192s	user 0.021s	sys 0.015s
I20260709 13:20:29.092352 19995 tablet_bootstrap.cc:654] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:29.093139 19993 raft_consensus.cc:359] T f012387a39df446bbd9dbea32dc03202 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:29.093847 19993 raft_consensus.cc:385] T f012387a39df446bbd9dbea32dc03202 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:29.093779 20132 consensus_queue.cc:1048] T 63868398f3604f529519d926d2831f4e P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:20:29.094262 19993 raft_consensus.cc:740] T f012387a39df446bbd9dbea32dc03202 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Initialized, Role: FOLLOWER
I20260709 13:20:29.096386 19690 leader_election.cc:304] T c393f9d18341477c97fac9a4af7e3adf P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4; no voters: e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:29.097426 19993 consensus_queue.cc:260] T f012387a39df446bbd9dbea32dc03202 P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:29.097627 20083 raft_consensus.cc:2804] T c393f9d18341477c97fac9a4af7e3adf P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:29.099516 20083 raft_consensus.cc:697] T c393f9d18341477c97fac9a4af7e3adf P 8df38c4c53c44e94b36c1e8992476d7c [term 1 LEADER]: Becoming Leader. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Running, Role: LEADER
I20260709 13:20:29.103595 19993 ts_tablet_manager.cc:1434] T f012387a39df446bbd9dbea32dc03202 P e90ffed3ba174ababeb21d9415d4d918: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.004s
I20260709 13:20:29.104940 19993 tablet_bootstrap.cc:492] T e3ca37daad8341ad99cbba926f37232e P e90ffed3ba174ababeb21d9415d4d918: Bootstrap starting.
I20260709 13:20:29.108618 20083 consensus_queue.cc:237] T c393f9d18341477c97fac9a4af7e3adf P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:29.115200 19995 tablet_bootstrap.cc:492] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430: No bootstrap required, opened a new log
I20260709 13:20:29.115770 19995 ts_tablet_manager.cc:1403] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430: Time spent bootstrapping tablet: real 0.083s	user 0.009s	sys 0.008s
I20260709 13:20:29.120970 19622 catalog_manager.cc:5697] T 0413f22bd6d7447ca630b75be496555f P b80b3a62d7d549a9a5c42def46769ac4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131). New cstate: current_term: 1 leader_uuid: "b80b3a62d7d549a9a5c42def46769ac4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:29.125301 19995 raft_consensus.cc:359] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:29.127468 19995 raft_consensus.cc:385] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:29.127866 19995 raft_consensus.cc:740] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62716bf66c24330860573535b8b9430, State: Initialized, Role: FOLLOWER
I20260709 13:20:29.126834 19622 catalog_manager.cc:5697] T 3aaced67782041fa9960c745301a96e7 P b80b3a62d7d549a9a5c42def46769ac4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131). New cstate: current_term: 1 leader_uuid: "b80b3a62d7d549a9a5c42def46769ac4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:29.128727 19995 consensus_queue.cc:260] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:29.124788 20285 raft_consensus.cc:493] T 8bdf042a063d4596970f49730ffa04b4 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:29.133839 20285 raft_consensus.cc:515] T 8bdf042a063d4596970f49730ffa04b4 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:29.136054 20285 leader_election.cc:290] T 8bdf042a063d4596970f49730ffa04b4 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:29.136988 19994 tablet_bootstrap.cc:492] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c: No bootstrap required, opened a new log
I20260709 13:20:29.137521 19994 ts_tablet_manager.cc:1403] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.165s	user 0.010s	sys 0.013s
I20260709 13:20:29.137620 19721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bdf042a063d4596970f49730ffa04b4" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:29.138478 19721 raft_consensus.cc:2393] T 8bdf042a063d4596970f49730ffa04b4 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b80b3a62d7d549a9a5c42def46769ac4 in current term 1: Already voted for candidate e90ffed3ba174ababeb21d9415d4d918 in this term.
I20260709 13:20:29.141360 19995 ts_tablet_manager.cc:1434] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.002s
I20260709 13:20:29.142720 19995 tablet_bootstrap.cc:492] T ece79531bbd749d3b9300fe2fbeef9af P c62716bf66c24330860573535b8b9430: Bootstrap starting.
I20260709 13:20:29.146095 19798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bdf042a063d4596970f49730ffa04b4" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:29.149221 19839 leader_election.cc:304] T 8bdf042a063d4596970f49730ffa04b4 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4; no voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:29.152293 20065 consensus_queue.cc:1048] T 0c709292a0304b3c84a777427a754bb5 P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:29.152864 20285 raft_consensus.cc:3060] T 8bdf042a063d4596970f49730ffa04b4 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:29.156116 19620 catalog_manager.cc:5697] T c393f9d18341477c97fac9a4af7e3adf P 8df38c4c53c44e94b36c1e8992476d7c reported cstate change: term changed from 0 to 1, leader changed from <none> to 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129). New cstate: current_term: 1 leader_uuid: "8df38c4c53c44e94b36c1e8992476d7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:29.152158 19994 raft_consensus.cc:359] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:29.160629 19994 raft_consensus.cc:385] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:29.161046 19994 raft_consensus.cc:740] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:29.165793 19994 consensus_queue.cc:260] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:29.169803 19994 ts_tablet_manager.cc:1434] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.032s	user 0.009s	sys 0.001s
I20260709 13:20:29.177312 19997 tablet_bootstrap.cc:492] T b832efa5a2e34b3894e33629110e6fef P b80b3a62d7d549a9a5c42def46769ac4: No bootstrap required, opened a new log
I20260709 13:20:29.192106 19997 ts_tablet_manager.cc:1403] T b832efa5a2e34b3894e33629110e6fef P b80b3a62d7d549a9a5c42def46769ac4: Time spent bootstrapping tablet: real 0.191s	user 0.020s	sys 0.009s
I20260709 13:20:29.182948 20000 raft_consensus.cc:493] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:29.186549 20278 consensus_queue.cc:1048] T 63868398f3604f529519d926d2831f4e P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:20:29.190887 20285 raft_consensus.cc:2749] T 8bdf042a063d4596970f49730ffa04b4 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:29.182562 19993 tablet_bootstrap.cc:654] T e3ca37daad8341ad99cbba926f37232e P e90ffed3ba174ababeb21d9415d4d918: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:29.193487 20000 raft_consensus.cc:515] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:29.196640 19994 tablet_bootstrap.cc:492] T 3aaced67782041fa9960c745301a96e7 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:29.200245 20037 raft_consensus.cc:515] T 1fa8a0db61be45c3aa42edb11db4c5ff P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:29.202965 19873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7bc0eba7e94b9eb1eeec9bbd1a6e97" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:29.204344 20037 leader_election.cc:290] T 1fa8a0db61be45c3aa42edb11db4c5ff P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 election: Requested vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
W20260709 13:20:29.206394 19913 leader_election.cc:343] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:29.206823 19800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa8a0db61be45c3aa42edb11db4c5ff" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
I20260709 13:20:29.201086 20000 leader_election.cc:290] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:29.198127 20065 raft_consensus.cc:493] T f012387a39df446bbd9dbea32dc03202 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:29.210712 19946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa8a0db61be45c3aa42edb11db4c5ff" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:29.215878 19994 tablet_bootstrap.cc:654] T 3aaced67782041fa9960c745301a96e7 P 8df38c4c53c44e94b36c1e8992476d7c: Neither blocks nor log segments found. Creating new log.
W20260709 13:20:29.209110 19839 leader_election.cc:343] T 1fa8a0db61be45c3aa42edb11db4c5ff P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:29.218151 19946 raft_consensus.cc:3060] T 1fa8a0db61be45c3aa42edb11db4c5ff P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:29.219305 19800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7bc0eba7e94b9eb1eeec9bbd1a6e97" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:29.220568 20065 raft_consensus.cc:515] T f012387a39df446bbd9dbea32dc03202 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:29.226984 20040 consensus_queue.cc:1048] T 0c709292a0304b3c84a777427a754bb5 P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:20:29.227253 20275 consensus_queue.cc:1048] T f012387a39df446bbd9dbea32dc03202 P c62716bf66c24330860573535b8b9430 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:29.238302 19946 raft_consensus.cc:2468] T 1fa8a0db61be45c3aa42edb11db4c5ff P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 1.
I20260709 13:20:29.242551 19995 tablet_bootstrap.cc:654] T ece79531bbd749d3b9300fe2fbeef9af P c62716bf66c24330860573535b8b9430: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:29.247355 19839 leader_election.cc:304] T 1fa8a0db61be45c3aa42edb11db4c5ff P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430; no voters: e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:29.262542 20285 raft_consensus.cc:2804] T 1fa8a0db61be45c3aa42edb11db4c5ff P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:20:29.261389 19913 leader_election.cc:343] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:29.267789 20000 raft_consensus.cc:493] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:29.279351 19997 raft_consensus.cc:359] T b832efa5a2e34b3894e33629110e6fef P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:29.279943 19997 raft_consensus.cc:385] T b832efa5a2e34b3894e33629110e6fef P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:29.280169 19997 raft_consensus.cc:740] T b832efa5a2e34b3894e33629110e6fef P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Initialized, Role: FOLLOWER
I20260709 13:20:29.273658 20285 raft_consensus.cc:697] T 1fa8a0db61be45c3aa42edb11db4c5ff P b80b3a62d7d549a9a5c42def46769ac4 [term 1 LEADER]: Becoming Leader. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Running, Role: LEADER
I20260709 13:20:29.290105 20285 consensus_queue.cc:237] T 1fa8a0db61be45c3aa42edb11db4c5ff P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:29.273782 19913 leader_election.cc:304] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c62716bf66c24330860573535b8b9430; no voters: b80b3a62d7d549a9a5c42def46769ac4, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:29.296428 20237 raft_consensus.cc:2749] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:29.278455 20000 raft_consensus.cc:515] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:29.301743 20000 leader_election.cc:290] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:29.303476 20065 leader_election.cc:290] T f012387a39df446bbd9dbea32dc03202 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:29.307229 19993 tablet_bootstrap.cc:492] T e3ca37daad8341ad99cbba926f37232e P e90ffed3ba174ababeb21d9415d4d918: No bootstrap required, opened a new log
I20260709 13:20:29.307713 19993 ts_tablet_manager.cc:1403] T e3ca37daad8341ad99cbba926f37232e P e90ffed3ba174ababeb21d9415d4d918: Time spent bootstrapping tablet: real 0.203s	user 0.015s	sys 0.008s
I20260709 13:20:29.311136 19721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8abfa1aa35c44699960a8166fa79c41d" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:29.310376 19993 raft_consensus.cc:359] T e3ca37daad8341ad99cbba926f37232e P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:29.311941 19993 raft_consensus.cc:385] T e3ca37daad8341ad99cbba926f37232e P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:29.312368 19993 raft_consensus.cc:740] T e3ca37daad8341ad99cbba926f37232e P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Initialized, Role: FOLLOWER
W20260709 13:20:29.313915 19914 leader_election.cc:343] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:29.314260 19799 raft_consensus.cc:3060] T f012387a39df446bbd9dbea32dc03202 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:29.293184 20278 raft_consensus.cc:493] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:29.316306 20278 raft_consensus.cc:515] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:29.318578 20278 leader_election.cc:290] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:29.321023 19874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5476f5d68d496eaedce93e80bff5e4" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:29.321213 19942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5476f5d68d496eaedce93e80bff5e4" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:29.322139 19942 raft_consensus.cc:2468] T 7a5476f5d68d496eaedce93e80bff5e4 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 0.
W20260709 13:20:29.322995 19690 leader_election.cc:343] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:29.326288 19690 leader_election.cc:304] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, c62716bf66c24330860573535b8b9430; no voters: b80b3a62d7d549a9a5c42def46769ac4
I20260709 13:20:29.329895 20278 raft_consensus.cc:2804] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:29.330442 20278 raft_consensus.cc:493] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:29.330879 20278 raft_consensus.cc:3060] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:29.313298 19993 consensus_queue.cc:260] T e3ca37daad8341ad99cbba926f37232e P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:29.336953 19798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8abfa1aa35c44699960a8166fa79c41d" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:29.340149 19997 consensus_queue.cc:260] T b832efa5a2e34b3894e33629110e6fef P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:29.355044 19993 ts_tablet_manager.cc:1434] T e3ca37daad8341ad99cbba926f37232e P e90ffed3ba174ababeb21d9415d4d918: Time spent starting tablet: real 0.047s	user 0.005s	sys 0.002s
I20260709 13:20:29.353422 19872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f012387a39df446bbd9dbea32dc03202" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:29.365927 20298 consensus_queue.cc:1048] T 8bdf042a063d4596970f49730ffa04b4 P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:20:29.367427 20181 raft_consensus.cc:493] T 0413f22bd6d7447ca630b75be496555f P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:29.372485 20181 raft_consensus.cc:515] T 0413f22bd6d7447ca630b75be496555f P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:29.377888 19942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0413f22bd6d7447ca630b75be496555f" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:29.378755 19942 raft_consensus.cc:2393] T 0413f22bd6d7447ca630b75be496555f P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8df38c4c53c44e94b36c1e8992476d7c in current term 1: Already voted for candidate b80b3a62d7d549a9a5c42def46769ac4 in this term.
W20260709 13:20:29.377916 19913 leader_election.cc:343] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:29.383353 19913 leader_election.cc:304] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:29.385208 20237 raft_consensus.cc:2749] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:29.391657 19874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0413f22bd6d7447ca630b75be496555f" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:29.374567 20181 leader_election.cc:290] T 0413f22bd6d7447ca630b75be496555f P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:29.393231 19690 leader_election.cc:304] T 0413f22bd6d7447ca630b75be496555f P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c; no voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430
I20260709 13:20:29.393393 20275 consensus_queue.cc:1048] T f012387a39df446bbd9dbea32dc03202 P c62716bf66c24330860573535b8b9430 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:29.391948 19995 tablet_bootstrap.cc:492] T ece79531bbd749d3b9300fe2fbeef9af P c62716bf66c24330860573535b8b9430: No bootstrap required, opened a new log
I20260709 13:20:29.395998 20181 raft_consensus.cc:3060] T 0413f22bd6d7447ca630b75be496555f P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:29.400406 19944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f012387a39df446bbd9dbea32dc03202" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:29.403757 19765 leader_election.cc:304] T f012387a39df446bbd9dbea32dc03202 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e90ffed3ba174ababeb21d9415d4d918; no voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430
W20260709 13:20:29.406992 19839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:42643, user_credentials={real_user=slave}} blocked reactor thread for 63044.5us
I20260709 13:20:29.407793 20040 raft_consensus.cc:2749] T f012387a39df446bbd9dbea32dc03202 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:29.396476 19995 ts_tablet_manager.cc:1403] T ece79531bbd749d3b9300fe2fbeef9af P c62716bf66c24330860573535b8b9430: Time spent bootstrapping tablet: real 0.254s	user 0.014s	sys 0.007s
I20260709 13:20:29.362917 19997 ts_tablet_manager.cc:1434] T b832efa5a2e34b3894e33629110e6fef P b80b3a62d7d549a9a5c42def46769ac4: Time spent starting tablet: real 0.170s	user 0.006s	sys 0.002s
I20260709 13:20:29.399128 19994 tablet_bootstrap.cc:492] T 3aaced67782041fa9960c745301a96e7 P 8df38c4c53c44e94b36c1e8992476d7c: No bootstrap required, opened a new log
I20260709 13:20:29.411128 19994 ts_tablet_manager.cc:1403] T 3aaced67782041fa9960c745301a96e7 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.215s	user 0.010s	sys 0.016s
I20260709 13:20:29.410432 19997 tablet_bootstrap.cc:492] T dc5c36b41652443a8130592df8a66eb9 P b80b3a62d7d549a9a5c42def46769ac4: Bootstrap starting.
I20260709 13:20:29.423834 20132 consensus_queue.cc:1048] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:20:29.432323 19993 tablet_bootstrap.cc:492] T c393f9d18341477c97fac9a4af7e3adf P e90ffed3ba174ababeb21d9415d4d918: Bootstrap starting.
I20260709 13:20:29.440186 19997 tablet_bootstrap.cc:654] T dc5c36b41652443a8130592df8a66eb9 P b80b3a62d7d549a9a5c42def46769ac4: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:29.449806 19995 raft_consensus.cc:359] T ece79531bbd749d3b9300fe2fbeef9af P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:29.451084 19995 raft_consensus.cc:385] T ece79531bbd749d3b9300fe2fbeef9af P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:29.451428 19995 raft_consensus.cc:740] T ece79531bbd749d3b9300fe2fbeef9af P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62716bf66c24330860573535b8b9430, State: Initialized, Role: FOLLOWER
I20260709 13:20:29.452217 19995 consensus_queue.cc:260] T ece79531bbd749d3b9300fe2fbeef9af P c62716bf66c24330860573535b8b9430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:29.457332 19995 ts_tablet_manager.cc:1434] T ece79531bbd749d3b9300fe2fbeef9af P c62716bf66c24330860573535b8b9430: Time spent starting tablet: real 0.048s	user 0.008s	sys 0.000s
I20260709 13:20:29.461731 20181 raft_consensus.cc:2749] T 0413f22bd6d7447ca630b75be496555f P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:29.458606 19995 tablet_bootstrap.cc:492] T b832efa5a2e34b3894e33629110e6fef P c62716bf66c24330860573535b8b9430: Bootstrap starting.
I20260709 13:20:29.370333 20278 raft_consensus.cc:515] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:29.468689 20300 raft_consensus.cc:493] T e3ca37daad8341ad99cbba926f37232e P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:29.477451 20300 raft_consensus.cc:515] T e3ca37daad8341ad99cbba926f37232e P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
W20260709 13:20:29.478236 19690 consensus_peers.cc:597] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c -> Peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Couldn't send request to peer e90ffed3ba174ababeb21d9415d4d918. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:29.479774 20300 leader_election.cc:290] T e3ca37daad8341ad99cbba926f37232e P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:29.486517 19722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3ca37daad8341ad99cbba926f37232e" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
W20260709 13:20:29.488346 19766 leader_election.cc:343] T e3ca37daad8341ad99cbba926f37232e P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:29.490070 19944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3ca37daad8341ad99cbba926f37232e" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:29.490823 19944 raft_consensus.cc:2468] T e3ca37daad8341ad99cbba926f37232e P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 0.
I20260709 13:20:29.491914 20297 consensus_queue.cc:1048] T 8bdf042a063d4596970f49730ffa04b4 P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:29.498903 19765 leader_election.cc:304] T e3ca37daad8341ad99cbba926f37232e P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c62716bf66c24330860573535b8b9430, e90ffed3ba174ababeb21d9415d4d918; no voters: 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:29.500342 20212 raft_consensus.cc:2804] T e3ca37daad8341ad99cbba926f37232e P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:29.500765 20212 raft_consensus.cc:493] T e3ca37daad8341ad99cbba926f37232e P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:29.501125 20212 raft_consensus.cc:3060] T e3ca37daad8341ad99cbba926f37232e P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:29.501590 19993 tablet_bootstrap.cc:654] T c393f9d18341477c97fac9a4af7e3adf P e90ffed3ba174ababeb21d9415d4d918: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:29.504807 20278 leader_election.cc:290] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Requested vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:29.506222 19874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5476f5d68d496eaedce93e80bff5e4" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
I20260709 13:20:29.506683 19946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5476f5d68d496eaedce93e80bff5e4" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:29.507297 19946 raft_consensus.cc:3060] T 7a5476f5d68d496eaedce93e80bff5e4 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:20:29.507906 19690 leader_election.cc:343] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:29.509228 20009 raft_consensus.cc:493] T b832efa5a2e34b3894e33629110e6fef P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:29.510722 19840 consensus_peers.cc:597] T 3aaced67782041fa9960c745301a96e7 P b80b3a62d7d549a9a5c42def46769ac4 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:29.509718 20009 raft_consensus.cc:515] T b832efa5a2e34b3894e33629110e6fef P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:29.514164 19798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b832efa5a2e34b3894e33629110e6fef" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:29.515156 20009 leader_election.cc:290] T b832efa5a2e34b3894e33629110e6fef P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
W20260709 13:20:29.515755 19839 leader_election.cc:343] T b832efa5a2e34b3894e33629110e6fef P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:29.517664 19620 catalog_manager.cc:5697] T 1fa8a0db61be45c3aa42edb11db4c5ff P b80b3a62d7d549a9a5c42def46769ac4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131). New cstate: current_term: 1 leader_uuid: "b80b3a62d7d549a9a5c42def46769ac4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:29.519649 19946 raft_consensus.cc:2468] T 7a5476f5d68d496eaedce93e80bff5e4 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 1.
I20260709 13:20:29.520017 19944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b832efa5a2e34b3894e33629110e6fef" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:29.509585 19994 raft_consensus.cc:359] T 3aaced67782041fa9960c745301a96e7 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:29.525641 19994 raft_consensus.cc:385] T 3aaced67782041fa9960c745301a96e7 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:29.525993 19994 raft_consensus.cc:740] T 3aaced67782041fa9960c745301a96e7 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:29.527410 19690 leader_election.cc:304] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, c62716bf66c24330860573535b8b9430; no voters: b80b3a62d7d549a9a5c42def46769ac4
I20260709 13:20:29.527364 19994 consensus_queue.cc:260] T 3aaced67782041fa9960c745301a96e7 P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
W20260709 13:20:29.530033 19839 leader_election.cc:343] T b832efa5a2e34b3894e33629110e6fef P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c62716bf66c24330860573535b8b9430 (127.9.116.132:42643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:29.530668 19839 leader_election.cc:304] T b832efa5a2e34b3894e33629110e6fef P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4; no voters: c62716bf66c24330860573535b8b9430, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:29.531905 20285 raft_consensus.cc:2749] T b832efa5a2e34b3894e33629110e6fef P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:29.525142 20212 raft_consensus.cc:515] T e3ca37daad8341ad99cbba926f37232e P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:29.535399 19994 ts_tablet_manager.cc:1434] T 3aaced67782041fa9960c745301a96e7 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.124s	user 0.008s	sys 0.000s
I20260709 13:20:29.536109 20212 leader_election.cc:290] T e3ca37daad8341ad99cbba926f37232e P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 election: Requested vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:29.535861 19946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3ca37daad8341ad99cbba926f37232e" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:29.536970 19946 raft_consensus.cc:3060] T e3ca37daad8341ad99cbba926f37232e P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:29.532670 20278 raft_consensus.cc:2804] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:29.539112 19725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3ca37daad8341ad99cbba926f37232e" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
I20260709 13:20:29.540283 20278 raft_consensus.cc:697] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 LEADER]: Becoming Leader. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Running, Role: LEADER
I20260709 13:20:29.541208 20278 consensus_queue.cc:237] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
W20260709 13:20:29.543486 19766 leader_election.cc:343] T e3ca37daad8341ad99cbba926f37232e P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:29.545749 19995 tablet_bootstrap.cc:654] T b832efa5a2e34b3894e33629110e6fef P c62716bf66c24330860573535b8b9430: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:29.536497 19994 tablet_bootstrap.cc:492] T e3ca37daad8341ad99cbba926f37232e P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:29.550453 19946 raft_consensus.cc:2468] T e3ca37daad8341ad99cbba926f37232e P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 1.
I20260709 13:20:29.556689 19765 leader_election.cc:304] T e3ca37daad8341ad99cbba926f37232e P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c62716bf66c24330860573535b8b9430, e90ffed3ba174ababeb21d9415d4d918; no voters: 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:29.558223 20300 raft_consensus.cc:2804] T e3ca37daad8341ad99cbba926f37232e P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:29.558741 20300 raft_consensus.cc:697] T e3ca37daad8341ad99cbba926f37232e P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Becoming Leader. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Running, Role: LEADER
I20260709 13:20:29.559612 20300 consensus_queue.cc:237] T e3ca37daad8341ad99cbba926f37232e P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:29.564777 20285 consensus_queue.cc:1048] T 0413f22bd6d7447ca630b75be496555f P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:20:29.565665 19690 consensus_peers.cc:597] T c393f9d18341477c97fac9a4af7e3adf P 8df38c4c53c44e94b36c1e8992476d7c -> Peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Couldn't send request to peer e90ffed3ba174ababeb21d9415d4d918. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:29.569701 20132 consensus_queue.cc:1048] T c393f9d18341477c97fac9a4af7e3adf P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:20:29.577562 19997 tablet_bootstrap.cc:492] T dc5c36b41652443a8130592df8a66eb9 P b80b3a62d7d549a9a5c42def46769ac4: No bootstrap required, opened a new log
I20260709 13:20:29.578176 19997 ts_tablet_manager.cc:1403] T dc5c36b41652443a8130592df8a66eb9 P b80b3a62d7d549a9a5c42def46769ac4: Time spent bootstrapping tablet: real 0.168s	user 0.013s	sys 0.003s
I20260709 13:20:29.580153 20285 consensus_queue.cc:1048] T 3aaced67782041fa9960c745301a96e7 P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:29.595326 20285 consensus_queue.cc:1048] T 0413f22bd6d7447ca630b75be496555f P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:20:29.601577 19994 tablet_bootstrap.cc:654] T e3ca37daad8341ad99cbba926f37232e P 8df38c4c53c44e94b36c1e8992476d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:29.614230 19997 raft_consensus.cc:359] T dc5c36b41652443a8130592df8a66eb9 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:29.615309 19997 raft_consensus.cc:385] T dc5c36b41652443a8130592df8a66eb9 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:29.615657 19997 raft_consensus.cc:740] T dc5c36b41652443a8130592df8a66eb9 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Initialized, Role: FOLLOWER
I20260709 13:20:29.616643 19997 consensus_queue.cc:260] T dc5c36b41652443a8130592df8a66eb9 P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:29.620245 19997 ts_tablet_manager.cc:1434] T dc5c36b41652443a8130592df8a66eb9 P b80b3a62d7d549a9a5c42def46769ac4: Time spent starting tablet: real 0.042s	user 0.005s	sys 0.003s
I20260709 13:20:29.621562 19997 tablet_bootstrap.cc:492] T bc04a038495940c1bbeda4a2e6f17b82 P b80b3a62d7d549a9a5c42def46769ac4: Bootstrap starting.
I20260709 13:20:29.652189 19995 tablet_bootstrap.cc:492] T b832efa5a2e34b3894e33629110e6fef P c62716bf66c24330860573535b8b9430: No bootstrap required, opened a new log
I20260709 13:20:29.652765 19995 ts_tablet_manager.cc:1403] T b832efa5a2e34b3894e33629110e6fef P c62716bf66c24330860573535b8b9430: Time spent bootstrapping tablet: real 0.194s	user 0.021s	sys 0.011s
I20260709 13:20:29.653016 19997 tablet_bootstrap.cc:654] T bc04a038495940c1bbeda4a2e6f17b82 P b80b3a62d7d549a9a5c42def46769ac4: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:29.655616 19995 raft_consensus.cc:359] T b832efa5a2e34b3894e33629110e6fef P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:29.656387 19995 raft_consensus.cc:385] T b832efa5a2e34b3894e33629110e6fef P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:29.657612 19995 raft_consensus.cc:740] T b832efa5a2e34b3894e33629110e6fef P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62716bf66c24330860573535b8b9430, State: Initialized, Role: FOLLOWER
I20260709 13:20:29.658675 19993 tablet_bootstrap.cc:492] T c393f9d18341477c97fac9a4af7e3adf P e90ffed3ba174ababeb21d9415d4d918: No bootstrap required, opened a new log
I20260709 13:20:29.659307 19993 ts_tablet_manager.cc:1403] T c393f9d18341477c97fac9a4af7e3adf P e90ffed3ba174ababeb21d9415d4d918: Time spent bootstrapping tablet: real 0.227s	user 0.015s	sys 0.043s
I20260709 13:20:29.659171 19995 consensus_queue.cc:260] T b832efa5a2e34b3894e33629110e6fef P c62716bf66c24330860573535b8b9430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:29.668931 19995 ts_tablet_manager.cc:1434] T b832efa5a2e34b3894e33629110e6fef P c62716bf66c24330860573535b8b9430: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:20:29.687147 19995 tablet_bootstrap.cc:492] T bc04a038495940c1bbeda4a2e6f17b82 P c62716bf66c24330860573535b8b9430: Bootstrap starting.
I20260709 13:20:29.692332 19620 catalog_manager.cc:5697] T e3ca37daad8341ad99cbba926f37232e P e90ffed3ba174ababeb21d9415d4d918 reported cstate change: term changed from 0 to 1, leader changed from <none> to e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "e90ffed3ba174ababeb21d9415d4d918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:29.713713 19995 tablet_bootstrap.cc:654] T bc04a038495940c1bbeda4a2e6f17b82 P c62716bf66c24330860573535b8b9430: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:29.726330 19993 raft_consensus.cc:359] T c393f9d18341477c97fac9a4af7e3adf P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:29.727505 19993 raft_consensus.cc:385] T c393f9d18341477c97fac9a4af7e3adf P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:29.727890 19993 raft_consensus.cc:740] T c393f9d18341477c97fac9a4af7e3adf P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Initialized, Role: FOLLOWER
I20260709 13:20:29.728899 19993 consensus_queue.cc:260] T c393f9d18341477c97fac9a4af7e3adf P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:29.741597 19994 tablet_bootstrap.cc:492] T e3ca37daad8341ad99cbba926f37232e P 8df38c4c53c44e94b36c1e8992476d7c: No bootstrap required, opened a new log
I20260709 13:20:29.742165 19994 ts_tablet_manager.cc:1403] T e3ca37daad8341ad99cbba926f37232e P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.206s	user 0.017s	sys 0.003s
I20260709 13:20:29.745504 19997 tablet_bootstrap.cc:492] T bc04a038495940c1bbeda4a2e6f17b82 P b80b3a62d7d549a9a5c42def46769ac4: No bootstrap required, opened a new log
I20260709 13:20:29.746052 19997 ts_tablet_manager.cc:1403] T bc04a038495940c1bbeda4a2e6f17b82 P b80b3a62d7d549a9a5c42def46769ac4: Time spent bootstrapping tablet: real 0.125s	user 0.008s	sys 0.020s
I20260709 13:20:29.748967 19997 raft_consensus.cc:359] T bc04a038495940c1bbeda4a2e6f17b82 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:29.749682 19997 raft_consensus.cc:385] T bc04a038495940c1bbeda4a2e6f17b82 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:20:29.749851 19839 consensus_peers.cc:597] T 1fa8a0db61be45c3aa42edb11db4c5ff P b80b3a62d7d549a9a5c42def46769ac4 -> Peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Couldn't send request to peer e90ffed3ba174ababeb21d9415d4d918. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:29.749969 19997 raft_consensus.cc:740] T bc04a038495940c1bbeda4a2e6f17b82 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Initialized, Role: FOLLOWER
I20260709 13:20:29.750769 19997 consensus_queue.cc:260] T bc04a038495940c1bbeda4a2e6f17b82 P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:29.754566 19997 ts_tablet_manager.cc:1434] T bc04a038495940c1bbeda4a2e6f17b82 P b80b3a62d7d549a9a5c42def46769ac4: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.007s
I20260709 13:20:29.755561 19997 tablet_bootstrap.cc:492] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4: Bootstrap starting.
I20260709 13:20:29.761520 19997 tablet_bootstrap.cc:654] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:29.774396 20278 raft_consensus.cc:493] T 3aaced67782041fa9960c745301a96e7 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:29.775012 20278 raft_consensus.cc:515] T 3aaced67782041fa9960c745301a96e7 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:29.777174 20278 leader_election.cc:290] T 3aaced67782041fa9960c745301a96e7 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:29.790444 19874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaced67782041fa9960c745301a96e7" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:29.790874 19944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaced67782041fa9960c745301a96e7" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:29.796380 20080 raft_consensus.cc:493] T dc5c36b41652443a8130592df8a66eb9 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:29.796968 20080 raft_consensus.cc:515] T dc5c36b41652443a8130592df8a66eb9 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:29.798925 19993 ts_tablet_manager.cc:1434] T c393f9d18341477c97fac9a4af7e3adf P e90ffed3ba174ababeb21d9415d4d918: Time spent starting tablet: real 0.139s	user 0.009s	sys 0.002s
I20260709 13:20:29.799757 19622 catalog_manager.cc:5697] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c reported cstate change: term changed from 0 to 1, leader changed from <none> to 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129). New cstate: current_term: 1 leader_uuid: "8df38c4c53c44e94b36c1e8992476d7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:29.800691 19690 leader_election.cc:304] T 3aaced67782041fa9960c745301a96e7 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c; no voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430
I20260709 13:20:29.808203 19993 tablet_bootstrap.cc:492] T 1fa8a0db61be45c3aa42edb11db4c5ff P e90ffed3ba174ababeb21d9415d4d918: Bootstrap starting.
I20260709 13:20:29.808189 20132 raft_consensus.cc:2749] T 3aaced67782041fa9960c745301a96e7 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:29.805043 20080 leader_election.cc:290] T dc5c36b41652443a8130592df8a66eb9 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:29.814256 19994 raft_consensus.cc:359] T e3ca37daad8341ad99cbba926f37232e P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:29.815104 19994 raft_consensus.cc:385] T e3ca37daad8341ad99cbba926f37232e P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:29.820811 19797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5c36b41652443a8130592df8a66eb9" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:29.824643 19722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5c36b41652443a8130592df8a66eb9" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
W20260709 13:20:29.830673 19840 leader_election.cc:343] T dc5c36b41652443a8130592df8a66eb9 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:20:29.836269 19839 leader_election.cc:343] T dc5c36b41652443a8130592df8a66eb9 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:29.836766 19839 leader_election.cc:304] T dc5c36b41652443a8130592df8a66eb9 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4; no voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918
W20260709 13:20:29.838624 19670 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:20:29.838851 20080 raft_consensus.cc:2749] T dc5c36b41652443a8130592df8a66eb9 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:29.832634 19995 tablet_bootstrap.cc:492] T bc04a038495940c1bbeda4a2e6f17b82 P c62716bf66c24330860573535b8b9430: No bootstrap required, opened a new log
I20260709 13:20:29.842511 19995 ts_tablet_manager.cc:1403] T bc04a038495940c1bbeda4a2e6f17b82 P c62716bf66c24330860573535b8b9430: Time spent bootstrapping tablet: real 0.156s	user 0.021s	sys 0.003s
I20260709 13:20:29.815407 19994 raft_consensus.cc:740] T e3ca37daad8341ad99cbba926f37232e P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:29.846860 19994 consensus_queue.cc:260] T e3ca37daad8341ad99cbba926f37232e P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:29.851111 19994 ts_tablet_manager.cc:1434] T e3ca37daad8341ad99cbba926f37232e P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.109s	user 0.008s	sys 0.003s
I20260709 13:20:29.853035 19994 tablet_bootstrap.cc:492] T 563940ad96444ce59e393ea303cc3352 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:29.856204 19993 tablet_bootstrap.cc:654] T 1fa8a0db61be45c3aa42edb11db4c5ff P e90ffed3ba174ababeb21d9415d4d918: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:29.857239 19995 raft_consensus.cc:359] T bc04a038495940c1bbeda4a2e6f17b82 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:29.858577 19995 raft_consensus.cc:385] T bc04a038495940c1bbeda4a2e6f17b82 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:29.859464 19994 tablet_bootstrap.cc:654] T 563940ad96444ce59e393ea303cc3352 P 8df38c4c53c44e94b36c1e8992476d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:29.861141 19995 raft_consensus.cc:740] T bc04a038495940c1bbeda4a2e6f17b82 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62716bf66c24330860573535b8b9430, State: Initialized, Role: FOLLOWER
I20260709 13:20:29.862076 19995 consensus_queue.cc:260] T bc04a038495940c1bbeda4a2e6f17b82 P c62716bf66c24330860573535b8b9430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
W20260709 13:20:29.906616 19690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:43319, user_credentials={real_user=slave}} blocked reactor thread for 100997us
I20260709 13:20:29.922957 19997 tablet_bootstrap.cc:492] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4: No bootstrap required, opened a new log
I20260709 13:20:29.924822 19997 ts_tablet_manager.cc:1403] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4: Time spent bootstrapping tablet: real 0.169s	user 0.028s	sys 0.057s
I20260709 13:20:29.935163 19997 raft_consensus.cc:359] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:29.936721 19995 ts_tablet_manager.cc:1434] T bc04a038495940c1bbeda4a2e6f17b82 P c62716bf66c24330860573535b8b9430: Time spent starting tablet: real 0.094s	user 0.006s	sys 0.001s
I20260709 13:20:29.937116 19997 raft_consensus.cc:385] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:29.939611 20275 raft_consensus.cc:493] T ece79531bbd749d3b9300fe2fbeef9af P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:29.940814 20275 raft_consensus.cc:515] T ece79531bbd749d3b9300fe2fbeef9af P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:29.937692 19997 raft_consensus.cc:740] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Initialized, Role: FOLLOWER
I20260709 13:20:29.949793 20275 leader_election.cc:290] T ece79531bbd749d3b9300fe2fbeef9af P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:29.969967 19995 tablet_bootstrap.cc:492] T 9f60d4b859264720a20f5670c3c71a6f P c62716bf66c24330860573535b8b9430: Bootstrap starting.
I20260709 13:20:29.977301 20275 raft_consensus.cc:493] T b832efa5a2e34b3894e33629110e6fef P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:29.978612 20275 raft_consensus.cc:515] T b832efa5a2e34b3894e33629110e6fef P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:29.980108 19874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece79531bbd749d3b9300fe2fbeef9af" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:30.001001 19993 tablet_bootstrap.cc:492] T 1fa8a0db61be45c3aa42edb11db4c5ff P e90ffed3ba174ababeb21d9415d4d918: No bootstrap required, opened a new log
I20260709 13:20:29.991714 19995 tablet_bootstrap.cc:654] T 9f60d4b859264720a20f5670c3c71a6f P c62716bf66c24330860573535b8b9430: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:30.024534 19993 ts_tablet_manager.cc:1403] T 1fa8a0db61be45c3aa42edb11db4c5ff P e90ffed3ba174ababeb21d9415d4d918: Time spent bootstrapping tablet: real 0.217s	user 0.020s	sys 0.000s
I20260709 13:20:29.985349 20275 leader_election.cc:290] T b832efa5a2e34b3894e33629110e6fef P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:30.021560 19721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece79531bbd749d3b9300fe2fbeef9af" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:30.029292 19993 raft_consensus.cc:359] T 1fa8a0db61be45c3aa42edb11db4c5ff P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:30.030277 19993 raft_consensus.cc:385] T 1fa8a0db61be45c3aa42edb11db4c5ff P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:20:30.031494 19913 leader_election.cc:343] T ece79531bbd749d3b9300fe2fbeef9af P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:30.030658 19993 raft_consensus.cc:740] T 1fa8a0db61be45c3aa42edb11db4c5ff P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Initialized, Role: FOLLOWER
I20260709 13:20:30.022850 20037 raft_consensus.cc:493] T bc04a038495940c1bbeda4a2e6f17b82 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:30.038903 20037 raft_consensus.cc:515] T bc04a038495940c1bbeda4a2e6f17b82 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:30.044037 19874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b832efa5a2e34b3894e33629110e6fef" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:30.045405 19874 raft_consensus.cc:2468] T b832efa5a2e34b3894e33629110e6fef P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c62716bf66c24330860573535b8b9430 in term 0.
I20260709 13:20:30.075291 19913 leader_election.cc:304] T b832efa5a2e34b3894e33629110e6fef P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430; no voters: 
I20260709 13:20:30.076352 20275 raft_consensus.cc:2804] T b832efa5a2e34b3894e33629110e6fef P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:30.046715 19942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc04a038495940c1bbeda4a2e6f17b82" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:30.051379 19993 consensus_queue.cc:260] T 1fa8a0db61be45c3aa42edb11db4c5ff P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:30.055666 19722 raft_consensus.cc:3060] T 3aaced67782041fa9960c745301a96e7 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:30.077190 19942 raft_consensus.cc:2468] T bc04a038495940c1bbeda4a2e6f17b82 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 0.
I20260709 13:20:30.079419 19839 leader_election.cc:304] T bc04a038495940c1bbeda4a2e6f17b82 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430; no voters: 
I20260709 13:20:30.080916 19993 ts_tablet_manager.cc:1434] T 1fa8a0db61be45c3aa42edb11db4c5ff P e90ffed3ba174ababeb21d9415d4d918: Time spent starting tablet: real 0.056s	user 0.011s	sys 0.000s
I20260709 13:20:30.083199 20009 raft_consensus.cc:2804] T bc04a038495940c1bbeda4a2e6f17b82 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:30.083793 20009 raft_consensus.cc:493] T bc04a038495940c1bbeda4a2e6f17b82 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:30.084321 20009 raft_consensus.cc:3060] T bc04a038495940c1bbeda4a2e6f17b82 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:29.983108 20079 consensus_queue.cc:1048] T 1fa8a0db61be45c3aa42edb11db4c5ff P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.076s
I20260709 13:20:30.076751 20275 raft_consensus.cc:493] T b832efa5a2e34b3894e33629110e6fef P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:30.087167 20275 raft_consensus.cc:3060] T b832efa5a2e34b3894e33629110e6fef P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:30.083422 19993 tablet_bootstrap.cc:492] T d5f8619b1f9f46adaa41774af856d9b1 P e90ffed3ba174ababeb21d9415d4d918: Bootstrap starting.
I20260709 13:20:30.091509 20037 leader_election.cc:290] T bc04a038495940c1bbeda4a2e6f17b82 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:30.093425 19721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc04a038495940c1bbeda4a2e6f17b82" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:30.093899 20298 raft_consensus.cc:493] T 1fa8a0db61be45c3aa42edb11db4c5ff P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:30.094565 20298 raft_consensus.cc:515] T 1fa8a0db61be45c3aa42edb11db4c5ff P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
W20260709 13:20:30.095580 19840 leader_election.cc:343] T bc04a038495940c1bbeda4a2e6f17b82 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:20:30.073673 19914 leader_election.cc:343] T ece79531bbd749d3b9300fe2fbeef9af P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:30.099069 19914 leader_election.cc:304] T ece79531bbd749d3b9300fe2fbeef9af P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4
I20260709 13:20:30.097805 20298 leader_election.cc:290] T 1fa8a0db61be45c3aa42edb11db4c5ff P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:30.100971 20237 raft_consensus.cc:2749] T ece79531bbd749d3b9300fe2fbeef9af P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:30.106941 19993 tablet_bootstrap.cc:654] T d5f8619b1f9f46adaa41774af856d9b1 P e90ffed3ba174ababeb21d9415d4d918: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:30.107175 19997 consensus_queue.cc:260] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:30.109153 19997 ts_tablet_manager.cc:1434] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4: Time spent starting tablet: real 0.184s	user 0.013s	sys 0.000s
I20260709 13:20:30.115803 19997 tablet_bootstrap.cc:492] T ece79531bbd749d3b9300fe2fbeef9af P b80b3a62d7d549a9a5c42def46769ac4: Bootstrap starting.
I20260709 13:20:30.126919 20009 raft_consensus.cc:515] T bc04a038495940c1bbeda4a2e6f17b82 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:30.129388 19995 tablet_bootstrap.cc:492] T 9f60d4b859264720a20f5670c3c71a6f P c62716bf66c24330860573535b8b9430: No bootstrap required, opened a new log
I20260709 13:20:30.129971 19995 ts_tablet_manager.cc:1403] T 9f60d4b859264720a20f5670c3c71a6f P c62716bf66c24330860573535b8b9430: Time spent bootstrapping tablet: real 0.161s	user 0.026s	sys 0.005s
I20260709 13:20:30.130154 19944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc04a038495940c1bbeda4a2e6f17b82" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:30.130944 19944 raft_consensus.cc:3060] T bc04a038495940c1bbeda4a2e6f17b82 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:30.133630 19995 raft_consensus.cc:359] T 9f60d4b859264720a20f5670c3c71a6f P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:30.134421 19995 raft_consensus.cc:385] T 9f60d4b859264720a20f5670c3c71a6f P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:20:30.135454 19690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:42643, user_credentials={real_user=slave}} blocked reactor thread for 70622.3us
I20260709 13:20:30.134703 19995 raft_consensus.cc:740] T 9f60d4b859264720a20f5670c3c71a6f P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62716bf66c24330860573535b8b9430, State: Initialized, Role: FOLLOWER
I20260709 13:20:30.136099 20009 leader_election.cc:290] T bc04a038495940c1bbeda4a2e6f17b82 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 election: Requested vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:30.136946 19995 consensus_queue.cc:260] T 9f60d4b859264720a20f5670c3c71a6f P c62716bf66c24330860573535b8b9430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:30.151424 19721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc04a038495940c1bbeda4a2e6f17b82" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
I20260709 13:20:30.156430 19995 ts_tablet_manager.cc:1434] T 9f60d4b859264720a20f5670c3c71a6f P c62716bf66c24330860573535b8b9430: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.004s
I20260709 13:20:30.170459 19948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa8a0db61be45c3aa42edb11db4c5ff" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:30.174700 19800 raft_consensus.cc:3060] T c393f9d18341477c97fac9a4af7e3adf P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:20:30.182087 19840 leader_election.cc:343] T bc04a038495940c1bbeda4a2e6f17b82 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:30.190260 19721 raft_consensus.cc:3060] T e3ca37daad8341ad99cbba926f37232e P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:30.180810 20275 raft_consensus.cc:515] T b832efa5a2e34b3894e33629110e6fef P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:30.194187 20275 leader_election.cc:290] T b832efa5a2e34b3894e33629110e6fef P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 election: Requested vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:30.207085 19868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b832efa5a2e34b3894e33629110e6fef" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
I20260709 13:20:30.207944 19868 raft_consensus.cc:3060] T b832efa5a2e34b3894e33629110e6fef P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:30.216818 19870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa8a0db61be45c3aa42edb11db4c5ff" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:30.217485 19997 tablet_bootstrap.cc:654] T ece79531bbd749d3b9300fe2fbeef9af P b80b3a62d7d549a9a5c42def46769ac4: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:30.224752 19798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b832efa5a2e34b3894e33629110e6fef" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:30.227975 19799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b832efa5a2e34b3894e33629110e6fef" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
I20260709 13:20:30.238518 19944 raft_consensus.cc:2468] T bc04a038495940c1bbeda4a2e6f17b82 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 1.
I20260709 13:20:30.240419 19993 tablet_bootstrap.cc:492] T d5f8619b1f9f46adaa41774af856d9b1 P e90ffed3ba174ababeb21d9415d4d918: No bootstrap required, opened a new log
I20260709 13:20:30.241573 19993 ts_tablet_manager.cc:1403] T d5f8619b1f9f46adaa41774af856d9b1 P e90ffed3ba174ababeb21d9415d4d918: Time spent bootstrapping tablet: real 0.159s	user 0.020s	sys 0.011s
I20260709 13:20:30.243494 19839 leader_election.cc:304] T bc04a038495940c1bbeda4a2e6f17b82 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:30.248090 20009 raft_consensus.cc:2804] T bc04a038495940c1bbeda4a2e6f17b82 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:30.248569 20009 raft_consensus.cc:697] T bc04a038495940c1bbeda4a2e6f17b82 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 LEADER]: Becoming Leader. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Running, Role: LEADER
I20260709 13:20:30.239329 20209 consensus_queue.cc:1048] T e3ca37daad8341ad99cbba926f37232e P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:20:30.260617 19995 tablet_bootstrap.cc:492] T 87364a1289ed4aa1b9b2a7a9951ed776 P c62716bf66c24330860573535b8b9430: Bootstrap starting.
I20260709 13:20:30.267620 19799 raft_consensus.cc:3060] T 1fa8a0db61be45c3aa42edb11db4c5ff P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:20:30.269438 19913 leader_election.cc:343] T b832efa5a2e34b3894e33629110e6fef P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:20:30.270826 19913 leader_election.cc:343] T b832efa5a2e34b3894e33629110e6fef P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:30.271162 19874 raft_consensus.cc:3060] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:30.274631 20009 consensus_queue.cc:237] T bc04a038495940c1bbeda4a2e6f17b82 P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:30.279615 19994 tablet_bootstrap.cc:492] T 563940ad96444ce59e393ea303cc3352 P 8df38c4c53c44e94b36c1e8992476d7c: No bootstrap required, opened a new log
I20260709 13:20:30.280423 19994 ts_tablet_manager.cc:1403] T 563940ad96444ce59e393ea303cc3352 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.428s	user 0.031s	sys 0.049s
I20260709 13:20:30.291514 19995 tablet_bootstrap.cc:654] T 87364a1289ed4aa1b9b2a7a9951ed776 P c62716bf66c24330860573535b8b9430: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:30.300042 19993 raft_consensus.cc:359] T d5f8619b1f9f46adaa41774af856d9b1 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:30.303877 19993 raft_consensus.cc:385] T d5f8619b1f9f46adaa41774af856d9b1 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:30.304392 19993 raft_consensus.cc:740] T d5f8619b1f9f46adaa41774af856d9b1 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Initialized, Role: FOLLOWER
I20260709 13:20:30.305204 20300 consensus_queue.cc:1048] T e3ca37daad8341ad99cbba926f37232e P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:20:30.305258 19993 consensus_queue.cc:260] T d5f8619b1f9f46adaa41774af856d9b1 P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:30.288138 20332 consensus_queue.cc:1048] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260709 13:20:30.308710 19993 ts_tablet_manager.cc:1434] T d5f8619b1f9f46adaa41774af856d9b1 P e90ffed3ba174ababeb21d9415d4d918: Time spent starting tablet: real 0.063s	user 0.008s	sys 0.000s
I20260709 13:20:30.308931 20079 raft_consensus.cc:493] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:30.309489 20079 raft_consensus.cc:515] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:30.310163 19993 tablet_bootstrap.cc:492] T 563940ad96444ce59e393ea303cc3352 P e90ffed3ba174ababeb21d9415d4d918: Bootstrap starting.
I20260709 13:20:30.317503 19993 tablet_bootstrap.cc:654] T 563940ad96444ce59e393ea303cc3352 P e90ffed3ba174ababeb21d9415d4d918: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:30.324247 19725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5476f5d68d496eaedce93e80bff5e4" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:30.328650 19623 catalog_manager.cc:5697] T bc04a038495940c1bbeda4a2e6f17b82 P b80b3a62d7d549a9a5c42def46769ac4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131). New cstate: current_term: 1 leader_uuid: "b80b3a62d7d549a9a5c42def46769ac4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:30.334192 19765 leader_election.cc:304] T 1fa8a0db61be45c3aa42edb11db4c5ff P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e90ffed3ba174ababeb21d9415d4d918; no voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430
I20260709 13:20:30.363003 20332 consensus_queue.cc:1048] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:20:30.336161 20079 leader_election.cc:290] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:30.332640 19868 raft_consensus.cc:2468] T b832efa5a2e34b3894e33629110e6fef P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c62716bf66c24330860573535b8b9430 in term 1.
I20260709 13:20:30.365339 19913 leader_election.cc:304] T b832efa5a2e34b3894e33629110e6fef P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430; no voters: e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:30.346050 19946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5476f5d68d496eaedce93e80bff5e4" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:30.361987 19994 raft_consensus.cc:359] T 563940ad96444ce59e393ea303cc3352 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:30.371666 19994 raft_consensus.cc:385] T 563940ad96444ce59e393ea303cc3352 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:30.372035 19994 raft_consensus.cc:740] T 563940ad96444ce59e393ea303cc3352 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:30.373113 19997 tablet_bootstrap.cc:492] T ece79531bbd749d3b9300fe2fbeef9af P b80b3a62d7d549a9a5c42def46769ac4: No bootstrap required, opened a new log
I20260709 13:20:30.373876 19997 ts_tablet_manager.cc:1403] T ece79531bbd749d3b9300fe2fbeef9af P b80b3a62d7d549a9a5c42def46769ac4: Time spent bootstrapping tablet: real 0.258s	user 0.019s	sys 0.005s
I20260709 13:20:30.376089 20301 raft_consensus.cc:2804] T b832efa5a2e34b3894e33629110e6fef P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:30.376901 20301 raft_consensus.cc:697] T b832efa5a2e34b3894e33629110e6fef P c62716bf66c24330860573535b8b9430 [term 1 LEADER]: Becoming Leader. State: Replica: c62716bf66c24330860573535b8b9430, State: Running, Role: LEADER
I20260709 13:20:30.378189 20301 consensus_queue.cc:237] T b832efa5a2e34b3894e33629110e6fef P c62716bf66c24330860573535b8b9430 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:30.390975 19839 leader_election.cc:304] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4; no voters: 8df38c4c53c44e94b36c1e8992476d7c, c62716bf66c24330860573535b8b9430
I20260709 13:20:30.397428 19997 raft_consensus.cc:359] T ece79531bbd749d3b9300fe2fbeef9af P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:30.398156 19997 raft_consensus.cc:385] T ece79531bbd749d3b9300fe2fbeef9af P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:30.398414 19997 raft_consensus.cc:740] T ece79531bbd749d3b9300fe2fbeef9af P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Initialized, Role: FOLLOWER
I20260709 13:20:30.403079 19994 consensus_queue.cc:260] T 563940ad96444ce59e393ea303cc3352 P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:30.419039 19997 consensus_queue.cc:260] T ece79531bbd749d3b9300fe2fbeef9af P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:30.441411 19994 ts_tablet_manager.cc:1434] T 563940ad96444ce59e393ea303cc3352 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.161s	user 0.009s	sys 0.003s
I20260709 13:20:30.444370 19997 ts_tablet_manager.cc:1434] T ece79531bbd749d3b9300fe2fbeef9af P b80b3a62d7d549a9a5c42def46769ac4: Time spent starting tablet: real 0.070s	user 0.008s	sys 0.000s
I20260709 13:20:30.444967 20285 raft_consensus.cc:493] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:30.445705 19997 tablet_bootstrap.cc:492] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P b80b3a62d7d549a9a5c42def46769ac4: Bootstrap starting.
I20260709 13:20:30.445617 20285 raft_consensus.cc:515] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:30.446908 20009 raft_consensus.cc:2749] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:20:30.446252 19994 tablet_bootstrap.cc:492] T 1cceb7f75d7040c2b98049590928b891 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:30.464756 20009 raft_consensus.cc:493] T ece79531bbd749d3b9300fe2fbeef9af P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:30.465395 20009 raft_consensus.cc:515] T ece79531bbd749d3b9300fe2fbeef9af P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:30.531173 19997 tablet_bootstrap.cc:654] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P b80b3a62d7d549a9a5c42def46769ac4: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:30.531489 19993 tablet_bootstrap.cc:492] T 563940ad96444ce59e393ea303cc3352 P e90ffed3ba174ababeb21d9415d4d918: No bootstrap required, opened a new log
I20260709 13:20:30.532219 19993 ts_tablet_manager.cc:1403] T 563940ad96444ce59e393ea303cc3352 P e90ffed3ba174ababeb21d9415d4d918: Time spent bootstrapping tablet: real 0.223s	user 0.033s	sys 0.062s
I20260709 13:20:30.530956 20275 raft_consensus.cc:493] T 9f60d4b859264720a20f5670c3c71a6f P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:30.561317 20275 raft_consensus.cc:515] T 9f60d4b859264720a20f5670c3c71a6f P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:30.567077 20275 leader_election.cc:290] T 9f60d4b859264720a20f5670c3c71a6f P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:30.567862 19994 tablet_bootstrap.cc:654] T 1cceb7f75d7040c2b98049590928b891 P 8df38c4c53c44e94b36c1e8992476d7c: Neither blocks nor log segments found. Creating new log.
W20260709 13:20:30.568881 19766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:41855, user_credentials={real_user=slave}} blocked reactor thread for 132430us
I20260709 13:20:30.571686 19722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f60d4b859264720a20f5670c3c71a6f" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
W20260709 13:20:30.573202 19914 leader_election.cc:343] T 9f60d4b859264720a20f5670c3c71a6f P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:30.575688 20132 raft_consensus.cc:493] T 563940ad96444ce59e393ea303cc3352 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:30.576171 20132 raft_consensus.cc:515] T 563940ad96444ce59e393ea303cc3352 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:30.577906 20132 leader_election.cc:290] T 563940ad96444ce59e393ea303cc3352 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:30.578325 20298 raft_consensus.cc:493] T c393f9d18341477c97fac9a4af7e3adf P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:30.578684 20298 raft_consensus.cc:515] T c393f9d18341477c97fac9a4af7e3adf P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:30.577369 19993 raft_consensus.cc:359] T 563940ad96444ce59e393ea303cc3352 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:30.579382 20285 leader_election.cc:290] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:30.579510 19993 raft_consensus.cc:385] T 563940ad96444ce59e393ea303cc3352 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:30.580026 19993 raft_consensus.cc:740] T 563940ad96444ce59e393ea303cc3352 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Initialized, Role: FOLLOWER
I20260709 13:20:30.580888 19993 consensus_queue.cc:260] T 563940ad96444ce59e393ea303cc3352 P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:30.581987 19800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563940ad96444ce59e393ea303cc3352" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:30.582692 19800 raft_consensus.cc:2468] T 563940ad96444ce59e393ea303cc3352 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 0.
I20260709 13:20:30.583998 19690 leader_election.cc:304] T 563940ad96444ce59e393ea303cc3352 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918; no voters: 
I20260709 13:20:30.584316 20212 raft_consensus.cc:2749] T 1fa8a0db61be45c3aa42edb11db4c5ff P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:30.584808 20132 raft_consensus.cc:2804] T 563940ad96444ce59e393ea303cc3352 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:30.585219 20132 raft_consensus.cc:493] T 563940ad96444ce59e393ea303cc3352 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:30.585541 20132 raft_consensus.cc:3060] T 563940ad96444ce59e393ea303cc3352 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:30.586935 19993 ts_tablet_manager.cc:1434] T 563940ad96444ce59e393ea303cc3352 P e90ffed3ba174ababeb21d9415d4d918: Time spent starting tablet: real 0.054s	user 0.007s	sys 0.003s
I20260709 13:20:30.588411 19993 tablet_bootstrap.cc:492] T 1cceb7f75d7040c2b98049590928b891 P e90ffed3ba174ababeb21d9415d4d918: Bootstrap starting.
I20260709 13:20:30.617362 19946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece79531bbd749d3b9300fe2fbeef9af" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:30.618223 19946 raft_consensus.cc:2468] T ece79531bbd749d3b9300fe2fbeef9af P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 0.
I20260709 13:20:30.635246 19874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f60d4b859264720a20f5670c3c71a6f" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:30.636447 19995 tablet_bootstrap.cc:492] T 87364a1289ed4aa1b9b2a7a9951ed776 P c62716bf66c24330860573535b8b9430: No bootstrap required, opened a new log
I20260709 13:20:30.634608 19839 leader_election.cc:304] T ece79531bbd749d3b9300fe2fbeef9af P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430; no voters: 
I20260709 13:20:30.637003 19995 ts_tablet_manager.cc:1403] T 87364a1289ed4aa1b9b2a7a9951ed776 P c62716bf66c24330860573535b8b9430: Time spent bootstrapping tablet: real 0.377s	user 0.023s	sys 0.062s
I20260709 13:20:30.639205 19797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563940ad96444ce59e393ea303cc3352" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:30.639956 19797 raft_consensus.cc:2468] T 563940ad96444ce59e393ea303cc3352 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 0.
I20260709 13:20:30.641840 20009 leader_election.cc:290] T ece79531bbd749d3b9300fe2fbeef9af P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:30.642489 20132 raft_consensus.cc:515] T 563940ad96444ce59e393ea303cc3352 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:30.644187 19724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece79531bbd749d3b9300fe2fbeef9af" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:30.644614 20132 leader_election.cc:290] T 563940ad96444ce59e393ea303cc3352 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Requested vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:30.646612 19994 tablet_bootstrap.cc:492] T 1cceb7f75d7040c2b98049590928b891 P 8df38c4c53c44e94b36c1e8992476d7c: No bootstrap required, opened a new log
I20260709 13:20:30.647173 19994 ts_tablet_manager.cc:1403] T 1cceb7f75d7040c2b98049590928b891 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.205s	user 0.015s	sys 0.008s
I20260709 13:20:30.648998 19995 raft_consensus.cc:359] T 87364a1289ed4aa1b9b2a7a9951ed776 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:30.649739 19995 raft_consensus.cc:385] T 87364a1289ed4aa1b9b2a7a9951ed776 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:30.649952 19796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563940ad96444ce59e393ea303cc3352" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
I20260709 13:20:30.650621 19796 raft_consensus.cc:3060] T 563940ad96444ce59e393ea303cc3352 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:30.653123 19997 tablet_bootstrap.cc:492] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P b80b3a62d7d549a9a5c42def46769ac4: No bootstrap required, opened a new log
I20260709 13:20:30.653622 19997 ts_tablet_manager.cc:1403] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P b80b3a62d7d549a9a5c42def46769ac4: Time spent bootstrapping tablet: real 0.208s	user 0.016s	sys 0.004s
I20260709 13:20:30.655452 20009 raft_consensus.cc:2804] T ece79531bbd749d3b9300fe2fbeef9af P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:30.655905 20009 raft_consensus.cc:493] T ece79531bbd749d3b9300fe2fbeef9af P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:30.656215 20009 raft_consensus.cc:3060] T ece79531bbd749d3b9300fe2fbeef9af P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:30.650010 19995 raft_consensus.cc:740] T 87364a1289ed4aa1b9b2a7a9951ed776 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62716bf66c24330860573535b8b9430, State: Initialized, Role: FOLLOWER
I20260709 13:20:30.663043 19839 leader_election.cc:304] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, e90ffed3ba174ababeb21d9415d4d918; no voters: 
I20260709 13:20:30.666204 20080 raft_consensus.cc:2804] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:20:30.660390 19913 leader_election.cc:343] T 9f60d4b859264720a20f5670c3c71a6f P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:30.667181 19913 leader_election.cc:304] T 9f60d4b859264720a20f5670c3c71a6f P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4
I20260709 13:20:30.668113 20009 raft_consensus.cc:515] T ece79531bbd749d3b9300fe2fbeef9af P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:30.672291 20298 leader_election.cc:290] T c393f9d18341477c97fac9a4af7e3adf P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:30.673434 20300 raft_consensus.cc:493] T d5f8619b1f9f46adaa41774af856d9b1 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:30.673972 20300 raft_consensus.cc:515] T d5f8619b1f9f46adaa41774af856d9b1 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:30.675218 20275 raft_consensus.cc:2749] T 9f60d4b859264720a20f5670c3c71a6f P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:30.675038 19722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c393f9d18341477c97fac9a4af7e3adf" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:30.666615 20080 raft_consensus.cc:493] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:30.676213 20080 raft_consensus.cc:3060] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:30.676348 20300 leader_election.cc:290] T d5f8619b1f9f46adaa41774af856d9b1 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:30.649950 19994 raft_consensus.cc:359] T 1cceb7f75d7040c2b98049590928b891 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:30.678383 19994 raft_consensus.cc:385] T 1cceb7f75d7040c2b98049590928b891 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:30.679940 19994 raft_consensus.cc:740] T 1cceb7f75d7040c2b98049590928b891 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:30.683198 20009 leader_election.cc:290] T ece79531bbd749d3b9300fe2fbeef9af P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 election: Requested vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:30.688050 19997 raft_consensus.cc:359] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:30.688849 19997 raft_consensus.cc:385] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:30.684985 19995 consensus_queue.cc:260] T 87364a1289ed4aa1b9b2a7a9951ed776 P c62716bf66c24330860573535b8b9430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:30.689102 19997 raft_consensus.cc:740] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Initialized, Role: FOLLOWER
I20260709 13:20:30.682385 19941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece79531bbd749d3b9300fe2fbeef9af" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:30.691490 19941 raft_consensus.cc:3060] T ece79531bbd749d3b9300fe2fbeef9af P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:30.694656 19993 tablet_bootstrap.cc:654] T 1cceb7f75d7040c2b98049590928b891 P e90ffed3ba174ababeb21d9415d4d918: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:30.696059 19994 consensus_queue.cc:260] T 1cceb7f75d7040c2b98049590928b891 P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:30.704742 19874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563940ad96444ce59e393ea303cc3352" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:30.700659 19997 consensus_queue.cc:260] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:30.698979 19724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563940ad96444ce59e393ea303cc3352" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:30.711720 19871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563940ad96444ce59e393ea303cc3352" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
I20260709 13:20:30.712334 19724 raft_consensus.cc:2393] T 563940ad96444ce59e393ea303cc3352 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b80b3a62d7d549a9a5c42def46769ac4 in current term 1: Already voted for candidate 8df38c4c53c44e94b36c1e8992476d7c in this term.
I20260709 13:20:30.713682 19997 ts_tablet_manager.cc:1434] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P b80b3a62d7d549a9a5c42def46769ac4: Time spent starting tablet: real 0.060s	user 0.009s	sys 0.000s
I20260709 13:20:30.714844 19997 tablet_bootstrap.cc:492] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4: Bootstrap starting.
I20260709 13:20:30.715289 19945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5f8619b1f9f46adaa41774af856d9b1" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:30.719974 19994 ts_tablet_manager.cc:1434] T 1cceb7f75d7040c2b98049590928b891 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.072s	user 0.007s	sys 0.001s
I20260709 13:20:30.723747 19995 ts_tablet_manager.cc:1434] T 87364a1289ed4aa1b9b2a7a9951ed776 P c62716bf66c24330860573535b8b9430: Time spent starting tablet: real 0.086s	user 0.006s	sys 0.002s
I20260709 13:20:30.725075 19995 tablet_bootstrap.cc:492] T 8fd58e41ecee4779b3e72df3aea4463a P c62716bf66c24330860573535b8b9430: Bootstrap starting.
I20260709 13:20:30.726212 19796 raft_consensus.cc:2468] T 563940ad96444ce59e393ea303cc3352 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 1.
I20260709 13:20:30.751497 19994 tablet_bootstrap.cc:492] T dc5c36b41652443a8130592df8a66eb9 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:30.754431 19690 leader_election.cc:304] T 563940ad96444ce59e393ea303cc3352 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918; no voters: 
I20260709 13:20:30.755740 19796 raft_consensus.cc:3060] T d5f8619b1f9f46adaa41774af856d9b1 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:30.761067 19722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5f8619b1f9f46adaa41774af856d9b1" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:30.762848 19994 tablet_bootstrap.cc:654] T dc5c36b41652443a8130592df8a66eb9 P 8df38c4c53c44e94b36c1e8992476d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:30.765528 20181 raft_consensus.cc:2804] T 563940ad96444ce59e393ea303cc3352 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:30.766116 20181 raft_consensus.cc:697] T 563940ad96444ce59e393ea303cc3352 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 LEADER]: Becoming Leader. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Running, Role: LEADER
I20260709 13:20:30.767025 20181 consensus_queue.cc:237] T 563940ad96444ce59e393ea303cc3352 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
W20260709 13:20:30.774325 19840 leader_election.cc:343] T ece79531bbd749d3b9300fe2fbeef9af P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:30.789126 19941 raft_consensus.cc:2468] T ece79531bbd749d3b9300fe2fbeef9af P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 1.
I20260709 13:20:30.792320 19839 leader_election.cc:304] T ece79531bbd749d3b9300fe2fbeef9af P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430; no voters: 
I20260709 13:20:30.793473 20350 raft_consensus.cc:493] T 87364a1289ed4aa1b9b2a7a9951ed776 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:30.793984 20353 raft_consensus.cc:2804] T ece79531bbd749d3b9300fe2fbeef9af P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:30.794538 19995 tablet_bootstrap.cc:654] T 8fd58e41ecee4779b3e72df3aea4463a P c62716bf66c24330860573535b8b9430: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:30.794677 20353 raft_consensus.cc:697] T ece79531bbd749d3b9300fe2fbeef9af P b80b3a62d7d549a9a5c42def46769ac4 [term 1 LEADER]: Becoming Leader. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Running, Role: LEADER
I20260709 13:20:30.819226 19766 leader_election.cc:304] T d5f8619b1f9f46adaa41774af856d9b1 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e90ffed3ba174ababeb21d9415d4d918; no voters: 8df38c4c53c44e94b36c1e8992476d7c, c62716bf66c24330860573535b8b9430
I20260709 13:20:30.819957 20353 consensus_queue.cc:237] T ece79531bbd749d3b9300fe2fbeef9af P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:30.799646 19868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c393f9d18341477c97fac9a4af7e3adf" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:30.827831 19997 tablet_bootstrap.cc:654] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:30.834304 19622 catalog_manager.cc:5697] T b832efa5a2e34b3894e33629110e6fef P c62716bf66c24330860573535b8b9430 reported cstate change: term changed from 0 to 1, leader changed from <none> to c62716bf66c24330860573535b8b9430 (127.9.116.132). New cstate: current_term: 1 leader_uuid: "c62716bf66c24330860573535b8b9430" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:30.794039 20350 raft_consensus.cc:515] T 87364a1289ed4aa1b9b2a7a9951ed776 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:30.808431 20275 raft_consensus.cc:493] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:30.801560 20080 raft_consensus.cc:515] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:30.814049 19994 tablet_bootstrap.cc:492] T dc5c36b41652443a8130592df8a66eb9 P 8df38c4c53c44e94b36c1e8992476d7c: No bootstrap required, opened a new log
I20260709 13:20:30.837761 20275 raft_consensus.cc:515] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:30.838713 19994 ts_tablet_manager.cc:1403] T dc5c36b41652443a8130592df8a66eb9 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.088s	user 0.015s	sys 0.002s
I20260709 13:20:30.840273 19874 raft_consensus.cc:2393] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8df38c4c53c44e94b36c1e8992476d7c in current term 1: Already voted for candidate b80b3a62d7d549a9a5c42def46769ac4 in this term.
I20260709 13:20:30.841303 20350 leader_election.cc:290] T 87364a1289ed4aa1b9b2a7a9951ed776 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:30.844364 20275 leader_election.cc:290] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:30.847330 19994 raft_consensus.cc:359] T dc5c36b41652443a8130592df8a66eb9 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:30.848479 19994 raft_consensus.cc:385] T dc5c36b41652443a8130592df8a66eb9 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:30.848831 19994 raft_consensus.cc:740] T dc5c36b41652443a8130592df8a66eb9 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:30.850049 19868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7bc0eba7e94b9eb1eeec9bbd1a6e97" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:30.850747 19868 raft_consensus.cc:2468] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c62716bf66c24330860573535b8b9430 in term 0.
I20260709 13:20:30.852504 19913 leader_election.cc:304] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430; no voters: 
I20260709 13:20:30.854594 20301 raft_consensus.cc:2804] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:30.855140 20301 raft_consensus.cc:493] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:30.855613 20301 raft_consensus.cc:3060] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:30.861496 19874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87364a1289ed4aa1b9b2a7a9951ed776" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:30.862839 20009 consensus_queue.cc:1048] T bc04a038495940c1bbeda4a2e6f17b82 P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:20:30.863431 19913 leader_election.cc:343] T 87364a1289ed4aa1b9b2a7a9951ed776 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:30.868376 20080 leader_election.cc:290] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 election: Requested vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:30.870198 19799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563940ad96444ce59e393ea303cc3352" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
I20260709 13:20:30.871258 19799 raft_consensus.cc:2393] T 563940ad96444ce59e393ea303cc3352 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b80b3a62d7d549a9a5c42def46769ac4 in current term 1: Already voted for candidate 8df38c4c53c44e94b36c1e8992476d7c in this term.
I20260709 13:20:30.877264 19725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece79531bbd749d3b9300fe2fbeef9af" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
I20260709 13:20:30.849633 19994 consensus_queue.cc:260] T dc5c36b41652443a8130592df8a66eb9 P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:30.889919 19994 ts_tablet_manager.cc:1434] T dc5c36b41652443a8130592df8a66eb9 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.050s	user 0.004s	sys 0.002s
I20260709 13:20:30.898295 19722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563940ad96444ce59e393ea303cc3352" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
I20260709 13:20:30.912178 19994 tablet_bootstrap.cc:492] T 0a9c6b64c8e04c4283d24ae6290a52ea P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:30.923208 20301 raft_consensus.cc:515] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:30.938758 19994 tablet_bootstrap.cc:654] T 0a9c6b64c8e04c4283d24ae6290a52ea P 8df38c4c53c44e94b36c1e8992476d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:30.941535 19997 tablet_bootstrap.cc:492] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4: No bootstrap required, opened a new log
I20260709 13:20:30.942044 19997 ts_tablet_manager.cc:1403] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4: Time spent bootstrapping tablet: real 0.228s	user 0.021s	sys 0.017s
I20260709 13:20:30.942406 20301 leader_election.cc:290] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 election: Requested vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:30.943642 19874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7bc0eba7e94b9eb1eeec9bbd1a6e97" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
I20260709 13:20:30.944321 19874 raft_consensus.cc:3060] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:30.944895 19997 raft_consensus.cc:359] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:30.907760 19995 tablet_bootstrap.cc:492] T 8fd58e41ecee4779b3e72df3aea4463a P c62716bf66c24330860573535b8b9430: No bootstrap required, opened a new log
I20260709 13:20:30.960656 19995 ts_tablet_manager.cc:1403] T 8fd58e41ecee4779b3e72df3aea4463a P c62716bf66c24330860573535b8b9430: Time spent bootstrapping tablet: real 0.236s	user 0.010s	sys 0.019s
I20260709 13:20:30.963660 19995 raft_consensus.cc:359] T 8fd58e41ecee4779b3e72df3aea4463a P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:30.964354 19995 raft_consensus.cc:385] T 8fd58e41ecee4779b3e72df3aea4463a P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:30.964629 19995 raft_consensus.cc:740] T 8fd58e41ecee4779b3e72df3aea4463a P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62716bf66c24330860573535b8b9430, State: Initialized, Role: FOLLOWER
I20260709 13:20:30.965530 19995 consensus_queue.cc:260] T 8fd58e41ecee4779b3e72df3aea4463a P c62716bf66c24330860573535b8b9430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:30.967689 19995 ts_tablet_manager.cc:1434] T 8fd58e41ecee4779b3e72df3aea4463a P c62716bf66c24330860573535b8b9430: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.002s
I20260709 13:20:30.968202 20275 consensus_queue.cc:1048] T b832efa5a2e34b3894e33629110e6fef P c62716bf66c24330860573535b8b9430 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:20:30.945959 19840 leader_election.cc:343] T ece79531bbd749d3b9300fe2fbeef9af P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:20:30.970345 19840 consensus_peers.cc:597] T bc04a038495940c1bbeda4a2e6f17b82 P b80b3a62d7d549a9a5c42def46769ac4 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:30.971796 19997 raft_consensus.cc:385] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:30.972642 19840 leader_election.cc:304] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4; no voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:30.949256 19797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7bc0eba7e94b9eb1eeec9bbd1a6e97" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:30.906319 19993 tablet_bootstrap.cc:492] T 1cceb7f75d7040c2b98049590928b891 P e90ffed3ba174ababeb21d9415d4d918: No bootstrap required, opened a new log
I20260709 13:20:30.956385 19800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7bc0eba7e94b9eb1eeec9bbd1a6e97" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
I20260709 13:20:30.974488 20079 raft_consensus.cc:2749] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:20:30.975171 19670 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:20:30.957180 19765 leader_election.cc:304] T c393f9d18341477c97fac9a4af7e3adf P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e90ffed3ba174ababeb21d9415d4d918; no voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4
I20260709 13:20:30.947299 19799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87364a1289ed4aa1b9b2a7a9951ed776" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:30.976509 20354 raft_consensus.cc:2749] T c393f9d18341477c97fac9a4af7e3adf P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:20:30.974480 19993 ts_tablet_manager.cc:1403] T 1cceb7f75d7040c2b98049590928b891 P e90ffed3ba174ababeb21d9415d4d918: Time spent bootstrapping tablet: real 0.386s	user 0.015s	sys 0.025s
I20260709 13:20:30.968765 19995 tablet_bootstrap.cc:492] T bd80f9f8533b4cd3b5861ec11917a03d P c62716bf66c24330860573535b8b9430: Bootstrap starting.
I20260709 13:20:30.981444 19993 raft_consensus.cc:359] T 1cceb7f75d7040c2b98049590928b891 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:30.982257 19993 raft_consensus.cc:385] T 1cceb7f75d7040c2b98049590928b891 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:30.982579 19993 raft_consensus.cc:740] T 1cceb7f75d7040c2b98049590928b891 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Initialized, Role: FOLLOWER
I20260709 13:20:30.987087 19993 consensus_queue.cc:260] T 1cceb7f75d7040c2b98049590928b891 P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:30.989367 19995 tablet_bootstrap.cc:654] T bd80f9f8533b4cd3b5861ec11917a03d P c62716bf66c24330860573535b8b9430: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:30.972126 19997 raft_consensus.cc:740] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Initialized, Role: FOLLOWER
I20260709 13:20:30.877200 19620 catalog_manager.cc:5697] T 563940ad96444ce59e393ea303cc3352 P 8df38c4c53c44e94b36c1e8992476d7c reported cstate change: term changed from 0 to 1, leader changed from <none> to 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129). New cstate: current_term: 1 leader_uuid: "8df38c4c53c44e94b36c1e8992476d7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:31.001298 19993 ts_tablet_manager.cc:1434] T 1cceb7f75d7040c2b98049590928b891 P e90ffed3ba174ababeb21d9415d4d918: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.000s
I20260709 13:20:30.997424 19997 consensus_queue.cc:260] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:31.016352 19993 tablet_bootstrap.cc:492] T dc5c36b41652443a8130592df8a66eb9 P e90ffed3ba174ababeb21d9415d4d918: Bootstrap starting.
I20260709 13:20:31.017251 19997 ts_tablet_manager.cc:1434] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4: Time spent starting tablet: real 0.075s	user 0.004s	sys 0.003s
W20260709 13:20:31.011319 19913 leader_election.cc:343] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:31.012403 19622 catalog_manager.cc:5697] T ece79531bbd749d3b9300fe2fbeef9af P b80b3a62d7d549a9a5c42def46769ac4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131). New cstate: current_term: 1 leader_uuid: "b80b3a62d7d549a9a5c42def46769ac4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:31.018364 19997 tablet_bootstrap.cc:492] T 87364a1289ed4aa1b9b2a7a9951ed776 P b80b3a62d7d549a9a5c42def46769ac4: Bootstrap starting.
W20260709 13:20:31.018802 19913 leader_election.cc:343] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:20:31.019734 19913 leader_election.cc:343] T 87364a1289ed4aa1b9b2a7a9951ed776 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:31.020110 19913 leader_election.cc:304] T 87364a1289ed4aa1b9b2a7a9951ed776 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c62716bf66c24330860573535b8b9430; no voters: b80b3a62d7d549a9a5c42def46769ac4, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:31.025313 20350 raft_consensus.cc:493] T 1cceb7f75d7040c2b98049590928b891 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:31.026018 19997 tablet_bootstrap.cc:654] T 87364a1289ed4aa1b9b2a7a9951ed776 P b80b3a62d7d549a9a5c42def46769ac4: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:31.025873 20350 raft_consensus.cc:515] T 1cceb7f75d7040c2b98049590928b891 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:31.027311 20370 raft_consensus.cc:2749] T 87364a1289ed4aa1b9b2a7a9951ed776 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:31.030332 20350 leader_election.cc:290] T 1cceb7f75d7040c2b98049590928b891 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:31.031057 19725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cceb7f75d7040c2b98049590928b891" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:31.033502 19725 raft_consensus.cc:2468] T 1cceb7f75d7040c2b98049590928b891 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c62716bf66c24330860573535b8b9430 in term 0.
I20260709 13:20:31.036580 19914 leader_election.cc:304] T 1cceb7f75d7040c2b98049590928b891 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, c62716bf66c24330860573535b8b9430; no voters: 
I20260709 13:20:31.038175 20370 raft_consensus.cc:2804] T 1cceb7f75d7040c2b98049590928b891 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:31.038966 20370 raft_consensus.cc:493] T 1cceb7f75d7040c2b98049590928b891 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:31.039536 20370 raft_consensus.cc:3060] T 1cceb7f75d7040c2b98049590928b891 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:31.052132 19799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cceb7f75d7040c2b98049590928b891" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:31.052876 19799 raft_consensus.cc:2468] T 1cceb7f75d7040c2b98049590928b891 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c62716bf66c24330860573535b8b9430 in term 0.
W20260709 13:20:31.075184 19913 consensus_peers.cc:597] T b832efa5a2e34b3894e33629110e6fef P c62716bf66c24330860573535b8b9430 -> Peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Couldn't send request to peer e90ffed3ba174ababeb21d9415d4d918. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:31.054927 19874 raft_consensus.cc:2468] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c62716bf66c24330860573535b8b9430 in term 1.
I20260709 13:20:31.084204 19995 tablet_bootstrap.cc:492] T bd80f9f8533b4cd3b5861ec11917a03d P c62716bf66c24330860573535b8b9430: No bootstrap required, opened a new log
I20260709 13:20:31.098069 19995 ts_tablet_manager.cc:1403] T bd80f9f8533b4cd3b5861ec11917a03d P c62716bf66c24330860573535b8b9430: Time spent bootstrapping tablet: real 0.130s	user 0.011s	sys 0.000s
I20260709 13:20:31.086083 19997 tablet_bootstrap.cc:492] T 87364a1289ed4aa1b9b2a7a9951ed776 P b80b3a62d7d549a9a5c42def46769ac4: No bootstrap required, opened a new log
I20260709 13:20:31.099381 19997 ts_tablet_manager.cc:1403] T 87364a1289ed4aa1b9b2a7a9951ed776 P b80b3a62d7d549a9a5c42def46769ac4: Time spent bootstrapping tablet: real 0.081s	user 0.037s	sys 0.006s
I20260709 13:20:31.100682 19995 raft_consensus.cc:359] T bd80f9f8533b4cd3b5861ec11917a03d P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:31.101363 19995 raft_consensus.cc:385] T bd80f9f8533b4cd3b5861ec11917a03d P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:31.101643 19995 raft_consensus.cc:740] T bd80f9f8533b4cd3b5861ec11917a03d P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62716bf66c24330860573535b8b9430, State: Initialized, Role: FOLLOWER
I20260709 13:20:31.102025 19997 raft_consensus.cc:359] T 87364a1289ed4aa1b9b2a7a9951ed776 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:31.094694 20370 raft_consensus.cc:515] T 1cceb7f75d7040c2b98049590928b891 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:31.102385 19995 consensus_queue.cc:260] T bd80f9f8533b4cd3b5861ec11917a03d P c62716bf66c24330860573535b8b9430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:31.069793 20212 raft_consensus.cc:2749] T d5f8619b1f9f46adaa41774af856d9b1 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:31.106063 19993 tablet_bootstrap.cc:654] T dc5c36b41652443a8130592df8a66eb9 P e90ffed3ba174ababeb21d9415d4d918: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:31.104169 19913 leader_election.cc:304] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430; no voters: e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:31.107389 20370 leader_election.cc:290] T 1cceb7f75d7040c2b98049590928b891 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 election: Requested vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:31.108034 20370 raft_consensus.cc:2804] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:31.102727 19997 raft_consensus.cc:385] T 87364a1289ed4aa1b9b2a7a9951ed776 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:31.109102 20370 raft_consensus.cc:697] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P c62716bf66c24330860573535b8b9430 [term 1 LEADER]: Becoming Leader. State: Replica: c62716bf66c24330860573535b8b9430, State: Running, Role: LEADER
I20260709 13:20:31.109381 19997 raft_consensus.cc:740] T 87364a1289ed4aa1b9b2a7a9951ed776 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Initialized, Role: FOLLOWER
I20260709 13:20:31.110006 20370 consensus_queue.cc:237] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P c62716bf66c24330860573535b8b9430 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:31.110219 19796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cceb7f75d7040c2b98049590928b891" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
I20260709 13:20:31.110174 19997 consensus_queue.cc:260] T 87364a1289ed4aa1b9b2a7a9951ed776 P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:31.111004 19796 raft_consensus.cc:3060] T 1cceb7f75d7040c2b98049590928b891 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:31.111490 19725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cceb7f75d7040c2b98049590928b891" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
I20260709 13:20:31.112074 19725 raft_consensus.cc:3060] T 1cceb7f75d7040c2b98049590928b891 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:31.114535 19997 ts_tablet_manager.cc:1434] T 87364a1289ed4aa1b9b2a7a9951ed776 P b80b3a62d7d549a9a5c42def46769ac4: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:20:31.115722 19997 tablet_bootstrap.cc:492] T cb33acfa7a9f46ab95b0e77e1394f258 P b80b3a62d7d549a9a5c42def46769ac4: Bootstrap starting.
I20260709 13:20:31.094823 19994 tablet_bootstrap.cc:492] T 0a9c6b64c8e04c4283d24ae6290a52ea P 8df38c4c53c44e94b36c1e8992476d7c: No bootstrap required, opened a new log
I20260709 13:20:31.118824 19994 ts_tablet_manager.cc:1403] T 0a9c6b64c8e04c4283d24ae6290a52ea P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.207s	user 0.045s	sys 0.084s
I20260709 13:20:31.124181 19995 ts_tablet_manager.cc:1434] T bd80f9f8533b4cd3b5861ec11917a03d P c62716bf66c24330860573535b8b9430: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260709 13:20:31.125207 19796 raft_consensus.cc:2468] T 1cceb7f75d7040c2b98049590928b891 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c62716bf66c24330860573535b8b9430 in term 1.
I20260709 13:20:31.125772 19995 tablet_bootstrap.cc:492] T 29f92a880ce04259ae0ee9c5288949ff P c62716bf66c24330860573535b8b9430: Bootstrap starting.
I20260709 13:20:31.126571 19913 leader_election.cc:304] T 1cceb7f75d7040c2b98049590928b891 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c62716bf66c24330860573535b8b9430, e90ffed3ba174ababeb21d9415d4d918; no voters: 
I20260709 13:20:31.127540 20275 raft_consensus.cc:2804] T 1cceb7f75d7040c2b98049590928b891 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:31.128068 20275 raft_consensus.cc:697] T 1cceb7f75d7040c2b98049590928b891 P c62716bf66c24330860573535b8b9430 [term 1 LEADER]: Becoming Leader. State: Replica: c62716bf66c24330860573535b8b9430, State: Running, Role: LEADER
I20260709 13:20:31.128985 20275 consensus_queue.cc:237] T 1cceb7f75d7040c2b98049590928b891 P c62716bf66c24330860573535b8b9430 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:31.141961 19997 tablet_bootstrap.cc:654] T cb33acfa7a9f46ab95b0e77e1394f258 P b80b3a62d7d549a9a5c42def46769ac4: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:31.144568 19994 raft_consensus.cc:359] T 0a9c6b64c8e04c4283d24ae6290a52ea P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:31.145586 19994 raft_consensus.cc:385] T 0a9c6b64c8e04c4283d24ae6290a52ea P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:31.145931 19994 raft_consensus.cc:740] T 0a9c6b64c8e04c4283d24ae6290a52ea P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:31.147933 19994 consensus_queue.cc:260] T 0a9c6b64c8e04c4283d24ae6290a52ea P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:31.149360 19725 raft_consensus.cc:2468] T 1cceb7f75d7040c2b98049590928b891 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c62716bf66c24330860573535b8b9430 in term 1.
I20260709 13:20:31.150828 19994 ts_tablet_manager.cc:1434] T 0a9c6b64c8e04c4283d24ae6290a52ea P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.000s
I20260709 13:20:31.154095 19995 tablet_bootstrap.cc:654] T 29f92a880ce04259ae0ee9c5288949ff P c62716bf66c24330860573535b8b9430: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:31.155664 19994 tablet_bootstrap.cc:492] T 8abfa1aa35c44699960a8166fa79c41d P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:31.156575 19993 tablet_bootstrap.cc:492] T dc5c36b41652443a8130592df8a66eb9 P e90ffed3ba174ababeb21d9415d4d918: No bootstrap required, opened a new log
I20260709 13:20:31.157063 19993 ts_tablet_manager.cc:1403] T dc5c36b41652443a8130592df8a66eb9 P e90ffed3ba174ababeb21d9415d4d918: Time spent bootstrapping tablet: real 0.141s	user 0.023s	sys 0.010s
I20260709 13:20:31.159875 19993 raft_consensus.cc:359] T dc5c36b41652443a8130592df8a66eb9 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:31.160591 19993 raft_consensus.cc:385] T dc5c36b41652443a8130592df8a66eb9 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:31.160859 19993 raft_consensus.cc:740] T dc5c36b41652443a8130592df8a66eb9 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Initialized, Role: FOLLOWER
I20260709 13:20:31.161523 19993 consensus_queue.cc:260] T dc5c36b41652443a8130592df8a66eb9 P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:31.162422 19994 tablet_bootstrap.cc:654] T 8abfa1aa35c44699960a8166fa79c41d P 8df38c4c53c44e94b36c1e8992476d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:31.166325 19997 tablet_bootstrap.cc:492] T cb33acfa7a9f46ab95b0e77e1394f258 P b80b3a62d7d549a9a5c42def46769ac4: No bootstrap required, opened a new log
I20260709 13:20:31.166896 19997 ts_tablet_manager.cc:1403] T cb33acfa7a9f46ab95b0e77e1394f258 P b80b3a62d7d549a9a5c42def46769ac4: Time spent bootstrapping tablet: real 0.051s	user 0.017s	sys 0.000s
I20260709 13:20:31.172812 20350 raft_consensus.cc:493] T bd80f9f8533b4cd3b5861ec11917a03d P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:31.170348 19997 raft_consensus.cc:359] T cb33acfa7a9f46ab95b0e77e1394f258 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:31.173386 20350 raft_consensus.cc:515] T bd80f9f8533b4cd3b5861ec11917a03d P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:31.173925 19997 raft_consensus.cc:385] T cb33acfa7a9f46ab95b0e77e1394f258 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:31.174242 19997 raft_consensus.cc:740] T cb33acfa7a9f46ab95b0e77e1394f258 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Initialized, Role: FOLLOWER
I20260709 13:20:31.174959 19997 consensus_queue.cc:260] T cb33acfa7a9f46ab95b0e77e1394f258 P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:31.180682 19997 ts_tablet_manager.cc:1434] T cb33acfa7a9f46ab95b0e77e1394f258 P b80b3a62d7d549a9a5c42def46769ac4: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:20:31.180773 20350 leader_election.cc:290] T bd80f9f8533b4cd3b5861ec11917a03d P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:31.181842 19997 tablet_bootstrap.cc:492] T 9f60d4b859264720a20f5670c3c71a6f P b80b3a62d7d549a9a5c42def46769ac4: Bootstrap starting.
I20260709 13:20:31.181694 19725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd80f9f8533b4cd3b5861ec11917a03d" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
W20260709 13:20:31.183420 19914 leader_election.cc:343] T bd80f9f8533b4cd3b5861ec11917a03d P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:31.182374 19796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd80f9f8533b4cd3b5861ec11917a03d" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
W20260709 13:20:31.185395 19913 leader_election.cc:343] T bd80f9f8533b4cd3b5861ec11917a03d P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:31.185880 19913 leader_election.cc:304] T bd80f9f8533b4cd3b5861ec11917a03d P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:31.187180 20350 raft_consensus.cc:2749] T bd80f9f8533b4cd3b5861ec11917a03d P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:31.194247 19997 tablet_bootstrap.cc:654] T 9f60d4b859264720a20f5670c3c71a6f P b80b3a62d7d549a9a5c42def46769ac4: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:31.201429 19993 ts_tablet_manager.cc:1434] T dc5c36b41652443a8130592df8a66eb9 P e90ffed3ba174ababeb21d9415d4d918: Time spent starting tablet: real 0.044s	user 0.007s	sys 0.001s
I20260709 13:20:31.202059 19994 tablet_bootstrap.cc:492] T 8abfa1aa35c44699960a8166fa79c41d P 8df38c4c53c44e94b36c1e8992476d7c: No bootstrap required, opened a new log
I20260709 13:20:31.202581 19994 ts_tablet_manager.cc:1403] T 8abfa1aa35c44699960a8166fa79c41d P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.047s	user 0.009s	sys 0.004s
I20260709 13:20:31.205528 19994 raft_consensus.cc:359] T 8abfa1aa35c44699960a8166fa79c41d P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:31.206212 19994 raft_consensus.cc:385] T 8abfa1aa35c44699960a8166fa79c41d P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:31.206477 19994 raft_consensus.cc:740] T 8abfa1aa35c44699960a8166fa79c41d P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:31.207279 19994 consensus_queue.cc:260] T 8abfa1aa35c44699960a8166fa79c41d P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:31.210397 19994 ts_tablet_manager.cc:1434] T 8abfa1aa35c44699960a8166fa79c41d P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:20:31.202591 19993 tablet_bootstrap.cc:492] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P e90ffed3ba174ababeb21d9415d4d918: Bootstrap starting.
I20260709 13:20:31.211560 19994 tablet_bootstrap.cc:492] T ece79531bbd749d3b9300fe2fbeef9af P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:31.222496 19997 tablet_bootstrap.cc:492] T 9f60d4b859264720a20f5670c3c71a6f P b80b3a62d7d549a9a5c42def46769ac4: No bootstrap required, opened a new log
I20260709 13:20:31.223078 19997 ts_tablet_manager.cc:1403] T 9f60d4b859264720a20f5670c3c71a6f P b80b3a62d7d549a9a5c42def46769ac4: Time spent bootstrapping tablet: real 0.042s	user 0.012s	sys 0.000s
I20260709 13:20:31.225934 19997 raft_consensus.cc:359] T 9f60d4b859264720a20f5670c3c71a6f P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:31.226619 19997 raft_consensus.cc:385] T 9f60d4b859264720a20f5670c3c71a6f P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:31.227013 19997 raft_consensus.cc:740] T 9f60d4b859264720a20f5670c3c71a6f P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Initialized, Role: FOLLOWER
I20260709 13:20:31.227830 19994 tablet_bootstrap.cc:654] T ece79531bbd749d3b9300fe2fbeef9af P 8df38c4c53c44e94b36c1e8992476d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:31.232569 19993 tablet_bootstrap.cc:654] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P e90ffed3ba174ababeb21d9415d4d918: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:31.233873 19997 consensus_queue.cc:260] T 9f60d4b859264720a20f5670c3c71a6f P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
W20260709 13:20:31.245146 19840 consensus_peers.cc:597] T ece79531bbd749d3b9300fe2fbeef9af P b80b3a62d7d549a9a5c42def46769ac4 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:31.246829 20325 consensus_queue.cc:1048] T 563940ad96444ce59e393ea303cc3352 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:20:31.256702 20325 raft_consensus.cc:493] T dc5c36b41652443a8130592df8a66eb9 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:31.257355 20325 raft_consensus.cc:515] T dc5c36b41652443a8130592df8a66eb9 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:31.261397 20325 leader_election.cc:290] T dc5c36b41652443a8130592df8a66eb9 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:31.264026 19874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5c36b41652443a8130592df8a66eb9" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:31.264992 19874 raft_consensus.cc:2468] T dc5c36b41652443a8130592df8a66eb9 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 0.
I20260709 13:20:31.267234 19797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5c36b41652443a8130592df8a66eb9" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:31.267607 19690 leader_election.cc:304] T dc5c36b41652443a8130592df8a66eb9 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4; no voters: 
I20260709 13:20:31.268358 19797 raft_consensus.cc:2468] T dc5c36b41652443a8130592df8a66eb9 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 0.
I20260709 13:20:31.270547 20132 raft_consensus.cc:2804] T dc5c36b41652443a8130592df8a66eb9 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:31.271169 20132 raft_consensus.cc:493] T dc5c36b41652443a8130592df8a66eb9 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:31.271625 20132 raft_consensus.cc:3060] T dc5c36b41652443a8130592df8a66eb9 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:31.276065 19997 ts_tablet_manager.cc:1434] T 9f60d4b859264720a20f5670c3c71a6f P b80b3a62d7d549a9a5c42def46769ac4: Time spent starting tablet: real 0.053s	user 0.007s	sys 0.000s
I20260709 13:20:31.277645 19997 tablet_bootstrap.cc:492] T 8fd58e41ecee4779b3e72df3aea4463a P b80b3a62d7d549a9a5c42def46769ac4: Bootstrap starting.
I20260709 13:20:31.297055 20355 raft_consensus.cc:493] T 87364a1289ed4aa1b9b2a7a9951ed776 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:31.297649 20355 raft_consensus.cc:515] T 87364a1289ed4aa1b9b2a7a9951ed776 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:31.303697 19994 tablet_bootstrap.cc:492] T ece79531bbd749d3b9300fe2fbeef9af P 8df38c4c53c44e94b36c1e8992476d7c: No bootstrap required, opened a new log
I20260709 13:20:31.304265 19994 ts_tablet_manager.cc:1403] T ece79531bbd749d3b9300fe2fbeef9af P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.093s	user 0.013s	sys 0.000s
I20260709 13:20:31.332710 20355 leader_election.cc:290] T 87364a1289ed4aa1b9b2a7a9951ed776 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:31.334616 20355 raft_consensus.cc:493] T 9f60d4b859264720a20f5670c3c71a6f P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:31.335165 20355 raft_consensus.cc:515] T 9f60d4b859264720a20f5670c3c71a6f P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:31.327428 19994 raft_consensus.cc:359] T ece79531bbd749d3b9300fe2fbeef9af P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:31.340545 19994 raft_consensus.cc:385] T ece79531bbd749d3b9300fe2fbeef9af P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:31.341004 19994 raft_consensus.cc:740] T ece79531bbd749d3b9300fe2fbeef9af P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:31.344064 20355 leader_election.cc:290] T 9f60d4b859264720a20f5670c3c71a6f P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:31.345248 19945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f60d4b859264720a20f5670c3c71a6f" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:31.346010 19722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f60d4b859264720a20f5670c3c71a6f" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:31.347378 19941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87364a1289ed4aa1b9b2a7a9951ed776" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
W20260709 13:20:31.348377 19840 leader_election.cc:343] T 9f60d4b859264720a20f5670c3c71a6f P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:31.349967 19797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87364a1289ed4aa1b9b2a7a9951ed776" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:31.346341 19945 raft_consensus.cc:2468] T 9f60d4b859264720a20f5670c3c71a6f P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 0.
I20260709 13:20:31.354696 19994 consensus_queue.cc:260] T ece79531bbd749d3b9300fe2fbeef9af P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:31.357311 20132 raft_consensus.cc:515] T dc5c36b41652443a8130592df8a66eb9 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:31.365795 19941 raft_consensus.cc:2468] T 87364a1289ed4aa1b9b2a7a9951ed776 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 0.
I20260709 13:20:31.365396 20181 raft_consensus.cc:493] T 0a9c6b64c8e04c4283d24ae6290a52ea P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:31.373888 20080 raft_consensus.cc:493] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:31.376539 20181 raft_consensus.cc:515] T 0a9c6b64c8e04c4283d24ae6290a52ea P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:31.376678 20080 raft_consensus.cc:515] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:31.382475 19874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9c6b64c8e04c4283d24ae6290a52ea" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:31.383201 19874 raft_consensus.cc:2468] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 0.
I20260709 13:20:31.384191 19722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9c6b64c8e04c4283d24ae6290a52ea" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:31.384912 19722 raft_consensus.cc:2468] T 0a9c6b64c8e04c4283d24ae6290a52ea P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 0.
I20260709 13:20:31.387554 19997 tablet_bootstrap.cc:654] T 8fd58e41ecee4779b3e72df3aea4463a P b80b3a62d7d549a9a5c42def46769ac4: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:31.387835 19690 leader_election.cc:304] T 0a9c6b64c8e04c4283d24ae6290a52ea P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4; no voters: 
I20260709 13:20:31.388980 20325 raft_consensus.cc:2804] T 0a9c6b64c8e04c4283d24ae6290a52ea P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:31.389365 20325 raft_consensus.cc:493] T 0a9c6b64c8e04c4283d24ae6290a52ea P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:31.389703 20325 raft_consensus.cc:3060] T 0a9c6b64c8e04c4283d24ae6290a52ea P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:31.386274 19800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9c6b64c8e04c4283d24ae6290a52ea" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:31.390273 20083 consensus_queue.cc:1048] T 563940ad96444ce59e393ea303cc3352 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:20:31.391824 19798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9c6b64c8e04c4283d24ae6290a52ea" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:31.393765 20181 leader_election.cc:290] T 0a9c6b64c8e04c4283d24ae6290a52ea P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:31.398350 19994 ts_tablet_manager.cc:1434] T ece79531bbd749d3b9300fe2fbeef9af P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.094s	user 0.011s	sys 0.000s
I20260709 13:20:31.399621 19994 tablet_bootstrap.cc:492] T bc04a038495940c1bbeda4a2e6f17b82 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
W20260709 13:20:31.404451 19690 leader_election.cc:343] T 0a9c6b64c8e04c4283d24ae6290a52ea P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:31.408403 19839 leader_election.cc:304] T 87364a1289ed4aa1b9b2a7a9951ed776 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430; no voters: 
I20260709 13:20:31.409615 20353 raft_consensus.cc:2804] T 87364a1289ed4aa1b9b2a7a9951ed776 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:31.410117 20353 raft_consensus.cc:493] T 87364a1289ed4aa1b9b2a7a9951ed776 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:31.410490 20353 raft_consensus.cc:3060] T 87364a1289ed4aa1b9b2a7a9951ed776 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:31.381397 20080 leader_election.cc:290] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:31.415305 19840 leader_election.cc:304] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4; no voters: 
I20260709 13:20:31.416635 20355 raft_consensus.cc:2804] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:31.417048 20355 raft_consensus.cc:493] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:31.417418 20355 raft_consensus.cc:3060] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:31.417173 19839 leader_election.cc:304] T 9f60d4b859264720a20f5670c3c71a6f P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:31.420306 20350 raft_consensus.cc:493] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:31.420874 20350 raft_consensus.cc:515] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:31.422010 20080 raft_consensus.cc:2804] T 9f60d4b859264720a20f5670c3c71a6f P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:31.422717 20080 raft_consensus.cc:493] T 9f60d4b859264720a20f5670c3c71a6f P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:31.423197 20080 raft_consensus.cc:3060] T 9f60d4b859264720a20f5670c3c71a6f P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:31.423399 20350 leader_election.cc:290] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:31.424482 19722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8abfa1aa35c44699960a8166fa79c41d" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:31.426095 19722 raft_consensus.cc:2468] T 8abfa1aa35c44699960a8166fa79c41d P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c62716bf66c24330860573535b8b9430 in term 0.
I20260709 13:20:31.427184 19914 leader_election.cc:304] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, c62716bf66c24330860573535b8b9430; no voters: 
I20260709 13:20:31.428289 20237 raft_consensus.cc:2804] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:31.428746 20237 raft_consensus.cc:493] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:31.429108 20237 raft_consensus.cc:3060] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:31.436883 20325 raft_consensus.cc:515] T 0a9c6b64c8e04c4283d24ae6290a52ea P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:31.438068 19795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5c36b41652443a8130592df8a66eb9" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
I20260709 13:20:31.441151 20132 leader_election.cc:290] T dc5c36b41652443a8130592df8a66eb9 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Requested vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
W20260709 13:20:31.442688 19839 leader_election.cc:343] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:31.448004 19874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5c36b41652443a8130592df8a66eb9" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
W20260709 13:20:31.447219 19839 leader_election.cc:343] T 87364a1289ed4aa1b9b2a7a9951ed776 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:31.449021 19874 raft_consensus.cc:3060] T dc5c36b41652443a8130592df8a66eb9 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:31.454008 19798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8abfa1aa35c44699960a8166fa79c41d" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
W20260709 13:20:31.455886 19913 leader_election.cc:343] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:31.445441 19795 raft_consensus.cc:3060] T dc5c36b41652443a8130592df8a66eb9 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:31.459472 19995 tablet_bootstrap.cc:492] T 29f92a880ce04259ae0ee9c5288949ff P c62716bf66c24330860573535b8b9430: No bootstrap required, opened a new log
I20260709 13:20:31.461643 19995 ts_tablet_manager.cc:1403] T 29f92a880ce04259ae0ee9c5288949ff P c62716bf66c24330860573535b8b9430: Time spent bootstrapping tablet: real 0.336s	user 0.028s	sys 0.046s
I20260709 13:20:31.465865 20350 raft_consensus.cc:493] T 8fd58e41ecee4779b3e72df3aea4463a P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:31.450712 20355 raft_consensus.cc:515] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:31.466738 19797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9c6b64c8e04c4283d24ae6290a52ea" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
I20260709 13:20:31.490696 19871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9c6b64c8e04c4283d24ae6290a52ea" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
W20260709 13:20:31.494414 19690 leader_election.cc:343] T 0a9c6b64c8e04c4283d24ae6290a52ea P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:31.497848 19993 tablet_bootstrap.cc:492] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P e90ffed3ba174ababeb21d9415d4d918: No bootstrap required, opened a new log
I20260709 13:20:31.498301 19993 ts_tablet_manager.cc:1403] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P e90ffed3ba174ababeb21d9415d4d918: Time spent bootstrapping tablet: real 0.296s	user 0.008s	sys 0.051s
W20260709 13:20:31.500327 19839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:43319, user_credentials={real_user=slave}} blocked reactor thread for 49923.1us
I20260709 13:20:31.485942 20325 leader_election.cc:290] T 0a9c6b64c8e04c4283d24ae6290a52ea P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Requested vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:31.467387 20353 raft_consensus.cc:515] T 87364a1289ed4aa1b9b2a7a9951ed776 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:31.506527 20079 consensus_queue.cc:1048] T ece79531bbd749d3b9300fe2fbeef9af P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:20:31.509415 19871 raft_consensus.cc:2393] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8df38c4c53c44e94b36c1e8992476d7c in current term 1: Already voted for candidate b80b3a62d7d549a9a5c42def46769ac4 in this term.
I20260709 13:20:31.510943 19994 tablet_bootstrap.cc:654] T bc04a038495940c1bbeda4a2e6f17b82 P 8df38c4c53c44e94b36c1e8992476d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:31.516309 20353 leader_election.cc:290] T 87364a1289ed4aa1b9b2a7a9951ed776 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 election: Requested vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:31.522898 19725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9c6b64c8e04c4283d24ae6290a52ea" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
I20260709 13:20:31.524222 19993 raft_consensus.cc:359] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:31.524973 19993 raft_consensus.cc:385] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:31.523834 19725 raft_consensus.cc:2393] T 0a9c6b64c8e04c4283d24ae6290a52ea P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b80b3a62d7d549a9a5c42def46769ac4 in current term 1: Already voted for candidate 8df38c4c53c44e94b36c1e8992476d7c in this term.
I20260709 13:20:31.525234 19993 raft_consensus.cc:740] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Initialized, Role: FOLLOWER
I20260709 13:20:31.531697 19874 raft_consensus.cc:2468] T dc5c36b41652443a8130592df8a66eb9 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 1.
I20260709 13:20:31.533215 19993 consensus_queue.cc:260] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:31.543186 19941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87364a1289ed4aa1b9b2a7a9951ed776" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:31.543916 19941 raft_consensus.cc:3060] T 87364a1289ed4aa1b9b2a7a9951ed776 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:31.545496 19997 tablet_bootstrap.cc:492] T 8fd58e41ecee4779b3e72df3aea4463a P b80b3a62d7d549a9a5c42def46769ac4: No bootstrap required, opened a new log
I20260709 13:20:31.546252 19997 ts_tablet_manager.cc:1403] T 8fd58e41ecee4779b3e72df3aea4463a P b80b3a62d7d549a9a5c42def46769ac4: Time spent bootstrapping tablet: real 0.269s	user 0.015s	sys 0.006s
I20260709 13:20:31.548538 19690 leader_election.cc:304] T dc5c36b41652443a8130592df8a66eb9 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4; no voters: 
I20260709 13:20:31.551692 20355 leader_election.cc:290] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 election: Requested vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:31.553514 19795 raft_consensus.cc:2468] T dc5c36b41652443a8130592df8a66eb9 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 1.
I20260709 13:20:31.553931 19993 ts_tablet_manager.cc:1434] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P e90ffed3ba174ababeb21d9415d4d918: Time spent starting tablet: real 0.055s	user 0.005s	sys 0.003s
I20260709 13:20:31.553645 20080 raft_consensus.cc:515] T 9f60d4b859264720a20f5670c3c71a6f P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:31.555766 20079 raft_consensus.cc:493] T cb33acfa7a9f46ab95b0e77e1394f258 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:31.556358 20079 raft_consensus.cc:515] T cb33acfa7a9f46ab95b0e77e1394f258 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:31.496656 20350 raft_consensus.cc:515] T 8fd58e41ecee4779b3e72df3aea4463a P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:31.564522 19690 leader_election.cc:304] T 0a9c6b64c8e04c4283d24ae6290a52ea P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c; no voters: b80b3a62d7d549a9a5c42def46769ac4, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:31.542259 19995 raft_consensus.cc:359] T 29f92a880ce04259ae0ee9c5288949ff P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:31.566442 19995 raft_consensus.cc:385] T 29f92a880ce04259ae0ee9c5288949ff P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:31.567093 19995 raft_consensus.cc:740] T 29f92a880ce04259ae0ee9c5288949ff P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62716bf66c24330860573535b8b9430, State: Initialized, Role: FOLLOWER
I20260709 13:20:31.558691 20079 leader_election.cc:290] T cb33acfa7a9f46ab95b0e77e1394f258 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:31.563083 20083 raft_consensus.cc:2804] T dc5c36b41652443a8130592df8a66eb9 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:31.570182 19995 consensus_queue.cc:260] T 29f92a880ce04259ae0ee9c5288949ff P c62716bf66c24330860573535b8b9430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:31.590881 19995 ts_tablet_manager.cc:1434] T 29f92a880ce04259ae0ee9c5288949ff P c62716bf66c24330860573535b8b9430: Time spent starting tablet: real 0.129s	user 0.015s	sys 0.000s
I20260709 13:20:31.603730 19722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb33acfa7a9f46ab95b0e77e1394f258" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
W20260709 13:20:31.605362 20377 log.cc:927] Time spent T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P c62716bf66c24330860573535b8b9430: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:20:31.605567 20237 raft_consensus.cc:515] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:31.608357 20350 leader_election.cc:290] T 8fd58e41ecee4779b3e72df3aea4463a P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:31.608852 20237 leader_election.cc:290] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 election: Requested vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:31.631554 19995 tablet_bootstrap.cc:492] T 6551866a22dd4aaf8ee318b9d91b3a02 P c62716bf66c24330860573535b8b9430: Bootstrap starting.
W20260709 13:20:31.633242 19690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:45249, user_credentials={real_user=slave}} blocked reactor thread for 68808.2us
I20260709 13:20:31.601864 20325 raft_consensus.cc:2749] T 0a9c6b64c8e04c4283d24ae6290a52ea P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:20:31.595842 20301 consensus_queue.cc:1048] T 1cceb7f75d7040c2b98049590928b891 P c62716bf66c24330860573535b8b9430 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.041s
I20260709 13:20:31.643544 19994 tablet_bootstrap.cc:492] T bc04a038495940c1bbeda4a2e6f17b82 P 8df38c4c53c44e94b36c1e8992476d7c: No bootstrap required, opened a new log
I20260709 13:20:31.644088 19994 ts_tablet_manager.cc:1403] T bc04a038495940c1bbeda4a2e6f17b82 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.245s	user 0.015s	sys 0.022s
I20260709 13:20:31.647563 19798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb33acfa7a9f46ab95b0e77e1394f258" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:31.647745 19797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9c6b64c8e04c4283d24ae6290a52ea" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
W20260709 13:20:31.640336 19840 leader_election.cc:343] T cb33acfa7a9f46ab95b0e77e1394f258 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:31.657009 19796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87364a1289ed4aa1b9b2a7a9951ed776" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
I20260709 13:20:31.661129 19993 tablet_bootstrap.cc:492] T b832efa5a2e34b3894e33629110e6fef P e90ffed3ba174ababeb21d9415d4d918: Bootstrap starting.
I20260709 13:20:31.661697 19941 raft_consensus.cc:2468] T 87364a1289ed4aa1b9b2a7a9951ed776 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 1.
I20260709 13:20:31.674084 19997 raft_consensus.cc:359] T 8fd58e41ecee4779b3e72df3aea4463a P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:31.675087 19997 raft_consensus.cc:385] T 8fd58e41ecee4779b3e72df3aea4463a P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:31.675385 19997 raft_consensus.cc:740] T 8fd58e41ecee4779b3e72df3aea4463a P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Initialized, Role: FOLLOWER
I20260709 13:20:31.676923 19997 consensus_queue.cc:260] T 8fd58e41ecee4779b3e72df3aea4463a P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:31.678520 20080 leader_election.cc:290] T 9f60d4b859264720a20f5670c3c71a6f P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 election: Requested vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:31.676066 19623 catalog_manager.cc:5697] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P c62716bf66c24330860573535b8b9430 reported cstate change: term changed from 0 to 1, leader changed from <none> to c62716bf66c24330860573535b8b9430 (127.9.116.132). New cstate: current_term: 1 leader_uuid: "c62716bf66c24330860573535b8b9430" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:31.681320 19723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f60d4b859264720a20f5670c3c71a6f" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
I20260709 13:20:31.682071 19993 tablet_bootstrap.cc:654] T b832efa5a2e34b3894e33629110e6fef P e90ffed3ba174ababeb21d9415d4d918: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:31.681308 19997 ts_tablet_manager.cc:1434] T 8fd58e41ecee4779b3e72df3aea4463a P b80b3a62d7d549a9a5c42def46769ac4: Time spent starting tablet: real 0.135s	user 0.008s	sys 0.000s
W20260709 13:20:31.708289 19913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:43319, user_credentials={real_user=slave}} blocked reactor thread for 110062us
I20260709 13:20:31.718159 19839 leader_election.cc:304] T 87364a1289ed4aa1b9b2a7a9951ed776 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430; no voters: 
I20260709 13:20:31.724540 19945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f60d4b859264720a20f5670c3c71a6f" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:31.725274 19945 raft_consensus.cc:3060] T 9f60d4b859264720a20f5670c3c71a6f P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:31.725770 19997 tablet_bootstrap.cc:492] T 59fc86c7e9f648e3b3f51e936e2bcfec P b80b3a62d7d549a9a5c42def46769ac4: Bootstrap starting.
I20260709 13:20:31.708000 20083 raft_consensus.cc:697] T dc5c36b41652443a8130592df8a66eb9 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 LEADER]: Becoming Leader. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Running, Role: LEADER
I20260709 13:20:31.732818 19997 tablet_bootstrap.cc:654] T 59fc86c7e9f648e3b3f51e936e2bcfec P b80b3a62d7d549a9a5c42def46769ac4: Neither blocks nor log segments found. Creating new log.
W20260709 13:20:31.711678 19840 leader_election.cc:343] T 9f60d4b859264720a20f5670c3c71a6f P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:31.735121 20388 raft_consensus.cc:2804] T 87364a1289ed4aa1b9b2a7a9951ed776 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:31.754952 19796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8abfa1aa35c44699960a8166fa79c41d" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
I20260709 13:20:31.755722 19722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd58e41ecee4779b3e72df3aea4463a" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:31.755646 19994 raft_consensus.cc:359] T bc04a038495940c1bbeda4a2e6f17b82 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:31.756516 19994 raft_consensus.cc:385] T bc04a038495940c1bbeda4a2e6f17b82 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:31.756812 19994 raft_consensus.cc:740] T bc04a038495940c1bbeda4a2e6f17b82 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:31.758036 19994 consensus_queue.cc:260] T bc04a038495940c1bbeda4a2e6f17b82 P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:31.740702 19871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd58e41ecee4779b3e72df3aea4463a" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:31.761600 19871 raft_consensus.cc:2468] T 8fd58e41ecee4779b3e72df3aea4463a P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c62716bf66c24330860573535b8b9430 in term 0.
W20260709 13:20:31.761935 19839 leader_election.cc:343] T cb33acfa7a9f46ab95b0e77e1394f258 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:31.762429 19839 leader_election.cc:304] T cb33acfa7a9f46ab95b0e77e1394f258 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4; no voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:31.762878 19994 ts_tablet_manager.cc:1434] T bc04a038495940c1bbeda4a2e6f17b82 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.119s	user 0.007s	sys 0.001s
I20260709 13:20:31.763769 20353 raft_consensus.cc:2749] T cb33acfa7a9f46ab95b0e77e1394f258 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:31.763983 19994 tablet_bootstrap.cc:492] T cb33acfa7a9f46ab95b0e77e1394f258 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:31.764483 19995 tablet_bootstrap.cc:654] T 6551866a22dd4aaf8ee318b9d91b3a02 P c62716bf66c24330860573535b8b9430: Neither blocks nor log segments found. Creating new log.
W20260709 13:20:31.769062 19839 leader_election.cc:343] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:31.769562 19839 leader_election.cc:304] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4; no voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:31.770255 20392 raft_consensus.cc:2749] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:20:31.771400 19839 leader_election.cc:343] T 87364a1289ed4aa1b9b2a7a9951ed776 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:31.777336 19945 raft_consensus.cc:2468] T 9f60d4b859264720a20f5670c3c71a6f P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 1.
I20260709 13:20:31.757961 20083 consensus_queue.cc:237] T dc5c36b41652443a8130592df8a66eb9 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
W20260709 13:20:31.786764 19914 leader_election.cc:343] T 8fd58e41ecee4779b3e72df3aea4463a P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:20:31.789156 19913 leader_election.cc:343] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:31.791910 19994 tablet_bootstrap.cc:654] T cb33acfa7a9f46ab95b0e77e1394f258 P 8df38c4c53c44e94b36c1e8992476d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:31.793087 19839 leader_election.cc:304] T 9f60d4b859264720a20f5670c3c71a6f P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:31.793627 20388 raft_consensus.cc:697] T 87364a1289ed4aa1b9b2a7a9951ed776 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 LEADER]: Becoming Leader. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Running, Role: LEADER
I20260709 13:20:31.794622 20388 consensus_queue.cc:237] T 87364a1289ed4aa1b9b2a7a9951ed776 P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:31.796741 20353 raft_consensus.cc:2804] T 9f60d4b859264720a20f5670c3c71a6f P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:31.797313 20353 raft_consensus.cc:697] T 9f60d4b859264720a20f5670c3c71a6f P b80b3a62d7d549a9a5c42def46769ac4 [term 1 LEADER]: Becoming Leader. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Running, Role: LEADER
I20260709 13:20:31.798170 20353 consensus_queue.cc:237] T 9f60d4b859264720a20f5670c3c71a6f P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:31.798939 19997 tablet_bootstrap.cc:492] T 59fc86c7e9f648e3b3f51e936e2bcfec P b80b3a62d7d549a9a5c42def46769ac4: No bootstrap required, opened a new log
I20260709 13:20:31.798259 19995 tablet_bootstrap.cc:492] T 6551866a22dd4aaf8ee318b9d91b3a02 P c62716bf66c24330860573535b8b9430: No bootstrap required, opened a new log
I20260709 13:20:31.800011 19997 ts_tablet_manager.cc:1403] T 59fc86c7e9f648e3b3f51e936e2bcfec P b80b3a62d7d549a9a5c42def46769ac4: Time spent bootstrapping tablet: real 0.078s	user 0.012s	sys 0.001s
I20260709 13:20:31.800160 19995 ts_tablet_manager.cc:1403] T 6551866a22dd4aaf8ee318b9d91b3a02 P c62716bf66c24330860573535b8b9430: Time spent bootstrapping tablet: real 0.189s	user 0.015s	sys 0.000s
I20260709 13:20:31.811839 19995 raft_consensus.cc:359] T 6551866a22dd4aaf8ee318b9d91b3a02 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:31.813750 19995 raft_consensus.cc:385] T 6551866a22dd4aaf8ee318b9d91b3a02 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:31.814080 19995 raft_consensus.cc:740] T 6551866a22dd4aaf8ee318b9d91b3a02 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62716bf66c24330860573535b8b9430, State: Initialized, Role: FOLLOWER
I20260709 13:20:31.814970 19995 consensus_queue.cc:260] T 6551866a22dd4aaf8ee318b9d91b3a02 P c62716bf66c24330860573535b8b9430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:31.815198 19913 leader_election.cc:304] T 8fd58e41ecee4779b3e72df3aea4463a P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:31.829427 19721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8abfa1aa35c44699960a8166fa79c41d" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
I20260709 13:20:31.834236 20370 raft_consensus.cc:2804] T 8fd58e41ecee4779b3e72df3aea4463a P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:31.834354 20325 raft_consensus.cc:493] T ece79531bbd749d3b9300fe2fbeef9af P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:31.839588 20325 raft_consensus.cc:515] T ece79531bbd749d3b9300fe2fbeef9af P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:31.841717 20325 leader_election.cc:290] T ece79531bbd749d3b9300fe2fbeef9af P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:31.838397 20370 raft_consensus.cc:493] T 8fd58e41ecee4779b3e72df3aea4463a P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:31.842702 19796 raft_consensus.cc:3060] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:31.838717 19721 raft_consensus.cc:3060] T 8abfa1aa35c44699960a8166fa79c41d P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:31.846179 19997 raft_consensus.cc:359] T 59fc86c7e9f648e3b3f51e936e2bcfec P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:31.852684 19997 raft_consensus.cc:385] T 59fc86c7e9f648e3b3f51e936e2bcfec P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:31.852999 19997 raft_consensus.cc:740] T 59fc86c7e9f648e3b3f51e936e2bcfec P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Initialized, Role: FOLLOWER
I20260709 13:20:31.853724 19997 consensus_queue.cc:260] T 59fc86c7e9f648e3b3f51e936e2bcfec P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:31.852699 19622 catalog_manager.cc:5697] T dc5c36b41652443a8130592df8a66eb9 P 8df38c4c53c44e94b36c1e8992476d7c reported cstate change: term changed from 0 to 1, leader changed from <none> to 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129). New cstate: current_term: 1 leader_uuid: "8df38c4c53c44e94b36c1e8992476d7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:31.858698 19995 ts_tablet_manager.cc:1434] T 6551866a22dd4aaf8ee318b9d91b3a02 P c62716bf66c24330860573535b8b9430: Time spent starting tablet: real 0.058s	user 0.007s	sys 0.002s
I20260709 13:20:31.842890 20370 raft_consensus.cc:3060] T 8fd58e41ecee4779b3e72df3aea4463a P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:31.864499 19997 ts_tablet_manager.cc:1434] T 59fc86c7e9f648e3b3f51e936e2bcfec P b80b3a62d7d549a9a5c42def46769ac4: Time spent starting tablet: real 0.064s	user 0.005s	sys 0.001s
I20260709 13:20:31.865787 19997 tablet_bootstrap.cc:492] T 29f92a880ce04259ae0ee9c5288949ff P b80b3a62d7d549a9a5c42def46769ac4: Bootstrap starting.
I20260709 13:20:31.868625 19723 raft_consensus.cc:3060] T ece79531bbd749d3b9300fe2fbeef9af P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:31.869572 19994 tablet_bootstrap.cc:492] T cb33acfa7a9f46ab95b0e77e1394f258 P 8df38c4c53c44e94b36c1e8992476d7c: No bootstrap required, opened a new log
I20260709 13:20:31.870107 19994 ts_tablet_manager.cc:1403] T cb33acfa7a9f46ab95b0e77e1394f258 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.106s	user 0.007s	sys 0.008s
I20260709 13:20:31.878484 19997 tablet_bootstrap.cc:654] T 29f92a880ce04259ae0ee9c5288949ff P b80b3a62d7d549a9a5c42def46769ac4: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:31.883905 19944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece79531bbd749d3b9300fe2fbeef9af" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:31.885823 19994 raft_consensus.cc:359] T cb33acfa7a9f46ab95b0e77e1394f258 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:31.888378 20237 consensus_queue.cc:1048] T 1cceb7f75d7040c2b98049590928b891 P c62716bf66c24330860573535b8b9430 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:31.886611 19994 raft_consensus.cc:385] T cb33acfa7a9f46ab95b0e77e1394f258 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:31.892284 19994 raft_consensus.cc:740] T cb33acfa7a9f46ab95b0e77e1394f258 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:31.902190 19994 consensus_queue.cc:260] T cb33acfa7a9f46ab95b0e77e1394f258 P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:31.904723 19874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece79531bbd749d3b9300fe2fbeef9af" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:31.878199 20370 raft_consensus.cc:515] T 8fd58e41ecee4779b3e72df3aea4463a P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:31.908761 19993 tablet_bootstrap.cc:492] T b832efa5a2e34b3894e33629110e6fef P e90ffed3ba174ababeb21d9415d4d918: No bootstrap required, opened a new log
I20260709 13:20:31.909327 19993 ts_tablet_manager.cc:1403] T b832efa5a2e34b3894e33629110e6fef P e90ffed3ba174ababeb21d9415d4d918: Time spent bootstrapping tablet: real 0.248s	user 0.011s	sys 0.014s
I20260709 13:20:31.914364 20414 raft_consensus.cc:493] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:31.915122 20414 raft_consensus.cc:515] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:31.924634 19725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd58e41ecee4779b3e72df3aea4463a" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
I20260709 13:20:31.924968 19994 ts_tablet_manager.cc:1434] T cb33acfa7a9f46ab95b0e77e1394f258 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.055s	user 0.008s	sys 0.002s
I20260709 13:20:31.924732 20397 consensus_queue.cc:1048] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P c62716bf66c24330860573535b8b9430 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:20:31.926352 19914 leader_election.cc:343] T 8fd58e41ecee4779b3e72df3aea4463a P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:31.909432 20370 leader_election.cc:290] T 8fd58e41ecee4779b3e72df3aea4463a P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 election: Requested vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:31.922600 19868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7bc0eba7e94b9eb1eeec9bbd1a6e97" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:31.945477 19870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd58e41ecee4779b3e72df3aea4463a" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
I20260709 13:20:31.946218 19870 raft_consensus.cc:3060] T 8fd58e41ecee4779b3e72df3aea4463a P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:31.947973 19994 tablet_bootstrap.cc:492] T bd80f9f8533b4cd3b5861ec11917a03d P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:31.955665 19620 catalog_manager.cc:5697] T 1cceb7f75d7040c2b98049590928b891 P e90ffed3ba174ababeb21d9415d4d918 reported cstate change: term changed from 0 to 1, leader changed from <none> to c62716bf66c24330860573535b8b9430 (127.9.116.132). New cstate: current_term: 1 leader_uuid: "c62716bf66c24330860573535b8b9430" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } }
I20260709 13:20:31.939951 19721 raft_consensus.cc:2468] T 8abfa1aa35c44699960a8166fa79c41d P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c62716bf66c24330860573535b8b9430 in term 1.
I20260709 13:20:31.975456 19993 raft_consensus.cc:359] T b832efa5a2e34b3894e33629110e6fef P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:31.977103 19993 raft_consensus.cc:385] T b832efa5a2e34b3894e33629110e6fef P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:31.977329 19993 raft_consensus.cc:740] T b832efa5a2e34b3894e33629110e6fef P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Initialized, Role: FOLLOWER
I20260709 13:20:31.977759 19993 consensus_queue.cc:260] T b832efa5a2e34b3894e33629110e6fef P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:31.971879 19690 leader_election.cc:304] T ece79531bbd749d3b9300fe2fbeef9af P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c; no voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430
I20260709 13:20:31.989166 19941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7bc0eba7e94b9eb1eeec9bbd1a6e97" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:31.998493 19993 ts_tablet_manager.cc:1434] T b832efa5a2e34b3894e33629110e6fef P e90ffed3ba174ababeb21d9415d4d918: Time spent starting tablet: real 0.089s	user 0.006s	sys 0.001s
I20260709 13:20:31.993507 20301 raft_consensus.cc:493] T 29f92a880ce04259ae0ee9c5288949ff P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:31.968281 20414 leader_election.cc:290] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:32.014628 20399 raft_consensus.cc:493] T cb33acfa7a9f46ab95b0e77e1394f258 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:32.015596 19993 tablet_bootstrap.cc:492] T 0a9c6b64c8e04c4283d24ae6290a52ea P e90ffed3ba174ababeb21d9415d4d918: Bootstrap starting.
I20260709 13:20:32.015285 20399 raft_consensus.cc:515] T cb33acfa7a9f46ab95b0e77e1394f258 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:32.019171 20350 raft_consensus.cc:493] T 6551866a22dd4aaf8ee318b9d91b3a02 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:32.019768 20350 raft_consensus.cc:515] T 6551866a22dd4aaf8ee318b9d91b3a02 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:32.020231 20399 leader_election.cc:290] T cb33acfa7a9f46ab95b0e77e1394f258 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:32.021927 20350 leader_election.cc:290] T 6551866a22dd4aaf8ee318b9d91b3a02 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:32.024446 19872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb33acfa7a9f46ab95b0e77e1394f258" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:32.025285 19872 raft_consensus.cc:2468] T cb33acfa7a9f46ab95b0e77e1394f258 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 0.
I20260709 13:20:32.027961 19795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6551866a22dd4aaf8ee318b9d91b3a02" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:32.007097 20301 raft_consensus.cc:515] T 29f92a880ce04259ae0ee9c5288949ff P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
W20260709 13:20:32.030175 19913 leader_election.cc:343] T 6551866a22dd4aaf8ee318b9d91b3a02 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:32.029510 19690 leader_election.cc:304] T cb33acfa7a9f46ab95b0e77e1394f258 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4; no voters: 
I20260709 13:20:32.034030 20399 raft_consensus.cc:2804] T cb33acfa7a9f46ab95b0e77e1394f258 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:32.034919 20399 raft_consensus.cc:493] T cb33acfa7a9f46ab95b0e77e1394f258 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:32.035274 20399 raft_consensus.cc:3060] T cb33acfa7a9f46ab95b0e77e1394f258 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:32.040343 19994 tablet_bootstrap.cc:654] T bd80f9f8533b4cd3b5861ec11917a03d P 8df38c4c53c44e94b36c1e8992476d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:32.031279 20301 leader_election.cc:290] T 29f92a880ce04259ae0ee9c5288949ff P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:32.042443 19765 leader_election.cc:304] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e90ffed3ba174ababeb21d9415d4d918; no voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430
I20260709 13:20:32.060458 19868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29f92a880ce04259ae0ee9c5288949ff" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:32.065899 19993 tablet_bootstrap.cc:654] T 0a9c6b64c8e04c4283d24ae6290a52ea P e90ffed3ba174ababeb21d9415d4d918: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:32.071095 19795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29f92a880ce04259ae0ee9c5288949ff" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
W20260709 13:20:32.072860 19913 leader_election.cc:343] T 29f92a880ce04259ae0ee9c5288949ff P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:32.073632 19796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb33acfa7a9f46ab95b0e77e1394f258" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:32.067898 20083 raft_consensus.cc:493] T bc04a038495940c1bbeda4a2e6f17b82 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:32.084344 19690 leader_election.cc:343] T cb33acfa7a9f46ab95b0e77e1394f258 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:20:32.085062 19914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:41855, user_credentials={real_user=slave}} blocked reactor thread for 91030.7us
I20260709 13:20:32.095213 19914 leader_election.cc:304] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, c62716bf66c24330860573535b8b9430; no voters: e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:32.095603 19724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6551866a22dd4aaf8ee318b9d91b3a02" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:32.096868 20237 raft_consensus.cc:2804] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:32.097430 20237 raft_consensus.cc:697] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 [term 1 LEADER]: Becoming Leader. State: Replica: c62716bf66c24330860573535b8b9430, State: Running, Role: LEADER
W20260709 13:20:32.097486 19914 leader_election.cc:343] T 6551866a22dd4aaf8ee318b9d91b3a02 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:32.098126 19914 leader_election.cc:304] T 6551866a22dd4aaf8ee318b9d91b3a02 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:32.098523 20237 consensus_queue.cc:237] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:32.098953 20420 raft_consensus.cc:2749] T 6551866a22dd4aaf8ee318b9d91b3a02 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:32.086225 20083 raft_consensus.cc:515] T bc04a038495940c1bbeda4a2e6f17b82 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:32.134670 20040 raft_consensus.cc:493] T b832efa5a2e34b3894e33629110e6fef P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:32.135326 20040 raft_consensus.cc:515] T b832efa5a2e34b3894e33629110e6fef P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:32.137380 20040 leader_election.cc:290] T b832efa5a2e34b3894e33629110e6fef P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:32.140702 20394 raft_consensus.cc:2749] T ece79531bbd749d3b9300fe2fbeef9af P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:32.142287 19945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b832efa5a2e34b3894e33629110e6fef" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:32.142890 19874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b832efa5a2e34b3894e33629110e6fef" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:32.137259 20351 raft_consensus.cc:2749] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:20:32.147563 19765 leader_election.cc:304] T b832efa5a2e34b3894e33629110e6fef P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e90ffed3ba174ababeb21d9415d4d918; no voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430
W20260709 13:20:32.162320 19913 leader_election.cc:343] T 29f92a880ce04259ae0ee9c5288949ff P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:32.165593 19913 leader_election.cc:304] T 29f92a880ce04259ae0ee9c5288949ff P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c62716bf66c24330860573535b8b9430; no voters: b80b3a62d7d549a9a5c42def46769ac4, e90ffed3ba174ababeb21d9415d4d918
W20260709 13:20:32.170398 19670 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:20:32.172156 20400 raft_consensus.cc:2749] T b832efa5a2e34b3894e33629110e6fef P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:32.173317 19722 raft_consensus.cc:3060] T bc04a038495940c1bbeda4a2e6f17b82 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:32.173528 19870 raft_consensus.cc:2468] T 8fd58e41ecee4779b3e72df3aea4463a P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c62716bf66c24330860573535b8b9430 in term 1.
I20260709 13:20:32.176704 20424 raft_consensus.cc:2749] T 29f92a880ce04259ae0ee9c5288949ff P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:32.179806 19913 leader_election.cc:304] T 8fd58e41ecee4779b3e72df3aea4463a P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:32.180994 20370 raft_consensus.cc:2804] T 8fd58e41ecee4779b3e72df3aea4463a P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:32.181450 20370 raft_consensus.cc:697] T 8fd58e41ecee4779b3e72df3aea4463a P c62716bf66c24330860573535b8b9430 [term 1 LEADER]: Becoming Leader. State: Replica: c62716bf66c24330860573535b8b9430, State: Running, Role: LEADER
I20260709 13:20:32.182240 20370 consensus_queue.cc:237] T 8fd58e41ecee4779b3e72df3aea4463a P c62716bf66c24330860573535b8b9430 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:32.185328 20083 leader_election.cc:290] T bc04a038495940c1bbeda4a2e6f17b82 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:32.186125 19994 tablet_bootstrap.cc:492] T bd80f9f8533b4cd3b5861ec11917a03d P 8df38c4c53c44e94b36c1e8992476d7c: No bootstrap required, opened a new log
I20260709 13:20:32.186638 19994 ts_tablet_manager.cc:1403] T bd80f9f8533b4cd3b5861ec11917a03d P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.244s	user 0.016s	sys 0.006s
I20260709 13:20:32.187152 20399 raft_consensus.cc:515] T cb33acfa7a9f46ab95b0e77e1394f258 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:32.189529 19994 raft_consensus.cc:359] T bd80f9f8533b4cd3b5861ec11917a03d P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:32.190316 19994 raft_consensus.cc:385] T bd80f9f8533b4cd3b5861ec11917a03d P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:32.194344 19997 tablet_bootstrap.cc:492] T 29f92a880ce04259ae0ee9c5288949ff P b80b3a62d7d549a9a5c42def46769ac4: No bootstrap required, opened a new log
I20260709 13:20:32.194898 19997 ts_tablet_manager.cc:1403] T 29f92a880ce04259ae0ee9c5288949ff P b80b3a62d7d549a9a5c42def46769ac4: Time spent bootstrapping tablet: real 0.329s	user 0.011s	sys 0.023s
I20260709 13:20:32.190598 19994 raft_consensus.cc:740] T bd80f9f8533b4cd3b5861ec11917a03d P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:32.196528 19994 consensus_queue.cc:260] T bd80f9f8533b4cd3b5861ec11917a03d P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:32.198964 19997 raft_consensus.cc:359] T 29f92a880ce04259ae0ee9c5288949ff P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:32.200474 19997 raft_consensus.cc:385] T 29f92a880ce04259ae0ee9c5288949ff P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:32.200903 19997 raft_consensus.cc:740] T 29f92a880ce04259ae0ee9c5288949ff P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Initialized, Role: FOLLOWER
I20260709 13:20:32.201644 19997 consensus_queue.cc:260] T 29f92a880ce04259ae0ee9c5288949ff P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:32.203648 19994 ts_tablet_manager.cc:1434] T bd80f9f8533b4cd3b5861ec11917a03d P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:20:32.205516 19994 tablet_bootstrap.cc:492] T 9f60d4b859264720a20f5670c3c71a6f P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:32.208962 20080 consensus_queue.cc:1048] T 9f60d4b859264720a20f5670c3c71a6f P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:32.191241 20399 leader_election.cc:290] T cb33acfa7a9f46ab95b0e77e1394f258 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Requested vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:32.214352 19994 tablet_bootstrap.cc:654] T 9f60d4b859264720a20f5670c3c71a6f P 8df38c4c53c44e94b36c1e8992476d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:32.214968 19997 ts_tablet_manager.cc:1434] T 29f92a880ce04259ae0ee9c5288949ff P b80b3a62d7d549a9a5c42def46769ac4: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.001s
I20260709 13:20:32.218447 19993 tablet_bootstrap.cc:492] T 0a9c6b64c8e04c4283d24ae6290a52ea P e90ffed3ba174ababeb21d9415d4d918: No bootstrap required, opened a new log
I20260709 13:20:32.219206 19993 ts_tablet_manager.cc:1403] T 0a9c6b64c8e04c4283d24ae6290a52ea P e90ffed3ba174ababeb21d9415d4d918: Time spent bootstrapping tablet: real 0.204s	user 0.006s	sys 0.009s
I20260709 13:20:32.215591 19620 catalog_manager.cc:5697] T 87364a1289ed4aa1b9b2a7a9951ed776 P b80b3a62d7d549a9a5c42def46769ac4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131). New cstate: current_term: 1 leader_uuid: "b80b3a62d7d549a9a5c42def46769ac4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:32.222743 19993 raft_consensus.cc:359] T 0a9c6b64c8e04c4283d24ae6290a52ea P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:32.228538 19993 raft_consensus.cc:385] T 0a9c6b64c8e04c4283d24ae6290a52ea P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:32.228893 19993 raft_consensus.cc:740] T 0a9c6b64c8e04c4283d24ae6290a52ea P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Initialized, Role: FOLLOWER
I20260709 13:20:32.229786 19993 consensus_queue.cc:260] T 0a9c6b64c8e04c4283d24ae6290a52ea P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
W20260709 13:20:32.230571 19690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:43319, user_credentials={real_user=slave}} blocked reactor thread for 77417.6us
I20260709 13:20:32.224397 19620 catalog_manager.cc:5697] T 9f60d4b859264720a20f5670c3c71a6f P b80b3a62d7d549a9a5c42def46769ac4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131). New cstate: current_term: 1 leader_uuid: "b80b3a62d7d549a9a5c42def46769ac4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:32.232662 19944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc04a038495940c1bbeda4a2e6f17b82" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:32.235513 19994 tablet_bootstrap.cc:492] T 9f60d4b859264720a20f5670c3c71a6f P 8df38c4c53c44e94b36c1e8992476d7c: No bootstrap required, opened a new log
I20260709 13:20:32.236064 19994 ts_tablet_manager.cc:1403] T 9f60d4b859264720a20f5670c3c71a6f P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.031s	user 0.015s	sys 0.002s
I20260709 13:20:32.242812 19994 raft_consensus.cc:359] T 9f60d4b859264720a20f5670c3c71a6f P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:32.243625 19994 raft_consensus.cc:385] T 9f60d4b859264720a20f5670c3c71a6f P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:32.243911 19994 raft_consensus.cc:740] T 9f60d4b859264720a20f5670c3c71a6f P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:32.244640 19994 consensus_queue.cc:260] T 9f60d4b859264720a20f5670c3c71a6f P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:32.246928 19994 ts_tablet_manager.cc:1434] T 9f60d4b859264720a20f5670c3c71a6f P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:20:32.247957 19994 tablet_bootstrap.cc:492] T 8fd58e41ecee4779b3e72df3aea4463a P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:32.252173 19873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc04a038495940c1bbeda4a2e6f17b82" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:32.253408 20388 consensus_queue.cc:1048] T 87364a1289ed4aa1b9b2a7a9951ed776 P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:32.255864 19723 raft_consensus.cc:3060] T 9f60d4b859264720a20f5670c3c71a6f P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:32.258107 19994 tablet_bootstrap.cc:654] T 8fd58e41ecee4779b3e72df3aea4463a P 8df38c4c53c44e94b36c1e8992476d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:32.265452 20388 raft_consensus.cc:493] T 59fc86c7e9f648e3b3f51e936e2bcfec P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:32.266043 20388 raft_consensus.cc:515] T 59fc86c7e9f648e3b3f51e936e2bcfec P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:32.266913 19993 ts_tablet_manager.cc:1434] T 0a9c6b64c8e04c4283d24ae6290a52ea P e90ffed3ba174ababeb21d9415d4d918: Time spent starting tablet: real 0.047s	user 0.001s	sys 0.007s
I20260709 13:20:32.268033 19993 tablet_bootstrap.cc:492] T 87364a1289ed4aa1b9b2a7a9951ed776 P e90ffed3ba174ababeb21d9415d4d918: Bootstrap starting.
I20260709 13:20:32.269457 20388 leader_election.cc:290] T 59fc86c7e9f648e3b3f51e936e2bcfec P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:32.270841 19797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59fc86c7e9f648e3b3f51e936e2bcfec" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
W20260709 13:20:32.272363 19839 leader_election.cc:343] T 59fc86c7e9f648e3b3f51e936e2bcfec P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:32.273104 19724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59fc86c7e9f648e3b3f51e936e2bcfec" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
W20260709 13:20:32.275034 19840 leader_election.cc:343] T 59fc86c7e9f648e3b3f51e936e2bcfec P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:32.276762 19840 leader_election.cc:304] T 59fc86c7e9f648e3b3f51e936e2bcfec P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4; no voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:32.278017 20080 raft_consensus.cc:2749] T 59fc86c7e9f648e3b3f51e936e2bcfec P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:32.296845 19994 tablet_bootstrap.cc:492] T 8fd58e41ecee4779b3e72df3aea4463a P 8df38c4c53c44e94b36c1e8992476d7c: No bootstrap required, opened a new log
I20260709 13:20:32.297917 19994 ts_tablet_manager.cc:1403] T 8fd58e41ecee4779b3e72df3aea4463a P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.050s	user 0.025s	sys 0.008s
W20260709 13:20:32.298954 19766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:41855, user_credentials={real_user=slave}} blocked reactor thread for 68652.6us
I20260709 13:20:32.301132 19994 raft_consensus.cc:359] T 8fd58e41ecee4779b3e72df3aea4463a P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:32.301932 19994 raft_consensus.cc:385] T 8fd58e41ecee4779b3e72df3aea4463a P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:32.302199 19994 raft_consensus.cc:740] T 8fd58e41ecee4779b3e72df3aea4463a P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:32.303025 19994 consensus_queue.cc:260] T 8fd58e41ecee4779b3e72df3aea4463a P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:32.305377 19994 ts_tablet_manager.cc:1434] T 8fd58e41ecee4779b3e72df3aea4463a P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:20:32.305227 19870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb33acfa7a9f46ab95b0e77e1394f258" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
I20260709 13:20:32.305984 19870 raft_consensus.cc:3060] T cb33acfa7a9f46ab95b0e77e1394f258 P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:32.306514 19994 tablet_bootstrap.cc:492] T 59fc86c7e9f648e3b3f51e936e2bcfec P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:32.307439 19797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb33acfa7a9f46ab95b0e77e1394f258" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
I20260709 13:20:32.314119 19994 tablet_bootstrap.cc:654] T 59fc86c7e9f648e3b3f51e936e2bcfec P 8df38c4c53c44e94b36c1e8992476d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:32.333698 19870 raft_consensus.cc:2468] T cb33acfa7a9f46ab95b0e77e1394f258 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 1.
W20260709 13:20:32.342815 19839 consensus_peers.cc:597] T 87364a1289ed4aa1b9b2a7a9951ed776 P b80b3a62d7d549a9a5c42def46769ac4 -> Peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Couldn't send request to peer e90ffed3ba174ababeb21d9415d4d918. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:32.348080 20080 consensus_queue.cc:1048] T 9f60d4b859264720a20f5670c3c71a6f P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:20:32.360335 20388 raft_consensus.cc:493] T 29f92a880ce04259ae0ee9c5288949ff P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:32.360960 20388 raft_consensus.cc:515] T 29f92a880ce04259ae0ee9c5288949ff P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:32.363662 20388 leader_election.cc:290] T 29f92a880ce04259ae0ee9c5288949ff P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:32.369032 20395 raft_consensus.cc:493] T 8fd58e41ecee4779b3e72df3aea4463a P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:32.367398 19948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29f92a880ce04259ae0ee9c5288949ff" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:32.372255 19948 raft_consensus.cc:2468] T 29f92a880ce04259ae0ee9c5288949ff P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 0.
I20260709 13:20:32.376472 19839 leader_election.cc:304] T 29f92a880ce04259ae0ee9c5288949ff P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430; no voters: 
I20260709 13:20:32.377483 20388 raft_consensus.cc:2804] T 29f92a880ce04259ae0ee9c5288949ff P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:32.377942 20388 raft_consensus.cc:493] T 29f92a880ce04259ae0ee9c5288949ff P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:32.378321 20388 raft_consensus.cc:3060] T 29f92a880ce04259ae0ee9c5288949ff P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:32.369596 20395 raft_consensus.cc:515] T 8fd58e41ecee4779b3e72df3aea4463a P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:32.391182 20395 leader_election.cc:290] T 8fd58e41ecee4779b3e72df3aea4463a P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:32.387326 19797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29f92a880ce04259ae0ee9c5288949ff" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
W20260709 13:20:32.393440 19839 leader_election.cc:343] T 29f92a880ce04259ae0ee9c5288949ff P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:32.409068 20388 raft_consensus.cc:515] T 29f92a880ce04259ae0ee9c5288949ff P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:32.411586 20388 leader_election.cc:290] T 29f92a880ce04259ae0ee9c5288949ff P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 election: Requested vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:32.413411 19797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29f92a880ce04259ae0ee9c5288949ff" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
I20260709 13:20:32.415198 19945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29f92a880ce04259ae0ee9c5288949ff" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:32.415820 19945 raft_consensus.cc:3060] T 29f92a880ce04259ae0ee9c5288949ff P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:32.425899 19993 tablet_bootstrap.cc:654] T 87364a1289ed4aa1b9b2a7a9951ed776 P e90ffed3ba174ababeb21d9415d4d918: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:32.436890 19690 leader_election.cc:304] T bc04a038495940c1bbeda4a2e6f17b82 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c; no voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430
W20260709 13:20:32.439662 19839 leader_election.cc:343] T 29f92a880ce04259ae0ee9c5288949ff P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:32.442209 20083 raft_consensus.cc:2749] T bc04a038495940c1bbeda4a2e6f17b82 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:32.446487 19690 leader_election.cc:304] T cb33acfa7a9f46ab95b0e77e1394f258 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4; no voters: 
I20260709 13:20:32.447407 20083 raft_consensus.cc:2804] T cb33acfa7a9f46ab95b0e77e1394f258 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:32.447926 20083 raft_consensus.cc:697] T cb33acfa7a9f46ab95b0e77e1394f258 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 LEADER]: Becoming Leader. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Running, Role: LEADER
I20260709 13:20:32.448771 20083 consensus_queue.cc:237] T cb33acfa7a9f46ab95b0e77e1394f258 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:32.463972 20395 consensus_queue.cc:1048] T dc5c36b41652443a8130592df8a66eb9 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:20:32.466995 19690 leader_election.cc:343] T cb33acfa7a9f46ab95b0e77e1394f258 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:32.473656 19945 raft_consensus.cc:2468] T 29f92a880ce04259ae0ee9c5288949ff P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 1.
I20260709 13:20:32.475033 19839 leader_election.cc:304] T 29f92a880ce04259ae0ee9c5288949ff P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430; no voters: e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:32.476343 20388 raft_consensus.cc:2804] T 29f92a880ce04259ae0ee9c5288949ff P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:32.477061 19622 catalog_manager.cc:5697] T 8fd58e41ecee4779b3e72df3aea4463a P c62716bf66c24330860573535b8b9430 reported cstate change: term changed from 0 to 1, leader changed from <none> to c62716bf66c24330860573535b8b9430 (127.9.116.132). New cstate: current_term: 1 leader_uuid: "c62716bf66c24330860573535b8b9430" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:32.480374 19622 catalog_manager.cc:5697] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 reported cstate change: term changed from 0 to 1, leader changed from <none> to c62716bf66c24330860573535b8b9430 (127.9.116.132). New cstate: current_term: 1 leader_uuid: "c62716bf66c24330860573535b8b9430" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:32.484660 20388 raft_consensus.cc:697] T 29f92a880ce04259ae0ee9c5288949ff P b80b3a62d7d549a9a5c42def46769ac4 [term 1 LEADER]: Becoming Leader. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Running, Role: LEADER
I20260709 13:20:32.486630 20388 consensus_queue.cc:237] T 29f92a880ce04259ae0ee9c5288949ff P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:32.498498 19993 tablet_bootstrap.cc:492] T 87364a1289ed4aa1b9b2a7a9951ed776 P e90ffed3ba174ababeb21d9415d4d918: No bootstrap required, opened a new log
I20260709 13:20:32.499184 19994 tablet_bootstrap.cc:492] T 59fc86c7e9f648e3b3f51e936e2bcfec P 8df38c4c53c44e94b36c1e8992476d7c: No bootstrap required, opened a new log
I20260709 13:20:32.499825 19994 ts_tablet_manager.cc:1403] T 59fc86c7e9f648e3b3f51e936e2bcfec P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.193s	user 0.038s	sys 0.076s
I20260709 13:20:32.500922 19993 ts_tablet_manager.cc:1403] T 87364a1289ed4aa1b9b2a7a9951ed776 P e90ffed3ba174ababeb21d9415d4d918: Time spent bootstrapping tablet: real 0.233s	user 0.006s	sys 0.007s
I20260709 13:20:32.502942 19994 raft_consensus.cc:359] T 59fc86c7e9f648e3b3f51e936e2bcfec P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:32.503705 19994 raft_consensus.cc:385] T 59fc86c7e9f648e3b3f51e936e2bcfec P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:32.503996 19994 raft_consensus.cc:740] T 59fc86c7e9f648e3b3f51e936e2bcfec P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:32.503899 19993 raft_consensus.cc:359] T 87364a1289ed4aa1b9b2a7a9951ed776 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:32.504668 19993 raft_consensus.cc:385] T 87364a1289ed4aa1b9b2a7a9951ed776 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:32.504977 19993 raft_consensus.cc:740] T 87364a1289ed4aa1b9b2a7a9951ed776 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Initialized, Role: FOLLOWER
I20260709 13:20:32.504817 19994 consensus_queue.cc:260] T 59fc86c7e9f648e3b3f51e936e2bcfec P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:32.505810 19993 consensus_queue.cc:260] T 87364a1289ed4aa1b9b2a7a9951ed776 P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:32.508529 19993 ts_tablet_manager.cc:1434] T 87364a1289ed4aa1b9b2a7a9951ed776 P e90ffed3ba174ababeb21d9415d4d918: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 13:20:32.511014 19993 tablet_bootstrap.cc:492] T 8abfa1aa35c44699960a8166fa79c41d P e90ffed3ba174ababeb21d9415d4d918: Bootstrap starting.
I20260709 13:20:32.513190 19797 raft_consensus.cc:3060] T b832efa5a2e34b3894e33629110e6fef P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:32.516202 19873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd58e41ecee4779b3e72df3aea4463a" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:32.517138 19873 raft_consensus.cc:2393] T 8fd58e41ecee4779b3e72df3aea4463a P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8df38c4c53c44e94b36c1e8992476d7c in current term 1: Already voted for candidate c62716bf66c24330860573535b8b9430 in this term.
I20260709 13:20:32.519830 19948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd58e41ecee4779b3e72df3aea4463a" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:32.530561 19993 tablet_bootstrap.cc:654] T 8abfa1aa35c44699960a8166fa79c41d P e90ffed3ba174ababeb21d9415d4d918: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:32.531828 19994 ts_tablet_manager.cc:1434] T 59fc86c7e9f648e3b3f51e936e2bcfec P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.000s
I20260709 13:20:32.538857 20395 raft_consensus.cc:493] T bd80f9f8533b4cd3b5861ec11917a03d P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:32.539472 20395 raft_consensus.cc:515] T bd80f9f8533b4cd3b5861ec11917a03d P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:32.541477 20395 leader_election.cc:290] T bd80f9f8533b4cd3b5861ec11917a03d P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:32.548879 19622 catalog_manager.cc:5697] T 29f92a880ce04259ae0ee9c5288949ff P b80b3a62d7d549a9a5c42def46769ac4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131). New cstate: current_term: 1 leader_uuid: "b80b3a62d7d549a9a5c42def46769ac4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:32.551546 19994 tablet_bootstrap.cc:492] T 6551866a22dd4aaf8ee318b9d91b3a02 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:32.555962 19948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd80f9f8533b4cd3b5861ec11917a03d" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:32.556715 19948 raft_consensus.cc:2468] T bd80f9f8533b4cd3b5861ec11917a03d P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 0.
I20260709 13:20:32.559831 19798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd80f9f8533b4cd3b5861ec11917a03d" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:32.572813 19993 tablet_bootstrap.cc:492] T 8abfa1aa35c44699960a8166fa79c41d P e90ffed3ba174ababeb21d9415d4d918: No bootstrap required, opened a new log
I20260709 13:20:32.573344 19993 ts_tablet_manager.cc:1403] T 8abfa1aa35c44699960a8166fa79c41d P e90ffed3ba174ababeb21d9415d4d918: Time spent bootstrapping tablet: real 0.063s	user 0.016s	sys 0.008s
I20260709 13:20:32.574647 19994 tablet_bootstrap.cc:654] T 6551866a22dd4aaf8ee318b9d91b3a02 P 8df38c4c53c44e94b36c1e8992476d7c: Neither blocks nor log segments found. Creating new log.
W20260709 13:20:32.582008 19690 leader_election.cc:343] T bd80f9f8533b4cd3b5861ec11917a03d P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:32.587251 19690 leader_election.cc:304] T bd80f9f8533b4cd3b5861ec11917a03d P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, c62716bf66c24330860573535b8b9430; no voters: e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:32.588001 20395 raft_consensus.cc:2804] T bd80f9f8533b4cd3b5861ec11917a03d P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:32.588598 20395 raft_consensus.cc:493] T bd80f9f8533b4cd3b5861ec11917a03d P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:32.588943 20395 raft_consensus.cc:3060] T bd80f9f8533b4cd3b5861ec11917a03d P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:32.591024 19623 catalog_manager.cc:5697] T cb33acfa7a9f46ab95b0e77e1394f258 P 8df38c4c53c44e94b36c1e8992476d7c reported cstate change: term changed from 0 to 1, leader changed from <none> to 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129). New cstate: current_term: 1 leader_uuid: "8df38c4c53c44e94b36c1e8992476d7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:32.592813 19993 raft_consensus.cc:359] T 8abfa1aa35c44699960a8166fa79c41d P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:32.594089 19993 raft_consensus.cc:385] T 8abfa1aa35c44699960a8166fa79c41d P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:32.594425 19993 raft_consensus.cc:740] T 8abfa1aa35c44699960a8166fa79c41d P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Initialized, Role: FOLLOWER
I20260709 13:20:32.595201 19993 consensus_queue.cc:260] T 8abfa1aa35c44699960a8166fa79c41d P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:32.602336 19993 ts_tablet_manager.cc:1434] T 8abfa1aa35c44699960a8166fa79c41d P e90ffed3ba174ababeb21d9415d4d918: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.000s
I20260709 13:20:32.596712 20395 raft_consensus.cc:515] T bd80f9f8533b4cd3b5861ec11917a03d P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:32.598393 20212 raft_consensus.cc:493] T 0a9c6b64c8e04c4283d24ae6290a52ea P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:32.603152 20212 raft_consensus.cc:515] T 0a9c6b64c8e04c4283d24ae6290a52ea P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:32.605405 20212 leader_election.cc:290] T 0a9c6b64c8e04c4283d24ae6290a52ea P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:32.598466 19690 leader_election.cc:304] T 8fd58e41ecee4779b3e72df3aea4463a P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c; no voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430
I20260709 13:20:32.607347 20394 raft_consensus.cc:3060] T 8fd58e41ecee4779b3e72df3aea4463a P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:32.608079 19723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9c6b64c8e04c4283d24ae6290a52ea" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:32.608954 20399 consensus_queue.cc:1048] T dc5c36b41652443a8130592df8a66eb9 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:32.610858 19873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9c6b64c8e04c4283d24ae6290a52ea" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:32.611763 19873 raft_consensus.cc:2393] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e90ffed3ba174ababeb21d9415d4d918 in current term 1: Already voted for candidate b80b3a62d7d549a9a5c42def46769ac4 in this term.
I20260709 13:20:32.608975 19723 raft_consensus.cc:2393] T 0a9c6b64c8e04c4283d24ae6290a52ea P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e90ffed3ba174ababeb21d9415d4d918 in current term 1: Already voted for candidate 8df38c4c53c44e94b36c1e8992476d7c in this term.
I20260709 13:20:32.614909 19765 leader_election.cc:304] T 0a9c6b64c8e04c4283d24ae6290a52ea P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e90ffed3ba174ababeb21d9415d4d918; no voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4
I20260709 13:20:32.616722 20414 raft_consensus.cc:3060] T 0a9c6b64c8e04c4283d24ae6290a52ea P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:32.619568 20394 raft_consensus.cc:2749] T 8fd58e41ecee4779b3e72df3aea4463a P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:32.625465 19797 raft_consensus.cc:3060] T 8abfa1aa35c44699960a8166fa79c41d P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:32.623276 19994 tablet_bootstrap.cc:492] T 6551866a22dd4aaf8ee318b9d91b3a02 P 8df38c4c53c44e94b36c1e8992476d7c: No bootstrap required, opened a new log
I20260709 13:20:32.628064 19994 ts_tablet_manager.cc:1403] T 6551866a22dd4aaf8ee318b9d91b3a02 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.077s	user 0.012s	sys 0.000s
I20260709 13:20:32.628383 20395 leader_election.cc:290] T bd80f9f8533b4cd3b5861ec11917a03d P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Requested vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:32.629628 19993 tablet_bootstrap.cc:492] T bd80f9f8533b4cd3b5861ec11917a03d P e90ffed3ba174ababeb21d9415d4d918: Bootstrap starting.
I20260709 13:20:32.631098 19994 raft_consensus.cc:359] T 6551866a22dd4aaf8ee318b9d91b3a02 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:32.631814 19948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd80f9f8533b4cd3b5861ec11917a03d" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:32.635063 19948 raft_consensus.cc:3060] T bd80f9f8533b4cd3b5861ec11917a03d P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:32.638806 19798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd80f9f8533b4cd3b5861ec11917a03d" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
I20260709 13:20:32.643456 19948 raft_consensus.cc:2468] T bd80f9f8533b4cd3b5861ec11917a03d P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 1.
I20260709 13:20:32.644685 19994 raft_consensus.cc:385] T 6551866a22dd4aaf8ee318b9d91b3a02 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:32.645036 19994 raft_consensus.cc:740] T 6551866a22dd4aaf8ee318b9d91b3a02 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
W20260709 13:20:32.644800 19690 leader_election.cc:343] T bd80f9f8533b4cd3b5861ec11917a03d P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:32.645916 19994 consensus_queue.cc:260] T 6551866a22dd4aaf8ee318b9d91b3a02 P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:32.647418 19690 leader_election.cc:304] T bd80f9f8533b4cd3b5861ec11917a03d P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, c62716bf66c24330860573535b8b9430; no voters: e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:32.660212 20083 raft_consensus.cc:2804] T bd80f9f8533b4cd3b5861ec11917a03d P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:32.660727 20083 raft_consensus.cc:697] T bd80f9f8533b4cd3b5861ec11917a03d P 8df38c4c53c44e94b36c1e8992476d7c [term 1 LEADER]: Becoming Leader. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Running, Role: LEADER
I20260709 13:20:32.661484 20083 consensus_queue.cc:237] T bd80f9f8533b4cd3b5861ec11917a03d P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:32.647568 20424 consensus_queue.cc:1048] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:20:32.648569 19994 ts_tablet_manager.cc:1434] T 6551866a22dd4aaf8ee318b9d91b3a02 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 13:20:32.654250 20414 raft_consensus.cc:2749] T 0a9c6b64c8e04c4283d24ae6290a52ea P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:32.666488 19993 tablet_bootstrap.cc:654] T bd80f9f8533b4cd3b5861ec11917a03d P e90ffed3ba174ababeb21d9415d4d918: Neither blocks nor log segments found. Creating new log.
W20260709 13:20:32.820412 19914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:41855, user_credentials={real_user=slave}} blocked reactor thread for 70221.9us
I20260709 13:20:32.821887 20237 consensus_queue.cc:1048] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:32.827658 20209 raft_consensus.cc:493] T 87364a1289ed4aa1b9b2a7a9951ed776 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:32.828254 20209 raft_consensus.cc:515] T 87364a1289ed4aa1b9b2a7a9951ed776 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:32.830361 20209 leader_election.cc:290] T 87364a1289ed4aa1b9b2a7a9951ed776 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:32.836567 20181 raft_consensus.cc:493] T 59fc86c7e9f648e3b3f51e936e2bcfec P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:32.837175 20181 raft_consensus.cc:515] T 59fc86c7e9f648e3b3f51e936e2bcfec P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:32.840517 20181 leader_election.cc:290] T 59fc86c7e9f648e3b3f51e936e2bcfec P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:32.848023 19797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59fc86c7e9f648e3b3f51e936e2bcfec" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
W20260709 13:20:32.851059 19690 leader_election.cc:343] T 59fc86c7e9f648e3b3f51e936e2bcfec P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:32.863692 19948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87364a1289ed4aa1b9b2a7a9951ed776" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:32.863858 19621 catalog_manager.cc:5697] T bd80f9f8533b4cd3b5861ec11917a03d P 8df38c4c53c44e94b36c1e8992476d7c reported cstate change: term changed from 0 to 1, leader changed from <none> to 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129). New cstate: current_term: 1 leader_uuid: "8df38c4c53c44e94b36c1e8992476d7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:32.863693 20370 consensus_queue.cc:1048] T 8fd58e41ecee4779b3e72df3aea4463a P c62716bf66c24330860573535b8b9430 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:20:32.867741 19870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59fc86c7e9f648e3b3f51e936e2bcfec" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:32.868431 19870 raft_consensus.cc:2468] T 59fc86c7e9f648e3b3f51e936e2bcfec P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 0.
I20260709 13:20:32.869951 19993 tablet_bootstrap.cc:492] T bd80f9f8533b4cd3b5861ec11917a03d P e90ffed3ba174ababeb21d9415d4d918: No bootstrap required, opened a new log
I20260709 13:20:32.870420 19993 ts_tablet_manager.cc:1403] T bd80f9f8533b4cd3b5861ec11917a03d P e90ffed3ba174ababeb21d9415d4d918: Time spent bootstrapping tablet: real 0.241s	user 0.014s	sys 0.007s
I20260709 13:20:32.875481 19873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87364a1289ed4aa1b9b2a7a9951ed776" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:32.876693 19690 leader_election.cc:304] T 59fc86c7e9f648e3b3f51e936e2bcfec P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4; no voters: e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:32.878019 20399 raft_consensus.cc:2804] T 59fc86c7e9f648e3b3f51e936e2bcfec P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:32.878427 20399 raft_consensus.cc:493] T 59fc86c7e9f648e3b3f51e936e2bcfec P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:32.880970 20399 raft_consensus.cc:3060] T 59fc86c7e9f648e3b3f51e936e2bcfec P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:32.884878 19797 raft_consensus.cc:3060] T 87364a1289ed4aa1b9b2a7a9951ed776 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:32.893559 19993 raft_consensus.cc:359] T bd80f9f8533b4cd3b5861ec11917a03d P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:32.894398 19993 raft_consensus.cc:385] T bd80f9f8533b4cd3b5861ec11917a03d P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:32.894541 19765 leader_election.cc:304] T 87364a1289ed4aa1b9b2a7a9951ed776 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e90ffed3ba174ababeb21d9415d4d918; no voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430
I20260709 13:20:32.898169 20399 raft_consensus.cc:515] T 59fc86c7e9f648e3b3f51e936e2bcfec P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:32.900867 20399 leader_election.cc:290] T 59fc86c7e9f648e3b3f51e936e2bcfec P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Requested vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:32.902279 19798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59fc86c7e9f648e3b3f51e936e2bcfec" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
I20260709 13:20:32.902954 19868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59fc86c7e9f648e3b3f51e936e2bcfec" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
I20260709 13:20:32.903939 19868 raft_consensus.cc:3060] T 59fc86c7e9f648e3b3f51e936e2bcfec P b80b3a62d7d549a9a5c42def46769ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:32.907279 20424 consensus_queue.cc:1048] T 8fd58e41ecee4779b3e72df3aea4463a P c62716bf66c24330860573535b8b9430 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:20:32.908650 19690 leader_election.cc:343] T 59fc86c7e9f648e3b3f51e936e2bcfec P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:32.909173 20083 raft_consensus.cc:493] T 6551866a22dd4aaf8ee318b9d91b3a02 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:32.909745 20083 raft_consensus.cc:515] T 6551866a22dd4aaf8ee318b9d91b3a02 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:32.911610 20083 leader_election.cc:290] T 6551866a22dd4aaf8ee318b9d91b3a02 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:32.894739 19993 raft_consensus.cc:740] T bd80f9f8533b4cd3b5861ec11917a03d P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Initialized, Role: FOLLOWER
I20260709 13:20:32.919445 19993 consensus_queue.cc:260] T bd80f9f8533b4cd3b5861ec11917a03d P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:32.920858 19798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6551866a22dd4aaf8ee318b9d91b3a02" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
W20260709 13:20:32.924659 20446 log.cc:927] Time spent T bd80f9f8533b4cd3b5861ec11917a03d P 8df38c4c53c44e94b36c1e8992476d7c: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:20:32.928915 19993 ts_tablet_manager.cc:1434] T bd80f9f8533b4cd3b5861ec11917a03d P e90ffed3ba174ababeb21d9415d4d918: Time spent starting tablet: real 0.058s	user 0.007s	sys 0.000s
W20260709 13:20:32.933063 19690 leader_election.cc:343] T 6551866a22dd4aaf8ee318b9d91b3a02 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:32.938140 20351 raft_consensus.cc:2749] T 87364a1289ed4aa1b9b2a7a9951ed776 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:32.942447 19944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6551866a22dd4aaf8ee318b9d91b3a02" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:32.943249 19944 raft_consensus.cc:2468] T 6551866a22dd4aaf8ee318b9d91b3a02 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 0.
I20260709 13:20:32.947599 19993 tablet_bootstrap.cc:492] T 59fc86c7e9f648e3b3f51e936e2bcfec P e90ffed3ba174ababeb21d9415d4d918: Bootstrap starting.
I20260709 13:20:32.949477 19690 leader_election.cc:304] T 6551866a22dd4aaf8ee318b9d91b3a02 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, c62716bf66c24330860573535b8b9430; no voters: e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:32.952994 20083 raft_consensus.cc:2804] T 6551866a22dd4aaf8ee318b9d91b3a02 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:32.953400 20083 raft_consensus.cc:493] T 6551866a22dd4aaf8ee318b9d91b3a02 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:32.953752 20083 raft_consensus.cc:3060] T 6551866a22dd4aaf8ee318b9d91b3a02 P 8df38c4c53c44e94b36c1e8992476d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:32.971155 20351 raft_consensus.cc:493] T bd80f9f8533b4cd3b5861ec11917a03d P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:32.971908 20351 raft_consensus.cc:515] T bd80f9f8533b4cd3b5861ec11917a03d P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:32.976330 19724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd80f9f8533b4cd3b5861ec11917a03d" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:32.977401 20351 leader_election.cc:290] T bd80f9f8533b4cd3b5861ec11917a03d P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:32.938095 19868 raft_consensus.cc:2468] T 59fc86c7e9f648e3b3f51e936e2bcfec P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 1.
I20260709 13:20:32.982504 19690 leader_election.cc:304] T 59fc86c7e9f648e3b3f51e936e2bcfec P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4; no voters: e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:32.984129 20395 raft_consensus.cc:2804] T 59fc86c7e9f648e3b3f51e936e2bcfec P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:32.984339 19944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd80f9f8533b4cd3b5861ec11917a03d" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:32.984725 20395 raft_consensus.cc:697] T 59fc86c7e9f648e3b3f51e936e2bcfec P 8df38c4c53c44e94b36c1e8992476d7c [term 1 LEADER]: Becoming Leader. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Running, Role: LEADER
I20260709 13:20:32.986150 20083 raft_consensus.cc:515] T 6551866a22dd4aaf8ee318b9d91b3a02 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:32.985257 19944 raft_consensus.cc:2393] T bd80f9f8533b4cd3b5861ec11917a03d P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e90ffed3ba174ababeb21d9415d4d918 in current term 1: Already voted for candidate 8df38c4c53c44e94b36c1e8992476d7c in this term.
I20260709 13:20:32.991169 20395 consensus_queue.cc:237] T 59fc86c7e9f648e3b3f51e936e2bcfec P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:33.003285 20083 leader_election.cc:290] T 6551866a22dd4aaf8ee318b9d91b3a02 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Requested vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:33.005908 19765 leader_election.cc:304] T bd80f9f8533b4cd3b5861ec11917a03d P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e90ffed3ba174ababeb21d9415d4d918; no voters: 8df38c4c53c44e94b36c1e8992476d7c, c62716bf66c24330860573535b8b9430
I20260709 13:20:33.010206 19798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6551866a22dd4aaf8ee318b9d91b3a02" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
W20260709 13:20:33.015524 19690 leader_election.cc:343] T 6551866a22dd4aaf8ee318b9d91b3a02 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:33.036712 19944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6551866a22dd4aaf8ee318b9d91b3a02" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:33.037328 19993 tablet_bootstrap.cc:654] T 59fc86c7e9f648e3b3f51e936e2bcfec P e90ffed3ba174ababeb21d9415d4d918: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:33.014714 20212 raft_consensus.cc:3060] T bd80f9f8533b4cd3b5861ec11917a03d P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:33.045014 20388 consensus_queue.cc:1048] T 29f92a880ce04259ae0ee9c5288949ff P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:20:33.046408 19690 consensus_peers.cc:597] T cb33acfa7a9f46ab95b0e77e1394f258 P 8df38c4c53c44e94b36c1e8992476d7c -> Peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Couldn't send request to peer e90ffed3ba174ababeb21d9415d4d918. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:33.040512 19944 raft_consensus.cc:3060] T 6551866a22dd4aaf8ee318b9d91b3a02 P c62716bf66c24330860573535b8b9430 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:33.058943 19993 tablet_bootstrap.cc:492] T 59fc86c7e9f648e3b3f51e936e2bcfec P e90ffed3ba174ababeb21d9415d4d918: No bootstrap required, opened a new log
I20260709 13:20:33.059641 19993 ts_tablet_manager.cc:1403] T 59fc86c7e9f648e3b3f51e936e2bcfec P e90ffed3ba174ababeb21d9415d4d918: Time spent bootstrapping tablet: real 0.112s	user 0.015s	sys 0.010s
I20260709 13:20:33.062436 19993 raft_consensus.cc:359] T 59fc86c7e9f648e3b3f51e936e2bcfec P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:33.063189 19993 raft_consensus.cc:385] T 59fc86c7e9f648e3b3f51e936e2bcfec P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:33.063474 19993 raft_consensus.cc:740] T 59fc86c7e9f648e3b3f51e936e2bcfec P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Initialized, Role: FOLLOWER
I20260709 13:20:33.064316 19993 consensus_queue.cc:260] T 59fc86c7e9f648e3b3f51e936e2bcfec P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:33.065394 20212 raft_consensus.cc:2749] T bd80f9f8533b4cd3b5861ec11917a03d P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:33.066733 19993 ts_tablet_manager.cc:1434] T 59fc86c7e9f648e3b3f51e936e2bcfec P e90ffed3ba174ababeb21d9415d4d918: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:20:33.068005 19993 tablet_bootstrap.cc:492] T 6551866a22dd4aaf8ee318b9d91b3a02 P e90ffed3ba174ababeb21d9415d4d918: Bootstrap starting.
I20260709 13:20:33.080323 19993 tablet_bootstrap.cc:654] T 6551866a22dd4aaf8ee318b9d91b3a02 P e90ffed3ba174ababeb21d9415d4d918: Neither blocks nor log segments found. Creating new log.
W20260709 13:20:33.081475 19839 consensus_peers.cc:597] T 29f92a880ce04259ae0ee9c5288949ff P b80b3a62d7d549a9a5c42def46769ac4 -> Peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Couldn't send request to peer e90ffed3ba174ababeb21d9415d4d918. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:33.090173 20395 consensus_queue.cc:1048] T cb33acfa7a9f46ab95b0e77e1394f258 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:33.102999 19623 catalog_manager.cc:5697] T 59fc86c7e9f648e3b3f51e936e2bcfec P 8df38c4c53c44e94b36c1e8992476d7c reported cstate change: term changed from 0 to 1, leader changed from <none> to 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129). New cstate: current_term: 1 leader_uuid: "8df38c4c53c44e94b36c1e8992476d7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:33.112476 19944 raft_consensus.cc:2468] T 6551866a22dd4aaf8ee318b9d91b3a02 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 1.
I20260709 13:20:33.123790 19690 leader_election.cc:304] T 6551866a22dd4aaf8ee318b9d91b3a02 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, c62716bf66c24330860573535b8b9430; no voters: e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:33.124711 20181 raft_consensus.cc:2804] T 6551866a22dd4aaf8ee318b9d91b3a02 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:33.125468 20181 raft_consensus.cc:697] T 6551866a22dd4aaf8ee318b9d91b3a02 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 LEADER]: Becoming Leader. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Running, Role: LEADER
I20260709 13:20:33.126369 20181 consensus_queue.cc:237] T 6551866a22dd4aaf8ee318b9d91b3a02 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:33.131804 19993 tablet_bootstrap.cc:492] T 6551866a22dd4aaf8ee318b9d91b3a02 P e90ffed3ba174ababeb21d9415d4d918: No bootstrap required, opened a new log
I20260709 13:20:33.132362 19993 ts_tablet_manager.cc:1403] T 6551866a22dd4aaf8ee318b9d91b3a02 P e90ffed3ba174ababeb21d9415d4d918: Time spent bootstrapping tablet: real 0.065s	user 0.009s	sys 0.019s
I20260709 13:20:33.150643 19993 raft_consensus.cc:359] T 6551866a22dd4aaf8ee318b9d91b3a02 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:33.151468 19993 raft_consensus.cc:385] T 6551866a22dd4aaf8ee318b9d91b3a02 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:33.151782 19993 raft_consensus.cc:740] T 6551866a22dd4aaf8ee318b9d91b3a02 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Initialized, Role: FOLLOWER
I20260709 13:20:33.156188 19993 consensus_queue.cc:260] T 6551866a22dd4aaf8ee318b9d91b3a02 P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:33.157914 19993 ts_tablet_manager.cc:1434] T 6551866a22dd4aaf8ee318b9d91b3a02 P e90ffed3ba174ababeb21d9415d4d918: Time spent starting tablet: real 0.025s	user 0.004s	sys 0.003s
I20260709 13:20:33.162976 20212 raft_consensus.cc:493] T 59fc86c7e9f648e3b3f51e936e2bcfec P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:33.163559 20212 raft_consensus.cc:515] T 59fc86c7e9f648e3b3f51e936e2bcfec P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:33.166916 19993 tablet_bootstrap.cc:492] T 29f92a880ce04259ae0ee9c5288949ff P e90ffed3ba174ababeb21d9415d4d918: Bootstrap starting.
I20260709 13:20:33.167135 19722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59fc86c7e9f648e3b3f51e936e2bcfec" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:33.166333 20212 leader_election.cc:290] T 59fc86c7e9f648e3b3f51e936e2bcfec P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:33.174448 19868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59fc86c7e9f648e3b3f51e936e2bcfec" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:33.175324 19868 raft_consensus.cc:2393] T 59fc86c7e9f648e3b3f51e936e2bcfec P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e90ffed3ba174ababeb21d9415d4d918 in current term 1: Already voted for candidate 8df38c4c53c44e94b36c1e8992476d7c in this term.
I20260709 13:20:33.177068 19765 leader_election.cc:304] T 59fc86c7e9f648e3b3f51e936e2bcfec P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e90ffed3ba174ababeb21d9415d4d918; no voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4
I20260709 13:20:33.179273 19993 tablet_bootstrap.cc:654] T 29f92a880ce04259ae0ee9c5288949ff P e90ffed3ba174ababeb21d9415d4d918: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:33.180487 20212 raft_consensus.cc:3060] T 59fc86c7e9f648e3b3f51e936e2bcfec P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:33.198299 19621 catalog_manager.cc:5697] T 6551866a22dd4aaf8ee318b9d91b3a02 P 8df38c4c53c44e94b36c1e8992476d7c reported cstate change: term changed from 0 to 1, leader changed from <none> to 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129). New cstate: current_term: 1 leader_uuid: "8df38c4c53c44e94b36c1e8992476d7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:33.204077 20212 raft_consensus.cc:2749] T 59fc86c7e9f648e3b3f51e936e2bcfec P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:33.226841 19993 tablet_bootstrap.cc:492] T 29f92a880ce04259ae0ee9c5288949ff P e90ffed3ba174ababeb21d9415d4d918: No bootstrap required, opened a new log
I20260709 13:20:33.227348 19993 ts_tablet_manager.cc:1403] T 29f92a880ce04259ae0ee9c5288949ff P e90ffed3ba174ababeb21d9415d4d918: Time spent bootstrapping tablet: real 0.061s	user 0.015s	sys 0.033s
I20260709 13:20:33.230402 19993 raft_consensus.cc:359] T 29f92a880ce04259ae0ee9c5288949ff P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:33.231176 19993 raft_consensus.cc:385] T 29f92a880ce04259ae0ee9c5288949ff P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:33.231452 19993 raft_consensus.cc:740] T 29f92a880ce04259ae0ee9c5288949ff P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Initialized, Role: FOLLOWER
I20260709 13:20:33.232237 19993 consensus_queue.cc:260] T 29f92a880ce04259ae0ee9c5288949ff P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:33.234542 19993 ts_tablet_manager.cc:1434] T 29f92a880ce04259ae0ee9c5288949ff P e90ffed3ba174ababeb21d9415d4d918: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:20:33.235608 19993 tablet_bootstrap.cc:492] T cb33acfa7a9f46ab95b0e77e1394f258 P e90ffed3ba174ababeb21d9415d4d918: Bootstrap starting.
I20260709 13:20:33.241822 19993 tablet_bootstrap.cc:654] T cb33acfa7a9f46ab95b0e77e1394f258 P e90ffed3ba174ababeb21d9415d4d918: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:33.241917 20181 consensus_queue.cc:1048] T bd80f9f8533b4cd3b5861ec11917a03d P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:33.257436 19993 tablet_bootstrap.cc:492] T cb33acfa7a9f46ab95b0e77e1394f258 P e90ffed3ba174ababeb21d9415d4d918: No bootstrap required, opened a new log
I20260709 13:20:33.257951 19993 ts_tablet_manager.cc:1403] T cb33acfa7a9f46ab95b0e77e1394f258 P e90ffed3ba174ababeb21d9415d4d918: Time spent bootstrapping tablet: real 0.023s	user 0.016s	sys 0.003s
I20260709 13:20:33.260919 19993 raft_consensus.cc:359] T cb33acfa7a9f46ab95b0e77e1394f258 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:33.261755 19993 raft_consensus.cc:385] T cb33acfa7a9f46ab95b0e77e1394f258 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:33.262045 19993 raft_consensus.cc:740] T cb33acfa7a9f46ab95b0e77e1394f258 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Initialized, Role: FOLLOWER
W20260709 13:20:33.265508 19670 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:20:33.262897 19993 consensus_queue.cc:260] T cb33acfa7a9f46ab95b0e77e1394f258 P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:33.269637 19993 ts_tablet_manager.cc:1434] T cb33acfa7a9f46ab95b0e77e1394f258 P e90ffed3ba174ababeb21d9415d4d918: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:20:33.290827 20394 consensus_queue.cc:1048] T bd80f9f8533b4cd3b5861ec11917a03d P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:33.392871 20083 consensus_queue.cc:1048] T 59fc86c7e9f648e3b3f51e936e2bcfec P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:33.409534 20399 consensus_queue.cc:1048] T 59fc86c7e9f648e3b3f51e936e2bcfec P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:33.419891 20392 raft_consensus.cc:493] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:33.420868 20392 raft_consensus.cc:515] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:33.423352 19722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9c6b64c8e04c4283d24ae6290a52ea" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:33.424049 19722 raft_consensus.cc:2468] T 0a9c6b64c8e04c4283d24ae6290a52ea P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 1.
I20260709 13:20:33.424340 19796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9c6b64c8e04c4283d24ae6290a52ea" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:33.425035 19796 raft_consensus.cc:2468] T 0a9c6b64c8e04c4283d24ae6290a52ea P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 1.
I20260709 13:20:33.425784 20392 leader_election.cc:290] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:33.427021 19840 leader_election.cc:304] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4; no voters: 
I20260709 13:20:33.427986 20353 raft_consensus.cc:2804] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:20:33.428380 20353 raft_consensus.cc:493] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:33.431852 20353 raft_consensus.cc:3060] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:33.444898 20353 raft_consensus.cc:515] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:33.448366 19722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9c6b64c8e04c4283d24ae6290a52ea" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
I20260709 13:20:33.448992 19722 raft_consensus.cc:3060] T 0a9c6b64c8e04c4283d24ae6290a52ea P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:33.451740 19796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9c6b64c8e04c4283d24ae6290a52ea" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
I20260709 13:20:33.452844 19796 raft_consensus.cc:3060] T 0a9c6b64c8e04c4283d24ae6290a52ea P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:33.458932 20353 leader_election.cc:290] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 election: Requested vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:33.459424 19722 raft_consensus.cc:2468] T 0a9c6b64c8e04c4283d24ae6290a52ea P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 2.
I20260709 13:20:33.459838 19796 raft_consensus.cc:2468] T 0a9c6b64c8e04c4283d24ae6290a52ea P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 2.
I20260709 13:20:33.460632 19840 leader_election.cc:304] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4; no voters: 
I20260709 13:20:33.461442 20353 raft_consensus.cc:2804] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:20:33.462162 20353 raft_consensus.cc:697] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4 [term 2 LEADER]: Becoming Leader. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Running, Role: LEADER
I20260709 13:20:33.463956 20353 consensus_queue.cc:237] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:33.472838 19798 raft_consensus.cc:3060] T 29f92a880ce04259ae0ee9c5288949ff P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:33.483774 19619 catalog_manager.cc:5697] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4 reported cstate change: term changed from 0 to 2, leader changed from <none> to b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131). New cstate: current_term: 2 leader_uuid: "b80b3a62d7d549a9a5c42def46769ac4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:33.522996 19795 raft_consensus.cc:3060] T cb33acfa7a9f46ab95b0e77e1394f258 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:33.523147 19796 raft_consensus.cc:3060] T 6551866a22dd4aaf8ee318b9d91b3a02 P e90ffed3ba174ababeb21d9415d4d918 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:33.530897 20395 consensus_queue.cc:1048] T 6551866a22dd4aaf8ee318b9d91b3a02 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:20:33.553043 20083 consensus_queue.cc:1048] T 6551866a22dd4aaf8ee318b9d91b3a02 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:34.067260 19795 raft_consensus.cc:1275] T 0a9c6b64c8e04c4283d24ae6290a52ea P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Refusing update from remote peer b80b3a62d7d549a9a5c42def46769ac4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:20:34.068650 20353 consensus_queue.cc:1048] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:34.085484 19722 raft_consensus.cc:1275] T 0a9c6b64c8e04c4283d24ae6290a52ea P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Refusing update from remote peer b80b3a62d7d549a9a5c42def46769ac4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:20:34.087051 20353 consensus_queue.cc:1048] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:35.329116  9682 tablet_server.cc:179] TabletServer@127.9.116.129:0 shutting down...
W20260709 13:20:35.373637 19766 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:35.377902  9682 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:20:35.381918 19766 consensus_peers.cc:597] T d762eace063c4e04821c749d21702c3f P e90ffed3ba174ababeb21d9415d4d918 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:35.383198  9682 tablet_replica.cc:333] T 6551866a22dd4aaf8ee318b9d91b3a02 P 8df38c4c53c44e94b36c1e8992476d7c: stopping tablet replica
I20260709 13:20:35.383992  9682 raft_consensus.cc:2243] T 6551866a22dd4aaf8ee318b9d91b3a02 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:35.384979  9682 raft_consensus.cc:2272] T 6551866a22dd4aaf8ee318b9d91b3a02 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:35.386968  9682 tablet_replica.cc:333] T 8fd58e41ecee4779b3e72df3aea4463a P 8df38c4c53c44e94b36c1e8992476d7c: stopping tablet replica
I20260709 13:20:35.387581  9682 raft_consensus.cc:2243] T 8fd58e41ecee4779b3e72df3aea4463a P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:35.388010  9682 raft_consensus.cc:2272] T 8fd58e41ecee4779b3e72df3aea4463a P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:35.389714  9682 tablet_replica.cc:333] T 9f60d4b859264720a20f5670c3c71a6f P 8df38c4c53c44e94b36c1e8992476d7c: stopping tablet replica
I20260709 13:20:35.390298  9682 raft_consensus.cc:2243] T 9f60d4b859264720a20f5670c3c71a6f P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:35.390694  9682 raft_consensus.cc:2272] T 9f60d4b859264720a20f5670c3c71a6f P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:35.392707  9682 tablet_replica.cc:333] T cb33acfa7a9f46ab95b0e77e1394f258 P 8df38c4c53c44e94b36c1e8992476d7c: stopping tablet replica
I20260709 13:20:35.393572  9682 raft_consensus.cc:2243] T cb33acfa7a9f46ab95b0e77e1394f258 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:35.394556  9682 raft_consensus.cc:2272] T cb33acfa7a9f46ab95b0e77e1394f258 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:35.395725 19766 consensus_peers.cc:597] T e3ca37daad8341ad99cbba926f37232e P e90ffed3ba174ababeb21d9415d4d918 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:35.397069  9682 tablet_replica.cc:333] T dc5c36b41652443a8130592df8a66eb9 P 8df38c4c53c44e94b36c1e8992476d7c: stopping tablet replica
I20260709 13:20:35.398010  9682 raft_consensus.cc:2243] T dc5c36b41652443a8130592df8a66eb9 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:35.399052  9682 raft_consensus.cc:2272] T dc5c36b41652443a8130592df8a66eb9 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:35.401012  9682 tablet_replica.cc:333] T 1cceb7f75d7040c2b98049590928b891 P 8df38c4c53c44e94b36c1e8992476d7c: stopping tablet replica
I20260709 13:20:35.401590  9682 raft_consensus.cc:2243] T 1cceb7f75d7040c2b98049590928b891 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:35.402009  9682 raft_consensus.cc:2272] T 1cceb7f75d7040c2b98049590928b891 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:35.404924  9682 tablet_replica.cc:333] T e3ca37daad8341ad99cbba926f37232e P 8df38c4c53c44e94b36c1e8992476d7c: stopping tablet replica
I20260709 13:20:35.405778  9682 raft_consensus.cc:2243] T e3ca37daad8341ad99cbba926f37232e P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:35.406363  9682 raft_consensus.cc:2272] T e3ca37daad8341ad99cbba926f37232e P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:35.408871  9682 tablet_replica.cc:333] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c: stopping tablet replica
I20260709 13:20:35.409595  9682 raft_consensus.cc:2243] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:35.410465  9682 raft_consensus.cc:2272] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:35.412977  9682 tablet_replica.cc:333] T 0413f22bd6d7447ca630b75be496555f P 8df38c4c53c44e94b36c1e8992476d7c: stopping tablet replica
I20260709 13:20:35.413753  9682 raft_consensus.cc:2243] T 0413f22bd6d7447ca630b75be496555f P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:35.414263  9682 raft_consensus.cc:2272] T 0413f22bd6d7447ca630b75be496555f P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:35.416708  9682 tablet_replica.cc:333] T c393f9d18341477c97fac9a4af7e3adf P 8df38c4c53c44e94b36c1e8992476d7c: stopping tablet replica
I20260709 13:20:35.417253  9682 raft_consensus.cc:2243] T c393f9d18341477c97fac9a4af7e3adf P 8df38c4c53c44e94b36c1e8992476d7c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:35.418000  9682 raft_consensus.cc:2272] T c393f9d18341477c97fac9a4af7e3adf P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:35.420540  9682 tablet_replica.cc:333] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c: stopping tablet replica
I20260709 13:20:35.421190  9682 raft_consensus.cc:2243] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:35.422127  9682 raft_consensus.cc:2272] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:35.424501  9682 tablet_replica.cc:333] T 9b9137146e3644a6b0026778530fa8a6 P 8df38c4c53c44e94b36c1e8992476d7c: stopping tablet replica
I20260709 13:20:35.425120  9682 raft_consensus.cc:2243] T 9b9137146e3644a6b0026778530fa8a6 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:35.425644  9682 raft_consensus.cc:2272] T 9b9137146e3644a6b0026778530fa8a6 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:35.428447  9682 tablet_replica.cc:333] T 0a9c6b64c8e04c4283d24ae6290a52ea P 8df38c4c53c44e94b36c1e8992476d7c: stopping tablet replica
I20260709 13:20:35.429217  9682 raft_consensus.cc:2243] T 0a9c6b64c8e04c4283d24ae6290a52ea P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:35.429828  9682 raft_consensus.cc:2272] T 0a9c6b64c8e04c4283d24ae6290a52ea P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:35.431972  9682 tablet_replica.cc:333] T 8bdf042a063d4596970f49730ffa04b4 P 8df38c4c53c44e94b36c1e8992476d7c: stopping tablet replica
I20260709 13:20:35.432603  9682 raft_consensus.cc:2243] T 8bdf042a063d4596970f49730ffa04b4 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:35.433085  9682 raft_consensus.cc:2272] T 8bdf042a063d4596970f49730ffa04b4 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:35.435576  9682 tablet_replica.cc:333] T a323eb80c5084159803e7d4ea5d9c410 P 8df38c4c53c44e94b36c1e8992476d7c: stopping tablet replica
I20260709 13:20:35.436174  9682 raft_consensus.cc:2243] T a323eb80c5084159803e7d4ea5d9c410 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:35.437250  9682 raft_consensus.cc:2272] T a323eb80c5084159803e7d4ea5d9c410 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:35.438035 19766 consensus_peers.cc:597] T 8bdf042a063d4596970f49730ffa04b4 P e90ffed3ba174ababeb21d9415d4d918 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:35.440014  9682 tablet_replica.cc:333] T 8abfa1aa35c44699960a8166fa79c41d P 8df38c4c53c44e94b36c1e8992476d7c: stopping tablet replica
I20260709 13:20:35.440994  9682 raft_consensus.cc:2243] T 8abfa1aa35c44699960a8166fa79c41d P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:35.441574  9682 raft_consensus.cc:2272] T 8abfa1aa35c44699960a8166fa79c41d P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:35.444038  9682 tablet_replica.cc:333] T 7e3a9d11df99411d8f0edb0df6f14d87 P 8df38c4c53c44e94b36c1e8992476d7c: stopping tablet replica
I20260709 13:20:35.444943  9682 raft_consensus.cc:2243] T 7e3a9d11df99411d8f0edb0df6f14d87 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:35.446084  9682 raft_consensus.cc:2272] T 7e3a9d11df99411d8f0edb0df6f14d87 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:35.448899  9682 tablet_replica.cc:333] T d1da95785dbf4ef4bde6f5e6ab35b444 P 8df38c4c53c44e94b36c1e8992476d7c: stopping tablet replica
W20260709 13:20:35.449759 19840 consensus_peers.cc:597] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P b80b3a62d7d549a9a5c42def46769ac4 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:35.449779  9682 raft_consensus.cc:2243] T d1da95785dbf4ef4bde6f5e6ab35b444 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:35.450678  9682 raft_consensus.cc:2272] T d1da95785dbf4ef4bde6f5e6ab35b444 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:35.453984  9682 tablet_replica.cc:333] T 353a546cf8c54671971c239502ef165a P 8df38c4c53c44e94b36c1e8992476d7c: stopping tablet replica
I20260709 13:20:35.455217  9682 raft_consensus.cc:2243] T 353a546cf8c54671971c239502ef165a P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:35.455938  9682 raft_consensus.cc:2272] T 353a546cf8c54671971c239502ef165a P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:35.459379  9682 tablet_replica.cc:333] T 563940ad96444ce59e393ea303cc3352 P 8df38c4c53c44e94b36c1e8992476d7c: stopping tablet replica
I20260709 13:20:35.460335  9682 raft_consensus.cc:2243] T 563940ad96444ce59e393ea303cc3352 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:35.461350  9682 raft_consensus.cc:2272] T 563940ad96444ce59e393ea303cc3352 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:35.462636 19914 consensus_peers.cc:597] T 353a546cf8c54671971c239502ef165a P c62716bf66c24330860573535b8b9430 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:35.464581  9682 tablet_replica.cc:333] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c: stopping tablet replica
I20260709 13:20:35.465850  9682 raft_consensus.cc:2243] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:35.467237  9682 raft_consensus.cc:2272] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:35.470454  9682 tablet_replica.cc:333] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c: stopping tablet replica
I20260709 13:20:35.471285  9682 raft_consensus.cc:2243] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:35.471854  9682 raft_consensus.cc:2272] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:35.474375 19766 consensus_peers.cc:597] T 180230b77b7640b29d055ae8f4358ffe P e90ffed3ba174ababeb21d9415d4d918 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:35.474872  9682 tablet_replica.cc:333] T 86964b8a61ce494f9096c4fefdc6efbf P 8df38c4c53c44e94b36c1e8992476d7c: stopping tablet replica
W20260709 13:20:35.475246 19840 consensus_peers.cc:597] T 3aaced67782041fa9960c745301a96e7 P b80b3a62d7d549a9a5c42def46769ac4 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:35.475746  9682 raft_consensus.cc:2243] T 86964b8a61ce494f9096c4fefdc6efbf P 8df38c4c53c44e94b36c1e8992476d7c [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:20:35.475859 19840 consensus_peers.cc:597] T 19e83b82855d4129bb8525e6ff05316b P b80b3a62d7d549a9a5c42def46769ac4 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:35.476624  9682 raft_consensus.cc:2272] T 86964b8a61ce494f9096c4fefdc6efbf P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:35.478605  9682 tablet_replica.cc:333] T 0587017774214f24bdfa4a3da36e367e P 8df38c4c53c44e94b36c1e8992476d7c: stopping tablet replica
I20260709 13:20:35.479177  9682 raft_consensus.cc:2243] T 0587017774214f24bdfa4a3da36e367e P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:35.479565  9682 raft_consensus.cc:2272] T 0587017774214f24bdfa4a3da36e367e P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:35.481307  9682 tablet_replica.cc:333] T 984fa8de2ccb49359c2b475e4adfc1ce P 8df38c4c53c44e94b36c1e8992476d7c: stopping tablet replica
I20260709 13:20:35.481797  9682 raft_consensus.cc:2243] T 984fa8de2ccb49359c2b475e4adfc1ce P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:35.482154  9682 raft_consensus.cc:2272] T 984fa8de2ccb49359c2b475e4adfc1ce P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:35.484776  9682 tablet_replica.cc:333] T 7e3099f1888c4ed18c30523bf9ec94f5 P 8df38c4c53c44e94b36c1e8992476d7c: stopping tablet replica
I20260709 13:20:35.485569  9682 raft_consensus.cc:2243] T 7e3099f1888c4ed18c30523bf9ec94f5 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:35.486076  9682 raft_consensus.cc:2272] T 7e3099f1888c4ed18c30523bf9ec94f5 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:35.488785  9682 tablet_replica.cc:333] T 2eef64a7d31b4e1e899aee0857cb0e07 P 8df38c4c53c44e94b36c1e8992476d7c: stopping tablet replica
I20260709 13:20:35.489390  9682 raft_consensus.cc:2243] T 2eef64a7d31b4e1e899aee0857cb0e07 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:35.489825  9682 raft_consensus.cc:2272] T 2eef64a7d31b4e1e899aee0857cb0e07 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:35.492509  9682 tablet_replica.cc:333] T 5b9511148896441a948b772784f62a3d P 8df38c4c53c44e94b36c1e8992476d7c: stopping tablet replica
I20260709 13:20:35.493355  9682 raft_consensus.cc:2243] T 5b9511148896441a948b772784f62a3d P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:35.493914  9682 raft_consensus.cc:2272] T 5b9511148896441a948b772784f62a3d P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:35.493922 19766 consensus_peers.cc:597] T 6a066b96902f41dbb1511eccf26820ce P e90ffed3ba174ababeb21d9415d4d918 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:35.496955  9682 tablet_replica.cc:333] T bc04a038495940c1bbeda4a2e6f17b82 P 8df38c4c53c44e94b36c1e8992476d7c: stopping tablet replica
I20260709 13:20:35.497640  9682 raft_consensus.cc:2243] T bc04a038495940c1bbeda4a2e6f17b82 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:35.498096  9682 raft_consensus.cc:2272] T bc04a038495940c1bbeda4a2e6f17b82 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:35.500239  9682 tablet_replica.cc:333] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c: stopping tablet replica
I20260709 13:20:35.501060  9682 raft_consensus.cc:2243] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:35.501917  9682 raft_consensus.cc:2272] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:35.504374  9682 tablet_replica.cc:333] T d762eace063c4e04821c749d21702c3f P 8df38c4c53c44e94b36c1e8992476d7c: stopping tablet replica
I20260709 13:20:35.504985  9682 raft_consensus.cc:2243] T d762eace063c4e04821c749d21702c3f P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:35.505451  9682 raft_consensus.cc:2272] T d762eace063c4e04821c749d21702c3f P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:35.507712  9682 tablet_replica.cc:333] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P 8df38c4c53c44e94b36c1e8992476d7c: stopping tablet replica
I20260709 13:20:35.508286  9682 raft_consensus.cc:2243] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:35.508788  9682 raft_consensus.cc:2272] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:35.511204  9682 tablet_replica.cc:333] T 59fc86c7e9f648e3b3f51e936e2bcfec P 8df38c4c53c44e94b36c1e8992476d7c: stopping tablet replica
I20260709 13:20:35.511746  9682 raft_consensus.cc:2243] T 59fc86c7e9f648e3b3f51e936e2bcfec P 8df38c4c53c44e94b36c1e8992476d7c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:35.512555  9682 raft_consensus.cc:2272] T 59fc86c7e9f648e3b3f51e936e2bcfec P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:35.514618  9682 tablet_replica.cc:333] T ece79531bbd749d3b9300fe2fbeef9af P 8df38c4c53c44e94b36c1e8992476d7c: stopping tablet replica
I20260709 13:20:35.515367  9682 raft_consensus.cc:2243] T ece79531bbd749d3b9300fe2fbeef9af P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:35.515885  9682 raft_consensus.cc:2272] T ece79531bbd749d3b9300fe2fbeef9af P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:35.516196 19766 consensus_peers.cc:597] T 9b9137146e3644a6b0026778530fa8a6 P e90ffed3ba174ababeb21d9415d4d918 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:35.518172  9682 tablet_replica.cc:333] T 180230b77b7640b29d055ae8f4358ffe P 8df38c4c53c44e94b36c1e8992476d7c: stopping tablet replica
I20260709 13:20:35.519022  9682 raft_consensus.cc:2243] T 180230b77b7640b29d055ae8f4358ffe P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:35.519565  9682 raft_consensus.cc:2272] T 180230b77b7640b29d055ae8f4358ffe P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:35.522258  9682 tablet_replica.cc:333] T 2a411802fdaa42e7863236f0af595fe2 P 8df38c4c53c44e94b36c1e8992476d7c: stopping tablet replica
I20260709 13:20:35.522747  9682 raft_consensus.cc:2243] T 2a411802fdaa42e7863236f0af595fe2 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:35.523478  9682 raft_consensus.cc:2272] T 2a411802fdaa42e7863236f0af595fe2 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:35.525679  9682 tablet_replica.cc:333] T e8cfec59c63741869c636df6356546d0 P 8df38c4c53c44e94b36c1e8992476d7c: stopping tablet replica
I20260709 13:20:35.526381  9682 raft_consensus.cc:2243] T e8cfec59c63741869c636df6356546d0 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:35.526890  9682 raft_consensus.cc:2272] T e8cfec59c63741869c636df6356546d0 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:35.529438  9682 tablet_replica.cc:333] T 3aaced67782041fa9960c745301a96e7 P 8df38c4c53c44e94b36c1e8992476d7c: stopping tablet replica
I20260709 13:20:35.530067  9682 raft_consensus.cc:2243] T 3aaced67782041fa9960c745301a96e7 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:35.530534  9682 raft_consensus.cc:2272] T 3aaced67782041fa9960c745301a96e7 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:35.532873  9682 tablet_replica.cc:333] T 63868398f3604f529519d926d2831f4e P 8df38c4c53c44e94b36c1e8992476d7c: stopping tablet replica
I20260709 13:20:35.533540  9682 raft_consensus.cc:2243] T 63868398f3604f529519d926d2831f4e P 8df38c4c53c44e94b36c1e8992476d7c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:35.534548  9682 raft_consensus.cc:2272] T 63868398f3604f529519d926d2831f4e P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:35.537242  9682 tablet_replica.cc:333] T af881b097b4245b1be6072579197cb6b P 8df38c4c53c44e94b36c1e8992476d7c: stopping tablet replica
I20260709 13:20:35.537904  9682 raft_consensus.cc:2243] T af881b097b4245b1be6072579197cb6b P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:35.538388  9682 raft_consensus.cc:2272] T af881b097b4245b1be6072579197cb6b P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:35.541007  9682 tablet_replica.cc:333] T f820e93ffdcd428ea4adfca88589a850 P 8df38c4c53c44e94b36c1e8992476d7c: stopping tablet replica
I20260709 13:20:35.541517  9682 raft_consensus.cc:2243] T f820e93ffdcd428ea4adfca88589a850 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:35.542001  9682 raft_consensus.cc:2272] T f820e93ffdcd428ea4adfca88589a850 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:35.544044  9682 tablet_replica.cc:333] T ad38a1519fb748588f5a322589274004 P 8df38c4c53c44e94b36c1e8992476d7c: stopping tablet replica
I20260709 13:20:35.544682  9682 raft_consensus.cc:2243] T ad38a1519fb748588f5a322589274004 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:35.545186  9682 raft_consensus.cc:2272] T ad38a1519fb748588f5a322589274004 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:35.547993  9682 tablet_replica.cc:333] T ac3bca5bb7644a8fbc78df43ef3331a0 P 8df38c4c53c44e94b36c1e8992476d7c: stopping tablet replica
I20260709 13:20:35.548693  9682 raft_consensus.cc:2243] T ac3bca5bb7644a8fbc78df43ef3331a0 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:35.549183  9682 raft_consensus.cc:2272] T ac3bca5bb7644a8fbc78df43ef3331a0 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:35.549259 19766 consensus_peers.cc:597] T 984fa8de2ccb49359c2b475e4adfc1ce P e90ffed3ba174ababeb21d9415d4d918 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:35.551744  9682 tablet_replica.cc:333] T bd80f9f8533b4cd3b5861ec11917a03d P 8df38c4c53c44e94b36c1e8992476d7c: stopping tablet replica
I20260709 13:20:35.552404  9682 raft_consensus.cc:2243] T bd80f9f8533b4cd3b5861ec11917a03d P 8df38c4c53c44e94b36c1e8992476d7c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:35.553128  9682 raft_consensus.cc:2272] T bd80f9f8533b4cd3b5861ec11917a03d P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:35.554927  9682 tablet_replica.cc:333] T 6a066b96902f41dbb1511eccf26820ce P 8df38c4c53c44e94b36c1e8992476d7c: stopping tablet replica
I20260709 13:20:35.555430  9682 raft_consensus.cc:2243] T 6a066b96902f41dbb1511eccf26820ce P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:35.555856  9682 raft_consensus.cc:2272] T 6a066b96902f41dbb1511eccf26820ce P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:35.561465 19914 consensus_peers.cc:597] T 8fd58e41ecee4779b3e72df3aea4463a P c62716bf66c24330860573535b8b9430 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:35.570670 19914 consensus_peers.cc:597] T e8cfec59c63741869c636df6356546d0 P c62716bf66c24330860573535b8b9430 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:35.574188 19840 consensus_peers.cc:597] T bc04a038495940c1bbeda4a2e6f17b82 P b80b3a62d7d549a9a5c42def46769ac4 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:35.599998 19840 consensus_peers.cc:597] T 0413f22bd6d7447ca630b75be496555f P b80b3a62d7d549a9a5c42def46769ac4 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:35.627919 19840 consensus_peers.cc:597] T f820e93ffdcd428ea4adfca88589a850 P b80b3a62d7d549a9a5c42def46769ac4 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:41855: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:35.629751  9682 tablet_server.cc:196] TabletServer@127.9.116.129:0 shutdown complete.
W20260709 13:20:35.636610 19840 consensus_peers.cc:597] T 7e3099f1888c4ed18c30523bf9ec94f5 P b80b3a62d7d549a9a5c42def46769ac4 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:41855: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:35.652119 19914 consensus_peers.cc:597] T ac3bca5bb7644a8fbc78df43ef3331a0 P c62716bf66c24330860573535b8b9430 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:41855: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:35.655300 19766 consensus_peers.cc:597] T af881b097b4245b1be6072579197cb6b P e90ffed3ba174ababeb21d9415d4d918 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:41855: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:35.696714 19766 consensus_peers.cc:597] T 2eef64a7d31b4e1e899aee0857cb0e07 P e90ffed3ba174ababeb21d9415d4d918 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:41855: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:35.702937  9682 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:20:35.710266 19766 consensus_peers.cc:597] T d1da95785dbf4ef4bde6f5e6ab35b444 P e90ffed3ba174ababeb21d9415d4d918 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:41855: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:35.710628 20484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:20:35.712003 20483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:20:35.714998 20486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:20:35.715101  9682 server_base.cc:1061] running on GCE node
W20260709 13:20:35.718176 19914 consensus_peers.cc:597] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:41855: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:35.718833  9682 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:20:35.719134  9682 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:20:35.719350  9682 hybrid_clock.cc:648] HybridClock initialized: now 1783603235719333 us; error 0 us; skew 500 ppm
I20260709 13:20:35.719947  9682 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:20:35.722594  9682 webserver.cc:533] Webserver started at http://127.9.116.129:44883/ using document root <none> and password file <none>
I20260709 13:20:35.723138  9682 fs_manager.cc:362] Metadata directory not provided
I20260709 13:20:35.723358  9682 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:20:35.728564  9682 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.005s	sys 0.000s
W20260709 13:20:35.729375 19840 consensus_peers.cc:597] T 9f60d4b859264720a20f5670c3c71a6f P b80b3a62d7d549a9a5c42def46769ac4 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:41855: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:35.731848 19914 consensus_peers.cc:597] T 1cceb7f75d7040c2b98049590928b891 P c62716bf66c24330860573535b8b9430 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:41855: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:35.737452 20491 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:20:35.738276  9682 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:20:35.738605  9682 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602916530764-9682-0/minicluster-data/ts-0-root
uuid: "8df38c4c53c44e94b36c1e8992476d7c"
format_stamp: "Formatted at 2026-07-09 13:20:22 on dist-test-slave-1t6m"
I20260709 13:20:35.739019  9682 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602916530764-9682-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602916530764-9682-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602916530764-9682-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260709 13:20:35.744987 19840 consensus_peers.cc:597] T 0587017774214f24bdfa4a3da36e367e P b80b3a62d7d549a9a5c42def46769ac4 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:41855: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:35.748838 19840 consensus_peers.cc:597] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:41855: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:35.754724  9682 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:20:35.756062  9682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:20:35.757488  9682 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:20:35.763316 20498 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
W20260709 13:20:35.794937 19766 consensus_peers.cc:597] T 5b9511148896441a948b772784f62a3d P e90ffed3ba174ababeb21d9415d4d918 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:41855: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:35.838660 19840 consensus_peers.cc:597] T ece79531bbd749d3b9300fe2fbeef9af P b80b3a62d7d549a9a5c42def46769ac4 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:41855: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:35.875365 19840 consensus_peers.cc:597] T ad38a1519fb748588f5a322589274004 P b80b3a62d7d549a9a5c42def46769ac4 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:41855: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:36.026973  9682 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 13:20:36.027415  9682 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.265s	user 0.011s	sys 0.000s
I20260709 13:20:36.027809  9682 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
I20260709 13:20:36.039433 20498 tablet_bootstrap.cc:492] T 984fa8de2ccb49359c2b475e4adfc1ce P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:36.057848 20498 tablet_bootstrap.cc:492] T 984fa8de2ccb49359c2b475e4adfc1ce P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:36.058856 20498 tablet_bootstrap.cc:492] T 984fa8de2ccb49359c2b475e4adfc1ce P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap complete.
I20260709 13:20:36.059520 20498 ts_tablet_manager.cc:1403] T 984fa8de2ccb49359c2b475e4adfc1ce P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.004s
I20260709 13:20:36.061944 20498 raft_consensus.cc:359] T 984fa8de2ccb49359c2b475e4adfc1ce P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:36.062633 20498 raft_consensus.cc:740] T 984fa8de2ccb49359c2b475e4adfc1ce P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:36.063513 20498 consensus_queue.cc:260] T 984fa8de2ccb49359c2b475e4adfc1ce P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:36.107223 20498 ts_tablet_manager.cc:1434] T 984fa8de2ccb49359c2b475e4adfc1ce P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.047s	user 0.011s	sys 0.020s
I20260709 13:20:36.108063 20498 tablet_bootstrap.cc:492] T 0413f22bd6d7447ca630b75be496555f P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:36.124882 20498 tablet_bootstrap.cc:492] T 0413f22bd6d7447ca630b75be496555f P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:36.125945 20498 tablet_bootstrap.cc:492] T 0413f22bd6d7447ca630b75be496555f P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap complete.
I20260709 13:20:36.126646 20498 ts_tablet_manager.cc:1403] T 0413f22bd6d7447ca630b75be496555f P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.019s	user 0.004s	sys 0.012s
I20260709 13:20:36.129711 20498 raft_consensus.cc:359] T 0413f22bd6d7447ca630b75be496555f P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:36.130460 20498 raft_consensus.cc:740] T 0413f22bd6d7447ca630b75be496555f P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:36.131218 20498 consensus_queue.cc:260] T 0413f22bd6d7447ca630b75be496555f P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:36.135174 20498 ts_tablet_manager.cc:1434] T 0413f22bd6d7447ca630b75be496555f P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:20:36.136059 20498 tablet_bootstrap.cc:492] T 563940ad96444ce59e393ea303cc3352 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:36.153492 20498 tablet_bootstrap.cc:492] T 563940ad96444ce59e393ea303cc3352 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:36.156517 20498 tablet_bootstrap.cc:492] T 563940ad96444ce59e393ea303cc3352 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap complete.
I20260709 13:20:36.157222 20498 ts_tablet_manager.cc:1403] T 563940ad96444ce59e393ea303cc3352 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.021s	user 0.018s	sys 0.000s
I20260709 13:20:36.161170 20498 raft_consensus.cc:359] T 563940ad96444ce59e393ea303cc3352 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:36.162233 20498 raft_consensus.cc:740] T 563940ad96444ce59e393ea303cc3352 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:36.162855 20498 consensus_queue.cc:260] T 563940ad96444ce59e393ea303cc3352 P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:36.164649 20498 ts_tablet_manager.cc:1434] T 563940ad96444ce59e393ea303cc3352 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:20:36.165403 20498 tablet_bootstrap.cc:492] T 6a066b96902f41dbb1511eccf26820ce P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:36.179654 20498 tablet_bootstrap.cc:492] T 6a066b96902f41dbb1511eccf26820ce P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:36.180708 20498 tablet_bootstrap.cc:492] T 6a066b96902f41dbb1511eccf26820ce P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap complete.
I20260709 13:20:36.181438 20498 ts_tablet_manager.cc:1403] T 6a066b96902f41dbb1511eccf26820ce P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.016s	user 0.015s	sys 0.000s
I20260709 13:20:36.183650 20498 raft_consensus.cc:359] T 6a066b96902f41dbb1511eccf26820ce P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:36.184401 20498 raft_consensus.cc:740] T 6a066b96902f41dbb1511eccf26820ce P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:36.185209 20498 consensus_queue.cc:260] T 6a066b96902f41dbb1511eccf26820ce P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:36.186580 20498 ts_tablet_manager.cc:1434] T 6a066b96902f41dbb1511eccf26820ce P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.000s
I20260709 13:20:36.187260 20498 tablet_bootstrap.cc:492] T ac3bca5bb7644a8fbc78df43ef3331a0 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:36.200915 20498 tablet_bootstrap.cc:492] T ac3bca5bb7644a8fbc78df43ef3331a0 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:36.201661 20498 tablet_bootstrap.cc:492] T ac3bca5bb7644a8fbc78df43ef3331a0 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap complete.
I20260709 13:20:36.202200 20498 ts_tablet_manager.cc:1403] T ac3bca5bb7644a8fbc78df43ef3331a0 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.015s	user 0.012s	sys 0.000s
I20260709 13:20:36.204030 20498 raft_consensus.cc:359] T ac3bca5bb7644a8fbc78df43ef3331a0 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:36.204540 20498 raft_consensus.cc:740] T ac3bca5bb7644a8fbc78df43ef3331a0 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:36.205153 20498 consensus_queue.cc:260] T ac3bca5bb7644a8fbc78df43ef3331a0 P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:36.207115 20498 ts_tablet_manager.cc:1434] T ac3bca5bb7644a8fbc78df43ef3331a0 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:20:36.207901 20498 tablet_bootstrap.cc:492] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:36.251060 20498 tablet_bootstrap.cc:492] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:36.252009 20498 tablet_bootstrap.cc:492] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap complete.
I20260709 13:20:36.252648 20498 ts_tablet_manager.cc:1403] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.045s	user 0.024s	sys 0.012s
I20260709 13:20:36.255091 20498 raft_consensus.cc:359] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:36.255769 20498 raft_consensus.cc:740] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:36.256412 20498 consensus_queue.cc:260] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:36.258515 20498 ts_tablet_manager.cc:1434] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:20:36.259284 20498 tablet_bootstrap.cc:492] T 2a411802fdaa42e7863236f0af595fe2 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:36.274314 20498 tablet_bootstrap.cc:492] T 2a411802fdaa42e7863236f0af595fe2 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:36.275267 20498 tablet_bootstrap.cc:492] T 2a411802fdaa42e7863236f0af595fe2 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap complete.
I20260709 13:20:36.275889 20498 ts_tablet_manager.cc:1403] T 2a411802fdaa42e7863236f0af595fe2 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.017s	user 0.008s	sys 0.007s
I20260709 13:20:36.278239 20498 raft_consensus.cc:359] T 2a411802fdaa42e7863236f0af595fe2 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:36.278939 20498 raft_consensus.cc:740] T 2a411802fdaa42e7863236f0af595fe2 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:36.279610 20498 consensus_queue.cc:260] T 2a411802fdaa42e7863236f0af595fe2 P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:36.281569 20498 ts_tablet_manager.cc:1434] T 2a411802fdaa42e7863236f0af595fe2 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.001s
I20260709 13:20:36.282335 20498 tablet_bootstrap.cc:492] T 3aaced67782041fa9960c745301a96e7 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:36.315330 20498 tablet_bootstrap.cc:492] T 3aaced67782041fa9960c745301a96e7 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:36.316247 20498 tablet_bootstrap.cc:492] T 3aaced67782041fa9960c745301a96e7 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap complete.
I20260709 13:20:36.316875 20498 ts_tablet_manager.cc:1403] T 3aaced67782041fa9960c745301a96e7 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.035s	user 0.011s	sys 0.003s
I20260709 13:20:36.319196 20498 raft_consensus.cc:359] T 3aaced67782041fa9960c745301a96e7 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:36.319859 20498 raft_consensus.cc:740] T 3aaced67782041fa9960c745301a96e7 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:36.320533 20498 consensus_queue.cc:260] T 3aaced67782041fa9960c745301a96e7 P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:36.322459 20498 ts_tablet_manager.cc:1434] T 3aaced67782041fa9960c745301a96e7 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:20:36.323251 20498 tablet_bootstrap.cc:492] T c393f9d18341477c97fac9a4af7e3adf P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:36.326604  9682 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 13:20:36.327016  9682 ts_tablet_manager.cc:595] Time spent register tablets: real 0.299s	user 0.270s	sys 0.007s
I20260709 13:20:36.337538 20498 tablet_bootstrap.cc:492] T c393f9d18341477c97fac9a4af7e3adf P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:36.338470 20498 tablet_bootstrap.cc:492] T c393f9d18341477c97fac9a4af7e3adf P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap complete.
I20260709 13:20:36.339183 20498 ts_tablet_manager.cc:1403] T c393f9d18341477c97fac9a4af7e3adf P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.016s	user 0.015s	sys 0.000s
I20260709 13:20:36.341835 20498 raft_consensus.cc:359] T c393f9d18341477c97fac9a4af7e3adf P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:36.342702 20498 raft_consensus.cc:740] T c393f9d18341477c97fac9a4af7e3adf P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:36.343468 20498 consensus_queue.cc:260] T c393f9d18341477c97fac9a4af7e3adf P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:36.345567 20498 ts_tablet_manager.cc:1434] T c393f9d18341477c97fac9a4af7e3adf P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:20:36.346349 20498 tablet_bootstrap.cc:492] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:36.373638 20212 raft_consensus.cc:493] T c393f9d18341477c97fac9a4af7e3adf P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:36.374208 20212 raft_consensus.cc:515] T c393f9d18341477c97fac9a4af7e3adf P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:36.377018 20212 leader_election.cc:290] T c393f9d18341477c97fac9a4af7e3adf P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:36.378755 20424 raft_consensus.cc:493] T 7a5476f5d68d496eaedce93e80bff5e4 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:36.379402 20424 raft_consensus.cc:515] T 7a5476f5d68d496eaedce93e80bff5e4 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:36.381337 19873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c393f9d18341477c97fac9a4af7e3adf" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
W20260709 13:20:36.382537 19766 leader_election.cc:336] T c393f9d18341477c97fac9a4af7e3adf P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Network error: Client connection negotiation failed: client connection to 127.9.116.129:41855: connect: Connection refused (error 111)
I20260709 13:20:36.383347 20500 raft_consensus.cc:493] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:36.383896 20500 raft_consensus.cc:515] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:36.381520 20424 leader_election.cc:290] T 7a5476f5d68d496eaedce93e80bff5e4 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
W20260709 13:20:36.393604 19840 leader_election.cc:336] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Network error: Client connection negotiation failed: client connection to 127.9.116.129:41855: connect: Connection refused (error 111)
I20260709 13:20:36.394949 19765 leader_election.cc:304] T c393f9d18341477c97fac9a4af7e3adf P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e90ffed3ba174ababeb21d9415d4d918; no voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4
I20260709 13:20:36.396099 20351 raft_consensus.cc:2749] T c393f9d18341477c97fac9a4af7e3adf P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:20:36.401732 19948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5476f5d68d496eaedce93e80bff5e4" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:36.402490 19948 raft_consensus.cc:2468] T 7a5476f5d68d496eaedce93e80bff5e4 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 1.
I20260709 13:20:36.403759 19873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5476f5d68d496eaedce93e80bff5e4" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:36.404397 19873 raft_consensus.cc:2468] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c62716bf66c24330860573535b8b9430 in term 1.
I20260709 13:20:36.412384 19839 leader_election.cc:304] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:36.413290 19913 leader_election.cc:304] T 7a5476f5d68d496eaedce93e80bff5e4 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430; no voters: 
I20260709 13:20:36.414458 20424 raft_consensus.cc:2804] T 7a5476f5d68d496eaedce93e80bff5e4 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:20:36.414865 20424 raft_consensus.cc:493] T 7a5476f5d68d496eaedce93e80bff5e4 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:36.415247 20424 raft_consensus.cc:3060] T 7a5476f5d68d496eaedce93e80bff5e4 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:36.422256 20212 raft_consensus.cc:493] T 7e3a9d11df99411d8f0edb0df6f14d87 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:36.422878 20212 raft_consensus.cc:515] T 7e3a9d11df99411d8f0edb0df6f14d87 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:36.425900 20212 leader_election.cc:290] T 7e3a9d11df99411d8f0edb0df6f14d87 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
W20260709 13:20:36.428309 19914 leader_election.cc:336] T 7a5476f5d68d496eaedce93e80bff5e4 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Network error: Client connection negotiation failed: client connection to 127.9.116.129:41855: connect: Connection refused (error 111)
I20260709 13:20:36.430923 20500 leader_election.cc:290] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:36.434949 19873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3a9d11df99411d8f0edb0df6f14d87" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:36.437062 20388 raft_consensus.cc:2804] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:20:36.437466 20388 raft_consensus.cc:493] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:36.437817 20388 raft_consensus.cc:3060] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:36.440156 20498 tablet_bootstrap.cc:492] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:36.441275 20498 tablet_bootstrap.cc:492] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap complete.
I20260709 13:20:36.441910 20498 ts_tablet_manager.cc:1403] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.096s	user 0.039s	sys 0.048s
I20260709 13:20:36.444689 20498 raft_consensus.cc:359] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:36.445339 20498 raft_consensus.cc:740] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:36.445982 20498 consensus_queue.cc:260] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:36.452443 20498 ts_tablet_manager.cc:1434] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 13:20:36.453311 20498 tablet_bootstrap.cc:492] T 8abfa1aa35c44699960a8166fa79c41d P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
W20260709 13:20:36.463343 19766 leader_election.cc:336] T 7e3a9d11df99411d8f0edb0df6f14d87 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Network error: Client connection negotiation failed: client connection to 127.9.116.129:41855: connect: Connection refused (error 111)
I20260709 13:20:36.464115 19766 leader_election.cc:304] T 7e3a9d11df99411d8f0edb0df6f14d87 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e90ffed3ba174ababeb21d9415d4d918; no voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4
I20260709 13:20:36.464864 20212 raft_consensus.cc:2749] T 7e3a9d11df99411d8f0edb0df6f14d87 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:20:36.467408 20424 raft_consensus.cc:515] T 7a5476f5d68d496eaedce93e80bff5e4 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:36.465880 20388 raft_consensus.cc:515] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:36.473079 19948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5476f5d68d496eaedce93e80bff5e4" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:36.473884 19948 raft_consensus.cc:2393] T 7a5476f5d68d496eaedce93e80bff5e4 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b80b3a62d7d549a9a5c42def46769ac4 in current term 2: Already voted for candidate c62716bf66c24330860573535b8b9430 in this term.
I20260709 13:20:36.470494 19873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5476f5d68d496eaedce93e80bff5e4" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
I20260709 13:20:36.475276 19873 raft_consensus.cc:2393] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate c62716bf66c24330860573535b8b9430 in current term 2: Already voted for candidate b80b3a62d7d549a9a5c42def46769ac4 in this term.
I20260709 13:20:36.478458 20388 leader_election.cc:290] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 election: Requested vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
W20260709 13:20:36.481179 19840 leader_election.cc:336] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Network error: Client connection negotiation failed: client connection to 127.9.116.129:41855: connect: Connection refused (error 111)
I20260709 13:20:36.481768 19840 leader_election.cc:304] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4; no voters: 8df38c4c53c44e94b36c1e8992476d7c, c62716bf66c24330860573535b8b9430
I20260709 13:20:36.482574 20499 raft_consensus.cc:2749] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:20:36.487140 19914 leader_election.cc:336] T 7a5476f5d68d496eaedce93e80bff5e4 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Network error: Client connection negotiation failed: client connection to 127.9.116.129:41855: connect: Connection refused (error 111)
I20260709 13:20:36.487706 19914 leader_election.cc:304] T 7a5476f5d68d496eaedce93e80bff5e4 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4
I20260709 13:20:36.488520 20501 raft_consensus.cc:2749] T 7a5476f5d68d496eaedce93e80bff5e4 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:20:36.491042 20498 tablet_bootstrap.cc:492] T 8abfa1aa35c44699960a8166fa79c41d P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:36.492115 20498 tablet_bootstrap.cc:492] T 8abfa1aa35c44699960a8166fa79c41d P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap complete.
I20260709 13:20:36.492811 20498 ts_tablet_manager.cc:1403] T 8abfa1aa35c44699960a8166fa79c41d P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.040s	user 0.014s	sys 0.001s
I20260709 13:20:36.494239 20424 leader_election.cc:290] T 7a5476f5d68d496eaedce93e80bff5e4 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 2 election: Requested vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:36.495478 20498 raft_consensus.cc:359] T 8abfa1aa35c44699960a8166fa79c41d P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:36.496282 20498 raft_consensus.cc:740] T 8abfa1aa35c44699960a8166fa79c41d P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:36.497015 20498 consensus_queue.cc:260] T 8abfa1aa35c44699960a8166fa79c41d P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:36.498976 20498 ts_tablet_manager.cc:1434] T 8abfa1aa35c44699960a8166fa79c41d P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:20:36.499769 20498 tablet_bootstrap.cc:492] T e8cfec59c63741869c636df6356546d0 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:36.502812 20351 raft_consensus.cc:493] T 2a411802fdaa42e7863236f0af595fe2 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:36.503561 20351 raft_consensus.cc:515] T 2a411802fdaa42e7863236f0af595fe2 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:36.505813 20351 leader_election.cc:290] T 2a411802fdaa42e7863236f0af595fe2 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:36.508939 19873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a411802fdaa42e7863236f0af595fe2" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:36.524736 20498 tablet_bootstrap.cc:492] T e8cfec59c63741869c636df6356546d0 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:36.526093 20498 tablet_bootstrap.cc:492] T e8cfec59c63741869c636df6356546d0 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap complete.
I20260709 13:20:36.531610 20212 raft_consensus.cc:493] T bd80f9f8533b4cd3b5861ec11917a03d P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:36.532189 20212 raft_consensus.cc:515] T bd80f9f8533b4cd3b5861ec11917a03d P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:36.534235 20212 leader_election.cc:290] T bd80f9f8533b4cd3b5861ec11917a03d P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:36.535833 19948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd80f9f8533b4cd3b5861ec11917a03d" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
W20260709 13:20:36.538169 19766 leader_election.cc:336] T 2a411802fdaa42e7863236f0af595fe2 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Network error: Client connection negotiation failed: client connection to 127.9.116.129:41855: connect: Connection refused (error 111)
I20260709 13:20:36.538717 19766 leader_election.cc:304] T 2a411802fdaa42e7863236f0af595fe2 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e90ffed3ba174ababeb21d9415d4d918; no voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4
I20260709 13:20:36.539654 20478 raft_consensus.cc:2749] T 2a411802fdaa42e7863236f0af595fe2 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:20:36.546154 19766 leader_election.cc:336] T bd80f9f8533b4cd3b5861ec11917a03d P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Network error: Client connection negotiation failed: client connection to 127.9.116.129:41855: connect: Connection refused (error 111)
I20260709 13:20:36.546844 19766 leader_election.cc:304] T bd80f9f8533b4cd3b5861ec11917a03d P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e90ffed3ba174ababeb21d9415d4d918; no voters: 8df38c4c53c44e94b36c1e8992476d7c, c62716bf66c24330860573535b8b9430
I20260709 13:20:36.550419 20212 raft_consensus.cc:2749] T bd80f9f8533b4cd3b5861ec11917a03d P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:20:36.532202 20498 ts_tablet_manager.cc:1403] T e8cfec59c63741869c636df6356546d0 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.033s	user 0.017s	sys 0.008s
I20260709 13:20:36.554750 20498 raft_consensus.cc:359] T e8cfec59c63741869c636df6356546d0 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:36.557677 20498 raft_consensus.cc:740] T e8cfec59c63741869c636df6356546d0 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:36.558602 20498 consensus_queue.cc:260] T e8cfec59c63741869c636df6356546d0 P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:36.561014 20498 ts_tablet_manager.cc:1434] T e8cfec59c63741869c636df6356546d0 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.003s
I20260709 13:20:36.568768 20498 tablet_bootstrap.cc:492] T d1da95785dbf4ef4bde6f5e6ab35b444 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:36.588989 20498 tablet_bootstrap.cc:492] T d1da95785dbf4ef4bde6f5e6ab35b444 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:36.590301 20498 tablet_bootstrap.cc:492] T d1da95785dbf4ef4bde6f5e6ab35b444 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap complete.
I20260709 13:20:36.591122 20498 ts_tablet_manager.cc:1403] T d1da95785dbf4ef4bde6f5e6ab35b444 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.023s	user 0.019s	sys 0.000s
I20260709 13:20:36.594223 20498 raft_consensus.cc:359] T d1da95785dbf4ef4bde6f5e6ab35b444 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:36.594847 20500 raft_consensus.cc:493] T cb33acfa7a9f46ab95b0e77e1394f258 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:36.595139 20498 raft_consensus.cc:740] T d1da95785dbf4ef4bde6f5e6ab35b444 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:36.595417 20500 raft_consensus.cc:515] T cb33acfa7a9f46ab95b0e77e1394f258 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:36.596184 20498 consensus_queue.cc:260] T d1da95785dbf4ef4bde6f5e6ab35b444 P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:36.598547 20498 ts_tablet_manager.cc:1434] T d1da95785dbf4ef4bde6f5e6ab35b444 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:20:36.599538 20498 tablet_bootstrap.cc:492] T 8bdf042a063d4596970f49730ffa04b4 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:36.601064 19795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb33acfa7a9f46ab95b0e77e1394f258" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:36.603142 20388 raft_consensus.cc:493] T 59fc86c7e9f648e3b3f51e936e2bcfec P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:36.603718 20388 raft_consensus.cc:515] T 59fc86c7e9f648e3b3f51e936e2bcfec P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:36.597635 20500 leader_election.cc:290] T cb33acfa7a9f46ab95b0e77e1394f258 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:36.605499 20388 leader_election.cc:290] T 59fc86c7e9f648e3b3f51e936e2bcfec P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
W20260709 13:20:36.606101 19840 leader_election.cc:336] T cb33acfa7a9f46ab95b0e77e1394f258 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Network error: Client connection negotiation failed: client connection to 127.9.116.129:41855: connect: Connection refused (error 111)
I20260709 13:20:36.606508 19840 leader_election.cc:304] T cb33acfa7a9f46ab95b0e77e1394f258 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4; no voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:36.607224 20388 raft_consensus.cc:2749] T cb33acfa7a9f46ab95b0e77e1394f258 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:20:36.608693 19795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59fc86c7e9f648e3b3f51e936e2bcfec" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
W20260709 13:20:36.610260 19840 leader_election.cc:336] T 59fc86c7e9f648e3b3f51e936e2bcfec P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Network error: Client connection negotiation failed: client connection to 127.9.116.129:41855: connect: Connection refused (error 111)
I20260709 13:20:36.611028 19839 leader_election.cc:304] T 59fc86c7e9f648e3b3f51e936e2bcfec P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4; no voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:36.612102 20388 raft_consensus.cc:2749] T 59fc86c7e9f648e3b3f51e936e2bcfec P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:20:36.619199 20498 tablet_bootstrap.cc:492] T 8bdf042a063d4596970f49730ffa04b4 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:36.620376 20498 tablet_bootstrap.cc:492] T 8bdf042a063d4596970f49730ffa04b4 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap complete.
I20260709 13:20:36.621058 20498 ts_tablet_manager.cc:1403] T 8bdf042a063d4596970f49730ffa04b4 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.022s	user 0.013s	sys 0.006s
I20260709 13:20:36.623508 20498 raft_consensus.cc:359] T 8bdf042a063d4596970f49730ffa04b4 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:36.624198 20498 raft_consensus.cc:740] T 8bdf042a063d4596970f49730ffa04b4 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:36.624869 20498 consensus_queue.cc:260] T 8bdf042a063d4596970f49730ffa04b4 P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:36.627691 20498 ts_tablet_manager.cc:1434] T 8bdf042a063d4596970f49730ffa04b4 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:20:36.628851 20498 tablet_bootstrap.cc:492] T 9f60d4b859264720a20f5670c3c71a6f P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:36.637887  9682 rpc_server.cc:307] RPC server started. Bound to: 127.9.116.129:41855
I20260709 13:20:36.638054 20575 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.116.129:41855 every 8 connection(s)
I20260709 13:20:36.639240 20351 raft_consensus.cc:493] T a323eb80c5084159803e7d4ea5d9c410 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:36.639864 20351 raft_consensus.cc:515] T a323eb80c5084159803e7d4ea5d9c410 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:36.642323 20501 raft_consensus.cc:493] T 6551866a22dd4aaf8ee318b9d91b3a02 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:36.643230 20501 raft_consensus.cc:515] T 6551866a22dd4aaf8ee318b9d91b3a02 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:36.648672 19948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a323eb80c5084159803e7d4ea5d9c410" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:36.653133 19795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6551866a22dd4aaf8ee318b9d91b3a02" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:36.654140 20501 leader_election.cc:290] T 6551866a22dd4aaf8ee318b9d91b3a02 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:36.654732 20351 leader_election.cc:290] T a323eb80c5084159803e7d4ea5d9c410 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:36.656246 20388 raft_consensus.cc:493] T 86964b8a61ce494f9096c4fefdc6efbf P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:36.657021 20388 raft_consensus.cc:515] T 86964b8a61ce494f9096c4fefdc6efbf P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:36.661942 19944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86964b8a61ce494f9096c4fefdc6efbf" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:36.664414 20388 leader_election.cc:290] T 86964b8a61ce494f9096c4fefdc6efbf P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:36.667811 20212 raft_consensus.cc:493] T cb33acfa7a9f46ab95b0e77e1394f258 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:36.668378 20212 raft_consensus.cc:515] T cb33acfa7a9f46ab95b0e77e1394f258 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:36.670289 20212 leader_election.cc:290] T cb33acfa7a9f46ab95b0e77e1394f258 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:36.673720 19868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb33acfa7a9f46ab95b0e77e1394f258" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:36.674453 19868 raft_consensus.cc:2468] T cb33acfa7a9f46ab95b0e77e1394f258 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 1.
I20260709 13:20:36.680753 19765 leader_election.cc:304] T cb33acfa7a9f46ab95b0e77e1394f258 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, e90ffed3ba174ababeb21d9415d4d918; no voters: 
I20260709 13:20:36.658563 20501 raft_consensus.cc:493] T 9b2e4bf3fb5c456dab400b096f7bd729 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:36.686611 20501 raft_consensus.cc:515] T 9b2e4bf3fb5c456dab400b096f7bd729 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:36.689033 20478 raft_consensus.cc:2804] T cb33acfa7a9f46ab95b0e77e1394f258 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:20:36.689494 20478 raft_consensus.cc:493] T cb33acfa7a9f46ab95b0e77e1394f258 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:36.689922 20478 raft_consensus.cc:3060] T cb33acfa7a9f46ab95b0e77e1394f258 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:36.699505 19868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b2e4bf3fb5c456dab400b096f7bd729" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:36.704090 20556 raft_consensus.cc:493] T d5f8619b1f9f46adaa41774af856d9b1 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:36.704844 20545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6551866a22dd4aaf8ee318b9d91b3a02" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:36.704731 20556 raft_consensus.cc:515] T d5f8619b1f9f46adaa41774af856d9b1 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
W20260709 13:20:36.706353 19914 leader_election.cc:343] T 6551866a22dd4aaf8ee318b9d91b3a02 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:36.706847 19914 leader_election.cc:304] T 6551866a22dd4aaf8ee318b9d91b3a02 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:36.707253 20556 leader_election.cc:290] T d5f8619b1f9f46adaa41774af856d9b1 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:36.709118 20424 raft_consensus.cc:2749] T 6551866a22dd4aaf8ee318b9d91b3a02 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:20:36.711181 19944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5f8619b1f9f46adaa41774af856d9b1" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:36.723192 20544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a323eb80c5084159803e7d4ea5d9c410" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:36.725670 20424 raft_consensus.cc:493] T a323eb80c5084159803e7d4ea5d9c410 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:36.726267 20424 raft_consensus.cc:515] T a323eb80c5084159803e7d4ea5d9c410 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:36.726842 20543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5f8619b1f9f46adaa41774af856d9b1" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:36.727288 20545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb33acfa7a9f46ab95b0e77e1394f258" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:36.727823 20543 raft_consensus.cc:2468] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 1.
I20260709 13:20:36.728530 20424 leader_election.cc:290] T a323eb80c5084159803e7d4ea5d9c410 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:36.729457 20498 tablet_bootstrap.cc:492] T 9f60d4b859264720a20f5670c3c71a6f P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:36.729686 19796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a323eb80c5084159803e7d4ea5d9c410" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:36.730311 19796 raft_consensus.cc:2468] T a323eb80c5084159803e7d4ea5d9c410 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c62716bf66c24330860573535b8b9430 in term 1.
I20260709 13:20:36.730506 20498 tablet_bootstrap.cc:492] T 9f60d4b859264720a20f5670c3c71a6f P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap complete.
I20260709 13:20:36.731314 19913 leader_election.cc:304] T a323eb80c5084159803e7d4ea5d9c410 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c62716bf66c24330860573535b8b9430, e90ffed3ba174ababeb21d9415d4d918; no voters: 
I20260709 13:20:36.731592 20498 ts_tablet_manager.cc:1403] T 9f60d4b859264720a20f5670c3c71a6f P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.103s	user 0.027s	sys 0.037s
I20260709 13:20:36.731618 20544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b2e4bf3fb5c456dab400b096f7bd729" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:36.732160 20424 raft_consensus.cc:2804] T a323eb80c5084159803e7d4ea5d9c410 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:20:36.732795 20424 raft_consensus.cc:493] T a323eb80c5084159803e7d4ea5d9c410 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:36.733148 20424 raft_consensus.cc:3060] T a323eb80c5084159803e7d4ea5d9c410 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:36.734439 20498 raft_consensus.cc:359] T 9f60d4b859264720a20f5670c3c71a6f P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:36.735172 20498 raft_consensus.cc:740] T 9f60d4b859264720a20f5670c3c71a6f P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:36.735459 20543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a323eb80c5084159803e7d4ea5d9c410" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:36.735889 20498 consensus_queue.cc:260] T 9f60d4b859264720a20f5670c3c71a6f P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:36.737542 20498 ts_tablet_manager.cc:1434] T 9f60d4b859264720a20f5670c3c71a6f P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
W20260709 13:20:36.737895 19766 leader_election.cc:343] T a323eb80c5084159803e7d4ea5d9c410 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:36.738353 20498 tablet_bootstrap.cc:492] T ece79531bbd749d3b9300fe2fbeef9af P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:36.738349 19766 leader_election.cc:304] T a323eb80c5084159803e7d4ea5d9c410 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e90ffed3ba174ababeb21d9415d4d918; no voters: 8df38c4c53c44e94b36c1e8992476d7c, c62716bf66c24330860573535b8b9430
I20260709 13:20:36.739444 20556 raft_consensus.cc:2749] T a323eb80c5084159803e7d4ea5d9c410 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:20:36.740615 19766 leader_election.cc:343] T cb33acfa7a9f46ab95b0e77e1394f258 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:36.740612 20424 raft_consensus.cc:515] T a323eb80c5084159803e7d4ea5d9c410 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:36.743484 19766 leader_election.cc:304] T d5f8619b1f9f46adaa41774af856d9b1 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918; no voters: c62716bf66c24330860573535b8b9430
I20260709 13:20:36.744598 20556 raft_consensus.cc:2804] T d5f8619b1f9f46adaa41774af856d9b1 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:20:36.745049 20556 raft_consensus.cc:493] T d5f8619b1f9f46adaa41774af856d9b1 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:36.745429 20556 raft_consensus.cc:3060] T d5f8619b1f9f46adaa41774af856d9b1 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:36.746085 20543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86964b8a61ce494f9096c4fefdc6efbf" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:36.746490 19796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a323eb80c5084159803e7d4ea5d9c410" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
I20260709 13:20:36.747126 19796 raft_consensus.cc:3060] T a323eb80c5084159803e7d4ea5d9c410 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:36.748798 20500 raft_consensus.cc:493] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:36.749398 20500 raft_consensus.cc:515] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
W20260709 13:20:36.751111 19914 leader_election.cc:343] T 9b2e4bf3fb5c456dab400b096f7bd729 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:36.751596 19914 leader_election.cc:304] T 9b2e4bf3fb5c456dab400b096f7bd729 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4
I20260709 13:20:36.752224 20500 leader_election.cc:290] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:36.752381 20370 raft_consensus.cc:2749] T 9b2e4bf3fb5c456dab400b096f7bd729 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:20:36.754419 19795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563940ad96444ce59e393ea303cc3352" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
W20260709 13:20:36.755316 19840 leader_election.cc:343] T 86964b8a61ce494f9096c4fefdc6efbf P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:36.755779 19840 leader_election.cc:304] T 86964b8a61ce494f9096c4fefdc6efbf P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4; no voters: 8df38c4c53c44e94b36c1e8992476d7c, c62716bf66c24330860573535b8b9430
I20260709 13:20:36.757025 20500 raft_consensus.cc:2749] T 86964b8a61ce494f9096c4fefdc6efbf P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:20:36.720456 20501 leader_election.cc:290] T 9b2e4bf3fb5c456dab400b096f7bd729 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
W20260709 13:20:36.759382 19914 leader_election.cc:343] T a323eb80c5084159803e7d4ea5d9c410 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:36.762573 20544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a323eb80c5084159803e7d4ea5d9c410" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
I20260709 13:20:36.763047 20424 leader_election.cc:290] T a323eb80c5084159803e7d4ea5d9c410 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 2 election: Requested vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:36.763986 20500 raft_consensus.cc:493] T c393f9d18341477c97fac9a4af7e3adf P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:36.764554 20500 raft_consensus.cc:515] T c393f9d18341477c97fac9a4af7e3adf P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:36.765125 19796 raft_consensus.cc:2468] T a323eb80c5084159803e7d4ea5d9c410 P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c62716bf66c24330860573535b8b9430 in term 2.
I20260709 13:20:36.765658 20543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563940ad96444ce59e393ea303cc3352" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:36.766760 20500 leader_election.cc:290] T c393f9d18341477c97fac9a4af7e3adf P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:36.766909 19913 leader_election.cc:304] T a323eb80c5084159803e7d4ea5d9c410 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c62716bf66c24330860573535b8b9430, e90ffed3ba174ababeb21d9415d4d918; no voters: 
I20260709 13:20:36.767925 20544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c393f9d18341477c97fac9a4af7e3adf" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:36.766906 20543 raft_consensus.cc:2468] T 563940ad96444ce59e393ea303cc3352 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 1.
I20260709 13:20:36.768347 19795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c393f9d18341477c97fac9a4af7e3adf" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:36.768782 20544 raft_consensus.cc:2468] T c393f9d18341477c97fac9a4af7e3adf P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 1.
I20260709 13:20:36.769066 19795 raft_consensus.cc:2468] T c393f9d18341477c97fac9a4af7e3adf P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 1.
I20260709 13:20:36.769326 20424 raft_consensus.cc:2804] T a323eb80c5084159803e7d4ea5d9c410 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:20:36.770102 20424 raft_consensus.cc:697] T a323eb80c5084159803e7d4ea5d9c410 P c62716bf66c24330860573535b8b9430 [term 2 LEADER]: Becoming Leader. State: Replica: c62716bf66c24330860573535b8b9430, State: Running, Role: LEADER
W20260709 13:20:36.770696 19914 leader_election.cc:343] T a323eb80c5084159803e7d4ea5d9c410 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:36.770918 19840 leader_election.cc:304] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4; no voters: e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:36.772210 20499 raft_consensus.cc:2804] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:20:36.772648 20499 raft_consensus.cc:493] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:36.771962 20424 consensus_queue.cc:237] T a323eb80c5084159803e7d4ea5d9c410 P c62716bf66c24330860573535b8b9430 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:36.773039 20499 raft_consensus.cc:3060] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:36.773250 19840 leader_election.cc:304] T c393f9d18341477c97fac9a4af7e3adf P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4; no voters: 
I20260709 13:20:36.774737 20584 raft_consensus.cc:493] T dc5c36b41652443a8130592df8a66eb9 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:36.775904 20388 raft_consensus.cc:2804] T c393f9d18341477c97fac9a4af7e3adf P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:20:36.775467 20584 raft_consensus.cc:515] T dc5c36b41652443a8130592df8a66eb9 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:36.776319 20388 raft_consensus.cc:493] T c393f9d18341477c97fac9a4af7e3adf P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:36.776835 20388 raft_consensus.cc:3060] T c393f9d18341477c97fac9a4af7e3adf P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:36.779163 20584 leader_election.cc:290] T dc5c36b41652443a8130592df8a66eb9 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:36.788372 20544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5c36b41652443a8130592df8a66eb9" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:36.788573 20499 raft_consensus.cc:515] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:36.791663 20577 heartbeater.cc:344] Connected to a master server at 127.9.116.190:44649
I20260709 13:20:36.791836 20499 leader_election.cc:290] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 election: Requested vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:36.792111 20577 heartbeater.cc:461] Registering TS with master...
W20260709 13:20:36.792192 19840 leader_election.cc:343] T dc5c36b41652443a8130592df8a66eb9 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:36.793114 20577 heartbeater.cc:507] Master 127.9.116.190:44649 requested a full tablet report, sending...
I20260709 13:20:36.794518 20351 raft_consensus.cc:493] T dc5c36b41652443a8130592df8a66eb9 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:36.795307 20351 raft_consensus.cc:515] T dc5c36b41652443a8130592df8a66eb9 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:36.797948 20351 leader_election.cc:290] T dc5c36b41652443a8130592df8a66eb9 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:36.800444 19873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5c36b41652443a8130592df8a66eb9" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:36.801124 19873 raft_consensus.cc:2468] T dc5c36b41652443a8130592df8a66eb9 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 1.
I20260709 13:20:36.802641 19765 leader_election.cc:304] T dc5c36b41652443a8130592df8a66eb9 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, e90ffed3ba174ababeb21d9415d4d918; no voters: 
I20260709 13:20:36.800314 20543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563940ad96444ce59e393ea303cc3352" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
I20260709 13:20:36.803862 19798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563940ad96444ce59e393ea303cc3352" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
I20260709 13:20:36.803985 20543 raft_consensus.cc:3060] T 563940ad96444ce59e393ea303cc3352 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:36.807814 20498 tablet_bootstrap.cc:492] T ece79531bbd749d3b9300fe2fbeef9af P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:36.807930 20212 raft_consensus.cc:2804] T dc5c36b41652443a8130592df8a66eb9 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:20:36.808368 20212 raft_consensus.cc:493] T dc5c36b41652443a8130592df8a66eb9 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:36.808763 20212 raft_consensus.cc:3060] T dc5c36b41652443a8130592df8a66eb9 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:36.808923 20498 tablet_bootstrap.cc:492] T ece79531bbd749d3b9300fe2fbeef9af P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap complete.
I20260709 13:20:36.809643 20498 ts_tablet_manager.cc:1403] T ece79531bbd749d3b9300fe2fbeef9af P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.072s	user 0.024s	sys 0.031s
I20260709 13:20:36.810444 19796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5c36b41652443a8130592df8a66eb9" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:36.812348 20498 raft_consensus.cc:359] T ece79531bbd749d3b9300fe2fbeef9af P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:36.813109 20498 raft_consensus.cc:740] T ece79531bbd749d3b9300fe2fbeef9af P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:36.813961 20498 consensus_queue.cc:260] T ece79531bbd749d3b9300fe2fbeef9af P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:36.817442 20544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5c36b41652443a8130592df8a66eb9" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:36.822108 20498 ts_tablet_manager.cc:1434] T ece79531bbd749d3b9300fe2fbeef9af P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.000s
I20260709 13:20:36.823113 20498 tablet_bootstrap.cc:492] T bc04a038495940c1bbeda4a2e6f17b82 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:36.825165 20388 raft_consensus.cc:515] T c393f9d18341477c97fac9a4af7e3adf P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:36.821786 20212 raft_consensus.cc:515] T dc5c36b41652443a8130592df8a66eb9 P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:36.828778 20212 leader_election.cc:290] T dc5c36b41652443a8130592df8a66eb9 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 election: Requested vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:36.830678 19619 ts_manager.cc:194] Re-registered known tserver with Master: 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
W20260709 13:20:36.832556 19766 leader_election.cc:343] T dc5c36b41652443a8130592df8a66eb9 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:36.831223 19796 raft_consensus.cc:2393] T dc5c36b41652443a8130592df8a66eb9 P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b80b3a62d7d549a9a5c42def46769ac4 in current term 2: Already voted for candidate e90ffed3ba174ababeb21d9415d4d918 in this term.
I20260709 13:20:36.834177 20544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5c36b41652443a8130592df8a66eb9" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
W20260709 13:20:36.835531 19766 leader_election.cc:343] T dc5c36b41652443a8130592df8a66eb9 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:36.718127 20478 raft_consensus.cc:515] T cb33acfa7a9f46ab95b0e77e1394f258 P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:36.838385 20478 leader_election.cc:290] T cb33acfa7a9f46ab95b0e77e1394f258 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 election: Requested vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:36.841899 19839 leader_election.cc:304] T dc5c36b41652443a8130592df8a66eb9 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4; no voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:36.842722 20500 raft_consensus.cc:3060] T dc5c36b41652443a8130592df8a66eb9 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:36.851182 20500 raft_consensus.cc:2749] T dc5c36b41652443a8130592df8a66eb9 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:20:36.853125 20556 raft_consensus.cc:515] T d5f8619b1f9f46adaa41774af856d9b1 P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:36.856988 20556 leader_election.cc:290] T d5f8619b1f9f46adaa41774af856d9b1 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 election: Requested vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:36.858083 20544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb33acfa7a9f46ab95b0e77e1394f258" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
W20260709 13:20:36.859967 19766 leader_election.cc:343] T cb33acfa7a9f46ab95b0e77e1394f258 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:36.860605 20587 raft_consensus.cc:493] T 6551866a22dd4aaf8ee318b9d91b3a02 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:36.861387 20587 raft_consensus.cc:515] T 6551866a22dd4aaf8ee318b9d91b3a02 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:36.861768 20544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5f8619b1f9f46adaa41774af856d9b1" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
I20260709 13:20:36.862641 20544 raft_consensus.cc:3060] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:36.864307 19868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5c36b41652443a8130592df8a66eb9" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
I20260709 13:20:36.867682 19873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb33acfa7a9f46ab95b0e77e1394f258" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
I20260709 13:20:36.868552 19873 raft_consensus.cc:3060] T cb33acfa7a9f46ab95b0e77e1394f258 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:36.872644 20388 leader_election.cc:290] T c393f9d18341477c97fac9a4af7e3adf P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 election: Requested vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:36.877352 20545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c393f9d18341477c97fac9a4af7e3adf" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
I20260709 13:20:36.878274 20545 raft_consensus.cc:3060] T c393f9d18341477c97fac9a4af7e3adf P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:36.881367 20556 raft_consensus.cc:493] T 59fc86c7e9f648e3b3f51e936e2bcfec P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:36.881983 20556 raft_consensus.cc:515] T 59fc86c7e9f648e3b3f51e936e2bcfec P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:36.884231 20556 leader_election.cc:290] T 59fc86c7e9f648e3b3f51e936e2bcfec P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:36.887449 19796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c393f9d18341477c97fac9a4af7e3adf" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
I20260709 13:20:36.888101 19796 raft_consensus.cc:3060] T c393f9d18341477c97fac9a4af7e3adf P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:36.889307 20545 raft_consensus.cc:2468] T c393f9d18341477c97fac9a4af7e3adf P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 2.
I20260709 13:20:36.890412 19840 leader_election.cc:304] T c393f9d18341477c97fac9a4af7e3adf P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4; no voters: 
I20260709 13:20:36.891743 19944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5f8619b1f9f46adaa41774af856d9b1" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:36.893254 20541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59fc86c7e9f648e3b3f51e936e2bcfec" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:36.894389 19873 raft_consensus.cc:2468] T cb33acfa7a9f46ab95b0e77e1394f258 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 2.
W20260709 13:20:36.894470 19766 leader_election.cc:343] T 59fc86c7e9f648e3b3f51e936e2bcfec P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:36.896238 19871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59fc86c7e9f648e3b3f51e936e2bcfec" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:36.896919 19871 raft_consensus.cc:2468] T 59fc86c7e9f648e3b3f51e936e2bcfec P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 1.
I20260709 13:20:36.894196 20500 raft_consensus.cc:2804] T c393f9d18341477c97fac9a4af7e3adf P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:20:36.898125 20500 raft_consensus.cc:697] T c393f9d18341477c97fac9a4af7e3adf P b80b3a62d7d549a9a5c42def46769ac4 [term 2 LEADER]: Becoming Leader. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Running, Role: LEADER
I20260709 13:20:36.900399 20500 consensus_queue.cc:237] T c393f9d18341477c97fac9a4af7e3adf P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:36.906184 19621 catalog_manager.cc:5697] T a323eb80c5084159803e7d4ea5d9c410 P c62716bf66c24330860573535b8b9430 reported cstate change: term changed from 1 to 2, leader changed from 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129) to c62716bf66c24330860573535b8b9430 (127.9.116.132). New cstate: current_term: 2 leader_uuid: "c62716bf66c24330860573535b8b9430" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:36.911845 19796 raft_consensus.cc:2468] T c393f9d18341477c97fac9a4af7e3adf P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 2.
I20260709 13:20:36.915647 20424 raft_consensus.cc:493] T d5f8619b1f9f46adaa41774af856d9b1 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:36.916260 20424 raft_consensus.cc:515] T d5f8619b1f9f46adaa41774af856d9b1 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:36.917909 19765 leader_election.cc:304] T 59fc86c7e9f648e3b3f51e936e2bcfec P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, e90ffed3ba174ababeb21d9415d4d918; no voters: 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:36.922884 20212 raft_consensus.cc:2804] T 59fc86c7e9f648e3b3f51e936e2bcfec P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:20:36.923262 20212 raft_consensus.cc:493] T 59fc86c7e9f648e3b3f51e936e2bcfec P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:36.923596 20212 raft_consensus.cc:3060] T 59fc86c7e9f648e3b3f51e936e2bcfec P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:36.925093 20556 raft_consensus.cc:493] T 563940ad96444ce59e393ea303cc3352 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:36.925773 20556 raft_consensus.cc:515] T 563940ad96444ce59e393ea303cc3352 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:36.926928 20370 raft_consensus.cc:493] T 86964b8a61ce494f9096c4fefdc6efbf P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:36.927413 20370 raft_consensus.cc:515] T 86964b8a61ce494f9096c4fefdc6efbf P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:36.927999 20556 leader_election.cc:290] T 563940ad96444ce59e393ea303cc3352 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:36.937261 20370 leader_election.cc:290] T 86964b8a61ce494f9096c4fefdc6efbf P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:36.945670 20424 leader_election.cc:290] T d5f8619b1f9f46adaa41774af856d9b1 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:36.951910 19873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563940ad96444ce59e393ea303cc3352" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:36.952687 19873 raft_consensus.cc:2393] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e90ffed3ba174ababeb21d9415d4d918 in current term 2: Already voted for candidate b80b3a62d7d549a9a5c42def46769ac4 in this term.
I20260709 13:20:36.928544 19868 raft_consensus.cc:2468] T dc5c36b41652443a8130592df8a66eb9 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 2.
I20260709 13:20:36.948022 20584 raft_consensus.cc:493] T e804b7d8fd7e44f59516fb299bea7a2c P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:36.954173 20370 raft_consensus.cc:493] T e804b7d8fd7e44f59516fb299bea7a2c P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:36.954663 20370 raft_consensus.cc:515] T e804b7d8fd7e44f59516fb299bea7a2c P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:36.949658 20544 raft_consensus.cc:2468] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 2.
I20260709 13:20:36.950623 20543 raft_consensus.cc:2468] T 563940ad96444ce59e393ea303cc3352 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 2.
I20260709 13:20:36.958738 20541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563940ad96444ce59e393ea303cc3352" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:36.972338 20541 raft_consensus.cc:2393] T 563940ad96444ce59e393ea303cc3352 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e90ffed3ba174ababeb21d9415d4d918 in current term 2: Already voted for candidate b80b3a62d7d549a9a5c42def46769ac4 in this term.
I20260709 13:20:36.972209 19840 leader_election.cc:304] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4; no voters: e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:36.974061 20499 raft_consensus.cc:2804] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:20:36.974094 20370 leader_election.cc:290] T e804b7d8fd7e44f59516fb299bea7a2c P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:36.974771 20499 raft_consensus.cc:697] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 LEADER]: Becoming Leader. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Running, Role: LEADER
I20260709 13:20:36.973969 19766 leader_election.cc:304] T d5f8619b1f9f46adaa41774af856d9b1 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918; no voters: c62716bf66c24330860573535b8b9430
I20260709 13:20:36.975785 20499 consensus_queue.cc:237] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:36.976449 20556 raft_consensus.cc:2804] T d5f8619b1f9f46adaa41774af856d9b1 P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:20:36.976886 20498 tablet_bootstrap.cc:492] T bc04a038495940c1bbeda4a2e6f17b82 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:36.977023 20556 raft_consensus.cc:697] T d5f8619b1f9f46adaa41774af856d9b1 P e90ffed3ba174ababeb21d9415d4d918 [term 2 LEADER]: Becoming Leader. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Running, Role: LEADER
I20260709 13:20:36.977241 20587 leader_election.cc:290] T 6551866a22dd4aaf8ee318b9d91b3a02 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:36.977886 20498 tablet_bootstrap.cc:492] T bc04a038495940c1bbeda4a2e6f17b82 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap complete.
I20260709 13:20:36.960000 20540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5f8619b1f9f46adaa41774af856d9b1" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:36.978916 20540 raft_consensus.cc:2393] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c62716bf66c24330860573535b8b9430 in current term 2: Already voted for candidate e90ffed3ba174ababeb21d9415d4d918 in this term.
I20260709 13:20:36.979916 20538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6551866a22dd4aaf8ee318b9d91b3a02" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:36.977941 20556 consensus_queue.cc:237] T d5f8619b1f9f46adaa41774af856d9b1 P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:36.981554 20498 ts_tablet_manager.cc:1403] T bc04a038495940c1bbeda4a2e6f17b82 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.159s	user 0.032s	sys 0.054s
W20260709 13:20:36.982046 19766 leader_election.cc:343] T 6551866a22dd4aaf8ee318b9d91b3a02 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:36.962622 20545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86964b8a61ce494f9096c4fefdc6efbf" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
W20260709 13:20:36.984514 19914 leader_election.cc:343] T 86964b8a61ce494f9096c4fefdc6efbf P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:36.984658 20498 raft_consensus.cc:359] T bc04a038495940c1bbeda4a2e6f17b82 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:36.959259 20584 raft_consensus.cc:515] T e804b7d8fd7e44f59516fb299bea7a2c P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:36.986330 20544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e804b7d8fd7e44f59516fb299bea7a2c" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
W20260709 13:20:36.989039 19914 leader_election.cc:343] T e804b7d8fd7e44f59516fb299bea7a2c P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:36.989887 20584 leader_election.cc:290] T e804b7d8fd7e44f59516fb299bea7a2c P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:36.958458 20212 raft_consensus.cc:515] T 59fc86c7e9f648e3b3f51e936e2bcfec P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:36.985410 20498 raft_consensus.cc:740] T bc04a038495940c1bbeda4a2e6f17b82 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:36.993960 20370 raft_consensus.cc:493] T 63868398f3604f529519d926d2831f4e P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
W20260709 13:20:36.995123 19839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:42643, user_credentials={real_user=slave}} blocked reactor thread for 73876.9us
I20260709 13:20:36.996644 20370 raft_consensus.cc:515] T 63868398f3604f529519d926d2831f4e P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:36.997006 20498 consensus_queue.cc:260] T bc04a038495940c1bbeda4a2e6f17b82 P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:36.998090 20544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e804b7d8fd7e44f59516fb299bea7a2c" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:36.998914 20424 raft_consensus.cc:493] T bd80f9f8533b4cd3b5861ec11917a03d P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:37.012856 20424 raft_consensus.cc:515] T bd80f9f8533b4cd3b5861ec11917a03d P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:37.014596 20424 leader_election.cc:290] T bd80f9f8533b4cd3b5861ec11917a03d P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:36.998613 20370 leader_election.cc:290] T 63868398f3604f529519d926d2831f4e P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:37.001010 20498 ts_tablet_manager.cc:1434] T bc04a038495940c1bbeda4a2e6f17b82 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.001s
I20260709 13:20:37.015365 20545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd80f9f8533b4cd3b5861ec11917a03d" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:37.016407 20498 tablet_bootstrap.cc:492] T 7e3099f1888c4ed18c30523bf9ec94f5 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
W20260709 13:20:37.016769 19840 leader_election.cc:343] T e804b7d8fd7e44f59516fb299bea7a2c P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:20:37.016978 19914 leader_election.cc:343] T bd80f9f8533b4cd3b5861ec11917a03d P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:37.019496 20499 raft_consensus.cc:493] T 2a411802fdaa42e7863236f0af595fe2 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:37.020396 20499 raft_consensus.cc:515] T 2a411802fdaa42e7863236f0af595fe2 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:37.000943 20541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63868398f3604f529519d926d2831f4e" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:37.002833 19948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e804b7d8fd7e44f59516fb299bea7a2c" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:37.023499 19948 raft_consensus.cc:2468] T e804b7d8fd7e44f59516fb299bea7a2c P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 1.
W20260709 13:20:37.025352 19914 leader_election.cc:343] T 63868398f3604f529519d926d2831f4e P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:37.026206 19839 leader_election.cc:304] T e804b7d8fd7e44f59516fb299bea7a2c P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:37.003369 20212 leader_election.cc:290] T 59fc86c7e9f648e3b3f51e936e2bcfec P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 election: Requested vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:37.004707 20540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59fc86c7e9f648e3b3f51e936e2bcfec" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
I20260709 13:20:37.007601 20388 raft_consensus.cc:493] T 7e3a9d11df99411d8f0edb0df6f14d87 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:37.028194 20592 raft_consensus.cc:2804] T e804b7d8fd7e44f59516fb299bea7a2c P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:20:37.028007 20388 raft_consensus.cc:515] T 7e3a9d11df99411d8f0edb0df6f14d87 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:37.028589 20592 raft_consensus.cc:493] T e804b7d8fd7e44f59516fb299bea7a2c P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:37.028980 20592 raft_consensus.cc:3060] T e804b7d8fd7e44f59516fb299bea7a2c P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:37.029968 20388 leader_election.cc:290] T 7e3a9d11df99411d8f0edb0df6f14d87 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:37.031569 20540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3a9d11df99411d8f0edb0df6f14d87" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
W20260709 13:20:37.033063 19840 leader_election.cc:343] T 7e3a9d11df99411d8f0edb0df6f14d87 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:37.008477 20584 raft_consensus.cc:493] T 9b2e4bf3fb5c456dab400b096f7bd729 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:37.033506 19796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3a9d11df99411d8f0edb0df6f14d87" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:37.033771 20584 raft_consensus.cc:515] T 9b2e4bf3fb5c456dab400b096f7bd729 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:37.034273 19796 raft_consensus.cc:2468] T 7e3a9d11df99411d8f0edb0df6f14d87 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 1.
I20260709 13:20:37.009128 19868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86964b8a61ce494f9096c4fefdc6efbf" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:37.036294 19868 raft_consensus.cc:2468] T 86964b8a61ce494f9096c4fefdc6efbf P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c62716bf66c24330860573535b8b9430 in term 1.
W20260709 13:20:37.036743 19765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.132:42643, user_credentials={real_user=slave}} blocked reactor thread for 106765us
I20260709 13:20:37.036969 20540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b2e4bf3fb5c456dab400b096f7bd729" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:37.010193 19873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e804b7d8fd7e44f59516fb299bea7a2c" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4" is_pre_election: true
I20260709 13:20:37.040007 20499 leader_election.cc:290] T 2a411802fdaa42e7863236f0af595fe2 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:37.040659 20540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a411802fdaa42e7863236f0af595fe2" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:37.041431 20540 raft_consensus.cc:2468] T 2a411802fdaa42e7863236f0af595fe2 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 1.
I20260709 13:20:37.042124 19948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b2e4bf3fb5c456dab400b096f7bd729" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:37.042238 19913 leader_election.cc:304] T 86964b8a61ce494f9096c4fefdc6efbf P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c
W20260709 13:20:37.042662 19840 leader_election.cc:343] T 9b2e4bf3fb5c456dab400b096f7bd729 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:37.042865 19948 raft_consensus.cc:2468] T 9b2e4bf3fb5c456dab400b096f7bd729 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 1.
I20260709 13:20:37.043447 19840 leader_election.cc:304] T 2a411802fdaa42e7863236f0af595fe2 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4; no voters: 
I20260709 13:20:37.043542 20370 raft_consensus.cc:2804] T 86964b8a61ce494f9096c4fefdc6efbf P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:20:37.044409 20370 raft_consensus.cc:493] T 86964b8a61ce494f9096c4fefdc6efbf P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
W20260709 13:20:37.044476 19766 leader_election.cc:343] T 59fc86c7e9f648e3b3f51e936e2bcfec P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:37.044497 19948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6551866a22dd4aaf8ee318b9d91b3a02" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:37.044840 20370 raft_consensus.cc:3060] T 86964b8a61ce494f9096c4fefdc6efbf P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:37.044986 19868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59fc86c7e9f648e3b3f51e936e2bcfec" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
I20260709 13:20:37.045323 19948 raft_consensus.cc:2468] T 6551866a22dd4aaf8ee318b9d91b3a02 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 1.
I20260709 13:20:37.045812 19868 raft_consensus.cc:3060] T 59fc86c7e9f648e3b3f51e936e2bcfec P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:37.050364 20584 leader_election.cc:290] T 9b2e4bf3fb5c456dab400b096f7bd729 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:37.050814 20499 raft_consensus.cc:2804] T 2a411802fdaa42e7863236f0af595fe2 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:20:37.051326 20499 raft_consensus.cc:493] T 2a411802fdaa42e7863236f0af595fe2 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:37.051735 20499 raft_consensus.cc:3060] T 2a411802fdaa42e7863236f0af595fe2 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:37.054939 20556 raft_consensus.cc:493] T 63868398f3604f529519d926d2831f4e P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:37.055576 20556 raft_consensus.cc:515] T 63868398f3604f529519d926d2831f4e P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:37.060158 20556 leader_election.cc:290] T 63868398f3604f529519d926d2831f4e P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:37.061333 19796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a411802fdaa42e7863236f0af595fe2" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:37.062026 19796 raft_consensus.cc:2468] T 2a411802fdaa42e7863236f0af595fe2 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 1.
I20260709 13:20:37.064489 20370 raft_consensus.cc:515] T 86964b8a61ce494f9096c4fefdc6efbf P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:37.066483 19868 raft_consensus.cc:2468] T 59fc86c7e9f648e3b3f51e936e2bcfec P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 2.
I20260709 13:20:37.066972 19839 leader_election.cc:304] T 7e3a9d11df99411d8f0edb0df6f14d87 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, e90ffed3ba174ababeb21d9415d4d918; no voters: 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:37.067780 19796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5f8619b1f9f46adaa41774af856d9b1" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:37.080138 19623 catalog_manager.cc:5697] T c393f9d18341477c97fac9a4af7e3adf P b80b3a62d7d549a9a5c42def46769ac4 reported cstate change: term changed from 1 to 2, leader changed from 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129) to b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131). New cstate: current_term: 2 leader_uuid: "b80b3a62d7d549a9a5c42def46769ac4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:37.068274 19795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63868398f3604f529519d926d2831f4e" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:37.069036 19765 leader_election.cc:304] T cb33acfa7a9f46ab95b0e77e1394f258 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, e90ffed3ba174ababeb21d9415d4d918; no voters: 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:37.083122 19795 raft_consensus.cc:2468] T 63868398f3604f529519d926d2831f4e P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c62716bf66c24330860573535b8b9430 in term 1.
I20260709 13:20:37.084240 19765 leader_election.cc:304] T 563940ad96444ce59e393ea303cc3352 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e90ffed3ba174ababeb21d9415d4d918; no voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4
I20260709 13:20:37.071179 19799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd80f9f8533b4cd3b5861ec11917a03d" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918" is_pre_election: true
I20260709 13:20:37.071890 20544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63868398f3604f529519d926d2831f4e" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
I20260709 13:20:37.085440 19799 raft_consensus.cc:2468] T bd80f9f8533b4cd3b5861ec11917a03d P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c62716bf66c24330860573535b8b9430 in term 1.
I20260709 13:20:37.085507 20556 raft_consensus.cc:2804] T cb33acfa7a9f46ab95b0e77e1394f258 P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:20:37.086129 20556 raft_consensus.cc:697] T cb33acfa7a9f46ab95b0e77e1394f258 P e90ffed3ba174ababeb21d9415d4d918 [term 2 LEADER]: Becoming Leader. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Running, Role: LEADER
W20260709 13:20:37.086341 19766 leader_election.cc:343] T 63868398f3604f529519d926d2831f4e P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:37.087046 20556 consensus_queue.cc:237] T cb33acfa7a9f46ab95b0e77e1394f258 P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:37.087982 20351 raft_consensus.cc:3060] T 563940ad96444ce59e393ea303cc3352 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:37.091599 19765 leader_election.cc:304] T dc5c36b41652443a8130592df8a66eb9 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, e90ffed3ba174ababeb21d9415d4d918; no voters: 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:37.076249 20540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86964b8a61ce494f9096c4fefdc6efbf" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
I20260709 13:20:37.078215 20499 raft_consensus.cc:515] T 2a411802fdaa42e7863236f0af595fe2 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
W20260709 13:20:37.095892 19914 leader_election.cc:343] T 86964b8a61ce494f9096c4fefdc6efbf P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:37.073980 20592 raft_consensus.cc:515] T e804b7d8fd7e44f59516fb299bea7a2c P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:37.075898 20370 leader_election.cc:290] T 86964b8a61ce494f9096c4fefdc6efbf P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 2 election: Requested vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:37.097213 19765 leader_election.cc:304] T 59fc86c7e9f648e3b3f51e936e2bcfec P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, e90ffed3ba174ababeb21d9415d4d918; no voters: 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:37.098167 20592 leader_election.cc:290] T e804b7d8fd7e44f59516fb299bea7a2c P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 election: Requested vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:37.095965 19871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86964b8a61ce494f9096c4fefdc6efbf" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
I20260709 13:20:37.098997 20592 raft_consensus.cc:2804] T 7e3a9d11df99411d8f0edb0df6f14d87 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:20:37.099390 20592 raft_consensus.cc:493] T 7e3a9d11df99411d8f0edb0df6f14d87 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:37.099426 19871 raft_consensus.cc:3060] T 86964b8a61ce494f9096c4fefdc6efbf P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:37.099929 20592 raft_consensus.cc:3060] T 7e3a9d11df99411d8f0edb0df6f14d87 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:37.098172 20591 raft_consensus.cc:2804] T dc5c36b41652443a8130592df8a66eb9 P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:20:37.102977 20478 raft_consensus.cc:2804] T 59fc86c7e9f648e3b3f51e936e2bcfec P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:20:37.103183 20540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a411802fdaa42e7863236f0af595fe2" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
I20260709 13:20:37.103571 20478 raft_consensus.cc:697] T 59fc86c7e9f648e3b3f51e936e2bcfec P e90ffed3ba174ababeb21d9415d4d918 [term 2 LEADER]: Becoming Leader. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Running, Role: LEADER
I20260709 13:20:37.103935 20540 raft_consensus.cc:3060] T 2a411802fdaa42e7863236f0af595fe2 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:37.104502 20478 consensus_queue.cc:237] T 59fc86c7e9f648e3b3f51e936e2bcfec P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:37.103207 20591 raft_consensus.cc:697] T dc5c36b41652443a8130592df8a66eb9 P e90ffed3ba174ababeb21d9415d4d918 [term 2 LEADER]: Becoming Leader. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Running, Role: LEADER
I20260709 13:20:37.106220 20591 consensus_queue.cc:237] T dc5c36b41652443a8130592df8a66eb9 P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:37.107203 19873 raft_consensus.cc:2393] T e804b7d8fd7e44f59516fb299bea7a2c P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c62716bf66c24330860573535b8b9430 in current term 2: Already voted for candidate b80b3a62d7d549a9a5c42def46769ac4 in this term.
I20260709 13:20:37.111606 20544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e804b7d8fd7e44f59516fb299bea7a2c" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
W20260709 13:20:37.113312 19840 leader_election.cc:343] T e804b7d8fd7e44f59516fb299bea7a2c P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:37.093537 19622 catalog_manager.cc:5697] T d5f8619b1f9f46adaa41774af856d9b1 P e90ffed3ba174ababeb21d9415d4d918 reported cstate change: term changed from 1 to 2, leader changed from 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129) to e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130). New cstate: current_term: 2 leader_uuid: "e90ffed3ba174ababeb21d9415d4d918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:37.095928 19839 leader_election.cc:304] T 9b2e4bf3fb5c456dab400b096f7bd729 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:37.118196 20584 raft_consensus.cc:2804] T 9b2e4bf3fb5c456dab400b096f7bd729 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:20:37.118568 20584 raft_consensus.cc:493] T 9b2e4bf3fb5c456dab400b096f7bd729 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:37.118942 20584 raft_consensus.cc:3060] T 9b2e4bf3fb5c456dab400b096f7bd729 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:37.119177 20540 raft_consensus.cc:2468] T 2a411802fdaa42e7863236f0af595fe2 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 2.
I20260709 13:20:37.120404 19840 leader_election.cc:304] T 2a411802fdaa42e7863236f0af595fe2 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4; no voters: 
I20260709 13:20:37.121315 20500 raft_consensus.cc:2804] T 2a411802fdaa42e7863236f0af595fe2 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:20:37.121958 20500 raft_consensus.cc:697] T 2a411802fdaa42e7863236f0af595fe2 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 LEADER]: Becoming Leader. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Running, Role: LEADER
I20260709 13:20:37.124413 19948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e804b7d8fd7e44f59516fb299bea7a2c" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:37.124995 19948 raft_consensus.cc:3060] T e804b7d8fd7e44f59516fb299bea7a2c P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:37.125015 19913 leader_election.cc:304] T d5f8619b1f9f46adaa41774af856d9b1 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:37.126040 20501 raft_consensus.cc:3060] T d5f8619b1f9f46adaa41774af856d9b1 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:37.128005 19871 raft_consensus.cc:2468] T 86964b8a61ce494f9096c4fefdc6efbf P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c62716bf66c24330860573535b8b9430 in term 2.
I20260709 13:20:37.123040 20500 consensus_queue.cc:237] T 2a411802fdaa42e7863236f0af595fe2 P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:37.134660 20498 tablet_bootstrap.cc:492] T 7e3099f1888c4ed18c30523bf9ec94f5 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:37.135648 20498 tablet_bootstrap.cc:492] T 7e3099f1888c4ed18c30523bf9ec94f5 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap complete.
I20260709 13:20:37.131693 19913 leader_election.cc:304] T 63868398f3604f529519d926d2831f4e P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c62716bf66c24330860573535b8b9430, e90ffed3ba174ababeb21d9415d4d918; no voters: 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:37.131906 20499 leader_election.cc:290] T 2a411802fdaa42e7863236f0af595fe2 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 election: Requested vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:37.137228 19913 leader_election.cc:304] T bd80f9f8533b4cd3b5861ec11917a03d P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c62716bf66c24330860573535b8b9430, e90ffed3ba174ababeb21d9415d4d918; no voters: 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:37.137360 19948 raft_consensus.cc:2468] T e804b7d8fd7e44f59516fb299bea7a2c P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 2.
I20260709 13:20:37.139042 20370 raft_consensus.cc:2804] T 63868398f3604f529519d926d2831f4e P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:20:37.139544 20370 raft_consensus.cc:493] T 63868398f3604f529519d926d2831f4e P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:37.139671 20424 raft_consensus.cc:2804] T bd80f9f8533b4cd3b5861ec11917a03d P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:20:37.140053 20370 raft_consensus.cc:3060] T 63868398f3604f529519d926d2831f4e P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:37.140123 20424 raft_consensus.cc:493] T bd80f9f8533b4cd3b5861ec11917a03d P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:37.140893 20424 raft_consensus.cc:3060] T bd80f9f8533b4cd3b5861ec11917a03d P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:37.141184 20592 raft_consensus.cc:515] T 7e3a9d11df99411d8f0edb0df6f14d87 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:37.150668 19913 leader_election.cc:304] T e804b7d8fd7e44f59516fb299bea7a2c P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4
I20260709 13:20:37.154191 20351 raft_consensus.cc:2749] T 563940ad96444ce59e393ea303cc3352 P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:20:37.133522 20501 raft_consensus.cc:2749] T d5f8619b1f9f46adaa41774af856d9b1 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:20:37.155350 20501 raft_consensus.cc:2749] T e804b7d8fd7e44f59516fb299bea7a2c P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:20:37.157809 19795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a411802fdaa42e7863236f0af595fe2" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
I20260709 13:20:37.158710 19795 raft_consensus.cc:3060] T 2a411802fdaa42e7863236f0af595fe2 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:37.160676 20370 raft_consensus.cc:515] T 63868398f3604f529519d926d2831f4e P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:37.161013 20424 raft_consensus.cc:515] T bd80f9f8533b4cd3b5861ec11917a03d P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:37.177826 20424 leader_election.cc:290] T bd80f9f8533b4cd3b5861ec11917a03d P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 2 election: Requested vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:37.153929 20584 raft_consensus.cc:515] T 9b2e4bf3fb5c456dab400b096f7bd729 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:37.136318 20498 ts_tablet_manager.cc:1403] T 7e3099f1888c4ed18c30523bf9ec94f5 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.120s	user 0.025s	sys 0.021s
I20260709 13:20:37.163708 20540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63868398f3604f529519d926d2831f4e" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
I20260709 13:20:37.182431 20584 leader_election.cc:290] T 9b2e4bf3fb5c456dab400b096f7bd729 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 election: Requested vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
W20260709 13:20:37.183503 19914 leader_election.cc:343] T 63868398f3604f529519d926d2831f4e P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:37.164635 19913 leader_election.cc:304] T 86964b8a61ce494f9096c4fefdc6efbf P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:37.184615 20424 raft_consensus.cc:2804] T 86964b8a61ce494f9096c4fefdc6efbf P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:20:37.185122 20424 raft_consensus.cc:697] T 86964b8a61ce494f9096c4fefdc6efbf P c62716bf66c24330860573535b8b9430 [term 2 LEADER]: Becoming Leader. State: Replica: c62716bf66c24330860573535b8b9430, State: Running, Role: LEADER
I20260709 13:20:37.185892 20424 consensus_queue.cc:237] T 86964b8a61ce494f9096c4fefdc6efbf P c62716bf66c24330860573535b8b9430 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:37.166739 19839 leader_election.cc:304] T e804b7d8fd7e44f59516fb299bea7a2c P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:37.188617 20584 raft_consensus.cc:2804] T e804b7d8fd7e44f59516fb299bea7a2c P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:20:37.189756 20584 raft_consensus.cc:697] T e804b7d8fd7e44f59516fb299bea7a2c P b80b3a62d7d549a9a5c42def46769ac4 [term 2 LEADER]: Becoming Leader. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Running, Role: LEADER
I20260709 13:20:37.191860 19799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3a9d11df99411d8f0edb0df6f14d87" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
I20260709 13:20:37.162721 19765 leader_election.cc:304] T 6551866a22dd4aaf8ee318b9d91b3a02 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c62716bf66c24330860573535b8b9430, e90ffed3ba174ababeb21d9415d4d918; no voters: 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:37.192519 19799 raft_consensus.cc:3060] T 7e3a9d11df99411d8f0edb0df6f14d87 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:37.192236 20498 raft_consensus.cc:359] T 7e3099f1888c4ed18c30523bf9ec94f5 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:37.193001 20498 raft_consensus.cc:740] T 7e3099f1888c4ed18c30523bf9ec94f5 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:37.193027 20591 raft_consensus.cc:2804] T 6551866a22dd4aaf8ee318b9d91b3a02 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:20:37.193562 20591 raft_consensus.cc:493] T 6551866a22dd4aaf8ee318b9d91b3a02 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:37.194522 19800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63868398f3604f529519d926d2831f4e" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
I20260709 13:20:37.194748 20540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b2e4bf3fb5c456dab400b096f7bd729" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
I20260709 13:20:37.195235 19800 raft_consensus.cc:3060] T 63868398f3604f529519d926d2831f4e P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:37.178300 20541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3a9d11df99411d8f0edb0df6f14d87" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
W20260709 13:20:37.196103 19840 leader_election.cc:343] T 9b2e4bf3fb5c456dab400b096f7bd729 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:37.193895 20498 consensus_queue.cc:260] T 7e3099f1888c4ed18c30523bf9ec94f5 P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
W20260709 13:20:37.197286 19840 leader_election.cc:343] T 7e3a9d11df99411d8f0edb0df6f14d87 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:37.193917 20591 raft_consensus.cc:3060] T 6551866a22dd4aaf8ee318b9d91b3a02 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:37.179327 20592 leader_election.cc:290] T 7e3a9d11df99411d8f0edb0df6f14d87 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 election: Requested vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:37.191622 20584 consensus_queue.cc:237] T e804b7d8fd7e44f59516fb299bea7a2c P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:37.195091 19794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd80f9f8533b4cd3b5861ec11917a03d" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
I20260709 13:20:37.201088 19794 raft_consensus.cc:3060] T bd80f9f8533b4cd3b5861ec11917a03d P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:37.200412 20498 ts_tablet_manager.cc:1434] T 7e3099f1888c4ed18c30523bf9ec94f5 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.002s
I20260709 13:20:37.180328 19795 raft_consensus.cc:2468] T 2a411802fdaa42e7863236f0af595fe2 P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 2.
I20260709 13:20:37.178947 20544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd80f9f8533b4cd3b5861ec11917a03d" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
W20260709 13:20:37.209266 19914 leader_election.cc:343] T bd80f9f8533b4cd3b5861ec11917a03d P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:37.210713 19794 raft_consensus.cc:2468] T bd80f9f8533b4cd3b5861ec11917a03d P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c62716bf66c24330860573535b8b9430 in term 2.
I20260709 13:20:37.211272 19800 raft_consensus.cc:2468] T 63868398f3604f529519d926d2831f4e P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c62716bf66c24330860573535b8b9430 in term 2.
I20260709 13:20:37.167830 20370 leader_election.cc:290] T 63868398f3604f529519d926d2831f4e P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 2 election: Requested vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:37.213788 20498 tablet_bootstrap.cc:492] T 7e3a9d11df99411d8f0edb0df6f14d87 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:37.214354 19799 raft_consensus.cc:2468] T 7e3a9d11df99411d8f0edb0df6f14d87 P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 2.
I20260709 13:20:37.216295 19839 leader_election.cc:304] T 7e3a9d11df99411d8f0edb0df6f14d87 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, e90ffed3ba174ababeb21d9415d4d918; no voters: 8df38c4c53c44e94b36c1e8992476d7c
W20260709 13:20:37.217103 19914 consensus_peers.cc:597] T a323eb80c5084159803e7d4ea5d9c410 P c62716bf66c24330860573535b8b9430 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:37.219746 19913 leader_election.cc:304] T bd80f9f8533b4cd3b5861ec11917a03d P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c62716bf66c24330860573535b8b9430, e90ffed3ba174ababeb21d9415d4d918; no voters: 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:37.220969 20600 raft_consensus.cc:2804] T bd80f9f8533b4cd3b5861ec11917a03d P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:20:37.221551 20600 raft_consensus.cc:697] T bd80f9f8533b4cd3b5861ec11917a03d P c62716bf66c24330860573535b8b9430 [term 2 LEADER]: Becoming Leader. State: Replica: c62716bf66c24330860573535b8b9430, State: Running, Role: LEADER
I20260709 13:20:37.223155 20600 consensus_queue.cc:237] T bd80f9f8533b4cd3b5861ec11917a03d P c62716bf66c24330860573535b8b9430 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:37.224727 19948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63868398f3604f529519d926d2831f4e" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:37.225600 19948 raft_consensus.cc:2393] T 63868398f3604f529519d926d2831f4e P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e90ffed3ba174ababeb21d9415d4d918 in current term 2: Already voted for candidate c62716bf66c24330860573535b8b9430 in this term.
I20260709 13:20:37.227656 19913 leader_election.cc:304] T 63868398f3604f529519d926d2831f4e P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c62716bf66c24330860573535b8b9430, e90ffed3ba174ababeb21d9415d4d918; no voters: 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:37.228593 20370 raft_consensus.cc:2804] T 63868398f3604f529519d926d2831f4e P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:20:37.229102 20370 raft_consensus.cc:697] T 63868398f3604f529519d926d2831f4e P c62716bf66c24330860573535b8b9430 [term 2 LEADER]: Becoming Leader. State: Replica: c62716bf66c24330860573535b8b9430, State: Running, Role: LEADER
I20260709 13:20:37.230201 19765 leader_election.cc:304] T 63868398f3604f529519d926d2831f4e P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e90ffed3ba174ababeb21d9415d4d918; no voters: 8df38c4c53c44e94b36c1e8992476d7c, c62716bf66c24330860573535b8b9430
I20260709 13:20:37.230129 20370 consensus_queue.cc:237] T 63868398f3604f529519d926d2831f4e P c62716bf66c24330860573535b8b9430 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:37.230967 20499 raft_consensus.cc:2804] T 7e3a9d11df99411d8f0edb0df6f14d87 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:20:37.231653 20499 raft_consensus.cc:697] T 7e3a9d11df99411d8f0edb0df6f14d87 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 LEADER]: Becoming Leader. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Running, Role: LEADER
I20260709 13:20:37.232373 20587 raft_consensus.cc:2749] T 63868398f3604f529519d926d2831f4e P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:20:37.231981 19623 catalog_manager.cc:5697] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4 reported cstate change: term changed from 1 to 2, leader changed from 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129) to b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131). New cstate: current_term: 2 leader_uuid: "b80b3a62d7d549a9a5c42def46769ac4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:37.232522 20499 consensus_queue.cc:237] T 7e3a9d11df99411d8f0edb0df6f14d87 P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:37.233660 19623 catalog_manager.cc:5697] T 2a411802fdaa42e7863236f0af595fe2 P b80b3a62d7d549a9a5c42def46769ac4 reported cstate change: term changed from 1 to 2, leader changed from 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129) to b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131). New cstate: current_term: 2 leader_uuid: "b80b3a62d7d549a9a5c42def46769ac4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:37.250757 19944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b2e4bf3fb5c456dab400b096f7bd729" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:37.251854 19944 raft_consensus.cc:3060] T 9b2e4bf3fb5c456dab400b096f7bd729 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:37.255297 20591 raft_consensus.cc:515] T 6551866a22dd4aaf8ee318b9d91b3a02 P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:37.258594 20540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6551866a22dd4aaf8ee318b9d91b3a02" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
W20260709 13:20:37.260085 19766 leader_election.cc:343] T 6551866a22dd4aaf8ee318b9d91b3a02 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:37.261161 19944 raft_consensus.cc:2468] T 9b2e4bf3fb5c456dab400b096f7bd729 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 2.
I20260709 13:20:37.262311 19948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6551866a22dd4aaf8ee318b9d91b3a02" candidate_uuid: "e90ffed3ba174ababeb21d9415d4d918" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:37.262951 19948 raft_consensus.cc:3060] T 6551866a22dd4aaf8ee318b9d91b3a02 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:37.267899 19622 catalog_manager.cc:5697] T cb33acfa7a9f46ab95b0e77e1394f258 P e90ffed3ba174ababeb21d9415d4d918 reported cstate change: term changed from 1 to 2, leader changed from 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129) to e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130). New cstate: current_term: 2 leader_uuid: "e90ffed3ba174ababeb21d9415d4d918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:37.270429 19948 raft_consensus.cc:2468] T 6551866a22dd4aaf8ee318b9d91b3a02 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e90ffed3ba174ababeb21d9415d4d918 in term 2.
I20260709 13:20:37.270043 19622 catalog_manager.cc:5697] T dc5c36b41652443a8130592df8a66eb9 P e90ffed3ba174ababeb21d9415d4d918 reported cstate change: term changed from 1 to 2, leader changed from 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129) to e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130). New cstate: current_term: 2 leader_uuid: "e90ffed3ba174ababeb21d9415d4d918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:37.270977 20498 tablet_bootstrap.cc:492] T 7e3a9d11df99411d8f0edb0df6f14d87 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:37.271564 19765 leader_election.cc:304] T 6551866a22dd4aaf8ee318b9d91b3a02 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c62716bf66c24330860573535b8b9430, e90ffed3ba174ababeb21d9415d4d918; no voters: 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:37.272120 20498 tablet_bootstrap.cc:492] T 7e3a9d11df99411d8f0edb0df6f14d87 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap complete.
I20260709 13:20:37.271840 19622 catalog_manager.cc:5697] T 59fc86c7e9f648e3b3f51e936e2bcfec P e90ffed3ba174ababeb21d9415d4d918 reported cstate change: term changed from 1 to 2, leader changed from 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129) to e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130). New cstate: current_term: 2 leader_uuid: "e90ffed3ba174ababeb21d9415d4d918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:37.272519 20351 raft_consensus.cc:2804] T 6551866a22dd4aaf8ee318b9d91b3a02 P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:20:37.272855 20498 ts_tablet_manager.cc:1403] T 7e3a9d11df99411d8f0edb0df6f14d87 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.068s	user 0.021s	sys 0.007s
I20260709 13:20:37.273063 20351 raft_consensus.cc:697] T 6551866a22dd4aaf8ee318b9d91b3a02 P e90ffed3ba174ababeb21d9415d4d918 [term 2 LEADER]: Becoming Leader. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Running, Role: LEADER
I20260709 13:20:37.273854 20351 consensus_queue.cc:237] T 6551866a22dd4aaf8ee318b9d91b3a02 P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:37.276937 20498 raft_consensus.cc:359] T 7e3a9d11df99411d8f0edb0df6f14d87 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:37.277631 20498 raft_consensus.cc:740] T 7e3a9d11df99411d8f0edb0df6f14d87 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:37.277833 20591 leader_election.cc:290] T 6551866a22dd4aaf8ee318b9d91b3a02 P e90ffed3ba174ababeb21d9415d4d918 [CANDIDATE]: Term 2 election: Requested vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:37.278575 20498 consensus_queue.cc:260] T 7e3a9d11df99411d8f0edb0df6f14d87 P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:37.281230 19839 leader_election.cc:304] T 9b2e4bf3fb5c456dab400b096f7bd729 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:37.285521 20499 raft_consensus.cc:2804] T 9b2e4bf3fb5c456dab400b096f7bd729 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:20:37.289113 19794 raft_consensus.cc:1275] T a323eb80c5084159803e7d4ea5d9c410 P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Refusing update from remote peer c62716bf66c24330860573535b8b9430: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:37.291342 20499 raft_consensus.cc:697] T 9b2e4bf3fb5c456dab400b096f7bd729 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 LEADER]: Becoming Leader. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Running, Role: LEADER
I20260709 13:20:37.292585 20499 consensus_queue.cc:237] T 9b2e4bf3fb5c456dab400b096f7bd729 P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:37.295832 20370 consensus_queue.cc:1048] T a323eb80c5084159803e7d4ea5d9c410 P c62716bf66c24330860573535b8b9430 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:20:37.303118 20498 ts_tablet_manager.cc:1434] T 7e3a9d11df99411d8f0edb0df6f14d87 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.030s	user 0.008s	sys 0.000s
I20260709 13:20:37.302188 19623 catalog_manager.cc:5697] T 7e3a9d11df99411d8f0edb0df6f14d87 P b80b3a62d7d549a9a5c42def46769ac4 reported cstate change: term changed from 1 to 2, leader changed from 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129) to b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131). New cstate: current_term: 2 leader_uuid: "b80b3a62d7d549a9a5c42def46769ac4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:37.305698 19621 catalog_manager.cc:5697] T 86964b8a61ce494f9096c4fefdc6efbf P c62716bf66c24330860573535b8b9430 reported cstate change: term changed from 1 to 2, leader changed from 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129) to c62716bf66c24330860573535b8b9430 (127.9.116.132). New cstate: current_term: 2 leader_uuid: "c62716bf66c24330860573535b8b9430" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:37.306977 19621 catalog_manager.cc:5697] T bd80f9f8533b4cd3b5861ec11917a03d P c62716bf66c24330860573535b8b9430 reported cstate change: term changed from 1 to 2, leader changed from 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129) to c62716bf66c24330860573535b8b9430 (127.9.116.132). New cstate: current_term: 2 leader_uuid: "c62716bf66c24330860573535b8b9430" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:37.308175 19621 catalog_manager.cc:5697] T 63868398f3604f529519d926d2831f4e P c62716bf66c24330860573535b8b9430 reported cstate change: term changed from 1 to 2, leader changed from 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129) to c62716bf66c24330860573535b8b9430 (127.9.116.132). New cstate: current_term: 2 leader_uuid: "c62716bf66c24330860573535b8b9430" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:37.305123 19623 catalog_manager.cc:5697] T e804b7d8fd7e44f59516fb299bea7a2c P b80b3a62d7d549a9a5c42def46769ac4 reported cstate change: term changed from 1 to 2, leader changed from 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129) to b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131). New cstate: current_term: 2 leader_uuid: "b80b3a62d7d549a9a5c42def46769ac4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:37.304040 20498 tablet_bootstrap.cc:492] T a323eb80c5084159803e7d4ea5d9c410 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:37.337481 19622 catalog_manager.cc:5697] T 9b2e4bf3fb5c456dab400b096f7bd729 P b80b3a62d7d549a9a5c42def46769ac4 reported cstate change: term changed from 1 to 2, leader changed from 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129) to b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131). New cstate: current_term: 2 leader_uuid: "b80b3a62d7d549a9a5c42def46769ac4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:37.346453 19619 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53944
I20260709 13:20:37.355255 20577 heartbeater.cc:499] Master 127.9.116.190:44649 was elected leader, sending a full tablet report...
I20260709 13:20:37.359702 20498 tablet_bootstrap.cc:492] T a323eb80c5084159803e7d4ea5d9c410 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:37.359545 19619 catalog_manager.cc:5697] T 6551866a22dd4aaf8ee318b9d91b3a02 P e90ffed3ba174ababeb21d9415d4d918 reported cstate change: term changed from 1 to 2, leader changed from 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129) to e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130). New cstate: current_term: 2 leader_uuid: "e90ffed3ba174ababeb21d9415d4d918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:37.361056 20498 tablet_bootstrap.cc:492] T a323eb80c5084159803e7d4ea5d9c410 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap complete.
I20260709 13:20:37.361984 20498 ts_tablet_manager.cc:1403] T a323eb80c5084159803e7d4ea5d9c410 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.058s	user 0.021s	sys 0.002s
I20260709 13:20:37.365517 20498 raft_consensus.cc:359] T a323eb80c5084159803e7d4ea5d9c410 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:37.368005 20498 raft_consensus.cc:740] T a323eb80c5084159803e7d4ea5d9c410 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:37.368813 20498 consensus_queue.cc:260] T a323eb80c5084159803e7d4ea5d9c410 P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:37.370921 20498 ts_tablet_manager.cc:1434] T a323eb80c5084159803e7d4ea5d9c410 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:20:37.371774 20498 tablet_bootstrap.cc:492] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:37.384022 20540 raft_consensus.cc:1275] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Refusing update from remote peer e90ffed3ba174ababeb21d9415d4d918: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:37.385154 20498 tablet_bootstrap.cc:492] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:37.386111 20498 tablet_bootstrap.cc:492] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap complete.
I20260709 13:20:37.386718 20498 ts_tablet_manager.cc:1403] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.015s	user 0.007s	sys 0.004s
I20260709 13:20:37.389256 20498 raft_consensus.cc:359] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:37.389955 20498 raft_consensus.cc:740] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:37.390692 20498 consensus_queue.cc:260] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:37.391335 20591 consensus_queue.cc:1048] T d5f8619b1f9f46adaa41774af856d9b1 P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:20:37.393136 20498 ts_tablet_manager.cc:1434] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:20:37.393998 20498 tablet_bootstrap.cc:492] T 1cceb7f75d7040c2b98049590928b891 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:37.397311 19794 raft_consensus.cc:1275] T 563940ad96444ce59e393ea303cc3352 P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Refusing update from remote peer b80b3a62d7d549a9a5c42def46769ac4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:37.399266 20499 consensus_queue.cc:1048] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:20:37.419246 19948 raft_consensus.cc:1275] T d5f8619b1f9f46adaa41774af856d9b1 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Refusing update from remote peer e90ffed3ba174ababeb21d9415d4d918: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:37.421588 20587 consensus_queue.cc:1048] T d5f8619b1f9f46adaa41774af856d9b1 P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:20:37.475289 20498 tablet_bootstrap.cc:492] T 1cceb7f75d7040c2b98049590928b891 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:37.476354 20498 tablet_bootstrap.cc:492] T 1cceb7f75d7040c2b98049590928b891 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap complete.
I20260709 13:20:37.477077 20498 ts_tablet_manager.cc:1403] T 1cceb7f75d7040c2b98049590928b891 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.083s	user 0.026s	sys 0.021s
I20260709 13:20:37.477702 20545 raft_consensus.cc:1275] T 563940ad96444ce59e393ea303cc3352 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Refusing update from remote peer b80b3a62d7d549a9a5c42def46769ac4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:37.479599 20498 raft_consensus.cc:359] T 1cceb7f75d7040c2b98049590928b891 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:37.480487 20498 raft_consensus.cc:740] T 1cceb7f75d7040c2b98049590928b891 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:37.481364 20498 consensus_queue.cc:260] T 1cceb7f75d7040c2b98049590928b891 P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:37.482148 20541 raft_consensus.cc:1275] T c393f9d18341477c97fac9a4af7e3adf P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Refusing update from remote peer b80b3a62d7d549a9a5c42def46769ac4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:37.483781 20498 ts_tablet_manager.cc:1434] T 1cceb7f75d7040c2b98049590928b891 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:20:37.484764 20498 tablet_bootstrap.cc:492] T e3ca37daad8341ad99cbba926f37232e P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:37.485029 19794 raft_consensus.cc:1275] T c393f9d18341477c97fac9a4af7e3adf P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Refusing update from remote peer b80b3a62d7d549a9a5c42def46769ac4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:37.487310 20584 consensus_queue.cc:1048] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:20:37.487403 20499 consensus_queue.cc:1048] T c393f9d18341477c97fac9a4af7e3adf P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:20:37.490774 20584 consensus_queue.cc:1048] T c393f9d18341477c97fac9a4af7e3adf P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:20:37.505831 20498 tablet_bootstrap.cc:492] T e3ca37daad8341ad99cbba926f37232e P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:37.507021 20498 tablet_bootstrap.cc:492] T e3ca37daad8341ad99cbba926f37232e P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap complete.
I20260709 13:20:37.507741 20498 ts_tablet_manager.cc:1403] T e3ca37daad8341ad99cbba926f37232e P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.023s	user 0.019s	sys 0.000s
I20260709 13:20:37.510234 20498 raft_consensus.cc:359] T e3ca37daad8341ad99cbba926f37232e P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:37.510990 20498 raft_consensus.cc:740] T e3ca37daad8341ad99cbba926f37232e P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:37.511817 20498 consensus_queue.cc:260] T e3ca37daad8341ad99cbba926f37232e P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:37.514053 20498 ts_tablet_manager.cc:1434] T e3ca37daad8341ad99cbba926f37232e P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:20:37.514938 20498 tablet_bootstrap.cc:492] T bd80f9f8533b4cd3b5861ec11917a03d P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
W20260709 13:20:37.516175 19766 consensus_peers.cc:597] T cb33acfa7a9f46ab95b0e77e1394f258 P e90ffed3ba174ababeb21d9415d4d918 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:37.545845 20498 tablet_bootstrap.cc:492] T bd80f9f8533b4cd3b5861ec11917a03d P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:37.546952 20498 tablet_bootstrap.cc:492] T bd80f9f8533b4cd3b5861ec11917a03d P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap complete.
I20260709 13:20:37.547678 20498 ts_tablet_manager.cc:1403] T bd80f9f8533b4cd3b5861ec11917a03d P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.033s	user 0.015s	sys 0.006s
I20260709 13:20:37.550343 20498 raft_consensus.cc:359] T bd80f9f8533b4cd3b5861ec11917a03d P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:37.551082 20498 raft_consensus.cc:740] T bd80f9f8533b4cd3b5861ec11917a03d P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:37.551884 20498 consensus_queue.cc:260] T bd80f9f8533b4cd3b5861ec11917a03d P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:37.564504 20498 ts_tablet_manager.cc:1434] T bd80f9f8533b4cd3b5861ec11917a03d P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.000s
I20260709 13:20:37.565419 20498 tablet_bootstrap.cc:492] T 0a9c6b64c8e04c4283d24ae6290a52ea P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:37.573065 19871 raft_consensus.cc:1275] T dc5c36b41652443a8130592df8a66eb9 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Refusing update from remote peer e90ffed3ba174ababeb21d9415d4d918: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:37.574755 20587 consensus_queue.cc:1048] T dc5c36b41652443a8130592df8a66eb9 P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:20:37.582223 19766 consensus_peers.cc:597] T d762eace063c4e04821c749d21702c3f P e90ffed3ba174ababeb21d9415d4d918 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:37.589681 19766 consensus_peers.cc:597] T dc5c36b41652443a8130592df8a66eb9 P e90ffed3ba174ababeb21d9415d4d918 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:37.593806 20498 tablet_bootstrap.cc:492] T 0a9c6b64c8e04c4283d24ae6290a52ea P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:37.594640 20498 tablet_bootstrap.cc:492] T 0a9c6b64c8e04c4283d24ae6290a52ea P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap complete.
I20260709 13:20:37.595261 20498 ts_tablet_manager.cc:1403] T 0a9c6b64c8e04c4283d24ae6290a52ea P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.030s	user 0.017s	sys 0.000s
I20260709 13:20:37.598134 20498 raft_consensus.cc:359] T 0a9c6b64c8e04c4283d24ae6290a52ea P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:37.598942 20498 raft_consensus.cc:740] T 0a9c6b64c8e04c4283d24ae6290a52ea P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:37.599756 20498 consensus_queue.cc:260] T 0a9c6b64c8e04c4283d24ae6290a52ea P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:37.609977 20498 ts_tablet_manager.cc:1434] T 0a9c6b64c8e04c4283d24ae6290a52ea P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:20:37.610877 20498 tablet_bootstrap.cc:492] T d762eace063c4e04821c749d21702c3f P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:37.630745 20544 raft_consensus.cc:3060] T bd80f9f8533b4cd3b5861ec11917a03d P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:37.638720 20544 raft_consensus.cc:1275] T bd80f9f8533b4cd3b5861ec11917a03d P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Refusing update from remote peer c62716bf66c24330860573535b8b9430: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:37.641516 20370 consensus_queue.cc:1048] T bd80f9f8533b4cd3b5861ec11917a03d P c62716bf66c24330860573535b8b9430 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:20:37.643041 20498 tablet_bootstrap.cc:492] T d762eace063c4e04821c749d21702c3f P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:37.644078 20498 tablet_bootstrap.cc:492] T d762eace063c4e04821c749d21702c3f P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap complete.
I20260709 13:20:37.644814 20498 ts_tablet_manager.cc:1403] T d762eace063c4e04821c749d21702c3f P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.034s	user 0.019s	sys 0.008s
I20260709 13:20:37.646930 19794 raft_consensus.cc:1275] T 2a411802fdaa42e7863236f0af595fe2 P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Refusing update from remote peer b80b3a62d7d549a9a5c42def46769ac4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:37.647496 20498 raft_consensus.cc:359] T d762eace063c4e04821c749d21702c3f P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:37.648300 20498 raft_consensus.cc:740] T d762eace063c4e04821c749d21702c3f P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:37.649026 20498 consensus_queue.cc:260] T d762eace063c4e04821c749d21702c3f P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:37.651325 20498 ts_tablet_manager.cc:1434] T d762eace063c4e04821c749d21702c3f P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:20:37.651533 19871 raft_consensus.cc:1275] T cb33acfa7a9f46ab95b0e77e1394f258 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Refusing update from remote peer e90ffed3ba174ababeb21d9415d4d918: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:37.652092 20498 tablet_bootstrap.cc:492] T 6551866a22dd4aaf8ee318b9d91b3a02 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:37.653682 20212 consensus_queue.cc:1048] T cb33acfa7a9f46ab95b0e77e1394f258 P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:20:37.662242 19871 raft_consensus.cc:1275] T 86964b8a61ce494f9096c4fefdc6efbf P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Refusing update from remote peer c62716bf66c24330860573535b8b9430: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:37.663388 20584 consensus_queue.cc:1048] T 2a411802fdaa42e7863236f0af595fe2 P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:20:37.664059 19794 raft_consensus.cc:1275] T 7e3a9d11df99411d8f0edb0df6f14d87 P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Refusing update from remote peer b80b3a62d7d549a9a5c42def46769ac4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:37.667249 20541 raft_consensus.cc:1275] T 2a411802fdaa42e7863236f0af595fe2 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Refusing update from remote peer b80b3a62d7d549a9a5c42def46769ac4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:20:37.668279 19766 consensus_peers.cc:597] T 59fc86c7e9f648e3b3f51e936e2bcfec P e90ffed3ba174ababeb21d9415d4d918 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:37.669268 20584 consensus_queue.cc:1048] T 2a411802fdaa42e7863236f0af595fe2 P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:20:37.675010 20584 consensus_queue.cc:1048] T 7e3a9d11df99411d8f0edb0df6f14d87 P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:20:37.689314 20498 tablet_bootstrap.cc:492] T 6551866a22dd4aaf8ee318b9d91b3a02 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:37.690392 20498 tablet_bootstrap.cc:492] T 6551866a22dd4aaf8ee318b9d91b3a02 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap complete.
I20260709 13:20:37.691135 20498 ts_tablet_manager.cc:1403] T 6551866a22dd4aaf8ee318b9d91b3a02 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.039s	user 0.031s	sys 0.003s
I20260709 13:20:37.693645 20498 raft_consensus.cc:359] T 6551866a22dd4aaf8ee318b9d91b3a02 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:37.694391 20498 raft_consensus.cc:740] T 6551866a22dd4aaf8ee318b9d91b3a02 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:37.695554 20498 consensus_queue.cc:260] T 6551866a22dd4aaf8ee318b9d91b3a02 P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:37.697715 20498 ts_tablet_manager.cc:1434] T 6551866a22dd4aaf8ee318b9d91b3a02 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:20:37.698561 20498 tablet_bootstrap.cc:492] T 0587017774214f24bdfa4a3da36e367e P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:37.705462 19800 raft_consensus.cc:1275] T 63868398f3604f529519d926d2831f4e P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Refusing update from remote peer c62716bf66c24330860573535b8b9430: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:37.710817 20424 consensus_queue.cc:1048] T 86964b8a61ce494f9096c4fefdc6efbf P c62716bf66c24330860573535b8b9430 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:20:37.716845 19914 consensus_peers.cc:597] T 86964b8a61ce494f9096c4fefdc6efbf P c62716bf66c24330860573535b8b9430 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:37.722317 19941 raft_consensus.cc:1275] T 9b2e4bf3fb5c456dab400b096f7bd729 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Refusing update from remote peer b80b3a62d7d549a9a5c42def46769ac4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:20:37.736613 19840 consensus_peers.cc:597] T e804b7d8fd7e44f59516fb299bea7a2c P b80b3a62d7d549a9a5c42def46769ac4 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:37.738018 20541 raft_consensus.cc:3060] T 7e3a9d11df99411d8f0edb0df6f14d87 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:37.750147 20584 consensus_queue.cc:1048] T 9b2e4bf3fb5c456dab400b096f7bd729 P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:20:37.753512 19941 raft_consensus.cc:1275] T e804b7d8fd7e44f59516fb299bea7a2c P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Refusing update from remote peer b80b3a62d7d549a9a5c42def46769ac4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:37.755802 20424 consensus_queue.cc:1048] T 63868398f3604f529519d926d2831f4e P c62716bf66c24330860573535b8b9430 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:20:37.759397 19873 raft_consensus.cc:1275] T 59fc86c7e9f648e3b3f51e936e2bcfec P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Refusing update from remote peer e90ffed3ba174ababeb21d9415d4d918: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:37.765805 20545 raft_consensus.cc:3060] T 6551866a22dd4aaf8ee318b9d91b3a02 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:20:37.786228 19914 consensus_peers.cc:597] T 63868398f3604f529519d926d2831f4e P c62716bf66c24330860573535b8b9430 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:37.789005 20541 raft_consensus.cc:1275] T 7e3a9d11df99411d8f0edb0df6f14d87 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Refusing update from remote peer b80b3a62d7d549a9a5c42def46769ac4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:37.791891 20584 consensus_queue.cc:1048] T 7e3a9d11df99411d8f0edb0df6f14d87 P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:20:37.795214 19941 raft_consensus.cc:1275] T 6551866a22dd4aaf8ee318b9d91b3a02 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Refusing update from remote peer e90ffed3ba174ababeb21d9415d4d918: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:37.802304 20500 consensus_queue.cc:1048] T e804b7d8fd7e44f59516fb299bea7a2c P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:20:37.803241 19914 consensus_peers.cc:597] T 353a546cf8c54671971c239502ef165a P c62716bf66c24330860573535b8b9430 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:37.826911 20542 raft_consensus.cc:3060] T a323eb80c5084159803e7d4ea5d9c410 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:37.835186 20591 consensus_queue.cc:1048] T 59fc86c7e9f648e3b3f51e936e2bcfec P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:20:37.842553 20545 raft_consensus.cc:1275] T 6551866a22dd4aaf8ee318b9d91b3a02 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Refusing update from remote peer e90ffed3ba174ababeb21d9415d4d918: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:37.844223 20591 consensus_queue.cc:1048] T 6551866a22dd4aaf8ee318b9d91b3a02 P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:20:37.885842 20556 consensus_queue.cc:1048] T 6551866a22dd4aaf8ee318b9d91b3a02 P e90ffed3ba174ababeb21d9415d4d918 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:20:37.903999 20545 raft_consensus.cc:3060] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:20:37.906389 19840 consensus_peers.cc:597] T 19e83b82855d4129bb8525e6ff05316b P b80b3a62d7d549a9a5c42def46769ac4 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:37.909950 19914 consensus_peers.cc:597] T 8fd58e41ecee4779b3e72df3aea4463a P c62716bf66c24330860573535b8b9430 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:37.918901 20498 tablet_bootstrap.cc:492] T 0587017774214f24bdfa4a3da36e367e P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:37.930037 19800 raft_consensus.cc:1275] T bd80f9f8533b4cd3b5861ec11917a03d P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Refusing update from remote peer c62716bf66c24330860573535b8b9430: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:37.933395 20498 tablet_bootstrap.cc:492] T 0587017774214f24bdfa4a3da36e367e P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap complete.
I20260709 13:20:37.934227 20498 ts_tablet_manager.cc:1403] T 0587017774214f24bdfa4a3da36e367e P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.236s	user 0.071s	sys 0.096s
I20260709 13:20:37.936196 20545 raft_consensus.cc:1275] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Refusing update from remote peer b80b3a62d7d549a9a5c42def46769ac4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:37.936853 20498 raft_consensus.cc:359] T 0587017774214f24bdfa4a3da36e367e P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:37.937546 20498 raft_consensus.cc:740] T 0587017774214f24bdfa4a3da36e367e P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:37.938221 20498 consensus_queue.cc:260] T 0587017774214f24bdfa4a3da36e367e P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:37.948463 20498 ts_tablet_manager.cc:1434] T 0587017774214f24bdfa4a3da36e367e P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:20:37.949335 20498 tablet_bootstrap.cc:492] T 9b9137146e3644a6b0026778530fa8a6 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:37.949117 20584 consensus_queue.cc:1048] T 9b2e4bf3fb5c456dab400b096f7bd729 P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:20:37.965570 19766 consensus_peers.cc:597] T af881b097b4245b1be6072579197cb6b P e90ffed3ba174ababeb21d9415d4d918 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:37.968120 20424 consensus_queue.cc:1048] T bd80f9f8533b4cd3b5861ec11917a03d P c62716bf66c24330860573535b8b9430 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:20:37.980309 20498 tablet_bootstrap.cc:492] T 9b9137146e3644a6b0026778530fa8a6 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:37.981272 20498 tablet_bootstrap.cc:492] T 9b9137146e3644a6b0026778530fa8a6 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap complete.
I20260709 13:20:37.981927 20498 ts_tablet_manager.cc:1403] T 9b9137146e3644a6b0026778530fa8a6 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.033s	user 0.019s	sys 0.007s
I20260709 13:20:37.996230 20498 raft_consensus.cc:359] T 9b9137146e3644a6b0026778530fa8a6 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:37.997043 20498 raft_consensus.cc:740] T 9b9137146e3644a6b0026778530fa8a6 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:38.000720 20498 consensus_queue.cc:260] T 9b9137146e3644a6b0026778530fa8a6 P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:38.006222 20498 ts_tablet_manager.cc:1434] T 9b9137146e3644a6b0026778530fa8a6 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.024s	user 0.010s	sys 0.006s
I20260709 13:20:38.007121 20498 tablet_bootstrap.cc:492] T 5b9511148896441a948b772784f62a3d P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:38.012207 20388 raft_consensus.cc:493] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:38.012781 20388 raft_consensus.cc:515] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:38.014686 20388 leader_election.cc:290] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:38.017297 19944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5476f5d68d496eaedce93e80bff5e4" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430" is_pre_election: true
I20260709 13:20:38.018558 20538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5476f5d68d496eaedce93e80bff5e4" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c" is_pre_election: true
W20260709 13:20:38.020141 19840 leader_election.cc:343] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:38.020536 19944 raft_consensus.cc:2468] T 7a5476f5d68d496eaedce93e80bff5e4 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 2.
I20260709 13:20:38.031970 20498 tablet_bootstrap.cc:492] T 5b9511148896441a948b772784f62a3d P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:38.036082 20498 tablet_bootstrap.cc:492] T 5b9511148896441a948b772784f62a3d P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap complete.
I20260709 13:20:38.036796 20498 ts_tablet_manager.cc:1403] T 5b9511148896441a948b772784f62a3d P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.030s	user 0.015s	sys 0.008s
I20260709 13:20:38.039836 20498 raft_consensus.cc:359] T 5b9511148896441a948b772784f62a3d P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:38.040632 20498 raft_consensus.cc:740] T 5b9511148896441a948b772784f62a3d P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:38.041361 20498 consensus_queue.cc:260] T 5b9511148896441a948b772784f62a3d P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:38.042373 19839 leader_election.cc:304] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:38.044765 20584 raft_consensus.cc:2804] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:20:38.045148 20584 raft_consensus.cc:493] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:38.045464 20584 raft_consensus.cc:3060] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:20:38.064181 20584 raft_consensus.cc:515] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:38.072049 20498 ts_tablet_manager.cc:1434] T 5b9511148896441a948b772784f62a3d P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.035s	user 0.015s	sys 0.014s
I20260709 13:20:38.077960 20498 tablet_bootstrap.cc:492] T 8fd58e41ecee4779b3e72df3aea4463a P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:38.080983 19948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5476f5d68d496eaedce93e80bff5e4" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:38.081684 19948 raft_consensus.cc:3060] T 7a5476f5d68d496eaedce93e80bff5e4 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:20:38.082319 20538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5476f5d68d496eaedce93e80bff5e4" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
W20260709 13:20:38.083741 19840 leader_election.cc:343] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 3 election: Tablet error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:38.090849 19948 raft_consensus.cc:2468] T 7a5476f5d68d496eaedce93e80bff5e4 P c62716bf66c24330860573535b8b9430 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 3.
I20260709 13:20:38.081046 20584 leader_election.cc:290] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 3 election: Requested vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
I20260709 13:20:38.092438 19839 leader_election.cc:304] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:38.093490 20499 raft_consensus.cc:2804] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:20:38.094198 20499 raft_consensus.cc:697] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [term 3 LEADER]: Becoming Leader. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Running, Role: LEADER
I20260709 13:20:38.095081 20499 consensus_queue.cc:237] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:38.143482 20498 tablet_bootstrap.cc:492] T 8fd58e41ecee4779b3e72df3aea4463a P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:38.144579 20498 tablet_bootstrap.cc:492] T 8fd58e41ecee4779b3e72df3aea4463a P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap complete.
I20260709 13:20:38.145256 20498 ts_tablet_manager.cc:1403] T 8fd58e41ecee4779b3e72df3aea4463a P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.068s	user 0.024s	sys 0.016s
I20260709 13:20:38.147709 20498 raft_consensus.cc:359] T 8fd58e41ecee4779b3e72df3aea4463a P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:38.148398 20498 raft_consensus.cc:740] T 8fd58e41ecee4779b3e72df3aea4463a P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:38.149219 20498 consensus_queue.cc:260] T 8fd58e41ecee4779b3e72df3aea4463a P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:38.151351 20498 ts_tablet_manager.cc:1434] T 8fd58e41ecee4779b3e72df3aea4463a P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:20:38.154387 20498 tablet_bootstrap.cc:492] T f820e93ffdcd428ea4adfca88589a850 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:38.160068 19623 catalog_manager.cc:5697] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 reported cstate change: term changed from 1 to 3, leader changed from 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129) to b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131). New cstate: current_term: 3 leader_uuid: "b80b3a62d7d549a9a5c42def46769ac4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:38.190217 20498 tablet_bootstrap.cc:492] T f820e93ffdcd428ea4adfca88589a850 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:38.191190 20498 tablet_bootstrap.cc:492] T f820e93ffdcd428ea4adfca88589a850 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap complete.
I20260709 13:20:38.191812 20498 ts_tablet_manager.cc:1403] T f820e93ffdcd428ea4adfca88589a850 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.038s	user 0.019s	sys 0.008s
I20260709 13:20:38.194516 20498 raft_consensus.cc:359] T f820e93ffdcd428ea4adfca88589a850 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:38.195065 20498 raft_consensus.cc:740] T f820e93ffdcd428ea4adfca88589a850 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:38.195704 20498 consensus_queue.cc:260] T f820e93ffdcd428ea4adfca88589a850 P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:38.197573 20498 ts_tablet_manager.cc:1434] T f820e93ffdcd428ea4adfca88589a850 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.005s	user 0.001s	sys 0.001s
I20260709 13:20:38.199432 20498 tablet_bootstrap.cc:492] T 63868398f3604f529519d926d2831f4e P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
W20260709 13:20:38.203716 19766 consensus_peers.cc:597] T 180230b77b7640b29d055ae8f4358ffe P e90ffed3ba174ababeb21d9415d4d918 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:38.214360 20498 tablet_bootstrap.cc:492] T 63868398f3604f529519d926d2831f4e P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:38.215396 20498 tablet_bootstrap.cc:492] T 63868398f3604f529519d926d2831f4e P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap complete.
I20260709 13:20:38.215976 20498 ts_tablet_manager.cc:1403] T 63868398f3604f529519d926d2831f4e P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.017s	user 0.014s	sys 0.001s
I20260709 13:20:38.217732 20498 raft_consensus.cc:359] T 63868398f3604f529519d926d2831f4e P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:38.218259 20498 raft_consensus.cc:740] T 63868398f3604f529519d926d2831f4e P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:38.218921 20498 consensus_queue.cc:260] T 63868398f3604f529519d926d2831f4e P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:38.220907 20498 ts_tablet_manager.cc:1434] T 63868398f3604f529519d926d2831f4e P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:20:38.222270 20498 tablet_bootstrap.cc:492] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:38.224952 20540 raft_consensus.cc:3060] T 63868398f3604f529519d926d2831f4e P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:38.235639 20498 tablet_bootstrap.cc:492] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:38.236485 20498 tablet_bootstrap.cc:492] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap complete.
I20260709 13:20:38.237059 20498 ts_tablet_manager.cc:1403] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.015s	user 0.009s	sys 0.003s
I20260709 13:20:38.239095 20498 raft_consensus.cc:359] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:38.239768 20498 raft_consensus.cc:740] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:38.240429 20498 consensus_queue.cc:260] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:38.242470 20498 ts_tablet_manager.cc:1434] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:20:38.243269 20498 tablet_bootstrap.cc:492] T 59fc86c7e9f648e3b3f51e936e2bcfec P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:38.257499 20498 tablet_bootstrap.cc:492] T 59fc86c7e9f648e3b3f51e936e2bcfec P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:38.258370 20498 tablet_bootstrap.cc:492] T 59fc86c7e9f648e3b3f51e936e2bcfec P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap complete.
I20260709 13:20:38.259071 20498 ts_tablet_manager.cc:1403] T 59fc86c7e9f648e3b3f51e936e2bcfec P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.016s	user 0.012s	sys 0.003s
I20260709 13:20:38.261428 20498 raft_consensus.cc:359] T 59fc86c7e9f648e3b3f51e936e2bcfec P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:38.262104 20498 raft_consensus.cc:740] T 59fc86c7e9f648e3b3f51e936e2bcfec P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:38.262830 20498 consensus_queue.cc:260] T 59fc86c7e9f648e3b3f51e936e2bcfec P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:38.264609 20498 ts_tablet_manager.cc:1434] T 59fc86c7e9f648e3b3f51e936e2bcfec P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:20:38.265422 20498 tablet_bootstrap.cc:492] T 353a546cf8c54671971c239502ef165a P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
W20260709 13:20:38.270310 19766 consensus_peers.cc:597] T 2eef64a7d31b4e1e899aee0857cb0e07 P e90ffed3ba174ababeb21d9415d4d918 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:38.279644 20498 tablet_bootstrap.cc:492] T 353a546cf8c54671971c239502ef165a P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:38.280500 20498 tablet_bootstrap.cc:492] T 353a546cf8c54671971c239502ef165a P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap complete.
I20260709 13:20:38.281059 20498 ts_tablet_manager.cc:1403] T 353a546cf8c54671971c239502ef165a P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.016s	user 0.007s	sys 0.006s
I20260709 13:20:38.283033 20498 raft_consensus.cc:359] T 353a546cf8c54671971c239502ef165a P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:38.283538 20498 raft_consensus.cc:740] T 353a546cf8c54671971c239502ef165a P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:38.284240 20498 consensus_queue.cc:260] T 353a546cf8c54671971c239502ef165a P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:38.285912 20498 ts_tablet_manager.cc:1434] T 353a546cf8c54671971c239502ef165a P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.001s
I20260709 13:20:38.287300 20498 tablet_bootstrap.cc:492] T ad38a1519fb748588f5a322589274004 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:38.301484 20498 tablet_bootstrap.cc:492] T ad38a1519fb748588f5a322589274004 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:38.302469 20498 tablet_bootstrap.cc:492] T ad38a1519fb748588f5a322589274004 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap complete.
I20260709 13:20:38.303184 20498 ts_tablet_manager.cc:1403] T ad38a1519fb748588f5a322589274004 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.016s	user 0.014s	sys 0.000s
I20260709 13:20:38.305593 20498 raft_consensus.cc:359] T ad38a1519fb748588f5a322589274004 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:38.306278 20498 raft_consensus.cc:740] T ad38a1519fb748588f5a322589274004 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:38.306967 20498 consensus_queue.cc:260] T ad38a1519fb748588f5a322589274004 P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:38.309176 20498 ts_tablet_manager.cc:1434] T ad38a1519fb748588f5a322589274004 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:20:38.311547 20498 tablet_bootstrap.cc:492] T 86964b8a61ce494f9096c4fefdc6efbf P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:38.331534 20498 tablet_bootstrap.cc:492] T 86964b8a61ce494f9096c4fefdc6efbf P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:38.332499 20498 tablet_bootstrap.cc:492] T 86964b8a61ce494f9096c4fefdc6efbf P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap complete.
I20260709 13:20:38.333180 20498 ts_tablet_manager.cc:1403] T 86964b8a61ce494f9096c4fefdc6efbf P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.022s	user 0.019s	sys 0.000s
I20260709 13:20:38.335788 20498 raft_consensus.cc:359] T 86964b8a61ce494f9096c4fefdc6efbf P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:38.336498 20498 raft_consensus.cc:740] T 86964b8a61ce494f9096c4fefdc6efbf P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:38.337275 20498 consensus_queue.cc:260] T 86964b8a61ce494f9096c4fefdc6efbf P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:38.339412 20498 ts_tablet_manager.cc:1434] T 86964b8a61ce494f9096c4fefdc6efbf P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:20:38.340258 20498 tablet_bootstrap.cc:492] T cb33acfa7a9f46ab95b0e77e1394f258 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:38.358191 20498 tablet_bootstrap.cc:492] T cb33acfa7a9f46ab95b0e77e1394f258 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:38.359290 20498 tablet_bootstrap.cc:492] T cb33acfa7a9f46ab95b0e77e1394f258 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap complete.
I20260709 13:20:38.360510 20498 ts_tablet_manager.cc:1403] T cb33acfa7a9f46ab95b0e77e1394f258 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.021s	user 0.012s	sys 0.004s
I20260709 13:20:38.364890 20498 raft_consensus.cc:359] T cb33acfa7a9f46ab95b0e77e1394f258 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:38.365592 20498 raft_consensus.cc:740] T cb33acfa7a9f46ab95b0e77e1394f258 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:38.366307 20498 consensus_queue.cc:260] T cb33acfa7a9f46ab95b0e77e1394f258 P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:38.369135 20498 ts_tablet_manager.cc:1434] T cb33acfa7a9f46ab95b0e77e1394f258 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 13:20:38.370829 20498 tablet_bootstrap.cc:492] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:38.384326 20498 tablet_bootstrap.cc:492] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:38.385042 20498 tablet_bootstrap.cc:492] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap complete.
I20260709 13:20:38.385536 20498 ts_tablet_manager.cc:1403] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.015s	user 0.008s	sys 0.004s
I20260709 13:20:38.387317 20498 raft_consensus.cc:359] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:38.387795 20498 raft_consensus.cc:740] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:38.388280 20498 consensus_queue.cc:260] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:38.390033 20498 ts_tablet_manager.cc:1434] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:20:38.391201 20498 tablet_bootstrap.cc:492] T af881b097b4245b1be6072579197cb6b P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:38.406381 20498 tablet_bootstrap.cc:492] T af881b097b4245b1be6072579197cb6b P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:38.407424 20498 tablet_bootstrap.cc:492] T af881b097b4245b1be6072579197cb6b P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap complete.
I20260709 13:20:38.408106 20498 ts_tablet_manager.cc:1403] T af881b097b4245b1be6072579197cb6b P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.017s	user 0.011s	sys 0.004s
I20260709 13:20:38.410595 20498 raft_consensus.cc:359] T af881b097b4245b1be6072579197cb6b P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:38.411307 20498 raft_consensus.cc:740] T af881b097b4245b1be6072579197cb6b P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:38.411965 20498 consensus_queue.cc:260] T af881b097b4245b1be6072579197cb6b P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:38.413801 20498 ts_tablet_manager.cc:1434] T af881b097b4245b1be6072579197cb6b P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:20:38.415032 20498 tablet_bootstrap.cc:492] T dc5c36b41652443a8130592df8a66eb9 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:38.429319 20498 tablet_bootstrap.cc:492] T dc5c36b41652443a8130592df8a66eb9 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:38.430262 20498 tablet_bootstrap.cc:492] T dc5c36b41652443a8130592df8a66eb9 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap complete.
I20260709 13:20:38.430962 20498 ts_tablet_manager.cc:1403] T dc5c36b41652443a8130592df8a66eb9 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.016s	user 0.015s	sys 0.000s
I20260709 13:20:38.433259 20498 raft_consensus.cc:359] T dc5c36b41652443a8130592df8a66eb9 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:38.433892 20498 raft_consensus.cc:740] T dc5c36b41652443a8130592df8a66eb9 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:38.434533 20498 consensus_queue.cc:260] T dc5c36b41652443a8130592df8a66eb9 P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:38.436481 20498 ts_tablet_manager.cc:1434] T dc5c36b41652443a8130592df8a66eb9 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:20:38.437150 20498 tablet_bootstrap.cc:492] T 2eef64a7d31b4e1e899aee0857cb0e07 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:38.463513 20498 tablet_bootstrap.cc:492] T 2eef64a7d31b4e1e899aee0857cb0e07 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:38.464278 20498 tablet_bootstrap.cc:492] T 2eef64a7d31b4e1e899aee0857cb0e07 P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap complete.
I20260709 13:20:38.464856 20498 ts_tablet_manager.cc:1403] T 2eef64a7d31b4e1e899aee0857cb0e07 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.028s	user 0.009s	sys 0.016s
I20260709 13:20:38.467099 20498 raft_consensus.cc:359] T 2eef64a7d31b4e1e899aee0857cb0e07 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:38.467638 20498 raft_consensus.cc:740] T 2eef64a7d31b4e1e899aee0857cb0e07 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:38.468386 20498 consensus_queue.cc:260] T 2eef64a7d31b4e1e899aee0857cb0e07 P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:38.470621 20498 ts_tablet_manager.cc:1434] T 2eef64a7d31b4e1e899aee0857cb0e07 P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:20:38.471874 20498 tablet_bootstrap.cc:492] T 180230b77b7640b29d055ae8f4358ffe P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:38.485976 20498 tablet_bootstrap.cc:492] T 180230b77b7640b29d055ae8f4358ffe P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:38.487030 20498 tablet_bootstrap.cc:492] T 180230b77b7640b29d055ae8f4358ffe P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap complete.
I20260709 13:20:38.487643 20498 ts_tablet_manager.cc:1403] T 180230b77b7640b29d055ae8f4358ffe P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.016s	user 0.014s	sys 0.001s
I20260709 13:20:38.489861 20498 raft_consensus.cc:359] T 180230b77b7640b29d055ae8f4358ffe P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:38.490478 20498 raft_consensus.cc:740] T 180230b77b7640b29d055ae8f4358ffe P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:38.491134 20498 consensus_queue.cc:260] T 180230b77b7640b29d055ae8f4358ffe P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:38.493074 20498 ts_tablet_manager.cc:1434] T 180230b77b7640b29d055ae8f4358ffe P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:20:38.494256 20498 tablet_bootstrap.cc:492] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap starting.
I20260709 13:20:38.520416 20498 tablet_bootstrap.cc:492] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:38.521432 20498 tablet_bootstrap.cc:492] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c: Bootstrap complete.
I20260709 13:20:38.522146 20498 ts_tablet_manager.cc:1403] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c: Time spent bootstrapping tablet: real 0.028s	user 0.020s	sys 0.003s
I20260709 13:20:38.524632 20498 raft_consensus.cc:359] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:38.525447 20498 raft_consensus.cc:740] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Initialized, Role: FOLLOWER
I20260709 13:20:38.526111 20498 consensus_queue.cc:260] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:38.528147 20498 ts_tablet_manager.cc:1434] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:20:38.529707 20540 raft_consensus.cc:3060] T cb33acfa7a9f46ab95b0e77e1394f258 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:38.547782 20540 raft_consensus.cc:3060] T 86964b8a61ce494f9096c4fefdc6efbf P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:38.601541 20540 raft_consensus.cc:3060] T dc5c36b41652443a8130592df8a66eb9 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:38.649298 20540 raft_consensus.cc:3060] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:20:38.654820 19948 raft_consensus.cc:1275] T 7a5476f5d68d496eaedce93e80bff5e4 P c62716bf66c24330860573535b8b9430 [term 3 FOLLOWER]: Refusing update from remote peer b80b3a62d7d549a9a5c42def46769ac4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:20:38.656422 20584 consensus_queue.cc:1048] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:20:38.657222 20540 raft_consensus.cc:1275] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c [term 3 FOLLOWER]: Refusing update from remote peer b80b3a62d7d549a9a5c42def46769ac4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:20:38.658686 20584 consensus_queue.cc:1048] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:20:38.680931 20538 raft_consensus.cc:3060] T 59fc86c7e9f648e3b3f51e936e2bcfec P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:38.750728 20540 raft_consensus.cc:3060] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:46.717463  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:46.780687 19871 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "29f92a880ce04259ae0ee9c5288949ff"
dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
mode: GRACEFUL
new_leader_uuid: "c62716bf66c24330860573535b8b9430"
 from {username='slave'} at 127.0.0.1:39648
I20260709 13:20:46.781404 19871 raft_consensus.cc:606] T 29f92a880ce04259ae0ee9c5288949ff P b80b3a62d7d549a9a5c42def46769ac4 [term 1 LEADER]: Received request to transfer leadership to TS c62716bf66c24330860573535b8b9430
I20260709 13:20:46.784111 19871 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7a5476f5d68d496eaedce93e80bff5e4"
dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
mode: GRACEFUL
new_leader_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
 from {username='slave'} at 127.0.0.1:39648
I20260709 13:20:46.784621 19871 raft_consensus.cc:606] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [term 3 LEADER]: Received request to transfer leadership to TS 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:46.787104 19871 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9b2e4bf3fb5c456dab400b096f7bd729"
dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
mode: GRACEFUL
new_leader_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
 from {username='slave'} at 127.0.0.1:39648
I20260709 13:20:46.787590 19871 raft_consensus.cc:606] T 9b2e4bf3fb5c456dab400b096f7bd729 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 LEADER]: Received request to transfer leadership to TS 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:46.790026 19871 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bc04a038495940c1bbeda4a2e6f17b82"
dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
mode: GRACEFUL
new_leader_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
 from {username='slave'} at 127.0.0.1:39648
I20260709 13:20:46.790457 19871 raft_consensus.cc:606] T bc04a038495940c1bbeda4a2e6f17b82 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 LEADER]: Received request to transfer leadership to TS 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:46.802420 19795 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d5f8619b1f9f46adaa41774af856d9b1"
dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
mode: GRACEFUL
new_leader_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
 from {username='slave'} at 127.0.0.1:40112
I20260709 13:20:46.803078 19795 raft_consensus.cc:606] T d5f8619b1f9f46adaa41774af856d9b1 P e90ffed3ba174ababeb21d9415d4d918 [term 2 LEADER]: Received request to transfer leadership to TS 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:46.806114 19871 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e804b7d8fd7e44f59516fb299bea7a2c"
dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
mode: GRACEFUL
new_leader_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
 from {username='slave'} at 127.0.0.1:39648
I20260709 13:20:46.806712 19871 raft_consensus.cc:606] T e804b7d8fd7e44f59516fb299bea7a2c P b80b3a62d7d549a9a5c42def46769ac4 [term 2 LEADER]: Received request to transfer leadership to TS 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:46.807869  9682 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:20:46.809621  9682 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:20:46.921669 20703 raft_consensus.cc:993] T e804b7d8fd7e44f59516fb299bea7a2c P b80b3a62d7d549a9a5c42def46769ac4: : Instructing follower 8df38c4c53c44e94b36c1e8992476d7c to start an election
I20260709 13:20:46.922271 20703 raft_consensus.cc:1081] T e804b7d8fd7e44f59516fb299bea7a2c P b80b3a62d7d549a9a5c42def46769ac4 [term 2 LEADER]: Signalling peer 8df38c4c53c44e94b36c1e8992476d7c to start an election
I20260709 13:20:46.924490 20545 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e804b7d8fd7e44f59516fb299bea7a2c"
dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
 from {username='slave'} at 127.0.0.1:33934
I20260709 13:20:46.925123 20545 raft_consensus.cc:493] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:46.926134 20545 raft_consensus.cc:3060] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:20:46.931733 20388 raft_consensus.cc:993] T bc04a038495940c1bbeda4a2e6f17b82 P b80b3a62d7d549a9a5c42def46769ac4: : Instructing follower 8df38c4c53c44e94b36c1e8992476d7c to start an election
I20260709 13:20:46.932276 20703 raft_consensus.cc:1081] T bc04a038495940c1bbeda4a2e6f17b82 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 LEADER]: Signalling peer 8df38c4c53c44e94b36c1e8992476d7c to start an election
I20260709 13:20:46.934077 20538 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bc04a038495940c1bbeda4a2e6f17b82"
dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
 from {username='slave'} at 127.0.0.1:33934
I20260709 13:20:46.934682 20538 raft_consensus.cc:493] T bc04a038495940c1bbeda4a2e6f17b82 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:46.935141 20538 raft_consensus.cc:3060] T bc04a038495940c1bbeda4a2e6f17b82 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:46.941526 20545 raft_consensus.cc:515] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:46.943641 20538 raft_consensus.cc:515] T bc04a038495940c1bbeda4a2e6f17b82 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:46.945544 20545 leader_election.cc:290] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 3 election: Requested vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:46.945865 20538 leader_election.cc:290] T bc04a038495940c1bbeda4a2e6f17b82 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 2 election: Requested vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:46.958273 19868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc04a038495940c1bbeda4a2e6f17b82" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
I20260709 13:20:46.959017 19868 raft_consensus.cc:3055] T bc04a038495940c1bbeda4a2e6f17b82 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:20:46.959352 19868 raft_consensus.cc:740] T bc04a038495940c1bbeda4a2e6f17b82 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Running, Role: LEADER
I20260709 13:20:46.960317 19868 consensus_queue.cc:260] T bc04a038495940c1bbeda4a2e6f17b82 P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:46.961671 19868 raft_consensus.cc:3060] T bc04a038495940c1bbeda4a2e6f17b82 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:46.963825 19948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e804b7d8fd7e44f59516fb299bea7a2c" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:46.969146 19868 raft_consensus.cc:2468] T bc04a038495940c1bbeda4a2e6f17b82 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 2.
I20260709 13:20:46.969513 19948 raft_consensus.cc:3060] T e804b7d8fd7e44f59516fb299bea7a2c P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:20:46.957857 19871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e804b7d8fd7e44f59516fb299bea7a2c" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
I20260709 13:20:46.969481 19944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc04a038495940c1bbeda4a2e6f17b82" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:46.970852 19871 raft_consensus.cc:3055] T e804b7d8fd7e44f59516fb299bea7a2c P b80b3a62d7d549a9a5c42def46769ac4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:20:46.971661 19944 raft_consensus.cc:3060] T bc04a038495940c1bbeda4a2e6f17b82 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:46.972100 19871 raft_consensus.cc:740] T e804b7d8fd7e44f59516fb299bea7a2c P b80b3a62d7d549a9a5c42def46769ac4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Running, Role: LEADER
I20260709 13:20:46.973692 19871 consensus_queue.cc:260] T e804b7d8fd7e44f59516fb299bea7a2c P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:46.975070 19871 raft_consensus.cc:3060] T e804b7d8fd7e44f59516fb299bea7a2c P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:20:46.976039 20494 leader_election.cc:304] T bc04a038495940c1bbeda4a2e6f17b82 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4; no voters: 
I20260709 13:20:46.980285 19944 raft_consensus.cc:2468] T bc04a038495940c1bbeda4a2e6f17b82 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 2.
I20260709 13:20:46.981168 19948 raft_consensus.cc:2468] T e804b7d8fd7e44f59516fb299bea7a2c P c62716bf66c24330860573535b8b9430 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 3.
I20260709 13:20:46.990918 19871 raft_consensus.cc:2468] T e804b7d8fd7e44f59516fb299bea7a2c P b80b3a62d7d549a9a5c42def46769ac4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 3.
I20260709 13:20:46.993513 20714 raft_consensus.cc:2804] T bc04a038495940c1bbeda4a2e6f17b82 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:20:46.995100 20714 raft_consensus.cc:697] T bc04a038495940c1bbeda4a2e6f17b82 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 LEADER]: Becoming Leader. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Running, Role: LEADER
I20260709 13:20:46.996232 20714 consensus_queue.cc:237] T bc04a038495940c1bbeda4a2e6f17b82 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:46.996558 20494 leader_election.cc:304] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, c62716bf66c24330860573535b8b9430; no voters: 
I20260709 13:20:46.999202 20715 raft_consensus.cc:2804] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:20:47.000779 20715 raft_consensus.cc:697] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c [term 3 LEADER]: Becoming Leader. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Running, Role: LEADER
I20260709 13:20:47.001634 20715 consensus_queue.cc:237] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:47.011426 19621 catalog_manager.cc:5697] T bc04a038495940c1bbeda4a2e6f17b82 P 8df38c4c53c44e94b36c1e8992476d7c reported cstate change: term changed from 1 to 2, leader changed from b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131) to 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129). New cstate: current_term: 2 leader_uuid: "8df38c4c53c44e94b36c1e8992476d7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:47.035231 19621 catalog_manager.cc:5697] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c reported cstate change: term changed from 2 to 3, leader changed from b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131) to 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129). New cstate: current_term: 3 leader_uuid: "8df38c4c53c44e94b36c1e8992476d7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:47.129086 20703 raft_consensus.cc:993] T 9b2e4bf3fb5c456dab400b096f7bd729 P b80b3a62d7d549a9a5c42def46769ac4: : Instructing follower 8df38c4c53c44e94b36c1e8992476d7c to start an election
I20260709 13:20:47.129630 20710 raft_consensus.cc:1081] T 9b2e4bf3fb5c456dab400b096f7bd729 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 LEADER]: Signalling peer 8df38c4c53c44e94b36c1e8992476d7c to start an election
I20260709 13:20:47.131145 20545 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9b2e4bf3fb5c456dab400b096f7bd729"
dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
 from {username='slave'} at 127.0.0.1:33934
I20260709 13:20:47.131724 20545 raft_consensus.cc:493] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:47.132028 20545 raft_consensus.cc:3060] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:20:47.141276 20702 raft_consensus.cc:993] T d5f8619b1f9f46adaa41774af856d9b1 P e90ffed3ba174ababeb21d9415d4d918: : Instructing follower 8df38c4c53c44e94b36c1e8992476d7c to start an election
I20260709 13:20:47.142007 20704 raft_consensus.cc:1081] T d5f8619b1f9f46adaa41774af856d9b1 P e90ffed3ba174ababeb21d9415d4d918 [term 2 LEADER]: Signalling peer 8df38c4c53c44e94b36c1e8992476d7c to start an election
I20260709 13:20:47.143677 20538 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d5f8619b1f9f46adaa41774af856d9b1"
dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
 from {username='slave'} at 127.0.0.1:33928
I20260709 13:20:47.144375 20538 raft_consensus.cc:493] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:47.145150 20538 raft_consensus.cc:3060] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:20:47.144181 20545 raft_consensus.cc:515] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:47.149170 20545 leader_election.cc:290] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 3 election: Requested vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:47.149813 19871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b2e4bf3fb5c456dab400b096f7bd729" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
I20260709 13:20:47.150470 19871 raft_consensus.cc:3055] T 9b2e4bf3fb5c456dab400b096f7bd729 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:20:47.150857 19871 raft_consensus.cc:740] T 9b2e4bf3fb5c456dab400b096f7bd729 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Running, Role: LEADER
I20260709 13:20:47.151386 19948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b2e4bf3fb5c456dab400b096f7bd729" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:47.152081 19948 raft_consensus.cc:3060] T 9b2e4bf3fb5c456dab400b096f7bd729 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:20:47.151835 19871 consensus_queue.cc:260] T 9b2e4bf3fb5c456dab400b096f7bd729 P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:47.154228 19871 raft_consensus.cc:3060] T 9b2e4bf3fb5c456dab400b096f7bd729 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:20:47.155082 20538 raft_consensus.cc:515] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:47.158362 20538 leader_election.cc:290] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 3 election: Requested vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:47.158393 19944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5f8619b1f9f46adaa41774af856d9b1" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:47.159291 19944 raft_consensus.cc:3060] T d5f8619b1f9f46adaa41774af856d9b1 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:20:47.159665 19948 raft_consensus.cc:2468] T 9b2e4bf3fb5c456dab400b096f7bd729 P c62716bf66c24330860573535b8b9430 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 3.
I20260709 13:20:47.160661 20494 leader_election.cc:304] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, c62716bf66c24330860573535b8b9430; no voters: 
I20260709 13:20:47.161525 20715 raft_consensus.cc:2804] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:20:47.162017 20715 raft_consensus.cc:697] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c [term 3 LEADER]: Becoming Leader. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Running, Role: LEADER
I20260709 13:20:47.163040 20715 consensus_queue.cc:237] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:47.164441 19871 raft_consensus.cc:2468] T 9b2e4bf3fb5c456dab400b096f7bd729 P b80b3a62d7d549a9a5c42def46769ac4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 3.
I20260709 13:20:47.167040 19944 raft_consensus.cc:2468] T d5f8619b1f9f46adaa41774af856d9b1 P c62716bf66c24330860573535b8b9430 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 3.
I20260709 13:20:47.168356 20494 leader_election.cc:304] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, c62716bf66c24330860573535b8b9430; no voters: 
I20260709 13:20:47.169080 20714 raft_consensus.cc:2804] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:20:47.169623 20714 raft_consensus.cc:697] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c [term 3 LEADER]: Becoming Leader. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Running, Role: LEADER
I20260709 13:20:47.170583 20714 consensus_queue.cc:237] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:47.176143 19794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5f8619b1f9f46adaa41774af856d9b1" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
I20260709 13:20:47.176801 19794 raft_consensus.cc:3055] T d5f8619b1f9f46adaa41774af856d9b1 P e90ffed3ba174ababeb21d9415d4d918 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:20:47.177080 19794 raft_consensus.cc:740] T d5f8619b1f9f46adaa41774af856d9b1 P e90ffed3ba174ababeb21d9415d4d918 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Running, Role: LEADER
I20260709 13:20:47.177805 19794 consensus_queue.cc:260] T d5f8619b1f9f46adaa41774af856d9b1 P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:47.178776 19794 raft_consensus.cc:3060] T d5f8619b1f9f46adaa41774af856d9b1 P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:20:47.179821 19621 catalog_manager.cc:5697] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c reported cstate change: term changed from 2 to 3, leader changed from b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131) to 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129). New cstate: current_term: 3 leader_uuid: "8df38c4c53c44e94b36c1e8992476d7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:47.183751 20703 raft_consensus.cc:993] T 29f92a880ce04259ae0ee9c5288949ff P b80b3a62d7d549a9a5c42def46769ac4: : Instructing follower c62716bf66c24330860573535b8b9430 to start an election
I20260709 13:20:47.184310 20703 raft_consensus.cc:1081] T 29f92a880ce04259ae0ee9c5288949ff P b80b3a62d7d549a9a5c42def46769ac4 [term 1 LEADER]: Signalling peer c62716bf66c24330860573535b8b9430 to start an election
I20260709 13:20:47.186975 19944 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "29f92a880ce04259ae0ee9c5288949ff"
dest_uuid: "c62716bf66c24330860573535b8b9430"
 from {username='slave'} at 127.0.0.1:57490
I20260709 13:20:47.187578 19944 raft_consensus.cc:493] T 29f92a880ce04259ae0ee9c5288949ff P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:47.187933 19944 raft_consensus.cc:3060] T 29f92a880ce04259ae0ee9c5288949ff P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:47.188803 19794 raft_consensus.cc:2468] T d5f8619b1f9f46adaa41774af856d9b1 P e90ffed3ba174ababeb21d9415d4d918 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 3.
I20260709 13:20:47.197629 19944 raft_consensus.cc:515] T 29f92a880ce04259ae0ee9c5288949ff P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:47.200492 19944 leader_election.cc:290] T 29f92a880ce04259ae0ee9c5288949ff P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 2 election: Requested vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:47.201306 19871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29f92a880ce04259ae0ee9c5288949ff" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
I20260709 13:20:47.201387 19794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29f92a880ce04259ae0ee9c5288949ff" candidate_uuid: "c62716bf66c24330860573535b8b9430" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
I20260709 13:20:47.203933 19794 raft_consensus.cc:3060] T 29f92a880ce04259ae0ee9c5288949ff P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:47.201308 19621 catalog_manager.cc:5697] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c reported cstate change: term changed from 2 to 3, leader changed from e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130) to 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129). New cstate: current_term: 3 leader_uuid: "8df38c4c53c44e94b36c1e8992476d7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:47.205130 19871 raft_consensus.cc:3055] T 29f92a880ce04259ae0ee9c5288949ff P b80b3a62d7d549a9a5c42def46769ac4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:20:47.205447 19871 raft_consensus.cc:740] T 29f92a880ce04259ae0ee9c5288949ff P b80b3a62d7d549a9a5c42def46769ac4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Running, Role: LEADER
I20260709 13:20:47.206408 19871 consensus_queue.cc:260] T 29f92a880ce04259ae0ee9c5288949ff P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:47.207804 19871 raft_consensus.cc:3060] T 29f92a880ce04259ae0ee9c5288949ff P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:47.212520 19794 raft_consensus.cc:2468] T 29f92a880ce04259ae0ee9c5288949ff P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c62716bf66c24330860573535b8b9430 in term 2.
I20260709 13:20:47.212538 20388 raft_consensus.cc:993] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4: : Instructing follower 8df38c4c53c44e94b36c1e8992476d7c to start an election
I20260709 13:20:47.213377 20388 raft_consensus.cc:1081] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [term 3 LEADER]: Signalling peer 8df38c4c53c44e94b36c1e8992476d7c to start an election
I20260709 13:20:47.214848 20538 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7a5476f5d68d496eaedce93e80bff5e4"
dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
 from {username='slave'} at 127.0.0.1:33934
I20260709 13:20:47.215276 19913 leader_election.cc:304] T 29f92a880ce04259ae0ee9c5288949ff P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c62716bf66c24330860573535b8b9430, e90ffed3ba174ababeb21d9415d4d918; no voters: 
I20260709 13:20:47.215454 20538 raft_consensus.cc:493] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:47.216279 20538 raft_consensus.cc:3060] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:20:47.216295 20691 raft_consensus.cc:2804] T 29f92a880ce04259ae0ee9c5288949ff P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:20:47.216948 20691 raft_consensus.cc:697] T 29f92a880ce04259ae0ee9c5288949ff P c62716bf66c24330860573535b8b9430 [term 2 LEADER]: Becoming Leader. State: Replica: c62716bf66c24330860573535b8b9430, State: Running, Role: LEADER
I20260709 13:20:47.216976 19871 raft_consensus.cc:2468] T 29f92a880ce04259ae0ee9c5288949ff P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c62716bf66c24330860573535b8b9430 in term 2.
I20260709 13:20:47.217909 20691 consensus_queue.cc:237] T 29f92a880ce04259ae0ee9c5288949ff P c62716bf66c24330860573535b8b9430 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:47.225078 20538 raft_consensus.cc:515] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:47.228281 19944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5476f5d68d496eaedce93e80bff5e4" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:47.228847 19871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5476f5d68d496eaedce93e80bff5e4" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
I20260709 13:20:47.229003 19944 raft_consensus.cc:3060] T 7a5476f5d68d496eaedce93e80bff5e4 P c62716bf66c24330860573535b8b9430 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:20:47.229507 19871 raft_consensus.cc:3055] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:20:47.229862 19871 raft_consensus.cc:740] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Running, Role: LEADER
I20260709 13:20:47.230746 19871 consensus_queue.cc:260] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:47.231917 19871 raft_consensus.cc:3060] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:20:47.231427 19621 catalog_manager.cc:5697] T 29f92a880ce04259ae0ee9c5288949ff P c62716bf66c24330860573535b8b9430 reported cstate change: term changed from 1 to 2, leader changed from b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131) to c62716bf66c24330860573535b8b9430 (127.9.116.132). New cstate: current_term: 2 leader_uuid: "c62716bf66c24330860573535b8b9430" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:47.236037 19944 raft_consensus.cc:2468] T 7a5476f5d68d496eaedce93e80bff5e4 P c62716bf66c24330860573535b8b9430 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 4.
I20260709 13:20:47.227371 20538 leader_election.cc:290] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 4 election: Requested vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:47.237216 20494 leader_election.cc:304] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, c62716bf66c24330860573535b8b9430; no voters: 
I20260709 13:20:47.238456 20714 raft_consensus.cc:2804] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:20:47.238564 19871 raft_consensus.cc:2468] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 4.
I20260709 13:20:47.239050 20714 raft_consensus.cc:697] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c [term 4 LEADER]: Becoming Leader. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Running, Role: LEADER
I20260709 13:20:47.240065 20714 consensus_queue.cc:237] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:47.254200 19621 catalog_manager.cc:5697] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c reported cstate change: term changed from 3 to 4, leader changed from b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131) to 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129). New cstate: current_term: 4 leader_uuid: "8df38c4c53c44e94b36c1e8992476d7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:47.391700 19944 raft_consensus.cc:1275] T bc04a038495940c1bbeda4a2e6f17b82 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Refusing update from remote peer 8df38c4c53c44e94b36c1e8992476d7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:47.392958 20714 consensus_queue.cc:1048] T bc04a038495940c1bbeda4a2e6f17b82 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:20:47.407748 19871 raft_consensus.cc:1275] T bc04a038495940c1bbeda4a2e6f17b82 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Refusing update from remote peer 8df38c4c53c44e94b36c1e8992476d7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:47.409200 20717 consensus_queue.cc:1048] T bc04a038495940c1bbeda4a2e6f17b82 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:20:47.554332 19944 raft_consensus.cc:1275] T 9b2e4bf3fb5c456dab400b096f7bd729 P c62716bf66c24330860573535b8b9430 [term 3 FOLLOWER]: Refusing update from remote peer 8df38c4c53c44e94b36c1e8992476d7c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:20:47.555768 20717 consensus_queue.cc:1048] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:20:47.565958 19868 raft_consensus.cc:1275] T 9b2e4bf3fb5c456dab400b096f7bd729 P b80b3a62d7d549a9a5c42def46769ac4 [term 3 FOLLOWER]: Refusing update from remote peer 8df38c4c53c44e94b36c1e8992476d7c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:20:47.567746 20714 consensus_queue.cc:1048] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:20:47.571975 19871 raft_consensus.cc:1275] T e804b7d8fd7e44f59516fb299bea7a2c P b80b3a62d7d549a9a5c42def46769ac4 [term 3 FOLLOWER]: Refusing update from remote peer 8df38c4c53c44e94b36c1e8992476d7c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:20:47.573778 20717 consensus_queue.cc:1048] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:20:47.581519 19944 raft_consensus.cc:1275] T d5f8619b1f9f46adaa41774af856d9b1 P c62716bf66c24330860573535b8b9430 [term 3 FOLLOWER]: Refusing update from remote peer 8df38c4c53c44e94b36c1e8992476d7c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:20:47.583352 20717 consensus_queue.cc:1048] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:20:47.606776 19948 raft_consensus.cc:1275] T e804b7d8fd7e44f59516fb299bea7a2c P c62716bf66c24330860573535b8b9430 [term 3 FOLLOWER]: Refusing update from remote peer 8df38c4c53c44e94b36c1e8992476d7c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:20:47.608377 20715 consensus_queue.cc:1048] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:20:47.633733 19948 raft_consensus.cc:1275] T 7a5476f5d68d496eaedce93e80bff5e4 P c62716bf66c24330860573535b8b9430 [term 4 FOLLOWER]: Refusing update from remote peer 8df38c4c53c44e94b36c1e8992476d7c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:20:47.634764 19795 raft_consensus.cc:1275] T d5f8619b1f9f46adaa41774af856d9b1 P e90ffed3ba174ababeb21d9415d4d918 [term 3 FOLLOWER]: Refusing update from remote peer 8df38c4c53c44e94b36c1e8992476d7c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:20:47.636845 20714 consensus_queue.cc:1048] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:20:47.640542 20739 consensus_queue.cc:1048] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:20:47.667101 19871 raft_consensus.cc:1275] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [term 4 FOLLOWER]: Refusing update from remote peer 8df38c4c53c44e94b36c1e8992476d7c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:20:47.669173 20714 consensus_queue.cc:1048] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:20:47.673247 19868 raft_consensus.cc:1275] T 29f92a880ce04259ae0ee9c5288949ff P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Refusing update from remote peer c62716bf66c24330860573535b8b9430: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:47.680119 20691 consensus_queue.cc:1048] T 29f92a880ce04259ae0ee9c5288949ff P c62716bf66c24330860573535b8b9430 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:20:47.735590 19795 raft_consensus.cc:1275] T 29f92a880ce04259ae0ee9c5288949ff P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Refusing update from remote peer c62716bf66c24330860573535b8b9430: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:47.737201 20691 consensus_queue.cc:1048] T 29f92a880ce04259ae0ee9c5288949ff P c62716bf66c24330860573535b8b9430 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:20:47.810770  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:47.851049  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:47.887621 19868 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3aaced67782041fa9960c745301a96e7"
dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
mode: GRACEFUL
new_leader_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
 from {username='slave'} at 127.0.0.1:39648
I20260709 13:20:47.888319 19868 raft_consensus.cc:606] T 3aaced67782041fa9960c745301a96e7 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 LEADER]: Received request to transfer leadership to TS 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:47.891232 19868 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7e3099f1888c4ed18c30523bf9ec94f5"
dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
mode: GRACEFUL
new_leader_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
 from {username='slave'} at 127.0.0.1:39648
I20260709 13:20:47.891893 19868 raft_consensus.cc:606] T 7e3099f1888c4ed18c30523bf9ec94f5 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 LEADER]: Received request to transfer leadership to TS 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:47.905133 19945 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8abfa1aa35c44699960a8166fa79c41d"
dest_uuid: "c62716bf66c24330860573535b8b9430"
mode: GRACEFUL
new_leader_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
 from {username='slave'} at 127.0.0.1:38866
I20260709 13:20:47.905639 19945 raft_consensus.cc:606] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 [term 1 LEADER]: Received request to transfer leadership to TS 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:47.908116 19795 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e3ca37daad8341ad99cbba926f37232e"
dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
mode: GRACEFUL
new_leader_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
 from {username='slave'} at 127.0.0.1:40112
I20260709 13:20:47.908672 19795 raft_consensus.cc:606] T e3ca37daad8341ad99cbba926f37232e P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Received request to transfer leadership to TS 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:47.912175 19868 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ece79531bbd749d3b9300fe2fbeef9af"
dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
mode: GRACEFUL
new_leader_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
 from {username='slave'} at 127.0.0.1:39648
I20260709 13:20:47.912746 19868 raft_consensus.cc:606] T ece79531bbd749d3b9300fe2fbeef9af P b80b3a62d7d549a9a5c42def46769ac4 [term 1 LEADER]: Received request to transfer leadership to TS 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:47.914853 20703 raft_consensus.cc:993] T 7e3099f1888c4ed18c30523bf9ec94f5 P b80b3a62d7d549a9a5c42def46769ac4: : Instructing follower 8df38c4c53c44e94b36c1e8992476d7c to start an election
I20260709 13:20:47.915151  9682 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:20:47.915374 20710 raft_consensus.cc:1081] T 7e3099f1888c4ed18c30523bf9ec94f5 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 LEADER]: Signalling peer 8df38c4c53c44e94b36c1e8992476d7c to start an election
I20260709 13:20:47.916947  9682 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:20:47.917045 20545 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7e3099f1888c4ed18c30523bf9ec94f5"
dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
 from {username='slave'} at 127.0.0.1:33934
I20260709 13:20:47.917680 20545 raft_consensus.cc:493] T 7e3099f1888c4ed18c30523bf9ec94f5 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:47.918040 20545 raft_consensus.cc:3060] T 7e3099f1888c4ed18c30523bf9ec94f5 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:47.925132 20545 raft_consensus.cc:515] T 7e3099f1888c4ed18c30523bf9ec94f5 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:47.928906 20545 leader_election.cc:290] T 7e3099f1888c4ed18c30523bf9ec94f5 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 2 election: Requested vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:47.929960 19945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3099f1888c4ed18c30523bf9ec94f5" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:47.930274 19868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3099f1888c4ed18c30523bf9ec94f5" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
I20260709 13:20:47.930712 19945 raft_consensus.cc:3060] T 7e3099f1888c4ed18c30523bf9ec94f5 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:47.930908 19868 raft_consensus.cc:3055] T 7e3099f1888c4ed18c30523bf9ec94f5 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:20:47.931372 19868 raft_consensus.cc:740] T 7e3099f1888c4ed18c30523bf9ec94f5 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Running, Role: LEADER
I20260709 13:20:47.932331 19868 consensus_queue.cc:260] T 7e3099f1888c4ed18c30523bf9ec94f5 P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:47.933573 19868 raft_consensus.cc:3060] T 7e3099f1888c4ed18c30523bf9ec94f5 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:47.936581 19945 raft_consensus.cc:2468] T 7e3099f1888c4ed18c30523bf9ec94f5 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 2.
I20260709 13:20:47.937276 19944 raft_consensus.cc:1240] T 7e3099f1888c4ed18c30523bf9ec94f5 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Rejecting Update request from peer b80b3a62d7d549a9a5c42def46769ac4 for earlier term 1. Current term is 2. Ops: []
I20260709 13:20:47.937697 20494 leader_election.cc:304] T 7e3099f1888c4ed18c30523bf9ec94f5 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, c62716bf66c24330860573535b8b9430; no voters: 
I20260709 13:20:47.938651 20739 raft_consensus.cc:2804] T 7e3099f1888c4ed18c30523bf9ec94f5 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:20:47.939215 20739 raft_consensus.cc:697] T 7e3099f1888c4ed18c30523bf9ec94f5 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 LEADER]: Becoming Leader. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Running, Role: LEADER
I20260709 13:20:47.940090 19868 raft_consensus.cc:2468] T 7e3099f1888c4ed18c30523bf9ec94f5 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 2.
I20260709 13:20:47.940173 20739 consensus_queue.cc:237] T 7e3099f1888c4ed18c30523bf9ec94f5 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:47.951759 19623 catalog_manager.cc:5697] T 7e3099f1888c4ed18c30523bf9ec94f5 P 8df38c4c53c44e94b36c1e8992476d7c reported cstate change: term changed from 1 to 2, leader changed from b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131) to 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129). New cstate: current_term: 2 leader_uuid: "8df38c4c53c44e94b36c1e8992476d7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:48.100443 20710 raft_consensus.cc:993] T 3aaced67782041fa9960c745301a96e7 P b80b3a62d7d549a9a5c42def46769ac4: : Instructing follower 8df38c4c53c44e94b36c1e8992476d7c to start an election
I20260709 13:20:48.100942 20703 raft_consensus.cc:1081] T 3aaced67782041fa9960c745301a96e7 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 LEADER]: Signalling peer 8df38c4c53c44e94b36c1e8992476d7c to start an election
I20260709 13:20:48.102241 20545 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3aaced67782041fa9960c745301a96e7"
dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
 from {username='slave'} at 127.0.0.1:33934
I20260709 13:20:48.102635 20545 raft_consensus.cc:493] T 3aaced67782041fa9960c745301a96e7 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:48.102882 20545 raft_consensus.cc:3060] T 3aaced67782041fa9960c745301a96e7 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:48.109084 20545 raft_consensus.cc:515] T 3aaced67782041fa9960c745301a96e7 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:48.110852 20545 leader_election.cc:290] T 3aaced67782041fa9960c745301a96e7 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 2 election: Requested vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:48.111922 19868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaced67782041fa9960c745301a96e7" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
I20260709 13:20:48.112602 19945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaced67782041fa9960c745301a96e7" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:48.112655 19868 raft_consensus.cc:3055] T 3aaced67782041fa9960c745301a96e7 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:20:48.113229 19945 raft_consensus.cc:3060] T 3aaced67782041fa9960c745301a96e7 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:48.113433 19868 raft_consensus.cc:740] T 3aaced67782041fa9960c745301a96e7 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Running, Role: LEADER
I20260709 13:20:48.114305 19868 consensus_queue.cc:260] T 3aaced67782041fa9960c745301a96e7 P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:48.115633 19868 raft_consensus.cc:3060] T 3aaced67782041fa9960c745301a96e7 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:48.118506 19945 raft_consensus.cc:2468] T 3aaced67782041fa9960c745301a96e7 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 2.
I20260709 13:20:48.119551 20494 leader_election.cc:304] T 3aaced67782041fa9960c745301a96e7 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, c62716bf66c24330860573535b8b9430; no voters: 
I20260709 13:20:48.120271 20714 raft_consensus.cc:2804] T 3aaced67782041fa9960c745301a96e7 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:20:48.120183 19868 raft_consensus.cc:2468] T 3aaced67782041fa9960c745301a96e7 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 2.
I20260709 13:20:48.120976 20714 raft_consensus.cc:697] T 3aaced67782041fa9960c745301a96e7 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 LEADER]: Becoming Leader. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Running, Role: LEADER
I20260709 13:20:48.121906 20714 consensus_queue.cc:237] T 3aaced67782041fa9960c745301a96e7 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:48.132153 19623 catalog_manager.cc:5697] T 3aaced67782041fa9960c745301a96e7 P 8df38c4c53c44e94b36c1e8992476d7c reported cstate change: term changed from 1 to 2, leader changed from b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131) to 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129). New cstate: current_term: 2 leader_uuid: "8df38c4c53c44e94b36c1e8992476d7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:48.184630 20723 raft_consensus.cc:993] T e3ca37daad8341ad99cbba926f37232e P e90ffed3ba174ababeb21d9415d4d918: : Instructing follower 8df38c4c53c44e94b36c1e8992476d7c to start an election
I20260709 13:20:48.185279 20706 raft_consensus.cc:1081] T e3ca37daad8341ad99cbba926f37232e P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Signalling peer 8df38c4c53c44e94b36c1e8992476d7c to start an election
I20260709 13:20:48.186967 20545 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e3ca37daad8341ad99cbba926f37232e"
dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
 from {username='slave'} at 127.0.0.1:33928
I20260709 13:20:48.187503 20545 raft_consensus.cc:493] T e3ca37daad8341ad99cbba926f37232e P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:48.187870 20545 raft_consensus.cc:3060] T e3ca37daad8341ad99cbba926f37232e P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:48.194622 20545 raft_consensus.cc:515] T e3ca37daad8341ad99cbba926f37232e P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:48.196498 20545 leader_election.cc:290] T e3ca37daad8341ad99cbba926f37232e P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 2 election: Requested vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:48.197059 19945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3ca37daad8341ad99cbba926f37232e" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:48.197722 19945 raft_consensus.cc:3060] T e3ca37daad8341ad99cbba926f37232e P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:48.197880 19795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3ca37daad8341ad99cbba926f37232e" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
I20260709 13:20:48.198506 19795 raft_consensus.cc:3055] T e3ca37daad8341ad99cbba926f37232e P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:20:48.198720 19795 raft_consensus.cc:740] T e3ca37daad8341ad99cbba926f37232e P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Running, Role: LEADER
I20260709 13:20:48.199441 19795 consensus_queue.cc:260] T e3ca37daad8341ad99cbba926f37232e P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:48.200469 19795 raft_consensus.cc:3060] T e3ca37daad8341ad99cbba926f37232e P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:48.205191 19945 raft_consensus.cc:2468] T e3ca37daad8341ad99cbba926f37232e P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 2.
I20260709 13:20:48.206453 20494 leader_election.cc:304] T e3ca37daad8341ad99cbba926f37232e P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, c62716bf66c24330860573535b8b9430; no voters: 
I20260709 13:20:48.207444 20714 raft_consensus.cc:2804] T e3ca37daad8341ad99cbba926f37232e P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:20:48.206683 19795 raft_consensus.cc:2468] T e3ca37daad8341ad99cbba926f37232e P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 2.
I20260709 13:20:48.208031 20714 raft_consensus.cc:697] T e3ca37daad8341ad99cbba926f37232e P 8df38c4c53c44e94b36c1e8992476d7c [term 2 LEADER]: Becoming Leader. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Running, Role: LEADER
I20260709 13:20:48.209053 20714 consensus_queue.cc:237] T e3ca37daad8341ad99cbba926f37232e P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:48.221263 19622 catalog_manager.cc:5697] T e3ca37daad8341ad99cbba926f37232e P 8df38c4c53c44e94b36c1e8992476d7c reported cstate change: term changed from 1 to 2, leader changed from e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130) to 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129). New cstate: current_term: 2 leader_uuid: "8df38c4c53c44e94b36c1e8992476d7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:48.279182 20703 raft_consensus.cc:993] T ece79531bbd749d3b9300fe2fbeef9af P b80b3a62d7d549a9a5c42def46769ac4: : Instructing follower 8df38c4c53c44e94b36c1e8992476d7c to start an election
I20260709 13:20:48.279640 20710 raft_consensus.cc:1081] T ece79531bbd749d3b9300fe2fbeef9af P b80b3a62d7d549a9a5c42def46769ac4 [term 1 LEADER]: Signalling peer 8df38c4c53c44e94b36c1e8992476d7c to start an election
I20260709 13:20:48.280946 20545 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ece79531bbd749d3b9300fe2fbeef9af"
dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
 from {username='slave'} at 127.0.0.1:33934
I20260709 13:20:48.281350 20545 raft_consensus.cc:493] T ece79531bbd749d3b9300fe2fbeef9af P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:48.281571 20545 raft_consensus.cc:3060] T ece79531bbd749d3b9300fe2fbeef9af P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:48.286185 20545 raft_consensus.cc:515] T ece79531bbd749d3b9300fe2fbeef9af P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:48.287832 20545 leader_election.cc:290] T ece79531bbd749d3b9300fe2fbeef9af P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 2 election: Requested vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:48.288480 19868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece79531bbd749d3b9300fe2fbeef9af" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
I20260709 13:20:48.289113 19868 raft_consensus.cc:3055] T ece79531bbd749d3b9300fe2fbeef9af P b80b3a62d7d549a9a5c42def46769ac4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:20:48.289438 19868 raft_consensus.cc:740] T ece79531bbd749d3b9300fe2fbeef9af P b80b3a62d7d549a9a5c42def46769ac4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Running, Role: LEADER
I20260709 13:20:48.289237 19945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece79531bbd749d3b9300fe2fbeef9af" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:48.290005 19945 raft_consensus.cc:3060] T ece79531bbd749d3b9300fe2fbeef9af P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:48.290254 19868 consensus_queue.cc:260] T ece79531bbd749d3b9300fe2fbeef9af P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:48.291545 19868 raft_consensus.cc:3060] T ece79531bbd749d3b9300fe2fbeef9af P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:48.296902 19945 raft_consensus.cc:2468] T ece79531bbd749d3b9300fe2fbeef9af P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 2.
I20260709 13:20:48.298233 19868 raft_consensus.cc:2468] T ece79531bbd749d3b9300fe2fbeef9af P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 2.
I20260709 13:20:48.298167 20494 leader_election.cc:304] T ece79531bbd749d3b9300fe2fbeef9af P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, c62716bf66c24330860573535b8b9430; no voters: 
I20260709 13:20:48.299218 20715 raft_consensus.cc:2804] T ece79531bbd749d3b9300fe2fbeef9af P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:20:48.299798 20715 raft_consensus.cc:697] T ece79531bbd749d3b9300fe2fbeef9af P 8df38c4c53c44e94b36c1e8992476d7c [term 2 LEADER]: Becoming Leader. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Running, Role: LEADER
I20260709 13:20:48.300673 20715 consensus_queue.cc:237] T ece79531bbd749d3b9300fe2fbeef9af P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:48.310364 19622 catalog_manager.cc:5697] T ece79531bbd749d3b9300fe2fbeef9af P 8df38c4c53c44e94b36c1e8992476d7c reported cstate change: term changed from 1 to 2, leader changed from b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131) to 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129). New cstate: current_term: 2 leader_uuid: "8df38c4c53c44e94b36c1e8992476d7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:48.400844 20424 raft_consensus.cc:993] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430: : Instructing follower 8df38c4c53c44e94b36c1e8992476d7c to start an election
I20260709 13:20:48.401391 20691 raft_consensus.cc:1081] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 [term 1 LEADER]: Signalling peer 8df38c4c53c44e94b36c1e8992476d7c to start an election
I20260709 13:20:48.403582 20538 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8abfa1aa35c44699960a8166fa79c41d"
dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
 from {username='slave'} at 127.0.0.1:33950
I20260709 13:20:48.404166 20538 raft_consensus.cc:493] T 8abfa1aa35c44699960a8166fa79c41d P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:48.404891 20538 raft_consensus.cc:3060] T 8abfa1aa35c44699960a8166fa79c41d P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:48.411712 20538 raft_consensus.cc:515] T 8abfa1aa35c44699960a8166fa79c41d P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:48.413542 20538 leader_election.cc:290] T 8abfa1aa35c44699960a8166fa79c41d P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 2 election: Requested vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:48.414367 19945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8abfa1aa35c44699960a8166fa79c41d" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:48.414546 19795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8abfa1aa35c44699960a8166fa79c41d" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
I20260709 13:20:48.415083 19945 raft_consensus.cc:3055] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:20:48.415246 19795 raft_consensus.cc:3060] T 8abfa1aa35c44699960a8166fa79c41d P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:48.415511 19945 raft_consensus.cc:740] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c62716bf66c24330860573535b8b9430, State: Running, Role: LEADER
I20260709 13:20:48.416324 19945 consensus_queue.cc:260] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:48.417763 19945 raft_consensus.cc:3060] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:48.422055 19795 raft_consensus.cc:2468] T 8abfa1aa35c44699960a8166fa79c41d P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 2.
I20260709 13:20:48.423254 20494 leader_election.cc:304] T 8abfa1aa35c44699960a8166fa79c41d P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918; no voters: 
I20260709 13:20:48.423664 19945 raft_consensus.cc:2468] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 2.
I20260709 13:20:48.424021 20715 raft_consensus.cc:2804] T 8abfa1aa35c44699960a8166fa79c41d P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:20:48.424563 20715 raft_consensus.cc:697] T 8abfa1aa35c44699960a8166fa79c41d P 8df38c4c53c44e94b36c1e8992476d7c [term 2 LEADER]: Becoming Leader. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Running, Role: LEADER
I20260709 13:20:48.425460 20715 consensus_queue.cc:237] T 8abfa1aa35c44699960a8166fa79c41d P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:48.433792 19622 catalog_manager.cc:5697] T 8abfa1aa35c44699960a8166fa79c41d P 8df38c4c53c44e94b36c1e8992476d7c reported cstate change: term changed from 1 to 2, leader changed from c62716bf66c24330860573535b8b9430 (127.9.116.132) to 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129). New cstate: current_term: 2 leader_uuid: "8df38c4c53c44e94b36c1e8992476d7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:48.462395 19868 raft_consensus.cc:1275] T 7e3099f1888c4ed18c30523bf9ec94f5 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Refusing update from remote peer 8df38c4c53c44e94b36c1e8992476d7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:48.464067 20715 consensus_queue.cc:1048] T 7e3099f1888c4ed18c30523bf9ec94f5 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:20:48.475196 19945 raft_consensus.cc:1275] T 7e3099f1888c4ed18c30523bf9ec94f5 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Refusing update from remote peer 8df38c4c53c44e94b36c1e8992476d7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:48.476801 20715 consensus_queue.cc:1048] T 7e3099f1888c4ed18c30523bf9ec94f5 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:20:48.552246 19945 raft_consensus.cc:1275] T 3aaced67782041fa9960c745301a96e7 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Refusing update from remote peer 8df38c4c53c44e94b36c1e8992476d7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:48.553560 20715 consensus_queue.cc:1048] T 3aaced67782041fa9960c745301a96e7 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:20:48.572911 19868 raft_consensus.cc:1275] T 3aaced67782041fa9960c745301a96e7 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Refusing update from remote peer 8df38c4c53c44e94b36c1e8992476d7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:48.574576 20715 consensus_queue.cc:1048] T 3aaced67782041fa9960c745301a96e7 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:20:48.620100 19794 raft_consensus.cc:1275] T e3ca37daad8341ad99cbba926f37232e P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Refusing update from remote peer 8df38c4c53c44e94b36c1e8992476d7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:48.621711 20715 consensus_queue.cc:1048] T e3ca37daad8341ad99cbba926f37232e P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:20:48.624897 19945 raft_consensus.cc:1275] T e3ca37daad8341ad99cbba926f37232e P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Refusing update from remote peer 8df38c4c53c44e94b36c1e8992476d7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:48.627295 20715 consensus_queue.cc:1048] T e3ca37daad8341ad99cbba926f37232e P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:20:48.827629 19945 raft_consensus.cc:1275] T ece79531bbd749d3b9300fe2fbeef9af P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Refusing update from remote peer 8df38c4c53c44e94b36c1e8992476d7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:48.828958 20715 consensus_queue.cc:1048] T ece79531bbd749d3b9300fe2fbeef9af P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:20:48.842428 19868 raft_consensus.cc:1275] T ece79531bbd749d3b9300fe2fbeef9af P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Refusing update from remote peer 8df38c4c53c44e94b36c1e8992476d7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:48.845373 20717 consensus_queue.cc:1048] T ece79531bbd749d3b9300fe2fbeef9af P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:20:48.917790  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:48.955415  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:48.964164 19794 raft_consensus.cc:1275] T 8abfa1aa35c44699960a8166fa79c41d P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Refusing update from remote peer 8df38c4c53c44e94b36c1e8992476d7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:48.965569 20715 consensus_queue.cc:1048] T 8abfa1aa35c44699960a8166fa79c41d P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:20:48.976965 19945 raft_consensus.cc:1275] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Refusing update from remote peer 8df38c4c53c44e94b36c1e8992476d7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:48.981508 20715 consensus_queue.cc:1048] T 8abfa1aa35c44699960a8166fa79c41d P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:20:49.005641 19794 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "180230b77b7640b29d055ae8f4358ffe"
dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
mode: GRACEFUL
new_leader_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
 from {username='slave'} at 127.0.0.1:40112
I20260709 13:20:49.006268 19794 raft_consensus.cc:606] T 180230b77b7640b29d055ae8f4358ffe P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Received request to transfer leadership to TS 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:49.009018 19794 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2eef64a7d31b4e1e899aee0857cb0e07"
dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
mode: GRACEFUL
new_leader_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
 from {username='slave'} at 127.0.0.1:40112
I20260709 13:20:49.009545 19794 raft_consensus.cc:606] T 2eef64a7d31b4e1e899aee0857cb0e07 P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Received request to transfer leadership to TS 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:49.013088 19795 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5d9f7b34cb064bbe8a6938e4f92d57c5"
dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
mode: GRACEFUL
new_leader_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
 from {username='slave'} at 127.0.0.1:40112
I20260709 13:20:49.013669 19795 raft_consensus.cc:606] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Received request to transfer leadership to TS b80b3a62d7d549a9a5c42def46769ac4
I20260709 13:20:49.016469 19795 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9b9137146e3644a6b0026778530fa8a6"
dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
mode: GRACEFUL
new_leader_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
 from {username='slave'} at 127.0.0.1:40112
I20260709 13:20:49.017036 19795 raft_consensus.cc:606] T 9b9137146e3644a6b0026778530fa8a6 P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Received request to transfer leadership to TS 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:49.018307  9682 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:20:49.019836  9682 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:20:49.218933 20723 raft_consensus.cc:993] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P e90ffed3ba174ababeb21d9415d4d918: : Instructing follower b80b3a62d7d549a9a5c42def46769ac4 to start an election
I20260709 13:20:49.219560 20706 raft_consensus.cc:1081] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Signalling peer b80b3a62d7d549a9a5c42def46769ac4 to start an election
I20260709 13:20:49.221212 19871 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5d9f7b34cb064bbe8a6938e4f92d57c5"
dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
 from {username='slave'} at 127.0.0.1:40880
I20260709 13:20:49.221740 19871 raft_consensus.cc:493] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:49.222030 19871 raft_consensus.cc:3060] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:49.226938 19871 raft_consensus.cc:515] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:49.228693 19871 leader_election.cc:290] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 election: Requested vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), c62716bf66c24330860573535b8b9430 (127.9.116.132:42643)
I20260709 13:20:49.229539 19795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9f7b34cb064bbe8a6938e4f92d57c5" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
I20260709 13:20:49.229821 19944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9f7b34cb064bbe8a6938e4f92d57c5" candidate_uuid: "b80b3a62d7d549a9a5c42def46769ac4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:49.230180 19795 raft_consensus.cc:3055] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:20:49.230358 19944 raft_consensus.cc:3060] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:49.230542 19795 raft_consensus.cc:740] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Running, Role: LEADER
I20260709 13:20:49.231385 19795 consensus_queue.cc:260] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:49.232580 19795 raft_consensus.cc:3060] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:49.239061 19944 raft_consensus.cc:2468] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 2.
I20260709 13:20:49.239351 19795 raft_consensus.cc:2468] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b80b3a62d7d549a9a5c42def46769ac4 in term 2.
I20260709 13:20:49.241321 19839 leader_election.cc:304] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P b80b3a62d7d549a9a5c42def46769ac4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b80b3a62d7d549a9a5c42def46769ac4, e90ffed3ba174ababeb21d9415d4d918; no voters: 
I20260709 13:20:49.242448 20703 raft_consensus.cc:2804] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:20:49.242935 20703 raft_consensus.cc:697] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 LEADER]: Becoming Leader. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Running, Role: LEADER
I20260709 13:20:49.243743 20703 consensus_queue.cc:237] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } }
I20260709 13:20:49.255437 19623 catalog_manager.cc:5697] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P b80b3a62d7d549a9a5c42def46769ac4 reported cstate change: term changed from 1 to 2, leader changed from e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130) to b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131). New cstate: current_term: 2 leader_uuid: "b80b3a62d7d549a9a5c42def46769ac4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:49.301267 20706 raft_consensus.cc:993] T 180230b77b7640b29d055ae8f4358ffe P e90ffed3ba174ababeb21d9415d4d918: : Instructing follower 8df38c4c53c44e94b36c1e8992476d7c to start an election
I20260709 13:20:49.301822 20723 raft_consensus.cc:1081] T 180230b77b7640b29d055ae8f4358ffe P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Signalling peer 8df38c4c53c44e94b36c1e8992476d7c to start an election
I20260709 13:20:49.303295 20538 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "180230b77b7640b29d055ae8f4358ffe"
dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
 from {username='slave'} at 127.0.0.1:33928
I20260709 13:20:49.303843 20538 raft_consensus.cc:493] T 180230b77b7640b29d055ae8f4358ffe P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:49.304170 20538 raft_consensus.cc:3060] T 180230b77b7640b29d055ae8f4358ffe P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:49.310895 20538 raft_consensus.cc:515] T 180230b77b7640b29d055ae8f4358ffe P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:49.313261 20538 leader_election.cc:290] T 180230b77b7640b29d055ae8f4358ffe P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 2 election: Requested vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:49.313452 19944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "180230b77b7640b29d055ae8f4358ffe" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:49.314066 19944 raft_consensus.cc:3060] T 180230b77b7640b29d055ae8f4358ffe P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:49.314201 19795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "180230b77b7640b29d055ae8f4358ffe" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
I20260709 13:20:49.315337 19795 raft_consensus.cc:3055] T 180230b77b7640b29d055ae8f4358ffe P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:20:49.315672 19795 raft_consensus.cc:740] T 180230b77b7640b29d055ae8f4358ffe P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Running, Role: LEADER
I20260709 13:20:49.316329 19795 consensus_queue.cc:260] T 180230b77b7640b29d055ae8f4358ffe P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:49.317301 19795 raft_consensus.cc:3060] T 180230b77b7640b29d055ae8f4358ffe P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:49.319720 19944 raft_consensus.cc:2468] T 180230b77b7640b29d055ae8f4358ffe P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 2.
I20260709 13:20:49.320986 20494 leader_election.cc:304] T 180230b77b7640b29d055ae8f4358ffe P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, c62716bf66c24330860573535b8b9430; no voters: 
I20260709 13:20:49.321842 20715 raft_consensus.cc:2804] T 180230b77b7640b29d055ae8f4358ffe P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:20:49.322336 20715 raft_consensus.cc:697] T 180230b77b7640b29d055ae8f4358ffe P 8df38c4c53c44e94b36c1e8992476d7c [term 2 LEADER]: Becoming Leader. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Running, Role: LEADER
I20260709 13:20:49.322911 19795 raft_consensus.cc:2468] T 180230b77b7640b29d055ae8f4358ffe P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 2.
I20260709 13:20:49.323247 20715 consensus_queue.cc:237] T 180230b77b7640b29d055ae8f4358ffe P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:49.332512 19623 catalog_manager.cc:5697] T 180230b77b7640b29d055ae8f4358ffe P 8df38c4c53c44e94b36c1e8992476d7c reported cstate change: term changed from 1 to 2, leader changed from e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130) to 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129). New cstate: current_term: 2 leader_uuid: "8df38c4c53c44e94b36c1e8992476d7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:49.367507 20706 raft_consensus.cc:993] T 9b9137146e3644a6b0026778530fa8a6 P e90ffed3ba174ababeb21d9415d4d918: : Instructing follower 8df38c4c53c44e94b36c1e8992476d7c to start an election
I20260709 13:20:49.368012 20706 raft_consensus.cc:1081] T 9b9137146e3644a6b0026778530fa8a6 P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Signalling peer 8df38c4c53c44e94b36c1e8992476d7c to start an election
I20260709 13:20:49.369874 20538 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9b9137146e3644a6b0026778530fa8a6"
dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
 from {username='slave'} at 127.0.0.1:33928
I20260709 13:20:49.370380 20538 raft_consensus.cc:493] T 9b9137146e3644a6b0026778530fa8a6 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:49.370719 20538 raft_consensus.cc:3060] T 9b9137146e3644a6b0026778530fa8a6 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:49.376730 20538 raft_consensus.cc:515] T 9b9137146e3644a6b0026778530fa8a6 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:49.378329 20538 leader_election.cc:290] T 9b9137146e3644a6b0026778530fa8a6 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 2 election: Requested vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:49.379053 19944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9137146e3644a6b0026778530fa8a6" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:49.379714 19944 raft_consensus.cc:3060] T 9b9137146e3644a6b0026778530fa8a6 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:49.381186 19795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9137146e3644a6b0026778530fa8a6" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
I20260709 13:20:49.381835 19795 raft_consensus.cc:3055] T 9b9137146e3644a6b0026778530fa8a6 P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:20:49.382158 19795 raft_consensus.cc:740] T 9b9137146e3644a6b0026778530fa8a6 P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Running, Role: LEADER
I20260709 13:20:49.383003 19795 consensus_queue.cc:260] T 9b9137146e3644a6b0026778530fa8a6 P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:49.384310 19795 raft_consensus.cc:3060] T 9b9137146e3644a6b0026778530fa8a6 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:49.386466 19944 raft_consensus.cc:2468] T 9b9137146e3644a6b0026778530fa8a6 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 2.
I20260709 13:20:49.388767 20494 leader_election.cc:304] T 9b9137146e3644a6b0026778530fa8a6 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, c62716bf66c24330860573535b8b9430; no voters: 
I20260709 13:20:49.389979 20715 raft_consensus.cc:2804] T 9b9137146e3644a6b0026778530fa8a6 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:20:49.390445 20715 raft_consensus.cc:697] T 9b9137146e3644a6b0026778530fa8a6 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 LEADER]: Becoming Leader. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Running, Role: LEADER
I20260709 13:20:49.391091 19795 raft_consensus.cc:2468] T 9b9137146e3644a6b0026778530fa8a6 P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 2.
I20260709 13:20:49.392089 20715 consensus_queue.cc:237] T 9b9137146e3644a6b0026778530fa8a6 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:49.410025 19623 catalog_manager.cc:5697] T 9b9137146e3644a6b0026778530fa8a6 P 8df38c4c53c44e94b36c1e8992476d7c reported cstate change: term changed from 1 to 2, leader changed from e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130) to 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129). New cstate: current_term: 2 leader_uuid: "8df38c4c53c44e94b36c1e8992476d7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:49.439579 20776 raft_consensus.cc:993] T 2eef64a7d31b4e1e899aee0857cb0e07 P e90ffed3ba174ababeb21d9415d4d918: : Instructing follower 8df38c4c53c44e94b36c1e8992476d7c to start an election
I20260709 13:20:49.440616 20706 raft_consensus.cc:1081] T 2eef64a7d31b4e1e899aee0857cb0e07 P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Signalling peer 8df38c4c53c44e94b36c1e8992476d7c to start an election
I20260709 13:20:49.442256 20538 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2eef64a7d31b4e1e899aee0857cb0e07"
dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
 from {username='slave'} at 127.0.0.1:33928
I20260709 13:20:49.442761 20538 raft_consensus.cc:493] T 2eef64a7d31b4e1e899aee0857cb0e07 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:49.443122 20538 raft_consensus.cc:3060] T 2eef64a7d31b4e1e899aee0857cb0e07 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:49.449571 20538 raft_consensus.cc:515] T 2eef64a7d31b4e1e899aee0857cb0e07 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:49.451962 19945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eef64a7d31b4e1e899aee0857cb0e07" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c62716bf66c24330860573535b8b9430"
I20260709 13:20:49.452523 19945 raft_consensus.cc:3060] T 2eef64a7d31b4e1e899aee0857cb0e07 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:49.452721 19799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eef64a7d31b4e1e899aee0857cb0e07" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
I20260709 13:20:49.453285 19799 raft_consensus.cc:3055] T 2eef64a7d31b4e1e899aee0857cb0e07 P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:20:49.453624 19799 raft_consensus.cc:740] T 2eef64a7d31b4e1e899aee0857cb0e07 P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e90ffed3ba174ababeb21d9415d4d918, State: Running, Role: LEADER
I20260709 13:20:49.454380 19799 consensus_queue.cc:260] T 2eef64a7d31b4e1e899aee0857cb0e07 P e90ffed3ba174ababeb21d9415d4d918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:49.455315 19799 raft_consensus.cc:3060] T 2eef64a7d31b4e1e899aee0857cb0e07 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:49.460813 19945 raft_consensus.cc:2468] T 2eef64a7d31b4e1e899aee0857cb0e07 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 2.
I20260709 13:20:49.462369 20494 leader_election.cc:304] T 2eef64a7d31b4e1e899aee0857cb0e07 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, c62716bf66c24330860573535b8b9430; no voters: 
I20260709 13:20:49.463213 20715 raft_consensus.cc:2804] T 2eef64a7d31b4e1e899aee0857cb0e07 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:20:49.464749 20715 raft_consensus.cc:697] T 2eef64a7d31b4e1e899aee0857cb0e07 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 LEADER]: Becoming Leader. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Running, Role: LEADER
I20260709 13:20:49.452167 20538 leader_election.cc:290] T 2eef64a7d31b4e1e899aee0857cb0e07 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 2 election: Requested vote from peers c62716bf66c24330860573535b8b9430 (127.9.116.132:42643), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:49.464367 19799 raft_consensus.cc:2468] T 2eef64a7d31b4e1e899aee0857cb0e07 P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 2.
I20260709 13:20:49.465714 20715 consensus_queue.cc:237] T 2eef64a7d31b4e1e899aee0857cb0e07 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:49.478322 19623 catalog_manager.cc:5697] T 2eef64a7d31b4e1e899aee0857cb0e07 P 8df38c4c53c44e94b36c1e8992476d7c reported cstate change: term changed from 1 to 2, leader changed from e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130) to 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129). New cstate: current_term: 2 leader_uuid: "8df38c4c53c44e94b36c1e8992476d7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:49.639118 19799 raft_consensus.cc:1275] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Refusing update from remote peer b80b3a62d7d549a9a5c42def46769ac4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:49.640415 20710 consensus_queue.cc:1048] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:20:49.670647 19945 raft_consensus.cc:1275] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Refusing update from remote peer b80b3a62d7d549a9a5c42def46769ac4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:49.672349 20703 consensus_queue.cc:1048] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P b80b3a62d7d549a9a5c42def46769ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:20:49.853348 19944 raft_consensus.cc:1275] T 180230b77b7640b29d055ae8f4358ffe P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Refusing update from remote peer 8df38c4c53c44e94b36c1e8992476d7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:49.854507 20714 consensus_queue.cc:1048] T 180230b77b7640b29d055ae8f4358ffe P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:20:49.867808 19794 raft_consensus.cc:1275] T 180230b77b7640b29d055ae8f4358ffe P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Refusing update from remote peer 8df38c4c53c44e94b36c1e8992476d7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:49.869958 20714 consensus_queue.cc:1048] T 180230b77b7640b29d055ae8f4358ffe P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:20:49.882892 19945 raft_consensus.cc:1275] T 9b9137146e3644a6b0026778530fa8a6 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Refusing update from remote peer 8df38c4c53c44e94b36c1e8992476d7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:49.884320 20714 consensus_queue.cc:1048] T 9b9137146e3644a6b0026778530fa8a6 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:20:49.899451 19799 raft_consensus.cc:1275] T 9b9137146e3644a6b0026778530fa8a6 P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Refusing update from remote peer 8df38c4c53c44e94b36c1e8992476d7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:49.901010 20714 consensus_queue.cc:1048] T 9b9137146e3644a6b0026778530fa8a6 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:20:49.948625 19944 raft_consensus.cc:1275] T 2eef64a7d31b4e1e899aee0857cb0e07 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Refusing update from remote peer 8df38c4c53c44e94b36c1e8992476d7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:49.950081 20717 consensus_queue.cc:1048] T 2eef64a7d31b4e1e899aee0857cb0e07 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:20:49.961076 19799 raft_consensus.cc:1275] T 2eef64a7d31b4e1e899aee0857cb0e07 P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Refusing update from remote peer 8df38c4c53c44e94b36c1e8992476d7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:49.962833 20789 consensus_queue.cc:1048] T 2eef64a7d31b4e1e899aee0857cb0e07 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:20:50.020637  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:50.069432  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:50.190222 19868 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f820e93ffdcd428ea4adfca88589a850"
dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
mode: GRACEFUL
new_leader_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
 from {username='slave'} at 127.0.0.1:39648
I20260709 13:20:50.191960 19868 raft_consensus.cc:606] T f820e93ffdcd428ea4adfca88589a850 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 LEADER]: Received request to transfer leadership to TS 8df38c4c53c44e94b36c1e8992476d7c
I20260709 13:20:50.195202  9682 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:20:50.203083  9682 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:20:50.428056 20703 raft_consensus.cc:993] T f820e93ffdcd428ea4adfca88589a850 P b80b3a62d7d549a9a5c42def46769ac4: : Instructing follower 8df38c4c53c44e94b36c1e8992476d7c to start an election
I20260709 13:20:50.428599 20703 raft_consensus.cc:1081] T f820e93ffdcd428ea4adfca88589a850 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 LEADER]: Signalling peer 8df38c4c53c44e94b36c1e8992476d7c to start an election
I20260709 13:20:50.432067 20538 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f820e93ffdcd428ea4adfca88589a850"
dest_uuid: "8df38c4c53c44e94b36c1e8992476d7c"
 from {username='slave'} at 127.0.0.1:33934
I20260709 13:20:50.432744 20538 raft_consensus.cc:493] T f820e93ffdcd428ea4adfca88589a850 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:50.433251 20538 raft_consensus.cc:3060] T f820e93ffdcd428ea4adfca88589a850 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:50.449553 20538 raft_consensus.cc:515] T f820e93ffdcd428ea4adfca88589a850 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:50.452472 20538 leader_election.cc:290] T f820e93ffdcd428ea4adfca88589a850 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 2 election: Requested vote from peers e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319), b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249)
I20260709 13:20:50.456604 19868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f820e93ffdcd428ea4adfca88589a850" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b80b3a62d7d549a9a5c42def46769ac4"
I20260709 13:20:50.457680 19868 raft_consensus.cc:3055] T f820e93ffdcd428ea4adfca88589a850 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:20:50.458148 19868 raft_consensus.cc:740] T f820e93ffdcd428ea4adfca88589a850 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b80b3a62d7d549a9a5c42def46769ac4, State: Running, Role: LEADER
I20260709 13:20:50.459194 19799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f820e93ffdcd428ea4adfca88589a850" candidate_uuid: "8df38c4c53c44e94b36c1e8992476d7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e90ffed3ba174ababeb21d9415d4d918"
I20260709 13:20:50.459954 19799 raft_consensus.cc:3060] T f820e93ffdcd428ea4adfca88589a850 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:50.459225 19868 consensus_queue.cc:260] T f820e93ffdcd428ea4adfca88589a850 P b80b3a62d7d549a9a5c42def46769ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:50.465682 19868 raft_consensus.cc:3060] T f820e93ffdcd428ea4adfca88589a850 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:50.484229 19799 raft_consensus.cc:2468] T f820e93ffdcd428ea4adfca88589a850 P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 2.
I20260709 13:20:50.489084 20494 leader_election.cc:304] T f820e93ffdcd428ea4adfca88589a850 P 8df38c4c53c44e94b36c1e8992476d7c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918; no voters: 
I20260709 13:20:50.491376 20714 raft_consensus.cc:2804] T f820e93ffdcd428ea4adfca88589a850 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:20:50.491928 20714 raft_consensus.cc:697] T f820e93ffdcd428ea4adfca88589a850 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 LEADER]: Becoming Leader. State: Replica: 8df38c4c53c44e94b36c1e8992476d7c, State: Running, Role: LEADER
I20260709 13:20:50.483983 19868 raft_consensus.cc:2468] T f820e93ffdcd428ea4adfca88589a850 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df38c4c53c44e94b36c1e8992476d7c in term 2.
I20260709 13:20:50.492982 20714 consensus_queue.cc:237] T f820e93ffdcd428ea4adfca88589a850 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } }
I20260709 13:20:50.518432 19621 catalog_manager.cc:5697] T f820e93ffdcd428ea4adfca88589a850 P 8df38c4c53c44e94b36c1e8992476d7c reported cstate change: term changed from 1 to 2, leader changed from b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131) to 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129). New cstate: current_term: 2 leader_uuid: "8df38c4c53c44e94b36c1e8992476d7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:50.895659 19871 raft_consensus.cc:1275] T f820e93ffdcd428ea4adfca88589a850 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Refusing update from remote peer 8df38c4c53c44e94b36c1e8992476d7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:50.896903 20715 consensus_queue.cc:1048] T f820e93ffdcd428ea4adfca88589a850 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:20:50.915520 19795 raft_consensus.cc:1275] T f820e93ffdcd428ea4adfca88589a850 P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Refusing update from remote peer 8df38c4c53c44e94b36c1e8992476d7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:50.918283 20714 consensus_queue.cc:1048] T f820e93ffdcd428ea4adfca88589a850 P 8df38c4c53c44e94b36c1e8992476d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:20:51.204499  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:51.242347  9682 tablet_server.cc:179] TabletServer@127.9.116.129:41855 shutting down...
W20260709 13:20:51.250459 19840 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 85 similar messages]
W20260709 13:20:51.255404 19840 consensus_peers.cc:597] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P b80b3a62d7d549a9a5c42def46769ac4 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:51.274192 19914 consensus_peers.cc:597] T a323eb80c5084159803e7d4ea5d9c410 P c62716bf66c24330860573535b8b9430 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:51.276365 19766 consensus_peers.cc:597] T 984fa8de2ccb49359c2b475e4adfc1ce P e90ffed3ba174ababeb21d9415d4d918 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:51.299379 19766 consensus_peers.cc:597] T cb33acfa7a9f46ab95b0e77e1394f258 P e90ffed3ba174ababeb21d9415d4d918 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:51.311064  9682 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:20:51.312234  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:20:51.313143  9682 raft_consensus.cc:2243] T dc5c36b41652443a8130592df8a66eb9 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.313746  9682 raft_consensus.cc:2272] T dc5c36b41652443a8130592df8a66eb9 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:51.314908 19840 consensus_peers.cc:597] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:51.316785  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:20:51.317718  9682 raft_consensus.cc:2243] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:20:51.319000  9682 raft_consensus.cc:2272] T 7a5476f5d68d496eaedce93e80bff5e4 P 8df38c4c53c44e94b36c1e8992476d7c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.321878  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:20:51.322391  9682 raft_consensus.cc:2243] T cb33acfa7a9f46ab95b0e77e1394f258 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.322855  9682 raft_consensus.cc:2272] T cb33acfa7a9f46ab95b0e77e1394f258 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.325279  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:20:51.326177  9682 raft_consensus.cc:2243] T ad38a1519fb748588f5a322589274004 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.326639  9682 raft_consensus.cc:2272] T ad38a1519fb748588f5a322589274004 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:51.328755 19914 consensus_peers.cc:597] T e8cfec59c63741869c636df6356546d0 P c62716bf66c24330860573535b8b9430 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:51.330267  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:20:51.331532  9682 raft_consensus.cc:2243] T f820e93ffdcd428ea4adfca88589a850 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:20:51.332630  9682 raft_consensus.cc:2272] T f820e93ffdcd428ea4adfca88589a850 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:51.332793 19766 consensus_peers.cc:597] T d1da95785dbf4ef4bde6f5e6ab35b444 P e90ffed3ba174ababeb21d9415d4d918 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:51.336561  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:20:51.337791  9682 raft_consensus.cc:2243] T 8fd58e41ecee4779b3e72df3aea4463a P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.338385  9682 raft_consensus.cc:2272] T 8fd58e41ecee4779b3e72df3aea4463a P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.341809  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:20:51.342540  9682 raft_consensus.cc:2243] T 5b9511148896441a948b772784f62a3d P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.343048  9682 raft_consensus.cc:2272] T 5b9511148896441a948b772784f62a3d P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.346037  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:20:51.346961  9682 raft_consensus.cc:2243] T 9b9137146e3644a6b0026778530fa8a6 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:20:51.347909  9682 raft_consensus.cc:2272] T 9b9137146e3644a6b0026778530fa8a6 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:51.348034 19840 consensus_peers.cc:597] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:51.350100  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:20:51.351248  9682 raft_consensus.cc:2243] T 86964b8a61ce494f9096c4fefdc6efbf P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.351909  9682 raft_consensus.cc:2272] T 86964b8a61ce494f9096c4fefdc6efbf P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.355465  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:20:51.356178  9682 raft_consensus.cc:2243] T 0587017774214f24bdfa4a3da36e367e P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.356649  9682 raft_consensus.cc:2272] T 0587017774214f24bdfa4a3da36e367e P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.359333  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:20:51.359968  9682 raft_consensus.cc:2243] T 6551866a22dd4aaf8ee318b9d91b3a02 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.360585  9682 raft_consensus.cc:2272] T 6551866a22dd4aaf8ee318b9d91b3a02 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.363046  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:20:51.363649  9682 raft_consensus.cc:2243] T d762eace063c4e04821c749d21702c3f P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.364118  9682 raft_consensus.cc:2272] T d762eace063c4e04821c749d21702c3f P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.366449  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:20:51.367129  9682 raft_consensus.cc:2243] T bc04a038495940c1bbeda4a2e6f17b82 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:20:51.368077  9682 raft_consensus.cc:2272] T bc04a038495940c1bbeda4a2e6f17b82 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.371137  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:20:51.372138  9682 raft_consensus.cc:2243] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:20:51.373258  9682 raft_consensus.cc:2272] T 9b2e4bf3fb5c456dab400b096f7bd729 P 8df38c4c53c44e94b36c1e8992476d7c [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:51.373565 19766 consensus_peers.cc:597] T 8bdf042a063d4596970f49730ffa04b4 P e90ffed3ba174ababeb21d9415d4d918 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:51.376274  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:20:51.377000  9682 raft_consensus.cc:2243] T 8abfa1aa35c44699960a8166fa79c41d P 8df38c4c53c44e94b36c1e8992476d7c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:20:51.377995  9682 raft_consensus.cc:2272] T 8abfa1aa35c44699960a8166fa79c41d P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.380705  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:20:51.381580  9682 raft_consensus.cc:2243] T 7e3a9d11df99411d8f0edb0df6f14d87 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.382259  9682 raft_consensus.cc:2272] T 7e3a9d11df99411d8f0edb0df6f14d87 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.385604  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:20:51.386250  9682 raft_consensus.cc:2243] T 180230b77b7640b29d055ae8f4358ffe P 8df38c4c53c44e94b36c1e8992476d7c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:20:51.387324  9682 raft_consensus.cc:2272] T 180230b77b7640b29d055ae8f4358ffe P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.389539  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:20:51.390339  9682 raft_consensus.cc:2243] T 59fc86c7e9f648e3b3f51e936e2bcfec P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.390955  9682 raft_consensus.cc:2272] T 59fc86c7e9f648e3b3f51e936e2bcfec P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.393807  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:20:51.394330  9682 raft_consensus.cc:2243] T ece79531bbd749d3b9300fe2fbeef9af P 8df38c4c53c44e94b36c1e8992476d7c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:20:51.395191  9682 raft_consensus.cc:2272] T ece79531bbd749d3b9300fe2fbeef9af P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.397433  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:20:51.397953  9682 raft_consensus.cc:2243] T 0a9c6b64c8e04c4283d24ae6290a52ea P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.398384  9682 raft_consensus.cc:2272] T 0a9c6b64c8e04c4283d24ae6290a52ea P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.400910  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:20:51.401448  9682 raft_consensus.cc:2243] T a323eb80c5084159803e7d4ea5d9c410 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.401837  9682 raft_consensus.cc:2272] T a323eb80c5084159803e7d4ea5d9c410 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.404040  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:20:51.404665  9682 raft_consensus.cc:2243] T 8bdf042a063d4596970f49730ffa04b4 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.405129  9682 raft_consensus.cc:2272] T 8bdf042a063d4596970f49730ffa04b4 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:51.407943 19766 consensus_peers.cc:597] T d762eace063c4e04821c749d21702c3f P e90ffed3ba174ababeb21d9415d4d918 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:51.407990  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:20:51.408926  9682 raft_consensus.cc:2243] T e8cfec59c63741869c636df6356546d0 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.409520  9682 raft_consensus.cc:2272] T e8cfec59c63741869c636df6356546d0 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.414309  9682 tablet_replica.cc:333] stopping tablet replica
W20260709 13:20:51.414562 19840 consensus_peers.cc:597] T 0413f22bd6d7447ca630b75be496555f P b80b3a62d7d549a9a5c42def46769ac4 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:51.415143  9682 raft_consensus.cc:2243] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:20:51.416168  9682 raft_consensus.cc:2272] T d5f8619b1f9f46adaa41774af856d9b1 P 8df38c4c53c44e94b36c1e8992476d7c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.418313  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:20:51.418957  9682 raft_consensus.cc:2243] T af881b097b4245b1be6072579197cb6b P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.419387  9682 raft_consensus.cc:2272] T af881b097b4245b1be6072579197cb6b P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.421562  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:20:51.422127  9682 raft_consensus.cc:2243] T 63868398f3604f529519d926d2831f4e P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.422586  9682 raft_consensus.cc:2272] T 63868398f3604f529519d926d2831f4e P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.425074  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:20:51.425707  9682 raft_consensus.cc:2243] T 3aaced67782041fa9960c745301a96e7 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:20:51.426584  9682 raft_consensus.cc:2272] T 3aaced67782041fa9960c745301a96e7 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.428701  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:20:51.429268  9682 raft_consensus.cc:2243] T c393f9d18341477c97fac9a4af7e3adf P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.429716  9682 raft_consensus.cc:2272] T c393f9d18341477c97fac9a4af7e3adf P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.431744  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:20:51.432296  9682 raft_consensus.cc:2243] T 2eef64a7d31b4e1e899aee0857cb0e07 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:20:51.433141  9682 raft_consensus.cc:2272] T 2eef64a7d31b4e1e899aee0857cb0e07 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.435957  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:20:51.436796  9682 raft_consensus.cc:2243] T 7e3099f1888c4ed18c30523bf9ec94f5 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:20:51.438009  9682 raft_consensus.cc:2272] T 7e3099f1888c4ed18c30523bf9ec94f5 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:51.438333 19766 consensus_peers.cc:597] T 6551866a22dd4aaf8ee318b9d91b3a02 P e90ffed3ba174ababeb21d9415d4d918 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:51.440474  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:20:51.441327  9682 raft_consensus.cc:2243] T 353a546cf8c54671971c239502ef165a P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.441766  9682 raft_consensus.cc:2272] T 353a546cf8c54671971c239502ef165a P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.444111  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:20:51.444694  9682 raft_consensus.cc:2243] T d1da95785dbf4ef4bde6f5e6ab35b444 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.445116  9682 raft_consensus.cc:2272] T d1da95785dbf4ef4bde6f5e6ab35b444 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.447412  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:20:51.447993  9682 raft_consensus.cc:2243] T 2a411802fdaa42e7863236f0af595fe2 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.448582  9682 raft_consensus.cc:2272] T 2a411802fdaa42e7863236f0af595fe2 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.451822  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:20:51.452594  9682 raft_consensus.cc:2243] T 9f60d4b859264720a20f5670c3c71a6f P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.453104  9682 raft_consensus.cc:2272] T 9f60d4b859264720a20f5670c3c71a6f P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.456377  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:20:51.457182  9682 raft_consensus.cc:2243] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.457762  9682 raft_consensus.cc:2272] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:51.459582 19914 consensus_peers.cc:597] T ac3bca5bb7644a8fbc78df43ef3331a0 P c62716bf66c24330860573535b8b9430 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:51.460945  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:20:51.461647  9682 raft_consensus.cc:2243] T 984fa8de2ccb49359c2b475e4adfc1ce P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.462114  9682 raft_consensus.cc:2272] T 984fa8de2ccb49359c2b475e4adfc1ce P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.465935  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:20:51.466760  9682 raft_consensus.cc:2243] T 1cceb7f75d7040c2b98049590928b891 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:20:51.467303 19766 consensus_peers.cc:597] T 59fc86c7e9f648e3b3f51e936e2bcfec P e90ffed3ba174ababeb21d9415d4d918 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:51.467378  9682 raft_consensus.cc:2272] T 1cceb7f75d7040c2b98049590928b891 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:51.469761 19914 consensus_peers.cc:597] T 1cceb7f75d7040c2b98049590928b891 P c62716bf66c24330860573535b8b9430 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:51.472205  9682 tablet_replica.cc:333] stopping tablet replica
W20260709 13:20:51.472889 19766 consensus_peers.cc:597] T dc5c36b41652443a8130592df8a66eb9 P e90ffed3ba174ababeb21d9415d4d918 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:51.473057  9682 raft_consensus.cc:2243] T 0413f22bd6d7447ca630b75be496555f P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.473706  9682 raft_consensus.cc:2272] T 0413f22bd6d7447ca630b75be496555f P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.476545  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:20:51.477128  9682 raft_consensus.cc:2243] T ac3bca5bb7644a8fbc78df43ef3331a0 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.477629  9682 raft_consensus.cc:2272] T ac3bca5bb7644a8fbc78df43ef3331a0 P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.480212  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:20:51.480789  9682 raft_consensus.cc:2243] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.481204  9682 raft_consensus.cc:2272] T 19e83b82855d4129bb8525e6ff05316b P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.483809  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:20:51.484683  9682 raft_consensus.cc:2243] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:20:51.486202  9682 raft_consensus.cc:2272] T e804b7d8fd7e44f59516fb299bea7a2c P 8df38c4c53c44e94b36c1e8992476d7c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.489068  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:20:51.489673  9682 raft_consensus.cc:2243] T 563940ad96444ce59e393ea303cc3352 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.490160  9682 raft_consensus.cc:2272] T 563940ad96444ce59e393ea303cc3352 P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.492471  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:20:51.493031  9682 raft_consensus.cc:2243] T e3ca37daad8341ad99cbba926f37232e P 8df38c4c53c44e94b36c1e8992476d7c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:20:51.494063  9682 raft_consensus.cc:2272] T e3ca37daad8341ad99cbba926f37232e P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.496905  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:20:51.497622  9682 raft_consensus.cc:2243] T bd80f9f8533b4cd3b5861ec11917a03d P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.498238  9682 raft_consensus.cc:2272] T bd80f9f8533b4cd3b5861ec11917a03d P 8df38c4c53c44e94b36c1e8992476d7c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.500967  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:20:51.501447  9682 raft_consensus.cc:2243] T 6a066b96902f41dbb1511eccf26820ce P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.501933  9682 raft_consensus.cc:2272] T 6a066b96902f41dbb1511eccf26820ce P 8df38c4c53c44e94b36c1e8992476d7c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:51.505501 19914 consensus_peers.cc:597] T bd80f9f8533b4cd3b5861ec11917a03d P c62716bf66c24330860573535b8b9430 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:51.522984 19840 consensus_peers.cc:597] T 0587017774214f24bdfa4a3da36e367e P b80b3a62d7d549a9a5c42def46769ac4 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:51.562462 19840 consensus_peers.cc:597] T 7e3a9d11df99411d8f0edb0df6f14d87 P b80b3a62d7d549a9a5c42def46769ac4 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:51.590333 19840 consensus_peers.cc:597] T 19e83b82855d4129bb8525e6ff05316b P b80b3a62d7d549a9a5c42def46769ac4 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:41855: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:51.594597  9682 tablet_server.cc:196] TabletServer@127.9.116.129:41855 shutdown complete.
W20260709 13:20:51.599138 19914 consensus_peers.cc:597] T 63868398f3604f529519d926d2831f4e P c62716bf66c24330860573535b8b9430 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:41855: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:51.606329 19766 consensus_peers.cc:597] T 6a066b96902f41dbb1511eccf26820ce P e90ffed3ba174ababeb21d9415d4d918 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:41855: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:51.607110 19766 consensus_peers.cc:597] T 5b9511148896441a948b772784f62a3d P e90ffed3ba174ababeb21d9415d4d918 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:41855: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:51.614826 19914 consensus_peers.cc:597] T 353a546cf8c54671971c239502ef165a P c62716bf66c24330860573535b8b9430 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:41855: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:51.636572 19840 consensus_peers.cc:597] T 2a411802fdaa42e7863236f0af595fe2 P b80b3a62d7d549a9a5c42def46769ac4 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:41855: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:51.638871 19914 consensus_peers.cc:597] T 8fd58e41ecee4779b3e72df3aea4463a P c62716bf66c24330860573535b8b9430 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:41855: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:51.661194 19840 consensus_peers.cc:597] T c393f9d18341477c97fac9a4af7e3adf P b80b3a62d7d549a9a5c42def46769ac4 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:41855: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:51.663311 19914 consensus_peers.cc:597] T 86964b8a61ce494f9096c4fefdc6efbf P c62716bf66c24330860573535b8b9430 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:41855: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:51.679499 19840 consensus_peers.cc:597] T ad38a1519fb748588f5a322589274004 P b80b3a62d7d549a9a5c42def46769ac4 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:41855: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:51.697782 19766 consensus_peers.cc:597] T af881b097b4245b1be6072579197cb6b P e90ffed3ba174ababeb21d9415d4d918 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:41855: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:51.702486  9682 tablet_server.cc:179] TabletServer@127.9.116.130:0 shutting down...
W20260709 13:20:51.724462 19913 consensus_peers.cc:597] T 353a546cf8c54671971c239502ef165a P c62716bf66c24330860573535b8b9430 -> Peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Couldn't send request to peer e90ffed3ba174ababeb21d9415d4d918. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:51.764526 19839 consensus_peers.cc:597] T 19a59035bda747fdbc59b7e6914101dd P b80b3a62d7d549a9a5c42def46769ac4 -> Peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Couldn't send request to peer e90ffed3ba174ababeb21d9415d4d918. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:51.760689 19913 consensus_peers.cc:597] T a323eb80c5084159803e7d4ea5d9c410 P c62716bf66c24330860573535b8b9430 -> Peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Couldn't send request to peer e90ffed3ba174ababeb21d9415d4d918. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:51.770359 19913 consensus_peers.cc:597] T 63868398f3604f529519d926d2831f4e P c62716bf66c24330860573535b8b9430 -> Peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Couldn't send request to peer e90ffed3ba174ababeb21d9415d4d918. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:51.772244  9682 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:20:51.774138  9682 tablet_replica.cc:333] T cb33acfa7a9f46ab95b0e77e1394f258 P e90ffed3ba174ababeb21d9415d4d918: stopping tablet replica
W20260709 13:20:51.775090 19913 consensus_peers.cc:597] T 29f92a880ce04259ae0ee9c5288949ff P c62716bf66c24330860573535b8b9430 -> Peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Couldn't send request to peer e90ffed3ba174ababeb21d9415d4d918. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:51.775193  9682 raft_consensus.cc:2243] T cb33acfa7a9f46ab95b0e77e1394f258 P e90ffed3ba174ababeb21d9415d4d918 [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:20:51.775511 19840 consensus_peers.cc:597] T 9f60d4b859264720a20f5670c3c71a6f P b80b3a62d7d549a9a5c42def46769ac4 -> Peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Couldn't send request to peer 8df38c4c53c44e94b36c1e8992476d7c. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:41855: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:51.776162  9682 raft_consensus.cc:2272] T cb33acfa7a9f46ab95b0e77e1394f258 P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.778384  9682 tablet_replica.cc:333] T 6551866a22dd4aaf8ee318b9d91b3a02 P e90ffed3ba174ababeb21d9415d4d918: stopping tablet replica
I20260709 13:20:51.779093  9682 raft_consensus.cc:2243] T 6551866a22dd4aaf8ee318b9d91b3a02 P e90ffed3ba174ababeb21d9415d4d918 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:20:51.779963  9682 raft_consensus.cc:2272] T 6551866a22dd4aaf8ee318b9d91b3a02 P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.782059  9682 tablet_replica.cc:333] T 87364a1289ed4aa1b9b2a7a9951ed776 P e90ffed3ba174ababeb21d9415d4d918: stopping tablet replica
I20260709 13:20:51.782631  9682 raft_consensus.cc:2243] T 87364a1289ed4aa1b9b2a7a9951ed776 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.783103  9682 raft_consensus.cc:2272] T 87364a1289ed4aa1b9b2a7a9951ed776 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.785017  9682 tablet_replica.cc:333] T b832efa5a2e34b3894e33629110e6fef P e90ffed3ba174ababeb21d9415d4d918: stopping tablet replica
I20260709 13:20:51.785568  9682 raft_consensus.cc:2243] T b832efa5a2e34b3894e33629110e6fef P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.786082  9682 raft_consensus.cc:2272] T b832efa5a2e34b3894e33629110e6fef P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.788767  9682 tablet_replica.cc:333] T dc5c36b41652443a8130592df8a66eb9 P e90ffed3ba174ababeb21d9415d4d918: stopping tablet replica
I20260709 13:20:51.789628  9682 raft_consensus.cc:2243] T dc5c36b41652443a8130592df8a66eb9 P e90ffed3ba174ababeb21d9415d4d918 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:20:51.790772  9682 raft_consensus.cc:2272] T dc5c36b41652443a8130592df8a66eb9 P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:51.792351 19839 consensus_peers.cc:597] T 87364a1289ed4aa1b9b2a7a9951ed776 P b80b3a62d7d549a9a5c42def46769ac4 -> Peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Couldn't send request to peer e90ffed3ba174ababeb21d9415d4d918. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:51.793711  9682 tablet_replica.cc:333] T 1cceb7f75d7040c2b98049590928b891 P e90ffed3ba174ababeb21d9415d4d918: stopping tablet replica
I20260709 13:20:51.794646  9682 raft_consensus.cc:2243] T 1cceb7f75d7040c2b98049590928b891 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.795225  9682 raft_consensus.cc:2272] T 1cceb7f75d7040c2b98049590928b891 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.797600  9682 tablet_replica.cc:333] T 563940ad96444ce59e393ea303cc3352 P e90ffed3ba174ababeb21d9415d4d918: stopping tablet replica
I20260709 13:20:51.798206  9682 raft_consensus.cc:2243] T 563940ad96444ce59e393ea303cc3352 P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.798642  9682 raft_consensus.cc:2272] T 563940ad96444ce59e393ea303cc3352 P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.800904  9682 tablet_replica.cc:333] T d5f8619b1f9f46adaa41774af856d9b1 P e90ffed3ba174ababeb21d9415d4d918: stopping tablet replica
I20260709 13:20:51.801486  9682 raft_consensus.cc:2243] T d5f8619b1f9f46adaa41774af856d9b1 P e90ffed3ba174ababeb21d9415d4d918 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.801965  9682 raft_consensus.cc:2272] T d5f8619b1f9f46adaa41774af856d9b1 P e90ffed3ba174ababeb21d9415d4d918 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.804031  9682 tablet_replica.cc:333] T 1fa8a0db61be45c3aa42edb11db4c5ff P e90ffed3ba174ababeb21d9415d4d918: stopping tablet replica
I20260709 13:20:51.804617  9682 raft_consensus.cc:2243] T 1fa8a0db61be45c3aa42edb11db4c5ff P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.805055  9682 raft_consensus.cc:2272] T 1fa8a0db61be45c3aa42edb11db4c5ff P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.807585  9682 tablet_replica.cc:333] T 29f92a880ce04259ae0ee9c5288949ff P e90ffed3ba174ababeb21d9415d4d918: stopping tablet replica
I20260709 13:20:51.808434  9682 raft_consensus.cc:2243] T 29f92a880ce04259ae0ee9c5288949ff P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.808976  9682 raft_consensus.cc:2272] T 29f92a880ce04259ae0ee9c5288949ff P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:51.811632 19913 consensus_peers.cc:597] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P c62716bf66c24330860573535b8b9430 -> Peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Couldn't send request to peer e90ffed3ba174ababeb21d9415d4d918. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:51.811975  9682 tablet_replica.cc:333] T c393f9d18341477c97fac9a4af7e3adf P e90ffed3ba174ababeb21d9415d4d918: stopping tablet replica
I20260709 13:20:51.812745  9682 raft_consensus.cc:2243] T c393f9d18341477c97fac9a4af7e3adf P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:20:51.814595 19913 consensus_peers.cc:597] T 3cad7a5f46f9407a99ea6f2cc5a8322c P c62716bf66c24330860573535b8b9430 -> Peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Couldn't send request to peer e90ffed3ba174ababeb21d9415d4d918. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:51.815196  9682 raft_consensus.cc:2272] T c393f9d18341477c97fac9a4af7e3adf P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.818001  9682 tablet_replica.cc:333] T e3ca37daad8341ad99cbba926f37232e P e90ffed3ba174ababeb21d9415d4d918: stopping tablet replica
I20260709 13:20:51.818634  9682 raft_consensus.cc:2243] T e3ca37daad8341ad99cbba926f37232e P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.819307  9682 raft_consensus.cc:2272] T e3ca37daad8341ad99cbba926f37232e P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.821930  9682 tablet_replica.cc:333] T f012387a39df446bbd9dbea32dc03202 P e90ffed3ba174ababeb21d9415d4d918: stopping tablet replica
I20260709 13:20:51.822831  9682 raft_consensus.cc:2243] T f012387a39df446bbd9dbea32dc03202 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.823395  9682 raft_consensus.cc:2272] T f012387a39df446bbd9dbea32dc03202 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.826512  9682 tablet_replica.cc:333] T 8abfa1aa35c44699960a8166fa79c41d P e90ffed3ba174ababeb21d9415d4d918: stopping tablet replica
W20260709 13:20:51.827044 19839 consensus_peers.cc:597] T 1fa8a0db61be45c3aa42edb11db4c5ff P b80b3a62d7d549a9a5c42def46769ac4 -> Peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Couldn't send request to peer e90ffed3ba174ababeb21d9415d4d918. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:51.827384  9682 raft_consensus.cc:2243] T 8abfa1aa35c44699960a8166fa79c41d P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.827970  9682 raft_consensus.cc:2272] T 8abfa1aa35c44699960a8166fa79c41d P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.830637  9682 tablet_replica.cc:333] T 7e3a9d11df99411d8f0edb0df6f14d87 P e90ffed3ba174ababeb21d9415d4d918: stopping tablet replica
I20260709 13:20:51.831480  9682 raft_consensus.cc:2243] T 7e3a9d11df99411d8f0edb0df6f14d87 P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.831947  9682 raft_consensus.cc:2272] T 7e3a9d11df99411d8f0edb0df6f14d87 P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.834002  9682 tablet_replica.cc:333] T 0a9c6b64c8e04c4283d24ae6290a52ea P e90ffed3ba174ababeb21d9415d4d918: stopping tablet replica
I20260709 13:20:51.834609  9682 raft_consensus.cc:2243] T 0a9c6b64c8e04c4283d24ae6290a52ea P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.835076  9682 raft_consensus.cc:2272] T 0a9c6b64c8e04c4283d24ae6290a52ea P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.836923  9682 tablet_replica.cc:333] T 8bdf042a063d4596970f49730ffa04b4 P e90ffed3ba174ababeb21d9415d4d918: stopping tablet replica
I20260709 13:20:51.837483  9682 raft_consensus.cc:2243] T 8bdf042a063d4596970f49730ffa04b4 P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:51.838461  9682 raft_consensus.cc:2272] T 8bdf042a063d4596970f49730ffa04b4 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.840831  9682 tablet_replica.cc:333] T a323eb80c5084159803e7d4ea5d9c410 P e90ffed3ba174ababeb21d9415d4d918: stopping tablet replica
I20260709 13:20:51.841516  9682 raft_consensus.cc:2243] T a323eb80c5084159803e7d4ea5d9c410 P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.842033  9682 raft_consensus.cc:2272] T a323eb80c5084159803e7d4ea5d9c410 P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.844134  9682 tablet_replica.cc:333] T 65e7a5222efe43e49e1fbb3a3d7cf99c P e90ffed3ba174ababeb21d9415d4d918: stopping tablet replica
I20260709 13:20:51.844825  9682 raft_consensus.cc:2243] T 65e7a5222efe43e49e1fbb3a3d7cf99c P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:51.845664  9682 raft_consensus.cc:2272] T 65e7a5222efe43e49e1fbb3a3d7cf99c P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.848266  9682 tablet_replica.cc:333] T d762eace063c4e04821c749d21702c3f P e90ffed3ba174ababeb21d9415d4d918: stopping tablet replica
I20260709 13:20:51.849026  9682 raft_consensus.cc:2243] T d762eace063c4e04821c749d21702c3f P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:51.850133  9682 raft_consensus.cc:2272] T d762eace063c4e04821c749d21702c3f P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.852623  9682 tablet_replica.cc:333] T 0c709292a0304b3c84a777427a754bb5 P e90ffed3ba174ababeb21d9415d4d918: stopping tablet replica
I20260709 13:20:51.853127  9682 raft_consensus.cc:2243] T 0c709292a0304b3c84a777427a754bb5 P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:51.853885  9682 raft_consensus.cc:2272] T 0c709292a0304b3c84a777427a754bb5 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.855923  9682 tablet_replica.cc:333] T 9b9137146e3644a6b0026778530fa8a6 P e90ffed3ba174ababeb21d9415d4d918: stopping tablet replica
I20260709 13:20:51.856451  9682 raft_consensus.cc:2243] T 9b9137146e3644a6b0026778530fa8a6 P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.856848  9682 raft_consensus.cc:2272] T 9b9137146e3644a6b0026778530fa8a6 P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.859232  9682 tablet_replica.cc:333] T 85d493223fe04c939b6effd840955727 P e90ffed3ba174ababeb21d9415d4d918: stopping tablet replica
I20260709 13:20:51.860018  9682 raft_consensus.cc:2243] T 85d493223fe04c939b6effd840955727 P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:51.861065  9682 raft_consensus.cc:2272] T 85d493223fe04c939b6effd840955727 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:51.863253 19839 consensus_peers.cc:597] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4 -> Peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Couldn't send request to peer e90ffed3ba174ababeb21d9415d4d918. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:51.863826  9682 tablet_replica.cc:333] T 984fa8de2ccb49359c2b475e4adfc1ce P e90ffed3ba174ababeb21d9415d4d918: stopping tablet replica
I20260709 13:20:51.864722  9682 raft_consensus.cc:2243] T 984fa8de2ccb49359c2b475e4adfc1ce P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:51.865912  9682 raft_consensus.cc:2272] T 984fa8de2ccb49359c2b475e4adfc1ce P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.868904  9682 tablet_replica.cc:333] T ad38a1519fb748588f5a322589274004 P e90ffed3ba174ababeb21d9415d4d918: stopping tablet replica
I20260709 13:20:51.869660  9682 raft_consensus.cc:2243] T ad38a1519fb748588f5a322589274004 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.870275  9682 raft_consensus.cc:2272] T ad38a1519fb748588f5a322589274004 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.873155  9682 tablet_replica.cc:333] T 2eef64a7d31b4e1e899aee0857cb0e07 P e90ffed3ba174ababeb21d9415d4d918: stopping tablet replica
I20260709 13:20:51.873986  9682 raft_consensus.cc:2243] T 2eef64a7d31b4e1e899aee0857cb0e07 P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.874598  9682 raft_consensus.cc:2272] T 2eef64a7d31b4e1e899aee0857cb0e07 P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.877545  9682 tablet_replica.cc:333] T 3cad7a5f46f9407a99ea6f2cc5a8322c P e90ffed3ba174ababeb21d9415d4d918: stopping tablet replica
I20260709 13:20:51.878152  9682 raft_consensus.cc:2243] T 3cad7a5f46f9407a99ea6f2cc5a8322c P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.878628  9682 raft_consensus.cc:2272] T 3cad7a5f46f9407a99ea6f2cc5a8322c P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.880774  9682 tablet_replica.cc:333] T bd80f9f8533b4cd3b5861ec11917a03d P e90ffed3ba174ababeb21d9415d4d918: stopping tablet replica
I20260709 13:20:51.881366  9682 raft_consensus.cc:2243] T bd80f9f8533b4cd3b5861ec11917a03d P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.881834  9682 raft_consensus.cc:2272] T bd80f9f8533b4cd3b5861ec11917a03d P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.884464  9682 tablet_replica.cc:333] T 6a066b96902f41dbb1511eccf26820ce P e90ffed3ba174ababeb21d9415d4d918: stopping tablet replica
I20260709 13:20:51.885264  9682 raft_consensus.cc:2243] T 6a066b96902f41dbb1511eccf26820ce P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:51.886180  9682 raft_consensus.cc:2272] T 6a066b96902f41dbb1511eccf26820ce P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.888262  9682 tablet_replica.cc:333] T 5b9511148896441a948b772784f62a3d P e90ffed3ba174ababeb21d9415d4d918: stopping tablet replica
I20260709 13:20:51.888823  9682 raft_consensus.cc:2243] T 5b9511148896441a948b772784f62a3d P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:51.889639  9682 raft_consensus.cc:2272] T 5b9511148896441a948b772784f62a3d P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.891662  9682 tablet_replica.cc:333] T e9369d9b7f5649be932967d24689a936 P e90ffed3ba174ababeb21d9415d4d918: stopping tablet replica
I20260709 13:20:51.892302  9682 raft_consensus.cc:2243] T e9369d9b7f5649be932967d24689a936 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.892896  9682 raft_consensus.cc:2272] T e9369d9b7f5649be932967d24689a936 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.895939  9682 tablet_replica.cc:333] T 59fc86c7e9f648e3b3f51e936e2bcfec P e90ffed3ba174ababeb21d9415d4d918: stopping tablet replica
I20260709 13:20:51.896778  9682 raft_consensus.cc:2243] T 59fc86c7e9f648e3b3f51e936e2bcfec P e90ffed3ba174ababeb21d9415d4d918 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:20:51.897857  9682 raft_consensus.cc:2272] T 59fc86c7e9f648e3b3f51e936e2bcfec P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:51.897898 19913 consensus_peers.cc:597] T b832efa5a2e34b3894e33629110e6fef P c62716bf66c24330860573535b8b9430 -> Peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Couldn't send request to peer e90ffed3ba174ababeb21d9415d4d918. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:51.898983 19839 consensus_peers.cc:597] T 2a411802fdaa42e7863236f0af595fe2 P b80b3a62d7d549a9a5c42def46769ac4 -> Peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Couldn't send request to peer e90ffed3ba174ababeb21d9415d4d918. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:51.899946 19839 consensus_peers.cc:597] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4 -> Peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Couldn't send request to peer e90ffed3ba174ababeb21d9415d4d918. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:51.901086  9682 tablet_replica.cc:333] T 180230b77b7640b29d055ae8f4358ffe P e90ffed3ba174ababeb21d9415d4d918: stopping tablet replica
I20260709 13:20:51.901862  9682 raft_consensus.cc:2243] T 180230b77b7640b29d055ae8f4358ffe P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.902395  9682 raft_consensus.cc:2272] T 180230b77b7640b29d055ae8f4358ffe P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.905215  9682 tablet_replica.cc:333] T a36c11f368f34bf0aa4925a099ac1f91 P e90ffed3ba174ababeb21d9415d4d918: stopping tablet replica
I20260709 13:20:51.905846  9682 raft_consensus.cc:2243] T a36c11f368f34bf0aa4925a099ac1f91 P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:51.906746  9682 raft_consensus.cc:2272] T a36c11f368f34bf0aa4925a099ac1f91 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.909214  9682 tablet_replica.cc:333] T d1da95785dbf4ef4bde6f5e6ab35b444 P e90ffed3ba174ababeb21d9415d4d918: stopping tablet replica
I20260709 13:20:51.909796  9682 raft_consensus.cc:2243] T d1da95785dbf4ef4bde6f5e6ab35b444 P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:51.910612  9682 raft_consensus.cc:2272] T d1da95785dbf4ef4bde6f5e6ab35b444 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.912825  9682 tablet_replica.cc:333] T 353a546cf8c54671971c239502ef165a P e90ffed3ba174ababeb21d9415d4d918: stopping tablet replica
I20260709 13:20:51.913365  9682 raft_consensus.cc:2243] T 353a546cf8c54671971c239502ef165a P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.913820  9682 raft_consensus.cc:2272] T 353a546cf8c54671971c239502ef165a P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.915900  9682 tablet_replica.cc:333] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P e90ffed3ba174ababeb21d9415d4d918: stopping tablet replica
I20260709 13:20:51.916523  9682 raft_consensus.cc:2243] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.916967  9682 raft_consensus.cc:2272] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.918915  9682 tablet_replica.cc:333] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P e90ffed3ba174ababeb21d9415d4d918: stopping tablet replica
I20260709 13:20:51.919440  9682 raft_consensus.cc:2243] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.919890  9682 raft_consensus.cc:2272] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.921780  9682 tablet_replica.cc:333] T 63868398f3604f529519d926d2831f4e P e90ffed3ba174ababeb21d9415d4d918: stopping tablet replica
I20260709 13:20:51.922307  9682 raft_consensus.cc:2243] T 63868398f3604f529519d926d2831f4e P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.922729  9682 raft_consensus.cc:2272] T 63868398f3604f529519d926d2831f4e P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.924748  9682 tablet_replica.cc:333] T af881b097b4245b1be6072579197cb6b P e90ffed3ba174ababeb21d9415d4d918: stopping tablet replica
I20260709 13:20:51.925272  9682 raft_consensus.cc:2243] T af881b097b4245b1be6072579197cb6b P e90ffed3ba174ababeb21d9415d4d918 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:51.926220  9682 raft_consensus.cc:2272] T af881b097b4245b1be6072579197cb6b P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.928354  9682 tablet_replica.cc:333] T e8cfec59c63741869c636df6356546d0 P e90ffed3ba174ababeb21d9415d4d918: stopping tablet replica
I20260709 13:20:51.928896  9682 raft_consensus.cc:2243] T e8cfec59c63741869c636df6356546d0 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.929373  9682 raft_consensus.cc:2272] T e8cfec59c63741869c636df6356546d0 P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.931407  9682 tablet_replica.cc:333] T 19a59035bda747fdbc59b7e6914101dd P e90ffed3ba174ababeb21d9415d4d918: stopping tablet replica
I20260709 13:20:51.931965  9682 raft_consensus.cc:2243] T 19a59035bda747fdbc59b7e6914101dd P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.932428  9682 raft_consensus.cc:2272] T 19a59035bda747fdbc59b7e6914101dd P e90ffed3ba174ababeb21d9415d4d918 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.934507  9682 tablet_replica.cc:333] T 2a411802fdaa42e7863236f0af595fe2 P e90ffed3ba174ababeb21d9415d4d918: stopping tablet replica
I20260709 13:20:51.935060  9682 raft_consensus.cc:2243] T 2a411802fdaa42e7863236f0af595fe2 P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.935515  9682 raft_consensus.cc:2272] T 2a411802fdaa42e7863236f0af595fe2 P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:51.937567  9682 tablet_replica.cc:333] T f820e93ffdcd428ea4adfca88589a850 P e90ffed3ba174ababeb21d9415d4d918: stopping tablet replica
I20260709 13:20:51.938090  9682 raft_consensus.cc:2243] T f820e93ffdcd428ea4adfca88589a850 P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:51.938535  9682 raft_consensus.cc:2272] T f820e93ffdcd428ea4adfca88589a850 P e90ffed3ba174ababeb21d9415d4d918 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:51.960493 19913 consensus_peers.cc:597] T f012387a39df446bbd9dbea32dc03202 P c62716bf66c24330860573535b8b9430 -> Peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Couldn't send request to peer e90ffed3ba174ababeb21d9415d4d918. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:51.987246 19839 consensus_peers.cc:597] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P b80b3a62d7d549a9a5c42def46769ac4 -> Peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Couldn't send request to peer e90ffed3ba174ababeb21d9415d4d918. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:52.026170  9682 tablet_server.cc:196] TabletServer@127.9.116.130:0 shutdown complete.
W20260709 13:20:52.037335 19913 consensus_peers.cc:597] T e9369d9b7f5649be932967d24689a936 P c62716bf66c24330860573535b8b9430 -> Peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Couldn't send request to peer e90ffed3ba174ababeb21d9415d4d918. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:43319: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:52.040503 19913 consensus_peers.cc:597] T e8cfec59c63741869c636df6356546d0 P c62716bf66c24330860573535b8b9430 -> Peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Couldn't send request to peer e90ffed3ba174ababeb21d9415d4d918. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:43319: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:52.084367 19839 consensus_peers.cc:597] T ad38a1519fb748588f5a322589274004 P b80b3a62d7d549a9a5c42def46769ac4 -> Peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Couldn't send request to peer e90ffed3ba174ababeb21d9415d4d918. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:43319: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:52.093883 19839 consensus_peers.cc:597] T c393f9d18341477c97fac9a4af7e3adf P b80b3a62d7d549a9a5c42def46769ac4 -> Peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Couldn't send request to peer e90ffed3ba174ababeb21d9415d4d918. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:43319: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:52.113646  9682 tablet_server.cc:179] TabletServer@127.9.116.131:0 shutting down...
W20260709 13:20:52.130923 19913 consensus_peers.cc:597] T bd80f9f8533b4cd3b5861ec11917a03d P c62716bf66c24330860573535b8b9430 -> Peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Couldn't send request to peer e90ffed3ba174ababeb21d9415d4d918. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:43319: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:52.134233 19913 consensus_peers.cc:597] T 1cceb7f75d7040c2b98049590928b891 P c62716bf66c24330860573535b8b9430 -> Peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Couldn't send request to peer e90ffed3ba174ababeb21d9415d4d918. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:43319: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:52.150743 19839 consensus_peers.cc:597] T 7e3a9d11df99411d8f0edb0df6f14d87 P b80b3a62d7d549a9a5c42def46769ac4 -> Peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Couldn't send request to peer e90ffed3ba174ababeb21d9415d4d918. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:43319: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:52.177547 19913 consensus_peers.cc:597] T e9369d9b7f5649be932967d24689a936 P c62716bf66c24330860573535b8b9430 -> Peer b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249): Couldn't send request to peer b80b3a62d7d549a9a5c42def46769ac4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:52.187834  9682 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:20:52.188838  9682 tablet_replica.cc:333] T 8fd58e41ecee4779b3e72df3aea4463a P b80b3a62d7d549a9a5c42def46769ac4: stopping tablet replica
I20260709 13:20:52.189729  9682 raft_consensus.cc:2243] T 8fd58e41ecee4779b3e72df3aea4463a P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.190361  9682 raft_consensus.cc:2272] T 8fd58e41ecee4779b3e72df3aea4463a P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.193457  9682 tablet_replica.cc:333] T cb33acfa7a9f46ab95b0e77e1394f258 P b80b3a62d7d549a9a5c42def46769ac4: stopping tablet replica
I20260709 13:20:52.194347  9682 raft_consensus.cc:2243] T cb33acfa7a9f46ab95b0e77e1394f258 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.195030  9682 raft_consensus.cc:2272] T cb33acfa7a9f46ab95b0e77e1394f258 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.197496  9682 tablet_replica.cc:333] T 87364a1289ed4aa1b9b2a7a9951ed776 P b80b3a62d7d549a9a5c42def46769ac4: stopping tablet replica
I20260709 13:20:52.197993  9682 raft_consensus.cc:2243] T 87364a1289ed4aa1b9b2a7a9951ed776 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:52.198700  9682 raft_consensus.cc:2272] T 87364a1289ed4aa1b9b2a7a9951ed776 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.200771  9682 tablet_replica.cc:333] T 59fc86c7e9f648e3b3f51e936e2bcfec P b80b3a62d7d549a9a5c42def46769ac4: stopping tablet replica
I20260709 13:20:52.201372  9682 raft_consensus.cc:2243] T 59fc86c7e9f648e3b3f51e936e2bcfec P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.201823  9682 raft_consensus.cc:2272] T 59fc86c7e9f648e3b3f51e936e2bcfec P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.203789  9682 tablet_replica.cc:333] T ece79531bbd749d3b9300fe2fbeef9af P b80b3a62d7d549a9a5c42def46769ac4: stopping tablet replica
I20260709 13:20:52.204478  9682 raft_consensus.cc:2243] T ece79531bbd749d3b9300fe2fbeef9af P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.205003  9682 raft_consensus.cc:2272] T ece79531bbd749d3b9300fe2fbeef9af P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.207856  9682 tablet_replica.cc:333] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4: stopping tablet replica
I20260709 13:20:52.208630  9682 raft_consensus.cc:2243] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.209200  9682 raft_consensus.cc:2272] T 7a5476f5d68d496eaedce93e80bff5e4 P b80b3a62d7d549a9a5c42def46769ac4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.211862  9682 tablet_replica.cc:333] T dc5c36b41652443a8130592df8a66eb9 P b80b3a62d7d549a9a5c42def46769ac4: stopping tablet replica
I20260709 13:20:52.212620  9682 raft_consensus.cc:2243] T dc5c36b41652443a8130592df8a66eb9 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.213174  9682 raft_consensus.cc:2272] T dc5c36b41652443a8130592df8a66eb9 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.215736  9682 tablet_replica.cc:333] T b832efa5a2e34b3894e33629110e6fef P b80b3a62d7d549a9a5c42def46769ac4: stopping tablet replica
I20260709 13:20:52.216224  9682 raft_consensus.cc:2243] T b832efa5a2e34b3894e33629110e6fef P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.216578  9682 raft_consensus.cc:2272] T b832efa5a2e34b3894e33629110e6fef P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.218489  9682 tablet_replica.cc:333] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4: stopping tablet replica
I20260709 13:20:52.219152  9682 raft_consensus.cc:2243] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:20:52.220152  9682 raft_consensus.cc:2272] T 0a9c6b64c8e04c4283d24ae6290a52ea P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.222954  9682 tablet_replica.cc:333] T 8bdf042a063d4596970f49730ffa04b4 P b80b3a62d7d549a9a5c42def46769ac4: stopping tablet replica
W20260709 13:20:52.223664 19913 consensus_peers.cc:597] T 8fd58e41ecee4779b3e72df3aea4463a P c62716bf66c24330860573535b8b9430 -> Peer b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249): Couldn't send request to peer b80b3a62d7d549a9a5c42def46769ac4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:52.223698  9682 raft_consensus.cc:2243] T 8bdf042a063d4596970f49730ffa04b4 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.224416  9682 raft_consensus.cc:2272] T 8bdf042a063d4596970f49730ffa04b4 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.226627  9682 tablet_replica.cc:333] T 1fa8a0db61be45c3aa42edb11db4c5ff P b80b3a62d7d549a9a5c42def46769ac4: stopping tablet replica
I20260709 13:20:52.227278  9682 raft_consensus.cc:2243] T 1fa8a0db61be45c3aa42edb11db4c5ff P b80b3a62d7d549a9a5c42def46769ac4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:52.228099  9682 raft_consensus.cc:2272] T 1fa8a0db61be45c3aa42edb11db4c5ff P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.230125  9682 tablet_replica.cc:333] T f012387a39df446bbd9dbea32dc03202 P b80b3a62d7d549a9a5c42def46769ac4: stopping tablet replica
I20260709 13:20:52.230700  9682 raft_consensus.cc:2243] T f012387a39df446bbd9dbea32dc03202 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.231189  9682 raft_consensus.cc:2272] T f012387a39df446bbd9dbea32dc03202 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.233276  9682 tablet_replica.cc:333] T 0c709292a0304b3c84a777427a754bb5 P b80b3a62d7d549a9a5c42def46769ac4: stopping tablet replica
I20260709 13:20:52.233851  9682 raft_consensus.cc:2243] T 0c709292a0304b3c84a777427a754bb5 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.234252  9682 raft_consensus.cc:2272] T 0c709292a0304b3c84a777427a754bb5 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.236243  9682 tablet_replica.cc:333] T 0413f22bd6d7447ca630b75be496555f P b80b3a62d7d549a9a5c42def46769ac4: stopping tablet replica
I20260709 13:20:52.236824  9682 raft_consensus.cc:2243] T 0413f22bd6d7447ca630b75be496555f P b80b3a62d7d549a9a5c42def46769ac4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:52.237543  9682 raft_consensus.cc:2272] T 0413f22bd6d7447ca630b75be496555f P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.239574  9682 tablet_replica.cc:333] T 29f92a880ce04259ae0ee9c5288949ff P b80b3a62d7d549a9a5c42def46769ac4: stopping tablet replica
I20260709 13:20:52.240142  9682 raft_consensus.cc:2243] T 29f92a880ce04259ae0ee9c5288949ff P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.240592  9682 raft_consensus.cc:2272] T 29f92a880ce04259ae0ee9c5288949ff P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.242571  9682 tablet_replica.cc:333] T c393f9d18341477c97fac9a4af7e3adf P b80b3a62d7d549a9a5c42def46769ac4: stopping tablet replica
I20260709 13:20:52.243196  9682 raft_consensus.cc:2243] T c393f9d18341477c97fac9a4af7e3adf P b80b3a62d7d549a9a5c42def46769ac4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:20:52.243997  9682 raft_consensus.cc:2272] T c393f9d18341477c97fac9a4af7e3adf P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.245983  9682 tablet_replica.cc:333] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4: stopping tablet replica
I20260709 13:20:52.246577  9682 raft_consensus.cc:2243] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:20:52.247399  9682 raft_consensus.cc:2272] T 563940ad96444ce59e393ea303cc3352 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.249464  9682 tablet_replica.cc:333] T e804b7d8fd7e44f59516fb299bea7a2c P b80b3a62d7d549a9a5c42def46769ac4: stopping tablet replica
I20260709 13:20:52.250017  9682 raft_consensus.cc:2243] T e804b7d8fd7e44f59516fb299bea7a2c P b80b3a62d7d549a9a5c42def46769ac4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.250452  9682 raft_consensus.cc:2272] T e804b7d8fd7e44f59516fb299bea7a2c P b80b3a62d7d549a9a5c42def46769ac4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.252463  9682 tablet_replica.cc:333] T 19e83b82855d4129bb8525e6ff05316b P b80b3a62d7d549a9a5c42def46769ac4: stopping tablet replica
I20260709 13:20:52.253021  9682 raft_consensus.cc:2243] T 19e83b82855d4129bb8525e6ff05316b P b80b3a62d7d549a9a5c42def46769ac4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:20:52.253793  9682 raft_consensus.cc:2272] T 19e83b82855d4129bb8525e6ff05316b P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.255770  9682 tablet_replica.cc:333] T 7e3a9d11df99411d8f0edb0df6f14d87 P b80b3a62d7d549a9a5c42def46769ac4: stopping tablet replica
I20260709 13:20:52.256309  9682 raft_consensus.cc:2243] T 7e3a9d11df99411d8f0edb0df6f14d87 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:20:52.257059  9682 raft_consensus.cc:2272] T 7e3a9d11df99411d8f0edb0df6f14d87 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.258973  9682 tablet_replica.cc:333] T 9f60d4b859264720a20f5670c3c71a6f P b80b3a62d7d549a9a5c42def46769ac4: stopping tablet replica
I20260709 13:20:52.259574  9682 raft_consensus.cc:2243] T 9f60d4b859264720a20f5670c3c71a6f P b80b3a62d7d549a9a5c42def46769ac4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:52.260355  9682 raft_consensus.cc:2272] T 9f60d4b859264720a20f5670c3c71a6f P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.263348  9682 tablet_replica.cc:333] T e9369d9b7f5649be932967d24689a936 P b80b3a62d7d549a9a5c42def46769ac4: stopping tablet replica
I20260709 13:20:52.263881  9682 raft_consensus.cc:2243] T e9369d9b7f5649be932967d24689a936 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.264302  9682 raft_consensus.cc:2272] T e9369d9b7f5649be932967d24689a936 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.266004  9682 tablet_replica.cc:333] T 65e7a5222efe43e49e1fbb3a3d7cf99c P b80b3a62d7d549a9a5c42def46769ac4: stopping tablet replica
I20260709 13:20:52.266700  9682 raft_consensus.cc:2243] T 65e7a5222efe43e49e1fbb3a3d7cf99c P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.267154  9682 raft_consensus.cc:2272] T 65e7a5222efe43e49e1fbb3a3d7cf99c P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.269012  9682 tablet_replica.cc:333] T 86964b8a61ce494f9096c4fefdc6efbf P b80b3a62d7d549a9a5c42def46769ac4: stopping tablet replica
I20260709 13:20:52.269549  9682 raft_consensus.cc:2243] T 86964b8a61ce494f9096c4fefdc6efbf P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.269977  9682 raft_consensus.cc:2272] T 86964b8a61ce494f9096c4fefdc6efbf P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.272220  9682 tablet_replica.cc:333] T 0587017774214f24bdfa4a3da36e367e P b80b3a62d7d549a9a5c42def46769ac4: stopping tablet replica
I20260709 13:20:52.272842  9682 raft_consensus.cc:2243] T 0587017774214f24bdfa4a3da36e367e P b80b3a62d7d549a9a5c42def46769ac4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:52.273742  9682 raft_consensus.cc:2272] T 0587017774214f24bdfa4a3da36e367e P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.275979  9682 tablet_replica.cc:333] T 984fa8de2ccb49359c2b475e4adfc1ce P b80b3a62d7d549a9a5c42def46769ac4: stopping tablet replica
I20260709 13:20:52.276544  9682 raft_consensus.cc:2243] T 984fa8de2ccb49359c2b475e4adfc1ce P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.276921  9682 raft_consensus.cc:2272] T 984fa8de2ccb49359c2b475e4adfc1ce P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.278924  9682 tablet_replica.cc:333] T 6a066b96902f41dbb1511eccf26820ce P b80b3a62d7d549a9a5c42def46769ac4: stopping tablet replica
I20260709 13:20:52.279438  9682 raft_consensus.cc:2243] T 6a066b96902f41dbb1511eccf26820ce P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.279812  9682 raft_consensus.cc:2272] T 6a066b96902f41dbb1511eccf26820ce P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.281651  9682 tablet_replica.cc:333] T ad38a1519fb748588f5a322589274004 P b80b3a62d7d549a9a5c42def46769ac4: stopping tablet replica
I20260709 13:20:52.282112  9682 raft_consensus.cc:2243] T ad38a1519fb748588f5a322589274004 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:52.282850  9682 raft_consensus.cc:2272] T ad38a1519fb748588f5a322589274004 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.284833  9682 tablet_replica.cc:333] T 7e3099f1888c4ed18c30523bf9ec94f5 P b80b3a62d7d549a9a5c42def46769ac4: stopping tablet replica
I20260709 13:20:52.285501  9682 raft_consensus.cc:2243] T 7e3099f1888c4ed18c30523bf9ec94f5 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.286018  9682 raft_consensus.cc:2272] T 7e3099f1888c4ed18c30523bf9ec94f5 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.288553  9682 tablet_replica.cc:333] T 3cad7a5f46f9407a99ea6f2cc5a8322c P b80b3a62d7d549a9a5c42def46769ac4: stopping tablet replica
I20260709 13:20:52.289310  9682 raft_consensus.cc:2243] T 3cad7a5f46f9407a99ea6f2cc5a8322c P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.289757  9682 raft_consensus.cc:2272] T 3cad7a5f46f9407a99ea6f2cc5a8322c P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.291806  9682 tablet_replica.cc:333] T ac3bca5bb7644a8fbc78df43ef3331a0 P b80b3a62d7d549a9a5c42def46769ac4: stopping tablet replica
I20260709 13:20:52.292332  9682 raft_consensus.cc:2243] T ac3bca5bb7644a8fbc78df43ef3331a0 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.292753  9682 raft_consensus.cc:2272] T ac3bca5bb7644a8fbc78df43ef3331a0 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.294610  9682 tablet_replica.cc:333] T a36c11f368f34bf0aa4925a099ac1f91 P b80b3a62d7d549a9a5c42def46769ac4: stopping tablet replica
I20260709 13:20:52.295150  9682 raft_consensus.cc:2243] T a36c11f368f34bf0aa4925a099ac1f91 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.295571  9682 raft_consensus.cc:2272] T a36c11f368f34bf0aa4925a099ac1f91 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.297649  9682 tablet_replica.cc:333] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P b80b3a62d7d549a9a5c42def46769ac4: stopping tablet replica
I20260709 13:20:52.298184  9682 raft_consensus.cc:2243] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:52.299072  9682 raft_consensus.cc:2272] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.301299  9682 tablet_replica.cc:333] T 85d493223fe04c939b6effd840955727 P b80b3a62d7d549a9a5c42def46769ac4: stopping tablet replica
I20260709 13:20:52.301837  9682 raft_consensus.cc:2243] T 85d493223fe04c939b6effd840955727 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.302280  9682 raft_consensus.cc:2272] T 85d493223fe04c939b6effd840955727 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.304287  9682 tablet_replica.cc:333] T bc04a038495940c1bbeda4a2e6f17b82 P b80b3a62d7d549a9a5c42def46769ac4: stopping tablet replica
I20260709 13:20:52.304831  9682 raft_consensus.cc:2243] T bc04a038495940c1bbeda4a2e6f17b82 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.305274  9682 raft_consensus.cc:2272] T bc04a038495940c1bbeda4a2e6f17b82 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.307264  9682 tablet_replica.cc:333] T 9b2e4bf3fb5c456dab400b096f7bd729 P b80b3a62d7d549a9a5c42def46769ac4: stopping tablet replica
I20260709 13:20:52.307852  9682 raft_consensus.cc:2243] T 9b2e4bf3fb5c456dab400b096f7bd729 P b80b3a62d7d549a9a5c42def46769ac4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.308318  9682 raft_consensus.cc:2272] T 9b2e4bf3fb5c456dab400b096f7bd729 P b80b3a62d7d549a9a5c42def46769ac4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.310282  9682 tablet_replica.cc:333] T 19a59035bda747fdbc59b7e6914101dd P b80b3a62d7d549a9a5c42def46769ac4: stopping tablet replica
I20260709 13:20:52.310833  9682 raft_consensus.cc:2243] T 19a59035bda747fdbc59b7e6914101dd P b80b3a62d7d549a9a5c42def46769ac4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:52.311599  9682 raft_consensus.cc:2272] T 19a59035bda747fdbc59b7e6914101dd P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.313645  9682 tablet_replica.cc:333] T 2a411802fdaa42e7863236f0af595fe2 P b80b3a62d7d549a9a5c42def46769ac4: stopping tablet replica
I20260709 13:20:52.314232  9682 raft_consensus.cc:2243] T 2a411802fdaa42e7863236f0af595fe2 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:20:52.315092  9682 raft_consensus.cc:2272] T 2a411802fdaa42e7863236f0af595fe2 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.317128  9682 tablet_replica.cc:333] T 5b9511148896441a948b772784f62a3d P b80b3a62d7d549a9a5c42def46769ac4: stopping tablet replica
I20260709 13:20:52.317627  9682 raft_consensus.cc:2243] T 5b9511148896441a948b772784f62a3d P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.318125  9682 raft_consensus.cc:2272] T 5b9511148896441a948b772784f62a3d P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.320415  9682 tablet_replica.cc:333] T f820e93ffdcd428ea4adfca88589a850 P b80b3a62d7d549a9a5c42def46769ac4: stopping tablet replica
I20260709 13:20:52.321111  9682 raft_consensus.cc:2243] T f820e93ffdcd428ea4adfca88589a850 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.321602  9682 raft_consensus.cc:2272] T f820e93ffdcd428ea4adfca88589a850 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.323553  9682 tablet_replica.cc:333] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P b80b3a62d7d549a9a5c42def46769ac4: stopping tablet replica
I20260709 13:20:52.324052  9682 raft_consensus.cc:2243] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.324456  9682 raft_consensus.cc:2272] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.326311  9682 tablet_replica.cc:333] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P b80b3a62d7d549a9a5c42def46769ac4: stopping tablet replica
I20260709 13:20:52.326850  9682 raft_consensus.cc:2243] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:20:52.327654  9682 raft_consensus.cc:2272] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.329511  9682 tablet_replica.cc:333] T 3aaced67782041fa9960c745301a96e7 P b80b3a62d7d549a9a5c42def46769ac4: stopping tablet replica
I20260709 13:20:52.329986  9682 raft_consensus.cc:2243] T 3aaced67782041fa9960c745301a96e7 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.330410  9682 raft_consensus.cc:2272] T 3aaced67782041fa9960c745301a96e7 P b80b3a62d7d549a9a5c42def46769ac4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.332273  9682 tablet_replica.cc:333] T af881b097b4245b1be6072579197cb6b P b80b3a62d7d549a9a5c42def46769ac4: stopping tablet replica
I20260709 13:20:52.332772  9682 raft_consensus.cc:2243] T af881b097b4245b1be6072579197cb6b P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.333252  9682 raft_consensus.cc:2272] T af881b097b4245b1be6072579197cb6b P b80b3a62d7d549a9a5c42def46769ac4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.354238 20733 raft_consensus.cc:493] T 180230b77b7640b29d055ae8f4358ffe P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:52.354854 20733 raft_consensus.cc:515] T 180230b77b7640b29d055ae8f4358ffe P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:52.357338 20733 leader_election.cc:290] T 180230b77b7640b29d055ae8f4358ffe P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
W20260709 13:20:52.361908 19914 leader_election.cc:336] T 180230b77b7640b29d055ae8f4358ffe P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Network error: Client connection negotiation failed: client connection to 127.9.116.129:41855: connect: Connection refused (error 111)
W20260709 13:20:52.363521 19913 leader_election.cc:336] T 180230b77b7640b29d055ae8f4358ffe P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Network error: Client connection negotiation failed: client connection to 127.9.116.130:43319: connect: Connection refused (error 111)
I20260709 13:20:52.363895 19913 leader_election.cc:304] T 180230b77b7640b29d055ae8f4358ffe P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:52.364540 20733 raft_consensus.cc:2749] T 180230b77b7640b29d055ae8f4358ffe P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:20:52.376190 20733 raft_consensus.cc:493] T 9b9137146e3644a6b0026778530fa8a6 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:52.376739 20733 raft_consensus.cc:515] T 9b9137146e3644a6b0026778530fa8a6 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:52.378813 20733 leader_election.cc:290] T 9b9137146e3644a6b0026778530fa8a6 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
W20260709 13:20:52.382957 19914 leader_election.cc:336] T 9b9137146e3644a6b0026778530fa8a6 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Network error: Client connection negotiation failed: client connection to 127.9.116.129:41855: connect: Connection refused (error 111)
W20260709 13:20:52.383548 19913 leader_election.cc:336] T 9b9137146e3644a6b0026778530fa8a6 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Network error: Client connection negotiation failed: client connection to 127.9.116.130:43319: connect: Connection refused (error 111)
I20260709 13:20:52.383992 19913 leader_election.cc:304] T 9b9137146e3644a6b0026778530fa8a6 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:52.384701 20733 raft_consensus.cc:2749] T 9b9137146e3644a6b0026778530fa8a6 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:20:52.429167 19913 consensus_peers.cc:597] T b832efa5a2e34b3894e33629110e6fef P c62716bf66c24330860573535b8b9430 -> Peer b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249): Couldn't send request to peer b80b3a62d7d549a9a5c42def46769ac4. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.131:45249: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:52.430987  9682 tablet_server.cc:196] TabletServer@127.9.116.131:0 shutdown complete.
W20260709 13:20:52.446309 19913 consensus_peers.cc:597] T 29f92a880ce04259ae0ee9c5288949ff P c62716bf66c24330860573535b8b9430 -> Peer b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249): Couldn't send request to peer b80b3a62d7d549a9a5c42def46769ac4. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.131:45249: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:52.471498 19913 consensus_peers.cc:597] T f012387a39df446bbd9dbea32dc03202 P c62716bf66c24330860573535b8b9430 -> Peer b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249): Couldn't send request to peer b80b3a62d7d549a9a5c42def46769ac4. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.131:45249: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:52.473315 20424 raft_consensus.cc:493] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:52.473914 20424 raft_consensus.cc:515] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:52.476105 20424 leader_election.cc:290] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
W20260709 13:20:52.479557 19914 leader_election.cc:336] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Network error: Client connection negotiation failed: client connection to 127.9.116.129:41855: connect: Connection refused (error 111)
W20260709 13:20:52.482571 19913 consensus_peers.cc:597] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P c62716bf66c24330860573535b8b9430 -> Peer b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249): Couldn't send request to peer b80b3a62d7d549a9a5c42def46769ac4. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.131:45249: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:52.483464 19913 leader_election.cc:336] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Network error: Client connection negotiation failed: client connection to 127.9.116.130:43319: connect: Connection refused (error 111)
I20260709 13:20:52.483911 19913 leader_election.cc:304] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:52.484664 20733 raft_consensus.cc:2749] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:20:52.509706 19913 consensus_peers.cc:597] T ac3bca5bb7644a8fbc78df43ef3331a0 P c62716bf66c24330860573535b8b9430 -> Peer b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249): Couldn't send request to peer b80b3a62d7d549a9a5c42def46769ac4. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.131:45249: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:52.510931 20733 raft_consensus.cc:493] T 7e3099f1888c4ed18c30523bf9ec94f5 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:52.511498 20733 raft_consensus.cc:515] T 7e3099f1888c4ed18c30523bf9ec94f5 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b80b3a62d7d549a9a5c42def46769ac4" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 45249 } } peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } }
I20260709 13:20:52.513718 20733 leader_election.cc:290] T 7e3099f1888c4ed18c30523bf9ec94f5 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249), 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855)
W20260709 13:20:52.520891 19914 leader_election.cc:336] T 7e3099f1888c4ed18c30523bf9ec94f5 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Network error: Client connection negotiation failed: client connection to 127.9.116.129:41855: connect: Connection refused (error 111)
W20260709 13:20:52.521463 19913 leader_election.cc:336] T 7e3099f1888c4ed18c30523bf9ec94f5 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249): Network error: Client connection negotiation failed: client connection to 127.9.116.131:45249: connect: Connection refused (error 111)
I20260709 13:20:52.521948 19913 leader_election.cc:304] T 7e3099f1888c4ed18c30523bf9ec94f5 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c, b80b3a62d7d549a9a5c42def46769ac4
I20260709 13:20:52.522562 20733 raft_consensus.cc:2749] T 7e3099f1888c4ed18c30523bf9ec94f5 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:20:52.522805  9682 tablet_server.cc:179] TabletServer@127.9.116.132:0 shutting down...
W20260709 13:20:52.545797 19913 consensus_peers.cc:597] T 3cad7a5f46f9407a99ea6f2cc5a8322c P c62716bf66c24330860573535b8b9430 -> Peer b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249): Couldn't send request to peer b80b3a62d7d549a9a5c42def46769ac4. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.131:45249: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:52.549010 20733 raft_consensus.cc:493] T 2eef64a7d31b4e1e899aee0857cb0e07 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:52.549808 20733 raft_consensus.cc:515] T 2eef64a7d31b4e1e899aee0857cb0e07 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:52.550750 20424 raft_consensus.cc:493] T d5f8619b1f9f46adaa41774af856d9b1 P c62716bf66c24330860573535b8b9430 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:52.551250 20424 raft_consensus.cc:515] T d5f8619b1f9f46adaa41774af856d9b1 P c62716bf66c24330860573535b8b9430 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:52.551883 20733 leader_election.cc:290] T 2eef64a7d31b4e1e899aee0857cb0e07 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:52.553216 20424 leader_election.cc:290] T d5f8619b1f9f46adaa41774af856d9b1 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
W20260709 13:20:52.556332 19914 leader_election.cc:336] T 2eef64a7d31b4e1e899aee0857cb0e07 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Network error: Client connection negotiation failed: client connection to 127.9.116.129:41855: connect: Connection refused (error 111)
W20260709 13:20:52.556890 19914 leader_election.cc:336] T d5f8619b1f9f46adaa41774af856d9b1 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Network error: Client connection negotiation failed: client connection to 127.9.116.129:41855: connect: Connection refused (error 111)
W20260709 13:20:52.557608 19913 leader_election.cc:336] T 2eef64a7d31b4e1e899aee0857cb0e07 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Network error: Client connection negotiation failed: client connection to 127.9.116.130:43319: connect: Connection refused (error 111)
I20260709 13:20:52.558090 19913 leader_election.cc:304] T 2eef64a7d31b4e1e899aee0857cb0e07 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:52.558873 20424 raft_consensus.cc:2749] T 2eef64a7d31b4e1e899aee0857cb0e07 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:20:52.560243 19913 leader_election.cc:336] T d5f8619b1f9f46adaa41774af856d9b1 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Network error: Client connection negotiation failed: client connection to 127.9.116.130:43319: connect: Connection refused (error 111)
I20260709 13:20:52.560722 19913 leader_election.cc:304] T d5f8619b1f9f46adaa41774af856d9b1 P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:52.561493 20424 raft_consensus.cc:2749] T d5f8619b1f9f46adaa41774af856d9b1 P c62716bf66c24330860573535b8b9430 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:20:52.571158  9682 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:20:52.571976  9682 tablet_replica.cc:333] T 6551866a22dd4aaf8ee318b9d91b3a02 P c62716bf66c24330860573535b8b9430: stopping tablet replica
I20260709 13:20:52.572603  9682 raft_consensus.cc:2243] T 6551866a22dd4aaf8ee318b9d91b3a02 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.573141  9682 raft_consensus.cc:2272] T 6551866a22dd4aaf8ee318b9d91b3a02 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.575309  9682 tablet_replica.cc:333] T 29f92a880ce04259ae0ee9c5288949ff P c62716bf66c24330860573535b8b9430: stopping tablet replica
I20260709 13:20:52.575888  9682 raft_consensus.cc:2243] T 29f92a880ce04259ae0ee9c5288949ff P c62716bf66c24330860573535b8b9430 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:20:52.576802  9682 raft_consensus.cc:2272] T 29f92a880ce04259ae0ee9c5288949ff P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.578722  9682 tablet_replica.cc:333] T bd80f9f8533b4cd3b5861ec11917a03d P c62716bf66c24330860573535b8b9430: stopping tablet replica
I20260709 13:20:52.579494  9682 raft_consensus.cc:2243] T bd80f9f8533b4cd3b5861ec11917a03d P c62716bf66c24330860573535b8b9430 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:20:52.580485  9682 raft_consensus.cc:2272] T bd80f9f8533b4cd3b5861ec11917a03d P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.583145  9682 tablet_replica.cc:333] T 8fd58e41ecee4779b3e72df3aea4463a P c62716bf66c24330860573535b8b9430: stopping tablet replica
I20260709 13:20:52.583833  9682 raft_consensus.cc:2243] T 8fd58e41ecee4779b3e72df3aea4463a P c62716bf66c24330860573535b8b9430 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:52.584617  9682 raft_consensus.cc:2272] T 8fd58e41ecee4779b3e72df3aea4463a P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.586683  9682 tablet_replica.cc:333] T 87364a1289ed4aa1b9b2a7a9951ed776 P c62716bf66c24330860573535b8b9430: stopping tablet replica
I20260709 13:20:52.587301  9682 raft_consensus.cc:2243] T 87364a1289ed4aa1b9b2a7a9951ed776 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.587730  9682 raft_consensus.cc:2272] T 87364a1289ed4aa1b9b2a7a9951ed776 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.589521 20424 raft_consensus.cc:493] T e3ca37daad8341ad99cbba926f37232e P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8df38c4c53c44e94b36c1e8992476d7c)
I20260709 13:20:52.589812  9682 tablet_replica.cc:333] T b832efa5a2e34b3894e33629110e6fef P c62716bf66c24330860573535b8b9430: stopping tablet replica
I20260709 13:20:52.590154 20424 raft_consensus.cc:515] T e3ca37daad8341ad99cbba926f37232e P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62716bf66c24330860573535b8b9430" member_type: VOTER last_known_addr { host: "127.9.116.132" port: 42643 } } peers { permanent_uuid: "8df38c4c53c44e94b36c1e8992476d7c" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 41855 } } peers { permanent_uuid: "e90ffed3ba174ababeb21d9415d4d918" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43319 } }
I20260709 13:20:52.590647  9682 raft_consensus.cc:2243] T b832efa5a2e34b3894e33629110e6fef P c62716bf66c24330860573535b8b9430 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:52.591620  9682 raft_consensus.cc:2272] T b832efa5a2e34b3894e33629110e6fef P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.592291 20424 leader_election.cc:290] T e3ca37daad8341ad99cbba926f37232e P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855), e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319)
I20260709 13:20:52.594313  9682 tablet_replica.cc:333] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430: stopping tablet replica
I20260709 13:20:52.595244  9682 raft_consensus.cc:2243] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.595835  9682 raft_consensus.cc:2272] T 8abfa1aa35c44699960a8166fa79c41d P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:52.597697 19913 leader_election.cc:336] T e3ca37daad8341ad99cbba926f37232e P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e90ffed3ba174ababeb21d9415d4d918 (127.9.116.130:43319): Network error: Client connection negotiation failed: client connection to 127.9.116.130:43319: connect: Connection refused (error 111)
W20260709 13:20:52.598331 19914 leader_election.cc:336] T e3ca37daad8341ad99cbba926f37232e P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8df38c4c53c44e94b36c1e8992476d7c (127.9.116.129:41855): Network error: Client connection negotiation failed: client connection to 127.9.116.129:41855: connect: Connection refused (error 111)
I20260709 13:20:52.598683  9682 tablet_replica.cc:333] T 1fa8a0db61be45c3aa42edb11db4c5ff P c62716bf66c24330860573535b8b9430: stopping tablet replica
I20260709 13:20:52.598773 19914 leader_election.cc:304] T e3ca37daad8341ad99cbba926f37232e P c62716bf66c24330860573535b8b9430 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c62716bf66c24330860573535b8b9430; no voters: 8df38c4c53c44e94b36c1e8992476d7c, e90ffed3ba174ababeb21d9415d4d918
I20260709 13:20:52.599627  9682 raft_consensus.cc:2243] T 1fa8a0db61be45c3aa42edb11db4c5ff P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.599766 20733 raft_consensus.cc:2749] T e3ca37daad8341ad99cbba926f37232e P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:20:52.600299  9682 raft_consensus.cc:2272] T 1fa8a0db61be45c3aa42edb11db4c5ff P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.602427  9682 tablet_replica.cc:333] T 7a5476f5d68d496eaedce93e80bff5e4 P c62716bf66c24330860573535b8b9430: stopping tablet replica
I20260709 13:20:52.603137  9682 raft_consensus.cc:2243] T 7a5476f5d68d496eaedce93e80bff5e4 P c62716bf66c24330860573535b8b9430 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.603631  9682 raft_consensus.cc:2272] T 7a5476f5d68d496eaedce93e80bff5e4 P c62716bf66c24330860573535b8b9430 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.605587  9682 tablet_replica.cc:333] T 1cceb7f75d7040c2b98049590928b891 P c62716bf66c24330860573535b8b9430: stopping tablet replica
I20260709 13:20:52.606148  9682 raft_consensus.cc:2243] T 1cceb7f75d7040c2b98049590928b891 P c62716bf66c24330860573535b8b9430 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:52.606997  9682 raft_consensus.cc:2272] T 1cceb7f75d7040c2b98049590928b891 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.608873  9682 tablet_replica.cc:333] T d5f8619b1f9f46adaa41774af856d9b1 P c62716bf66c24330860573535b8b9430: stopping tablet replica
I20260709 13:20:52.609520  9682 raft_consensus.cc:2243] T d5f8619b1f9f46adaa41774af856d9b1 P c62716bf66c24330860573535b8b9430 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.609982  9682 raft_consensus.cc:2272] T d5f8619b1f9f46adaa41774af856d9b1 P c62716bf66c24330860573535b8b9430 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.612020  9682 tablet_replica.cc:333] T f012387a39df446bbd9dbea32dc03202 P c62716bf66c24330860573535b8b9430: stopping tablet replica
I20260709 13:20:52.612588  9682 raft_consensus.cc:2243] T f012387a39df446bbd9dbea32dc03202 P c62716bf66c24330860573535b8b9430 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:52.613339  9682 raft_consensus.cc:2272] T f012387a39df446bbd9dbea32dc03202 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.615413  9682 tablet_replica.cc:333] T 63868398f3604f529519d926d2831f4e P c62716bf66c24330860573535b8b9430: stopping tablet replica
I20260709 13:20:52.616195  9682 raft_consensus.cc:2243] T 63868398f3604f529519d926d2831f4e P c62716bf66c24330860573535b8b9430 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:20:52.616978  9682 raft_consensus.cc:2272] T 63868398f3604f529519d926d2831f4e P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.619729  9682 tablet_replica.cc:333] T 3aaced67782041fa9960c745301a96e7 P c62716bf66c24330860573535b8b9430: stopping tablet replica
I20260709 13:20:52.620491  9682 raft_consensus.cc:2243] T 3aaced67782041fa9960c745301a96e7 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.620893  9682 raft_consensus.cc:2272] T 3aaced67782041fa9960c745301a96e7 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.622725  9682 tablet_replica.cc:333] T e3ca37daad8341ad99cbba926f37232e P c62716bf66c24330860573535b8b9430: stopping tablet replica
I20260709 13:20:52.623315  9682 raft_consensus.cc:2243] T e3ca37daad8341ad99cbba926f37232e P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.623914  9682 raft_consensus.cc:2272] T e3ca37daad8341ad99cbba926f37232e P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.626617  9682 tablet_replica.cc:333] T 0413f22bd6d7447ca630b75be496555f P c62716bf66c24330860573535b8b9430: stopping tablet replica
I20260709 13:20:52.627424  9682 raft_consensus.cc:2243] T 0413f22bd6d7447ca630b75be496555f P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.627890  9682 raft_consensus.cc:2272] T 0413f22bd6d7447ca630b75be496555f P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.630214  9682 tablet_replica.cc:333] T a36c11f368f34bf0aa4925a099ac1f91 P c62716bf66c24330860573535b8b9430: stopping tablet replica
I20260709 13:20:52.630820  9682 raft_consensus.cc:2243] T a36c11f368f34bf0aa4925a099ac1f91 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:20:52.630898 19913 consensus_peers.cc:597] T 86964b8a61ce494f9096c4fefdc6efbf P c62716bf66c24330860573535b8b9430 -> Peer b80b3a62d7d549a9a5c42def46769ac4 (127.9.116.131:45249): Couldn't send request to peer b80b3a62d7d549a9a5c42def46769ac4. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.131:45249: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:52.631306  9682 raft_consensus.cc:2272] T a36c11f368f34bf0aa4925a099ac1f91 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.633592  9682 tablet_replica.cc:333] T d762eace063c4e04821c749d21702c3f P c62716bf66c24330860573535b8b9430: stopping tablet replica
I20260709 13:20:52.634146  9682 raft_consensus.cc:2243] T d762eace063c4e04821c749d21702c3f P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.634588  9682 raft_consensus.cc:2272] T d762eace063c4e04821c749d21702c3f P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.636976  9682 tablet_replica.cc:333] T 0c709292a0304b3c84a777427a754bb5 P c62716bf66c24330860573535b8b9430: stopping tablet replica
I20260709 13:20:52.637560  9682 raft_consensus.cc:2243] T 0c709292a0304b3c84a777427a754bb5 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.638000  9682 raft_consensus.cc:2272] T 0c709292a0304b3c84a777427a754bb5 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.639997  9682 tablet_replica.cc:333] T 9b9137146e3644a6b0026778530fa8a6 P c62716bf66c24330860573535b8b9430: stopping tablet replica
I20260709 13:20:52.640544  9682 raft_consensus.cc:2243] T 9b9137146e3644a6b0026778530fa8a6 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.641000  9682 raft_consensus.cc:2272] T 9b9137146e3644a6b0026778530fa8a6 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.643021  9682 tablet_replica.cc:333] T 85d493223fe04c939b6effd840955727 P c62716bf66c24330860573535b8b9430: stopping tablet replica
I20260709 13:20:52.643561  9682 raft_consensus.cc:2243] T 85d493223fe04c939b6effd840955727 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.644012  9682 raft_consensus.cc:2272] T 85d493223fe04c939b6effd840955727 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.646062  9682 tablet_replica.cc:333] T 65e7a5222efe43e49e1fbb3a3d7cf99c P c62716bf66c24330860573535b8b9430: stopping tablet replica
I20260709 13:20:52.646631  9682 raft_consensus.cc:2243] T 65e7a5222efe43e49e1fbb3a3d7cf99c P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.647138  9682 raft_consensus.cc:2272] T 65e7a5222efe43e49e1fbb3a3d7cf99c P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.649573  9682 tablet_replica.cc:333] T a323eb80c5084159803e7d4ea5d9c410 P c62716bf66c24330860573535b8b9430: stopping tablet replica
I20260709 13:20:52.650164  9682 raft_consensus.cc:2243] T a323eb80c5084159803e7d4ea5d9c410 P c62716bf66c24330860573535b8b9430 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:20:52.651036  9682 raft_consensus.cc:2272] T a323eb80c5084159803e7d4ea5d9c410 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.653095  9682 tablet_replica.cc:333] T 2eef64a7d31b4e1e899aee0857cb0e07 P c62716bf66c24330860573535b8b9430: stopping tablet replica
I20260709 13:20:52.653662  9682 raft_consensus.cc:2243] T 2eef64a7d31b4e1e899aee0857cb0e07 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.654117  9682 raft_consensus.cc:2272] T 2eef64a7d31b4e1e899aee0857cb0e07 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.655997  9682 tablet_replica.cc:333] T 3cad7a5f46f9407a99ea6f2cc5a8322c P c62716bf66c24330860573535b8b9430: stopping tablet replica
I20260709 13:20:52.656515  9682 raft_consensus.cc:2243] T 3cad7a5f46f9407a99ea6f2cc5a8322c P c62716bf66c24330860573535b8b9430 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:52.657312  9682 raft_consensus.cc:2272] T 3cad7a5f46f9407a99ea6f2cc5a8322c P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.659605  9682 tablet_replica.cc:333] T 7e3099f1888c4ed18c30523bf9ec94f5 P c62716bf66c24330860573535b8b9430: stopping tablet replica
I20260709 13:20:52.660135  9682 raft_consensus.cc:2243] T 7e3099f1888c4ed18c30523bf9ec94f5 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.660593  9682 raft_consensus.cc:2272] T 7e3099f1888c4ed18c30523bf9ec94f5 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.662559  9682 tablet_replica.cc:333] T 353a546cf8c54671971c239502ef165a P c62716bf66c24330860573535b8b9430: stopping tablet replica
I20260709 13:20:52.663162  9682 raft_consensus.cc:2243] T 353a546cf8c54671971c239502ef165a P c62716bf66c24330860573535b8b9430 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:52.663973  9682 raft_consensus.cc:2272] T 353a546cf8c54671971c239502ef165a P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.666008  9682 tablet_replica.cc:333] T d1da95785dbf4ef4bde6f5e6ab35b444 P c62716bf66c24330860573535b8b9430: stopping tablet replica
I20260709 13:20:52.666556  9682 raft_consensus.cc:2243] T d1da95785dbf4ef4bde6f5e6ab35b444 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.667042  9682 raft_consensus.cc:2272] T d1da95785dbf4ef4bde6f5e6ab35b444 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.669130  9682 tablet_replica.cc:333] T 86964b8a61ce494f9096c4fefdc6efbf P c62716bf66c24330860573535b8b9430: stopping tablet replica
I20260709 13:20:52.669672  9682 raft_consensus.cc:2243] T 86964b8a61ce494f9096c4fefdc6efbf P c62716bf66c24330860573535b8b9430 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:20:52.670451  9682 raft_consensus.cc:2272] T 86964b8a61ce494f9096c4fefdc6efbf P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.672626  9682 tablet_replica.cc:333] T 0587017774214f24bdfa4a3da36e367e P c62716bf66c24330860573535b8b9430: stopping tablet replica
I20260709 13:20:52.673215  9682 raft_consensus.cc:2243] T 0587017774214f24bdfa4a3da36e367e P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.673669  9682 raft_consensus.cc:2272] T 0587017774214f24bdfa4a3da36e367e P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.675814  9682 tablet_replica.cc:333] T e804b7d8fd7e44f59516fb299bea7a2c P c62716bf66c24330860573535b8b9430: stopping tablet replica
I20260709 13:20:52.676371  9682 raft_consensus.cc:2243] T e804b7d8fd7e44f59516fb299bea7a2c P c62716bf66c24330860573535b8b9430 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.676868  9682 raft_consensus.cc:2272] T e804b7d8fd7e44f59516fb299bea7a2c P c62716bf66c24330860573535b8b9430 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.679014  9682 tablet_replica.cc:333] T 19e83b82855d4129bb8525e6ff05316b P c62716bf66c24330860573535b8b9430: stopping tablet replica
I20260709 13:20:52.679620  9682 raft_consensus.cc:2243] T 19e83b82855d4129bb8525e6ff05316b P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.680085  9682 raft_consensus.cc:2272] T 19e83b82855d4129bb8525e6ff05316b P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.682063  9682 tablet_replica.cc:333] T ac3bca5bb7644a8fbc78df43ef3331a0 P c62716bf66c24330860573535b8b9430: stopping tablet replica
I20260709 13:20:52.682653  9682 raft_consensus.cc:2243] T ac3bca5bb7644a8fbc78df43ef3331a0 P c62716bf66c24330860573535b8b9430 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:52.683509  9682 raft_consensus.cc:2272] T ac3bca5bb7644a8fbc78df43ef3331a0 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.685602  9682 tablet_replica.cc:333] T bc04a038495940c1bbeda4a2e6f17b82 P c62716bf66c24330860573535b8b9430: stopping tablet replica
I20260709 13:20:52.686184  9682 raft_consensus.cc:2243] T bc04a038495940c1bbeda4a2e6f17b82 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.686691  9682 raft_consensus.cc:2272] T bc04a038495940c1bbeda4a2e6f17b82 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.689419  9682 tablet_replica.cc:333] T 19a59035bda747fdbc59b7e6914101dd P c62716bf66c24330860573535b8b9430: stopping tablet replica
I20260709 13:20:52.690034  9682 raft_consensus.cc:2243] T 19a59035bda747fdbc59b7e6914101dd P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.690517  9682 raft_consensus.cc:2272] T 19a59035bda747fdbc59b7e6914101dd P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.692628  9682 tablet_replica.cc:333] T 9b2e4bf3fb5c456dab400b096f7bd729 P c62716bf66c24330860573535b8b9430: stopping tablet replica
I20260709 13:20:52.693189  9682 raft_consensus.cc:2243] T 9b2e4bf3fb5c456dab400b096f7bd729 P c62716bf66c24330860573535b8b9430 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.693691  9682 raft_consensus.cc:2272] T 9b2e4bf3fb5c456dab400b096f7bd729 P c62716bf66c24330860573535b8b9430 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.695960  9682 tablet_replica.cc:333] T e8cfec59c63741869c636df6356546d0 P c62716bf66c24330860573535b8b9430: stopping tablet replica
I20260709 13:20:52.696625  9682 raft_consensus.cc:2243] T e8cfec59c63741869c636df6356546d0 P c62716bf66c24330860573535b8b9430 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:52.697484  9682 raft_consensus.cc:2272] T e8cfec59c63741869c636df6356546d0 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.699506  9682 tablet_replica.cc:333] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P c62716bf66c24330860573535b8b9430: stopping tablet replica
I20260709 13:20:52.700121  9682 raft_consensus.cc:2243] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P c62716bf66c24330860573535b8b9430 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:52.700872  9682 raft_consensus.cc:2272] T eb7bc0eba7e94b9eb1eeec9bbd1a6e97 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.702768  9682 tablet_replica.cc:333] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P c62716bf66c24330860573535b8b9430: stopping tablet replica
I20260709 13:20:52.703361  9682 raft_consensus.cc:2243] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.703787  9682 raft_consensus.cc:2272] T 5d9f7b34cb064bbe8a6938e4f92d57c5 P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.705595  9682 tablet_replica.cc:333] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P c62716bf66c24330860573535b8b9430: stopping tablet replica
I20260709 13:20:52.706110  9682 raft_consensus.cc:2243] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.706532  9682 raft_consensus.cc:2272] T 0f3d7bd0ea32401fa8cb8ab3292a7808 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.708412  9682 tablet_replica.cc:333] T 9f60d4b859264720a20f5670c3c71a6f P c62716bf66c24330860573535b8b9430: stopping tablet replica
I20260709 13:20:52.708925  9682 raft_consensus.cc:2243] T 9f60d4b859264720a20f5670c3c71a6f P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.709370  9682 raft_consensus.cc:2272] T 9f60d4b859264720a20f5670c3c71a6f P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.711395  9682 tablet_replica.cc:333] T e9369d9b7f5649be932967d24689a936 P c62716bf66c24330860573535b8b9430: stopping tablet replica
I20260709 13:20:52.711880  9682 raft_consensus.cc:2243] T e9369d9b7f5649be932967d24689a936 P c62716bf66c24330860573535b8b9430 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:52.712700  9682 raft_consensus.cc:2272] T e9369d9b7f5649be932967d24689a936 P c62716bf66c24330860573535b8b9430 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.714550  9682 tablet_replica.cc:333] T ece79531bbd749d3b9300fe2fbeef9af P c62716bf66c24330860573535b8b9430: stopping tablet replica
I20260709 13:20:52.715062  9682 raft_consensus.cc:2243] T ece79531bbd749d3b9300fe2fbeef9af P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.715507  9682 raft_consensus.cc:2272] T ece79531bbd749d3b9300fe2fbeef9af P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.717295  9682 tablet_replica.cc:333] T 180230b77b7640b29d055ae8f4358ffe P c62716bf66c24330860573535b8b9430: stopping tablet replica
I20260709 13:20:52.717737  9682 raft_consensus.cc:2243] T 180230b77b7640b29d055ae8f4358ffe P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:52.718145  9682 raft_consensus.cc:2272] T 180230b77b7640b29d055ae8f4358ffe P c62716bf66c24330860573535b8b9430 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.773430  9682 tablet_server.cc:196] TabletServer@127.9.116.132:0 shutdown complete.
I20260709 13:20:52.835505  9682 master.cc:562] Master@127.9.116.190:44649 shutting down...
I20260709 13:20:52.862190  9682 raft_consensus.cc:2243] T 00000000000000000000000000000000 P e027addf51a94fb8b711872407dc97b2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:52.862722  9682 raft_consensus.cc:2272] T 00000000000000000000000000000000 P e027addf51a94fb8b711872407dc97b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:52.863098  9682 tablet_replica.cc:333] T 00000000000000000000000000000000 P e027addf51a94fb8b711872407dc97b2: stopping tablet replica
I20260709 13:20:52.885414  9682 master.cc:584] Master@127.9.116.190:44649 shutdown complete.
[       OK ] LeaderRebalancerTest.RestartTserver (31011 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 13:20:52.975361  9682 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.9.116.190:40921
I20260709 13:20:52.976542  9682 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:20:52.982952 20819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:20:52.982980 20820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:20:52.983765 20822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:20:52.985846  9682 server_base.cc:1061] running on GCE node
I20260709 13:20:52.986910  9682 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:20:52.987090  9682 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:20:52.987205  9682 hybrid_clock.cc:648] HybridClock initialized: now 1783603252987194 us; error 0 us; skew 500 ppm
I20260709 13:20:52.987701  9682 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:20:52.990072  9682 webserver.cc:533] Webserver started at http://127.9.116.190:39461/ using document root <none> and password file <none>
I20260709 13:20:52.990523  9682 fs_manager.cc:362] Metadata directory not provided
I20260709 13:20:52.990684  9682 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:20:52.990970  9682 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:20:52.992053  9682 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602916530764-9682-0/minicluster-data/master-0-root/instance:
uuid: "d11a97679c174598899b62857a671ac1"
format_stamp: "Formatted at 2026-07-09 13:20:52 on dist-test-slave-1t6m"
I20260709 13:20:52.996450  9682 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:20:52.999758 20827 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:20:53.000495  9682 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.003s
I20260709 13:20:53.000825  9682 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602916530764-9682-0/minicluster-data/master-0-root
uuid: "d11a97679c174598899b62857a671ac1"
format_stamp: "Formatted at 2026-07-09 13:20:52 on dist-test-slave-1t6m"
I20260709 13:20:53.001123  9682 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602916530764-9682-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602916530764-9682-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602916530764-9682-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:20:53.013504  9682 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:20:53.014765  9682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:20:53.053547  9682 rpc_server.cc:307] RPC server started. Bound to: 127.9.116.190:40921
I20260709 13:20:53.053655 20878 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.116.190:40921 every 8 connection(s)
I20260709 13:20:53.057498 20879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:53.068562 20879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d11a97679c174598899b62857a671ac1: Bootstrap starting.
I20260709 13:20:53.073488 20879 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d11a97679c174598899b62857a671ac1: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:53.078248 20879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d11a97679c174598899b62857a671ac1: No bootstrap required, opened a new log
I20260709 13:20:53.080551 20879 raft_consensus.cc:359] T 00000000000000000000000000000000 P d11a97679c174598899b62857a671ac1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d11a97679c174598899b62857a671ac1" member_type: VOTER }
I20260709 13:20:53.080958 20879 raft_consensus.cc:385] T 00000000000000000000000000000000 P d11a97679c174598899b62857a671ac1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:53.081246 20879 raft_consensus.cc:740] T 00000000000000000000000000000000 P d11a97679c174598899b62857a671ac1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d11a97679c174598899b62857a671ac1, State: Initialized, Role: FOLLOWER
I20260709 13:20:53.081844 20879 consensus_queue.cc:260] T 00000000000000000000000000000000 P d11a97679c174598899b62857a671ac1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d11a97679c174598899b62857a671ac1" member_type: VOTER }
I20260709 13:20:53.082346 20879 raft_consensus.cc:399] T 00000000000000000000000000000000 P d11a97679c174598899b62857a671ac1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:20:53.082579 20879 raft_consensus.cc:493] T 00000000000000000000000000000000 P d11a97679c174598899b62857a671ac1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:20:53.082890 20879 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d11a97679c174598899b62857a671ac1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:53.088369 20879 raft_consensus.cc:515] T 00000000000000000000000000000000 P d11a97679c174598899b62857a671ac1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d11a97679c174598899b62857a671ac1" member_type: VOTER }
I20260709 13:20:53.088951 20879 leader_election.cc:304] T 00000000000000000000000000000000 P d11a97679c174598899b62857a671ac1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d11a97679c174598899b62857a671ac1; no voters: 
I20260709 13:20:53.090111 20879 leader_election.cc:290] T 00000000000000000000000000000000 P d11a97679c174598899b62857a671ac1 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:20:53.090494 20882 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d11a97679c174598899b62857a671ac1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:53.092001 20882 raft_consensus.cc:697] T 00000000000000000000000000000000 P d11a97679c174598899b62857a671ac1 [term 1 LEADER]: Becoming Leader. State: Replica: d11a97679c174598899b62857a671ac1, State: Running, Role: LEADER
I20260709 13:20:53.092653 20882 consensus_queue.cc:237] T 00000000000000000000000000000000 P d11a97679c174598899b62857a671ac1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d11a97679c174598899b62857a671ac1" member_type: VOTER }
I20260709 13:20:53.093218 20879 sys_catalog.cc:565] T 00000000000000000000000000000000 P d11a97679c174598899b62857a671ac1 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:20:53.097421 20883 sys_catalog.cc:455] T 00000000000000000000000000000000 P d11a97679c174598899b62857a671ac1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d11a97679c174598899b62857a671ac1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d11a97679c174598899b62857a671ac1" member_type: VOTER } }
I20260709 13:20:53.098156 20883 sys_catalog.cc:458] T 00000000000000000000000000000000 P d11a97679c174598899b62857a671ac1 [sys.catalog]: This master's current role is: LEADER
I20260709 13:20:53.097971 20884 sys_catalog.cc:455] T 00000000000000000000000000000000 P d11a97679c174598899b62857a671ac1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d11a97679c174598899b62857a671ac1. Latest consensus state: current_term: 1 leader_uuid: "d11a97679c174598899b62857a671ac1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d11a97679c174598899b62857a671ac1" member_type: VOTER } }
I20260709 13:20:53.098526 20884 sys_catalog.cc:458] T 00000000000000000000000000000000 P d11a97679c174598899b62857a671ac1 [sys.catalog]: This master's current role is: LEADER
I20260709 13:20:53.103989 20889 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:20:53.111922 20889 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:20:53.112944  9682 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:20:53.121018 20889 catalog_manager.cc:1383] Generated new cluster ID: 6c43c06dec8940d988c78910eb6084d2
I20260709 13:20:53.121336 20889 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:20:53.142733 20889 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:20:53.144075 20889 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:20:53.156333 20889 catalog_manager.cc:6070] T 00000000000000000000000000000000 P d11a97679c174598899b62857a671ac1: Generated new TSK 0
I20260709 13:20:53.156971 20889 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:20:53.180548  9682 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:20:53.187184 20900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:20:53.188060 20901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:20:53.189814 20903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:20:53.190408  9682 server_base.cc:1061] running on GCE node
I20260709 13:20:53.191421  9682 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:20:53.191612  9682 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:20:53.191741  9682 hybrid_clock.cc:648] HybridClock initialized: now 1783603253191730 us; error 0 us; skew 500 ppm
I20260709 13:20:53.192188  9682 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:20:53.194558  9682 webserver.cc:533] Webserver started at http://127.9.116.129:34965/ using document root <none> and password file <none>
I20260709 13:20:53.195134  9682 fs_manager.cc:362] Metadata directory not provided
I20260709 13:20:53.195327  9682 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:20:53.195547  9682 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:20:53.196596  9682 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602916530764-9682-0/minicluster-data/ts-0-root/instance:
uuid: "71bdb236d2304c75b7eabfec8f3d5a46"
format_stamp: "Formatted at 2026-07-09 13:20:53 on dist-test-slave-1t6m"
I20260709 13:20:53.201236  9682 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:20:53.205047 20908 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:20:53.205855  9682 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:20:53.206175  9682 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602916530764-9682-0/minicluster-data/ts-0-root
uuid: "71bdb236d2304c75b7eabfec8f3d5a46"
format_stamp: "Formatted at 2026-07-09 13:20:53 on dist-test-slave-1t6m"
I20260709 13:20:53.206470  9682 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602916530764-9682-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602916530764-9682-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602916530764-9682-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:20:53.222769  9682 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:20:53.224128  9682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:20:53.225627  9682 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:20:53.228128  9682 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:20:53.228358  9682 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:20:53.228615  9682 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:20:53.228816  9682 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:20:53.269862  9682 rpc_server.cc:307] RPC server started. Bound to: 127.9.116.129:40473
I20260709 13:20:53.269981 20970 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.116.129:40473 every 8 connection(s)
I20260709 13:20:53.275055  9682 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:20:53.282909 20975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:20:53.284171 20976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:20:53.292891 20978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:20:53.295328 20971 heartbeater.cc:344] Connected to a master server at 127.9.116.190:40921
I20260709 13:20:53.295854 20971 heartbeater.cc:461] Registering TS with master...
I20260709 13:20:53.296447  9682 server_base.cc:1061] running on GCE node
I20260709 13:20:53.296886 20971 heartbeater.cc:507] Master 127.9.116.190:40921 requested a full tablet report, sending...
I20260709 13:20:53.297487  9682 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:20:53.297753  9682 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:20:53.297946  9682 hybrid_clock.cc:648] HybridClock initialized: now 1783603253297925 us; error 0 us; skew 500 ppm
I20260709 13:20:53.298550  9682 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:20:53.299542 20844 ts_manager.cc:194] Registered new tserver with Master: 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473)
I20260709 13:20:53.301537  9682 webserver.cc:533] Webserver started at http://127.9.116.130:45271/ using document root <none> and password file <none>
I20260709 13:20:53.302205  9682 fs_manager.cc:362] Metadata directory not provided
I20260709 13:20:53.302284 20844 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34720
I20260709 13:20:53.302459  9682 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:20:53.302958  9682 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:20:53.304370  9682 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602916530764-9682-0/minicluster-data/ts-1-root/instance:
uuid: "ec8fff5744984f4695d768676cae8d17"
format_stamp: "Formatted at 2026-07-09 13:20:53 on dist-test-slave-1t6m"
I20260709 13:20:53.308981  9682 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.004s
I20260709 13:20:53.312390 20983 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:20:53.313164  9682 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:20:53.313472  9682 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602916530764-9682-0/minicluster-data/ts-1-root
uuid: "ec8fff5744984f4695d768676cae8d17"
format_stamp: "Formatted at 2026-07-09 13:20:53 on dist-test-slave-1t6m"
I20260709 13:20:53.313774  9682 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602916530764-9682-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602916530764-9682-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602916530764-9682-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:20:53.329589  9682 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:20:53.330963  9682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:20:53.332470  9682 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:20:53.334740  9682 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:20:53.335027  9682 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:20:53.335276  9682 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:20:53.335448  9682 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:20:53.375640  9682 rpc_server.cc:307] RPC server started. Bound to: 127.9.116.130:42659
I20260709 13:20:53.375762 21045 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.116.130:42659 every 8 connection(s)
I20260709 13:20:53.380527  9682 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:20:53.387796 21049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:20:53.389405 21050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:20:53.392462  9682 server_base.cc:1061] running on GCE node
W20260709 13:20:53.392671 21053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:20:53.393399 21046 heartbeater.cc:344] Connected to a master server at 127.9.116.190:40921
I20260709 13:20:53.393846 21046 heartbeater.cc:461] Registering TS with master...
I20260709 13:20:53.393965  9682 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:20:53.394263  9682 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:20:53.394572  9682 hybrid_clock.cc:648] HybridClock initialized: now 1783603253394548 us; error 0 us; skew 500 ppm
I20260709 13:20:53.394817 21046 heartbeater.cc:507] Master 127.9.116.190:40921 requested a full tablet report, sending...
I20260709 13:20:53.395406  9682 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:20:53.397310 20844 ts_manager.cc:194] Registered new tserver with Master: ec8fff5744984f4695d768676cae8d17 (127.9.116.130:42659)
I20260709 13:20:53.398057  9682 webserver.cc:533] Webserver started at http://127.9.116.131:43661/ using document root <none> and password file <none>
I20260709 13:20:53.398612  9682 fs_manager.cc:362] Metadata directory not provided
I20260709 13:20:53.398844  9682 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:20:53.398926 20844 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34732
I20260709 13:20:53.399124  9682 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:20:53.400591  9682 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602916530764-9682-0/minicluster-data/ts-2-root/instance:
uuid: "bd6a13ebcb53485a9cc5082a4f64d652"
format_stamp: "Formatted at 2026-07-09 13:20:53 on dist-test-slave-1t6m"
I20260709 13:20:53.405471  9682 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:20:53.408910 21057 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:20:53.409711  9682 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:20:53.410020  9682 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602916530764-9682-0/minicluster-data/ts-2-root
uuid: "bd6a13ebcb53485a9cc5082a4f64d652"
format_stamp: "Formatted at 2026-07-09 13:20:53 on dist-test-slave-1t6m"
I20260709 13:20:53.410316  9682 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602916530764-9682-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602916530764-9682-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602916530764-9682-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:20:53.425855  9682 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:20:53.427196  9682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:20:53.428680  9682 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:20:53.431020  9682 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:20:53.431208  9682 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:20:53.431507  9682 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:20:53.431669  9682 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:20:53.472535  9682 rpc_server.cc:307] RPC server started. Bound to: 127.9.116.131:37203
I20260709 13:20:53.472664 21119 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.116.131:37203 every 8 connection(s)
I20260709 13:20:53.487504 21120 heartbeater.cc:344] Connected to a master server at 127.9.116.190:40921
I20260709 13:20:53.487982 21120 heartbeater.cc:461] Registering TS with master...
I20260709 13:20:53.488746 21120 heartbeater.cc:507] Master 127.9.116.190:40921 requested a full tablet report, sending...
I20260709 13:20:53.490898 20844 ts_manager.cc:194] Registered new tserver with Master: bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131:37203)
I20260709 13:20:53.491350  9682 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014790949s
I20260709 13:20:53.492331 20844 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34736
I20260709 13:20:53.519184 20844 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34740:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:20:53.522194 20844 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:20:53.600181 21085 tablet_service.cc:1511] Processing CreateTablet for tablet b541a9ed669a420d8fbeb6e10ab3c47f (DEFAULT_TABLE table=test-workload [id=7d08ae5c97af4e99b9b9e290e2c165ac]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:20:53.606004 21085 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b541a9ed669a420d8fbeb6e10ab3c47f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:53.610164 21084 tablet_service.cc:1511] Processing CreateTablet for tablet 5e579b46cc154814ad25486ba520710f (DEFAULT_TABLE table=test-workload [id=7d08ae5c97af4e99b9b9e290e2c165ac]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:20:53.611802 21084 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e579b46cc154814ad25486ba520710f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:53.616698 21083 tablet_service.cc:1511] Processing CreateTablet for tablet ee997af6d38745549698356bcbc09a99 (DEFAULT_TABLE table=test-workload [id=7d08ae5c97af4e99b9b9e290e2c165ac]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:20:53.619588 21083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee997af6d38745549698356bcbc09a99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:53.619493 21008 tablet_service.cc:1511] Processing CreateTablet for tablet 2725109df1524e6c8eb897f0e6424829 (DEFAULT_TABLE table=test-workload [id=7d08ae5c97af4e99b9b9e290e2c165ac]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:20:53.619752 21006 tablet_service.cc:1511] Processing CreateTablet for tablet 0f79d8e2c55b4a1f8380919ef723efe7 (DEFAULT_TABLE table=test-workload [id=7d08ae5c97af4e99b9b9e290e2c165ac]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:20:53.620833 21008 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2725109df1524e6c8eb897f0e6424829. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:53.621423 21006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f79d8e2c55b4a1f8380919ef723efe7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:53.622015 21079 tablet_service.cc:1511] Processing CreateTablet for tablet 35602162000346e3bbb790f54467ed0d (DEFAULT_TABLE table=test-workload [id=7d08ae5c97af4e99b9b9e290e2c165ac]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:20:53.623420 21079 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35602162000346e3bbb790f54467ed0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:53.623844 21011 tablet_service.cc:1511] Processing CreateTablet for tablet b541a9ed669a420d8fbeb6e10ab3c47f (DEFAULT_TABLE table=test-workload [id=7d08ae5c97af4e99b9b9e290e2c165ac]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:20:53.625258 21011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b541a9ed669a420d8fbeb6e10ab3c47f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:53.625557 21082 tablet_service.cc:1511] Processing CreateTablet for tablet 2725109df1524e6c8eb897f0e6424829 (DEFAULT_TABLE table=test-workload [id=7d08ae5c97af4e99b9b9e290e2c165ac]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:20:53.626971 21082 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2725109df1524e6c8eb897f0e6424829. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:53.627774 21010 tablet_service.cc:1511] Processing CreateTablet for tablet 5e579b46cc154814ad25486ba520710f (DEFAULT_TABLE table=test-workload [id=7d08ae5c97af4e99b9b9e290e2c165ac]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:20:53.629175 21010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e579b46cc154814ad25486ba520710f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:53.631775 21009 tablet_service.cc:1511] Processing CreateTablet for tablet ee997af6d38745549698356bcbc09a99 (DEFAULT_TABLE table=test-workload [id=7d08ae5c97af4e99b9b9e290e2c165ac]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:20:53.633255 21009 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee997af6d38745549698356bcbc09a99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:53.636775 21007 tablet_service.cc:1511] Processing CreateTablet for tablet 7f3178c915104a27b9a3b6978249c5ed (DEFAULT_TABLE table=test-workload [id=7d08ae5c97af4e99b9b9e290e2c165ac]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:20:53.637746 21078 tablet_service.cc:1511] Processing CreateTablet for tablet c3add26e31364c95a9897ab53b516735 (DEFAULT_TABLE table=test-workload [id=7d08ae5c97af4e99b9b9e290e2c165ac]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:20:53.638370 21007 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f3178c915104a27b9a3b6978249c5ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:53.639252 21078 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3add26e31364c95a9897ab53b516735. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:53.639818 21080 tablet_service.cc:1511] Processing CreateTablet for tablet 0f79d8e2c55b4a1f8380919ef723efe7 (DEFAULT_TABLE table=test-workload [id=7d08ae5c97af4e99b9b9e290e2c165ac]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:20:53.641453 21080 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f79d8e2c55b4a1f8380919ef723efe7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:53.643667 21081 tablet_service.cc:1511] Processing CreateTablet for tablet 7f3178c915104a27b9a3b6978249c5ed (DEFAULT_TABLE table=test-workload [id=7d08ae5c97af4e99b9b9e290e2c165ac]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:20:53.649166 21005 tablet_service.cc:1511] Processing CreateTablet for tablet 35602162000346e3bbb790f54467ed0d (DEFAULT_TABLE table=test-workload [id=7d08ae5c97af4e99b9b9e290e2c165ac]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:20:53.652616 21004 tablet_service.cc:1511] Processing CreateTablet for tablet c3add26e31364c95a9897ab53b516735 (DEFAULT_TABLE table=test-workload [id=7d08ae5c97af4e99b9b9e290e2c165ac]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:20:53.654150 21004 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3add26e31364c95a9897ab53b516735. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:53.655268 21005 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35602162000346e3bbb790f54467ed0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:53.656858 21003 tablet_service.cc:1511] Processing CreateTablet for tablet c50ac60e07eb46dc8836de50bce12760 (DEFAULT_TABLE table=test-workload [id=7d08ae5c97af4e99b9b9e290e2c165ac]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:20:53.658321 21003 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c50ac60e07eb46dc8836de50bce12760. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:53.662730 21081 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f3178c915104a27b9a3b6978249c5ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:53.673923 21135 tablet_bootstrap.cc:492] T 0f79d8e2c55b4a1f8380919ef723efe7 P ec8fff5744984f4695d768676cae8d17: Bootstrap starting.
I20260709 13:20:53.717506 21135 tablet_bootstrap.cc:654] T 0f79d8e2c55b4a1f8380919ef723efe7 P ec8fff5744984f4695d768676cae8d17: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:53.719732 20936 tablet_service.cc:1511] Processing CreateTablet for tablet b541a9ed669a420d8fbeb6e10ab3c47f (DEFAULT_TABLE table=test-workload [id=7d08ae5c97af4e99b9b9e290e2c165ac]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:20:53.728523 20936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b541a9ed669a420d8fbeb6e10ab3c47f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:53.732002 20929 tablet_service.cc:1511] Processing CreateTablet for tablet c3add26e31364c95a9897ab53b516735 (DEFAULT_TABLE table=test-workload [id=7d08ae5c97af4e99b9b9e290e2c165ac]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:20:53.733521 20929 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3add26e31364c95a9897ab53b516735. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:53.733898 21084 tablet_service.cc:1511] Processing CreateTablet for tablet 35d6d5be27cb4096917b886d06a3cf58 (DEFAULT_TABLE table=test-workload [id=7d08ae5c97af4e99b9b9e290e2c165ac]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:20:53.734994 21136 tablet_bootstrap.cc:492] T 0f79d8e2c55b4a1f8380919ef723efe7 P bd6a13ebcb53485a9cc5082a4f64d652: Bootstrap starting.
I20260709 13:20:53.735765 21084 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35d6d5be27cb4096917b886d06a3cf58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:53.737048 20933 tablet_service.cc:1511] Processing CreateTablet for tablet 2725109df1524e6c8eb897f0e6424829 (DEFAULT_TABLE table=test-workload [id=7d08ae5c97af4e99b9b9e290e2c165ac]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:20:53.738507 20933 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2725109df1524e6c8eb897f0e6424829. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:53.721140 20935 tablet_service.cc:1511] Processing CreateTablet for tablet 5e579b46cc154814ad25486ba520710f (DEFAULT_TABLE table=test-workload [id=7d08ae5c97af4e99b9b9e290e2c165ac]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:20:53.722373 20934 tablet_service.cc:1511] Processing CreateTablet for tablet ee997af6d38745549698356bcbc09a99 (DEFAULT_TABLE table=test-workload [id=7d08ae5c97af4e99b9b9e290e2c165ac]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:20:53.723977 20932 tablet_service.cc:1511] Processing CreateTablet for tablet 7f3178c915104a27b9a3b6978249c5ed (DEFAULT_TABLE table=test-workload [id=7d08ae5c97af4e99b9b9e290e2c165ac]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:20:53.726385 20930 tablet_service.cc:1511] Processing CreateTablet for tablet 35602162000346e3bbb790f54467ed0d (DEFAULT_TABLE table=test-workload [id=7d08ae5c97af4e99b9b9e290e2c165ac]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:20:53.744295 20932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f3178c915104a27b9a3b6978249c5ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:53.745538 20930 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35602162000346e3bbb790f54467ed0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:53.725203 20931 tablet_service.cc:1511] Processing CreateTablet for tablet 0f79d8e2c55b4a1f8380919ef723efe7 (DEFAULT_TABLE table=test-workload [id=7d08ae5c97af4e99b9b9e290e2c165ac]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:20:53.747155 20928 tablet_service.cc:1511] Processing CreateTablet for tablet c50ac60e07eb46dc8836de50bce12760 (DEFAULT_TABLE table=test-workload [id=7d08ae5c97af4e99b9b9e290e2c165ac]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:20:53.748080 20934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee997af6d38745549698356bcbc09a99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:53.749954 20927 tablet_service.cc:1511] Processing CreateTablet for tablet 35d6d5be27cb4096917b886d06a3cf58 (DEFAULT_TABLE table=test-workload [id=7d08ae5c97af4e99b9b9e290e2c165ac]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:20:53.751278 20931 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f79d8e2c55b4a1f8380919ef723efe7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:53.752151 20928 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c50ac60e07eb46dc8836de50bce12760. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:53.752884 20935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e579b46cc154814ad25486ba520710f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:53.753539 20927 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35d6d5be27cb4096917b886d06a3cf58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:53.726711 21083 tablet_service.cc:1511] Processing CreateTablet for tablet c50ac60e07eb46dc8836de50bce12760 (DEFAULT_TABLE table=test-workload [id=7d08ae5c97af4e99b9b9e290e2c165ac]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:20:53.770278 21083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c50ac60e07eb46dc8836de50bce12760. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:53.784329 21008 tablet_service.cc:1511] Processing CreateTablet for tablet 35d6d5be27cb4096917b886d06a3cf58 (DEFAULT_TABLE table=test-workload [id=7d08ae5c97af4e99b9b9e290e2c165ac]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:20:53.785796 21008 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35d6d5be27cb4096917b886d06a3cf58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:53.786108 21135 tablet_bootstrap.cc:492] T 0f79d8e2c55b4a1f8380919ef723efe7 P ec8fff5744984f4695d768676cae8d17: No bootstrap required, opened a new log
I20260709 13:20:53.786712 21135 ts_tablet_manager.cc:1403] T 0f79d8e2c55b4a1f8380919ef723efe7 P ec8fff5744984f4695d768676cae8d17: Time spent bootstrapping tablet: real 0.113s	user 0.020s	sys 0.021s
I20260709 13:20:53.789983 21135 raft_consensus.cc:359] T 0f79d8e2c55b4a1f8380919ef723efe7 P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:53.792050 21135 raft_consensus.cc:385] T 0f79d8e2c55b4a1f8380919ef723efe7 P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:53.807224 21138 tablet_bootstrap.cc:492] T c3add26e31364c95a9897ab53b516735 P 71bdb236d2304c75b7eabfec8f3d5a46: Bootstrap starting.
I20260709 13:20:53.815450 21136 tablet_bootstrap.cc:654] T 0f79d8e2c55b4a1f8380919ef723efe7 P bd6a13ebcb53485a9cc5082a4f64d652: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:53.806103 21135 raft_consensus.cc:740] T 0f79d8e2c55b4a1f8380919ef723efe7 P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec8fff5744984f4695d768676cae8d17, State: Initialized, Role: FOLLOWER
I20260709 13:20:53.831826 21135 consensus_queue.cc:260] T 0f79d8e2c55b4a1f8380919ef723efe7 P ec8fff5744984f4695d768676cae8d17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:53.832684 21138 tablet_bootstrap.cc:654] T c3add26e31364c95a9897ab53b516735 P 71bdb236d2304c75b7eabfec8f3d5a46: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:53.847925 21135 ts_tablet_manager.cc:1434] T 0f79d8e2c55b4a1f8380919ef723efe7 P ec8fff5744984f4695d768676cae8d17: Time spent starting tablet: real 0.061s	user 0.006s	sys 0.002s
I20260709 13:20:53.849066 21135 tablet_bootstrap.cc:492] T c50ac60e07eb46dc8836de50bce12760 P ec8fff5744984f4695d768676cae8d17: Bootstrap starting.
I20260709 13:20:53.848843 21046 heartbeater.cc:499] Master 127.9.116.190:40921 was elected leader, sending a full tablet report...
I20260709 13:20:53.848428 21136 tablet_bootstrap.cc:492] T 0f79d8e2c55b4a1f8380919ef723efe7 P bd6a13ebcb53485a9cc5082a4f64d652: No bootstrap required, opened a new log
I20260709 13:20:53.851372 21136 ts_tablet_manager.cc:1403] T 0f79d8e2c55b4a1f8380919ef723efe7 P bd6a13ebcb53485a9cc5082a4f64d652: Time spent bootstrapping tablet: real 0.141s	user 0.012s	sys 0.019s
I20260709 13:20:53.855664 21136 raft_consensus.cc:359] T 0f79d8e2c55b4a1f8380919ef723efe7 P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:53.856679 21136 raft_consensus.cc:385] T 0f79d8e2c55b4a1f8380919ef723efe7 P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:53.857074 21136 raft_consensus.cc:740] T 0f79d8e2c55b4a1f8380919ef723efe7 P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd6a13ebcb53485a9cc5082a4f64d652, State: Initialized, Role: FOLLOWER
I20260709 13:20:53.864416 21135 tablet_bootstrap.cc:654] T c50ac60e07eb46dc8836de50bce12760 P ec8fff5744984f4695d768676cae8d17: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:53.858101 21136 consensus_queue.cc:260] T 0f79d8e2c55b4a1f8380919ef723efe7 P bd6a13ebcb53485a9cc5082a4f64d652 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:53.870054 21138 tablet_bootstrap.cc:492] T c3add26e31364c95a9897ab53b516735 P 71bdb236d2304c75b7eabfec8f3d5a46: No bootstrap required, opened a new log
I20260709 13:20:53.870748 21138 ts_tablet_manager.cc:1403] T c3add26e31364c95a9897ab53b516735 P 71bdb236d2304c75b7eabfec8f3d5a46: Time spent bootstrapping tablet: real 0.064s	user 0.016s	sys 0.023s
I20260709 13:20:53.875028 21138 raft_consensus.cc:359] T c3add26e31364c95a9897ab53b516735 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:53.876037 21138 raft_consensus.cc:385] T c3add26e31364c95a9897ab53b516735 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:53.877578 21120 heartbeater.cc:499] Master 127.9.116.190:40921 was elected leader, sending a full tablet report...
I20260709 13:20:53.878131 21136 ts_tablet_manager.cc:1434] T 0f79d8e2c55b4a1f8380919ef723efe7 P bd6a13ebcb53485a9cc5082a4f64d652: Time spent starting tablet: real 0.026s	user 0.014s	sys 0.002s
I20260709 13:20:53.879204 21136 tablet_bootstrap.cc:492] T 7f3178c915104a27b9a3b6978249c5ed P bd6a13ebcb53485a9cc5082a4f64d652: Bootstrap starting.
W20260709 13:20:53.883406 21047 tablet.cc:2406] T 0f79d8e2c55b4a1f8380919ef723efe7 P ec8fff5744984f4695d768676cae8d17: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:20:53.885545 21135 tablet_bootstrap.cc:492] T c50ac60e07eb46dc8836de50bce12760 P ec8fff5744984f4695d768676cae8d17: No bootstrap required, opened a new log
I20260709 13:20:53.885905 21136 tablet_bootstrap.cc:654] T 7f3178c915104a27b9a3b6978249c5ed P bd6a13ebcb53485a9cc5082a4f64d652: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:53.886126 21135 ts_tablet_manager.cc:1403] T c50ac60e07eb46dc8836de50bce12760 P ec8fff5744984f4695d768676cae8d17: Time spent bootstrapping tablet: real 0.037s	user 0.015s	sys 0.011s
I20260709 13:20:53.885973 21138 raft_consensus.cc:740] T c3add26e31364c95a9897ab53b516735 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71bdb236d2304c75b7eabfec8f3d5a46, State: Initialized, Role: FOLLOWER
I20260709 13:20:53.887334 21138 consensus_queue.cc:260] T c3add26e31364c95a9897ab53b516735 P 71bdb236d2304c75b7eabfec8f3d5a46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:53.890167 21135 raft_consensus.cc:359] T c50ac60e07eb46dc8836de50bce12760 P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:53.891275 21135 raft_consensus.cc:385] T c50ac60e07eb46dc8836de50bce12760 P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:53.891655 21135 raft_consensus.cc:740] T c50ac60e07eb46dc8836de50bce12760 P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec8fff5744984f4695d768676cae8d17, State: Initialized, Role: FOLLOWER
I20260709 13:20:53.892922 21135 consensus_queue.cc:260] T c50ac60e07eb46dc8836de50bce12760 P ec8fff5744984f4695d768676cae8d17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:53.898579 21136 tablet_bootstrap.cc:492] T 7f3178c915104a27b9a3b6978249c5ed P bd6a13ebcb53485a9cc5082a4f64d652: No bootstrap required, opened a new log
I20260709 13:20:53.899314 21136 ts_tablet_manager.cc:1403] T 7f3178c915104a27b9a3b6978249c5ed P bd6a13ebcb53485a9cc5082a4f64d652: Time spent bootstrapping tablet: real 0.020s	user 0.009s	sys 0.009s
I20260709 13:20:53.901875 21135 ts_tablet_manager.cc:1434] T c50ac60e07eb46dc8836de50bce12760 P ec8fff5744984f4695d768676cae8d17: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 13:20:53.902621 21136 raft_consensus.cc:359] T 7f3178c915104a27b9a3b6978249c5ed P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:53.903472 21136 raft_consensus.cc:385] T 7f3178c915104a27b9a3b6978249c5ed P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:53.903509 21135 tablet_bootstrap.cc:492] T c3add26e31364c95a9897ab53b516735 P ec8fff5744984f4695d768676cae8d17: Bootstrap starting.
I20260709 13:20:53.903833 21136 raft_consensus.cc:740] T 7f3178c915104a27b9a3b6978249c5ed P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd6a13ebcb53485a9cc5082a4f64d652, State: Initialized, Role: FOLLOWER
I20260709 13:20:53.904855 21136 consensus_queue.cc:260] T 7f3178c915104a27b9a3b6978249c5ed P bd6a13ebcb53485a9cc5082a4f64d652 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:53.907850 21136 ts_tablet_manager.cc:1434] T 7f3178c915104a27b9a3b6978249c5ed P bd6a13ebcb53485a9cc5082a4f64d652: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.003s
I20260709 13:20:53.910104 20971 heartbeater.cc:499] Master 127.9.116.190:40921 was elected leader, sending a full tablet report...
I20260709 13:20:53.910269 21138 ts_tablet_manager.cc:1434] T c3add26e31364c95a9897ab53b516735 P 71bdb236d2304c75b7eabfec8f3d5a46: Time spent starting tablet: real 0.039s	user 0.014s	sys 0.011s
I20260709 13:20:53.911489 21138 tablet_bootstrap.cc:492] T 35d6d5be27cb4096917b886d06a3cf58 P 71bdb236d2304c75b7eabfec8f3d5a46: Bootstrap starting.
I20260709 13:20:53.912966 21135 tablet_bootstrap.cc:654] T c3add26e31364c95a9897ab53b516735 P ec8fff5744984f4695d768676cae8d17: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:53.909657 21136 tablet_bootstrap.cc:492] T c3add26e31364c95a9897ab53b516735 P bd6a13ebcb53485a9cc5082a4f64d652: Bootstrap starting.
I20260709 13:20:53.918217 21138 tablet_bootstrap.cc:654] T 35d6d5be27cb4096917b886d06a3cf58 P 71bdb236d2304c75b7eabfec8f3d5a46: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:53.923926 21136 tablet_bootstrap.cc:654] T c3add26e31364c95a9897ab53b516735 P bd6a13ebcb53485a9cc5082a4f64d652: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:53.924818 21135 tablet_bootstrap.cc:492] T c3add26e31364c95a9897ab53b516735 P ec8fff5744984f4695d768676cae8d17: No bootstrap required, opened a new log
I20260709 13:20:53.925442 21135 ts_tablet_manager.cc:1403] T c3add26e31364c95a9897ab53b516735 P ec8fff5744984f4695d768676cae8d17: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.007s
I20260709 13:20:53.929289 21135 raft_consensus.cc:359] T c3add26e31364c95a9897ab53b516735 P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:53.930115 21138 tablet_bootstrap.cc:492] T 35d6d5be27cb4096917b886d06a3cf58 P 71bdb236d2304c75b7eabfec8f3d5a46: No bootstrap required, opened a new log
I20260709 13:20:53.930342 21135 raft_consensus.cc:385] T c3add26e31364c95a9897ab53b516735 P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:53.930827 21138 ts_tablet_manager.cc:1403] T 35d6d5be27cb4096917b886d06a3cf58 P 71bdb236d2304c75b7eabfec8f3d5a46: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.003s
I20260709 13:20:53.930833 21135 raft_consensus.cc:740] T c3add26e31364c95a9897ab53b516735 P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec8fff5744984f4695d768676cae8d17, State: Initialized, Role: FOLLOWER
I20260709 13:20:53.932376 21135 consensus_queue.cc:260] T c3add26e31364c95a9897ab53b516735 P ec8fff5744984f4695d768676cae8d17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:53.934119 21138 raft_consensus.cc:359] T 35d6d5be27cb4096917b886d06a3cf58 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:53.934973 21138 raft_consensus.cc:385] T 35d6d5be27cb4096917b886d06a3cf58 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:53.935271 21138 raft_consensus.cc:740] T 35d6d5be27cb4096917b886d06a3cf58 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71bdb236d2304c75b7eabfec8f3d5a46, State: Initialized, Role: FOLLOWER
I20260709 13:20:53.936333 21138 consensus_queue.cc:260] T 35d6d5be27cb4096917b886d06a3cf58 P 71bdb236d2304c75b7eabfec8f3d5a46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:53.938262 21135 ts_tablet_manager.cc:1434] T c3add26e31364c95a9897ab53b516735 P ec8fff5744984f4695d768676cae8d17: Time spent starting tablet: real 0.012s	user 0.010s	sys 0.001s
I20260709 13:20:53.938942 21138 ts_tablet_manager.cc:1434] T 35d6d5be27cb4096917b886d06a3cf58 P 71bdb236d2304c75b7eabfec8f3d5a46: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
I20260709 13:20:53.939842 21135 tablet_bootstrap.cc:492] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17: Bootstrap starting.
I20260709 13:20:53.940095 21138 tablet_bootstrap.cc:492] T 35602162000346e3bbb790f54467ed0d P 71bdb236d2304c75b7eabfec8f3d5a46: Bootstrap starting.
I20260709 13:20:53.947954 21136 tablet_bootstrap.cc:492] T c3add26e31364c95a9897ab53b516735 P bd6a13ebcb53485a9cc5082a4f64d652: No bootstrap required, opened a new log
I20260709 13:20:53.948657 21136 ts_tablet_manager.cc:1403] T c3add26e31364c95a9897ab53b516735 P bd6a13ebcb53485a9cc5082a4f64d652: Time spent bootstrapping tablet: real 0.040s	user 0.014s	sys 0.016s
I20260709 13:20:53.949196 21135 tablet_bootstrap.cc:654] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:53.949263 21138 tablet_bootstrap.cc:654] T 35602162000346e3bbb790f54467ed0d P 71bdb236d2304c75b7eabfec8f3d5a46: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:53.952044 21136 raft_consensus.cc:359] T c3add26e31364c95a9897ab53b516735 P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:53.953145 21136 raft_consensus.cc:385] T c3add26e31364c95a9897ab53b516735 P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:53.953635 21136 raft_consensus.cc:740] T c3add26e31364c95a9897ab53b516735 P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd6a13ebcb53485a9cc5082a4f64d652, State: Initialized, Role: FOLLOWER
I20260709 13:20:53.954949 21136 consensus_queue.cc:260] T c3add26e31364c95a9897ab53b516735 P bd6a13ebcb53485a9cc5082a4f64d652 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:53.960343 21136 ts_tablet_manager.cc:1434] T c3add26e31364c95a9897ab53b516735 P bd6a13ebcb53485a9cc5082a4f64d652: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:20:53.961407 21135 tablet_bootstrap.cc:492] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17: No bootstrap required, opened a new log
I20260709 13:20:53.961478 21136 tablet_bootstrap.cc:492] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652: Bootstrap starting.
I20260709 13:20:53.962152 21135 ts_tablet_manager.cc:1403] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17: Time spent bootstrapping tablet: real 0.023s	user 0.010s	sys 0.009s
I20260709 13:20:53.965322 21135 raft_consensus.cc:359] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:53.966161 21135 raft_consensus.cc:385] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:53.966475 21135 raft_consensus.cc:740] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec8fff5744984f4695d768676cae8d17, State: Initialized, Role: FOLLOWER
I20260709 13:20:53.967279 21135 consensus_queue.cc:260] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:53.969852 21135 ts_tablet_manager.cc:1434] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:20:53.970979 21135 tablet_bootstrap.cc:492] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17: Bootstrap starting.
I20260709 13:20:53.972990 21136 tablet_bootstrap.cc:654] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:53.978339 21135 tablet_bootstrap.cc:654] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:53.990886 21135 tablet_bootstrap.cc:492] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17: No bootstrap required, opened a new log
I20260709 13:20:53.991824 21140 raft_consensus.cc:493] T 0f79d8e2c55b4a1f8380919ef723efe7 P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:53.992412 21140 raft_consensus.cc:515] T 0f79d8e2c55b4a1f8380919ef723efe7 P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:53.995342 21140 leader_election.cc:290] T 0f79d8e2c55b4a1f8380919ef723efe7 P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473), bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131:37203)
I20260709 13:20:53.991490 21135 ts_tablet_manager.cc:1403] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17: Time spent bootstrapping tablet: real 0.021s	user 0.013s	sys 0.003s
I20260709 13:20:53.996091 21138 tablet_bootstrap.cc:492] T 35602162000346e3bbb790f54467ed0d P 71bdb236d2304c75b7eabfec8f3d5a46: No bootstrap required, opened a new log
I20260709 13:20:53.996623 21136 tablet_bootstrap.cc:492] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652: No bootstrap required, opened a new log
I20260709 13:20:53.998809 21136 ts_tablet_manager.cc:1403] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652: Time spent bootstrapping tablet: real 0.038s	user 0.015s	sys 0.005s
I20260709 13:20:53.998718 21138 ts_tablet_manager.cc:1403] T 35602162000346e3bbb790f54467ed0d P 71bdb236d2304c75b7eabfec8f3d5a46: Time spent bootstrapping tablet: real 0.059s	user 0.023s	sys 0.031s
I20260709 13:20:53.998623 21135 raft_consensus.cc:359] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.001214 21135 raft_consensus.cc:385] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:54.001535 21135 raft_consensus.cc:740] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec8fff5744984f4695d768676cae8d17, State: Initialized, Role: FOLLOWER
I20260709 13:20:54.002413 21135 consensus_queue.cc:260] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.002882 21136 raft_consensus.cc:359] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.005155 21136 raft_consensus.cc:385] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:54.005632 21136 raft_consensus.cc:740] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd6a13ebcb53485a9cc5082a4f64d652, State: Initialized, Role: FOLLOWER
I20260709 13:20:54.004909 21138 raft_consensus.cc:359] T 35602162000346e3bbb790f54467ed0d P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.006176 21138 raft_consensus.cc:385] T 35602162000346e3bbb790f54467ed0d P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:54.007658 21138 raft_consensus.cc:740] T 35602162000346e3bbb790f54467ed0d P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71bdb236d2304c75b7eabfec8f3d5a46, State: Initialized, Role: FOLLOWER
I20260709 13:20:54.007496 21136 consensus_queue.cc:260] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.010105 21136 ts_tablet_manager.cc:1434] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:20:54.011221 21136 tablet_bootstrap.cc:492] T 5e579b46cc154814ad25486ba520710f P bd6a13ebcb53485a9cc5082a4f64d652: Bootstrap starting.
I20260709 13:20:54.012521 21135 ts_tablet_manager.cc:1434] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:20:54.008680 21138 consensus_queue.cc:260] T 35602162000346e3bbb790f54467ed0d P 71bdb236d2304c75b7eabfec8f3d5a46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.017669 21136 tablet_bootstrap.cc:654] T 5e579b46cc154814ad25486ba520710f P bd6a13ebcb53485a9cc5082a4f64d652: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:54.017849 21138 ts_tablet_manager.cc:1434] T 35602162000346e3bbb790f54467ed0d P 71bdb236d2304c75b7eabfec8f3d5a46: Time spent starting tablet: real 0.018s	user 0.003s	sys 0.004s
I20260709 13:20:54.021142 21135 tablet_bootstrap.cc:492] T b541a9ed669a420d8fbeb6e10ab3c47f P ec8fff5744984f4695d768676cae8d17: Bootstrap starting.
I20260709 13:20:54.023617 21138 tablet_bootstrap.cc:492] T b541a9ed669a420d8fbeb6e10ab3c47f P 71bdb236d2304c75b7eabfec8f3d5a46: Bootstrap starting.
I20260709 13:20:54.030529 21138 tablet_bootstrap.cc:654] T b541a9ed669a420d8fbeb6e10ab3c47f P 71bdb236d2304c75b7eabfec8f3d5a46: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:54.038048 21135 tablet_bootstrap.cc:654] T b541a9ed669a420d8fbeb6e10ab3c47f P ec8fff5744984f4695d768676cae8d17: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:54.045312 21138 tablet_bootstrap.cc:492] T b541a9ed669a420d8fbeb6e10ab3c47f P 71bdb236d2304c75b7eabfec8f3d5a46: No bootstrap required, opened a new log
I20260709 13:20:54.046020 21138 ts_tablet_manager.cc:1403] T b541a9ed669a420d8fbeb6e10ab3c47f P 71bdb236d2304c75b7eabfec8f3d5a46: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.006s
I20260709 13:20:54.047399 20946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f79d8e2c55b4a1f8380919ef723efe7" candidate_uuid: "ec8fff5744984f4695d768676cae8d17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" is_pre_election: true
W20260709 13:20:54.050041 20985 leader_election.cc:343] T 0f79d8e2c55b4a1f8380919ef723efe7 P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:54.050204 21138 raft_consensus.cc:359] T b541a9ed669a420d8fbeb6e10ab3c47f P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } }
I20260709 13:20:54.051251 21138 raft_consensus.cc:385] T b541a9ed669a420d8fbeb6e10ab3c47f P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:54.051635 21138 raft_consensus.cc:740] T b541a9ed669a420d8fbeb6e10ab3c47f P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71bdb236d2304c75b7eabfec8f3d5a46, State: Initialized, Role: FOLLOWER
I20260709 13:20:54.052039 21135 tablet_bootstrap.cc:492] T b541a9ed669a420d8fbeb6e10ab3c47f P ec8fff5744984f4695d768676cae8d17: No bootstrap required, opened a new log
I20260709 13:20:54.052641 21135 ts_tablet_manager.cc:1403] T b541a9ed669a420d8fbeb6e10ab3c47f P ec8fff5744984f4695d768676cae8d17: Time spent bootstrapping tablet: real 0.032s	user 0.012s	sys 0.005s
I20260709 13:20:54.052698 21138 consensus_queue.cc:260] T b541a9ed669a420d8fbeb6e10ab3c47f P 71bdb236d2304c75b7eabfec8f3d5a46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } }
I20260709 13:20:54.053167 21095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f79d8e2c55b4a1f8380919ef723efe7" candidate_uuid: "ec8fff5744984f4695d768676cae8d17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" is_pre_election: true
I20260709 13:20:54.054276 21095 raft_consensus.cc:2468] T 0f79d8e2c55b4a1f8380919ef723efe7 P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec8fff5744984f4695d768676cae8d17 in term 0.
I20260709 13:20:54.056057 21135 raft_consensus.cc:359] T b541a9ed669a420d8fbeb6e10ab3c47f P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } }
I20260709 13:20:54.056918 21135 raft_consensus.cc:385] T b541a9ed669a420d8fbeb6e10ab3c47f P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:54.056499 20985 leader_election.cc:304] T 0f79d8e2c55b4a1f8380919ef723efe7 P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bd6a13ebcb53485a9cc5082a4f64d652, ec8fff5744984f4695d768676cae8d17; no voters: 71bdb236d2304c75b7eabfec8f3d5a46
I20260709 13:20:54.057260 21135 raft_consensus.cc:740] T b541a9ed669a420d8fbeb6e10ab3c47f P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec8fff5744984f4695d768676cae8d17, State: Initialized, Role: FOLLOWER
I20260709 13:20:54.058415 21140 raft_consensus.cc:2804] T 0f79d8e2c55b4a1f8380919ef723efe7 P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:54.058276 21135 consensus_queue.cc:260] T b541a9ed669a420d8fbeb6e10ab3c47f P ec8fff5744984f4695d768676cae8d17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } }
I20260709 13:20:54.059039 21140 raft_consensus.cc:493] T 0f79d8e2c55b4a1f8380919ef723efe7 P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:54.059599 21140 raft_consensus.cc:3060] T 0f79d8e2c55b4a1f8380919ef723efe7 P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:54.061367 21138 ts_tablet_manager.cc:1434] T b541a9ed669a420d8fbeb6e10ab3c47f P 71bdb236d2304c75b7eabfec8f3d5a46: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.004s
I20260709 13:20:54.069566 21144 raft_consensus.cc:493] T c3add26e31364c95a9897ab53b516735 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:54.069842 21138 tablet_bootstrap.cc:492] T 0f79d8e2c55b4a1f8380919ef723efe7 P 71bdb236d2304c75b7eabfec8f3d5a46: Bootstrap starting.
I20260709 13:20:54.070278 21144 raft_consensus.cc:515] T c3add26e31364c95a9897ab53b516735 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.078166 21138 tablet_bootstrap.cc:654] T 0f79d8e2c55b4a1f8380919ef723efe7 P 71bdb236d2304c75b7eabfec8f3d5a46: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:54.079416 21144 leader_election.cc:290] T c3add26e31364c95a9897ab53b516735 P 71bdb236d2304c75b7eabfec8f3d5a46 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec8fff5744984f4695d768676cae8d17 (127.9.116.130:42659), bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131:37203)
I20260709 13:20:54.077838 21140 raft_consensus.cc:515] T 0f79d8e2c55b4a1f8380919ef723efe7 P ec8fff5744984f4695d768676cae8d17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.085271 21095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f79d8e2c55b4a1f8380919ef723efe7" candidate_uuid: "ec8fff5744984f4695d768676cae8d17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652"
I20260709 13:20:54.086102 21095 raft_consensus.cc:3060] T 0f79d8e2c55b4a1f8380919ef723efe7 P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:54.087977 21135 ts_tablet_manager.cc:1434] T b541a9ed669a420d8fbeb6e10ab3c47f P ec8fff5744984f4695d768676cae8d17: Time spent starting tablet: real 0.035s	user 0.011s	sys 0.007s
I20260709 13:20:54.091010 20946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f79d8e2c55b4a1f8380919ef723efe7" candidate_uuid: "ec8fff5744984f4695d768676cae8d17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71bdb236d2304c75b7eabfec8f3d5a46"
I20260709 13:20:54.092005 21136 tablet_bootstrap.cc:492] T 5e579b46cc154814ad25486ba520710f P bd6a13ebcb53485a9cc5082a4f64d652: No bootstrap required, opened a new log
W20260709 13:20:54.092423 20985 leader_election.cc:343] T 0f79d8e2c55b4a1f8380919ef723efe7 P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:54.092581 21136 ts_tablet_manager.cc:1403] T 5e579b46cc154814ad25486ba520710f P bd6a13ebcb53485a9cc5082a4f64d652: Time spent bootstrapping tablet: real 0.082s	user 0.018s	sys 0.048s
I20260709 13:20:54.092492 21021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3add26e31364c95a9897ab53b516735" candidate_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec8fff5744984f4695d768676cae8d17" is_pre_election: true
I20260709 13:20:54.093500 21021 raft_consensus.cc:2468] T c3add26e31364c95a9897ab53b516735 P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71bdb236d2304c75b7eabfec8f3d5a46 in term 0.
I20260709 13:20:54.095036 20912 leader_election.cc:304] T c3add26e31364c95a9897ab53b516735 P 71bdb236d2304c75b7eabfec8f3d5a46 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71bdb236d2304c75b7eabfec8f3d5a46, ec8fff5744984f4695d768676cae8d17; no voters: 
I20260709 13:20:54.095638 21140 leader_election.cc:290] T 0f79d8e2c55b4a1f8380919ef723efe7 P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 1 election: Requested vote from peers 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473), bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131:37203)
I20260709 13:20:54.099825 21135 tablet_bootstrap.cc:492] T 5e579b46cc154814ad25486ba520710f P ec8fff5744984f4695d768676cae8d17: Bootstrap starting.
I20260709 13:20:54.096024 21144 raft_consensus.cc:2804] T c3add26e31364c95a9897ab53b516735 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:54.095873 21136 raft_consensus.cc:359] T 5e579b46cc154814ad25486ba520710f P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.099527 21138 tablet_bootstrap.cc:492] T 0f79d8e2c55b4a1f8380919ef723efe7 P 71bdb236d2304c75b7eabfec8f3d5a46: No bootstrap required, opened a new log
I20260709 13:20:54.100538 21144 raft_consensus.cc:493] T c3add26e31364c95a9897ab53b516735 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:54.100659 21136 raft_consensus.cc:385] T 5e579b46cc154814ad25486ba520710f P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:54.101006 21144 raft_consensus.cc:3060] T c3add26e31364c95a9897ab53b516735 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:54.101044 21138 ts_tablet_manager.cc:1403] T 0f79d8e2c55b4a1f8380919ef723efe7 P 71bdb236d2304c75b7eabfec8f3d5a46: Time spent bootstrapping tablet: real 0.031s	user 0.014s	sys 0.000s
I20260709 13:20:54.101114 21136 raft_consensus.cc:740] T 5e579b46cc154814ad25486ba520710f P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd6a13ebcb53485a9cc5082a4f64d652, State: Initialized, Role: FOLLOWER
I20260709 13:20:54.102281 21136 consensus_queue.cc:260] T 5e579b46cc154814ad25486ba520710f P bd6a13ebcb53485a9cc5082a4f64d652 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
W20260709 13:20:54.105235 20892 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:20:54.107128 21135 tablet_bootstrap.cc:654] T 5e579b46cc154814ad25486ba520710f P ec8fff5744984f4695d768676cae8d17: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:54.107029 21138 raft_consensus.cc:359] T 0f79d8e2c55b4a1f8380919ef723efe7 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.108109 21138 raft_consensus.cc:385] T 0f79d8e2c55b4a1f8380919ef723efe7 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:54.109015 21138 raft_consensus.cc:740] T 0f79d8e2c55b4a1f8380919ef723efe7 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71bdb236d2304c75b7eabfec8f3d5a46, State: Initialized, Role: FOLLOWER
I20260709 13:20:54.109392 21144 raft_consensus.cc:515] T c3add26e31364c95a9897ab53b516735 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.109965 21138 consensus_queue.cc:260] T 0f79d8e2c55b4a1f8380919ef723efe7 P 71bdb236d2304c75b7eabfec8f3d5a46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.112814 21136 ts_tablet_manager.cc:1434] T 5e579b46cc154814ad25486ba520710f P bd6a13ebcb53485a9cc5082a4f64d652: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 13:20:54.113106 21021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3add26e31364c95a9897ab53b516735" candidate_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec8fff5744984f4695d768676cae8d17"
I20260709 13:20:54.113972 21021 raft_consensus.cc:3060] T c3add26e31364c95a9897ab53b516735 P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:54.119818 21138 ts_tablet_manager.cc:1434] T 0f79d8e2c55b4a1f8380919ef723efe7 P 71bdb236d2304c75b7eabfec8f3d5a46: Time spent starting tablet: real 0.018s	user 0.013s	sys 0.000s
I20260709 13:20:54.122043 21136 tablet_bootstrap.cc:492] T ee997af6d38745549698356bcbc09a99 P bd6a13ebcb53485a9cc5082a4f64d652: Bootstrap starting.
I20260709 13:20:54.123723 21138 tablet_bootstrap.cc:492] T ee997af6d38745549698356bcbc09a99 P 71bdb236d2304c75b7eabfec8f3d5a46: Bootstrap starting.
I20260709 13:20:54.097694 21095 raft_consensus.cc:2468] T 0f79d8e2c55b4a1f8380919ef723efe7 P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec8fff5744984f4695d768676cae8d17 in term 1.
I20260709 13:20:54.127543 20985 leader_election.cc:304] T 0f79d8e2c55b4a1f8380919ef723efe7 P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bd6a13ebcb53485a9cc5082a4f64d652, ec8fff5744984f4695d768676cae8d17; no voters: 71bdb236d2304c75b7eabfec8f3d5a46
I20260709 13:20:54.129151 21140 raft_consensus.cc:2804] T 0f79d8e2c55b4a1f8380919ef723efe7 P ec8fff5744984f4695d768676cae8d17 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:54.129804 21140 raft_consensus.cc:697] T 0f79d8e2c55b4a1f8380919ef723efe7 P ec8fff5744984f4695d768676cae8d17 [term 1 LEADER]: Becoming Leader. State: Replica: ec8fff5744984f4695d768676cae8d17, State: Running, Role: LEADER
I20260709 13:20:54.130697 21140 consensus_queue.cc:237] T 0f79d8e2c55b4a1f8380919ef723efe7 P ec8fff5744984f4695d768676cae8d17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.134124 21138 tablet_bootstrap.cc:654] T ee997af6d38745549698356bcbc09a99 P 71bdb236d2304c75b7eabfec8f3d5a46: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:54.135619 21136 tablet_bootstrap.cc:654] T ee997af6d38745549698356bcbc09a99 P bd6a13ebcb53485a9cc5082a4f64d652: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:54.141398 21021 raft_consensus.cc:2468] T c3add26e31364c95a9897ab53b516735 P ec8fff5744984f4695d768676cae8d17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71bdb236d2304c75b7eabfec8f3d5a46 in term 1.
I20260709 13:20:54.143002 20912 leader_election.cc:304] T c3add26e31364c95a9897ab53b516735 P 71bdb236d2304c75b7eabfec8f3d5a46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71bdb236d2304c75b7eabfec8f3d5a46, ec8fff5744984f4695d768676cae8d17; no voters: 
I20260709 13:20:54.144335 21163 raft_consensus.cc:2804] T c3add26e31364c95a9897ab53b516735 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:54.145282 21144 leader_election.cc:290] T c3add26e31364c95a9897ab53b516735 P 71bdb236d2304c75b7eabfec8f3d5a46 [CANDIDATE]: Term 1 election: Requested vote from peers ec8fff5744984f4695d768676cae8d17 (127.9.116.130:42659), bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131:37203)
I20260709 13:20:54.148195 21163 raft_consensus.cc:697] T c3add26e31364c95a9897ab53b516735 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 LEADER]: Becoming Leader. State: Replica: 71bdb236d2304c75b7eabfec8f3d5a46, State: Running, Role: LEADER
I20260709 13:20:54.149647 21163 consensus_queue.cc:237] T c3add26e31364c95a9897ab53b516735 P 71bdb236d2304c75b7eabfec8f3d5a46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.153081 21094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3add26e31364c95a9897ab53b516735" candidate_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652"
I20260709 13:20:54.153815 21094 raft_consensus.cc:3060] T c3add26e31364c95a9897ab53b516735 P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:54.154457 21144 raft_consensus.cc:493] T 0f79d8e2c55b4a1f8380919ef723efe7 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:54.154943 20844 catalog_manager.cc:5697] T 0f79d8e2c55b4a1f8380919ef723efe7 P ec8fff5744984f4695d768676cae8d17 reported cstate change: term changed from 0 to 1, leader changed from <none> to ec8fff5744984f4695d768676cae8d17 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "ec8fff5744984f4695d768676cae8d17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:54.159060 21136 tablet_bootstrap.cc:492] T ee997af6d38745549698356bcbc09a99 P bd6a13ebcb53485a9cc5082a4f64d652: No bootstrap required, opened a new log
I20260709 13:20:54.159925 21136 ts_tablet_manager.cc:1403] T ee997af6d38745549698356bcbc09a99 P bd6a13ebcb53485a9cc5082a4f64d652: Time spent bootstrapping tablet: real 0.038s	user 0.011s	sys 0.019s
I20260709 13:20:54.159891 21095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3add26e31364c95a9897ab53b516735" candidate_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" is_pre_election: true
I20260709 13:20:54.159161 21144 raft_consensus.cc:515] T 0f79d8e2c55b4a1f8380919ef723efe7 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.163771 21136 raft_consensus.cc:359] T ee997af6d38745549698356bcbc09a99 P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.163968 21020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f79d8e2c55b4a1f8380919ef723efe7" candidate_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec8fff5744984f4695d768676cae8d17" is_pre_election: true
I20260709 13:20:54.164578 21136 raft_consensus.cc:385] T ee997af6d38745549698356bcbc09a99 P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:54.164969 21136 raft_consensus.cc:740] T ee997af6d38745549698356bcbc09a99 P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd6a13ebcb53485a9cc5082a4f64d652, State: Initialized, Role: FOLLOWER
I20260709 13:20:54.165457 21138 tablet_bootstrap.cc:492] T ee997af6d38745549698356bcbc09a99 P 71bdb236d2304c75b7eabfec8f3d5a46: No bootstrap required, opened a new log
I20260709 13:20:54.166111 21138 ts_tablet_manager.cc:1403] T ee997af6d38745549698356bcbc09a99 P 71bdb236d2304c75b7eabfec8f3d5a46: Time spent bootstrapping tablet: real 0.043s	user 0.015s	sys 0.004s
I20260709 13:20:54.165923 21136 consensus_queue.cc:260] T ee997af6d38745549698356bcbc09a99 P bd6a13ebcb53485a9cc5082a4f64d652 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.170042 21138 raft_consensus.cc:359] T ee997af6d38745549698356bcbc09a99 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.177346 21138 raft_consensus.cc:385] T ee997af6d38745549698356bcbc09a99 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:54.172940 21095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f79d8e2c55b4a1f8380919ef723efe7" candidate_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" is_pre_election: true
I20260709 13:20:54.173624 21094 raft_consensus.cc:2468] T c3add26e31364c95a9897ab53b516735 P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71bdb236d2304c75b7eabfec8f3d5a46 in term 1.
I20260709 13:20:54.172609 21144 leader_election.cc:290] T 0f79d8e2c55b4a1f8380919ef723efe7 P 71bdb236d2304c75b7eabfec8f3d5a46 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec8fff5744984f4695d768676cae8d17 (127.9.116.130:42659), bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131:37203)
I20260709 13:20:54.177865 21138 raft_consensus.cc:740] T ee997af6d38745549698356bcbc09a99 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71bdb236d2304c75b7eabfec8f3d5a46, State: Initialized, Role: FOLLOWER
I20260709 13:20:54.178215 21095 raft_consensus.cc:2393] T 0f79d8e2c55b4a1f8380919ef723efe7 P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 71bdb236d2304c75b7eabfec8f3d5a46 in current term 1: Already voted for candidate ec8fff5744984f4695d768676cae8d17 in this term.
I20260709 13:20:54.177217 21136 ts_tablet_manager.cc:1434] T ee997af6d38745549698356bcbc09a99 P bd6a13ebcb53485a9cc5082a4f64d652: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:20:54.179152 21138 consensus_queue.cc:260] T ee997af6d38745549698356bcbc09a99 P 71bdb236d2304c75b7eabfec8f3d5a46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.179978 20910 leader_election.cc:304] T 0f79d8e2c55b4a1f8380919ef723efe7 P 71bdb236d2304c75b7eabfec8f3d5a46 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71bdb236d2304c75b7eabfec8f3d5a46; no voters: bd6a13ebcb53485a9cc5082a4f64d652, ec8fff5744984f4695d768676cae8d17
I20260709 13:20:54.180869 21144 raft_consensus.cc:3060] T 0f79d8e2c55b4a1f8380919ef723efe7 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:54.181861 21135 tablet_bootstrap.cc:492] T 5e579b46cc154814ad25486ba520710f P ec8fff5744984f4695d768676cae8d17: No bootstrap required, opened a new log
I20260709 13:20:54.182078 21136 tablet_bootstrap.cc:492] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652: Bootstrap starting.
I20260709 13:20:54.182384 21135 ts_tablet_manager.cc:1403] T 5e579b46cc154814ad25486ba520710f P ec8fff5744984f4695d768676cae8d17: Time spent bootstrapping tablet: real 0.083s	user 0.011s	sys 0.043s
I20260709 13:20:54.184933 21135 raft_consensus.cc:359] T 5e579b46cc154814ad25486ba520710f P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.185462 21135 raft_consensus.cc:385] T 5e579b46cc154814ad25486ba520710f P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:54.185664 21135 raft_consensus.cc:740] T 5e579b46cc154814ad25486ba520710f P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec8fff5744984f4695d768676cae8d17, State: Initialized, Role: FOLLOWER
I20260709 13:20:54.186259 21135 consensus_queue.cc:260] T 5e579b46cc154814ad25486ba520710f P ec8fff5744984f4695d768676cae8d17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.187585 21140 raft_consensus.cc:493] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:54.188362 21138 ts_tablet_manager.cc:1434] T ee997af6d38745549698356bcbc09a99 P 71bdb236d2304c75b7eabfec8f3d5a46: Time spent starting tablet: real 0.021s	user 0.002s	sys 0.005s
I20260709 13:20:54.188087 21140 raft_consensus.cc:515] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.188498 21135 ts_tablet_manager.cc:1434] T 5e579b46cc154814ad25486ba520710f P ec8fff5744984f4695d768676cae8d17: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.003s
I20260709 13:20:54.189234 21136 tablet_bootstrap.cc:654] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:54.192137 21135 tablet_bootstrap.cc:492] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17: Bootstrap starting.
I20260709 13:20:54.192795 21140 leader_election.cc:290] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473), bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131:37203)
I20260709 13:20:54.195595 21095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35602162000346e3bbb790f54467ed0d" candidate_uuid: "ec8fff5744984f4695d768676cae8d17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" is_pre_election: true
I20260709 13:20:54.196842 21138 tablet_bootstrap.cc:492] T c50ac60e07eb46dc8836de50bce12760 P 71bdb236d2304c75b7eabfec8f3d5a46: Bootstrap starting.
W20260709 13:20:54.197593 20985 leader_election.cc:343] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131:37203): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:54.195990 20842 catalog_manager.cc:5697] T c3add26e31364c95a9897ab53b516735 P 71bdb236d2304c75b7eabfec8f3d5a46 reported cstate change: term changed from 0 to 1, leader changed from <none> to 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129). New cstate: current_term: 1 leader_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:54.197937 21143 raft_consensus.cc:493] T ee997af6d38745549698356bcbc09a99 P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:54.198302 21144 raft_consensus.cc:2749] T 0f79d8e2c55b4a1f8380919ef723efe7 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:54.198523 21143 raft_consensus.cc:515] T ee997af6d38745549698356bcbc09a99 P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.197885 20946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35602162000346e3bbb790f54467ed0d" candidate_uuid: "ec8fff5744984f4695d768676cae8d17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" is_pre_election: true
I20260709 13:20:54.203274 20946 raft_consensus.cc:2468] T 35602162000346e3bbb790f54467ed0d P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec8fff5744984f4695d768676cae8d17 in term 0.
I20260709 13:20:54.205346 20985 leader_election.cc:304] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 71bdb236d2304c75b7eabfec8f3d5a46, ec8fff5744984f4695d768676cae8d17; no voters: bd6a13ebcb53485a9cc5082a4f64d652
I20260709 13:20:54.205940 21136 tablet_bootstrap.cc:492] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652: No bootstrap required, opened a new log
I20260709 13:20:54.205543 21143 leader_election.cc:290] T ee997af6d38745549698356bcbc09a99 P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473), ec8fff5744984f4695d768676cae8d17 (127.9.116.130:42659)
I20260709 13:20:54.207139 21136 ts_tablet_manager.cc:1403] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.004s
I20260709 13:20:54.209729 21140 raft_consensus.cc:2804] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:54.203250 21135 tablet_bootstrap.cc:654] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:54.210361 21140 raft_consensus.cc:493] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:54.211220 21140 raft_consensus.cc:3060] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:54.211975 21136 raft_consensus.cc:359] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } }
I20260709 13:20:54.213922 21136 raft_consensus.cc:385] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:54.214234 21136 raft_consensus.cc:740] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd6a13ebcb53485a9cc5082a4f64d652, State: Initialized, Role: FOLLOWER
I20260709 13:20:54.215273 21136 consensus_queue.cc:260] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } }
I20260709 13:20:54.221868 21158 raft_consensus.cc:493] T c50ac60e07eb46dc8836de50bce12760 P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:54.222465 21158 raft_consensus.cc:515] T c50ac60e07eb46dc8836de50bce12760 P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.222908 21136 ts_tablet_manager.cc:1434] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.001s
I20260709 13:20:54.223994 21136 tablet_bootstrap.cc:492] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652: Bootstrap starting.
I20260709 13:20:54.224663 21158 leader_election.cc:290] T c50ac60e07eb46dc8836de50bce12760 P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473), bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131:37203)
I20260709 13:20:54.226635 20946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50ac60e07eb46dc8836de50bce12760" candidate_uuid: "ec8fff5744984f4695d768676cae8d17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" is_pre_election: true
W20260709 13:20:54.228294 20985 leader_election.cc:343] T c50ac60e07eb46dc8836de50bce12760 P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:54.237828 21136 tablet_bootstrap.cc:654] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:54.239588 21138 tablet_bootstrap.cc:654] T c50ac60e07eb46dc8836de50bce12760 P 71bdb236d2304c75b7eabfec8f3d5a46: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:54.244675 21140 raft_consensus.cc:515] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.246973 21095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50ac60e07eb46dc8836de50bce12760" candidate_uuid: "ec8fff5744984f4695d768676cae8d17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" is_pre_election: true
W20260709 13:20:54.248811 20985 leader_election.cc:343] T c50ac60e07eb46dc8836de50bce12760 P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131:37203): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:54.249457 20985 leader_election.cc:304] T c50ac60e07eb46dc8836de50bce12760 P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec8fff5744984f4695d768676cae8d17; no voters: 71bdb236d2304c75b7eabfec8f3d5a46, bd6a13ebcb53485a9cc5082a4f64d652
I20260709 13:20:54.250990 21158 raft_consensus.cc:2749] T c50ac60e07eb46dc8836de50bce12760 P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:54.256598 21143 raft_consensus.cc:493] T 7f3178c915104a27b9a3b6978249c5ed P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:54.257421 21143 raft_consensus.cc:515] T 7f3178c915104a27b9a3b6978249c5ed P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.262053 21143 leader_election.cc:290] T 7f3178c915104a27b9a3b6978249c5ed P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473), ec8fff5744984f4695d768676cae8d17 (127.9.116.130:42659)
I20260709 13:20:54.262751 20946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35602162000346e3bbb790f54467ed0d" candidate_uuid: "ec8fff5744984f4695d768676cae8d17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71bdb236d2304c75b7eabfec8f3d5a46"
I20260709 13:20:54.263507 20946 raft_consensus.cc:3060] T 35602162000346e3bbb790f54467ed0d P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:54.264076 21095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35602162000346e3bbb790f54467ed0d" candidate_uuid: "ec8fff5744984f4695d768676cae8d17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652"
I20260709 13:20:54.265736 21135 tablet_bootstrap.cc:492] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17: No bootstrap required, opened a new log
W20260709 13:20:54.265712 20985 leader_election.cc:343] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131:37203): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:54.266206 21140 leader_election.cc:290] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 1 election: Requested vote from peers 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473), bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131:37203)
I20260709 13:20:54.266855 21135 ts_tablet_manager.cc:1403] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17: Time spent bootstrapping tablet: real 0.075s	user 0.010s	sys 0.007s
I20260709 13:20:54.272934 21136 tablet_bootstrap.cc:492] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652: No bootstrap required, opened a new log
I20260709 13:20:54.272833 21021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee997af6d38745549698356bcbc09a99" candidate_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec8fff5744984f4695d768676cae8d17" is_pre_election: true
I20260709 13:20:54.273800 21136 ts_tablet_manager.cc:1403] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652: Time spent bootstrapping tablet: real 0.050s	user 0.006s	sys 0.023s
W20260709 13:20:54.275493 21061 leader_election.cc:343] T ee997af6d38745549698356bcbc09a99 P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ec8fff5744984f4695d768676cae8d17 (127.9.116.130:42659): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:54.273195 21020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f3178c915104a27b9a3b6978249c5ed" candidate_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec8fff5744984f4695d768676cae8d17" is_pre_election: true
I20260709 13:20:54.276713 21136 raft_consensus.cc:359] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.277446 21136 raft_consensus.cc:385] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:54.277719 21136 raft_consensus.cc:740] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd6a13ebcb53485a9cc5082a4f64d652, State: Initialized, Role: FOLLOWER
I20260709 13:20:54.278548 21136 consensus_queue.cc:260] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.280670 21136 ts_tablet_manager.cc:1434] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:20:54.281600 21136 tablet_bootstrap.cc:492] T 35d6d5be27cb4096917b886d06a3cf58 P bd6a13ebcb53485a9cc5082a4f64d652: Bootstrap starting.
W20260709 13:20:54.281409 21061 leader_election.cc:343] T 7f3178c915104a27b9a3b6978249c5ed P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ec8fff5744984f4695d768676cae8d17 (127.9.116.130:42659): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:54.272814 20946 raft_consensus.cc:2468] T 35602162000346e3bbb790f54467ed0d P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec8fff5744984f4695d768676cae8d17 in term 1.
I20260709 13:20:54.283826 20985 leader_election.cc:304] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 71bdb236d2304c75b7eabfec8f3d5a46, ec8fff5744984f4695d768676cae8d17; no voters: bd6a13ebcb53485a9cc5082a4f64d652
I20260709 13:20:54.285439 21140 raft_consensus.cc:2804] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:54.286070 21140 raft_consensus.cc:697] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [term 1 LEADER]: Becoming Leader. State: Replica: ec8fff5744984f4695d768676cae8d17, State: Running, Role: LEADER
I20260709 13:20:54.287410 21136 tablet_bootstrap.cc:654] T 35d6d5be27cb4096917b886d06a3cf58 P bd6a13ebcb53485a9cc5082a4f64d652: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:54.287042 21140 consensus_queue.cc:237] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.278883 21135 raft_consensus.cc:359] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.298971 20945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f3178c915104a27b9a3b6978249c5ed" candidate_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" is_pre_election: true
I20260709 13:20:54.299445 21138 tablet_bootstrap.cc:492] T c50ac60e07eb46dc8836de50bce12760 P 71bdb236d2304c75b7eabfec8f3d5a46: No bootstrap required, opened a new log
I20260709 13:20:54.302222 21138 ts_tablet_manager.cc:1403] T c50ac60e07eb46dc8836de50bce12760 P 71bdb236d2304c75b7eabfec8f3d5a46: Time spent bootstrapping tablet: real 0.106s	user 0.016s	sys 0.021s
W20260709 13:20:54.302074 21059 leader_election.cc:343] T 7f3178c915104a27b9a3b6978249c5ed P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:54.302819 21059 leader_election.cc:304] T 7f3178c915104a27b9a3b6978249c5ed P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd6a13ebcb53485a9cc5082a4f64d652; no voters: 71bdb236d2304c75b7eabfec8f3d5a46, ec8fff5744984f4695d768676cae8d17
I20260709 13:20:54.302117 20946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee997af6d38745549698356bcbc09a99" candidate_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" is_pre_election: true
I20260709 13:20:54.303951 20946 raft_consensus.cc:2468] T ee997af6d38745549698356bcbc09a99 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd6a13ebcb53485a9cc5082a4f64d652 in term 0.
I20260709 13:20:54.304122 21143 raft_consensus.cc:2749] T 7f3178c915104a27b9a3b6978249c5ed P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:54.305560 20842 catalog_manager.cc:5697] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 reported cstate change: term changed from 0 to 1, leader changed from <none> to ec8fff5744984f4695d768676cae8d17 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "ec8fff5744984f4695d768676cae8d17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:54.306598 21059 leader_election.cc:304] T ee997af6d38745549698356bcbc09a99 P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 71bdb236d2304c75b7eabfec8f3d5a46, bd6a13ebcb53485a9cc5082a4f64d652; no voters: ec8fff5744984f4695d768676cae8d17
I20260709 13:20:54.307790 21138 raft_consensus.cc:359] T c50ac60e07eb46dc8836de50bce12760 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.308193 21143 raft_consensus.cc:2804] T ee997af6d38745549698356bcbc09a99 P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:54.308802 21143 raft_consensus.cc:493] T ee997af6d38745549698356bcbc09a99 P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:54.309302 21143 raft_consensus.cc:3060] T ee997af6d38745549698356bcbc09a99 P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:54.308560 21138 raft_consensus.cc:385] T c50ac60e07eb46dc8836de50bce12760 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:54.312906 21138 raft_consensus.cc:740] T c50ac60e07eb46dc8836de50bce12760 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71bdb236d2304c75b7eabfec8f3d5a46, State: Initialized, Role: FOLLOWER
I20260709 13:20:54.313804 21138 consensus_queue.cc:260] T c50ac60e07eb46dc8836de50bce12760 P 71bdb236d2304c75b7eabfec8f3d5a46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.314641 21136 tablet_bootstrap.cc:492] T 35d6d5be27cb4096917b886d06a3cf58 P bd6a13ebcb53485a9cc5082a4f64d652: No bootstrap required, opened a new log
I20260709 13:20:54.315214 21136 ts_tablet_manager.cc:1403] T 35d6d5be27cb4096917b886d06a3cf58 P bd6a13ebcb53485a9cc5082a4f64d652: Time spent bootstrapping tablet: real 0.034s	user 0.017s	sys 0.015s
I20260709 13:20:54.318040 21136 raft_consensus.cc:359] T 35d6d5be27cb4096917b886d06a3cf58 P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.318753 21136 raft_consensus.cc:385] T 35d6d5be27cb4096917b886d06a3cf58 P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:54.319269 21138 ts_tablet_manager.cc:1434] T c50ac60e07eb46dc8836de50bce12760 P 71bdb236d2304c75b7eabfec8f3d5a46: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:20:54.319445 21136 raft_consensus.cc:740] T 35d6d5be27cb4096917b886d06a3cf58 P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd6a13ebcb53485a9cc5082a4f64d652, State: Initialized, Role: FOLLOWER
I20260709 13:20:54.318606 21143 raft_consensus.cc:515] T ee997af6d38745549698356bcbc09a99 P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.318846 21135 raft_consensus.cc:385] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:54.320511 21138 tablet_bootstrap.cc:492] T 5e579b46cc154814ad25486ba520710f P 71bdb236d2304c75b7eabfec8f3d5a46: Bootstrap starting.
I20260709 13:20:54.320595 21135 raft_consensus.cc:740] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec8fff5744984f4695d768676cae8d17, State: Initialized, Role: FOLLOWER
I20260709 13:20:54.321636 21135 consensus_queue.cc:260] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.322439 20946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee997af6d38745549698356bcbc09a99" candidate_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71bdb236d2304c75b7eabfec8f3d5a46"
I20260709 13:20:54.323071 20946 raft_consensus.cc:3060] T ee997af6d38745549698356bcbc09a99 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:54.323764 21135 ts_tablet_manager.cc:1434] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17: Time spent starting tablet: real 0.052s	user 0.006s	sys 0.000s
I20260709 13:20:54.324738 21135 tablet_bootstrap.cc:492] T ee997af6d38745549698356bcbc09a99 P ec8fff5744984f4695d768676cae8d17: Bootstrap starting.
I20260709 13:20:54.327068 21136 consensus_queue.cc:260] T 35d6d5be27cb4096917b886d06a3cf58 P bd6a13ebcb53485a9cc5082a4f64d652 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.327781 21138 tablet_bootstrap.cc:654] T 5e579b46cc154814ad25486ba520710f P 71bdb236d2304c75b7eabfec8f3d5a46: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:54.332245 21135 tablet_bootstrap.cc:654] T ee997af6d38745549698356bcbc09a99 P ec8fff5744984f4695d768676cae8d17: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:54.337968 20946 raft_consensus.cc:2468] T ee997af6d38745549698356bcbc09a99 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd6a13ebcb53485a9cc5082a4f64d652 in term 1.
I20260709 13:20:54.341552 21059 leader_election.cc:304] T ee997af6d38745549698356bcbc09a99 P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71bdb236d2304c75b7eabfec8f3d5a46, bd6a13ebcb53485a9cc5082a4f64d652; no voters: 
I20260709 13:20:54.343426 21143 leader_election.cc:290] T ee997af6d38745549698356bcbc09a99 P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 1 election: Requested vote from peers 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473), ec8fff5744984f4695d768676cae8d17 (127.9.116.130:42659)
I20260709 13:20:54.343875 21185 raft_consensus.cc:493] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:54.344391 21143 raft_consensus.cc:2804] T ee997af6d38745549698356bcbc09a99 P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:54.344444 21185 raft_consensus.cc:515] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.345037 21143 raft_consensus.cc:697] T ee997af6d38745549698356bcbc09a99 P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 LEADER]: Becoming Leader. State: Replica: bd6a13ebcb53485a9cc5082a4f64d652, State: Running, Role: LEADER
I20260709 13:20:54.345157 21020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee997af6d38745549698356bcbc09a99" candidate_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec8fff5744984f4695d768676cae8d17"
I20260709 13:20:54.346145 21143 consensus_queue.cc:237] T ee997af6d38745549698356bcbc09a99 P bd6a13ebcb53485a9cc5082a4f64d652 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.348312 20946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725109df1524e6c8eb897f0e6424829" candidate_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" is_pre_election: true
W20260709 13:20:54.350314 21061 leader_election.cc:343] T ee997af6d38745549698356bcbc09a99 P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ec8fff5744984f4695d768676cae8d17 (127.9.116.130:42659): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:20:54.351428 21059 leader_election.cc:343] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:54.354197 21135 tablet_bootstrap.cc:492] T ee997af6d38745549698356bcbc09a99 P ec8fff5744984f4695d768676cae8d17: No bootstrap required, opened a new log
I20260709 13:20:54.356276 21136 ts_tablet_manager.cc:1434] T 35d6d5be27cb4096917b886d06a3cf58 P bd6a13ebcb53485a9cc5082a4f64d652: Time spent starting tablet: real 0.041s	user 0.007s	sys 0.009s
I20260709 13:20:54.357496 21138 tablet_bootstrap.cc:492] T 5e579b46cc154814ad25486ba520710f P 71bdb236d2304c75b7eabfec8f3d5a46: No bootstrap required, opened a new log
I20260709 13:20:54.357682 21136 tablet_bootstrap.cc:492] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652: Bootstrap starting.
I20260709 13:20:54.358019 21138 ts_tablet_manager.cc:1403] T 5e579b46cc154814ad25486ba520710f P 71bdb236d2304c75b7eabfec8f3d5a46: Time spent bootstrapping tablet: real 0.038s	user 0.024s	sys 0.011s
I20260709 13:20:54.361097 21138 raft_consensus.cc:359] T 5e579b46cc154814ad25486ba520710f P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.361789 21138 raft_consensus.cc:385] T 5e579b46cc154814ad25486ba520710f P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:54.362051 21138 raft_consensus.cc:740] T 5e579b46cc154814ad25486ba520710f P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71bdb236d2304c75b7eabfec8f3d5a46, State: Initialized, Role: FOLLOWER
I20260709 13:20:54.362859 21138 consensus_queue.cc:260] T 5e579b46cc154814ad25486ba520710f P 71bdb236d2304c75b7eabfec8f3d5a46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.363452 21136 tablet_bootstrap.cc:654] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:54.363546 21185 leader_election.cc:290] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473), ec8fff5744984f4695d768676cae8d17 (127.9.116.130:42659)
I20260709 13:20:54.365340 21138 ts_tablet_manager.cc:1434] T 5e579b46cc154814ad25486ba520710f P 71bdb236d2304c75b7eabfec8f3d5a46: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:20:54.366305 21138 tablet_bootstrap.cc:492] T 7f3178c915104a27b9a3b6978249c5ed P 71bdb236d2304c75b7eabfec8f3d5a46: Bootstrap starting.
I20260709 13:20:54.365738 20842 catalog_manager.cc:5697] T ee997af6d38745549698356bcbc09a99 P bd6a13ebcb53485a9cc5082a4f64d652 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131). New cstate: current_term: 1 leader_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:54.362874 21135 ts_tablet_manager.cc:1403] T ee997af6d38745549698356bcbc09a99 P ec8fff5744984f4695d768676cae8d17: Time spent bootstrapping tablet: real 0.030s	user 0.010s	sys 0.002s
I20260709 13:20:54.368273 21020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725109df1524e6c8eb897f0e6424829" candidate_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec8fff5744984f4695d768676cae8d17" is_pre_election: true
I20260709 13:20:54.378832 21144 raft_consensus.cc:493] T b541a9ed669a420d8fbeb6e10ab3c47f P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:54.374253 21140 raft_consensus.cc:493] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:54.379456 21144 raft_consensus.cc:515] T b541a9ed669a420d8fbeb6e10ab3c47f P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } }
I20260709 13:20:54.372900 21138 tablet_bootstrap.cc:654] T 7f3178c915104a27b9a3b6978249c5ed P 71bdb236d2304c75b7eabfec8f3d5a46: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:54.380067 21140 raft_consensus.cc:515] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.382381 21140 leader_election.cc:290] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473), bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131:37203)
I20260709 13:20:54.383463 21135 raft_consensus.cc:359] T ee997af6d38745549698356bcbc09a99 P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.384208 21135 raft_consensus.cc:385] T ee997af6d38745549698356bcbc09a99 P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:54.384500 21135 raft_consensus.cc:740] T ee997af6d38745549698356bcbc09a99 P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec8fff5744984f4695d768676cae8d17, State: Initialized, Role: FOLLOWER
I20260709 13:20:54.385193 21021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b541a9ed669a420d8fbeb6e10ab3c47f" candidate_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec8fff5744984f4695d768676cae8d17" is_pre_election: true
I20260709 13:20:54.385419 21135 consensus_queue.cc:260] T ee997af6d38745549698356bcbc09a99 P ec8fff5744984f4695d768676cae8d17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.385936 21021 raft_consensus.cc:2468] T b541a9ed669a420d8fbeb6e10ab3c47f P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71bdb236d2304c75b7eabfec8f3d5a46 in term 0.
I20260709 13:20:54.384202 21020 raft_consensus.cc:2468] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd6a13ebcb53485a9cc5082a4f64d652 in term 0.
I20260709 13:20:54.387847 21135 ts_tablet_manager.cc:1434] T ee997af6d38745549698356bcbc09a99 P ec8fff5744984f4695d768676cae8d17: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:20:54.388285 20946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725109df1524e6c8eb897f0e6424829" candidate_uuid: "ec8fff5744984f4695d768676cae8d17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" is_pre_election: true
I20260709 13:20:54.393743 21094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725109df1524e6c8eb897f0e6424829" candidate_uuid: "ec8fff5744984f4695d768676cae8d17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" is_pre_election: true
I20260709 13:20:54.394415 21094 raft_consensus.cc:2468] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec8fff5744984f4695d768676cae8d17 in term 0.
I20260709 13:20:54.391973 21136 tablet_bootstrap.cc:492] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652: No bootstrap required, opened a new log
I20260709 13:20:54.395310 20912 leader_election.cc:304] T b541a9ed669a420d8fbeb6e10ab3c47f P 71bdb236d2304c75b7eabfec8f3d5a46 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71bdb236d2304c75b7eabfec8f3d5a46, ec8fff5744984f4695d768676cae8d17; no voters: 
I20260709 13:20:54.395843 21136 ts_tablet_manager.cc:1403] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652: Time spent bootstrapping tablet: real 0.038s	user 0.011s	sys 0.001s
I20260709 13:20:54.396539 21163 raft_consensus.cc:2804] T b541a9ed669a420d8fbeb6e10ab3c47f P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:54.396950 21163 raft_consensus.cc:493] T b541a9ed669a420d8fbeb6e10ab3c47f P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:54.396834 21061 leader_election.cc:304] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bd6a13ebcb53485a9cc5082a4f64d652, ec8fff5744984f4695d768676cae8d17; no voters: 71bdb236d2304c75b7eabfec8f3d5a46
I20260709 13:20:54.397409 21163 raft_consensus.cc:3060] T b541a9ed669a420d8fbeb6e10ab3c47f P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:54.392364 21095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b541a9ed669a420d8fbeb6e10ab3c47f" candidate_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" is_pre_election: true
I20260709 13:20:54.398312 21095 raft_consensus.cc:2468] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71bdb236d2304c75b7eabfec8f3d5a46 in term 0.
I20260709 13:20:54.399964 21136 raft_consensus.cc:359] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.400987 21136 raft_consensus.cc:385] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:54.401326 21136 raft_consensus.cc:740] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd6a13ebcb53485a9cc5082a4f64d652, State: Initialized, Role: FOLLOWER
I20260709 13:20:54.402361 21136 consensus_queue.cc:260] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.402978 21185 raft_consensus.cc:2804] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:54.403475 21185 raft_consensus.cc:493] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:54.403513 21144 leader_election.cc:290] T b541a9ed669a420d8fbeb6e10ab3c47f P 71bdb236d2304c75b7eabfec8f3d5a46 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131:37203), ec8fff5744984f4695d768676cae8d17 (127.9.116.130:42659)
I20260709 13:20:54.403929 21185 raft_consensus.cc:3060] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:54.405134 21135 tablet_bootstrap.cc:492] T 35d6d5be27cb4096917b886d06a3cf58 P ec8fff5744984f4695d768676cae8d17: Bootstrap starting.
I20260709 13:20:54.409478 21136 ts_tablet_manager.cc:1434] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
W20260709 13:20:54.395321 20985 leader_election.cc:343] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:54.411260 20985 leader_election.cc:304] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bd6a13ebcb53485a9cc5082a4f64d652, ec8fff5744984f4695d768676cae8d17; no voters: 71bdb236d2304c75b7eabfec8f3d5a46
I20260709 13:20:54.412403 21158 raft_consensus.cc:2804] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:54.412783 21158 raft_consensus.cc:493] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:54.413096 21158 raft_consensus.cc:3060] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:54.415465 21135 tablet_bootstrap.cc:654] T 35d6d5be27cb4096917b886d06a3cf58 P ec8fff5744984f4695d768676cae8d17: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:54.415930 21185 raft_consensus.cc:515] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.419068 20946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725109df1524e6c8eb897f0e6424829" candidate_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71bdb236d2304c75b7eabfec8f3d5a46"
I20260709 13:20:54.420657 21144 raft_consensus.cc:493] T 35d6d5be27cb4096917b886d06a3cf58 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:54.420744 21143 raft_consensus.cc:493] T 5e579b46cc154814ad25486ba520710f P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:54.421244 21144 raft_consensus.cc:515] T 35d6d5be27cb4096917b886d06a3cf58 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.421692 21163 raft_consensus.cc:515] T b541a9ed669a420d8fbeb6e10ab3c47f P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } }
I20260709 13:20:54.421553 21143 raft_consensus.cc:515] T 5e579b46cc154814ad25486ba520710f P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
W20260709 13:20:54.423101 21059 leader_election.cc:343] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:54.423846 21163 leader_election.cc:290] T b541a9ed669a420d8fbeb6e10ab3c47f P 71bdb236d2304c75b7eabfec8f3d5a46 [CANDIDATE]: Term 1 election: Requested vote from peers bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131:37203), ec8fff5744984f4695d768676cae8d17 (127.9.116.130:42659)
I20260709 13:20:54.425209 21094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b541a9ed669a420d8fbeb6e10ab3c47f" candidate_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652"
I20260709 13:20:54.425910 21138 tablet_bootstrap.cc:492] T 7f3178c915104a27b9a3b6978249c5ed P 71bdb236d2304c75b7eabfec8f3d5a46: No bootstrap required, opened a new log
I20260709 13:20:54.426473 21138 ts_tablet_manager.cc:1403] T 7f3178c915104a27b9a3b6978249c5ed P 71bdb236d2304c75b7eabfec8f3d5a46: Time spent bootstrapping tablet: real 0.060s	user 0.015s	sys 0.022s
I20260709 13:20:54.426956 20946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e579b46cc154814ad25486ba520710f" candidate_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" is_pre_election: true
I20260709 13:20:54.427660 20946 raft_consensus.cc:2468] T 5e579b46cc154814ad25486ba520710f P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd6a13ebcb53485a9cc5082a4f64d652 in term 0.
I20260709 13:20:54.427490 21021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b541a9ed669a420d8fbeb6e10ab3c47f" candidate_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec8fff5744984f4695d768676cae8d17"
I20260709 13:20:54.428723 21021 raft_consensus.cc:3060] T b541a9ed669a420d8fbeb6e10ab3c47f P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:54.429342 21138 raft_consensus.cc:359] T 7f3178c915104a27b9a3b6978249c5ed P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.430015 21138 raft_consensus.cc:385] T 7f3178c915104a27b9a3b6978249c5ed P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:54.430287 21138 raft_consensus.cc:740] T 7f3178c915104a27b9a3b6978249c5ed P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71bdb236d2304c75b7eabfec8f3d5a46, State: Initialized, Role: FOLLOWER
I20260709 13:20:54.425860 21094 raft_consensus.cc:3060] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:54.430958 21138 consensus_queue.cc:260] T 7f3178c915104a27b9a3b6978249c5ed P 71bdb236d2304c75b7eabfec8f3d5a46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.410418 21140 raft_consensus.cc:493] T 5e579b46cc154814ad25486ba520710f P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:54.435223 21140 raft_consensus.cc:515] T 5e579b46cc154814ad25486ba520710f P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.435461 21019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35d6d5be27cb4096917b886d06a3cf58" candidate_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec8fff5744984f4695d768676cae8d17" is_pre_election: true
I20260709 13:20:54.433476 21144 leader_election.cc:290] T 35d6d5be27cb4096917b886d06a3cf58 P 71bdb236d2304c75b7eabfec8f3d5a46 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec8fff5744984f4695d768676cae8d17 (127.9.116.130:42659), bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131:37203)
I20260709 13:20:54.433354 21185 leader_election.cc:290] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 1 election: Requested vote from peers 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473), ec8fff5744984f4695d768676cae8d17 (127.9.116.130:42659)
I20260709 13:20:54.433868 21138 ts_tablet_manager.cc:1434] T 7f3178c915104a27b9a3b6978249c5ed P 71bdb236d2304c75b7eabfec8f3d5a46: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:20:54.440524 21059 leader_election.cc:304] T 5e579b46cc154814ad25486ba520710f P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71bdb236d2304c75b7eabfec8f3d5a46, bd6a13ebcb53485a9cc5082a4f64d652; no voters: 
I20260709 13:20:54.441941 21019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725109df1524e6c8eb897f0e6424829" candidate_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec8fff5744984f4695d768676cae8d17"
I20260709 13:20:54.432253 21143 leader_election.cc:290] T 5e579b46cc154814ad25486ba520710f P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473), ec8fff5744984f4695d768676cae8d17 (127.9.116.130:42659)
I20260709 13:20:54.444579 21143 raft_consensus.cc:2804] T 5e579b46cc154814ad25486ba520710f P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:54.445644 21143 raft_consensus.cc:493] T 5e579b46cc154814ad25486ba520710f P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:54.446110 21143 raft_consensus.cc:3060] T 5e579b46cc154814ad25486ba520710f P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:54.451020 21138 tablet_bootstrap.cc:492] T 2725109df1524e6c8eb897f0e6424829 P 71bdb236d2304c75b7eabfec8f3d5a46: Bootstrap starting.
I20260709 13:20:54.437906 21158 raft_consensus.cc:515] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.453047 21095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35d6d5be27cb4096917b886d06a3cf58" candidate_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" is_pre_election: true
I20260709 13:20:54.453337 21019 raft_consensus.cc:2393] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate bd6a13ebcb53485a9cc5082a4f64d652 in current term 1: Already voted for candidate ec8fff5744984f4695d768676cae8d17 in this term.
I20260709 13:20:54.453779 21095 raft_consensus.cc:2468] T 35d6d5be27cb4096917b886d06a3cf58 P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71bdb236d2304c75b7eabfec8f3d5a46 in term 0.
I20260709 13:20:54.454435 21158 leader_election.cc:290] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 1 election: Requested vote from peers 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473), bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131:37203)
W20260709 13:20:54.450870 20912 leader_election.cc:343] T 35d6d5be27cb4096917b886d06a3cf58 P 71bdb236d2304c75b7eabfec8f3d5a46 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ec8fff5744984f4695d768676cae8d17 (127.9.116.130:42659): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:54.455473 21061 leader_election.cc:304] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd6a13ebcb53485a9cc5082a4f64d652; no voters: 71bdb236d2304c75b7eabfec8f3d5a46, ec8fff5744984f4695d768676cae8d17
I20260709 13:20:54.455641 21140 leader_election.cc:290] T 5e579b46cc154814ad25486ba520710f P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473), bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131:37203)
I20260709 13:20:54.456326 21185 raft_consensus.cc:2749] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:20:54.456969 20910 leader_election.cc:304] T 35d6d5be27cb4096917b886d06a3cf58 P 71bdb236d2304c75b7eabfec8f3d5a46 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 71bdb236d2304c75b7eabfec8f3d5a46, bd6a13ebcb53485a9cc5082a4f64d652; no voters: ec8fff5744984f4695d768676cae8d17
I20260709 13:20:54.457952 21094 raft_consensus.cc:2468] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71bdb236d2304c75b7eabfec8f3d5a46 in term 1.
I20260709 13:20:54.460507 21095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e579b46cc154814ad25486ba520710f" candidate_uuid: "ec8fff5744984f4695d768676cae8d17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" is_pre_election: true
I20260709 13:20:54.461438 21144 raft_consensus.cc:2804] T 35d6d5be27cb4096917b886d06a3cf58 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:54.461681 21020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e579b46cc154814ad25486ba520710f" candidate_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec8fff5744984f4695d768676cae8d17" is_pre_election: true
I20260709 13:20:54.461934 21144 raft_consensus.cc:493] T 35d6d5be27cb4096917b886d06a3cf58 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:54.462345 21144 raft_consensus.cc:3060] T 35d6d5be27cb4096917b886d06a3cf58 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:54.462662 20910 leader_election.cc:304] T b541a9ed669a420d8fbeb6e10ab3c47f P 71bdb236d2304c75b7eabfec8f3d5a46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71bdb236d2304c75b7eabfec8f3d5a46, bd6a13ebcb53485a9cc5082a4f64d652; no voters: 
I20260709 13:20:54.463070 21143 raft_consensus.cc:515] T 5e579b46cc154814ad25486ba520710f P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.463670 21138 tablet_bootstrap.cc:654] T 2725109df1524e6c8eb897f0e6424829 P 71bdb236d2304c75b7eabfec8f3d5a46: Neither blocks nor log segments found. Creating new log.
I20260709 13:20:54.464398 21095 raft_consensus.cc:2393] T 5e579b46cc154814ad25486ba520710f P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec8fff5744984f4695d768676cae8d17 in current term 1: Already voted for candidate bd6a13ebcb53485a9cc5082a4f64d652 in this term.
I20260709 13:20:54.465059 20946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725109df1524e6c8eb897f0e6424829" candidate_uuid: "ec8fff5744984f4695d768676cae8d17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71bdb236d2304c75b7eabfec8f3d5a46"
I20260709 13:20:54.463630 21163 raft_consensus.cc:2804] T b541a9ed669a420d8fbeb6e10ab3c47f P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:54.466434 21163 raft_consensus.cc:697] T b541a9ed669a420d8fbeb6e10ab3c47f P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 LEADER]: Becoming Leader. State: Replica: 71bdb236d2304c75b7eabfec8f3d5a46, State: Running, Role: LEADER
I20260709 13:20:54.466710 21185 raft_consensus.cc:493] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:54.467332 21185 raft_consensus.cc:515] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.467377 21163 consensus_queue.cc:237] T b541a9ed669a420d8fbeb6e10ab3c47f P 71bdb236d2304c75b7eabfec8f3d5a46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } }
I20260709 13:20:54.468765 21020 raft_consensus.cc:2468] T 5e579b46cc154814ad25486ba520710f P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd6a13ebcb53485a9cc5082a4f64d652 in term 0.
I20260709 13:20:54.469069 21135 tablet_bootstrap.cc:492] T 35d6d5be27cb4096917b886d06a3cf58 P ec8fff5744984f4695d768676cae8d17: No bootstrap required, opened a new log
I20260709 13:20:54.469619 21135 ts_tablet_manager.cc:1403] T 35d6d5be27cb4096917b886d06a3cf58 P ec8fff5744984f4695d768676cae8d17: Time spent bootstrapping tablet: real 0.065s	user 0.019s	sys 0.023s
I20260709 13:20:54.471113 21185 leader_election.cc:290] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473), ec8fff5744984f4695d768676cae8d17 (127.9.116.130:42659)
I20260709 13:20:54.471388 21144 raft_consensus.cc:515] T 35d6d5be27cb4096917b886d06a3cf58 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.472585 21135 raft_consensus.cc:359] T 35d6d5be27cb4096917b886d06a3cf58 P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.466770 21021 raft_consensus.cc:2468] T b541a9ed669a420d8fbeb6e10ab3c47f P ec8fff5744984f4695d768676cae8d17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71bdb236d2304c75b7eabfec8f3d5a46 in term 1.
I20260709 13:20:54.473110 21020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50ac60e07eb46dc8836de50bce12760" candidate_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec8fff5744984f4695d768676cae8d17" is_pre_election: true
I20260709 13:20:54.473865 21020 raft_consensus.cc:2468] T c50ac60e07eb46dc8836de50bce12760 P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd6a13ebcb53485a9cc5082a4f64d652 in term 0.
I20260709 13:20:54.473348 21135 raft_consensus.cc:385] T 35d6d5be27cb4096917b886d06a3cf58 P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:54.474737 21135 raft_consensus.cc:740] T 35d6d5be27cb4096917b886d06a3cf58 P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec8fff5744984f4695d768676cae8d17, State: Initialized, Role: FOLLOWER
I20260709 13:20:54.475395 21061 leader_election.cc:304] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd6a13ebcb53485a9cc5082a4f64d652, ec8fff5744984f4695d768676cae8d17; no voters: 
I20260709 13:20:54.475522 21135 consensus_queue.cc:260] T 35d6d5be27cb4096917b886d06a3cf58 P ec8fff5744984f4695d768676cae8d17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.476469 21185 raft_consensus.cc:2804] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:54.476876 21185 raft_consensus.cc:493] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:54.477212 21185 raft_consensus.cc:3060] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:54.477753 21135 ts_tablet_manager.cc:1434] T 35d6d5be27cb4096917b886d06a3cf58 P ec8fff5744984f4695d768676cae8d17: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:20:54.479276 21144 leader_election.cc:290] T 35d6d5be27cb4096917b886d06a3cf58 P 71bdb236d2304c75b7eabfec8f3d5a46 [CANDIDATE]: Term 1 election: Requested vote from peers ec8fff5744984f4695d768676cae8d17 (127.9.116.130:42659), bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131:37203)
I20260709 13:20:54.475049 21167 raft_consensus.cc:493] T c50ac60e07eb46dc8836de50bce12760 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:54.480394 21020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35d6d5be27cb4096917b886d06a3cf58" candidate_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec8fff5744984f4695d768676cae8d17"
I20260709 13:20:54.481078 21020 raft_consensus.cc:3060] T 35d6d5be27cb4096917b886d06a3cf58 P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:54.480572 21167 raft_consensus.cc:515] T c50ac60e07eb46dc8836de50bce12760 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.484550 21167 leader_election.cc:290] T c50ac60e07eb46dc8836de50bce12760 P 71bdb236d2304c75b7eabfec8f3d5a46 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec8fff5744984f4695d768676cae8d17 (127.9.116.130:42659), bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131:37203)
I20260709 13:20:54.487931 21143 leader_election.cc:290] T 5e579b46cc154814ad25486ba520710f P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 1 election: Requested vote from peers 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473), ec8fff5744984f4695d768676cae8d17 (127.9.116.130:42659)
I20260709 13:20:54.489228 21019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e579b46cc154814ad25486ba520710f" candidate_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec8fff5744984f4695d768676cae8d17"
I20260709 13:20:54.490304 21019 raft_consensus.cc:3060] T 5e579b46cc154814ad25486ba520710f P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:54.491115 21185 raft_consensus.cc:515] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.494024 21020 raft_consensus.cc:2468] T 35d6d5be27cb4096917b886d06a3cf58 P ec8fff5744984f4695d768676cae8d17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71bdb236d2304c75b7eabfec8f3d5a46 in term 1.
I20260709 13:20:54.495525 21018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50ac60e07eb46dc8836de50bce12760" candidate_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec8fff5744984f4695d768676cae8d17"
I20260709 13:20:54.496271 21018 raft_consensus.cc:3060] T c50ac60e07eb46dc8836de50bce12760 P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:54.501909 21140 raft_consensus.cc:493] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:54.502079 20912 leader_election.cc:304] T 35d6d5be27cb4096917b886d06a3cf58 P 71bdb236d2304c75b7eabfec8f3d5a46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71bdb236d2304c75b7eabfec8f3d5a46, ec8fff5744984f4695d768676cae8d17; no voters: 
I20260709 13:20:54.502503 21140 raft_consensus.cc:515] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
W20260709 13:20:54.503861 20985 leader_election.cc:343] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:20:54.504676 21140 leader_election.cc:290] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473), bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131:37203)
I20260709 13:20:54.506198 21095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f3178c915104a27b9a3b6978249c5ed" candidate_uuid: "ec8fff5744984f4695d768676cae8d17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" is_pre_election: true
I20260709 13:20:54.506587 21018 raft_consensus.cc:2468] T c50ac60e07eb46dc8836de50bce12760 P ec8fff5744984f4695d768676cae8d17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd6a13ebcb53485a9cc5082a4f64d652 in term 1.
I20260709 13:20:54.506992 21095 raft_consensus.cc:2468] T 7f3178c915104a27b9a3b6978249c5ed P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec8fff5744984f4695d768676cae8d17 in term 0.
I20260709 13:20:54.495311 21185 leader_election.cc:290] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 1 election: Requested vote from peers 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473), ec8fff5744984f4695d768676cae8d17 (127.9.116.130:42659)
I20260709 13:20:54.508129 20946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e579b46cc154814ad25486ba520710f" candidate_uuid: "ec8fff5744984f4695d768676cae8d17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" is_pre_election: true
I20260709 13:20:54.508563 21095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35d6d5be27cb4096917b886d06a3cf58" candidate_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652"
I20260709 13:20:54.508961 20945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50ac60e07eb46dc8836de50bce12760" candidate_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" is_pre_election: true
I20260709 13:20:54.509212 21095 raft_consensus.cc:3060] T 35d6d5be27cb4096917b886d06a3cf58 P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:54.509071 20946 raft_consensus.cc:2468] T 5e579b46cc154814ad25486ba520710f P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec8fff5744984f4695d768676cae8d17 in term 0.
I20260709 13:20:54.509943 21093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50ac60e07eb46dc8836de50bce12760" candidate_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" is_pre_election: true
I20260709 13:20:54.510821 21093 raft_consensus.cc:2393] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 71bdb236d2304c75b7eabfec8f3d5a46 in current term 1: Already voted for candidate bd6a13ebcb53485a9cc5082a4f64d652 in this term.
I20260709 13:20:54.511005 21021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50ac60e07eb46dc8836de50bce12760" candidate_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec8fff5744984f4695d768676cae8d17" is_pre_election: true
I20260709 13:20:54.512101 21021 raft_consensus.cc:2393] T c50ac60e07eb46dc8836de50bce12760 P ec8fff5744984f4695d768676cae8d17 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 71bdb236d2304c75b7eabfec8f3d5a46 in current term 1: Already voted for candidate bd6a13ebcb53485a9cc5082a4f64d652 in this term.
I20260709 13:20:54.512550 20985 leader_election.cc:304] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd6a13ebcb53485a9cc5082a4f64d652, ec8fff5744984f4695d768676cae8d17; no voters: 
I20260709 13:20:54.513077 21167 raft_consensus.cc:2804] T 35d6d5be27cb4096917b886d06a3cf58 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:54.513520 21140 raft_consensus.cc:2804] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:20:54.513964 21140 raft_consensus.cc:493] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:54.514055 21167 raft_consensus.cc:697] T 35d6d5be27cb4096917b886d06a3cf58 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 LEADER]: Becoming Leader. State: Replica: 71bdb236d2304c75b7eabfec8f3d5a46, State: Running, Role: LEADER
I20260709 13:20:54.514873 20945 raft_consensus.cc:2468] T c50ac60e07eb46dc8836de50bce12760 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd6a13ebcb53485a9cc5082a4f64d652 in term 0.
I20260709 13:20:54.514336 21140 raft_consensus.cc:3060] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:54.515674 21167 consensus_queue.cc:237] T 35d6d5be27cb4096917b886d06a3cf58 P 71bdb236d2304c75b7eabfec8f3d5a46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.517894 21095 raft_consensus.cc:2468] T 35d6d5be27cb4096917b886d06a3cf58 P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71bdb236d2304c75b7eabfec8f3d5a46 in term 1.
I20260709 13:20:54.518832 20944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e579b46cc154814ad25486ba520710f" candidate_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71bdb236d2304c75b7eabfec8f3d5a46"
I20260709 13:20:54.519524 21061 leader_election.cc:304] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd6a13ebcb53485a9cc5082a4f64d652, ec8fff5744984f4695d768676cae8d17; no voters: 
I20260709 13:20:54.519809 20944 raft_consensus.cc:3060] T 5e579b46cc154814ad25486ba520710f P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:54.521202 21185 raft_consensus.cc:2804] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:54.521735 21185 raft_consensus.cc:697] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 LEADER]: Becoming Leader. State: Replica: bd6a13ebcb53485a9cc5082a4f64d652, State: Running, Role: LEADER
I20260709 13:20:54.525247 20842 catalog_manager.cc:5697] T b541a9ed669a420d8fbeb6e10ab3c47f P 71bdb236d2304c75b7eabfec8f3d5a46 reported cstate change: term changed from 0 to 1, leader changed from <none> to 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129). New cstate: current_term: 1 leader_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:54.528124 21140 raft_consensus.cc:515] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.531196 20910 leader_election.cc:304] T c50ac60e07eb46dc8836de50bce12760 P 71bdb236d2304c75b7eabfec8f3d5a46 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71bdb236d2304c75b7eabfec8f3d5a46; no voters: bd6a13ebcb53485a9cc5082a4f64d652, ec8fff5744984f4695d768676cae8d17
I20260709 13:20:54.535526 20985 leader_election.cc:304] T 5e579b46cc154814ad25486ba520710f P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 71bdb236d2304c75b7eabfec8f3d5a46, ec8fff5744984f4695d768676cae8d17; no voters: bd6a13ebcb53485a9cc5082a4f64d652
I20260709 13:20:54.536427 21163 raft_consensus.cc:3060] T c50ac60e07eb46dc8836de50bce12760 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:54.522609 21185 consensus_queue.cc:237] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.532873 20944 raft_consensus.cc:2468] T 5e579b46cc154814ad25486ba520710f P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd6a13ebcb53485a9cc5082a4f64d652 in term 1.
I20260709 13:20:54.534720 20945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50ac60e07eb46dc8836de50bce12760" candidate_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71bdb236d2304c75b7eabfec8f3d5a46"
I20260709 13:20:54.543324 21095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f3178c915104a27b9a3b6978249c5ed" candidate_uuid: "ec8fff5744984f4695d768676cae8d17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652"
I20260709 13:20:54.544137 21095 raft_consensus.cc:3060] T 7f3178c915104a27b9a3b6978249c5ed P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:54.544725 21094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725109df1524e6c8eb897f0e6424829" candidate_uuid: "ec8fff5744984f4695d768676cae8d17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652"
I20260709 13:20:54.545657 21094 raft_consensus.cc:2393] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ec8fff5744984f4695d768676cae8d17 in current term 1: Already voted for candidate bd6a13ebcb53485a9cc5082a4f64d652 in this term.
I20260709 13:20:54.547036 20985 leader_election.cc:304] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec8fff5744984f4695d768676cae8d17; no voters: 71bdb236d2304c75b7eabfec8f3d5a46, bd6a13ebcb53485a9cc5082a4f64d652
I20260709 13:20:54.549582 20946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f3178c915104a27b9a3b6978249c5ed" candidate_uuid: "ec8fff5744984f4695d768676cae8d17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71bdb236d2304c75b7eabfec8f3d5a46"
I20260709 13:20:54.550736 20946 raft_consensus.cc:3060] T 7f3178c915104a27b9a3b6978249c5ed P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:54.551523 21140 leader_election.cc:290] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 1 election: Requested vote from peers 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473), bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131:37203)
I20260709 13:20:54.552446 21140 raft_consensus.cc:2749] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:20:54.563385 20944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f3178c915104a27b9a3b6978249c5ed" candidate_uuid: "ec8fff5744984f4695d768676cae8d17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" is_pre_election: true
I20260709 13:20:54.572206 21059 leader_election.cc:304] T 5e579b46cc154814ad25486ba520710f P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71bdb236d2304c75b7eabfec8f3d5a46, bd6a13ebcb53485a9cc5082a4f64d652; no voters: 
I20260709 13:20:54.571460 20842 catalog_manager.cc:5697] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131). New cstate: current_term: 1 leader_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:54.573503 21143 raft_consensus.cc:2804] T 5e579b46cc154814ad25486ba520710f P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:54.574204 21143 raft_consensus.cc:697] T 5e579b46cc154814ad25486ba520710f P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 LEADER]: Becoming Leader. State: Replica: bd6a13ebcb53485a9cc5082a4f64d652, State: Running, Role: LEADER
I20260709 13:20:54.575816 21019 raft_consensus.cc:2468] T 5e579b46cc154814ad25486ba520710f P ec8fff5744984f4695d768676cae8d17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd6a13ebcb53485a9cc5082a4f64d652 in term 1.
I20260709 13:20:54.575408 21143 consensus_queue.cc:237] T 5e579b46cc154814ad25486ba520710f P bd6a13ebcb53485a9cc5082a4f64d652 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.576522 21158 raft_consensus.cc:2764] T 5e579b46cc154814ad25486ba520710f P ec8fff5744984f4695d768676cae8d17 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:20:54.562477 20946 raft_consensus.cc:2468] T 7f3178c915104a27b9a3b6978249c5ed P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec8fff5744984f4695d768676cae8d17 in term 1.
I20260709 13:20:54.576231 21163 raft_consensus.cc:2749] T c50ac60e07eb46dc8836de50bce12760 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:54.583981 21138 tablet_bootstrap.cc:492] T 2725109df1524e6c8eb897f0e6424829 P 71bdb236d2304c75b7eabfec8f3d5a46: No bootstrap required, opened a new log
I20260709 13:20:54.584573 21138 ts_tablet_manager.cc:1403] T 2725109df1524e6c8eb897f0e6424829 P 71bdb236d2304c75b7eabfec8f3d5a46: Time spent bootstrapping tablet: real 0.145s	user 0.061s	sys 0.042s
I20260709 13:20:54.587234 21138 raft_consensus.cc:359] T 2725109df1524e6c8eb897f0e6424829 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.587903 21138 raft_consensus.cc:385] T 2725109df1524e6c8eb897f0e6424829 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:20:54.588173 21138 raft_consensus.cc:740] T 2725109df1524e6c8eb897f0e6424829 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71bdb236d2304c75b7eabfec8f3d5a46, State: Initialized, Role: FOLLOWER
I20260709 13:20:54.588868 21138 consensus_queue.cc:260] T 2725109df1524e6c8eb897f0e6424829 P 71bdb236d2304c75b7eabfec8f3d5a46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.590663 21095 raft_consensus.cc:2468] T 7f3178c915104a27b9a3b6978249c5ed P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec8fff5744984f4695d768676cae8d17 in term 1.
I20260709 13:20:54.591671 21138 ts_tablet_manager.cc:1434] T 2725109df1524e6c8eb897f0e6424829 P 71bdb236d2304c75b7eabfec8f3d5a46: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:20:54.590958 20844 catalog_manager.cc:5697] T 35d6d5be27cb4096917b886d06a3cf58 P 71bdb236d2304c75b7eabfec8f3d5a46 reported cstate change: term changed from 0 to 1, leader changed from <none> to 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129). New cstate: current_term: 1 leader_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } health_report { overall_health: UNKNOWN } } }
W20260709 13:20:54.594475 20985 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:37203, user_credentials={real_user=slave}} blocked reactor thread for 47553.2us
I20260709 13:20:54.591858 20945 raft_consensus.cc:2468] T c50ac60e07eb46dc8836de50bce12760 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd6a13ebcb53485a9cc5082a4f64d652 in term 1.
I20260709 13:20:54.597800 20985 leader_election.cc:304] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71bdb236d2304c75b7eabfec8f3d5a46, ec8fff5744984f4695d768676cae8d17; no voters: 
I20260709 13:20:54.599344 21193 raft_consensus.cc:2804] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:20:54.599862 21193 raft_consensus.cc:697] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17 [term 1 LEADER]: Becoming Leader. State: Replica: ec8fff5744984f4695d768676cae8d17, State: Running, Role: LEADER
I20260709 13:20:54.600759 21193 consensus_queue.cc:237] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.602540 21158 raft_consensus.cc:493] T ee997af6d38745549698356bcbc09a99 P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:54.603160 21158 raft_consensus.cc:515] T ee997af6d38745549698356bcbc09a99 P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.605657 21158 leader_election.cc:290] T ee997af6d38745549698356bcbc09a99 P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473), bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131:37203)
I20260709 13:20:54.608652 20945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee997af6d38745549698356bcbc09a99" candidate_uuid: "ec8fff5744984f4695d768676cae8d17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" is_pre_election: true
I20260709 13:20:54.609566 20945 raft_consensus.cc:2393] T ee997af6d38745549698356bcbc09a99 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec8fff5744984f4695d768676cae8d17 in current term 1: Already voted for candidate bd6a13ebcb53485a9cc5082a4f64d652 in this term.
I20260709 13:20:54.611016 21095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee997af6d38745549698356bcbc09a99" candidate_uuid: "ec8fff5744984f4695d768676cae8d17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" is_pre_election: true
I20260709 13:20:54.614252 20985 leader_election.cc:304] T ee997af6d38745549698356bcbc09a99 P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec8fff5744984f4695d768676cae8d17; no voters: 71bdb236d2304c75b7eabfec8f3d5a46, bd6a13ebcb53485a9cc5082a4f64d652
I20260709 13:20:54.614353 20843 catalog_manager.cc:5697] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17 reported cstate change: term changed from 0 to 1, leader changed from <none> to ec8fff5744984f4695d768676cae8d17 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "ec8fff5744984f4695d768676cae8d17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:54.615267 21193 raft_consensus.cc:3060] T ee997af6d38745549698356bcbc09a99 P ec8fff5744984f4695d768676cae8d17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:54.615881 21158 consensus_queue.cc:1048] T 0f79d8e2c55b4a1f8380919ef723efe7 P ec8fff5744984f4695d768676cae8d17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:54.615944 20842 catalog_manager.cc:5697] T 5e579b46cc154814ad25486ba520710f P bd6a13ebcb53485a9cc5082a4f64d652 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131). New cstate: current_term: 1 leader_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:54.622856 21143 raft_consensus.cc:493] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:54.623422 21143 raft_consensus.cc:515] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.624480 21193 raft_consensus.cc:2749] T ee997af6d38745549698356bcbc09a99 P ec8fff5744984f4695d768676cae8d17 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:54.625712 21143 leader_election.cc:290] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473), ec8fff5744984f4695d768676cae8d17 (127.9.116.130:42659)
I20260709 13:20:54.627929 21019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35602162000346e3bbb790f54467ed0d" candidate_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec8fff5744984f4695d768676cae8d17" is_pre_election: true
I20260709 13:20:54.630573 20945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35602162000346e3bbb790f54467ed0d" candidate_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" is_pre_election: true
I20260709 13:20:54.631453 20945 raft_consensus.cc:2393] T 35602162000346e3bbb790f54467ed0d P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bd6a13ebcb53485a9cc5082a4f64d652 in current term 1: Already voted for candidate ec8fff5744984f4695d768676cae8d17 in this term.
I20260709 13:20:54.632596 21059 leader_election.cc:304] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd6a13ebcb53485a9cc5082a4f64d652; no voters: 71bdb236d2304c75b7eabfec8f3d5a46, ec8fff5744984f4695d768676cae8d17
I20260709 13:20:54.633795 21143 raft_consensus.cc:3060] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:54.641484 21143 raft_consensus.cc:2749] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:54.653018 21158 consensus_queue.cc:1048] T 0f79d8e2c55b4a1f8380919ef723efe7 P ec8fff5744984f4695d768676cae8d17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:20:54.657230 21163 raft_consensus.cc:493] T 2725109df1524e6c8eb897f0e6424829 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:54.657976 21163 raft_consensus.cc:515] T 2725109df1524e6c8eb897f0e6424829 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:54.661124 21019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725109df1524e6c8eb897f0e6424829" candidate_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec8fff5744984f4695d768676cae8d17" is_pre_election: true
I20260709 13:20:54.661403 21095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725109df1524e6c8eb897f0e6424829" candidate_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" is_pre_election: true
I20260709 13:20:54.661948 21019 raft_consensus.cc:2393] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 71bdb236d2304c75b7eabfec8f3d5a46 in current term 1: Already voted for candidate ec8fff5744984f4695d768676cae8d17 in this term.
I20260709 13:20:54.662190 21163 leader_election.cc:290] T 2725109df1524e6c8eb897f0e6424829 P 71bdb236d2304c75b7eabfec8f3d5a46 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec8fff5744984f4695d768676cae8d17 (127.9.116.130:42659), bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131:37203)
I20260709 13:20:54.662478 21095 raft_consensus.cc:2393] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 71bdb236d2304c75b7eabfec8f3d5a46 in current term 1: Already voted for candidate bd6a13ebcb53485a9cc5082a4f64d652 in this term.
I20260709 13:20:54.663478 20910 leader_election.cc:304] T 2725109df1524e6c8eb897f0e6424829 P 71bdb236d2304c75b7eabfec8f3d5a46 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71bdb236d2304c75b7eabfec8f3d5a46; no voters: bd6a13ebcb53485a9cc5082a4f64d652, ec8fff5744984f4695d768676cae8d17
I20260709 13:20:54.664181 21163 raft_consensus.cc:3060] T 2725109df1524e6c8eb897f0e6424829 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:20:54.669648 21163 raft_consensus.cc:2749] T 2725109df1524e6c8eb897f0e6424829 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:20:54.717754 21163 consensus_queue.cc:1048] T c3add26e31364c95a9897ab53b516735 P 71bdb236d2304c75b7eabfec8f3d5a46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:54.728426 21144 consensus_queue.cc:1048] T c3add26e31364c95a9897ab53b516735 P 71bdb236d2304c75b7eabfec8f3d5a46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:54.733345 21158 consensus_queue.cc:1048] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:54.747548 21158 consensus_queue.cc:1048] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:54.776752 21143 consensus_queue.cc:1048] T ee997af6d38745549698356bcbc09a99 P bd6a13ebcb53485a9cc5082a4f64d652 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:54.787671 21185 consensus_queue.cc:1048] T ee997af6d38745549698356bcbc09a99 P bd6a13ebcb53485a9cc5082a4f64d652 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:54.993352 21144 consensus_queue.cc:1048] T 35d6d5be27cb4096917b886d06a3cf58 P 71bdb236d2304c75b7eabfec8f3d5a46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:55.004853 21167 consensus_queue.cc:1048] T 35d6d5be27cb4096917b886d06a3cf58 P 71bdb236d2304c75b7eabfec8f3d5a46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:55.012053 21167 consensus_queue.cc:1048] T b541a9ed669a420d8fbeb6e10ab3c47f P 71bdb236d2304c75b7eabfec8f3d5a46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:55.014526 21200 consensus_queue.cc:1048] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:55.019919 21185 consensus_queue.cc:1048] T 5e579b46cc154814ad25486ba520710f P bd6a13ebcb53485a9cc5082a4f64d652 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:20:55.035763 21144 consensus_queue.cc:1048] T b541a9ed669a420d8fbeb6e10ab3c47f P 71bdb236d2304c75b7eabfec8f3d5a46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:20:55.042583 21193 consensus_queue.cc:1048] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:20:55.047793 21185 consensus_queue.cc:1048] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:55.056250 21182 consensus_queue.cc:1048] T 5e579b46cc154814ad25486ba520710f P bd6a13ebcb53485a9cc5082a4f64d652 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:55.073295 21181 consensus_queue.cc:1048] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:20:55.108955 20892 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:20:56.112556 20892 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:20:56.386298 21143 raft_consensus.cc:493] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:56.386940 21143 raft_consensus.cc:515] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:56.388744 21143 leader_election.cc:290] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473), ec8fff5744984f4695d768676cae8d17 (127.9.116.130:42659)
I20260709 13:20:56.389739 20945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725109df1524e6c8eb897f0e6424829" candidate_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" is_pre_election: true
I20260709 13:20:56.389743 21021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725109df1524e6c8eb897f0e6424829" candidate_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec8fff5744984f4695d768676cae8d17" is_pre_election: true
I20260709 13:20:56.390717 20945 raft_consensus.cc:2468] T 2725109df1524e6c8eb897f0e6424829 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd6a13ebcb53485a9cc5082a4f64d652 in term 1.
I20260709 13:20:56.390875 21021 raft_consensus.cc:2468] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd6a13ebcb53485a9cc5082a4f64d652 in term 1.
I20260709 13:20:56.391954 21059 leader_election.cc:304] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71bdb236d2304c75b7eabfec8f3d5a46, bd6a13ebcb53485a9cc5082a4f64d652; no voters: 
I20260709 13:20:56.392741 21143 raft_consensus.cc:2804] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:20:56.393013 21143 raft_consensus.cc:493] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:56.393342 21143 raft_consensus.cc:3060] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:56.399176 21143 raft_consensus.cc:515] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:56.401021 21143 leader_election.cc:290] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 2 election: Requested vote from peers 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473), ec8fff5744984f4695d768676cae8d17 (127.9.116.130:42659)
I20260709 13:20:56.401935 20945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725109df1524e6c8eb897f0e6424829" candidate_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71bdb236d2304c75b7eabfec8f3d5a46"
I20260709 13:20:56.402021 21021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725109df1524e6c8eb897f0e6424829" candidate_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec8fff5744984f4695d768676cae8d17"
I20260709 13:20:56.402890 20945 raft_consensus.cc:3060] T 2725109df1524e6c8eb897f0e6424829 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:56.403219 21021 raft_consensus.cc:3060] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:56.408375 20945 raft_consensus.cc:2468] T 2725109df1524e6c8eb897f0e6424829 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd6a13ebcb53485a9cc5082a4f64d652 in term 2.
I20260709 13:20:56.408391 21021 raft_consensus.cc:2468] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd6a13ebcb53485a9cc5082a4f64d652 in term 2.
I20260709 13:20:56.409646 21059 leader_election.cc:304] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71bdb236d2304c75b7eabfec8f3d5a46, bd6a13ebcb53485a9cc5082a4f64d652; no voters: 
I20260709 13:20:56.410435 21143 raft_consensus.cc:2804] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:20:56.411455 21143 raft_consensus.cc:697] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 LEADER]: Becoming Leader. State: Replica: bd6a13ebcb53485a9cc5082a4f64d652, State: Running, Role: LEADER
I20260709 13:20:56.412153 21143 consensus_queue.cc:237] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:56.420301 20844 catalog_manager.cc:5697] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 reported cstate change: term changed from 0 to 2, leader changed from <none> to bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131). New cstate: current_term: 2 leader_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:56.873416 20945 raft_consensus.cc:1275] T 2725109df1524e6c8eb897f0e6424829 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 2 FOLLOWER]: Refusing update from remote peer bd6a13ebcb53485a9cc5082a4f64d652: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:20:56.874616 21143 consensus_queue.cc:1048] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:56.885233 21021 raft_consensus.cc:1275] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [term 2 FOLLOWER]: Refusing update from remote peer bd6a13ebcb53485a9cc5082a4f64d652: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:20:56.886950 21143 consensus_queue.cc:1048] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:20:56.974135  9682 ts_manager.cc:295] Set tserver state for 71bdb236d2304c75b7eabfec8f3d5a46 to MAINTENANCE_MODE
I20260709 13:20:56.974833  9682 tablet_server.cc:179] TabletServer@127.9.116.129:0 shutting down...
I20260709 13:20:56.996196  9682 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:20:56.996893  9682 tablet_replica.cc:333] T 7f3178c915104a27b9a3b6978249c5ed P 71bdb236d2304c75b7eabfec8f3d5a46: stopping tablet replica
I20260709 13:20:56.997852  9682 raft_consensus.cc:2243] T 7f3178c915104a27b9a3b6978249c5ed P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:56.998387  9682 raft_consensus.cc:2272] T 7f3178c915104a27b9a3b6978249c5ed P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:57.000458  9682 tablet_replica.cc:333] T 2725109df1524e6c8eb897f0e6424829 P 71bdb236d2304c75b7eabfec8f3d5a46: stopping tablet replica
I20260709 13:20:57.001094  9682 raft_consensus.cc:2243] T 2725109df1524e6c8eb897f0e6424829 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:57.001557  9682 raft_consensus.cc:2272] T 2725109df1524e6c8eb897f0e6424829 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:57.004313  9682 tablet_replica.cc:333] T c50ac60e07eb46dc8836de50bce12760 P 71bdb236d2304c75b7eabfec8f3d5a46: stopping tablet replica
I20260709 13:20:57.004935  9682 raft_consensus.cc:2243] T c50ac60e07eb46dc8836de50bce12760 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:57.005378  9682 raft_consensus.cc:2272] T c50ac60e07eb46dc8836de50bce12760 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:57.007234  9682 tablet_replica.cc:333] T ee997af6d38745549698356bcbc09a99 P 71bdb236d2304c75b7eabfec8f3d5a46: stopping tablet replica
I20260709 13:20:57.007768  9682 raft_consensus.cc:2243] T ee997af6d38745549698356bcbc09a99 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:57.008193  9682 raft_consensus.cc:2272] T ee997af6d38745549698356bcbc09a99 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:57.010001  9682 tablet_replica.cc:333] T 0f79d8e2c55b4a1f8380919ef723efe7 P 71bdb236d2304c75b7eabfec8f3d5a46: stopping tablet replica
I20260709 13:20:57.010509  9682 raft_consensus.cc:2243] T 0f79d8e2c55b4a1f8380919ef723efe7 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:57.010967  9682 raft_consensus.cc:2272] T 0f79d8e2c55b4a1f8380919ef723efe7 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:57.012663  9682 tablet_replica.cc:333] T c3add26e31364c95a9897ab53b516735 P 71bdb236d2304c75b7eabfec8f3d5a46: stopping tablet replica
I20260709 13:20:57.013230  9682 raft_consensus.cc:2243] T c3add26e31364c95a9897ab53b516735 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:57.014008  9682 raft_consensus.cc:2272] T c3add26e31364c95a9897ab53b516735 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:57.015996  9682 tablet_replica.cc:333] T b541a9ed669a420d8fbeb6e10ab3c47f P 71bdb236d2304c75b7eabfec8f3d5a46: stopping tablet replica
I20260709 13:20:57.016500  9682 raft_consensus.cc:2243] T b541a9ed669a420d8fbeb6e10ab3c47f P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:57.017249  9682 raft_consensus.cc:2272] T b541a9ed669a420d8fbeb6e10ab3c47f P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:57.019134  9682 tablet_replica.cc:333] T 35602162000346e3bbb790f54467ed0d P 71bdb236d2304c75b7eabfec8f3d5a46: stopping tablet replica
I20260709 13:20:57.019682  9682 raft_consensus.cc:2243] T 35602162000346e3bbb790f54467ed0d P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:57.020102  9682 raft_consensus.cc:2272] T 35602162000346e3bbb790f54467ed0d P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:57.021845  9682 tablet_replica.cc:333] T 5e579b46cc154814ad25486ba520710f P 71bdb236d2304c75b7eabfec8f3d5a46: stopping tablet replica
I20260709 13:20:57.022351  9682 raft_consensus.cc:2243] T 5e579b46cc154814ad25486ba520710f P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:57.022759  9682 raft_consensus.cc:2272] T 5e579b46cc154814ad25486ba520710f P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:57.024587  9682 tablet_replica.cc:333] T 35d6d5be27cb4096917b886d06a3cf58 P 71bdb236d2304c75b7eabfec8f3d5a46: stopping tablet replica
I20260709 13:20:57.025094  9682 raft_consensus.cc:2243] T 35d6d5be27cb4096917b886d06a3cf58 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:20:57.025867  9682 raft_consensus.cc:2272] T 35d6d5be27cb4096917b886d06a3cf58 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:57.043748  9682 tablet_server.cc:196] TabletServer@127.9.116.129:0 shutdown complete.
W20260709 13:20:57.113538 20892 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:20:57.237583 20985 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.116.129:40473: connect: Connection refused (error 111) [suppressed 120 similar messages]
W20260709 13:20:57.241019 20985 consensus_peers.cc:597] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 -> Peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Couldn't send request to peer 71bdb236d2304c75b7eabfec8f3d5a46. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:40473: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:57.252950 21059 consensus_peers.cc:597] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 -> Peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Couldn't send request to peer 71bdb236d2304c75b7eabfec8f3d5a46. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:40473: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:57.289261 21059 consensus_peers.cc:597] T ee997af6d38745549698356bcbc09a99 P bd6a13ebcb53485a9cc5082a4f64d652 -> Peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Couldn't send request to peer 71bdb236d2304c75b7eabfec8f3d5a46. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:40473: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:57.347783 20985 consensus_peers.cc:597] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17 -> Peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Couldn't send request to peer 71bdb236d2304c75b7eabfec8f3d5a46. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:40473: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:57.412881 21059 consensus_peers.cc:597] T 5e579b46cc154814ad25486ba520710f P bd6a13ebcb53485a9cc5082a4f64d652 -> Peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Couldn't send request to peer 71bdb236d2304c75b7eabfec8f3d5a46. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:40473: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:57.424918 20985 consensus_peers.cc:597] T 0f79d8e2c55b4a1f8380919ef723efe7 P ec8fff5744984f4695d768676cae8d17 -> Peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Couldn't send request to peer 71bdb236d2304c75b7eabfec8f3d5a46. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:40473: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:57.463034 21059 consensus_peers.cc:597] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 -> Peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Couldn't send request to peer 71bdb236d2304c75b7eabfec8f3d5a46. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:40473: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:57.957214 21143 raft_consensus.cc:493] T c3add26e31364c95a9897ab53b516735 P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 71bdb236d2304c75b7eabfec8f3d5a46)
I20260709 13:20:57.957754 21143 raft_consensus.cc:515] T c3add26e31364c95a9897ab53b516735 P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:57.959614 21143 leader_election.cc:290] T c3add26e31364c95a9897ab53b516735 P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473), ec8fff5744984f4695d768676cae8d17 (127.9.116.130:42659)
I20260709 13:20:57.960793 21021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3add26e31364c95a9897ab53b516735" candidate_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec8fff5744984f4695d768676cae8d17" is_pre_election: true
W20260709 13:20:57.963037 21059 leader_election.cc:336] T c3add26e31364c95a9897ab53b516735 P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Network error: Client connection negotiation failed: client connection to 127.9.116.129:40473: connect: Connection refused (error 111)
I20260709 13:20:57.963398 21059 leader_election.cc:304] T c3add26e31364c95a9897ab53b516735 P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd6a13ebcb53485a9cc5082a4f64d652; no voters: 71bdb236d2304c75b7eabfec8f3d5a46, ec8fff5744984f4695d768676cae8d17
I20260709 13:20:57.963992 21143 raft_consensus.cc:2749] T c3add26e31364c95a9897ab53b516735 P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:20:58.058279 21242 raft_consensus.cc:493] T 35d6d5be27cb4096917b886d06a3cf58 P ec8fff5744984f4695d768676cae8d17 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 71bdb236d2304c75b7eabfec8f3d5a46)
I20260709 13:20:58.058928 21242 raft_consensus.cc:515] T 35d6d5be27cb4096917b886d06a3cf58 P ec8fff5744984f4695d768676cae8d17 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:58.060638 21242 leader_election.cc:290] T 35d6d5be27cb4096917b886d06a3cf58 P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473), bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131:37203)
I20260709 13:20:58.062306 21094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35d6d5be27cb4096917b886d06a3cf58" candidate_uuid: "ec8fff5744984f4695d768676cae8d17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" is_pre_election: true
W20260709 13:20:58.064452 20985 leader_election.cc:336] T 35d6d5be27cb4096917b886d06a3cf58 P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Network error: Client connection negotiation failed: client connection to 127.9.116.129:40473: connect: Connection refused (error 111)
I20260709 13:20:58.064843 20985 leader_election.cc:304] T 35d6d5be27cb4096917b886d06a3cf58 P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec8fff5744984f4695d768676cae8d17; no voters: 71bdb236d2304c75b7eabfec8f3d5a46, bd6a13ebcb53485a9cc5082a4f64d652
I20260709 13:20:58.065495 21242 raft_consensus.cc:2749] T 35d6d5be27cb4096917b886d06a3cf58 P ec8fff5744984f4695d768676cae8d17 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:20:58.114486 20892 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:20:58.145361 21242 raft_consensus.cc:493] T c3add26e31364c95a9897ab53b516735 P ec8fff5744984f4695d768676cae8d17 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 71bdb236d2304c75b7eabfec8f3d5a46)
I20260709 13:20:58.145988 21242 raft_consensus.cc:515] T c3add26e31364c95a9897ab53b516735 P ec8fff5744984f4695d768676cae8d17 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:58.147943 21242 leader_election.cc:290] T c3add26e31364c95a9897ab53b516735 P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473), bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131:37203)
I20260709 13:20:58.149462 21094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3add26e31364c95a9897ab53b516735" candidate_uuid: "ec8fff5744984f4695d768676cae8d17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" is_pre_election: true
I20260709 13:20:58.150210 21094 raft_consensus.cc:2468] T c3add26e31364c95a9897ab53b516735 P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec8fff5744984f4695d768676cae8d17 in term 1.
I20260709 13:20:58.151243 20985 leader_election.cc:304] T c3add26e31364c95a9897ab53b516735 P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd6a13ebcb53485a9cc5082a4f64d652, ec8fff5744984f4695d768676cae8d17; no voters: 
I20260709 13:20:58.151948 21242 raft_consensus.cc:2804] T c3add26e31364c95a9897ab53b516735 P ec8fff5744984f4695d768676cae8d17 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:20:58.152011 20985 leader_election.cc:336] T c3add26e31364c95a9897ab53b516735 P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Network error: Client connection negotiation failed: client connection to 127.9.116.129:40473: connect: Connection refused (error 111)
I20260709 13:20:58.152362 21242 raft_consensus.cc:493] T c3add26e31364c95a9897ab53b516735 P ec8fff5744984f4695d768676cae8d17 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 71bdb236d2304c75b7eabfec8f3d5a46)
I20260709 13:20:58.152765 21242 raft_consensus.cc:3060] T c3add26e31364c95a9897ab53b516735 P ec8fff5744984f4695d768676cae8d17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:58.158205 21242 raft_consensus.cc:515] T c3add26e31364c95a9897ab53b516735 P ec8fff5744984f4695d768676cae8d17 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:58.160027 21242 leader_election.cc:290] T c3add26e31364c95a9897ab53b516735 P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 2 election: Requested vote from peers 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473), bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131:37203)
I20260709 13:20:58.161603 21094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3add26e31364c95a9897ab53b516735" candidate_uuid: "ec8fff5744984f4695d768676cae8d17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652"
I20260709 13:20:58.162352 21094 raft_consensus.cc:3060] T c3add26e31364c95a9897ab53b516735 P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:20:58.164098 20985 leader_election.cc:336] T c3add26e31364c95a9897ab53b516735 P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Network error: Client connection negotiation failed: client connection to 127.9.116.129:40473: connect: Connection refused (error 111)
I20260709 13:20:58.168148 21094 raft_consensus.cc:2468] T c3add26e31364c95a9897ab53b516735 P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec8fff5744984f4695d768676cae8d17 in term 2.
I20260709 13:20:58.169345 20985 leader_election.cc:304] T c3add26e31364c95a9897ab53b516735 P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bd6a13ebcb53485a9cc5082a4f64d652, ec8fff5744984f4695d768676cae8d17; no voters: 71bdb236d2304c75b7eabfec8f3d5a46
I20260709 13:20:58.170102 21242 raft_consensus.cc:2804] T c3add26e31364c95a9897ab53b516735 P ec8fff5744984f4695d768676cae8d17 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:20:58.170532 21242 raft_consensus.cc:697] T c3add26e31364c95a9897ab53b516735 P ec8fff5744984f4695d768676cae8d17 [term 2 LEADER]: Becoming Leader. State: Replica: ec8fff5744984f4695d768676cae8d17, State: Running, Role: LEADER
I20260709 13:20:58.171232 21242 consensus_queue.cc:237] T c3add26e31364c95a9897ab53b516735 P ec8fff5744984f4695d768676cae8d17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:58.178318 20844 catalog_manager.cc:5697] T c3add26e31364c95a9897ab53b516735 P ec8fff5744984f4695d768676cae8d17 reported cstate change: term changed from 1 to 2, leader changed from 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129) to ec8fff5744984f4695d768676cae8d17 (127.9.116.130). New cstate: current_term: 2 leader_uuid: "ec8fff5744984f4695d768676cae8d17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:58.343403 21143 raft_consensus.cc:493] T 35d6d5be27cb4096917b886d06a3cf58 P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 71bdb236d2304c75b7eabfec8f3d5a46)
I20260709 13:20:58.343956 21143 raft_consensus.cc:515] T 35d6d5be27cb4096917b886d06a3cf58 P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:58.345793 21143 leader_election.cc:290] T 35d6d5be27cb4096917b886d06a3cf58 P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473), ec8fff5744984f4695d768676cae8d17 (127.9.116.130:42659)
I20260709 13:20:58.347113 21021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35d6d5be27cb4096917b886d06a3cf58" candidate_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec8fff5744984f4695d768676cae8d17" is_pre_election: true
I20260709 13:20:58.347836 21021 raft_consensus.cc:2468] T 35d6d5be27cb4096917b886d06a3cf58 P ec8fff5744984f4695d768676cae8d17 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd6a13ebcb53485a9cc5082a4f64d652 in term 1.
I20260709 13:20:58.348905 21061 leader_election.cc:304] T 35d6d5be27cb4096917b886d06a3cf58 P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd6a13ebcb53485a9cc5082a4f64d652, ec8fff5744984f4695d768676cae8d17; no voters: 
W20260709 13:20:58.349546 21059 leader_election.cc:336] T 35d6d5be27cb4096917b886d06a3cf58 P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Network error: Client connection negotiation failed: client connection to 127.9.116.129:40473: connect: Connection refused (error 111)
I20260709 13:20:58.349593 21143 raft_consensus.cc:2804] T 35d6d5be27cb4096917b886d06a3cf58 P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:20:58.350069 21143 raft_consensus.cc:493] T 35d6d5be27cb4096917b886d06a3cf58 P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 71bdb236d2304c75b7eabfec8f3d5a46)
I20260709 13:20:58.350373 21143 raft_consensus.cc:3060] T 35d6d5be27cb4096917b886d06a3cf58 P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:58.355469 21143 raft_consensus.cc:515] T 35d6d5be27cb4096917b886d06a3cf58 P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:58.357368 21143 leader_election.cc:290] T 35d6d5be27cb4096917b886d06a3cf58 P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 2 election: Requested vote from peers 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473), ec8fff5744984f4695d768676cae8d17 (127.9.116.130:42659)
I20260709 13:20:58.357530 21242 raft_consensus.cc:493] T b541a9ed669a420d8fbeb6e10ab3c47f P ec8fff5744984f4695d768676cae8d17 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 71bdb236d2304c75b7eabfec8f3d5a46)
I20260709 13:20:58.358184 21242 raft_consensus.cc:515] T b541a9ed669a420d8fbeb6e10ab3c47f P ec8fff5744984f4695d768676cae8d17 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } }
I20260709 13:20:58.358556 21021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35d6d5be27cb4096917b886d06a3cf58" candidate_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec8fff5744984f4695d768676cae8d17"
I20260709 13:20:58.359196 21021 raft_consensus.cc:3060] T 35d6d5be27cb4096917b886d06a3cf58 P ec8fff5744984f4695d768676cae8d17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:20:58.360479 21242 leader_election.cc:290] T b541a9ed669a420d8fbeb6e10ab3c47f P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131:37203), 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473)
I20260709 13:20:58.361855 21094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b541a9ed669a420d8fbeb6e10ab3c47f" candidate_uuid: "ec8fff5744984f4695d768676cae8d17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" is_pre_election: true
W20260709 13:20:58.363626 21059 leader_election.cc:336] T 35d6d5be27cb4096917b886d06a3cf58 P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Network error: Client connection negotiation failed: client connection to 127.9.116.129:40473: connect: Connection refused (error 111)
W20260709 13:20:58.365370 20985 leader_election.cc:336] T b541a9ed669a420d8fbeb6e10ab3c47f P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Network error: Client connection negotiation failed: client connection to 127.9.116.129:40473: connect: Connection refused (error 111)
I20260709 13:20:58.365806 20985 leader_election.cc:304] T b541a9ed669a420d8fbeb6e10ab3c47f P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec8fff5744984f4695d768676cae8d17; no voters: 71bdb236d2304c75b7eabfec8f3d5a46, bd6a13ebcb53485a9cc5082a4f64d652
I20260709 13:20:58.366240 21021 raft_consensus.cc:2468] T 35d6d5be27cb4096917b886d06a3cf58 P ec8fff5744984f4695d768676cae8d17 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd6a13ebcb53485a9cc5082a4f64d652 in term 2.
I20260709 13:20:58.366572 21242 raft_consensus.cc:2749] T b541a9ed669a420d8fbeb6e10ab3c47f P ec8fff5744984f4695d768676cae8d17 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:20:58.367482 21061 leader_election.cc:304] T 35d6d5be27cb4096917b886d06a3cf58 P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bd6a13ebcb53485a9cc5082a4f64d652, ec8fff5744984f4695d768676cae8d17; no voters: 71bdb236d2304c75b7eabfec8f3d5a46
I20260709 13:20:58.368086 21143 raft_consensus.cc:2804] T 35d6d5be27cb4096917b886d06a3cf58 P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:20:58.368482 21143 raft_consensus.cc:697] T 35d6d5be27cb4096917b886d06a3cf58 P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 LEADER]: Becoming Leader. State: Replica: bd6a13ebcb53485a9cc5082a4f64d652, State: Running, Role: LEADER
I20260709 13:20:58.369280 21143 consensus_queue.cc:237] T 35d6d5be27cb4096917b886d06a3cf58 P bd6a13ebcb53485a9cc5082a4f64d652 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:20:58.377060 20844 catalog_manager.cc:5697] T 35d6d5be27cb4096917b886d06a3cf58 P bd6a13ebcb53485a9cc5082a4f64d652 reported cstate change: term changed from 1 to 2, leader changed from 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129) to bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131). New cstate: current_term: 2 leader_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:58.390484 21143 raft_consensus.cc:493] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 71bdb236d2304c75b7eabfec8f3d5a46)
I20260709 13:20:58.391084 21143 raft_consensus.cc:515] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } }
I20260709 13:20:58.393119 21143 leader_election.cc:290] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ec8fff5744984f4695d768676cae8d17 (127.9.116.130:42659), 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473)
I20260709 13:20:58.393965 21021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b541a9ed669a420d8fbeb6e10ab3c47f" candidate_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec8fff5744984f4695d768676cae8d17" is_pre_election: true
I20260709 13:20:58.394739 21021 raft_consensus.cc:2468] T b541a9ed669a420d8fbeb6e10ab3c47f P ec8fff5744984f4695d768676cae8d17 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd6a13ebcb53485a9cc5082a4f64d652 in term 1.
I20260709 13:20:58.396029 21061 leader_election.cc:304] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd6a13ebcb53485a9cc5082a4f64d652, ec8fff5744984f4695d768676cae8d17; no voters: 
I20260709 13:20:58.396849 21143 raft_consensus.cc:2804] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:20:58.397156 21143 raft_consensus.cc:493] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 71bdb236d2304c75b7eabfec8f3d5a46)
I20260709 13:20:58.397466 21143 raft_consensus.cc:3060] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:20:58.397611 21059 leader_election.cc:336] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Network error: Client connection negotiation failed: client connection to 127.9.116.129:40473: connect: Connection refused (error 111)
I20260709 13:20:58.403481 21143 raft_consensus.cc:515] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } }
I20260709 13:20:58.405133 21143 leader_election.cc:290] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 2 election: Requested vote from peers ec8fff5744984f4695d768676cae8d17 (127.9.116.130:42659), 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473)
I20260709 13:20:58.405906 21021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b541a9ed669a420d8fbeb6e10ab3c47f" candidate_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec8fff5744984f4695d768676cae8d17"
I20260709 13:20:58.406476 21021 raft_consensus.cc:3060] T b541a9ed669a420d8fbeb6e10ab3c47f P ec8fff5744984f4695d768676cae8d17 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:20:58.408892 21059 leader_election.cc:336] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Network error: Client connection negotiation failed: client connection to 127.9.116.129:40473: connect: Connection refused (error 111)
I20260709 13:20:58.412163 21021 raft_consensus.cc:2468] T b541a9ed669a420d8fbeb6e10ab3c47f P ec8fff5744984f4695d768676cae8d17 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd6a13ebcb53485a9cc5082a4f64d652 in term 2.
I20260709 13:20:58.413470 21061 leader_election.cc:304] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bd6a13ebcb53485a9cc5082a4f64d652, ec8fff5744984f4695d768676cae8d17; no voters: 71bdb236d2304c75b7eabfec8f3d5a46
I20260709 13:20:58.414311 21143 raft_consensus.cc:2804] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:20:58.414749 21143 raft_consensus.cc:697] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 LEADER]: Becoming Leader. State: Replica: bd6a13ebcb53485a9cc5082a4f64d652, State: Running, Role: LEADER
I20260709 13:20:58.415547 21143 consensus_queue.cc:237] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } }
I20260709 13:20:58.422307 20844 catalog_manager.cc:5697] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 reported cstate change: term changed from 1 to 2, leader changed from 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129) to bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131). New cstate: current_term: 2 leader_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:58.592548 21094 raft_consensus.cc:1275] T c3add26e31364c95a9897ab53b516735 P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 FOLLOWER]: Refusing update from remote peer ec8fff5744984f4695d768676cae8d17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:58.593840 21242 consensus_queue.cc:1048] T c3add26e31364c95a9897ab53b516735 P ec8fff5744984f4695d768676cae8d17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:20:58.607870 20985 consensus_peers.cc:597] T c3add26e31364c95a9897ab53b516735 P ec8fff5744984f4695d768676cae8d17 -> Peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Couldn't send request to peer 71bdb236d2304c75b7eabfec8f3d5a46. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:40473: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:58.772693 21059 consensus_peers.cc:597] T 35d6d5be27cb4096917b886d06a3cf58 P bd6a13ebcb53485a9cc5082a4f64d652 -> Peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Couldn't send request to peer 71bdb236d2304c75b7eabfec8f3d5a46. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:40473: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:58.930310 21059 consensus_peers.cc:597] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 -> Peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Couldn't send request to peer 71bdb236d2304c75b7eabfec8f3d5a46. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:40473: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:58.934418 21021 raft_consensus.cc:1275] T b541a9ed669a420d8fbeb6e10ab3c47f P ec8fff5744984f4695d768676cae8d17 [term 2 FOLLOWER]: Refusing update from remote peer bd6a13ebcb53485a9cc5082a4f64d652: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:58.935837 21143 consensus_queue.cc:1048] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:20:58.992592 21021 raft_consensus.cc:1275] T 35d6d5be27cb4096917b886d06a3cf58 P ec8fff5744984f4695d768676cae8d17 [term 2 FOLLOWER]: Refusing update from remote peer bd6a13ebcb53485a9cc5082a4f64d652: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:20:58.993825 21255 consensus_queue.cc:1048] T 35d6d5be27cb4096917b886d06a3cf58 P bd6a13ebcb53485a9cc5082a4f64d652 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:20:59.115659 20892 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:20:59.761336 21059 consensus_peers.cc:597] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 -> Peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Couldn't send request to peer 71bdb236d2304c75b7eabfec8f3d5a46. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:40473: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:59.803428 20985 consensus_peers.cc:597] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 -> Peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Couldn't send request to peer 71bdb236d2304c75b7eabfec8f3d5a46. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:40473: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:59.821491 20985 consensus_peers.cc:597] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17 -> Peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Couldn't send request to peer 71bdb236d2304c75b7eabfec8f3d5a46. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:40473: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:59.870229 21059 consensus_peers.cc:597] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 -> Peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Couldn't send request to peer 71bdb236d2304c75b7eabfec8f3d5a46. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:40473: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:59.950169 21059 consensus_peers.cc:597] T ee997af6d38745549698356bcbc09a99 P bd6a13ebcb53485a9cc5082a4f64d652 -> Peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Couldn't send request to peer 71bdb236d2304c75b7eabfec8f3d5a46. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:40473: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:00.057914 21059 consensus_peers.cc:597] T 5e579b46cc154814ad25486ba520710f P bd6a13ebcb53485a9cc5082a4f64d652 -> Peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Couldn't send request to peer 71bdb236d2304c75b7eabfec8f3d5a46. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:40473: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:00.083488 20985 consensus_peers.cc:597] T 0f79d8e2c55b4a1f8380919ef723efe7 P ec8fff5744984f4695d768676cae8d17 -> Peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Couldn't send request to peer 71bdb236d2304c75b7eabfec8f3d5a46. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:40473: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:00.116595 20892 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:21:00.804955 21116 debug-util.cc:398] Leaking SignalData structure 0x7b08000ffd60 after lost signal to thread 9685
W20260709 13:21:00.806286 21116 debug-util.cc:398] Leaking SignalData structure 0x7b0800208c40 after lost signal to thread 20878
W20260709 13:21:00.807996 21116 debug-util.cc:398] Leaking SignalData structure 0x7b08001365c0 after lost signal to thread 21045
W20260709 13:21:00.809319 21116 debug-util.cc:398] Leaking SignalData structure 0x7b08004028a0 after lost signal to thread 21119
W20260709 13:21:01.118065 20892 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:21:01.246325 21059 consensus_peers.cc:597] T 35d6d5be27cb4096917b886d06a3cf58 P bd6a13ebcb53485a9cc5082a4f64d652 -> Peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Couldn't send request to peer 71bdb236d2304c75b7eabfec8f3d5a46. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:40473: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:01.347219 20985 consensus_peers.cc:597] T c3add26e31364c95a9897ab53b516735 P ec8fff5744984f4695d768676cae8d17 -> Peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Couldn't send request to peer 71bdb236d2304c75b7eabfec8f3d5a46. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:40473: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:01.569995 21059 consensus_peers.cc:597] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 -> Peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Couldn't send request to peer 71bdb236d2304c75b7eabfec8f3d5a46. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:40473: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:01.948601 21059 consensus_peers.cc:597] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 -> Peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Couldn't send request to peer 71bdb236d2304c75b7eabfec8f3d5a46. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:40473: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:02.119448 20892 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:21:02.150758  9682 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:21:02.159709 20985 consensus_peers.cc:597] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17 -> Peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Couldn't send request to peer 71bdb236d2304c75b7eabfec8f3d5a46. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:40473: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:02.161466 21262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:02.166127 21263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:02.167725 21265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:02.169330  9682 server_base.cc:1061] running on GCE node
I20260709 13:21:02.171622  9682 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:21:02.171912  9682 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:21:02.172113  9682 hybrid_clock.cc:648] HybridClock initialized: now 1783603262172098 us; error 0 us; skew 500 ppm
I20260709 13:21:02.172708  9682 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:21:02.176191  9682 webserver.cc:533] Webserver started at http://127.9.116.129:34965/ using document root <none> and password file <none>
I20260709 13:21:02.176831  9682 fs_manager.cc:362] Metadata directory not provided
I20260709 13:21:02.177047  9682 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:21:02.181849  9682 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.001s	sys 0.004s
I20260709 13:21:02.186448 21270 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:02.187515  9682 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.003s
I20260709 13:21:02.187963  9682 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602916530764-9682-0/minicluster-data/ts-0-root
uuid: "71bdb236d2304c75b7eabfec8f3d5a46"
format_stamp: "Formatted at 2026-07-09 13:20:53 on dist-test-slave-1t6m"
I20260709 13:21:02.188658  9682 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602916530764-9682-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602916530764-9682-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602916530764-9682-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:21:02.206364  9682 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:21:02.207794  9682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:21:02.209671  9682 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:21:02.214557 21277 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
I20260709 13:21:02.270869  9682 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 13:21:02.271314  9682 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.058s	user 0.003s	sys 0.000s
I20260709 13:21:02.271708  9682 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
W20260709 13:21:02.275835 21059 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.116.129:40473: connect: Connection refused (error 111) [suppressed 100 similar messages]
I20260709 13:21:02.282593 21277 tablet_bootstrap.cc:492] T 7f3178c915104a27b9a3b6978249c5ed P 71bdb236d2304c75b7eabfec8f3d5a46: Bootstrap starting.
W20260709 13:21:02.305558 20985 consensus_peers.cc:597] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 -> Peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Couldn't send request to peer 71bdb236d2304c75b7eabfec8f3d5a46. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:40473: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:02.308709 21277 tablet_bootstrap.cc:492] T 7f3178c915104a27b9a3b6978249c5ed P 71bdb236d2304c75b7eabfec8f3d5a46: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:02.309865 21277 tablet_bootstrap.cc:492] T 7f3178c915104a27b9a3b6978249c5ed P 71bdb236d2304c75b7eabfec8f3d5a46: Bootstrap complete.
I20260709 13:21:02.310627 21277 ts_tablet_manager.cc:1403] T 7f3178c915104a27b9a3b6978249c5ed P 71bdb236d2304c75b7eabfec8f3d5a46: Time spent bootstrapping tablet: real 0.028s	user 0.013s	sys 0.005s
I20260709 13:21:02.314347 21277 raft_consensus.cc:359] T 7f3178c915104a27b9a3b6978249c5ed P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:02.315754 21277 raft_consensus.cc:740] T 7f3178c915104a27b9a3b6978249c5ed P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71bdb236d2304c75b7eabfec8f3d5a46, State: Initialized, Role: FOLLOWER
I20260709 13:21:02.316797 21277 consensus_queue.cc:260] T 7f3178c915104a27b9a3b6978249c5ed P 71bdb236d2304c75b7eabfec8f3d5a46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:02.320994 21277 ts_tablet_manager.cc:1434] T 7f3178c915104a27b9a3b6978249c5ed P 71bdb236d2304c75b7eabfec8f3d5a46: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:21:02.322069 21277 tablet_bootstrap.cc:492] T 35d6d5be27cb4096917b886d06a3cf58 P 71bdb236d2304c75b7eabfec8f3d5a46: Bootstrap starting.
I20260709 13:21:02.336462  9682 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 13:21:02.336933  9682 ts_tablet_manager.cc:595] Time spent register tablets: real 0.065s	user 0.060s	sys 0.000s
I20260709 13:21:02.341480 21277 tablet_bootstrap.cc:492] T 35d6d5be27cb4096917b886d06a3cf58 P 71bdb236d2304c75b7eabfec8f3d5a46: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:02.342743 21277 tablet_bootstrap.cc:492] T 35d6d5be27cb4096917b886d06a3cf58 P 71bdb236d2304c75b7eabfec8f3d5a46: Bootstrap complete.
I20260709 13:21:02.343647 21277 ts_tablet_manager.cc:1403] T 35d6d5be27cb4096917b886d06a3cf58 P 71bdb236d2304c75b7eabfec8f3d5a46: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.004s
I20260709 13:21:02.347167 21277 raft_consensus.cc:359] T 35d6d5be27cb4096917b886d06a3cf58 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:02.348022 21277 raft_consensus.cc:740] T 35d6d5be27cb4096917b886d06a3cf58 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71bdb236d2304c75b7eabfec8f3d5a46, State: Initialized, Role: FOLLOWER
I20260709 13:21:02.348915 21277 consensus_queue.cc:260] T 35d6d5be27cb4096917b886d06a3cf58 P 71bdb236d2304c75b7eabfec8f3d5a46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:02.352221 21277 ts_tablet_manager.cc:1434] T 35d6d5be27cb4096917b886d06a3cf58 P 71bdb236d2304c75b7eabfec8f3d5a46: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:21:02.353313 21277 tablet_bootstrap.cc:492] T c50ac60e07eb46dc8836de50bce12760 P 71bdb236d2304c75b7eabfec8f3d5a46: Bootstrap starting.
I20260709 13:21:02.378022 21277 tablet_bootstrap.cc:492] T c50ac60e07eb46dc8836de50bce12760 P 71bdb236d2304c75b7eabfec8f3d5a46: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:02.379499 21277 tablet_bootstrap.cc:492] T c50ac60e07eb46dc8836de50bce12760 P 71bdb236d2304c75b7eabfec8f3d5a46: Bootstrap complete.
I20260709 13:21:02.380452 21277 ts_tablet_manager.cc:1403] T c50ac60e07eb46dc8836de50bce12760 P 71bdb236d2304c75b7eabfec8f3d5a46: Time spent bootstrapping tablet: real 0.027s	user 0.017s	sys 0.007s
I20260709 13:21:02.384375 21277 raft_consensus.cc:359] T c50ac60e07eb46dc8836de50bce12760 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:02.385389 21277 raft_consensus.cc:740] T c50ac60e07eb46dc8836de50bce12760 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71bdb236d2304c75b7eabfec8f3d5a46, State: Initialized, Role: FOLLOWER
I20260709 13:21:02.386425 21277 consensus_queue.cc:260] T c50ac60e07eb46dc8836de50bce12760 P 71bdb236d2304c75b7eabfec8f3d5a46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:02.388876 21277 ts_tablet_manager.cc:1434] T c50ac60e07eb46dc8836de50bce12760 P 71bdb236d2304c75b7eabfec8f3d5a46: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:21:02.389787 21277 tablet_bootstrap.cc:492] T ee997af6d38745549698356bcbc09a99 P 71bdb236d2304c75b7eabfec8f3d5a46: Bootstrap starting.
I20260709 13:21:02.406895 21277 tablet_bootstrap.cc:492] T ee997af6d38745549698356bcbc09a99 P 71bdb236d2304c75b7eabfec8f3d5a46: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:02.408037 21277 tablet_bootstrap.cc:492] T ee997af6d38745549698356bcbc09a99 P 71bdb236d2304c75b7eabfec8f3d5a46: Bootstrap complete.
I20260709 13:21:02.408747 21277 ts_tablet_manager.cc:1403] T ee997af6d38745549698356bcbc09a99 P 71bdb236d2304c75b7eabfec8f3d5a46: Time spent bootstrapping tablet: real 0.019s	user 0.016s	sys 0.001s
I20260709 13:21:02.411751 21277 raft_consensus.cc:359] T ee997af6d38745549698356bcbc09a99 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:02.412671 21277 raft_consensus.cc:740] T ee997af6d38745549698356bcbc09a99 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71bdb236d2304c75b7eabfec8f3d5a46, State: Initialized, Role: FOLLOWER
I20260709 13:21:02.413604 21277 consensus_queue.cc:260] T ee997af6d38745549698356bcbc09a99 P 71bdb236d2304c75b7eabfec8f3d5a46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:02.415776 21277 ts_tablet_manager.cc:1434] T ee997af6d38745549698356bcbc09a99 P 71bdb236d2304c75b7eabfec8f3d5a46: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:21:02.416679 21277 tablet_bootstrap.cc:492] T 35602162000346e3bbb790f54467ed0d P 71bdb236d2304c75b7eabfec8f3d5a46: Bootstrap starting.
I20260709 13:21:02.418398  9682 rpc_server.cc:307] RPC server started. Bound to: 127.9.116.129:40473
I20260709 13:21:02.418673 21338 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.116.129:40473 every 8 connection(s)
I20260709 13:21:02.444303  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:21:02.459918 21277 tablet_bootstrap.cc:492] T 35602162000346e3bbb790f54467ed0d P 71bdb236d2304c75b7eabfec8f3d5a46: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:02.461344 21277 tablet_bootstrap.cc:492] T 35602162000346e3bbb790f54467ed0d P 71bdb236d2304c75b7eabfec8f3d5a46: Bootstrap complete.
I20260709 13:21:02.462252 21277 ts_tablet_manager.cc:1403] T 35602162000346e3bbb790f54467ed0d P 71bdb236d2304c75b7eabfec8f3d5a46: Time spent bootstrapping tablet: real 0.046s	user 0.021s	sys 0.016s
I20260709 13:21:02.465938 21277 raft_consensus.cc:359] T 35602162000346e3bbb790f54467ed0d P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:02.472903 21277 raft_consensus.cc:740] T 35602162000346e3bbb790f54467ed0d P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71bdb236d2304c75b7eabfec8f3d5a46, State: Initialized, Role: FOLLOWER
I20260709 13:21:02.474010 21277 consensus_queue.cc:260] T 35602162000346e3bbb790f54467ed0d P 71bdb236d2304c75b7eabfec8f3d5a46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:02.474669 21339 heartbeater.cc:344] Connected to a master server at 127.9.116.190:40921
I20260709 13:21:02.475188 21339 heartbeater.cc:461] Registering TS with master...
I20260709 13:21:02.476194 21339 heartbeater.cc:507] Master 127.9.116.190:40921 requested a full tablet report, sending...
I20260709 13:21:02.476588 21277 ts_tablet_manager.cc:1434] T 35602162000346e3bbb790f54467ed0d P 71bdb236d2304c75b7eabfec8f3d5a46: Time spent starting tablet: real 0.014s	user 0.003s	sys 0.004s
I20260709 13:21:02.477496 21277 tablet_bootstrap.cc:492] T c3add26e31364c95a9897ab53b516735 P 71bdb236d2304c75b7eabfec8f3d5a46: Bootstrap starting.
I20260709 13:21:02.484155 20843 ts_manager.cc:194] Re-registered known tserver with Master: 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473)
I20260709 13:21:02.503803 21277 tablet_bootstrap.cc:492] T c3add26e31364c95a9897ab53b516735 P 71bdb236d2304c75b7eabfec8f3d5a46: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:02.505285 21277 tablet_bootstrap.cc:492] T c3add26e31364c95a9897ab53b516735 P 71bdb236d2304c75b7eabfec8f3d5a46: Bootstrap complete.
I20260709 13:21:02.506033 21277 ts_tablet_manager.cc:1403] T c3add26e31364c95a9897ab53b516735 P 71bdb236d2304c75b7eabfec8f3d5a46: Time spent bootstrapping tablet: real 0.029s	user 0.018s	sys 0.008s
I20260709 13:21:02.509066 21277 raft_consensus.cc:359] T c3add26e31364c95a9897ab53b516735 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:02.509881 21277 raft_consensus.cc:740] T c3add26e31364c95a9897ab53b516735 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71bdb236d2304c75b7eabfec8f3d5a46, State: Initialized, Role: FOLLOWER
I20260709 13:21:02.510633 21277 consensus_queue.cc:260] T c3add26e31364c95a9897ab53b516735 P 71bdb236d2304c75b7eabfec8f3d5a46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:02.513222 21277 ts_tablet_manager.cc:1434] T c3add26e31364c95a9897ab53b516735 P 71bdb236d2304c75b7eabfec8f3d5a46: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:21:02.514110 21277 tablet_bootstrap.cc:492] T b541a9ed669a420d8fbeb6e10ab3c47f P 71bdb236d2304c75b7eabfec8f3d5a46: Bootstrap starting.
W20260709 13:21:02.525832 21059 consensus_peers.cc:597] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 -> Peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Couldn't send request to peer 71bdb236d2304c75b7eabfec8f3d5a46. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:02.543114 20985 consensus_peers.cc:597] T 0f79d8e2c55b4a1f8380919ef723efe7 P ec8fff5744984f4695d768676cae8d17 -> Peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Couldn't send request to peer 71bdb236d2304c75b7eabfec8f3d5a46. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:02.544615 21019 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7f3178c915104a27b9a3b6978249c5ed"
dest_uuid: "ec8fff5744984f4695d768676cae8d17"
mode: GRACEFUL
new_leader_uuid: "bd6a13ebcb53485a9cc5082a4f64d652"
 from {username='slave'} at 127.0.0.1:36740
I20260709 13:21:02.545295 21019 raft_consensus.cc:606] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17 [term 1 LEADER]: Received request to transfer leadership to TS bd6a13ebcb53485a9cc5082a4f64d652
I20260709 13:21:02.545375 21277 tablet_bootstrap.cc:492] T b541a9ed669a420d8fbeb6e10ab3c47f P 71bdb236d2304c75b7eabfec8f3d5a46: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:02.546520 21277 tablet_bootstrap.cc:492] T b541a9ed669a420d8fbeb6e10ab3c47f P 71bdb236d2304c75b7eabfec8f3d5a46: Bootstrap complete.
I20260709 13:21:02.547226 21277 ts_tablet_manager.cc:1403] T b541a9ed669a420d8fbeb6e10ab3c47f P 71bdb236d2304c75b7eabfec8f3d5a46: Time spent bootstrapping tablet: real 0.033s	user 0.021s	sys 0.011s
I20260709 13:21:02.549463 20843 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59894
I20260709 13:21:02.550309 21277 raft_consensus.cc:359] T b541a9ed669a420d8fbeb6e10ab3c47f P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } }
I20260709 13:21:02.551265 21277 raft_consensus.cc:740] T b541a9ed669a420d8fbeb6e10ab3c47f P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71bdb236d2304c75b7eabfec8f3d5a46, State: Initialized, Role: FOLLOWER
I20260709 13:21:02.552119 21277 consensus_queue.cc:260] T b541a9ed669a420d8fbeb6e10ab3c47f P 71bdb236d2304c75b7eabfec8f3d5a46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } }
I20260709 13:21:02.554618 21339 heartbeater.cc:499] Master 127.9.116.190:40921 was elected leader, sending a full tablet report...
I20260709 13:21:02.555349 21277 ts_tablet_manager.cc:1434] T b541a9ed669a420d8fbeb6e10ab3c47f P 71bdb236d2304c75b7eabfec8f3d5a46: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:21:02.556242 21277 tablet_bootstrap.cc:492] T 2725109df1524e6c8eb897f0e6424829 P 71bdb236d2304c75b7eabfec8f3d5a46: Bootstrap starting.
I20260709 13:21:02.566854 21094 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b541a9ed669a420d8fbeb6e10ab3c47f"
dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652"
mode: GRACEFUL
new_leader_uuid: "ec8fff5744984f4695d768676cae8d17"
 from {username='slave'} at 127.0.0.1:42722
I20260709 13:21:02.567569 21094 raft_consensus.cc:606] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 LEADER]: Received request to transfer leadership to TS ec8fff5744984f4695d768676cae8d17
I20260709 13:21:02.571936 21094 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c50ac60e07eb46dc8836de50bce12760"
dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652"
mode: GRACEFUL
new_leader_uuid: "ec8fff5744984f4695d768676cae8d17"
 from {username='slave'} at 127.0.0.1:42722
I20260709 13:21:02.572708 21094 raft_consensus.cc:606] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 LEADER]: Received request to transfer leadership to TS ec8fff5744984f4695d768676cae8d17
I20260709 13:21:02.574651  9682 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:21:02.575495 21277 tablet_bootstrap.cc:492] T 2725109df1524e6c8eb897f0e6424829 P 71bdb236d2304c75b7eabfec8f3d5a46: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:02.575616  9682 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:21:02.576738  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:21:02.576759 21277 tablet_bootstrap.cc:492] T 2725109df1524e6c8eb897f0e6424829 P 71bdb236d2304c75b7eabfec8f3d5a46: Bootstrap complete.
I20260709 13:21:02.577667 21277 ts_tablet_manager.cc:1403] T 2725109df1524e6c8eb897f0e6424829 P 71bdb236d2304c75b7eabfec8f3d5a46: Time spent bootstrapping tablet: real 0.022s	user 0.020s	sys 0.000s
I20260709 13:21:02.581252 21277 raft_consensus.cc:359] T 2725109df1524e6c8eb897f0e6424829 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:02.581929 21277 raft_consensus.cc:740] T 2725109df1524e6c8eb897f0e6424829 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71bdb236d2304c75b7eabfec8f3d5a46, State: Initialized, Role: FOLLOWER
I20260709 13:21:02.582517 21277 consensus_queue.cc:260] T 2725109df1524e6c8eb897f0e6424829 P 71bdb236d2304c75b7eabfec8f3d5a46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:02.584587 21277 ts_tablet_manager.cc:1434] T 2725109df1524e6c8eb897f0e6424829 P 71bdb236d2304c75b7eabfec8f3d5a46: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:21:02.589520 21277 tablet_bootstrap.cc:492] T 5e579b46cc154814ad25486ba520710f P 71bdb236d2304c75b7eabfec8f3d5a46: Bootstrap starting.
I20260709 13:21:02.603974 21277 tablet_bootstrap.cc:492] T 5e579b46cc154814ad25486ba520710f P 71bdb236d2304c75b7eabfec8f3d5a46: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:02.604863 21277 tablet_bootstrap.cc:492] T 5e579b46cc154814ad25486ba520710f P 71bdb236d2304c75b7eabfec8f3d5a46: Bootstrap complete.
I20260709 13:21:02.605389 21277 ts_tablet_manager.cc:1403] T 5e579b46cc154814ad25486ba520710f P 71bdb236d2304c75b7eabfec8f3d5a46: Time spent bootstrapping tablet: real 0.020s	user 0.008s	sys 0.005s
I20260709 13:21:02.607465 21277 raft_consensus.cc:359] T 5e579b46cc154814ad25486ba520710f P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:02.607962 21277 raft_consensus.cc:740] T 5e579b46cc154814ad25486ba520710f P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71bdb236d2304c75b7eabfec8f3d5a46, State: Initialized, Role: FOLLOWER
I20260709 13:21:02.608547 21277 consensus_queue.cc:260] T 5e579b46cc154814ad25486ba520710f P 71bdb236d2304c75b7eabfec8f3d5a46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:02.610097 21277 ts_tablet_manager.cc:1434] T 5e579b46cc154814ad25486ba520710f P 71bdb236d2304c75b7eabfec8f3d5a46: Time spent starting tablet: real 0.004s	user 0.002s	sys 0.002s
I20260709 13:21:02.610924 21277 tablet_bootstrap.cc:492] T 0f79d8e2c55b4a1f8380919ef723efe7 P 71bdb236d2304c75b7eabfec8f3d5a46: Bootstrap starting.
I20260709 13:21:02.626369 21277 tablet_bootstrap.cc:492] T 0f79d8e2c55b4a1f8380919ef723efe7 P 71bdb236d2304c75b7eabfec8f3d5a46: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:02.627351 21277 tablet_bootstrap.cc:492] T 0f79d8e2c55b4a1f8380919ef723efe7 P 71bdb236d2304c75b7eabfec8f3d5a46: Bootstrap complete.
I20260709 13:21:02.627907 21277 ts_tablet_manager.cc:1403] T 0f79d8e2c55b4a1f8380919ef723efe7 P 71bdb236d2304c75b7eabfec8f3d5a46: Time spent bootstrapping tablet: real 0.017s	user 0.012s	sys 0.003s
I20260709 13:21:02.629755 21277 raft_consensus.cc:359] T 0f79d8e2c55b4a1f8380919ef723efe7 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:02.630246 21277 raft_consensus.cc:740] T 0f79d8e2c55b4a1f8380919ef723efe7 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71bdb236d2304c75b7eabfec8f3d5a46, State: Initialized, Role: FOLLOWER
I20260709 13:21:02.630892 21277 consensus_queue.cc:260] T 0f79d8e2c55b4a1f8380919ef723efe7 P 71bdb236d2304c75b7eabfec8f3d5a46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:02.632441 21277 ts_tablet_manager.cc:1434] T 0f79d8e2c55b4a1f8380919ef723efe7 P 71bdb236d2304c75b7eabfec8f3d5a46: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:21:02.750892 21261 raft_consensus.cc:993] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652: : Instructing follower ec8fff5744984f4695d768676cae8d17 to start an election
I20260709 13:21:02.751593 21353 raft_consensus.cc:1081] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 LEADER]: Signalling peer ec8fff5744984f4695d768676cae8d17 to start an election
I20260709 13:21:02.753428 21019 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c50ac60e07eb46dc8836de50bce12760"
dest_uuid: "ec8fff5744984f4695d768676cae8d17"
 from {username='slave'} at 127.0.0.1:54288
I20260709 13:21:02.753984 21019 raft_consensus.cc:493] T c50ac60e07eb46dc8836de50bce12760 P ec8fff5744984f4695d768676cae8d17 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:02.754330 21019 raft_consensus.cc:3060] T c50ac60e07eb46dc8836de50bce12760 P ec8fff5744984f4695d768676cae8d17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:02.764137 21019 raft_consensus.cc:515] T c50ac60e07eb46dc8836de50bce12760 P ec8fff5744984f4695d768676cae8d17 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:02.766527 21019 leader_election.cc:290] T c50ac60e07eb46dc8836de50bce12760 P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 2 election: Requested vote from peers 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473), bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131:37203)
I20260709 13:21:02.767409 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50ac60e07eb46dc8836de50bce12760" candidate_uuid: "ec8fff5744984f4695d768676cae8d17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "71bdb236d2304c75b7eabfec8f3d5a46"
I20260709 13:21:02.767686 21094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50ac60e07eb46dc8836de50bce12760" candidate_uuid: "ec8fff5744984f4695d768676cae8d17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652"
I20260709 13:21:02.768317 21312 raft_consensus.cc:3060] T c50ac60e07eb46dc8836de50bce12760 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:02.768453 21094 raft_consensus.cc:3055] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:02.768908 21094 raft_consensus.cc:740] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bd6a13ebcb53485a9cc5082a4f64d652, State: Running, Role: LEADER
I20260709 13:21:02.770076 21094 consensus_queue.cc:260] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:02.771864 21094 raft_consensus.cc:3060] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:02.775377 21312 raft_consensus.cc:2468] T c50ac60e07eb46dc8836de50bce12760 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec8fff5744984f4695d768676cae8d17 in term 2.
I20260709 13:21:02.777137 20985 leader_election.cc:304] T c50ac60e07eb46dc8836de50bce12760 P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71bdb236d2304c75b7eabfec8f3d5a46, ec8fff5744984f4695d768676cae8d17; no voters: 
I20260709 13:21:02.777280 21313 raft_consensus.cc:3060] T 35d6d5be27cb4096917b886d06a3cf58 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:02.778210 21258 raft_consensus.cc:2804] T c50ac60e07eb46dc8836de50bce12760 P ec8fff5744984f4695d768676cae8d17 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:02.783206 21258 raft_consensus.cc:697] T c50ac60e07eb46dc8836de50bce12760 P ec8fff5744984f4695d768676cae8d17 [term 2 LEADER]: Becoming Leader. State: Replica: ec8fff5744984f4695d768676cae8d17, State: Running, Role: LEADER
I20260709 13:21:02.784973 21258 consensus_queue.cc:237] T c50ac60e07eb46dc8836de50bce12760 P ec8fff5744984f4695d768676cae8d17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:02.788182 21094 raft_consensus.cc:2468] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec8fff5744984f4695d768676cae8d17 in term 2.
I20260709 13:21:02.800246 20843 catalog_manager.cc:5697] T c50ac60e07eb46dc8836de50bce12760 P ec8fff5744984f4695d768676cae8d17 reported cstate change: term changed from 1 to 2, leader changed from bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131) to ec8fff5744984f4695d768676cae8d17 (127.9.116.130). New cstate: current_term: 2 leader_uuid: "ec8fff5744984f4695d768676cae8d17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:02.802765 21258 raft_consensus.cc:993] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17: : Instructing follower bd6a13ebcb53485a9cc5082a4f64d652 to start an election
I20260709 13:21:02.803344 21258 raft_consensus.cc:1081] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17 [term 1 LEADER]: Signalling peer bd6a13ebcb53485a9cc5082a4f64d652 to start an election
I20260709 13:21:02.805624 21094 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7f3178c915104a27b9a3b6978249c5ed"
dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652"
 from {username='slave'} at 127.0.0.1:52450
I20260709 13:21:02.806383 21094 raft_consensus.cc:493] T 7f3178c915104a27b9a3b6978249c5ed P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:02.806923 21094 raft_consensus.cc:3060] T 7f3178c915104a27b9a3b6978249c5ed P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:02.818578 21094 raft_consensus.cc:515] T 7f3178c915104a27b9a3b6978249c5ed P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:02.821491 21094 leader_election.cc:290] T 7f3178c915104a27b9a3b6978249c5ed P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 2 election: Requested vote from peers 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473), ec8fff5744984f4695d768676cae8d17 (127.9.116.130:42659)
I20260709 13:21:02.822651 21019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f3178c915104a27b9a3b6978249c5ed" candidate_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ec8fff5744984f4695d768676cae8d17"
I20260709 13:21:02.823153 21313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f3178c915104a27b9a3b6978249c5ed" candidate_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "71bdb236d2304c75b7eabfec8f3d5a46"
I20260709 13:21:02.823463 21019 raft_consensus.cc:3055] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:02.823810 21019 raft_consensus.cc:740] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ec8fff5744984f4695d768676cae8d17, State: Running, Role: LEADER
I20260709 13:21:02.823812 21313 raft_consensus.cc:3060] T 7f3178c915104a27b9a3b6978249c5ed P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:02.824875 21019 consensus_queue.cc:260] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:02.826387 21019 raft_consensus.cc:3060] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:02.832098 21313 raft_consensus.cc:2468] T 7f3178c915104a27b9a3b6978249c5ed P 71bdb236d2304c75b7eabfec8f3d5a46 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd6a13ebcb53485a9cc5082a4f64d652 in term 2.
I20260709 13:21:02.833595 21059 leader_election.cc:304] T 7f3178c915104a27b9a3b6978249c5ed P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71bdb236d2304c75b7eabfec8f3d5a46, bd6a13ebcb53485a9cc5082a4f64d652; no voters: 
I20260709 13:21:02.834611 21353 raft_consensus.cc:2804] T 7f3178c915104a27b9a3b6978249c5ed P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:02.835143 21353 raft_consensus.cc:697] T 7f3178c915104a27b9a3b6978249c5ed P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 LEADER]: Becoming Leader. State: Replica: bd6a13ebcb53485a9cc5082a4f64d652, State: Running, Role: LEADER
I20260709 13:21:02.835179 21019 raft_consensus.cc:2468] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd6a13ebcb53485a9cc5082a4f64d652 in term 2.
I20260709 13:21:02.836298 21353 consensus_queue.cc:237] T 7f3178c915104a27b9a3b6978249c5ed P bd6a13ebcb53485a9cc5082a4f64d652 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:02.850445 20843 catalog_manager.cc:5697] T 7f3178c915104a27b9a3b6978249c5ed P bd6a13ebcb53485a9cc5082a4f64d652 reported cstate change: term changed from 1 to 2, leader changed from ec8fff5744984f4695d768676cae8d17 (127.9.116.130) to bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131). New cstate: current_term: 2 leader_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:02.861280 21313 raft_consensus.cc:3060] T c3add26e31364c95a9897ab53b516735 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:02.944242 21261 raft_consensus.cc:993] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652: : Instructing follower ec8fff5744984f4695d768676cae8d17 to start an election
I20260709 13:21:02.944775 21353 raft_consensus.cc:1081] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 LEADER]: Signalling peer ec8fff5744984f4695d768676cae8d17 to start an election
I20260709 13:21:02.946597 21019 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b541a9ed669a420d8fbeb6e10ab3c47f"
dest_uuid: "ec8fff5744984f4695d768676cae8d17"
 from {username='slave'} at 127.0.0.1:54288
I20260709 13:21:02.947172 21019 raft_consensus.cc:493] T b541a9ed669a420d8fbeb6e10ab3c47f P ec8fff5744984f4695d768676cae8d17 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:02.947486 21019 raft_consensus.cc:3060] T b541a9ed669a420d8fbeb6e10ab3c47f P ec8fff5744984f4695d768676cae8d17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:02.954491 21019 raft_consensus.cc:515] T b541a9ed669a420d8fbeb6e10ab3c47f P ec8fff5744984f4695d768676cae8d17 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } }
I20260709 13:21:02.957393 21019 leader_election.cc:290] T b541a9ed669a420d8fbeb6e10ab3c47f P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 3 election: Requested vote from peers bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131:37203), 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473)
I20260709 13:21:02.957333 21094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b541a9ed669a420d8fbeb6e10ab3c47f" candidate_uuid: "ec8fff5744984f4695d768676cae8d17" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652"
I20260709 13:21:02.958246 21094 raft_consensus.cc:3055] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:21:02.958595 21094 raft_consensus.cc:740] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bd6a13ebcb53485a9cc5082a4f64d652, State: Running, Role: LEADER
I20260709 13:21:02.959511 21313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b541a9ed669a420d8fbeb6e10ab3c47f" candidate_uuid: "ec8fff5744984f4695d768676cae8d17" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "71bdb236d2304c75b7eabfec8f3d5a46"
I20260709 13:21:02.960199 21313 raft_consensus.cc:3060] T b541a9ed669a420d8fbeb6e10ab3c47f P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:21:02.960372 21094 consensus_queue.cc:260] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } }
I20260709 13:21:02.962026 21094 raft_consensus.cc:3060] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:02.966684 21313 raft_consensus.cc:2468] T b541a9ed669a420d8fbeb6e10ab3c47f P 71bdb236d2304c75b7eabfec8f3d5a46 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec8fff5744984f4695d768676cae8d17 in term 3.
I20260709 13:21:02.968155 20985 leader_election.cc:304] T b541a9ed669a420d8fbeb6e10ab3c47f P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71bdb236d2304c75b7eabfec8f3d5a46, ec8fff5744984f4695d768676cae8d17; no voters: 
I20260709 13:21:02.968998 21354 raft_consensus.cc:2804] T b541a9ed669a420d8fbeb6e10ab3c47f P ec8fff5744984f4695d768676cae8d17 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:21:02.969448 21354 raft_consensus.cc:697] T b541a9ed669a420d8fbeb6e10ab3c47f P ec8fff5744984f4695d768676cae8d17 [term 3 LEADER]: Becoming Leader. State: Replica: ec8fff5744984f4695d768676cae8d17, State: Running, Role: LEADER
I20260709 13:21:02.970602 21094 raft_consensus.cc:2468] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec8fff5744984f4695d768676cae8d17 in term 3.
I20260709 13:21:02.970398 21354 consensus_queue.cc:237] T b541a9ed669a420d8fbeb6e10ab3c47f P ec8fff5744984f4695d768676cae8d17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } }
I20260709 13:21:02.987705 20843 catalog_manager.cc:5697] T b541a9ed669a420d8fbeb6e10ab3c47f P ec8fff5744984f4695d768676cae8d17 reported cstate change: term changed from 2 to 3, leader changed from bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131) to ec8fff5744984f4695d768676cae8d17 (127.9.116.130). New cstate: current_term: 3 leader_uuid: "ec8fff5744984f4695d768676cae8d17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } health_report { overall_health: UNKNOWN } } }
W20260709 13:21:03.120555 20892 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:21:03.183411 21313 raft_consensus.cc:1275] T c50ac60e07eb46dc8836de50bce12760 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 2 FOLLOWER]: Refusing update from remote peer ec8fff5744984f4695d768676cae8d17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:03.184940 21354 consensus_queue.cc:1048] T c50ac60e07eb46dc8836de50bce12760 P ec8fff5744984f4695d768676cae8d17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:03.204071 21094 raft_consensus.cc:1275] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 FOLLOWER]: Refusing update from remote peer ec8fff5744984f4695d768676cae8d17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:03.205989 21354 consensus_queue.cc:1048] T c50ac60e07eb46dc8836de50bce12760 P ec8fff5744984f4695d768676cae8d17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:03.334679 21313 raft_consensus.cc:1275] T 7f3178c915104a27b9a3b6978249c5ed P 71bdb236d2304c75b7eabfec8f3d5a46 [term 2 FOLLOWER]: Refusing update from remote peer bd6a13ebcb53485a9cc5082a4f64d652: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:03.336210 21365 consensus_queue.cc:1048] T 7f3178c915104a27b9a3b6978249c5ed P bd6a13ebcb53485a9cc5082a4f64d652 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:03.346096 21019 raft_consensus.cc:1275] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17 [term 2 FOLLOWER]: Refusing update from remote peer bd6a13ebcb53485a9cc5082a4f64d652: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:03.353611 21261 consensus_queue.cc:1048] T 7f3178c915104a27b9a3b6978249c5ed P bd6a13ebcb53485a9cc5082a4f64d652 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:03.381050 21094 raft_consensus.cc:1275] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 [term 3 FOLLOWER]: Refusing update from remote peer ec8fff5744984f4695d768676cae8d17: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:21:03.382671 21368 consensus_queue.cc:1048] T b541a9ed669a420d8fbeb6e10ab3c47f P ec8fff5744984f4695d768676cae8d17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:21:03.394274 21313 raft_consensus.cc:1275] T b541a9ed669a420d8fbeb6e10ab3c47f P 71bdb236d2304c75b7eabfec8f3d5a46 [term 3 FOLLOWER]: Refusing update from remote peer ec8fff5744984f4695d768676cae8d17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:21:03.396351 21354 consensus_queue.cc:1048] T b541a9ed669a420d8fbeb6e10ab3c47f P ec8fff5744984f4695d768676cae8d17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:03.588671  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:21:03.596369 21094 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2725109df1524e6c8eb897f0e6424829"
dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652"
mode: GRACEFUL
new_leader_uuid: "ec8fff5744984f4695d768676cae8d17"
 from {username='slave'} at 127.0.0.1:42722
I20260709 13:21:03.596932 21094 raft_consensus.cc:606] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 LEADER]: Received request to transfer leadership to TS ec8fff5744984f4695d768676cae8d17
I20260709 13:21:03.599593 21019 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "35602162000346e3bbb790f54467ed0d"
dest_uuid: "ec8fff5744984f4695d768676cae8d17"
mode: GRACEFUL
new_leader_uuid: "bd6a13ebcb53485a9cc5082a4f64d652"
 from {username='slave'} at 127.0.0.1:36740
I20260709 13:21:03.600294 21019 raft_consensus.cc:606] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [term 1 LEADER]: Received request to transfer leadership to TS bd6a13ebcb53485a9cc5082a4f64d652
I20260709 13:21:03.603585 21019 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b541a9ed669a420d8fbeb6e10ab3c47f"
dest_uuid: "ec8fff5744984f4695d768676cae8d17"
mode: GRACEFUL
new_leader_uuid: "bd6a13ebcb53485a9cc5082a4f64d652"
 from {username='slave'} at 127.0.0.1:36740
I20260709 13:21:03.604332 21019 raft_consensus.cc:606] T b541a9ed669a420d8fbeb6e10ab3c47f P ec8fff5744984f4695d768676cae8d17 [term 3 LEADER]: Received request to transfer leadership to TS bd6a13ebcb53485a9cc5082a4f64d652
I20260709 13:21:03.605631  9682 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:21:03.606330  9682 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:21:03.607127  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:21:03.662441 21258 raft_consensus.cc:993] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17: : Instructing follower bd6a13ebcb53485a9cc5082a4f64d652 to start an election
I20260709 13:21:03.663023 21368 raft_consensus.cc:1081] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [term 1 LEADER]: Signalling peer bd6a13ebcb53485a9cc5082a4f64d652 to start an election
I20260709 13:21:03.664646 21094 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "35602162000346e3bbb790f54467ed0d"
dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652"
 from {username='slave'} at 127.0.0.1:52450
I20260709 13:21:03.665134 21094 raft_consensus.cc:493] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:03.665421 21094 raft_consensus.cc:3060] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:03.670336 21094 raft_consensus.cc:515] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:03.672230 21094 leader_election.cc:290] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 2 election: Requested vote from peers 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473), ec8fff5744984f4695d768676cae8d17 (127.9.116.130:42659)
I20260709 13:21:03.673223 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35602162000346e3bbb790f54467ed0d" candidate_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "71bdb236d2304c75b7eabfec8f3d5a46"
I20260709 13:21:03.673381 21019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35602162000346e3bbb790f54467ed0d" candidate_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ec8fff5744984f4695d768676cae8d17"
I20260709 13:21:03.673987 21312 raft_consensus.cc:3060] T 35602162000346e3bbb790f54467ed0d P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:03.674034 21019 raft_consensus.cc:3055] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:03.674642 21019 raft_consensus.cc:740] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ec8fff5744984f4695d768676cae8d17, State: Running, Role: LEADER
I20260709 13:21:03.675521 21019 consensus_queue.cc:260] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:03.676570 21019 raft_consensus.cc:3060] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:03.682116 21019 raft_consensus.cc:2468] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd6a13ebcb53485a9cc5082a4f64d652 in term 2.
I20260709 13:21:03.682610 21312 raft_consensus.cc:2468] T 35602162000346e3bbb790f54467ed0d P 71bdb236d2304c75b7eabfec8f3d5a46 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd6a13ebcb53485a9cc5082a4f64d652 in term 2.
I20260709 13:21:03.683687 21061 leader_election.cc:304] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd6a13ebcb53485a9cc5082a4f64d652, ec8fff5744984f4695d768676cae8d17; no voters: 
I20260709 13:21:03.684615 21261 raft_consensus.cc:2804] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:03.685070 21261 raft_consensus.cc:697] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 LEADER]: Becoming Leader. State: Replica: bd6a13ebcb53485a9cc5082a4f64d652, State: Running, Role: LEADER
I20260709 13:21:03.685943 21261 consensus_queue.cc:237] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:03.695274 20844 catalog_manager.cc:5697] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652 reported cstate change: term changed from 1 to 2, leader changed from ec8fff5744984f4695d768676cae8d17 (127.9.116.130) to bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131). New cstate: current_term: 2 leader_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:03.797253 21379 raft_consensus.cc:993] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652: : Instructing follower ec8fff5744984f4695d768676cae8d17 to start an election
I20260709 13:21:03.798629 21365 raft_consensus.cc:1081] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 LEADER]: Signalling peer ec8fff5744984f4695d768676cae8d17 to start an election
I20260709 13:21:03.800371 21019 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2725109df1524e6c8eb897f0e6424829"
dest_uuid: "ec8fff5744984f4695d768676cae8d17"
 from {username='slave'} at 127.0.0.1:54288
I20260709 13:21:03.801013 21019 raft_consensus.cc:493] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:03.801399 21019 raft_consensus.cc:3060] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:03.810313 21019 raft_consensus.cc:515] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:03.812664 21019 leader_election.cc:290] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 3 election: Requested vote from peers 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473), bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131:37203)
I20260709 13:21:03.813278 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725109df1524e6c8eb897f0e6424829" candidate_uuid: "ec8fff5744984f4695d768676cae8d17" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "71bdb236d2304c75b7eabfec8f3d5a46"
I20260709 13:21:03.813979 21094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725109df1524e6c8eb897f0e6424829" candidate_uuid: "ec8fff5744984f4695d768676cae8d17" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652"
I20260709 13:21:03.814275 21312 raft_consensus.cc:3060] T 2725109df1524e6c8eb897f0e6424829 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:03.814836 21094 raft_consensus.cc:3055] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:21:03.815207 21094 raft_consensus.cc:740] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bd6a13ebcb53485a9cc5082a4f64d652, State: Running, Role: LEADER
I20260709 13:21:03.816140 21094 consensus_queue.cc:260] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:03.817466 21094 raft_consensus.cc:3060] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:03.819973 21312 raft_consensus.cc:2468] T 2725109df1524e6c8eb897f0e6424829 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec8fff5744984f4695d768676cae8d17 in term 3.
I20260709 13:21:03.821195 20985 leader_election.cc:304] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71bdb236d2304c75b7eabfec8f3d5a46, ec8fff5744984f4695d768676cae8d17; no voters: 
I20260709 13:21:03.821897 21368 raft_consensus.cc:2804] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:21:03.822341 21368 raft_consensus.cc:697] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [term 3 LEADER]: Becoming Leader. State: Replica: ec8fff5744984f4695d768676cae8d17, State: Running, Role: LEADER
I20260709 13:21:03.823478 21368 consensus_queue.cc:237] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:03.825314 21094 raft_consensus.cc:2468] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec8fff5744984f4695d768676cae8d17 in term 3.
I20260709 13:21:03.830717 20844 catalog_manager.cc:5697] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 reported cstate change: term changed from 2 to 3, leader changed from bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131) to ec8fff5744984f4695d768676cae8d17 (127.9.116.130). New cstate: current_term: 3 leader_uuid: "ec8fff5744984f4695d768676cae8d17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:03.859408 21258 raft_consensus.cc:993] T b541a9ed669a420d8fbeb6e10ab3c47f P ec8fff5744984f4695d768676cae8d17: : Instructing follower bd6a13ebcb53485a9cc5082a4f64d652 to start an election
I20260709 13:21:03.859917 21368 raft_consensus.cc:1081] T b541a9ed669a420d8fbeb6e10ab3c47f P ec8fff5744984f4695d768676cae8d17 [term 3 LEADER]: Signalling peer bd6a13ebcb53485a9cc5082a4f64d652 to start an election
I20260709 13:21:03.861476 21094 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b541a9ed669a420d8fbeb6e10ab3c47f"
dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652"
 from {username='slave'} at 127.0.0.1:52450
I20260709 13:21:03.861959 21094 raft_consensus.cc:493] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:03.862249 21094 raft_consensus.cc:3060] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:21:03.867048 21094 raft_consensus.cc:515] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } }
I20260709 13:21:03.868603 21094 leader_election.cc:290] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 4 election: Requested vote from peers ec8fff5744984f4695d768676cae8d17 (127.9.116.130:42659), 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473)
I20260709 13:21:03.869441 21019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b541a9ed669a420d8fbeb6e10ab3c47f" candidate_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "ec8fff5744984f4695d768676cae8d17"
I20260709 13:21:03.869824 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b541a9ed669a420d8fbeb6e10ab3c47f" candidate_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "71bdb236d2304c75b7eabfec8f3d5a46"
I20260709 13:21:03.870199 21019 raft_consensus.cc:3055] T b541a9ed669a420d8fbeb6e10ab3c47f P ec8fff5744984f4695d768676cae8d17 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:21:03.870466 21312 raft_consensus.cc:3060] T b541a9ed669a420d8fbeb6e10ab3c47f P 71bdb236d2304c75b7eabfec8f3d5a46 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:21:03.870622 21019 raft_consensus.cc:740] T b541a9ed669a420d8fbeb6e10ab3c47f P ec8fff5744984f4695d768676cae8d17 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ec8fff5744984f4695d768676cae8d17, State: Running, Role: LEADER
I20260709 13:21:03.872417 21019 consensus_queue.cc:260] T b541a9ed669a420d8fbeb6e10ab3c47f P ec8fff5744984f4695d768676cae8d17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } }
I20260709 13:21:03.873898 21019 raft_consensus.cc:3060] T b541a9ed669a420d8fbeb6e10ab3c47f P ec8fff5744984f4695d768676cae8d17 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:21:03.876910 21312 raft_consensus.cc:2468] T b541a9ed669a420d8fbeb6e10ab3c47f P 71bdb236d2304c75b7eabfec8f3d5a46 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd6a13ebcb53485a9cc5082a4f64d652 in term 4.
I20260709 13:21:03.878082 21059 leader_election.cc:304] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71bdb236d2304c75b7eabfec8f3d5a46, bd6a13ebcb53485a9cc5082a4f64d652; no voters: 
I20260709 13:21:03.878837 21365 raft_consensus.cc:2804] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:21:03.879303 21365 raft_consensus.cc:697] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 [term 4 LEADER]: Becoming Leader. State: Replica: bd6a13ebcb53485a9cc5082a4f64d652, State: Running, Role: LEADER
I20260709 13:21:03.880119 21365 consensus_queue.cc:237] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } }
I20260709 13:21:03.882146 21019 raft_consensus.cc:2468] T b541a9ed669a420d8fbeb6e10ab3c47f P ec8fff5744984f4695d768676cae8d17 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd6a13ebcb53485a9cc5082a4f64d652 in term 4.
I20260709 13:21:03.887537 20844 catalog_manager.cc:5697] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 reported cstate change: term changed from 3 to 4, leader changed from ec8fff5744984f4695d768676cae8d17 (127.9.116.130) to bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131). New cstate: current_term: 4 leader_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:04.115442 21019 raft_consensus.cc:1275] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [term 2 FOLLOWER]: Refusing update from remote peer bd6a13ebcb53485a9cc5082a4f64d652: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:04.116865 21365 consensus_queue.cc:1048] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:21:04.121706 20892 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:21:04.130023 21312 raft_consensus.cc:1275] T 35602162000346e3bbb790f54467ed0d P 71bdb236d2304c75b7eabfec8f3d5a46 [term 2 FOLLOWER]: Refusing update from remote peer bd6a13ebcb53485a9cc5082a4f64d652: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:04.132123 21379 consensus_queue.cc:1048] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:04.279039 21312 raft_consensus.cc:1275] T 2725109df1524e6c8eb897f0e6424829 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 3 FOLLOWER]: Refusing update from remote peer ec8fff5744984f4695d768676cae8d17: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:21:04.280381 21368 consensus_queue.cc:1048] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:04.289989 21094 raft_consensus.cc:1275] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [term 3 FOLLOWER]: Refusing update from remote peer ec8fff5744984f4695d768676cae8d17: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:21:04.291628 21368 consensus_queue.cc:1048] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:21:04.343977 21312 raft_consensus.cc:1275] T b541a9ed669a420d8fbeb6e10ab3c47f P 71bdb236d2304c75b7eabfec8f3d5a46 [term 4 FOLLOWER]: Refusing update from remote peer bd6a13ebcb53485a9cc5082a4f64d652: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:21:04.345436 21379 consensus_queue.cc:1048] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:21:04.356350 21019 raft_consensus.cc:1275] T b541a9ed669a420d8fbeb6e10ab3c47f P ec8fff5744984f4695d768676cae8d17 [term 4 FOLLOWER]: Refusing update from remote peer bd6a13ebcb53485a9cc5082a4f64d652: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:21:04.358021 21379 consensus_queue.cc:1048] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:21:04.617769  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:21:04.625475 21021 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2725109df1524e6c8eb897f0e6424829"
dest_uuid: "ec8fff5744984f4695d768676cae8d17"
mode: GRACEFUL
new_leader_uuid: "bd6a13ebcb53485a9cc5082a4f64d652"
 from {username='slave'} at 127.0.0.1:36740
I20260709 13:21:04.626075 21021 raft_consensus.cc:606] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [term 3 LEADER]: Received request to transfer leadership to TS bd6a13ebcb53485a9cc5082a4f64d652
I20260709 13:21:04.628641 21094 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "35602162000346e3bbb790f54467ed0d"
dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652"
mode: GRACEFUL
new_leader_uuid: "ec8fff5744984f4695d768676cae8d17"
 from {username='slave'} at 127.0.0.1:42722
I20260709 13:21:04.629138 21094 raft_consensus.cc:606] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 LEADER]: Received request to transfer leadership to TS ec8fff5744984f4695d768676cae8d17
I20260709 13:21:04.631438 21094 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "35d6d5be27cb4096917b886d06a3cf58"
dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652"
mode: GRACEFUL
new_leader_uuid: "ec8fff5744984f4695d768676cae8d17"
 from {username='slave'} at 127.0.0.1:42722
I20260709 13:21:04.631879 21094 raft_consensus.cc:606] T 35d6d5be27cb4096917b886d06a3cf58 P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 LEADER]: Received request to transfer leadership to TS ec8fff5744984f4695d768676cae8d17
I20260709 13:21:04.632833  9682 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:21:04.633337  9682 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:21:04.633903  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:21:04.905607 21258 raft_consensus.cc:993] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17: : Instructing follower bd6a13ebcb53485a9cc5082a4f64d652 to start an election
I20260709 13:21:04.906266 21391 raft_consensus.cc:1081] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [term 3 LEADER]: Signalling peer bd6a13ebcb53485a9cc5082a4f64d652 to start an election
I20260709 13:21:04.907874 21094 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2725109df1524e6c8eb897f0e6424829"
dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652"
 from {username='slave'} at 127.0.0.1:52450
I20260709 13:21:04.908396 21094 raft_consensus.cc:493] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:04.908788 21094 raft_consensus.cc:3060] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:21:04.914503 21094 raft_consensus.cc:515] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:04.916519 21094 leader_election.cc:290] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 4 election: Requested vote from peers 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473), ec8fff5744984f4695d768676cae8d17 (127.9.116.130:42659)
I20260709 13:21:04.917258 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725109df1524e6c8eb897f0e6424829" candidate_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "71bdb236d2304c75b7eabfec8f3d5a46"
I20260709 13:21:04.917850 21312 raft_consensus.cc:3060] T 2725109df1524e6c8eb897f0e6424829 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:21:04.918416 21021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725109df1524e6c8eb897f0e6424829" candidate_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "ec8fff5744984f4695d768676cae8d17"
I20260709 13:21:04.919145 21021 raft_consensus.cc:3055] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:21:04.919492 21021 raft_consensus.cc:740] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ec8fff5744984f4695d768676cae8d17, State: Running, Role: LEADER
I20260709 13:21:04.920246 21021 consensus_queue.cc:260] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:04.921365 21021 raft_consensus.cc:3060] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:21:04.923745 21312 raft_consensus.cc:2468] T 2725109df1524e6c8eb897f0e6424829 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd6a13ebcb53485a9cc5082a4f64d652 in term 4.
I20260709 13:21:04.925019 21059 leader_election.cc:304] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71bdb236d2304c75b7eabfec8f3d5a46, bd6a13ebcb53485a9cc5082a4f64d652; no voters: 
I20260709 13:21:04.925969 21379 raft_consensus.cc:2804] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:21:04.926518 21379 raft_consensus.cc:697] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [term 4 LEADER]: Becoming Leader. State: Replica: bd6a13ebcb53485a9cc5082a4f64d652, State: Running, Role: LEADER
I20260709 13:21:04.927371 21379 consensus_queue.cc:237] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:04.930030 21021 raft_consensus.cc:2468] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd6a13ebcb53485a9cc5082a4f64d652 in term 4.
I20260709 13:21:04.934679 20843 catalog_manager.cc:5697] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 reported cstate change: term changed from 3 to 4, leader changed from ec8fff5744984f4695d768676cae8d17 (127.9.116.130) to bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131). New cstate: current_term: 4 leader_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:04.947180 21365 raft_consensus.cc:993] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652: : Instructing follower ec8fff5744984f4695d768676cae8d17 to start an election
I20260709 13:21:04.947649 21379 raft_consensus.cc:1081] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 LEADER]: Signalling peer ec8fff5744984f4695d768676cae8d17 to start an election
I20260709 13:21:04.949137 21021 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "35602162000346e3bbb790f54467ed0d"
dest_uuid: "ec8fff5744984f4695d768676cae8d17"
 from {username='slave'} at 127.0.0.1:54288
I20260709 13:21:04.949661 21021 raft_consensus.cc:493] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:04.950035 21021 raft_consensus.cc:3060] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:04.955952 21021 raft_consensus.cc:515] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:04.957655 21021 leader_election.cc:290] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 3 election: Requested vote from peers 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473), bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131:37203)
I20260709 13:21:04.958657 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35602162000346e3bbb790f54467ed0d" candidate_uuid: "ec8fff5744984f4695d768676cae8d17" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "71bdb236d2304c75b7eabfec8f3d5a46"
I20260709 13:21:04.958726 21094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35602162000346e3bbb790f54467ed0d" candidate_uuid: "ec8fff5744984f4695d768676cae8d17" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652"
I20260709 13:21:04.959403 21312 raft_consensus.cc:3060] T 35602162000346e3bbb790f54467ed0d P 71bdb236d2304c75b7eabfec8f3d5a46 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:04.959472 21094 raft_consensus.cc:3055] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:21:04.959903 21094 raft_consensus.cc:740] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bd6a13ebcb53485a9cc5082a4f64d652, State: Running, Role: LEADER
I20260709 13:21:04.960516 21094 consensus_queue.cc:260] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:04.961404 21094 raft_consensus.cc:3060] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:04.965139 21312 raft_consensus.cc:2468] T 35602162000346e3bbb790f54467ed0d P 71bdb236d2304c75b7eabfec8f3d5a46 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec8fff5744984f4695d768676cae8d17 in term 3.
I20260709 13:21:04.966336 20985 leader_election.cc:304] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71bdb236d2304c75b7eabfec8f3d5a46, ec8fff5744984f4695d768676cae8d17; no voters: 
I20260709 13:21:04.966712 21094 raft_consensus.cc:2468] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec8fff5744984f4695d768676cae8d17 in term 3.
I20260709 13:21:04.967085 21391 raft_consensus.cc:2804] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:21:04.967638 21391 raft_consensus.cc:697] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [term 3 LEADER]: Becoming Leader. State: Replica: ec8fff5744984f4695d768676cae8d17, State: Running, Role: LEADER
I20260709 13:21:04.968520 21391 consensus_queue.cc:237] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:04.975019 20843 catalog_manager.cc:5697] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 reported cstate change: term changed from 2 to 3, leader changed from bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131) to ec8fff5744984f4695d768676cae8d17 (127.9.116.130). New cstate: current_term: 3 leader_uuid: "ec8fff5744984f4695d768676cae8d17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:04.989948 21365 raft_consensus.cc:993] T 35d6d5be27cb4096917b886d06a3cf58 P bd6a13ebcb53485a9cc5082a4f64d652: : Instructing follower ec8fff5744984f4695d768676cae8d17 to start an election
I20260709 13:21:04.990428 21379 raft_consensus.cc:1081] T 35d6d5be27cb4096917b886d06a3cf58 P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 LEADER]: Signalling peer ec8fff5744984f4695d768676cae8d17 to start an election
I20260709 13:21:04.991951 21021 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "35d6d5be27cb4096917b886d06a3cf58"
dest_uuid: "ec8fff5744984f4695d768676cae8d17"
 from {username='slave'} at 127.0.0.1:54288
I20260709 13:21:04.992484 21021 raft_consensus.cc:493] T 35d6d5be27cb4096917b886d06a3cf58 P ec8fff5744984f4695d768676cae8d17 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:04.992812 21021 raft_consensus.cc:3060] T 35d6d5be27cb4096917b886d06a3cf58 P ec8fff5744984f4695d768676cae8d17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:04.998342 21021 raft_consensus.cc:515] T 35d6d5be27cb4096917b886d06a3cf58 P ec8fff5744984f4695d768676cae8d17 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:05.000018 21021 leader_election.cc:290] T 35d6d5be27cb4096917b886d06a3cf58 P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 3 election: Requested vote from peers 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473), bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131:37203)
I20260709 13:21:05.000883 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35d6d5be27cb4096917b886d06a3cf58" candidate_uuid: "ec8fff5744984f4695d768676cae8d17" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "71bdb236d2304c75b7eabfec8f3d5a46"
I20260709 13:21:05.001029 21094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35d6d5be27cb4096917b886d06a3cf58" candidate_uuid: "ec8fff5744984f4695d768676cae8d17" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652"
I20260709 13:21:05.001590 21312 raft_consensus.cc:3060] T 35d6d5be27cb4096917b886d06a3cf58 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:05.001681 21094 raft_consensus.cc:3055] T 35d6d5be27cb4096917b886d06a3cf58 P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:21:05.002274 21094 raft_consensus.cc:740] T 35d6d5be27cb4096917b886d06a3cf58 P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bd6a13ebcb53485a9cc5082a4f64d652, State: Running, Role: LEADER
I20260709 13:21:05.003125 21094 consensus_queue.cc:260] T 35d6d5be27cb4096917b886d06a3cf58 P bd6a13ebcb53485a9cc5082a4f64d652 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:05.004124 21094 raft_consensus.cc:3060] T 35d6d5be27cb4096917b886d06a3cf58 P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:05.008153 21312 raft_consensus.cc:2468] T 35d6d5be27cb4096917b886d06a3cf58 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec8fff5744984f4695d768676cae8d17 in term 3.
I20260709 13:21:05.009191 21094 raft_consensus.cc:2468] T 35d6d5be27cb4096917b886d06a3cf58 P bd6a13ebcb53485a9cc5082a4f64d652 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec8fff5744984f4695d768676cae8d17 in term 3.
I20260709 13:21:05.009456 20985 leader_election.cc:304] T 35d6d5be27cb4096917b886d06a3cf58 P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71bdb236d2304c75b7eabfec8f3d5a46, ec8fff5744984f4695d768676cae8d17; no voters: 
I20260709 13:21:05.010388 21391 raft_consensus.cc:2804] T 35d6d5be27cb4096917b886d06a3cf58 P ec8fff5744984f4695d768676cae8d17 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:21:05.010885 21391 raft_consensus.cc:697] T 35d6d5be27cb4096917b886d06a3cf58 P ec8fff5744984f4695d768676cae8d17 [term 3 LEADER]: Becoming Leader. State: Replica: ec8fff5744984f4695d768676cae8d17, State: Running, Role: LEADER
I20260709 13:21:05.011796 21391 consensus_queue.cc:237] T 35d6d5be27cb4096917b886d06a3cf58 P ec8fff5744984f4695d768676cae8d17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:05.019281 20843 catalog_manager.cc:5697] T 35d6d5be27cb4096917b886d06a3cf58 P ec8fff5744984f4695d768676cae8d17 reported cstate change: term changed from 2 to 3, leader changed from bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131) to ec8fff5744984f4695d768676cae8d17 (127.9.116.130). New cstate: current_term: 3 leader_uuid: "ec8fff5744984f4695d768676cae8d17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } health_report { overall_health: UNKNOWN } } }
W20260709 13:21:05.122886 20892 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:21:05.329677 21312 raft_consensus.cc:1275] T 2725109df1524e6c8eb897f0e6424829 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 4 FOLLOWER]: Refusing update from remote peer bd6a13ebcb53485a9cc5082a4f64d652: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:21:05.331200 21379 consensus_queue.cc:1048] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:21:05.341881 21021 raft_consensus.cc:1275] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [term 4 FOLLOWER]: Refusing update from remote peer bd6a13ebcb53485a9cc5082a4f64d652: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:21:05.343415 21365 consensus_queue.cc:1048] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:21:05.390899 21094 raft_consensus.cc:1275] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652 [term 3 FOLLOWER]: Refusing update from remote peer ec8fff5744984f4695d768676cae8d17: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:21:05.392069 21391 consensus_queue.cc:1048] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:21:05.402320 21312 raft_consensus.cc:1275] T 35602162000346e3bbb790f54467ed0d P 71bdb236d2304c75b7eabfec8f3d5a46 [term 3 FOLLOWER]: Refusing update from remote peer ec8fff5744984f4695d768676cae8d17: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:21:05.404772 21258 consensus_queue.cc:1048] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:21:05.450444 21094 raft_consensus.cc:1275] T 35d6d5be27cb4096917b886d06a3cf58 P bd6a13ebcb53485a9cc5082a4f64d652 [term 3 FOLLOWER]: Refusing update from remote peer ec8fff5744984f4695d768676cae8d17: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:21:05.451798 21258 consensus_queue.cc:1048] T 35d6d5be27cb4096917b886d06a3cf58 P ec8fff5744984f4695d768676cae8d17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:21:05.460608 21312 raft_consensus.cc:1275] T 35d6d5be27cb4096917b886d06a3cf58 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 3 FOLLOWER]: Refusing update from remote peer ec8fff5744984f4695d768676cae8d17: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:21:05.462474 21391 consensus_queue.cc:1048] T 35d6d5be27cb4096917b886d06a3cf58 P ec8fff5744984f4695d768676cae8d17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:21:05.640240  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:21:05.647611 21094 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2725109df1524e6c8eb897f0e6424829"
dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652"
mode: GRACEFUL
new_leader_uuid: "ec8fff5744984f4695d768676cae8d17"
 from {username='slave'} at 127.0.0.1:42722
I20260709 13:21:05.648101 21094 raft_consensus.cc:606] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [term 4 LEADER]: Received request to transfer leadership to TS ec8fff5744984f4695d768676cae8d17
I20260709 13:21:05.650480 21019 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "35602162000346e3bbb790f54467ed0d"
dest_uuid: "ec8fff5744984f4695d768676cae8d17"
mode: GRACEFUL
new_leader_uuid: "bd6a13ebcb53485a9cc5082a4f64d652"
 from {username='slave'} at 127.0.0.1:36740
I20260709 13:21:05.650985 21019 raft_consensus.cc:606] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [term 3 LEADER]: Received request to transfer leadership to TS bd6a13ebcb53485a9cc5082a4f64d652
I20260709 13:21:05.653326 21019 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c3add26e31364c95a9897ab53b516735"
dest_uuid: "ec8fff5744984f4695d768676cae8d17"
mode: GRACEFUL
new_leader_uuid: "bd6a13ebcb53485a9cc5082a4f64d652"
 from {username='slave'} at 127.0.0.1:36740
I20260709 13:21:05.653728 21019 raft_consensus.cc:606] T c3add26e31364c95a9897ab53b516735 P ec8fff5744984f4695d768676cae8d17 [term 2 LEADER]: Received request to transfer leadership to TS bd6a13ebcb53485a9cc5082a4f64d652
I20260709 13:21:05.654829  9682 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:21:05.655380  9682 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:21:05.656023  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:21:05.674700 21390 raft_consensus.cc:993] T c3add26e31364c95a9897ab53b516735 P ec8fff5744984f4695d768676cae8d17: : Instructing follower bd6a13ebcb53485a9cc5082a4f64d652 to start an election
I20260709 13:21:05.675300 21390 raft_consensus.cc:1081] T c3add26e31364c95a9897ab53b516735 P ec8fff5744984f4695d768676cae8d17 [term 2 LEADER]: Signalling peer bd6a13ebcb53485a9cc5082a4f64d652 to start an election
I20260709 13:21:05.676745 21094 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c3add26e31364c95a9897ab53b516735"
dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652"
 from {username='slave'} at 127.0.0.1:52450
I20260709 13:21:05.677207 21094 raft_consensus.cc:493] T c3add26e31364c95a9897ab53b516735 P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:05.677484 21094 raft_consensus.cc:3060] T c3add26e31364c95a9897ab53b516735 P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:05.682143 21094 raft_consensus.cc:515] T c3add26e31364c95a9897ab53b516735 P bd6a13ebcb53485a9cc5082a4f64d652 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:05.683641 21094 leader_election.cc:290] T c3add26e31364c95a9897ab53b516735 P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 3 election: Requested vote from peers 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473), ec8fff5744984f4695d768676cae8d17 (127.9.116.130:42659)
I20260709 13:21:05.684619 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3add26e31364c95a9897ab53b516735" candidate_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "71bdb236d2304c75b7eabfec8f3d5a46"
I20260709 13:21:05.684756 21019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3add26e31364c95a9897ab53b516735" candidate_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ec8fff5744984f4695d768676cae8d17"
I20260709 13:21:05.685293 21312 raft_consensus.cc:3060] T c3add26e31364c95a9897ab53b516735 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:05.685371 21019 raft_consensus.cc:3055] T c3add26e31364c95a9897ab53b516735 P ec8fff5744984f4695d768676cae8d17 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:21:05.685796 21019 raft_consensus.cc:740] T c3add26e31364c95a9897ab53b516735 P ec8fff5744984f4695d768676cae8d17 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ec8fff5744984f4695d768676cae8d17, State: Running, Role: LEADER
I20260709 13:21:05.686525 21019 consensus_queue.cc:260] T c3add26e31364c95a9897ab53b516735 P ec8fff5744984f4695d768676cae8d17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:05.687698 21019 raft_consensus.cc:3060] T c3add26e31364c95a9897ab53b516735 P ec8fff5744984f4695d768676cae8d17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:05.691691 21312 raft_consensus.cc:2468] T c3add26e31364c95a9897ab53b516735 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd6a13ebcb53485a9cc5082a4f64d652 in term 3.
I20260709 13:21:05.692379 21019 raft_consensus.cc:2468] T c3add26e31364c95a9897ab53b516735 P ec8fff5744984f4695d768676cae8d17 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd6a13ebcb53485a9cc5082a4f64d652 in term 3.
I20260709 13:21:05.693005 21059 leader_election.cc:304] T c3add26e31364c95a9897ab53b516735 P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71bdb236d2304c75b7eabfec8f3d5a46, bd6a13ebcb53485a9cc5082a4f64d652; no voters: 
I20260709 13:21:05.693907 21365 raft_consensus.cc:2804] T c3add26e31364c95a9897ab53b516735 P bd6a13ebcb53485a9cc5082a4f64d652 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:21:05.694412 21365 raft_consensus.cc:697] T c3add26e31364c95a9897ab53b516735 P bd6a13ebcb53485a9cc5082a4f64d652 [term 3 LEADER]: Becoming Leader. State: Replica: bd6a13ebcb53485a9cc5082a4f64d652, State: Running, Role: LEADER
I20260709 13:21:05.695464 21365 consensus_queue.cc:237] T c3add26e31364c95a9897ab53b516735 P bd6a13ebcb53485a9cc5082a4f64d652 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:05.702884 20842 catalog_manager.cc:5697] T c3add26e31364c95a9897ab53b516735 P bd6a13ebcb53485a9cc5082a4f64d652 reported cstate change: term changed from 2 to 3, leader changed from ec8fff5744984f4695d768676cae8d17 (127.9.116.130) to bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131). New cstate: current_term: 3 leader_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:05.782969 21391 raft_consensus.cc:993] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17: : Instructing follower bd6a13ebcb53485a9cc5082a4f64d652 to start an election
I20260709 13:21:05.783510 21390 raft_consensus.cc:1081] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [term 3 LEADER]: Signalling peer bd6a13ebcb53485a9cc5082a4f64d652 to start an election
I20260709 13:21:05.784873 21094 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "35602162000346e3bbb790f54467ed0d"
dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652"
 from {username='slave'} at 127.0.0.1:52450
I20260709 13:21:05.785315 21094 raft_consensus.cc:493] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:05.785544 21094 raft_consensus.cc:3060] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:21:05.790192 21094 raft_consensus.cc:515] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:05.791846 21094 leader_election.cc:290] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 4 election: Requested vote from peers 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473), ec8fff5744984f4695d768676cae8d17 (127.9.116.130:42659)
I20260709 13:21:05.792775 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35602162000346e3bbb790f54467ed0d" candidate_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "71bdb236d2304c75b7eabfec8f3d5a46"
I20260709 13:21:05.792901 21019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35602162000346e3bbb790f54467ed0d" candidate_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "ec8fff5744984f4695d768676cae8d17"
I20260709 13:21:05.793603 21312 raft_consensus.cc:3060] T 35602162000346e3bbb790f54467ed0d P 71bdb236d2304c75b7eabfec8f3d5a46 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:21:05.793617 21019 raft_consensus.cc:3055] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:21:05.794165 21019 raft_consensus.cc:740] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ec8fff5744984f4695d768676cae8d17, State: Running, Role: LEADER
I20260709 13:21:05.794976 21019 consensus_queue.cc:260] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:05.796382 21019 raft_consensus.cc:3060] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:21:05.801064 21379 raft_consensus.cc:993] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652: : Instructing follower ec8fff5744984f4695d768676cae8d17 to start an election
I20260709 13:21:05.801194 21312 raft_consensus.cc:2468] T 35602162000346e3bbb790f54467ed0d P 71bdb236d2304c75b7eabfec8f3d5a46 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd6a13ebcb53485a9cc5082a4f64d652 in term 4.
I20260709 13:21:05.801713 21365 raft_consensus.cc:1081] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [term 4 LEADER]: Signalling peer ec8fff5744984f4695d768676cae8d17 to start an election
I20260709 13:21:05.802502 21059 leader_election.cc:304] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71bdb236d2304c75b7eabfec8f3d5a46, bd6a13ebcb53485a9cc5082a4f64d652; no voters: 
I20260709 13:21:05.803269 21365 raft_consensus.cc:2804] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:21:05.803462 21021 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2725109df1524e6c8eb897f0e6424829"
dest_uuid: "ec8fff5744984f4695d768676cae8d17"
 from {username='slave'} at 127.0.0.1:54288
I20260709 13:21:05.803800 21365 raft_consensus.cc:697] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652 [term 4 LEADER]: Becoming Leader. State: Replica: bd6a13ebcb53485a9cc5082a4f64d652, State: Running, Role: LEADER
I20260709 13:21:05.804152 21021 raft_consensus.cc:493] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:05.804153 21019 raft_consensus.cc:2468] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd6a13ebcb53485a9cc5082a4f64d652 in term 4.
I20260709 13:21:05.804620 21021 raft_consensus.cc:3060] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:21:05.804749 21365 consensus_queue.cc:237] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:05.810042 21021 raft_consensus.cc:515] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:05.812075 21021 leader_election.cc:290] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 5 election: Requested vote from peers 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473), bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131:37203)
I20260709 13:21:05.811659 20842 catalog_manager.cc:5697] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652 reported cstate change: term changed from 3 to 4, leader changed from ec8fff5744984f4695d768676cae8d17 (127.9.116.130) to bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131). New cstate: current_term: 4 leader_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:05.812898 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725109df1524e6c8eb897f0e6424829" candidate_uuid: "ec8fff5744984f4695d768676cae8d17" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "71bdb236d2304c75b7eabfec8f3d5a46"
I20260709 13:21:05.813532 21312 raft_consensus.cc:3060] T 2725109df1524e6c8eb897f0e6424829 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:21:05.813485 21094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2725109df1524e6c8eb897f0e6424829" candidate_uuid: "ec8fff5744984f4695d768676cae8d17" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652"
I20260709 13:21:05.814528 21094 raft_consensus.cc:3055] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:21:05.814890 21094 raft_consensus.cc:740] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: bd6a13ebcb53485a9cc5082a4f64d652, State: Running, Role: LEADER
I20260709 13:21:05.815716 21094 consensus_queue.cc:260] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:05.816896 21094 raft_consensus.cc:3060] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:21:05.821602 21312 raft_consensus.cc:2468] T 2725109df1524e6c8eb897f0e6424829 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec8fff5744984f4695d768676cae8d17 in term 5.
I20260709 13:21:05.822985 20985 leader_election.cc:304] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71bdb236d2304c75b7eabfec8f3d5a46, ec8fff5744984f4695d768676cae8d17; no voters: 
I20260709 13:21:05.823822 21390 raft_consensus.cc:2804] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:21:05.824045 21094 raft_consensus.cc:2468] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec8fff5744984f4695d768676cae8d17 in term 5.
I20260709 13:21:05.824345 21390 raft_consensus.cc:697] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [term 5 LEADER]: Becoming Leader. State: Replica: ec8fff5744984f4695d768676cae8d17, State: Running, Role: LEADER
I20260709 13:21:05.825172 21390 consensus_queue.cc:237] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:05.831413 20842 catalog_manager.cc:5697] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 reported cstate change: term changed from 4 to 5, leader changed from bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131) to ec8fff5744984f4695d768676cae8d17 (127.9.116.130). New cstate: current_term: 5 leader_uuid: "ec8fff5744984f4695d768676cae8d17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } health_report { overall_health: UNKNOWN } } }
W20260709 13:21:06.124042 20892 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:21:06.137570 21021 raft_consensus.cc:1275] T c3add26e31364c95a9897ab53b516735 P ec8fff5744984f4695d768676cae8d17 [term 3 FOLLOWER]: Refusing update from remote peer bd6a13ebcb53485a9cc5082a4f64d652: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:21:06.138991 21365 consensus_queue.cc:1048] T c3add26e31364c95a9897ab53b516735 P bd6a13ebcb53485a9cc5082a4f64d652 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:21:06.149569 21312 raft_consensus.cc:1275] T c3add26e31364c95a9897ab53b516735 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 3 FOLLOWER]: Refusing update from remote peer bd6a13ebcb53485a9cc5082a4f64d652: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:21:06.154937 21353 consensus_queue.cc:1048] T c3add26e31364c95a9897ab53b516735 P bd6a13ebcb53485a9cc5082a4f64d652 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:21:06.218720 21312 raft_consensus.cc:1275] T 2725109df1524e6c8eb897f0e6424829 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 5 FOLLOWER]: Refusing update from remote peer ec8fff5744984f4695d768676cae8d17: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:21:06.220031 21390 consensus_queue.cc:1048] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:21:06.227833 21094 raft_consensus.cc:1275] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [term 5 FOLLOWER]: Refusing update from remote peer ec8fff5744984f4695d768676cae8d17: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:21:06.229570 21258 consensus_queue.cc:1048] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:21:06.332127 21021 raft_consensus.cc:1275] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [term 4 FOLLOWER]: Refusing update from remote peer bd6a13ebcb53485a9cc5082a4f64d652: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:21:06.333531 21353 consensus_queue.cc:1048] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:21:06.347121 21312 raft_consensus.cc:1275] T 35602162000346e3bbb790f54467ed0d P 71bdb236d2304c75b7eabfec8f3d5a46 [term 4 FOLLOWER]: Refusing update from remote peer bd6a13ebcb53485a9cc5082a4f64d652: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:21:06.348587 21417 consensus_queue.cc:1048] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:21:06.663269  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:21:06.670542 21094 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7f3178c915104a27b9a3b6978249c5ed"
dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652"
mode: GRACEFUL
new_leader_uuid: "ec8fff5744984f4695d768676cae8d17"
 from {username='slave'} at 127.0.0.1:42722
I20260709 13:21:06.671106 21094 raft_consensus.cc:606] T 7f3178c915104a27b9a3b6978249c5ed P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 LEADER]: Received request to transfer leadership to TS ec8fff5744984f4695d768676cae8d17
I20260709 13:21:06.673804 21021 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c50ac60e07eb46dc8836de50bce12760"
dest_uuid: "ec8fff5744984f4695d768676cae8d17"
mode: GRACEFUL
new_leader_uuid: "bd6a13ebcb53485a9cc5082a4f64d652"
 from {username='slave'} at 127.0.0.1:36740
I20260709 13:21:06.674306 21021 raft_consensus.cc:606] T c50ac60e07eb46dc8836de50bce12760 P ec8fff5744984f4695d768676cae8d17 [term 2 LEADER]: Received request to transfer leadership to TS bd6a13ebcb53485a9cc5082a4f64d652
I20260709 13:21:06.676787 21094 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ee997af6d38745549698356bcbc09a99"
dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652"
mode: GRACEFUL
new_leader_uuid: "ec8fff5744984f4695d768676cae8d17"
 from {username='slave'} at 127.0.0.1:42722
I20260709 13:21:06.677256 21094 raft_consensus.cc:606] T ee997af6d38745549698356bcbc09a99 P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 LEADER]: Received request to transfer leadership to TS ec8fff5744984f4695d768676cae8d17
I20260709 13:21:06.678366  9682 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:21:06.678992  9682 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:21:06.679602  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:21:06.793383 21353 raft_consensus.cc:993] T ee997af6d38745549698356bcbc09a99 P bd6a13ebcb53485a9cc5082a4f64d652: : Instructing follower ec8fff5744984f4695d768676cae8d17 to start an election
I20260709 13:21:06.793911 21417 raft_consensus.cc:1081] T ee997af6d38745549698356bcbc09a99 P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 LEADER]: Signalling peer ec8fff5744984f4695d768676cae8d17 to start an election
I20260709 13:21:06.795764 21021 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ee997af6d38745549698356bcbc09a99"
dest_uuid: "ec8fff5744984f4695d768676cae8d17"
 from {username='slave'} at 127.0.0.1:54288
I20260709 13:21:06.796363 21021 raft_consensus.cc:493] T ee997af6d38745549698356bcbc09a99 P ec8fff5744984f4695d768676cae8d17 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:06.796763 21021 raft_consensus.cc:3060] T ee997af6d38745549698356bcbc09a99 P ec8fff5744984f4695d768676cae8d17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:06.802107 21021 raft_consensus.cc:515] T ee997af6d38745549698356bcbc09a99 P ec8fff5744984f4695d768676cae8d17 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:06.803819 21021 leader_election.cc:290] T ee997af6d38745549698356bcbc09a99 P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 2 election: Requested vote from peers 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473), bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131:37203)
I20260709 13:21:06.804760 21313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee997af6d38745549698356bcbc09a99" candidate_uuid: "ec8fff5744984f4695d768676cae8d17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "71bdb236d2304c75b7eabfec8f3d5a46"
I20260709 13:21:06.805038 21094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee997af6d38745549698356bcbc09a99" candidate_uuid: "ec8fff5744984f4695d768676cae8d17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652"
I20260709 13:21:06.805464 21313 raft_consensus.cc:3060] T ee997af6d38745549698356bcbc09a99 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:06.805557 21094 raft_consensus.cc:3055] T ee997af6d38745549698356bcbc09a99 P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:06.805936 21094 raft_consensus.cc:740] T ee997af6d38745549698356bcbc09a99 P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bd6a13ebcb53485a9cc5082a4f64d652, State: Running, Role: LEADER
I20260709 13:21:06.806743 21094 consensus_queue.cc:260] T ee997af6d38745549698356bcbc09a99 P bd6a13ebcb53485a9cc5082a4f64d652 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:06.808077 21094 raft_consensus.cc:3060] T ee997af6d38745549698356bcbc09a99 P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:06.813037 21313 raft_consensus.cc:2468] T ee997af6d38745549698356bcbc09a99 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec8fff5744984f4695d768676cae8d17 in term 2.
I20260709 13:21:06.814438 20985 leader_election.cc:304] T ee997af6d38745549698356bcbc09a99 P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71bdb236d2304c75b7eabfec8f3d5a46, ec8fff5744984f4695d768676cae8d17; no voters: 
I20260709 13:21:06.814996 21094 raft_consensus.cc:2468] T ee997af6d38745549698356bcbc09a99 P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec8fff5744984f4695d768676cae8d17 in term 2.
I20260709 13:21:06.815433 21390 raft_consensus.cc:2804] T ee997af6d38745549698356bcbc09a99 P ec8fff5744984f4695d768676cae8d17 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:06.816010 21390 raft_consensus.cc:697] T ee997af6d38745549698356bcbc09a99 P ec8fff5744984f4695d768676cae8d17 [term 2 LEADER]: Becoming Leader. State: Replica: ec8fff5744984f4695d768676cae8d17, State: Running, Role: LEADER
I20260709 13:21:06.816895 21390 consensus_queue.cc:237] T ee997af6d38745549698356bcbc09a99 P ec8fff5744984f4695d768676cae8d17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:06.824011 20843 catalog_manager.cc:5697] T ee997af6d38745549698356bcbc09a99 P ec8fff5744984f4695d768676cae8d17 reported cstate change: term changed from 1 to 2, leader changed from bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131) to ec8fff5744984f4695d768676cae8d17 (127.9.116.130). New cstate: current_term: 2 leader_uuid: "ec8fff5744984f4695d768676cae8d17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:06.907771 21417 raft_consensus.cc:993] T 7f3178c915104a27b9a3b6978249c5ed P bd6a13ebcb53485a9cc5082a4f64d652: : Instructing follower ec8fff5744984f4695d768676cae8d17 to start an election
I20260709 13:21:06.908257 21417 raft_consensus.cc:1081] T 7f3178c915104a27b9a3b6978249c5ed P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 LEADER]: Signalling peer ec8fff5744984f4695d768676cae8d17 to start an election
I20260709 13:21:06.909775 21021 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7f3178c915104a27b9a3b6978249c5ed"
dest_uuid: "ec8fff5744984f4695d768676cae8d17"
 from {username='slave'} at 127.0.0.1:54288
I20260709 13:21:06.910235 21021 raft_consensus.cc:493] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:06.910517 21021 raft_consensus.cc:3060] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:06.916770 21021 raft_consensus.cc:515] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:06.918550 21021 leader_election.cc:290] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 3 election: Requested vote from peers 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473), bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131:37203)
I20260709 13:21:06.919227 21313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f3178c915104a27b9a3b6978249c5ed" candidate_uuid: "ec8fff5744984f4695d768676cae8d17" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "71bdb236d2304c75b7eabfec8f3d5a46"
I20260709 13:21:06.919456 21094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f3178c915104a27b9a3b6978249c5ed" candidate_uuid: "ec8fff5744984f4695d768676cae8d17" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652"
I20260709 13:21:06.919973 21313 raft_consensus.cc:3060] T 7f3178c915104a27b9a3b6978249c5ed P 71bdb236d2304c75b7eabfec8f3d5a46 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:06.920024 21094 raft_consensus.cc:3055] T 7f3178c915104a27b9a3b6978249c5ed P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:21:06.920466 21094 raft_consensus.cc:740] T 7f3178c915104a27b9a3b6978249c5ed P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bd6a13ebcb53485a9cc5082a4f64d652, State: Running, Role: LEADER
I20260709 13:21:06.921236 21094 consensus_queue.cc:260] T 7f3178c915104a27b9a3b6978249c5ed P bd6a13ebcb53485a9cc5082a4f64d652 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:06.922122 21094 raft_consensus.cc:3060] T 7f3178c915104a27b9a3b6978249c5ed P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:06.926038 21313 raft_consensus.cc:2468] T 7f3178c915104a27b9a3b6978249c5ed P 71bdb236d2304c75b7eabfec8f3d5a46 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec8fff5744984f4695d768676cae8d17 in term 3.
I20260709 13:21:06.927304 20985 leader_election.cc:304] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71bdb236d2304c75b7eabfec8f3d5a46, ec8fff5744984f4695d768676cae8d17; no voters: 
I20260709 13:21:06.928092 21390 raft_consensus.cc:2804] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:21:06.928541 21390 raft_consensus.cc:697] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17 [term 3 LEADER]: Becoming Leader. State: Replica: ec8fff5744984f4695d768676cae8d17, State: Running, Role: LEADER
I20260709 13:21:06.928929 21094 raft_consensus.cc:2468] T 7f3178c915104a27b9a3b6978249c5ed P bd6a13ebcb53485a9cc5082a4f64d652 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec8fff5744984f4695d768676cae8d17 in term 3.
I20260709 13:21:06.929324 21390 consensus_queue.cc:237] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:06.936004 20843 catalog_manager.cc:5697] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17 reported cstate change: term changed from 2 to 3, leader changed from bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131) to ec8fff5744984f4695d768676cae8d17 (127.9.116.130). New cstate: current_term: 3 leader_uuid: "ec8fff5744984f4695d768676cae8d17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:07.002166 21258 raft_consensus.cc:993] T c50ac60e07eb46dc8836de50bce12760 P ec8fff5744984f4695d768676cae8d17: : Instructing follower bd6a13ebcb53485a9cc5082a4f64d652 to start an election
I20260709 13:21:07.002684 21390 raft_consensus.cc:1081] T c50ac60e07eb46dc8836de50bce12760 P ec8fff5744984f4695d768676cae8d17 [term 2 LEADER]: Signalling peer bd6a13ebcb53485a9cc5082a4f64d652 to start an election
I20260709 13:21:07.004258 21094 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c50ac60e07eb46dc8836de50bce12760"
dest_uuid: "bd6a13ebcb53485a9cc5082a4f64d652"
 from {username='slave'} at 127.0.0.1:52450
I20260709 13:21:07.004736 21094 raft_consensus.cc:493] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:07.005012 21094 raft_consensus.cc:3060] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:07.011338 21094 raft_consensus.cc:515] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:07.012789 21094 leader_election.cc:290] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 3 election: Requested vote from peers 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473), ec8fff5744984f4695d768676cae8d17 (127.9.116.130:42659)
I20260709 13:21:07.013765 21021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50ac60e07eb46dc8836de50bce12760" candidate_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ec8fff5744984f4695d768676cae8d17"
I20260709 13:21:07.014014 21313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50ac60e07eb46dc8836de50bce12760" candidate_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "71bdb236d2304c75b7eabfec8f3d5a46"
I20260709 13:21:07.014392 21021 raft_consensus.cc:3055] T c50ac60e07eb46dc8836de50bce12760 P ec8fff5744984f4695d768676cae8d17 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:21:07.014715 21313 raft_consensus.cc:3060] T c50ac60e07eb46dc8836de50bce12760 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:07.014736 21021 raft_consensus.cc:740] T c50ac60e07eb46dc8836de50bce12760 P ec8fff5744984f4695d768676cae8d17 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ec8fff5744984f4695d768676cae8d17, State: Running, Role: LEADER
I20260709 13:21:07.015730 21021 consensus_queue.cc:260] T c50ac60e07eb46dc8836de50bce12760 P ec8fff5744984f4695d768676cae8d17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:07.016774 21021 raft_consensus.cc:3060] T c50ac60e07eb46dc8836de50bce12760 P ec8fff5744984f4695d768676cae8d17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:07.022218 21021 raft_consensus.cc:2468] T c50ac60e07eb46dc8836de50bce12760 P ec8fff5744984f4695d768676cae8d17 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd6a13ebcb53485a9cc5082a4f64d652 in term 3.
I20260709 13:21:07.023401 21061 leader_election.cc:304] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd6a13ebcb53485a9cc5082a4f64d652, ec8fff5744984f4695d768676cae8d17; no voters: 
I20260709 13:21:07.023993 21313 raft_consensus.cc:2468] T c50ac60e07eb46dc8836de50bce12760 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd6a13ebcb53485a9cc5082a4f64d652 in term 3.
I20260709 13:21:07.024317 21365 raft_consensus.cc:2804] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:21:07.024943 21365 raft_consensus.cc:697] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 [term 3 LEADER]: Becoming Leader. State: Replica: bd6a13ebcb53485a9cc5082a4f64d652, State: Running, Role: LEADER
I20260709 13:21:07.025758 21365 consensus_queue.cc:237] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } }
I20260709 13:21:07.032577 20843 catalog_manager.cc:5697] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 reported cstate change: term changed from 2 to 3, leader changed from ec8fff5744984f4695d768676cae8d17 (127.9.116.130) to bd6a13ebcb53485a9cc5082a4f64d652 (127.9.116.131). New cstate: current_term: 3 leader_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 } health_report { overall_health: HEALTHY } } }
W20260709 13:21:07.125085 20892 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:21:07.247195 21313 raft_consensus.cc:1275] T ee997af6d38745549698356bcbc09a99 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 2 FOLLOWER]: Refusing update from remote peer ec8fff5744984f4695d768676cae8d17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:07.248598 21390 consensus_queue.cc:1048] T ee997af6d38745549698356bcbc09a99 P ec8fff5744984f4695d768676cae8d17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:07.260540 21094 raft_consensus.cc:1275] T ee997af6d38745549698356bcbc09a99 P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 FOLLOWER]: Refusing update from remote peer ec8fff5744984f4695d768676cae8d17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:07.262030 21390 consensus_queue.cc:1048] T ee997af6d38745549698356bcbc09a99 P ec8fff5744984f4695d768676cae8d17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:07.395151 21094 raft_consensus.cc:1275] T 7f3178c915104a27b9a3b6978249c5ed P bd6a13ebcb53485a9cc5082a4f64d652 [term 3 FOLLOWER]: Refusing update from remote peer ec8fff5744984f4695d768676cae8d17: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:21:07.396668 21258 consensus_queue.cc:1048] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd6a13ebcb53485a9cc5082a4f64d652" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:21:07.399191 21313 raft_consensus.cc:1275] T 7f3178c915104a27b9a3b6978249c5ed P 71bdb236d2304c75b7eabfec8f3d5a46 [term 3 FOLLOWER]: Refusing update from remote peer ec8fff5744984f4695d768676cae8d17: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:21:07.400863 21258 consensus_queue.cc:1048] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:21:07.446907 21021 raft_consensus.cc:1275] T c50ac60e07eb46dc8836de50bce12760 P ec8fff5744984f4695d768676cae8d17 [term 3 FOLLOWER]: Refusing update from remote peer bd6a13ebcb53485a9cc5082a4f64d652: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:21:07.448179 21365 consensus_queue.cc:1048] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec8fff5744984f4695d768676cae8d17" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 42659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:21:07.456686 21313 raft_consensus.cc:1275] T c50ac60e07eb46dc8836de50bce12760 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 3 FOLLOWER]: Refusing update from remote peer bd6a13ebcb53485a9cc5082a4f64d652: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:21:07.458323 21434 consensus_queue.cc:1048] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71bdb236d2304c75b7eabfec8f3d5a46" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:21:07.685854  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:21:07.715752  9682 tablet_server.cc:179] TabletServer@127.9.116.129:40473 shutting down...
I20260709 13:21:07.735090  9682 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:21:07.735845  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:07.736558  9682 raft_consensus.cc:2243] T 0f79d8e2c55b4a1f8380919ef723efe7 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:07.736990  9682 raft_consensus.cc:2272] T 0f79d8e2c55b4a1f8380919ef723efe7 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:07.739032  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:07.739625  9682 raft_consensus.cc:2243] T ee997af6d38745549698356bcbc09a99 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:07.740058  9682 raft_consensus.cc:2272] T ee997af6d38745549698356bcbc09a99 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:07.742336  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:07.742969  9682 raft_consensus.cc:2243] T 5e579b46cc154814ad25486ba520710f P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:07.743368  9682 raft_consensus.cc:2272] T 5e579b46cc154814ad25486ba520710f P 71bdb236d2304c75b7eabfec8f3d5a46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:07.745154  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:07.745683  9682 raft_consensus.cc:2243] T 35d6d5be27cb4096917b886d06a3cf58 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:07.746111  9682 raft_consensus.cc:2272] T 35d6d5be27cb4096917b886d06a3cf58 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:07.748039  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:07.748587  9682 raft_consensus.cc:2243] T b541a9ed669a420d8fbeb6e10ab3c47f P 71bdb236d2304c75b7eabfec8f3d5a46 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:07.749076  9682 raft_consensus.cc:2272] T b541a9ed669a420d8fbeb6e10ab3c47f P 71bdb236d2304c75b7eabfec8f3d5a46 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:07.751003  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:07.751513  9682 raft_consensus.cc:2243] T c3add26e31364c95a9897ab53b516735 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:07.751931  9682 raft_consensus.cc:2272] T c3add26e31364c95a9897ab53b516735 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:07.753686  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:07.754217  9682 raft_consensus.cc:2243] T 35602162000346e3bbb790f54467ed0d P 71bdb236d2304c75b7eabfec8f3d5a46 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:07.754648  9682 raft_consensus.cc:2272] T 35602162000346e3bbb790f54467ed0d P 71bdb236d2304c75b7eabfec8f3d5a46 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:07.756423  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:07.756925  9682 raft_consensus.cc:2243] T 2725109df1524e6c8eb897f0e6424829 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:07.757362  9682 raft_consensus.cc:2272] T 2725109df1524e6c8eb897f0e6424829 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:07.759192  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:07.759720  9682 raft_consensus.cc:2243] T c50ac60e07eb46dc8836de50bce12760 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:07.760161  9682 raft_consensus.cc:2272] T c50ac60e07eb46dc8836de50bce12760 P 71bdb236d2304c75b7eabfec8f3d5a46 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:07.762238 20985 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 4 similar messages]
I20260709 13:21:07.762825  9682 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:07.763662  9682 raft_consensus.cc:2243] T 7f3178c915104a27b9a3b6978249c5ed P 71bdb236d2304c75b7eabfec8f3d5a46 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:07.764159  9682 raft_consensus.cc:2272] T 7f3178c915104a27b9a3b6978249c5ed P 71bdb236d2304c75b7eabfec8f3d5a46 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:07.765457 20985 consensus_peers.cc:597] T ee997af6d38745549698356bcbc09a99 P ec8fff5744984f4695d768676cae8d17 -> Peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Couldn't send request to peer 71bdb236d2304c75b7eabfec8f3d5a46. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:07.794448  9682 tablet_server.cc:196] TabletServer@127.9.116.129:40473 shutdown complete.
I20260709 13:21:07.814771  9682 tablet_server.cc:179] TabletServer@127.9.116.130:0 shutting down...
W20260709 13:21:07.835279 20985 consensus_peers.cc:597] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17 -> Peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Couldn't send request to peer 71bdb236d2304c75b7eabfec8f3d5a46. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:40473: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:07.847432  9682 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:21:07.848368  9682 tablet_replica.cc:333] T ee997af6d38745549698356bcbc09a99 P ec8fff5744984f4695d768676cae8d17: stopping tablet replica
W20260709 13:21:07.849403 21059 consensus_peers.cc:597] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 -> Peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Couldn't send request to peer 71bdb236d2304c75b7eabfec8f3d5a46. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:40473: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:07.849517  9682 raft_consensus.cc:2243] T ee997af6d38745549698356bcbc09a99 P ec8fff5744984f4695d768676cae8d17 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:07.850744  9682 raft_consensus.cc:2272] T ee997af6d38745549698356bcbc09a99 P ec8fff5744984f4695d768676cae8d17 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:07.853492  9682 tablet_replica.cc:333] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17: stopping tablet replica
I20260709 13:21:07.854140  9682 raft_consensus.cc:2243] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:21:07.854996  9682 raft_consensus.cc:2272] T 7f3178c915104a27b9a3b6978249c5ed P ec8fff5744984f4695d768676cae8d17 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:07.857519  9682 tablet_replica.cc:333] T 35d6d5be27cb4096917b886d06a3cf58 P ec8fff5744984f4695d768676cae8d17: stopping tablet replica
I20260709 13:21:07.858170  9682 raft_consensus.cc:2243] T 35d6d5be27cb4096917b886d06a3cf58 P ec8fff5744984f4695d768676cae8d17 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:21:07.859009  9682 raft_consensus.cc:2272] T 35d6d5be27cb4096917b886d06a3cf58 P ec8fff5744984f4695d768676cae8d17 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:07.861101  9682 tablet_replica.cc:333] T 5e579b46cc154814ad25486ba520710f P ec8fff5744984f4695d768676cae8d17: stopping tablet replica
I20260709 13:21:07.861629  9682 raft_consensus.cc:2243] T 5e579b46cc154814ad25486ba520710f P ec8fff5744984f4695d768676cae8d17 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:07.862116  9682 raft_consensus.cc:2272] T 5e579b46cc154814ad25486ba520710f P ec8fff5744984f4695d768676cae8d17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:07.864362  9682 tablet_replica.cc:333] T c50ac60e07eb46dc8836de50bce12760 P ec8fff5744984f4695d768676cae8d17: stopping tablet replica
I20260709 13:21:07.864933  9682 raft_consensus.cc:2243] T c50ac60e07eb46dc8836de50bce12760 P ec8fff5744984f4695d768676cae8d17 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:07.865357  9682 raft_consensus.cc:2272] T c50ac60e07eb46dc8836de50bce12760 P ec8fff5744984f4695d768676cae8d17 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:07.868182  9682 tablet_replica.cc:333] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17: stopping tablet replica
I20260709 13:21:07.868778  9682 raft_consensus.cc:2243] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:21:07.869549  9682 raft_consensus.cc:2272] T 2725109df1524e6c8eb897f0e6424829 P ec8fff5744984f4695d768676cae8d17 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:07.871749  9682 tablet_replica.cc:333] T b541a9ed669a420d8fbeb6e10ab3c47f P ec8fff5744984f4695d768676cae8d17: stopping tablet replica
I20260709 13:21:07.872268  9682 raft_consensus.cc:2243] T b541a9ed669a420d8fbeb6e10ab3c47f P ec8fff5744984f4695d768676cae8d17 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:07.872712  9682 raft_consensus.cc:2272] T b541a9ed669a420d8fbeb6e10ab3c47f P ec8fff5744984f4695d768676cae8d17 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:07.874615  9682 tablet_replica.cc:333] T c3add26e31364c95a9897ab53b516735 P ec8fff5744984f4695d768676cae8d17: stopping tablet replica
I20260709 13:21:07.875147  9682 raft_consensus.cc:2243] T c3add26e31364c95a9897ab53b516735 P ec8fff5744984f4695d768676cae8d17 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:07.875568  9682 raft_consensus.cc:2272] T c3add26e31364c95a9897ab53b516735 P ec8fff5744984f4695d768676cae8d17 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:07.878180  9682 tablet_replica.cc:333] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17: stopping tablet replica
I20260709 13:21:07.878695  9682 raft_consensus.cc:2243] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:07.879266  9682 raft_consensus.cc:2272] T 35602162000346e3bbb790f54467ed0d P ec8fff5744984f4695d768676cae8d17 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:07.880496 21061 consensus_peers.cc:597] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 -> Peer ec8fff5744984f4695d768676cae8d17 (127.9.116.130:42659): Couldn't send request to peer ec8fff5744984f4695d768676cae8d17. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:07.881660  9682 tablet_replica.cc:333] T 0f79d8e2c55b4a1f8380919ef723efe7 P ec8fff5744984f4695d768676cae8d17: stopping tablet replica
I20260709 13:21:07.882273  9682 raft_consensus.cc:2243] T 0f79d8e2c55b4a1f8380919ef723efe7 P ec8fff5744984f4695d768676cae8d17 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:07.883131  9682 raft_consensus.cc:2272] T 0f79d8e2c55b4a1f8380919ef723efe7 P ec8fff5744984f4695d768676cae8d17 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:07.901466 21059 consensus_peers.cc:597] T c3add26e31364c95a9897ab53b516735 P bd6a13ebcb53485a9cc5082a4f64d652 -> Peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Couldn't send request to peer 71bdb236d2304c75b7eabfec8f3d5a46. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:40473: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:07.917133  9682 tablet_server.cc:196] TabletServer@127.9.116.130:0 shutdown complete.
W20260709 13:21:07.932878 21061 consensus_peers.cc:597] T 5e579b46cc154814ad25486ba520710f P bd6a13ebcb53485a9cc5082a4f64d652 -> Peer ec8fff5744984f4695d768676cae8d17 (127.9.116.130:42659): Couldn't send request to peer ec8fff5744984f4695d768676cae8d17. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:42659: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:07.943346  9682 tablet_server.cc:179] TabletServer@127.9.116.131:0 shutting down...
W20260709 13:21:07.964555 21061 consensus_peers.cc:597] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652 -> Peer ec8fff5744984f4695d768676cae8d17 (127.9.116.130:42659): Couldn't send request to peer ec8fff5744984f4695d768676cae8d17. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:42659: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:07.973238  9682 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:21:07.974031  9682 tablet_replica.cc:333] T ee997af6d38745549698356bcbc09a99 P bd6a13ebcb53485a9cc5082a4f64d652: stopping tablet replica
I20260709 13:21:07.974762  9682 raft_consensus.cc:2243] T ee997af6d38745549698356bcbc09a99 P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:07.975337  9682 raft_consensus.cc:2272] T ee997af6d38745549698356bcbc09a99 P bd6a13ebcb53485a9cc5082a4f64d652 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:07.978024  9682 tablet_replica.cc:333] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652: stopping tablet replica
I20260709 13:21:07.978991  9682 raft_consensus.cc:2243] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:21:07.979866  9682 raft_consensus.cc:2272] T c50ac60e07eb46dc8836de50bce12760 P bd6a13ebcb53485a9cc5082a4f64d652 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:07.982363  9682 tablet_replica.cc:333] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652: stopping tablet replica
I20260709 13:21:07.983049  9682 raft_consensus.cc:2243] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:07.983613  9682 raft_consensus.cc:2272] T 2725109df1524e6c8eb897f0e6424829 P bd6a13ebcb53485a9cc5082a4f64d652 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:07.987054  9682 tablet_replica.cc:333] T 35d6d5be27cb4096917b886d06a3cf58 P bd6a13ebcb53485a9cc5082a4f64d652: stopping tablet replica
I20260709 13:21:07.987985  9682 raft_consensus.cc:2243] T 35d6d5be27cb4096917b886d06a3cf58 P bd6a13ebcb53485a9cc5082a4f64d652 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:07.988605  9682 raft_consensus.cc:2272] T 35d6d5be27cb4096917b886d06a3cf58 P bd6a13ebcb53485a9cc5082a4f64d652 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:07.992029  9682 tablet_replica.cc:333] T 5e579b46cc154814ad25486ba520710f P bd6a13ebcb53485a9cc5082a4f64d652: stopping tablet replica
W20260709 13:21:07.992545 21061 consensus_peers.cc:597] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 -> Peer ec8fff5744984f4695d768676cae8d17 (127.9.116.130:42659): Couldn't send request to peer ec8fff5744984f4695d768676cae8d17. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:42659: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:07.992919  9682 raft_consensus.cc:2243] T 5e579b46cc154814ad25486ba520710f P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:07.994179  9682 raft_consensus.cc:2272] T 5e579b46cc154814ad25486ba520710f P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:07.995832 21059 consensus_peers.cc:597] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 -> Peer 71bdb236d2304c75b7eabfec8f3d5a46 (127.9.116.129:40473): Couldn't send request to peer 71bdb236d2304c75b7eabfec8f3d5a46. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:40473: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:07.997077  9682 tablet_replica.cc:333] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652: stopping tablet replica
I20260709 13:21:07.997756  9682 raft_consensus.cc:2243] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:21:07.998575  9682 raft_consensus.cc:2272] T 35602162000346e3bbb790f54467ed0d P bd6a13ebcb53485a9cc5082a4f64d652 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:08.000893  9682 tablet_replica.cc:333] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652: stopping tablet replica
I20260709 13:21:08.001518  9682 raft_consensus.cc:2243] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:21:08.002346  9682 raft_consensus.cc:2272] T b541a9ed669a420d8fbeb6e10ab3c47f P bd6a13ebcb53485a9cc5082a4f64d652 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:08.004806  9682 tablet_replica.cc:333] T c3add26e31364c95a9897ab53b516735 P bd6a13ebcb53485a9cc5082a4f64d652: stopping tablet replica
I20260709 13:21:08.005430  9682 raft_consensus.cc:2243] T c3add26e31364c95a9897ab53b516735 P bd6a13ebcb53485a9cc5082a4f64d652 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:21:08.006232  9682 raft_consensus.cc:2272] T c3add26e31364c95a9897ab53b516735 P bd6a13ebcb53485a9cc5082a4f64d652 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:08.008435  9682 tablet_replica.cc:333] T 7f3178c915104a27b9a3b6978249c5ed P bd6a13ebcb53485a9cc5082a4f64d652: stopping tablet replica
I20260709 13:21:08.009004  9682 raft_consensus.cc:2243] T 7f3178c915104a27b9a3b6978249c5ed P bd6a13ebcb53485a9cc5082a4f64d652 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:08.009471  9682 raft_consensus.cc:2272] T 7f3178c915104a27b9a3b6978249c5ed P bd6a13ebcb53485a9cc5082a4f64d652 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:08.012414  9682 tablet_replica.cc:333] T 0f79d8e2c55b4a1f8380919ef723efe7 P bd6a13ebcb53485a9cc5082a4f64d652: stopping tablet replica
I20260709 13:21:08.013115  9682 raft_consensus.cc:2243] T 0f79d8e2c55b4a1f8380919ef723efe7 P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:08.013607  9682 raft_consensus.cc:2272] T 0f79d8e2c55b4a1f8380919ef723efe7 P bd6a13ebcb53485a9cc5082a4f64d652 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:08.065110  9682 tablet_server.cc:196] TabletServer@127.9.116.131:0 shutdown complete.
I20260709 13:21:08.090764  9682 master.cc:562] Master@127.9.116.190:40921 shutting down...
I20260709 13:21:08.117515  9682 raft_consensus.cc:2243] T 00000000000000000000000000000000 P d11a97679c174598899b62857a671ac1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:08.118101  9682 raft_consensus.cc:2272] T 00000000000000000000000000000000 P d11a97679c174598899b62857a671ac1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:08.118454  9682 tablet_replica.cc:333] T 00000000000000000000000000000000 P d11a97679c174598899b62857a671ac1: stopping tablet replica
I20260709 13:21:08.138533  9682 master.cc:584] Master@127.9.116.190:40921 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (15207 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 13:21:08.183288  9682 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.9.116.190:45781
I20260709 13:21:08.184402  9682 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:21:08.189517 21443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:08.190412 21444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:08.192909 21446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:08.192966  9682 server_base.cc:1061] running on GCE node
I20260709 13:21:08.194151  9682 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:21:08.194334  9682 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:21:08.194548  9682 hybrid_clock.cc:648] HybridClock initialized: now 1783603268194512 us; error 0 us; skew 500 ppm
I20260709 13:21:08.195452  9682 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:21:08.197921  9682 webserver.cc:533] Webserver started at http://127.9.116.190:38887/ using document root <none> and password file <none>
I20260709 13:21:08.198450  9682 fs_manager.cc:362] Metadata directory not provided
I20260709 13:21:08.198635  9682 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:21:08.198987  9682 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:21:08.200124  9682 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602916530764-9682-0/minicluster-data/master-0-root/instance:
uuid: "2d23126fb62441beb1e910daf013f7cb"
format_stamp: "Formatted at 2026-07-09 13:21:08 on dist-test-slave-1t6m"
I20260709 13:21:08.204605  9682 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:21:08.208154 21451 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:08.208967  9682 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 13:21:08.209307  9682 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602916530764-9682-0/minicluster-data/master-0-root
uuid: "2d23126fb62441beb1e910daf013f7cb"
format_stamp: "Formatted at 2026-07-09 13:21:08 on dist-test-slave-1t6m"
I20260709 13:21:08.209609  9682 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602916530764-9682-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602916530764-9682-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602916530764-9682-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:21:08.220381  9682 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:21:08.221668  9682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:21:08.260178  9682 rpc_server.cc:307] RPC server started. Bound to: 127.9.116.190:45781
I20260709 13:21:08.260284 21502 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.116.190:45781 every 8 connection(s)
I20260709 13:21:08.264091 21503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:08.275645 21503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d23126fb62441beb1e910daf013f7cb: Bootstrap starting.
I20260709 13:21:08.280603 21503 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d23126fb62441beb1e910daf013f7cb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:08.285193 21503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d23126fb62441beb1e910daf013f7cb: No bootstrap required, opened a new log
I20260709 13:21:08.287563 21503 raft_consensus.cc:359] T 00000000000000000000000000000000 P 2d23126fb62441beb1e910daf013f7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d23126fb62441beb1e910daf013f7cb" member_type: VOTER }
I20260709 13:21:08.287992 21503 raft_consensus.cc:385] T 00000000000000000000000000000000 P 2d23126fb62441beb1e910daf013f7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:08.288239 21503 raft_consensus.cc:740] T 00000000000000000000000000000000 P 2d23126fb62441beb1e910daf013f7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d23126fb62441beb1e910daf013f7cb, State: Initialized, Role: FOLLOWER
I20260709 13:21:08.288858 21503 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d23126fb62441beb1e910daf013f7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d23126fb62441beb1e910daf013f7cb" member_type: VOTER }
I20260709 13:21:08.289345 21503 raft_consensus.cc:399] T 00000000000000000000000000000000 P 2d23126fb62441beb1e910daf013f7cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:21:08.289585 21503 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2d23126fb62441beb1e910daf013f7cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:21:08.289867 21503 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2d23126fb62441beb1e910daf013f7cb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:08.294734 21503 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2d23126fb62441beb1e910daf013f7cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d23126fb62441beb1e910daf013f7cb" member_type: VOTER }
I20260709 13:21:08.295360 21503 leader_election.cc:304] T 00000000000000000000000000000000 P 2d23126fb62441beb1e910daf013f7cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d23126fb62441beb1e910daf013f7cb; no voters: 
I20260709 13:21:08.296458 21503 leader_election.cc:290] T 00000000000000000000000000000000 P 2d23126fb62441beb1e910daf013f7cb [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:21:08.296846 21506 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2d23126fb62441beb1e910daf013f7cb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:08.298249 21506 raft_consensus.cc:697] T 00000000000000000000000000000000 P 2d23126fb62441beb1e910daf013f7cb [term 1 LEADER]: Becoming Leader. State: Replica: 2d23126fb62441beb1e910daf013f7cb, State: Running, Role: LEADER
I20260709 13:21:08.299012 21506 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d23126fb62441beb1e910daf013f7cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d23126fb62441beb1e910daf013f7cb" member_type: VOTER }
I20260709 13:21:08.299499 21503 sys_catalog.cc:565] T 00000000000000000000000000000000 P 2d23126fb62441beb1e910daf013f7cb [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:21:08.303552 21508 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d23126fb62441beb1e910daf013f7cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d23126fb62441beb1e910daf013f7cb. Latest consensus state: current_term: 1 leader_uuid: "2d23126fb62441beb1e910daf013f7cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d23126fb62441beb1e910daf013f7cb" member_type: VOTER } }
I20260709 13:21:08.303593 21507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d23126fb62441beb1e910daf013f7cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d23126fb62441beb1e910daf013f7cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d23126fb62441beb1e910daf013f7cb" member_type: VOTER } }
I20260709 13:21:08.304386 21507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d23126fb62441beb1e910daf013f7cb [sys.catalog]: This master's current role is: LEADER
I20260709 13:21:08.306165 21508 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d23126fb62441beb1e910daf013f7cb [sys.catalog]: This master's current role is: LEADER
I20260709 13:21:08.306425 21514 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:21:08.312567 21514 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:21:08.314976  9682 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:21:08.322062 21514 catalog_manager.cc:1383] Generated new cluster ID: 847725cafd7b43f7a75f82bb563ef5c8
I20260709 13:21:08.322374 21514 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:21:08.350028 21514 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:21:08.351433 21514 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:21:08.370766 21514 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 2d23126fb62441beb1e910daf013f7cb: Generated new TSK 0
I20260709 13:21:08.371429 21514 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:21:08.385075  9682 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:21:08.391397 21524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:08.392051 21525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:08.393802 21527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:08.394352  9682 server_base.cc:1061] running on GCE node
I20260709 13:21:08.395323  9682 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:21:08.395520  9682 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:21:08.395651  9682 hybrid_clock.cc:648] HybridClock initialized: now 1783603268395640 us; error 0 us; skew 500 ppm
I20260709 13:21:08.396133  9682 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:21:08.398694  9682 webserver.cc:533] Webserver started at http://127.9.116.129:38087/ using document root <none> and password file <none>
I20260709 13:21:08.399214  9682 fs_manager.cc:362] Metadata directory not provided
I20260709 13:21:08.399394  9682 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:21:08.399613  9682 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:21:08.400658  9682 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602916530764-9682-0/minicluster-data/ts-0-root/instance:
uuid: "c49029e9cb7546728f69e9c919b01d90"
format_stamp: "Formatted at 2026-07-09 13:21:08 on dist-test-slave-1t6m"
I20260709 13:21:08.405418  9682 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:21:08.408974 21532 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:08.409973  9682 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:21:08.410315  9682 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602916530764-9682-0/minicluster-data/ts-0-root
uuid: "c49029e9cb7546728f69e9c919b01d90"
format_stamp: "Formatted at 2026-07-09 13:21:08 on dist-test-slave-1t6m"
I20260709 13:21:08.410673  9682 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602916530764-9682-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602916530764-9682-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602916530764-9682-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:21:08.432104  9682 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:21:08.433336  9682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:21:08.435081  9682 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:21:08.437426  9682 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:21:08.437668  9682 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:08.437907  9682 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:21:08.438086  9682 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:08.480444  9682 rpc_server.cc:307] RPC server started. Bound to: 127.9.116.129:32957
I20260709 13:21:08.480582 21594 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.116.129:32957 every 8 connection(s)
I20260709 13:21:08.485387  9682 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:21:08.493155 21599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:08.496622 21600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:08.499233  9682 server_base.cc:1061] running on GCE node
I20260709 13:21:08.500384 21595 heartbeater.cc:344] Connected to a master server at 127.9.116.190:45781
I20260709 13:21:08.500870 21595 heartbeater.cc:461] Registering TS with master...
W20260709 13:21:08.501236 21602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:08.501812 21595 heartbeater.cc:507] Master 127.9.116.190:45781 requested a full tablet report, sending...
I20260709 13:21:08.502583  9682 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:21:08.503048  9682 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:21:08.503261  9682 hybrid_clock.cc:648] HybridClock initialized: now 1783603268503222 us; error 0 us; skew 500 ppm
I20260709 13:21:08.504070  9682 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:21:08.504503 21468 ts_manager.cc:194] Registered new tserver with Master: c49029e9cb7546728f69e9c919b01d90 (127.9.116.129:32957)
I20260709 13:21:08.506891 21468 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43510
I20260709 13:21:08.506760  9682 webserver.cc:533] Webserver started at http://127.9.116.130:46493/ using document root <none> and password file <none>
I20260709 13:21:08.507783  9682 fs_manager.cc:362] Metadata directory not provided
I20260709 13:21:08.508010  9682 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:21:08.508351  9682 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:21:08.509944  9682 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602916530764-9682-0/minicluster-data/ts-1-root/instance:
uuid: "30f6f31e9a314e4099b7466a143ed434"
format_stamp: "Formatted at 2026-07-09 13:21:08 on dist-test-slave-1t6m"
I20260709 13:21:08.514861  9682 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.004s
I20260709 13:21:08.518183 21607 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:08.519025  9682 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:21:08.519308  9682 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602916530764-9682-0/minicluster-data/ts-1-root
uuid: "30f6f31e9a314e4099b7466a143ed434"
format_stamp: "Formatted at 2026-07-09 13:21:08 on dist-test-slave-1t6m"
I20260709 13:21:08.519565  9682 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602916530764-9682-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602916530764-9682-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602916530764-9682-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:21:08.542461  9682 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:21:08.543737  9682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:21:08.545176  9682 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:21:08.547566  9682 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:21:08.547757  9682 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:08.548060  9682 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:21:08.548218  9682 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:08.589543  9682 rpc_server.cc:307] RPC server started. Bound to: 127.9.116.130:37109
I20260709 13:21:08.589715 21669 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.116.130:37109 every 8 connection(s)
I20260709 13:21:08.594527  9682 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:21:08.601660 21673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:08.602766 21674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:08.606384  9682 server_base.cc:1061] running on GCE node
W20260709 13:21:08.606520 21676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:08.606961 21670 heartbeater.cc:344] Connected to a master server at 127.9.116.190:45781
I20260709 13:21:08.607493 21670 heartbeater.cc:461] Registering TS with master...
I20260709 13:21:08.607873  9682 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:21:08.608180  9682 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:21:08.608407  9682 hybrid_clock.cc:648] HybridClock initialized: now 1783603268608387 us; error 0 us; skew 500 ppm
I20260709 13:21:08.608479 21670 heartbeater.cc:507] Master 127.9.116.190:45781 requested a full tablet report, sending...
I20260709 13:21:08.609241  9682 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:21:08.611229 21468 ts_manager.cc:194] Registered new tserver with Master: 30f6f31e9a314e4099b7466a143ed434 (127.9.116.130:37109)
I20260709 13:21:08.612179  9682 webserver.cc:533] Webserver started at http://127.9.116.131:37195/ using document root <none> and password file <none>
I20260709 13:21:08.612775  9682 fs_manager.cc:362] Metadata directory not provided
I20260709 13:21:08.612982  9682 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:21:08.613132 21468 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43514
I20260709 13:21:08.613271  9682 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:21:08.614753  9682 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602916530764-9682-0/minicluster-data/ts-2-root/instance:
uuid: "73c089c6be7a453a818ca37081639799"
format_stamp: "Formatted at 2026-07-09 13:21:08 on dist-test-slave-1t6m"
I20260709 13:21:08.619350  9682 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:21:08.622922 21681 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:08.623741  9682 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 13:21:08.624068  9682 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602916530764-9682-0/minicluster-data/ts-2-root
uuid: "73c089c6be7a453a818ca37081639799"
format_stamp: "Formatted at 2026-07-09 13:21:08 on dist-test-slave-1t6m"
I20260709 13:21:08.624390  9682 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602916530764-9682-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602916530764-9682-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602916530764-9682-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:21:08.640602  9682 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:21:08.641837  9682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:21:08.643409  9682 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:21:08.645673  9682 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:21:08.645910  9682 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:08.646155  9682 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:21:08.646342  9682 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:08.687844  9682 rpc_server.cc:307] RPC server started. Bound to: 127.9.116.131:35735
I20260709 13:21:08.687954 21743 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.116.131:35735 every 8 connection(s)
I20260709 13:21:08.702495 21744 heartbeater.cc:344] Connected to a master server at 127.9.116.190:45781
I20260709 13:21:08.702986 21744 heartbeater.cc:461] Registering TS with master...
I20260709 13:21:08.703855 21744 heartbeater.cc:507] Master 127.9.116.190:45781 requested a full tablet report, sending...
I20260709 13:21:08.705861 21468 ts_manager.cc:194] Registered new tserver with Master: 73c089c6be7a453a818ca37081639799 (127.9.116.131:35735)
I20260709 13:21:08.706118  9682 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014524286s
I20260709 13:21:08.707592 21468 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43524
I20260709 13:21:08.732770 21468 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43530:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:21:08.735116 21468 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:21:08.793705 21560 tablet_service.cc:1511] Processing CreateTablet for tablet 83f44d0e05dd4b1ea9ff6706edf996fa (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=5d6dd25f80354fb684de38586adeb6d2]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:21:08.794126 21559 tablet_service.cc:1511] Processing CreateTablet for tablet 67e0d782f225460f8d7bf6e3a519e0c5 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=5d6dd25f80354fb684de38586adeb6d2]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:21:08.795253 21560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83f44d0e05dd4b1ea9ff6706edf996fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:08.795913 21559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67e0d782f225460f8d7bf6e3a519e0c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:08.798913 21709 tablet_service.cc:1511] Processing CreateTablet for tablet 83f44d0e05dd4b1ea9ff6706edf996fa (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=5d6dd25f80354fb684de38586adeb6d2]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:21:08.800376 21709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83f44d0e05dd4b1ea9ff6706edf996fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:08.802162 21708 tablet_service.cc:1511] Processing CreateTablet for tablet 67e0d782f225460f8d7bf6e3a519e0c5 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=5d6dd25f80354fb684de38586adeb6d2]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:21:08.802450 21635 tablet_service.cc:1511] Processing CreateTablet for tablet 83f44d0e05dd4b1ea9ff6706edf996fa (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=5d6dd25f80354fb684de38586adeb6d2]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:21:08.803735 21708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67e0d782f225460f8d7bf6e3a519e0c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:08.803893 21635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83f44d0e05dd4b1ea9ff6706edf996fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:08.807721 21634 tablet_service.cc:1511] Processing CreateTablet for tablet 67e0d782f225460f8d7bf6e3a519e0c5 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=5d6dd25f80354fb684de38586adeb6d2]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:21:08.809278 21634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67e0d782f225460f8d7bf6e3a519e0c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:08.826444 21759 tablet_bootstrap.cc:492] T 83f44d0e05dd4b1ea9ff6706edf996fa P 73c089c6be7a453a818ca37081639799: Bootstrap starting.
I20260709 13:21:08.834614 21759 tablet_bootstrap.cc:654] T 83f44d0e05dd4b1ea9ff6706edf996fa P 73c089c6be7a453a818ca37081639799: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:08.844079 21760 tablet_bootstrap.cc:492] T 83f44d0e05dd4b1ea9ff6706edf996fa P 30f6f31e9a314e4099b7466a143ed434: Bootstrap starting.
I20260709 13:21:08.846487 21759 tablet_bootstrap.cc:492] T 83f44d0e05dd4b1ea9ff6706edf996fa P 73c089c6be7a453a818ca37081639799: No bootstrap required, opened a new log
I20260709 13:21:08.847143 21759 ts_tablet_manager.cc:1403] T 83f44d0e05dd4b1ea9ff6706edf996fa P 73c089c6be7a453a818ca37081639799: Time spent bootstrapping tablet: real 0.021s	user 0.009s	sys 0.009s
I20260709 13:21:08.850404 21759 raft_consensus.cc:359] T 83f44d0e05dd4b1ea9ff6706edf996fa P 73c089c6be7a453a818ca37081639799 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } }
I20260709 13:21:08.851241 21759 raft_consensus.cc:385] T 83f44d0e05dd4b1ea9ff6706edf996fa P 73c089c6be7a453a818ca37081639799 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:08.851562 21759 raft_consensus.cc:740] T 83f44d0e05dd4b1ea9ff6706edf996fa P 73c089c6be7a453a818ca37081639799 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73c089c6be7a453a818ca37081639799, State: Initialized, Role: FOLLOWER
I20260709 13:21:08.852499 21759 consensus_queue.cc:260] T 83f44d0e05dd4b1ea9ff6706edf996fa P 73c089c6be7a453a818ca37081639799 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } }
I20260709 13:21:08.853639 21762 tablet_bootstrap.cc:492] T 67e0d782f225460f8d7bf6e3a519e0c5 P c49029e9cb7546728f69e9c919b01d90: Bootstrap starting.
I20260709 13:21:08.853883 21760 tablet_bootstrap.cc:654] T 83f44d0e05dd4b1ea9ff6706edf996fa P 30f6f31e9a314e4099b7466a143ed434: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:08.855230 21744 heartbeater.cc:499] Master 127.9.116.190:45781 was elected leader, sending a full tablet report...
I20260709 13:21:08.855731 21759 ts_tablet_manager.cc:1434] T 83f44d0e05dd4b1ea9ff6706edf996fa P 73c089c6be7a453a818ca37081639799: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.002s
I20260709 13:21:08.856745 21759 tablet_bootstrap.cc:492] T 67e0d782f225460f8d7bf6e3a519e0c5 P 73c089c6be7a453a818ca37081639799: Bootstrap starting.
I20260709 13:21:08.863559 21762 tablet_bootstrap.cc:654] T 67e0d782f225460f8d7bf6e3a519e0c5 P c49029e9cb7546728f69e9c919b01d90: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:08.863564 21759 tablet_bootstrap.cc:654] T 67e0d782f225460f8d7bf6e3a519e0c5 P 73c089c6be7a453a818ca37081639799: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:08.875562 21760 tablet_bootstrap.cc:492] T 83f44d0e05dd4b1ea9ff6706edf996fa P 30f6f31e9a314e4099b7466a143ed434: No bootstrap required, opened a new log
I20260709 13:21:08.875676 21762 tablet_bootstrap.cc:492] T 67e0d782f225460f8d7bf6e3a519e0c5 P c49029e9cb7546728f69e9c919b01d90: No bootstrap required, opened a new log
I20260709 13:21:08.876094 21760 ts_tablet_manager.cc:1403] T 83f44d0e05dd4b1ea9ff6706edf996fa P 30f6f31e9a314e4099b7466a143ed434: Time spent bootstrapping tablet: real 0.032s	user 0.015s	sys 0.009s
I20260709 13:21:08.876273 21762 ts_tablet_manager.cc:1403] T 67e0d782f225460f8d7bf6e3a519e0c5 P c49029e9cb7546728f69e9c919b01d90: Time spent bootstrapping tablet: real 0.023s	user 0.020s	sys 0.000s
I20260709 13:21:08.876586 21759 tablet_bootstrap.cc:492] T 67e0d782f225460f8d7bf6e3a519e0c5 P 73c089c6be7a453a818ca37081639799: No bootstrap required, opened a new log
I20260709 13:21:08.877143 21759 ts_tablet_manager.cc:1403] T 67e0d782f225460f8d7bf6e3a519e0c5 P 73c089c6be7a453a818ca37081639799: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.004s
I20260709 13:21:08.879535 21762 raft_consensus.cc:359] T 67e0d782f225460f8d7bf6e3a519e0c5 P c49029e9cb7546728f69e9c919b01d90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } }
I20260709 13:21:08.879354 21760 raft_consensus.cc:359] T 83f44d0e05dd4b1ea9ff6706edf996fa P 30f6f31e9a314e4099b7466a143ed434 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } }
I20260709 13:21:08.880244 21762 raft_consensus.cc:385] T 67e0d782f225460f8d7bf6e3a519e0c5 P c49029e9cb7546728f69e9c919b01d90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:08.880455 21760 raft_consensus.cc:385] T 83f44d0e05dd4b1ea9ff6706edf996fa P 30f6f31e9a314e4099b7466a143ed434 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:08.880700 21762 raft_consensus.cc:740] T 67e0d782f225460f8d7bf6e3a519e0c5 P c49029e9cb7546728f69e9c919b01d90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c49029e9cb7546728f69e9c919b01d90, State: Initialized, Role: FOLLOWER
I20260709 13:21:08.880257 21759 raft_consensus.cc:359] T 67e0d782f225460f8d7bf6e3a519e0c5 P 73c089c6be7a453a818ca37081639799 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } }
I20260709 13:21:08.880964 21760 raft_consensus.cc:740] T 83f44d0e05dd4b1ea9ff6706edf996fa P 30f6f31e9a314e4099b7466a143ed434 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30f6f31e9a314e4099b7466a143ed434, State: Initialized, Role: FOLLOWER
I20260709 13:21:08.881351 21759 raft_consensus.cc:385] T 67e0d782f225460f8d7bf6e3a519e0c5 P 73c089c6be7a453a818ca37081639799 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:08.881703 21759 raft_consensus.cc:740] T 67e0d782f225460f8d7bf6e3a519e0c5 P 73c089c6be7a453a818ca37081639799 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73c089c6be7a453a818ca37081639799, State: Initialized, Role: FOLLOWER
I20260709 13:21:08.881589 21762 consensus_queue.cc:260] T 67e0d782f225460f8d7bf6e3a519e0c5 P c49029e9cb7546728f69e9c919b01d90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } }
I20260709 13:21:08.882138 21760 consensus_queue.cc:260] T 83f44d0e05dd4b1ea9ff6706edf996fa P 30f6f31e9a314e4099b7466a143ed434 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } }
I20260709 13:21:08.882668 21759 consensus_queue.cc:260] T 67e0d782f225460f8d7bf6e3a519e0c5 P 73c089c6be7a453a818ca37081639799 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } }
I20260709 13:21:08.885924 21759 ts_tablet_manager.cc:1434] T 67e0d782f225460f8d7bf6e3a519e0c5 P 73c089c6be7a453a818ca37081639799: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:21:08.888019 21670 heartbeater.cc:499] Master 127.9.116.190:45781 was elected leader, sending a full tablet report...
I20260709 13:21:08.888334 21760 ts_tablet_manager.cc:1434] T 83f44d0e05dd4b1ea9ff6706edf996fa P 30f6f31e9a314e4099b7466a143ed434: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.006s
I20260709 13:21:08.889500 21760 tablet_bootstrap.cc:492] T 67e0d782f225460f8d7bf6e3a519e0c5 P 30f6f31e9a314e4099b7466a143ed434: Bootstrap starting.
I20260709 13:21:08.898905 21760 tablet_bootstrap.cc:654] T 67e0d782f225460f8d7bf6e3a519e0c5 P 30f6f31e9a314e4099b7466a143ed434: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:08.902326 21595 heartbeater.cc:499] Master 127.9.116.190:45781 was elected leader, sending a full tablet report...
I20260709 13:21:08.902851 21762 ts_tablet_manager.cc:1434] T 67e0d782f225460f8d7bf6e3a519e0c5 P c49029e9cb7546728f69e9c919b01d90: Time spent starting tablet: real 0.026s	user 0.011s	sys 0.015s
I20260709 13:21:08.904021 21762 tablet_bootstrap.cc:492] T 83f44d0e05dd4b1ea9ff6706edf996fa P c49029e9cb7546728f69e9c919b01d90: Bootstrap starting.
I20260709 13:21:08.904280 21760 tablet_bootstrap.cc:492] T 67e0d782f225460f8d7bf6e3a519e0c5 P 30f6f31e9a314e4099b7466a143ed434: No bootstrap required, opened a new log
I20260709 13:21:08.904827 21760 ts_tablet_manager.cc:1403] T 67e0d782f225460f8d7bf6e3a519e0c5 P 30f6f31e9a314e4099b7466a143ed434: Time spent bootstrapping tablet: real 0.016s	user 0.008s	sys 0.001s
I20260709 13:21:08.907222 21760 raft_consensus.cc:359] T 67e0d782f225460f8d7bf6e3a519e0c5 P 30f6f31e9a314e4099b7466a143ed434 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } }
I20260709 13:21:08.907882 21760 raft_consensus.cc:385] T 67e0d782f225460f8d7bf6e3a519e0c5 P 30f6f31e9a314e4099b7466a143ed434 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:08.908130 21760 raft_consensus.cc:740] T 67e0d782f225460f8d7bf6e3a519e0c5 P 30f6f31e9a314e4099b7466a143ed434 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30f6f31e9a314e4099b7466a143ed434, State: Initialized, Role: FOLLOWER
I20260709 13:21:08.908861 21760 consensus_queue.cc:260] T 67e0d782f225460f8d7bf6e3a519e0c5 P 30f6f31e9a314e4099b7466a143ed434 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } }
I20260709 13:21:08.911027 21762 tablet_bootstrap.cc:654] T 83f44d0e05dd4b1ea9ff6706edf996fa P c49029e9cb7546728f69e9c919b01d90: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:08.911070 21760 ts_tablet_manager.cc:1434] T 67e0d782f225460f8d7bf6e3a519e0c5 P 30f6f31e9a314e4099b7466a143ed434: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.003s
I20260709 13:21:08.916277 21762 tablet_bootstrap.cc:492] T 83f44d0e05dd4b1ea9ff6706edf996fa P c49029e9cb7546728f69e9c919b01d90: No bootstrap required, opened a new log
I20260709 13:21:08.916884 21762 ts_tablet_manager.cc:1403] T 83f44d0e05dd4b1ea9ff6706edf996fa P c49029e9cb7546728f69e9c919b01d90: Time spent bootstrapping tablet: real 0.013s	user 0.006s	sys 0.004s
I20260709 13:21:08.920116 21762 raft_consensus.cc:359] T 83f44d0e05dd4b1ea9ff6706edf996fa P c49029e9cb7546728f69e9c919b01d90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } }
I20260709 13:21:08.920863 21762 raft_consensus.cc:385] T 83f44d0e05dd4b1ea9ff6706edf996fa P c49029e9cb7546728f69e9c919b01d90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:08.921201 21762 raft_consensus.cc:740] T 83f44d0e05dd4b1ea9ff6706edf996fa P c49029e9cb7546728f69e9c919b01d90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c49029e9cb7546728f69e9c919b01d90, State: Initialized, Role: FOLLOWER
I20260709 13:21:08.921919 21762 consensus_queue.cc:260] T 83f44d0e05dd4b1ea9ff6706edf996fa P c49029e9cb7546728f69e9c919b01d90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } }
I20260709 13:21:08.924186 21762 ts_tablet_manager.cc:1434] T 83f44d0e05dd4b1ea9ff6706edf996fa P c49029e9cb7546728f69e9c919b01d90: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:21:08.929060 21763 raft_consensus.cc:493] T 83f44d0e05dd4b1ea9ff6706edf996fa P 73c089c6be7a453a818ca37081639799 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:08.929687 21763 raft_consensus.cc:515] T 83f44d0e05dd4b1ea9ff6706edf996fa P 73c089c6be7a453a818ca37081639799 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } }
I20260709 13:21:08.932111 21763 leader_election.cc:290] T 83f44d0e05dd4b1ea9ff6706edf996fa P 73c089c6be7a453a818ca37081639799 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c49029e9cb7546728f69e9c919b01d90 (127.9.116.129:32957), 30f6f31e9a314e4099b7466a143ed434 (127.9.116.130:37109)
I20260709 13:21:08.942665 21570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f44d0e05dd4b1ea9ff6706edf996fa" candidate_uuid: "73c089c6be7a453a818ca37081639799" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c49029e9cb7546728f69e9c919b01d90" is_pre_election: true
I20260709 13:21:08.943568 21570 raft_consensus.cc:2468] T 83f44d0e05dd4b1ea9ff6706edf996fa P c49029e9cb7546728f69e9c919b01d90 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73c089c6be7a453a818ca37081639799 in term 0.
I20260709 13:21:08.944258 21645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f44d0e05dd4b1ea9ff6706edf996fa" candidate_uuid: "73c089c6be7a453a818ca37081639799" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f6f31e9a314e4099b7466a143ed434" is_pre_election: true
I20260709 13:21:08.944922 21685 leader_election.cc:304] T 83f44d0e05dd4b1ea9ff6706edf996fa P 73c089c6be7a453a818ca37081639799 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73c089c6be7a453a818ca37081639799, c49029e9cb7546728f69e9c919b01d90; no voters: 
I20260709 13:21:08.945048 21645 raft_consensus.cc:2468] T 83f44d0e05dd4b1ea9ff6706edf996fa P 30f6f31e9a314e4099b7466a143ed434 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73c089c6be7a453a818ca37081639799 in term 0.
I20260709 13:21:08.945780 21763 raft_consensus.cc:2804] T 83f44d0e05dd4b1ea9ff6706edf996fa P 73c089c6be7a453a818ca37081639799 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:08.946130 21763 raft_consensus.cc:493] T 83f44d0e05dd4b1ea9ff6706edf996fa P 73c089c6be7a453a818ca37081639799 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:08.946466 21763 raft_consensus.cc:3060] T 83f44d0e05dd4b1ea9ff6706edf996fa P 73c089c6be7a453a818ca37081639799 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:08.951781 21763 raft_consensus.cc:515] T 83f44d0e05dd4b1ea9ff6706edf996fa P 73c089c6be7a453a818ca37081639799 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } }
I20260709 13:21:08.953476 21763 leader_election.cc:290] T 83f44d0e05dd4b1ea9ff6706edf996fa P 73c089c6be7a453a818ca37081639799 [CANDIDATE]: Term 1 election: Requested vote from peers c49029e9cb7546728f69e9c919b01d90 (127.9.116.129:32957), 30f6f31e9a314e4099b7466a143ed434 (127.9.116.130:37109)
I20260709 13:21:08.954238 21570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f44d0e05dd4b1ea9ff6706edf996fa" candidate_uuid: "73c089c6be7a453a818ca37081639799" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c49029e9cb7546728f69e9c919b01d90"
I20260709 13:21:08.954586 21645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f44d0e05dd4b1ea9ff6706edf996fa" candidate_uuid: "73c089c6be7a453a818ca37081639799" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f6f31e9a314e4099b7466a143ed434"
I20260709 13:21:08.954978 21570 raft_consensus.cc:3060] T 83f44d0e05dd4b1ea9ff6706edf996fa P c49029e9cb7546728f69e9c919b01d90 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:08.955215 21645 raft_consensus.cc:3060] T 83f44d0e05dd4b1ea9ff6706edf996fa P 30f6f31e9a314e4099b7466a143ed434 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:08.962836 21645 raft_consensus.cc:2468] T 83f44d0e05dd4b1ea9ff6706edf996fa P 30f6f31e9a314e4099b7466a143ed434 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73c089c6be7a453a818ca37081639799 in term 1.
I20260709 13:21:08.962836 21570 raft_consensus.cc:2468] T 83f44d0e05dd4b1ea9ff6706edf996fa P c49029e9cb7546728f69e9c919b01d90 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73c089c6be7a453a818ca37081639799 in term 1.
I20260709 13:21:08.964218 21685 leader_election.cc:304] T 83f44d0e05dd4b1ea9ff6706edf996fa P 73c089c6be7a453a818ca37081639799 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73c089c6be7a453a818ca37081639799, c49029e9cb7546728f69e9c919b01d90; no voters: 
I20260709 13:21:08.964862 21763 raft_consensus.cc:2804] T 83f44d0e05dd4b1ea9ff6706edf996fa P 73c089c6be7a453a818ca37081639799 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:08.966125 21763 raft_consensus.cc:697] T 83f44d0e05dd4b1ea9ff6706edf996fa P 73c089c6be7a453a818ca37081639799 [term 1 LEADER]: Becoming Leader. State: Replica: 73c089c6be7a453a818ca37081639799, State: Running, Role: LEADER
I20260709 13:21:08.967586 21763 consensus_queue.cc:237] T 83f44d0e05dd4b1ea9ff6706edf996fa P 73c089c6be7a453a818ca37081639799 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } }
I20260709 13:21:08.975056 21466 catalog_manager.cc:5697] T 83f44d0e05dd4b1ea9ff6706edf996fa P 73c089c6be7a453a818ca37081639799 reported cstate change: term changed from 0 to 1, leader changed from <none> to 73c089c6be7a453a818ca37081639799 (127.9.116.131). New cstate: current_term: 1 leader_uuid: "73c089c6be7a453a818ca37081639799" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:09.203528 21767 raft_consensus.cc:493] T 67e0d782f225460f8d7bf6e3a519e0c5 P c49029e9cb7546728f69e9c919b01d90 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:09.204113 21767 raft_consensus.cc:515] T 67e0d782f225460f8d7bf6e3a519e0c5 P c49029e9cb7546728f69e9c919b01d90 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } }
I20260709 13:21:09.206135 21767 leader_election.cc:290] T 67e0d782f225460f8d7bf6e3a519e0c5 P c49029e9cb7546728f69e9c919b01d90 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30f6f31e9a314e4099b7466a143ed434 (127.9.116.130:37109), 73c089c6be7a453a818ca37081639799 (127.9.116.131:35735)
I20260709 13:21:09.217288 21719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e0d782f225460f8d7bf6e3a519e0c5" candidate_uuid: "c49029e9cb7546728f69e9c919b01d90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73c089c6be7a453a818ca37081639799" is_pre_election: true
I20260709 13:21:09.217316 21645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e0d782f225460f8d7bf6e3a519e0c5" candidate_uuid: "c49029e9cb7546728f69e9c919b01d90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f6f31e9a314e4099b7466a143ed434" is_pre_election: true
I20260709 13:21:09.218058 21719 raft_consensus.cc:2468] T 67e0d782f225460f8d7bf6e3a519e0c5 P 73c089c6be7a453a818ca37081639799 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c49029e9cb7546728f69e9c919b01d90 in term 0.
I20260709 13:21:09.218170 21645 raft_consensus.cc:2468] T 67e0d782f225460f8d7bf6e3a519e0c5 P 30f6f31e9a314e4099b7466a143ed434 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c49029e9cb7546728f69e9c919b01d90 in term 0.
I20260709 13:21:09.219259 21534 leader_election.cc:304] T 67e0d782f225460f8d7bf6e3a519e0c5 P c49029e9cb7546728f69e9c919b01d90 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73c089c6be7a453a818ca37081639799, c49029e9cb7546728f69e9c919b01d90; no voters: 
I20260709 13:21:09.220022 21767 raft_consensus.cc:2804] T 67e0d782f225460f8d7bf6e3a519e0c5 P c49029e9cb7546728f69e9c919b01d90 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:09.220386 21767 raft_consensus.cc:493] T 67e0d782f225460f8d7bf6e3a519e0c5 P c49029e9cb7546728f69e9c919b01d90 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:09.220680 21767 raft_consensus.cc:3060] T 67e0d782f225460f8d7bf6e3a519e0c5 P c49029e9cb7546728f69e9c919b01d90 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:09.228659 21767 raft_consensus.cc:515] T 67e0d782f225460f8d7bf6e3a519e0c5 P c49029e9cb7546728f69e9c919b01d90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } }
I20260709 13:21:09.230760 21767 leader_election.cc:290] T 67e0d782f225460f8d7bf6e3a519e0c5 P c49029e9cb7546728f69e9c919b01d90 [CANDIDATE]: Term 1 election: Requested vote from peers 30f6f31e9a314e4099b7466a143ed434 (127.9.116.130:37109), 73c089c6be7a453a818ca37081639799 (127.9.116.131:35735)
I20260709 13:21:09.231576 21645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e0d782f225460f8d7bf6e3a519e0c5" candidate_uuid: "c49029e9cb7546728f69e9c919b01d90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f6f31e9a314e4099b7466a143ed434"
I20260709 13:21:09.231782 21719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e0d782f225460f8d7bf6e3a519e0c5" candidate_uuid: "c49029e9cb7546728f69e9c919b01d90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73c089c6be7a453a818ca37081639799"
I20260709 13:21:09.232153 21645 raft_consensus.cc:3060] T 67e0d782f225460f8d7bf6e3a519e0c5 P 30f6f31e9a314e4099b7466a143ed434 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:09.232373 21719 raft_consensus.cc:3060] T 67e0d782f225460f8d7bf6e3a519e0c5 P 73c089c6be7a453a818ca37081639799 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:09.237787 21719 raft_consensus.cc:2468] T 67e0d782f225460f8d7bf6e3a519e0c5 P 73c089c6be7a453a818ca37081639799 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c49029e9cb7546728f69e9c919b01d90 in term 1.
I20260709 13:21:09.237875 21645 raft_consensus.cc:2468] T 67e0d782f225460f8d7bf6e3a519e0c5 P 30f6f31e9a314e4099b7466a143ed434 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c49029e9cb7546728f69e9c919b01d90 in term 1.
I20260709 13:21:09.238955 21534 leader_election.cc:304] T 67e0d782f225460f8d7bf6e3a519e0c5 P c49029e9cb7546728f69e9c919b01d90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73c089c6be7a453a818ca37081639799, c49029e9cb7546728f69e9c919b01d90; no voters: 
I20260709 13:21:09.239681 21767 raft_consensus.cc:2804] T 67e0d782f225460f8d7bf6e3a519e0c5 P c49029e9cb7546728f69e9c919b01d90 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:09.240628 21767 raft_consensus.cc:697] T 67e0d782f225460f8d7bf6e3a519e0c5 P c49029e9cb7546728f69e9c919b01d90 [term 1 LEADER]: Becoming Leader. State: Replica: c49029e9cb7546728f69e9c919b01d90, State: Running, Role: LEADER
I20260709 13:21:09.241493 21767 consensus_queue.cc:237] T 67e0d782f225460f8d7bf6e3a519e0c5 P c49029e9cb7546728f69e9c919b01d90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } }
I20260709 13:21:09.248152 21466 catalog_manager.cc:5697] T 67e0d782f225460f8d7bf6e3a519e0c5 P c49029e9cb7546728f69e9c919b01d90 reported cstate change: term changed from 0 to 1, leader changed from <none> to c49029e9cb7546728f69e9c919b01d90 (127.9.116.129). New cstate: current_term: 1 leader_uuid: "c49029e9cb7546728f69e9c919b01d90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:09.306455 21466 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43546:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:21:09.308553 21466 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:21:09.333722 21559 tablet_service.cc:1511] Processing CreateTablet for tablet f38dbb095cde46cfbcca5917aac7d453 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=2f2e4db727fa45de9bcf17cea61dc846]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:21:09.335099 21559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f38dbb095cde46cfbcca5917aac7d453. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:09.335047 21634 tablet_service.cc:1511] Processing CreateTablet for tablet f38dbb095cde46cfbcca5917aac7d453 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=2f2e4db727fa45de9bcf17cea61dc846]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:21:09.336472 21634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f38dbb095cde46cfbcca5917aac7d453. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:09.336716 21708 tablet_service.cc:1511] Processing CreateTablet for tablet f38dbb095cde46cfbcca5917aac7d453 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=2f2e4db727fa45de9bcf17cea61dc846]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:21:09.337947 21708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f38dbb095cde46cfbcca5917aac7d453. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:09.345031 21560 tablet_service.cc:1511] Processing CreateTablet for tablet 32eb13c472e44e079221b8c6ec099fcb (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=2f2e4db727fa45de9bcf17cea61dc846]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:21:09.345438 21635 tablet_service.cc:1511] Processing CreateTablet for tablet 32eb13c472e44e079221b8c6ec099fcb (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=2f2e4db727fa45de9bcf17cea61dc846]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:21:09.346403 21560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32eb13c472e44e079221b8c6ec099fcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:09.348495 21635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32eb13c472e44e079221b8c6ec099fcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:09.349347 21709 tablet_service.cc:1511] Processing CreateTablet for tablet 32eb13c472e44e079221b8c6ec099fcb (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=2f2e4db727fa45de9bcf17cea61dc846]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:21:09.350301 21762 tablet_bootstrap.cc:492] T f38dbb095cde46cfbcca5917aac7d453 P c49029e9cb7546728f69e9c919b01d90: Bootstrap starting.
I20260709 13:21:09.351688 21709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32eb13c472e44e079221b8c6ec099fcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:09.357028 21762 tablet_bootstrap.cc:654] T f38dbb095cde46cfbcca5917aac7d453 P c49029e9cb7546728f69e9c919b01d90: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:09.366392 21762 tablet_bootstrap.cc:492] T f38dbb095cde46cfbcca5917aac7d453 P c49029e9cb7546728f69e9c919b01d90: No bootstrap required, opened a new log
I20260709 13:21:09.367034 21762 ts_tablet_manager.cc:1403] T f38dbb095cde46cfbcca5917aac7d453 P c49029e9cb7546728f69e9c919b01d90: Time spent bootstrapping tablet: real 0.017s	user 0.008s	sys 0.006s
I20260709 13:21:09.369192 21759 tablet_bootstrap.cc:492] T 32eb13c472e44e079221b8c6ec099fcb P 73c089c6be7a453a818ca37081639799: Bootstrap starting.
I20260709 13:21:09.370193 21762 raft_consensus.cc:359] T f38dbb095cde46cfbcca5917aac7d453 P c49029e9cb7546728f69e9c919b01d90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } }
I20260709 13:21:09.371019 21762 raft_consensus.cc:385] T f38dbb095cde46cfbcca5917aac7d453 P c49029e9cb7546728f69e9c919b01d90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:09.371322 21762 raft_consensus.cc:740] T f38dbb095cde46cfbcca5917aac7d453 P c49029e9cb7546728f69e9c919b01d90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c49029e9cb7546728f69e9c919b01d90, State: Initialized, Role: FOLLOWER
I20260709 13:21:09.371510 21760 tablet_bootstrap.cc:492] T f38dbb095cde46cfbcca5917aac7d453 P 30f6f31e9a314e4099b7466a143ed434: Bootstrap starting.
I20260709 13:21:09.372159 21762 consensus_queue.cc:260] T f38dbb095cde46cfbcca5917aac7d453 P c49029e9cb7546728f69e9c919b01d90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } }
I20260709 13:21:09.377206 21762 ts_tablet_manager.cc:1434] T f38dbb095cde46cfbcca5917aac7d453 P c49029e9cb7546728f69e9c919b01d90: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 13:21:09.380999 21762 tablet_bootstrap.cc:492] T 32eb13c472e44e079221b8c6ec099fcb P c49029e9cb7546728f69e9c919b01d90: Bootstrap starting.
I20260709 13:21:09.379668 21760 tablet_bootstrap.cc:654] T f38dbb095cde46cfbcca5917aac7d453 P 30f6f31e9a314e4099b7466a143ed434: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:09.379791 21759 tablet_bootstrap.cc:654] T 32eb13c472e44e079221b8c6ec099fcb P 73c089c6be7a453a818ca37081639799: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:09.387921 21762 tablet_bootstrap.cc:654] T 32eb13c472e44e079221b8c6ec099fcb P c49029e9cb7546728f69e9c919b01d90: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:09.392349 21759 tablet_bootstrap.cc:492] T 32eb13c472e44e079221b8c6ec099fcb P 73c089c6be7a453a818ca37081639799: No bootstrap required, opened a new log
I20260709 13:21:09.392805 21760 tablet_bootstrap.cc:492] T f38dbb095cde46cfbcca5917aac7d453 P 30f6f31e9a314e4099b7466a143ed434: No bootstrap required, opened a new log
I20260709 13:21:09.392953 21759 ts_tablet_manager.cc:1403] T 32eb13c472e44e079221b8c6ec099fcb P 73c089c6be7a453a818ca37081639799: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.002s
I20260709 13:21:09.393435 21760 ts_tablet_manager.cc:1403] T f38dbb095cde46cfbcca5917aac7d453 P 30f6f31e9a314e4099b7466a143ed434: Time spent bootstrapping tablet: real 0.022s	user 0.007s	sys 0.005s
I20260709 13:21:09.396248 21759 raft_consensus.cc:359] T 32eb13c472e44e079221b8c6ec099fcb P 73c089c6be7a453a818ca37081639799 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } }
I20260709 13:21:09.396497 21760 raft_consensus.cc:359] T f38dbb095cde46cfbcca5917aac7d453 P 30f6f31e9a314e4099b7466a143ed434 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } }
I20260709 13:21:09.397212 21759 raft_consensus.cc:385] T 32eb13c472e44e079221b8c6ec099fcb P 73c089c6be7a453a818ca37081639799 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:09.397326 21760 raft_consensus.cc:385] T f38dbb095cde46cfbcca5917aac7d453 P 30f6f31e9a314e4099b7466a143ed434 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:09.397624 21759 raft_consensus.cc:740] T 32eb13c472e44e079221b8c6ec099fcb P 73c089c6be7a453a818ca37081639799 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73c089c6be7a453a818ca37081639799, State: Initialized, Role: FOLLOWER
I20260709 13:21:09.397675 21760 raft_consensus.cc:740] T f38dbb095cde46cfbcca5917aac7d453 P 30f6f31e9a314e4099b7466a143ed434 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30f6f31e9a314e4099b7466a143ed434, State: Initialized, Role: FOLLOWER
I20260709 13:21:09.398609 21759 consensus_queue.cc:260] T 32eb13c472e44e079221b8c6ec099fcb P 73c089c6be7a453a818ca37081639799 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } }
I20260709 13:21:09.399482 21760 consensus_queue.cc:260] T f38dbb095cde46cfbcca5917aac7d453 P 30f6f31e9a314e4099b7466a143ed434 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } }
I20260709 13:21:09.400969 21762 tablet_bootstrap.cc:492] T 32eb13c472e44e079221b8c6ec099fcb P c49029e9cb7546728f69e9c919b01d90: No bootstrap required, opened a new log
I20260709 13:21:09.401005 21759 ts_tablet_manager.cc:1434] T 32eb13c472e44e079221b8c6ec099fcb P 73c089c6be7a453a818ca37081639799: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 13:21:09.401667 21762 ts_tablet_manager.cc:1403] T 32eb13c472e44e079221b8c6ec099fcb P c49029e9cb7546728f69e9c919b01d90: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.005s
I20260709 13:21:09.401854 21760 ts_tablet_manager.cc:1434] T f38dbb095cde46cfbcca5917aac7d453 P 30f6f31e9a314e4099b7466a143ed434: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:21:09.402321 21759 tablet_bootstrap.cc:492] T f38dbb095cde46cfbcca5917aac7d453 P 73c089c6be7a453a818ca37081639799: Bootstrap starting.
I20260709 13:21:09.403023 21760 tablet_bootstrap.cc:492] T 32eb13c472e44e079221b8c6ec099fcb P 30f6f31e9a314e4099b7466a143ed434: Bootstrap starting.
I20260709 13:21:09.404948 21762 raft_consensus.cc:359] T 32eb13c472e44e079221b8c6ec099fcb P c49029e9cb7546728f69e9c919b01d90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } }
I20260709 13:21:09.405742 21762 raft_consensus.cc:385] T 32eb13c472e44e079221b8c6ec099fcb P c49029e9cb7546728f69e9c919b01d90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:09.406075 21762 raft_consensus.cc:740] T 32eb13c472e44e079221b8c6ec099fcb P c49029e9cb7546728f69e9c919b01d90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c49029e9cb7546728f69e9c919b01d90, State: Initialized, Role: FOLLOWER
I20260709 13:21:09.407014 21762 consensus_queue.cc:260] T 32eb13c472e44e079221b8c6ec099fcb P c49029e9cb7546728f69e9c919b01d90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } }
I20260709 13:21:09.409448 21762 ts_tablet_manager.cc:1434] T 32eb13c472e44e079221b8c6ec099fcb P c49029e9cb7546728f69e9c919b01d90: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:21:09.411620 21759 tablet_bootstrap.cc:654] T f38dbb095cde46cfbcca5917aac7d453 P 73c089c6be7a453a818ca37081639799: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:09.413475 21760 tablet_bootstrap.cc:654] T 32eb13c472e44e079221b8c6ec099fcb P 30f6f31e9a314e4099b7466a143ed434: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:09.422533 21760 tablet_bootstrap.cc:492] T 32eb13c472e44e079221b8c6ec099fcb P 30f6f31e9a314e4099b7466a143ed434: No bootstrap required, opened a new log
I20260709 13:21:09.423182 21760 ts_tablet_manager.cc:1403] T 32eb13c472e44e079221b8c6ec099fcb P 30f6f31e9a314e4099b7466a143ed434: Time spent bootstrapping tablet: real 0.020s	user 0.010s	sys 0.002s
I20260709 13:21:09.425804 21760 raft_consensus.cc:359] T 32eb13c472e44e079221b8c6ec099fcb P 30f6f31e9a314e4099b7466a143ed434 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } }
I20260709 13:21:09.426541 21760 raft_consensus.cc:385] T 32eb13c472e44e079221b8c6ec099fcb P 30f6f31e9a314e4099b7466a143ed434 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:09.426942 21760 raft_consensus.cc:740] T 32eb13c472e44e079221b8c6ec099fcb P 30f6f31e9a314e4099b7466a143ed434 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30f6f31e9a314e4099b7466a143ed434, State: Initialized, Role: FOLLOWER
I20260709 13:21:09.427801 21760 consensus_queue.cc:260] T 32eb13c472e44e079221b8c6ec099fcb P 30f6f31e9a314e4099b7466a143ed434 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } }
I20260709 13:21:09.429600 21760 ts_tablet_manager.cc:1434] T 32eb13c472e44e079221b8c6ec099fcb P 30f6f31e9a314e4099b7466a143ed434: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.001s
I20260709 13:21:09.435650 21759 tablet_bootstrap.cc:492] T f38dbb095cde46cfbcca5917aac7d453 P 73c089c6be7a453a818ca37081639799: No bootstrap required, opened a new log
I20260709 13:21:09.436265 21759 ts_tablet_manager.cc:1403] T f38dbb095cde46cfbcca5917aac7d453 P 73c089c6be7a453a818ca37081639799: Time spent bootstrapping tablet: real 0.034s	user 0.016s	sys 0.013s
I20260709 13:21:09.439376 21759 raft_consensus.cc:359] T f38dbb095cde46cfbcca5917aac7d453 P 73c089c6be7a453a818ca37081639799 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } }
I20260709 13:21:09.439893 21759 raft_consensus.cc:385] T f38dbb095cde46cfbcca5917aac7d453 P 73c089c6be7a453a818ca37081639799 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:09.440140 21759 raft_consensus.cc:740] T f38dbb095cde46cfbcca5917aac7d453 P 73c089c6be7a453a818ca37081639799 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73c089c6be7a453a818ca37081639799, State: Initialized, Role: FOLLOWER
I20260709 13:21:09.440900 21759 consensus_queue.cc:260] T f38dbb095cde46cfbcca5917aac7d453 P 73c089c6be7a453a818ca37081639799 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } }
I20260709 13:21:09.442657 21759 ts_tablet_manager.cc:1434] T f38dbb095cde46cfbcca5917aac7d453 P 73c089c6be7a453a818ca37081639799: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.003s
I20260709 13:21:09.443274 21767 raft_consensus.cc:493] T 32eb13c472e44e079221b8c6ec099fcb P c49029e9cb7546728f69e9c919b01d90 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:09.443792 21767 raft_consensus.cc:515] T 32eb13c472e44e079221b8c6ec099fcb P c49029e9cb7546728f69e9c919b01d90 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } }
I20260709 13:21:09.445936 21767 leader_election.cc:290] T 32eb13c472e44e079221b8c6ec099fcb P c49029e9cb7546728f69e9c919b01d90 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30f6f31e9a314e4099b7466a143ed434 (127.9.116.130:37109), 73c089c6be7a453a818ca37081639799 (127.9.116.131:35735)
I20260709 13:21:09.446579 21645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32eb13c472e44e079221b8c6ec099fcb" candidate_uuid: "c49029e9cb7546728f69e9c919b01d90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f6f31e9a314e4099b7466a143ed434" is_pre_election: true
I20260709 13:21:09.447252 21645 raft_consensus.cc:2468] T 32eb13c472e44e079221b8c6ec099fcb P 30f6f31e9a314e4099b7466a143ed434 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c49029e9cb7546728f69e9c919b01d90 in term 0.
I20260709 13:21:09.448287 21536 leader_election.cc:304] T 32eb13c472e44e079221b8c6ec099fcb P c49029e9cb7546728f69e9c919b01d90 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30f6f31e9a314e4099b7466a143ed434, c49029e9cb7546728f69e9c919b01d90; no voters: 
I20260709 13:21:09.449030 21719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32eb13c472e44e079221b8c6ec099fcb" candidate_uuid: "c49029e9cb7546728f69e9c919b01d90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73c089c6be7a453a818ca37081639799" is_pre_election: true
I20260709 13:21:09.449093 21767 raft_consensus.cc:2804] T 32eb13c472e44e079221b8c6ec099fcb P c49029e9cb7546728f69e9c919b01d90 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:09.449667 21767 raft_consensus.cc:493] T 32eb13c472e44e079221b8c6ec099fcb P c49029e9cb7546728f69e9c919b01d90 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:09.449709 21719 raft_consensus.cc:2468] T 32eb13c472e44e079221b8c6ec099fcb P 73c089c6be7a453a818ca37081639799 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c49029e9cb7546728f69e9c919b01d90 in term 0.
I20260709 13:21:09.450138 21767 raft_consensus.cc:3060] T 32eb13c472e44e079221b8c6ec099fcb P c49029e9cb7546728f69e9c919b01d90 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:09.456699 21767 raft_consensus.cc:515] T 32eb13c472e44e079221b8c6ec099fcb P c49029e9cb7546728f69e9c919b01d90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } }
I20260709 13:21:09.458554 21767 leader_election.cc:290] T 32eb13c472e44e079221b8c6ec099fcb P c49029e9cb7546728f69e9c919b01d90 [CANDIDATE]: Term 1 election: Requested vote from peers 30f6f31e9a314e4099b7466a143ed434 (127.9.116.130:37109), 73c089c6be7a453a818ca37081639799 (127.9.116.131:35735)
I20260709 13:21:09.459340 21645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32eb13c472e44e079221b8c6ec099fcb" candidate_uuid: "c49029e9cb7546728f69e9c919b01d90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f6f31e9a314e4099b7466a143ed434"
I20260709 13:21:09.459657 21719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32eb13c472e44e079221b8c6ec099fcb" candidate_uuid: "c49029e9cb7546728f69e9c919b01d90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73c089c6be7a453a818ca37081639799"
I20260709 13:21:09.460027 21645 raft_consensus.cc:3060] T 32eb13c472e44e079221b8c6ec099fcb P 30f6f31e9a314e4099b7466a143ed434 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:09.460249 21719 raft_consensus.cc:3060] T 32eb13c472e44e079221b8c6ec099fcb P 73c089c6be7a453a818ca37081639799 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:09.462919 21763 consensus_queue.cc:1048] T 83f44d0e05dd4b1ea9ff6706edf996fa P 73c089c6be7a453a818ca37081639799 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:09.467386 21719 raft_consensus.cc:2468] T 32eb13c472e44e079221b8c6ec099fcb P 73c089c6be7a453a818ca37081639799 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c49029e9cb7546728f69e9c919b01d90 in term 1.
I20260709 13:21:09.467386 21645 raft_consensus.cc:2468] T 32eb13c472e44e079221b8c6ec099fcb P 30f6f31e9a314e4099b7466a143ed434 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c49029e9cb7546728f69e9c919b01d90 in term 1.
I20260709 13:21:09.469084 21534 leader_election.cc:304] T 32eb13c472e44e079221b8c6ec099fcb P c49029e9cb7546728f69e9c919b01d90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73c089c6be7a453a818ca37081639799, c49029e9cb7546728f69e9c919b01d90; no voters: 
I20260709 13:21:09.469951 21767 raft_consensus.cc:2804] T 32eb13c472e44e079221b8c6ec099fcb P c49029e9cb7546728f69e9c919b01d90 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:09.470427 21767 raft_consensus.cc:697] T 32eb13c472e44e079221b8c6ec099fcb P c49029e9cb7546728f69e9c919b01d90 [term 1 LEADER]: Becoming Leader. State: Replica: c49029e9cb7546728f69e9c919b01d90, State: Running, Role: LEADER
I20260709 13:21:09.471323 21767 consensus_queue.cc:237] T 32eb13c472e44e079221b8c6ec099fcb P c49029e9cb7546728f69e9c919b01d90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } }
I20260709 13:21:09.479583 21468 catalog_manager.cc:5697] T 32eb13c472e44e079221b8c6ec099fcb P c49029e9cb7546728f69e9c919b01d90 reported cstate change: term changed from 0 to 1, leader changed from <none> to c49029e9cb7546728f69e9c919b01d90 (127.9.116.129). New cstate: current_term: 1 leader_uuid: "c49029e9cb7546728f69e9c919b01d90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:09.485622 21795 consensus_queue.cc:1048] T 83f44d0e05dd4b1ea9ff6706edf996fa P 73c089c6be7a453a818ca37081639799 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:09.656133 21795 raft_consensus.cc:493] T f38dbb095cde46cfbcca5917aac7d453 P 73c089c6be7a453a818ca37081639799 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:09.656581 21795 raft_consensus.cc:515] T f38dbb095cde46cfbcca5917aac7d453 P 73c089c6be7a453a818ca37081639799 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } }
I20260709 13:21:09.658486 21795 leader_election.cc:290] T f38dbb095cde46cfbcca5917aac7d453 P 73c089c6be7a453a818ca37081639799 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c49029e9cb7546728f69e9c919b01d90 (127.9.116.129:32957), 30f6f31e9a314e4099b7466a143ed434 (127.9.116.130:37109)
I20260709 13:21:09.659247 21570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f38dbb095cde46cfbcca5917aac7d453" candidate_uuid: "73c089c6be7a453a818ca37081639799" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c49029e9cb7546728f69e9c919b01d90" is_pre_election: true
I20260709 13:21:09.659592 21645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f38dbb095cde46cfbcca5917aac7d453" candidate_uuid: "73c089c6be7a453a818ca37081639799" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f6f31e9a314e4099b7466a143ed434" is_pre_election: true
I20260709 13:21:09.659936 21570 raft_consensus.cc:2468] T f38dbb095cde46cfbcca5917aac7d453 P c49029e9cb7546728f69e9c919b01d90 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73c089c6be7a453a818ca37081639799 in term 0.
I20260709 13:21:09.660276 21645 raft_consensus.cc:2468] T f38dbb095cde46cfbcca5917aac7d453 P 30f6f31e9a314e4099b7466a143ed434 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73c089c6be7a453a818ca37081639799 in term 0.
I20260709 13:21:09.661082 21685 leader_election.cc:304] T f38dbb095cde46cfbcca5917aac7d453 P 73c089c6be7a453a818ca37081639799 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73c089c6be7a453a818ca37081639799, c49029e9cb7546728f69e9c919b01d90; no voters: 
I20260709 13:21:09.661808 21795 raft_consensus.cc:2804] T f38dbb095cde46cfbcca5917aac7d453 P 73c089c6be7a453a818ca37081639799 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:09.662168 21795 raft_consensus.cc:493] T f38dbb095cde46cfbcca5917aac7d453 P 73c089c6be7a453a818ca37081639799 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:09.662492 21795 raft_consensus.cc:3060] T f38dbb095cde46cfbcca5917aac7d453 P 73c089c6be7a453a818ca37081639799 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:09.667693 21795 raft_consensus.cc:515] T f38dbb095cde46cfbcca5917aac7d453 P 73c089c6be7a453a818ca37081639799 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } }
I20260709 13:21:09.669447 21795 leader_election.cc:290] T f38dbb095cde46cfbcca5917aac7d453 P 73c089c6be7a453a818ca37081639799 [CANDIDATE]: Term 1 election: Requested vote from peers c49029e9cb7546728f69e9c919b01d90 (127.9.116.129:32957), 30f6f31e9a314e4099b7466a143ed434 (127.9.116.130:37109)
I20260709 13:21:09.670262 21570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f38dbb095cde46cfbcca5917aac7d453" candidate_uuid: "73c089c6be7a453a818ca37081639799" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c49029e9cb7546728f69e9c919b01d90"
I20260709 13:21:09.670521 21645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f38dbb095cde46cfbcca5917aac7d453" candidate_uuid: "73c089c6be7a453a818ca37081639799" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f6f31e9a314e4099b7466a143ed434"
I20260709 13:21:09.671028 21570 raft_consensus.cc:3060] T f38dbb095cde46cfbcca5917aac7d453 P c49029e9cb7546728f69e9c919b01d90 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:09.671085 21645 raft_consensus.cc:3060] T f38dbb095cde46cfbcca5917aac7d453 P 30f6f31e9a314e4099b7466a143ed434 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:09.678254 21570 raft_consensus.cc:2468] T f38dbb095cde46cfbcca5917aac7d453 P c49029e9cb7546728f69e9c919b01d90 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73c089c6be7a453a818ca37081639799 in term 1.
I20260709 13:21:09.678254 21645 raft_consensus.cc:2468] T f38dbb095cde46cfbcca5917aac7d453 P 30f6f31e9a314e4099b7466a143ed434 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73c089c6be7a453a818ca37081639799 in term 1.
I20260709 13:21:09.679535 21685 leader_election.cc:304] T f38dbb095cde46cfbcca5917aac7d453 P 73c089c6be7a453a818ca37081639799 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73c089c6be7a453a818ca37081639799, c49029e9cb7546728f69e9c919b01d90; no voters: 
I20260709 13:21:09.680173 21795 raft_consensus.cc:2804] T f38dbb095cde46cfbcca5917aac7d453 P 73c089c6be7a453a818ca37081639799 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:09.680605 21795 raft_consensus.cc:697] T f38dbb095cde46cfbcca5917aac7d453 P 73c089c6be7a453a818ca37081639799 [term 1 LEADER]: Becoming Leader. State: Replica: 73c089c6be7a453a818ca37081639799, State: Running, Role: LEADER
I20260709 13:21:09.681277 21795 consensus_queue.cc:237] T f38dbb095cde46cfbcca5917aac7d453 P 73c089c6be7a453a818ca37081639799 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } }
I20260709 13:21:09.688972 21466 catalog_manager.cc:5697] T f38dbb095cde46cfbcca5917aac7d453 P 73c089c6be7a453a818ca37081639799 reported cstate change: term changed from 0 to 1, leader changed from <none> to 73c089c6be7a453a818ca37081639799 (127.9.116.131). New cstate: current_term: 1 leader_uuid: "73c089c6be7a453a818ca37081639799" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:09.701684 21767 consensus_queue.cc:1048] T 67e0d782f225460f8d7bf6e3a519e0c5 P c49029e9cb7546728f69e9c919b01d90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:09.713397 21767 consensus_queue.cc:1048] T 67e0d782f225460f8d7bf6e3a519e0c5 P c49029e9cb7546728f69e9c919b01d90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:09.763996 21719 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "83f44d0e05dd4b1ea9ff6706edf996fa"
dest_uuid: "73c089c6be7a453a818ca37081639799"
mode: GRACEFUL
new_leader_uuid: "c49029e9cb7546728f69e9c919b01d90"
 from {username='slave'} at 127.0.0.1:53520
I20260709 13:21:09.764580 21719 raft_consensus.cc:606] T 83f44d0e05dd4b1ea9ff6706edf996fa P 73c089c6be7a453a818ca37081639799 [term 1 LEADER]: Received request to transfer leadership to TS c49029e9cb7546728f69e9c919b01d90
I20260709 13:21:09.778257 21570 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "67e0d782f225460f8d7bf6e3a519e0c5"
dest_uuid: "c49029e9cb7546728f69e9c919b01d90"
mode: GRACEFUL
new_leader_uuid: "30f6f31e9a314e4099b7466a143ed434"
 from {username='slave'} at 127.0.0.1:33480
I20260709 13:21:09.778885 21570 raft_consensus.cc:606] T 67e0d782f225460f8d7bf6e3a519e0c5 P c49029e9cb7546728f69e9c919b01d90 [term 1 LEADER]: Received request to transfer leadership to TS 30f6f31e9a314e4099b7466a143ed434
I20260709 13:21:09.785012 21719 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f38dbb095cde46cfbcca5917aac7d453"
dest_uuid: "73c089c6be7a453a818ca37081639799"
mode: GRACEFUL
new_leader_uuid: "c49029e9cb7546728f69e9c919b01d90"
 from {username='slave'} at 127.0.0.1:53520
I20260709 13:21:09.785558 21719 raft_consensus.cc:606] T f38dbb095cde46cfbcca5917aac7d453 P 73c089c6be7a453a818ca37081639799 [term 1 LEADER]: Received request to transfer leadership to TS c49029e9cb7546728f69e9c919b01d90
I20260709 13:21:09.889891 21791 raft_consensus.cc:993] T 83f44d0e05dd4b1ea9ff6706edf996fa P 73c089c6be7a453a818ca37081639799: : Instructing follower c49029e9cb7546728f69e9c919b01d90 to start an election
I20260709 13:21:09.890424 21795 raft_consensus.cc:1081] T 83f44d0e05dd4b1ea9ff6706edf996fa P 73c089c6be7a453a818ca37081639799 [term 1 LEADER]: Signalling peer c49029e9cb7546728f69e9c919b01d90 to start an election
I20260709 13:21:09.892123 21570 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "83f44d0e05dd4b1ea9ff6706edf996fa"
dest_uuid: "c49029e9cb7546728f69e9c919b01d90"
 from {username='slave'} at 127.0.0.1:33470
I20260709 13:21:09.892740 21570 raft_consensus.cc:493] T 83f44d0e05dd4b1ea9ff6706edf996fa P c49029e9cb7546728f69e9c919b01d90 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:09.893131 21570 raft_consensus.cc:3060] T 83f44d0e05dd4b1ea9ff6706edf996fa P c49029e9cb7546728f69e9c919b01d90 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:09.898895 21570 raft_consensus.cc:515] T 83f44d0e05dd4b1ea9ff6706edf996fa P c49029e9cb7546728f69e9c919b01d90 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } }
I20260709 13:21:09.900774 21570 leader_election.cc:290] T 83f44d0e05dd4b1ea9ff6706edf996fa P c49029e9cb7546728f69e9c919b01d90 [CANDIDATE]: Term 2 election: Requested vote from peers 73c089c6be7a453a818ca37081639799 (127.9.116.131:35735), 30f6f31e9a314e4099b7466a143ed434 (127.9.116.130:37109)
I20260709 13:21:09.901795 21645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f44d0e05dd4b1ea9ff6706edf996fa" candidate_uuid: "c49029e9cb7546728f69e9c919b01d90" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "30f6f31e9a314e4099b7466a143ed434"
I20260709 13:21:09.901955 21719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f44d0e05dd4b1ea9ff6706edf996fa" candidate_uuid: "c49029e9cb7546728f69e9c919b01d90" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "73c089c6be7a453a818ca37081639799"
I20260709 13:21:09.902576 21645 raft_consensus.cc:3060] T 83f44d0e05dd4b1ea9ff6706edf996fa P 30f6f31e9a314e4099b7466a143ed434 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:09.902652 21719 raft_consensus.cc:3055] T 83f44d0e05dd4b1ea9ff6706edf996fa P 73c089c6be7a453a818ca37081639799 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:09.903120 21719 raft_consensus.cc:740] T 83f44d0e05dd4b1ea9ff6706edf996fa P 73c089c6be7a453a818ca37081639799 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 73c089c6be7a453a818ca37081639799, State: Running, Role: LEADER
I20260709 13:21:09.903978 21719 consensus_queue.cc:260] T 83f44d0e05dd4b1ea9ff6706edf996fa P 73c089c6be7a453a818ca37081639799 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } }
I20260709 13:21:09.905310 21719 raft_consensus.cc:3060] T 83f44d0e05dd4b1ea9ff6706edf996fa P 73c089c6be7a453a818ca37081639799 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:09.910163 21645 raft_consensus.cc:2468] T 83f44d0e05dd4b1ea9ff6706edf996fa P 30f6f31e9a314e4099b7466a143ed434 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c49029e9cb7546728f69e9c919b01d90 in term 2.
I20260709 13:21:09.911664 21536 leader_election.cc:304] T 83f44d0e05dd4b1ea9ff6706edf996fa P c49029e9cb7546728f69e9c919b01d90 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30f6f31e9a314e4099b7466a143ed434, c49029e9cb7546728f69e9c919b01d90; no voters: 
I20260709 13:21:09.912472 21767 raft_consensus.cc:2804] T 83f44d0e05dd4b1ea9ff6706edf996fa P c49029e9cb7546728f69e9c919b01d90 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:09.912902 21767 raft_consensus.cc:697] T 83f44d0e05dd4b1ea9ff6706edf996fa P c49029e9cb7546728f69e9c919b01d90 [term 2 LEADER]: Becoming Leader. State: Replica: c49029e9cb7546728f69e9c919b01d90, State: Running, Role: LEADER
I20260709 13:21:09.913002 21719 raft_consensus.cc:2468] T 83f44d0e05dd4b1ea9ff6706edf996fa P 73c089c6be7a453a818ca37081639799 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c49029e9cb7546728f69e9c919b01d90 in term 2.
I20260709 13:21:09.913705 21767 consensus_queue.cc:237] T 83f44d0e05dd4b1ea9ff6706edf996fa P c49029e9cb7546728f69e9c919b01d90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } }
I20260709 13:21:09.920147 21468 catalog_manager.cc:5697] T 83f44d0e05dd4b1ea9ff6706edf996fa P c49029e9cb7546728f69e9c919b01d90 reported cstate change: term changed from 1 to 2, leader changed from 73c089c6be7a453a818ca37081639799 (127.9.116.131) to c49029e9cb7546728f69e9c919b01d90 (127.9.116.129). New cstate: current_term: 2 leader_uuid: "c49029e9cb7546728f69e9c919b01d90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:09.996549 21767 consensus_queue.cc:1048] T 32eb13c472e44e079221b8c6ec099fcb P c49029e9cb7546728f69e9c919b01d90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:10.002310 21767 consensus_queue.cc:1048] T 32eb13c472e44e079221b8c6ec099fcb P c49029e9cb7546728f69e9c919b01d90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:10.076486 21795 consensus_queue.cc:1048] T f38dbb095cde46cfbcca5917aac7d453 P 73c089c6be7a453a818ca37081639799 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:10.088882 21791 consensus_queue.cc:1048] T f38dbb095cde46cfbcca5917aac7d453 P 73c089c6be7a453a818ca37081639799 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:10.098703 21795 raft_consensus.cc:993] T f38dbb095cde46cfbcca5917aac7d453 P 73c089c6be7a453a818ca37081639799: : Instructing follower c49029e9cb7546728f69e9c919b01d90 to start an election
I20260709 13:21:10.099206 21791 raft_consensus.cc:1081] T f38dbb095cde46cfbcca5917aac7d453 P 73c089c6be7a453a818ca37081639799 [term 1 LEADER]: Signalling peer c49029e9cb7546728f69e9c919b01d90 to start an election
I20260709 13:21:10.100941 21570 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f38dbb095cde46cfbcca5917aac7d453"
dest_uuid: "c49029e9cb7546728f69e9c919b01d90"
 from {username='slave'} at 127.0.0.1:33470
I20260709 13:21:10.101491 21570 raft_consensus.cc:493] T f38dbb095cde46cfbcca5917aac7d453 P c49029e9cb7546728f69e9c919b01d90 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:10.101886 21570 raft_consensus.cc:3060] T f38dbb095cde46cfbcca5917aac7d453 P c49029e9cb7546728f69e9c919b01d90 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:10.106936 21570 raft_consensus.cc:515] T f38dbb095cde46cfbcca5917aac7d453 P c49029e9cb7546728f69e9c919b01d90 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } }
I20260709 13:21:10.108631 21570 leader_election.cc:290] T f38dbb095cde46cfbcca5917aac7d453 P c49029e9cb7546728f69e9c919b01d90 [CANDIDATE]: Term 2 election: Requested vote from peers 30f6f31e9a314e4099b7466a143ed434 (127.9.116.130:37109), 73c089c6be7a453a818ca37081639799 (127.9.116.131:35735)
I20260709 13:21:10.109751 21645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f38dbb095cde46cfbcca5917aac7d453" candidate_uuid: "c49029e9cb7546728f69e9c919b01d90" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "30f6f31e9a314e4099b7466a143ed434"
I20260709 13:21:10.109803 21719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f38dbb095cde46cfbcca5917aac7d453" candidate_uuid: "c49029e9cb7546728f69e9c919b01d90" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "73c089c6be7a453a818ca37081639799"
I20260709 13:21:10.110514 21645 raft_consensus.cc:3060] T f38dbb095cde46cfbcca5917aac7d453 P 30f6f31e9a314e4099b7466a143ed434 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:10.110608 21719 raft_consensus.cc:3055] T f38dbb095cde46cfbcca5917aac7d453 P 73c089c6be7a453a818ca37081639799 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:10.111114 21719 raft_consensus.cc:740] T f38dbb095cde46cfbcca5917aac7d453 P 73c089c6be7a453a818ca37081639799 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 73c089c6be7a453a818ca37081639799, State: Running, Role: LEADER
I20260709 13:21:10.111940 21719 consensus_queue.cc:260] T f38dbb095cde46cfbcca5917aac7d453 P 73c089c6be7a453a818ca37081639799 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } }
I20260709 13:21:10.113145 21719 raft_consensus.cc:3060] T f38dbb095cde46cfbcca5917aac7d453 P 73c089c6be7a453a818ca37081639799 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:10.116911 21645 raft_consensus.cc:2468] T f38dbb095cde46cfbcca5917aac7d453 P 30f6f31e9a314e4099b7466a143ed434 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c49029e9cb7546728f69e9c919b01d90 in term 2.
I20260709 13:21:10.118325 21536 leader_election.cc:304] T f38dbb095cde46cfbcca5917aac7d453 P c49029e9cb7546728f69e9c919b01d90 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30f6f31e9a314e4099b7466a143ed434, c49029e9cb7546728f69e9c919b01d90; no voters: 
I20260709 13:21:10.118760 21719 raft_consensus.cc:2468] T f38dbb095cde46cfbcca5917aac7d453 P 73c089c6be7a453a818ca37081639799 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c49029e9cb7546728f69e9c919b01d90 in term 2.
I20260709 13:21:10.119339 21767 raft_consensus.cc:2804] T f38dbb095cde46cfbcca5917aac7d453 P c49029e9cb7546728f69e9c919b01d90 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:10.119818 21767 raft_consensus.cc:697] T f38dbb095cde46cfbcca5917aac7d453 P c49029e9cb7546728f69e9c919b01d90 [term 2 LEADER]: Becoming Leader. State: Replica: c49029e9cb7546728f69e9c919b01d90, State: Running, Role: LEADER
I20260709 13:21:10.120558 21767 consensus_queue.cc:237] T f38dbb095cde46cfbcca5917aac7d453 P c49029e9cb7546728f69e9c919b01d90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } }
I20260709 13:21:10.126677 21466 catalog_manager.cc:5697] T f38dbb095cde46cfbcca5917aac7d453 P c49029e9cb7546728f69e9c919b01d90 reported cstate change: term changed from 1 to 2, leader changed from 73c089c6be7a453a818ca37081639799 (127.9.116.131) to c49029e9cb7546728f69e9c919b01d90 (127.9.116.129). New cstate: current_term: 2 leader_uuid: "c49029e9cb7546728f69e9c919b01d90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:10.176234 21776 raft_consensus.cc:993] T 67e0d782f225460f8d7bf6e3a519e0c5 P c49029e9cb7546728f69e9c919b01d90: : Instructing follower 30f6f31e9a314e4099b7466a143ed434 to start an election
I20260709 13:21:10.176653 21767 raft_consensus.cc:1081] T 67e0d782f225460f8d7bf6e3a519e0c5 P c49029e9cb7546728f69e9c919b01d90 [term 1 LEADER]: Signalling peer 30f6f31e9a314e4099b7466a143ed434 to start an election
I20260709 13:21:10.178146 21645 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "67e0d782f225460f8d7bf6e3a519e0c5"
dest_uuid: "30f6f31e9a314e4099b7466a143ed434"
 from {username='slave'} at 127.0.0.1:36186
I20260709 13:21:10.178604 21645 raft_consensus.cc:493] T 67e0d782f225460f8d7bf6e3a519e0c5 P 30f6f31e9a314e4099b7466a143ed434 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:10.178886 21645 raft_consensus.cc:3060] T 67e0d782f225460f8d7bf6e3a519e0c5 P 30f6f31e9a314e4099b7466a143ed434 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:10.183820 21645 raft_consensus.cc:515] T 67e0d782f225460f8d7bf6e3a519e0c5 P 30f6f31e9a314e4099b7466a143ed434 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } }
I20260709 13:21:10.186019 21645 leader_election.cc:290] T 67e0d782f225460f8d7bf6e3a519e0c5 P 30f6f31e9a314e4099b7466a143ed434 [CANDIDATE]: Term 2 election: Requested vote from peers c49029e9cb7546728f69e9c919b01d90 (127.9.116.129:32957), 73c089c6be7a453a818ca37081639799 (127.9.116.131:35735)
I20260709 13:21:10.198274 21570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e0d782f225460f8d7bf6e3a519e0c5" candidate_uuid: "30f6f31e9a314e4099b7466a143ed434" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c49029e9cb7546728f69e9c919b01d90"
I20260709 13:21:10.199046 21570 raft_consensus.cc:3055] T 67e0d782f225460f8d7bf6e3a519e0c5 P c49029e9cb7546728f69e9c919b01d90 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:10.199486 21570 raft_consensus.cc:740] T 67e0d782f225460f8d7bf6e3a519e0c5 P c49029e9cb7546728f69e9c919b01d90 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c49029e9cb7546728f69e9c919b01d90, State: Running, Role: LEADER
I20260709 13:21:10.200402 21570 consensus_queue.cc:260] T 67e0d782f225460f8d7bf6e3a519e0c5 P c49029e9cb7546728f69e9c919b01d90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } }
I20260709 13:21:10.201377 21719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e0d782f225460f8d7bf6e3a519e0c5" candidate_uuid: "30f6f31e9a314e4099b7466a143ed434" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "73c089c6be7a453a818ca37081639799"
I20260709 13:21:10.201638 21570 raft_consensus.cc:3060] T 67e0d782f225460f8d7bf6e3a519e0c5 P c49029e9cb7546728f69e9c919b01d90 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:10.202031 21719 raft_consensus.cc:3060] T 67e0d782f225460f8d7bf6e3a519e0c5 P 73c089c6be7a453a818ca37081639799 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:10.256260 21719 raft_consensus.cc:2468] T 67e0d782f225460f8d7bf6e3a519e0c5 P 73c089c6be7a453a818ca37081639799 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30f6f31e9a314e4099b7466a143ed434 in term 2.
I20260709 13:21:10.259513 21609 leader_election.cc:304] T 67e0d782f225460f8d7bf6e3a519e0c5 P 30f6f31e9a314e4099b7466a143ed434 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30f6f31e9a314e4099b7466a143ed434, 73c089c6be7a453a818ca37081639799; no voters: 
I20260709 13:21:10.259785 21570 raft_consensus.cc:2468] T 67e0d782f225460f8d7bf6e3a519e0c5 P c49029e9cb7546728f69e9c919b01d90 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30f6f31e9a314e4099b7466a143ed434 in term 2.
I20260709 13:21:10.261622 21768 raft_consensus.cc:2804] T 67e0d782f225460f8d7bf6e3a519e0c5 P 30f6f31e9a314e4099b7466a143ed434 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:10.263734 21768 raft_consensus.cc:697] T 67e0d782f225460f8d7bf6e3a519e0c5 P 30f6f31e9a314e4099b7466a143ed434 [term 2 LEADER]: Becoming Leader. State: Replica: 30f6f31e9a314e4099b7466a143ed434, State: Running, Role: LEADER
I20260709 13:21:10.265555 21768 consensus_queue.cc:237] T 67e0d782f225460f8d7bf6e3a519e0c5 P 30f6f31e9a314e4099b7466a143ed434 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } }
I20260709 13:21:10.276153 21466 catalog_manager.cc:5697] T 67e0d782f225460f8d7bf6e3a519e0c5 P 30f6f31e9a314e4099b7466a143ed434 reported cstate change: term changed from 1 to 2, leader changed from c49029e9cb7546728f69e9c919b01d90 (127.9.116.129) to 30f6f31e9a314e4099b7466a143ed434 (127.9.116.130). New cstate: current_term: 2 leader_uuid: "30f6f31e9a314e4099b7466a143ed434" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:10.325757 21645 raft_consensus.cc:1275] T 83f44d0e05dd4b1ea9ff6706edf996fa P 30f6f31e9a314e4099b7466a143ed434 [term 2 FOLLOWER]: Refusing update from remote peer c49029e9cb7546728f69e9c919b01d90: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:10.327430 21767 consensus_queue.cc:1048] T 83f44d0e05dd4b1ea9ff6706edf996fa P c49029e9cb7546728f69e9c919b01d90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:10.338326 21719 raft_consensus.cc:1275] T 83f44d0e05dd4b1ea9ff6706edf996fa P 73c089c6be7a453a818ca37081639799 [term 2 FOLLOWER]: Refusing update from remote peer c49029e9cb7546728f69e9c919b01d90: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:10.340453 21767 consensus_queue.cc:1048] T 83f44d0e05dd4b1ea9ff6706edf996fa P c49029e9cb7546728f69e9c919b01d90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:10.555265  9682 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 13:21:10.556849  9682 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 13:21:10.557248  9682 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 13:21:10.571288 21570 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "32eb13c472e44e079221b8c6ec099fcb"
dest_uuid: "c49029e9cb7546728f69e9c919b01d90"
mode: GRACEFUL
new_leader_uuid: "73c089c6be7a453a818ca37081639799"
 from {username='slave'} at 127.0.0.1:33496
I20260709 13:21:10.571898 21570 raft_consensus.cc:606] T 32eb13c472e44e079221b8c6ec099fcb P c49029e9cb7546728f69e9c919b01d90 [term 1 LEADER]: Received request to transfer leadership to TS 73c089c6be7a453a818ca37081639799
I20260709 13:21:10.573103  9682 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 13:21:10.575441 21645 raft_consensus.cc:1275] T f38dbb095cde46cfbcca5917aac7d453 P 30f6f31e9a314e4099b7466a143ed434 [term 2 FOLLOWER]: Refusing update from remote peer c49029e9cb7546728f69e9c919b01d90: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:10.576959 21767 consensus_queue.cc:1048] T f38dbb095cde46cfbcca5917aac7d453 P c49029e9cb7546728f69e9c919b01d90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:10.586867 21719 raft_consensus.cc:1275] T f38dbb095cde46cfbcca5917aac7d453 P 73c089c6be7a453a818ca37081639799 [term 2 FOLLOWER]: Refusing update from remote peer c49029e9cb7546728f69e9c919b01d90: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:10.588812 21767 consensus_queue.cc:1048] T f38dbb095cde46cfbcca5917aac7d453 P c49029e9cb7546728f69e9c919b01d90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:10.724488 21719 raft_consensus.cc:1275] T 67e0d782f225460f8d7bf6e3a519e0c5 P 73c089c6be7a453a818ca37081639799 [term 2 FOLLOWER]: Refusing update from remote peer 30f6f31e9a314e4099b7466a143ed434: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:10.725919 21768 consensus_queue.cc:1048] T 67e0d782f225460f8d7bf6e3a519e0c5 P 30f6f31e9a314e4099b7466a143ed434 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:10.735791 21570 raft_consensus.cc:1275] T 67e0d782f225460f8d7bf6e3a519e0c5 P c49029e9cb7546728f69e9c919b01d90 [term 2 FOLLOWER]: Refusing update from remote peer 30f6f31e9a314e4099b7466a143ed434: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:10.737959 21812 consensus_queue.cc:1048] T 67e0d782f225460f8d7bf6e3a519e0c5 P 30f6f31e9a314e4099b7466a143ed434 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:21:10.938079 21776 raft_consensus.cc:993] T 32eb13c472e44e079221b8c6ec099fcb P c49029e9cb7546728f69e9c919b01d90: : Instructing follower 73c089c6be7a453a818ca37081639799 to start an election
I20260709 13:21:10.938534 21767 raft_consensus.cc:1081] T 32eb13c472e44e079221b8c6ec099fcb P c49029e9cb7546728f69e9c919b01d90 [term 1 LEADER]: Signalling peer 73c089c6be7a453a818ca37081639799 to start an election
I20260709 13:21:10.940104 21719 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "32eb13c472e44e079221b8c6ec099fcb"
dest_uuid: "73c089c6be7a453a818ca37081639799"
 from {username='slave'} at 127.0.0.1:53506
I20260709 13:21:10.940646 21719 raft_consensus.cc:493] T 32eb13c472e44e079221b8c6ec099fcb P 73c089c6be7a453a818ca37081639799 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:10.940972 21719 raft_consensus.cc:3060] T 32eb13c472e44e079221b8c6ec099fcb P 73c089c6be7a453a818ca37081639799 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:10.947510 21719 raft_consensus.cc:515] T 32eb13c472e44e079221b8c6ec099fcb P 73c089c6be7a453a818ca37081639799 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } }
I20260709 13:21:10.949532 21719 leader_election.cc:290] T 32eb13c472e44e079221b8c6ec099fcb P 73c089c6be7a453a818ca37081639799 [CANDIDATE]: Term 2 election: Requested vote from peers c49029e9cb7546728f69e9c919b01d90 (127.9.116.129:32957), 30f6f31e9a314e4099b7466a143ed434 (127.9.116.130:37109)
I20260709 13:21:10.950587 21570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32eb13c472e44e079221b8c6ec099fcb" candidate_uuid: "73c089c6be7a453a818ca37081639799" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c49029e9cb7546728f69e9c919b01d90"
I20260709 13:21:10.951120 21645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32eb13c472e44e079221b8c6ec099fcb" candidate_uuid: "73c089c6be7a453a818ca37081639799" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "30f6f31e9a314e4099b7466a143ed434"
I20260709 13:21:10.951464 21570 raft_consensus.cc:3055] T 32eb13c472e44e079221b8c6ec099fcb P c49029e9cb7546728f69e9c919b01d90 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:10.951851 21645 raft_consensus.cc:3060] T 32eb13c472e44e079221b8c6ec099fcb P 30f6f31e9a314e4099b7466a143ed434 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:10.951854 21570 raft_consensus.cc:740] T 32eb13c472e44e079221b8c6ec099fcb P c49029e9cb7546728f69e9c919b01d90 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c49029e9cb7546728f69e9c919b01d90, State: Running, Role: LEADER
I20260709 13:21:10.952953 21570 consensus_queue.cc:260] T 32eb13c472e44e079221b8c6ec099fcb P c49029e9cb7546728f69e9c919b01d90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } }
I20260709 13:21:10.954064 21570 raft_consensus.cc:3060] T 32eb13c472e44e079221b8c6ec099fcb P c49029e9cb7546728f69e9c919b01d90 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:10.959694 21570 raft_consensus.cc:2468] T 32eb13c472e44e079221b8c6ec099fcb P c49029e9cb7546728f69e9c919b01d90 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73c089c6be7a453a818ca37081639799 in term 2.
I20260709 13:21:10.959976 21645 raft_consensus.cc:2468] T 32eb13c472e44e079221b8c6ec099fcb P 30f6f31e9a314e4099b7466a143ed434 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73c089c6be7a453a818ca37081639799 in term 2.
I20260709 13:21:10.961012 21685 leader_election.cc:304] T 32eb13c472e44e079221b8c6ec099fcb P 73c089c6be7a453a818ca37081639799 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73c089c6be7a453a818ca37081639799, c49029e9cb7546728f69e9c919b01d90; no voters: 
I20260709 13:21:10.961993 21791 raft_consensus.cc:2804] T 32eb13c472e44e079221b8c6ec099fcb P 73c089c6be7a453a818ca37081639799 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:10.962944 21791 raft_consensus.cc:697] T 32eb13c472e44e079221b8c6ec099fcb P 73c089c6be7a453a818ca37081639799 [term 2 LEADER]: Becoming Leader. State: Replica: 73c089c6be7a453a818ca37081639799, State: Running, Role: LEADER
I20260709 13:21:10.964032 21791 consensus_queue.cc:237] T 32eb13c472e44e079221b8c6ec099fcb P 73c089c6be7a453a818ca37081639799 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } }
I20260709 13:21:10.971346 21467 catalog_manager.cc:5697] T 32eb13c472e44e079221b8c6ec099fcb P 73c089c6be7a453a818ca37081639799 reported cstate change: term changed from 1 to 2, leader changed from c49029e9cb7546728f69e9c919b01d90 (127.9.116.129) to 73c089c6be7a453a818ca37081639799 (127.9.116.131). New cstate: current_term: 2 leader_uuid: "73c089c6be7a453a818ca37081639799" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49029e9cb7546728f69e9c919b01d90" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 32957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30f6f31e9a314e4099b7466a143ed434" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 37109 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73c089c6be7a453a818ca37081639799" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 35735 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:11.202195  9682 tablet_server.cc:179] TabletServer@127.9.116.129:0 shutting down...
I20260709 13:21:11.229600  9682 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:21:11.230468  9682 tablet_replica.cc:333] T 32eb13c472e44e079221b8c6ec099fcb P c49029e9cb7546728f69e9c919b01d90: stopping tablet replica
I20260709 13:21:11.231335  9682 raft_consensus.cc:2243] T 32eb13c472e44e079221b8c6ec099fcb P c49029e9cb7546728f69e9c919b01d90 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:11.231853  9682 raft_consensus.cc:2272] T 32eb13c472e44e079221b8c6ec099fcb P c49029e9cb7546728f69e9c919b01d90 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:11.234853  9682 tablet_replica.cc:333] T f38dbb095cde46cfbcca5917aac7d453 P c49029e9cb7546728f69e9c919b01d90: stopping tablet replica
I20260709 13:21:11.235670  9682 raft_consensus.cc:2243] T f38dbb095cde46cfbcca5917aac7d453 P c49029e9cb7546728f69e9c919b01d90 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:11.236851  9682 raft_consensus.cc:2272] T f38dbb095cde46cfbcca5917aac7d453 P c49029e9cb7546728f69e9c919b01d90 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:11.239990  9682 tablet_replica.cc:333] T 83f44d0e05dd4b1ea9ff6706edf996fa P c49029e9cb7546728f69e9c919b01d90: stopping tablet replica
I20260709 13:21:11.240856  9682 raft_consensus.cc:2243] T 83f44d0e05dd4b1ea9ff6706edf996fa P c49029e9cb7546728f69e9c919b01d90 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:11.241827  9682 raft_consensus.cc:2272] T 83f44d0e05dd4b1ea9ff6706edf996fa P c49029e9cb7546728f69e9c919b01d90 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:11.244747  9682 tablet_replica.cc:333] T 67e0d782f225460f8d7bf6e3a519e0c5 P c49029e9cb7546728f69e9c919b01d90: stopping tablet replica
I20260709 13:21:11.245298  9682 raft_consensus.cc:2243] T 67e0d782f225460f8d7bf6e3a519e0c5 P c49029e9cb7546728f69e9c919b01d90 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:11.245782  9682 raft_consensus.cc:2272] T 67e0d782f225460f8d7bf6e3a519e0c5 P c49029e9cb7546728f69e9c919b01d90 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:11.255177 21611 consensus_peers.cc:597] T 67e0d782f225460f8d7bf6e3a519e0c5 P 30f6f31e9a314e4099b7466a143ed434 -> Peer c49029e9cb7546728f69e9c919b01d90 (127.9.116.129:32957): Couldn't send request to peer c49029e9cb7546728f69e9c919b01d90. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:11.275553  9682 tablet_server.cc:196] TabletServer@127.9.116.129:0 shutdown complete.
I20260709 13:21:11.294337  9682 tablet_server.cc:179] TabletServer@127.9.116.130:0 shutting down...
I20260709 13:21:11.321367  9682 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:21:11.322229  9682 tablet_replica.cc:333] T 32eb13c472e44e079221b8c6ec099fcb P 30f6f31e9a314e4099b7466a143ed434: stopping tablet replica
I20260709 13:21:11.323484  9682 raft_consensus.cc:2243] T 32eb13c472e44e079221b8c6ec099fcb P 30f6f31e9a314e4099b7466a143ed434 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:11.324327  9682 raft_consensus.cc:2272] T 32eb13c472e44e079221b8c6ec099fcb P 30f6f31e9a314e4099b7466a143ed434 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:11.328855  9682 tablet_replica.cc:333] T f38dbb095cde46cfbcca5917aac7d453 P 30f6f31e9a314e4099b7466a143ed434: stopping tablet replica
I20260709 13:21:11.329818  9682 raft_consensus.cc:2243] T f38dbb095cde46cfbcca5917aac7d453 P 30f6f31e9a314e4099b7466a143ed434 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:11.330550  9682 raft_consensus.cc:2272] T f38dbb095cde46cfbcca5917aac7d453 P 30f6f31e9a314e4099b7466a143ed434 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:11.335467  9682 tablet_replica.cc:333] T 67e0d782f225460f8d7bf6e3a519e0c5 P 30f6f31e9a314e4099b7466a143ed434: stopping tablet replica
I20260709 13:21:11.336301  9682 raft_consensus.cc:2243] T 67e0d782f225460f8d7bf6e3a519e0c5 P 30f6f31e9a314e4099b7466a143ed434 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:11.337212  9682 raft_consensus.cc:2272] T 67e0d782f225460f8d7bf6e3a519e0c5 P 30f6f31e9a314e4099b7466a143ed434 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:11.340981  9682 tablet_replica.cc:333] T 83f44d0e05dd4b1ea9ff6706edf996fa P 30f6f31e9a314e4099b7466a143ed434: stopping tablet replica
I20260709 13:21:11.341766  9682 raft_consensus.cc:2243] T 83f44d0e05dd4b1ea9ff6706edf996fa P 30f6f31e9a314e4099b7466a143ed434 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:11.342414  9682 raft_consensus.cc:2272] T 83f44d0e05dd4b1ea9ff6706edf996fa P 30f6f31e9a314e4099b7466a143ed434 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:11.349298 21685 consensus_peers.cc:597] T 32eb13c472e44e079221b8c6ec099fcb P 73c089c6be7a453a818ca37081639799 -> Peer c49029e9cb7546728f69e9c919b01d90 (127.9.116.129:32957): Couldn't send request to peer c49029e9cb7546728f69e9c919b01d90. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:32957: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:11.374946  9682 tablet_server.cc:196] TabletServer@127.9.116.130:0 shutdown complete.
I20260709 13:21:11.392776  9682 tablet_server.cc:179] TabletServer@127.9.116.131:0 shutting down...
I20260709 13:21:11.418648  9682 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:21:11.419512  9682 tablet_replica.cc:333] T f38dbb095cde46cfbcca5917aac7d453 P 73c089c6be7a453a818ca37081639799: stopping tablet replica
I20260709 13:21:11.420219  9682 raft_consensus.cc:2243] T f38dbb095cde46cfbcca5917aac7d453 P 73c089c6be7a453a818ca37081639799 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:11.420809  9682 raft_consensus.cc:2272] T f38dbb095cde46cfbcca5917aac7d453 P 73c089c6be7a453a818ca37081639799 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:11.423563  9682 tablet_replica.cc:333] T 32eb13c472e44e079221b8c6ec099fcb P 73c089c6be7a453a818ca37081639799: stopping tablet replica
I20260709 13:21:11.424180  9682 raft_consensus.cc:2243] T 32eb13c472e44e079221b8c6ec099fcb P 73c089c6be7a453a818ca37081639799 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:11.424929  9682 pending_rounds.cc:70] T 32eb13c472e44e079221b8c6ec099fcb P 73c089c6be7a453a818ca37081639799: Trying to abort 1 pending ops.
I20260709 13:21:11.425144  9682 pending_rounds.cc:77] T 32eb13c472e44e079221b8c6ec099fcb P 73c089c6be7a453a818ca37081639799: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305638997876617216 op_type: NO_OP noop_request { }
I20260709 13:21:11.425427  9682 raft_consensus.cc:2889] T 32eb13c472e44e079221b8c6ec099fcb P 73c089c6be7a453a818ca37081639799 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:21:11.425765  9682 raft_consensus.cc:2272] T 32eb13c472e44e079221b8c6ec099fcb P 73c089c6be7a453a818ca37081639799 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:11.428875  9682 tablet_replica.cc:333] T 67e0d782f225460f8d7bf6e3a519e0c5 P 73c089c6be7a453a818ca37081639799: stopping tablet replica
I20260709 13:21:11.429391  9682 raft_consensus.cc:2243] T 67e0d782f225460f8d7bf6e3a519e0c5 P 73c089c6be7a453a818ca37081639799 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:11.429807  9682 raft_consensus.cc:2272] T 67e0d782f225460f8d7bf6e3a519e0c5 P 73c089c6be7a453a818ca37081639799 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:11.432572  9682 tablet_replica.cc:333] T 83f44d0e05dd4b1ea9ff6706edf996fa P 73c089c6be7a453a818ca37081639799: stopping tablet replica
I20260709 13:21:11.433008  9682 raft_consensus.cc:2243] T 83f44d0e05dd4b1ea9ff6706edf996fa P 73c089c6be7a453a818ca37081639799 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:11.433408  9682 raft_consensus.cc:2272] T 83f44d0e05dd4b1ea9ff6706edf996fa P 73c089c6be7a453a818ca37081639799 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:11.458384  9682 tablet_server.cc:196] TabletServer@127.9.116.131:0 shutdown complete.
I20260709 13:21:11.476877  9682 master.cc:562] Master@127.9.116.190:45781 shutting down...
I20260709 13:21:11.496657  9682 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 2d23126fb62441beb1e910daf013f7cb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:11.497251  9682 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 2d23126fb62441beb1e910daf013f7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:11.497629  9682 tablet_replica.cc:333] T 00000000000000000000000000000000 P 2d23126fb62441beb1e910daf013f7cb: stopping tablet replica
I20260709 13:21:11.518905  9682 master.cc:584] Master@127.9.116.190:45781 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (3377 ms)
[----------] 6 tests from LeaderRebalancerTest (354863 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 13:21:11.561492  9682 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.9.116.190:39063
I20260709 13:21:11.562708  9682 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:21:11.568818 21827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:11.568873 21828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:11.569839 21830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:11.571489  9682 server_base.cc:1061] running on GCE node
I20260709 13:21:11.572361  9682 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:21:11.572547  9682 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:21:11.572726  9682 hybrid_clock.cc:648] HybridClock initialized: now 1783603271572712 us; error 0 us; skew 500 ppm
I20260709 13:21:11.573314  9682 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:21:11.575927  9682 webserver.cc:533] Webserver started at http://127.9.116.190:33159/ using document root <none> and password file <none>
I20260709 13:21:11.576447  9682 fs_manager.cc:362] Metadata directory not provided
I20260709 13:21:11.576653  9682 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:21:11.576942  9682 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:21:11.578133  9682 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602916530764-9682-0/minicluster-data/master-0-root/instance:
uuid: "848ddee1690240febd24df3095cdbbab"
format_stamp: "Formatted at 2026-07-09 13:21:11 on dist-test-slave-1t6m"
I20260709 13:21:11.582754  9682 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:21:11.586213 21835 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:11.587028  9682 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.003s
I20260709 13:21:11.587337  9682 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602916530764-9682-0/minicluster-data/master-0-root
uuid: "848ddee1690240febd24df3095cdbbab"
format_stamp: "Formatted at 2026-07-09 13:21:11 on dist-test-slave-1t6m"
I20260709 13:21:11.587617  9682 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602916530764-9682-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602916530764-9682-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602916530764-9682-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:21:11.608554  9682 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:21:11.609882  9682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:21:11.649375  9682 rpc_server.cc:307] RPC server started. Bound to: 127.9.116.190:39063
I20260709 13:21:11.649479 21886 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.116.190:39063 every 8 connection(s)
I20260709 13:21:11.653429 21887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:11.665510 21887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 848ddee1690240febd24df3095cdbbab: Bootstrap starting.
I20260709 13:21:11.670531 21887 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 848ddee1690240febd24df3095cdbbab: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:11.675037 21887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 848ddee1690240febd24df3095cdbbab: No bootstrap required, opened a new log
I20260709 13:21:11.677224 21887 raft_consensus.cc:359] T 00000000000000000000000000000000 P 848ddee1690240febd24df3095cdbbab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "848ddee1690240febd24df3095cdbbab" member_type: VOTER }
I20260709 13:21:11.677597 21887 raft_consensus.cc:385] T 00000000000000000000000000000000 P 848ddee1690240febd24df3095cdbbab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:11.677773 21887 raft_consensus.cc:740] T 00000000000000000000000000000000 P 848ddee1690240febd24df3095cdbbab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 848ddee1690240febd24df3095cdbbab, State: Initialized, Role: FOLLOWER
I20260709 13:21:11.678275 21887 consensus_queue.cc:260] T 00000000000000000000000000000000 P 848ddee1690240febd24df3095cdbbab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "848ddee1690240febd24df3095cdbbab" member_type: VOTER }
I20260709 13:21:11.678681 21887 raft_consensus.cc:399] T 00000000000000000000000000000000 P 848ddee1690240febd24df3095cdbbab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:21:11.678890 21887 raft_consensus.cc:493] T 00000000000000000000000000000000 P 848ddee1690240febd24df3095cdbbab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:21:11.679105 21887 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 848ddee1690240febd24df3095cdbbab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:11.684178 21887 raft_consensus.cc:515] T 00000000000000000000000000000000 P 848ddee1690240febd24df3095cdbbab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "848ddee1690240febd24df3095cdbbab" member_type: VOTER }
I20260709 13:21:11.684681 21887 leader_election.cc:304] T 00000000000000000000000000000000 P 848ddee1690240febd24df3095cdbbab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 848ddee1690240febd24df3095cdbbab; no voters: 
I20260709 13:21:11.685755 21887 leader_election.cc:290] T 00000000000000000000000000000000 P 848ddee1690240febd24df3095cdbbab [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:21:11.686117 21890 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 848ddee1690240febd24df3095cdbbab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:11.687650 21890 raft_consensus.cc:697] T 00000000000000000000000000000000 P 848ddee1690240febd24df3095cdbbab [term 1 LEADER]: Becoming Leader. State: Replica: 848ddee1690240febd24df3095cdbbab, State: Running, Role: LEADER
I20260709 13:21:11.688297 21890 consensus_queue.cc:237] T 00000000000000000000000000000000 P 848ddee1690240febd24df3095cdbbab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "848ddee1690240febd24df3095cdbbab" member_type: VOTER }
I20260709 13:21:11.688871 21887 sys_catalog.cc:565] T 00000000000000000000000000000000 P 848ddee1690240febd24df3095cdbbab [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:21:11.691483 21891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 848ddee1690240febd24df3095cdbbab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "848ddee1690240febd24df3095cdbbab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "848ddee1690240febd24df3095cdbbab" member_type: VOTER } }
I20260709 13:21:11.691583 21892 sys_catalog.cc:455] T 00000000000000000000000000000000 P 848ddee1690240febd24df3095cdbbab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 848ddee1690240febd24df3095cdbbab. Latest consensus state: current_term: 1 leader_uuid: "848ddee1690240febd24df3095cdbbab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "848ddee1690240febd24df3095cdbbab" member_type: VOTER } }
I20260709 13:21:11.692414 21892 sys_catalog.cc:458] T 00000000000000000000000000000000 P 848ddee1690240febd24df3095cdbbab [sys.catalog]: This master's current role is: LEADER
I20260709 13:21:11.692435 21891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 848ddee1690240febd24df3095cdbbab [sys.catalog]: This master's current role is: LEADER
I20260709 13:21:11.696786 21895 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:21:11.701889 21895 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:21:11.705302  9682 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:21:11.711891 21895 catalog_manager.cc:1383] Generated new cluster ID: 219bdea4c3ed4734add5088db2d67f7b
I20260709 13:21:11.712217 21895 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:21:11.738617 21895 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:21:11.740237 21895 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:21:11.755236 21895 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 848ddee1690240febd24df3095cdbbab: Generated new TSK 0
I20260709 13:21:11.755895 21895 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:21:11.773548  9682 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:21:11.780365 21908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:11.781697 21909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:11.782610 21911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:11.783067  9682 server_base.cc:1061] running on GCE node
I20260709 13:21:11.784080  9682 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:21:11.784266  9682 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:21:11.784395  9682 hybrid_clock.cc:648] HybridClock initialized: now 1783603271784383 us; error 0 us; skew 500 ppm
I20260709 13:21:11.784955  9682 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:21:11.787437  9682 webserver.cc:533] Webserver started at http://127.9.116.129:37659/ using document root <none> and password file <none>
I20260709 13:21:11.787921  9682 fs_manager.cc:362] Metadata directory not provided
I20260709 13:21:11.788092  9682 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:21:11.788308  9682 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:21:11.789485  9682 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602916530764-9682-0/minicluster-data/ts-0-root/instance:
uuid: "8507b5a0d0e6434aaa7302085c7f54cb"
format_stamp: "Formatted at 2026-07-09 13:21:11 on dist-test-slave-1t6m"
I20260709 13:21:11.794373  9682 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 13:21:11.798063 21916 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:11.799028  9682 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.003s
I20260709 13:21:11.799330  9682 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602916530764-9682-0/minicluster-data/ts-0-root
uuid: "8507b5a0d0e6434aaa7302085c7f54cb"
format_stamp: "Formatted at 2026-07-09 13:21:11 on dist-test-slave-1t6m"
I20260709 13:21:11.799592  9682 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602916530764-9682-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602916530764-9682-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602916530764-9682-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:21:11.820053  9682 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:21:11.821403  9682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:21:11.822937  9682 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:21:11.825286  9682 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:21:11.825505  9682 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:11.825794  9682 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:21:11.825975  9682 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:11.867839  9682 rpc_server.cc:307] RPC server started. Bound to: 127.9.116.129:35591
I20260709 13:21:11.867951 21978 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.116.129:35591 every 8 connection(s)
I20260709 13:21:11.873145  9682 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:21:11.881421 21983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:11.883343 21984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:11.886529 21986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:11.888200  9682 server_base.cc:1061] running on GCE node
I20260709 13:21:11.888456 21979 heartbeater.cc:344] Connected to a master server at 127.9.116.190:39063
I20260709 13:21:11.888963 21979 heartbeater.cc:461] Registering TS with master...
I20260709 13:21:11.889381  9682 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:21:11.889645  9682 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:21:11.889861  9682 hybrid_clock.cc:648] HybridClock initialized: now 1783603271889841 us; error 0 us; skew 500 ppm
I20260709 13:21:11.889892 21979 heartbeater.cc:507] Master 127.9.116.190:39063 requested a full tablet report, sending...
I20260709 13:21:11.890550  9682 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:21:11.892463 21852 ts_manager.cc:194] Registered new tserver with Master: 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591)
I20260709 13:21:11.893525  9682 webserver.cc:533] Webserver started at http://127.9.116.130:34497/ using document root <none> and password file <none>
I20260709 13:21:11.894263  9682 fs_manager.cc:362] Metadata directory not provided
I20260709 13:21:11.894501  9682 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:21:11.894819  9682 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:21:11.895072 21852 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45256
I20260709 13:21:11.896432  9682 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602916530764-9682-0/minicluster-data/ts-1-root/instance:
uuid: "c48712233a8a4eb0b39bb05cd60137e6"
format_stamp: "Formatted at 2026-07-09 13:21:11 on dist-test-slave-1t6m"
I20260709 13:21:11.901283  9682 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:21:11.904861 21991 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:11.905671  9682 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:21:11.905982  9682 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602916530764-9682-0/minicluster-data/ts-1-root
uuid: "c48712233a8a4eb0b39bb05cd60137e6"
format_stamp: "Formatted at 2026-07-09 13:21:11 on dist-test-slave-1t6m"
I20260709 13:21:11.906287  9682 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602916530764-9682-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602916530764-9682-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602916530764-9682-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:21:11.932621  9682 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:21:11.933874  9682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:21:11.935866  9682 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:21:11.938198  9682 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:21:11.938441  9682 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:11.938687  9682 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:21:11.938887  9682 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:21:11.980041  9682 rpc_server.cc:307] RPC server started. Bound to: 127.9.116.130:46529
I20260709 13:21:11.980196 22053 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.116.130:46529 every 8 connection(s)
I20260709 13:21:11.985082  9682 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:21:11.994415 22057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:12.000468 22054 heartbeater.cc:344] Connected to a master server at 127.9.116.190:39063
I20260709 13:21:12.001010 22054 heartbeater.cc:461] Registering TS with master...
I20260709 13:21:12.001977 22054 heartbeater.cc:507] Master 127.9.116.190:39063 requested a full tablet report, sending...
W20260709 13:21:12.002761 22060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:12.003463 22058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:12.004508 21852 ts_manager.cc:194] Registered new tserver with Master: c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529)
I20260709 13:21:12.006199  9682 server_base.cc:1061] running on GCE node
I20260709 13:21:12.006422 21852 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45266
I20260709 13:21:12.007311  9682 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:21:12.007658  9682 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:21:12.007879  9682 hybrid_clock.cc:648] HybridClock initialized: now 1783603272007860 us; error 0 us; skew 500 ppm
I20260709 13:21:12.008689  9682 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:21:12.011204  9682 webserver.cc:533] Webserver started at http://127.9.116.131:41691/ using document root <none> and password file <none>
I20260709 13:21:12.011726  9682 fs_manager.cc:362] Metadata directory not provided
I20260709 13:21:12.011929  9682 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:21:12.012225  9682 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:21:12.013465  9682 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602916530764-9682-0/minicluster-data/ts-2-root/instance:
uuid: "21a7bb1d6ef54b20befdf8ea6d293776"
format_stamp: "Formatted at 2026-07-09 13:21:12 on dist-test-slave-1t6m"
I20260709 13:21:12.018067  9682 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.001s
I20260709 13:21:12.021678 22065 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:12.022538  9682 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:21:12.022944  9682 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602916530764-9682-0/minicluster-data/ts-2-root
uuid: "21a7bb1d6ef54b20befdf8ea6d293776"
format_stamp: "Formatted at 2026-07-09 13:21:12 on dist-test-slave-1t6m"
I20260709 13:21:12.023236  9682 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602916530764-9682-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602916530764-9682-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602916530764-9682-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:21:12.054432  9682 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:21:12.055894  9682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:21:12.057459  9682 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:21:12.059855  9682 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:21:12.060105  9682 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:12.060369  9682 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:21:12.060555  9682 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:12.111439  9682 rpc_server.cc:307] RPC server started. Bound to: 127.9.116.131:33869
I20260709 13:21:12.111560 22127 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.116.131:33869 every 8 connection(s)
I20260709 13:21:12.130613 22128 heartbeater.cc:344] Connected to a master server at 127.9.116.190:39063
I20260709 13:21:12.131124 22128 heartbeater.cc:461] Registering TS with master...
I20260709 13:21:12.132171 22128 heartbeater.cc:507] Master 127.9.116.190:39063 requested a full tablet report, sending...
I20260709 13:21:12.134513 21852 ts_manager.cc:194] Registered new tserver with Master: 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131:33869)
I20260709 13:21:12.135457  9682 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.018860118s
I20260709 13:21:12.136336 21852 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45282
I20260709 13:21:12.167060 21852 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45286:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:21:12.169942 21852 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:21:12.235013 22093 tablet_service.cc:1511] Processing CreateTablet for tablet 0ca42dd42e4d46359256d9256f64d657 (DEFAULT_TABLE table=test-workload [id=c1a0d0f5418a4eb1bc7724e39cfb2ce9]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:21:12.235250 22092 tablet_service.cc:1511] Processing CreateTablet for tablet 1631e507d6714dc187fc9d7f2457aad9 (DEFAULT_TABLE table=test-workload [id=c1a0d0f5418a4eb1bc7724e39cfb2ce9]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:21:12.236819 22093 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ca42dd42e4d46359256d9256f64d657. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:12.237617 22092 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1631e507d6714dc187fc9d7f2457aad9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:12.240371 22091 tablet_service.cc:1511] Processing CreateTablet for tablet b8791ec386d148c19bb6fb734287e432 (DEFAULT_TABLE table=test-workload [id=c1a0d0f5418a4eb1bc7724e39cfb2ce9]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:21:12.241993 22091 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8791ec386d148c19bb6fb734287e432. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:12.245828 22090 tablet_service.cc:1511] Processing CreateTablet for tablet ea585ca33ebf4b329ce97d386bf3e7bc (DEFAULT_TABLE table=test-workload [id=c1a0d0f5418a4eb1bc7724e39cfb2ce9]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:21:12.247265 22090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea585ca33ebf4b329ce97d386bf3e7bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:12.251106 22089 tablet_service.cc:1511] Processing CreateTablet for tablet d74919ac686e49e694ca111d3b1a27f6 (DEFAULT_TABLE table=test-workload [id=c1a0d0f5418a4eb1bc7724e39cfb2ce9]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:21:12.252616 22089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d74919ac686e49e694ca111d3b1a27f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:12.271123 22088 tablet_service.cc:1511] Processing CreateTablet for tablet ba306baff7874796aba9a7c3fd5d628e (DEFAULT_TABLE table=test-workload [id=c1a0d0f5418a4eb1bc7724e39cfb2ce9]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:21:12.272861 22088 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba306baff7874796aba9a7c3fd5d628e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:12.279054 22142 tablet_bootstrap.cc:492] T 0ca42dd42e4d46359256d9256f64d657 P 21a7bb1d6ef54b20befdf8ea6d293776: Bootstrap starting.
I20260709 13:21:12.286049 22142 tablet_bootstrap.cc:654] T 0ca42dd42e4d46359256d9256f64d657 P 21a7bb1d6ef54b20befdf8ea6d293776: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:12.289000 22093 tablet_service.cc:1511] Processing CreateTablet for tablet 7f2c531192bb4639a60df411741abc7b (DEFAULT_TABLE table=test-workload [id=c1a0d0f5418a4eb1bc7724e39cfb2ce9]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:21:12.294840 22093 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f2c531192bb4639a60df411741abc7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:12.300551 22090 tablet_service.cc:1511] Processing CreateTablet for tablet f9dceef9cf6541039db61f39e2798430 (DEFAULT_TABLE table=test-workload [id=c1a0d0f5418a4eb1bc7724e39cfb2ce9]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:21:12.308394 22090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9dceef9cf6541039db61f39e2798430. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:12.309428 22092 tablet_service.cc:1511] Processing CreateTablet for tablet 5d0c4853c48a42938548ac8810a0bbc4 (DEFAULT_TABLE table=test-workload [id=c1a0d0f5418a4eb1bc7724e39cfb2ce9]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:21:12.310870 22092 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d0c4853c48a42938548ac8810a0bbc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:12.313967 21939 tablet_service.cc:1511] Processing CreateTablet for tablet ba306baff7874796aba9a7c3fd5d628e (DEFAULT_TABLE table=test-workload [id=c1a0d0f5418a4eb1bc7724e39cfb2ce9]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:21:12.315508 21939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba306baff7874796aba9a7c3fd5d628e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:12.315229 21944 tablet_service.cc:1511] Processing CreateTablet for tablet 0ca42dd42e4d46359256d9256f64d657 (DEFAULT_TABLE table=test-workload [id=c1a0d0f5418a4eb1bc7724e39cfb2ce9]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:21:12.316605 21944 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ca42dd42e4d46359256d9256f64d657. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:12.317670 21938 tablet_service.cc:1511] Processing CreateTablet for tablet 7f2c531192bb4639a60df411741abc7b (DEFAULT_TABLE table=test-workload [id=c1a0d0f5418a4eb1bc7724e39cfb2ce9]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:21:12.319144 21938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f2c531192bb4639a60df411741abc7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:12.320175 21943 tablet_service.cc:1511] Processing CreateTablet for tablet 1631e507d6714dc187fc9d7f2457aad9 (DEFAULT_TABLE table=test-workload [id=c1a0d0f5418a4eb1bc7724e39cfb2ce9]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:21:12.321635 21943 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1631e507d6714dc187fc9d7f2457aad9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:12.324131 21936 tablet_service.cc:1511] Processing CreateTablet for tablet 5d0c4853c48a42938548ac8810a0bbc4 (DEFAULT_TABLE table=test-workload [id=c1a0d0f5418a4eb1bc7724e39cfb2ce9]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:21:12.325516 21936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d0c4853c48a42938548ac8810a0bbc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:12.325958 21941 tablet_service.cc:1511] Processing CreateTablet for tablet ea585ca33ebf4b329ce97d386bf3e7bc (DEFAULT_TABLE table=test-workload [id=c1a0d0f5418a4eb1bc7724e39cfb2ce9]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:21:12.327407 21941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea585ca33ebf4b329ce97d386bf3e7bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:12.327469 21937 tablet_service.cc:1511] Processing CreateTablet for tablet f9dceef9cf6541039db61f39e2798430 (DEFAULT_TABLE table=test-workload [id=c1a0d0f5418a4eb1bc7724e39cfb2ce9]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:21:12.328809 21937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9dceef9cf6541039db61f39e2798430. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:12.323979 21942 tablet_service.cc:1511] Processing CreateTablet for tablet b8791ec386d148c19bb6fb734287e432 (DEFAULT_TABLE table=test-workload [id=c1a0d0f5418a4eb1bc7724e39cfb2ce9]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:21:12.329880 21940 tablet_service.cc:1511] Processing CreateTablet for tablet d74919ac686e49e694ca111d3b1a27f6 (DEFAULT_TABLE table=test-workload [id=c1a0d0f5418a4eb1bc7724e39cfb2ce9]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:21:12.330360 21942 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8791ec386d148c19bb6fb734287e432. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:12.331275 21940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d74919ac686e49e694ca111d3b1a27f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:12.381570 22142 tablet_bootstrap.cc:492] T 0ca42dd42e4d46359256d9256f64d657 P 21a7bb1d6ef54b20befdf8ea6d293776: No bootstrap required, opened a new log
I20260709 13:21:12.382280 22142 ts_tablet_manager.cc:1403] T 0ca42dd42e4d46359256d9256f64d657 P 21a7bb1d6ef54b20befdf8ea6d293776: Time spent bootstrapping tablet: real 0.104s	user 0.037s	sys 0.038s
I20260709 13:21:12.385619 22142 raft_consensus.cc:359] T 0ca42dd42e4d46359256d9256f64d657 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } } peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } }
I20260709 13:21:12.386438 22142 raft_consensus.cc:385] T 0ca42dd42e4d46359256d9256f64d657 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:12.386854 22142 raft_consensus.cc:740] T 0ca42dd42e4d46359256d9256f64d657 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21a7bb1d6ef54b20befdf8ea6d293776, State: Initialized, Role: FOLLOWER
I20260709 13:21:12.387889 22142 consensus_queue.cc:260] T 0ca42dd42e4d46359256d9256f64d657 P 21a7bb1d6ef54b20befdf8ea6d293776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } } peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } }
I20260709 13:21:12.388940 22019 tablet_service.cc:1511] Processing CreateTablet for tablet 0ca42dd42e4d46359256d9256f64d657 (DEFAULT_TABLE table=test-workload [id=c1a0d0f5418a4eb1bc7724e39cfb2ce9]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:21:12.390386 22019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ca42dd42e4d46359256d9256f64d657. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:12.392006 22018 tablet_service.cc:1511] Processing CreateTablet for tablet 1631e507d6714dc187fc9d7f2457aad9 (DEFAULT_TABLE table=test-workload [id=c1a0d0f5418a4eb1bc7724e39cfb2ce9]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:21:12.393486 22018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1631e507d6714dc187fc9d7f2457aad9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:12.396400 22017 tablet_service.cc:1511] Processing CreateTablet for tablet b8791ec386d148c19bb6fb734287e432 (DEFAULT_TABLE table=test-workload [id=c1a0d0f5418a4eb1bc7724e39cfb2ce9]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:21:12.397989 22017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8791ec386d148c19bb6fb734287e432. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:12.401880 22015 tablet_service.cc:1511] Processing CreateTablet for tablet d74919ac686e49e694ca111d3b1a27f6 (DEFAULT_TABLE table=test-workload [id=c1a0d0f5418a4eb1bc7724e39cfb2ce9]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:21:12.403463 22015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d74919ac686e49e694ca111d3b1a27f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:12.400108 22016 tablet_service.cc:1511] Processing CreateTablet for tablet ea585ca33ebf4b329ce97d386bf3e7bc (DEFAULT_TABLE table=test-workload [id=c1a0d0f5418a4eb1bc7724e39cfb2ce9]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:21:12.406378 22016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea585ca33ebf4b329ce97d386bf3e7bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:12.407153 22012 tablet_service.cc:1511] Processing CreateTablet for tablet f9dceef9cf6541039db61f39e2798430 (DEFAULT_TABLE table=test-workload [id=c1a0d0f5418a4eb1bc7724e39cfb2ce9]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:21:12.407793 22014 tablet_service.cc:1511] Processing CreateTablet for tablet ba306baff7874796aba9a7c3fd5d628e (DEFAULT_TABLE table=test-workload [id=c1a0d0f5418a4eb1bc7724e39cfb2ce9]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:21:12.408674 22012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9dceef9cf6541039db61f39e2798430. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:12.412048 22011 tablet_service.cc:1511] Processing CreateTablet for tablet 5d0c4853c48a42938548ac8810a0bbc4 (DEFAULT_TABLE table=test-workload [id=c1a0d0f5418a4eb1bc7724e39cfb2ce9]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:21:12.413457 22011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d0c4853c48a42938548ac8810a0bbc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:12.410131 22013 tablet_service.cc:1511] Processing CreateTablet for tablet 7f2c531192bb4639a60df411741abc7b (DEFAULT_TABLE table=test-workload [id=c1a0d0f5418a4eb1bc7724e39cfb2ce9]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:21:12.415717 22013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f2c531192bb4639a60df411741abc7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:12.423836 22014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba306baff7874796aba9a7c3fd5d628e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:12.431327 22145 tablet_bootstrap.cc:492] T f9dceef9cf6541039db61f39e2798430 P 8507b5a0d0e6434aaa7302085c7f54cb: Bootstrap starting.
I20260709 13:21:12.448104 22128 heartbeater.cc:499] Master 127.9.116.190:39063 was elected leader, sending a full tablet report...
I20260709 13:21:12.453236 22142 ts_tablet_manager.cc:1434] T 0ca42dd42e4d46359256d9256f64d657 P 21a7bb1d6ef54b20befdf8ea6d293776: Time spent starting tablet: real 0.071s	user 0.012s	sys 0.009s
I20260709 13:21:12.454348 22142 tablet_bootstrap.cc:492] T ea585ca33ebf4b329ce97d386bf3e7bc P 21a7bb1d6ef54b20befdf8ea6d293776: Bootstrap starting.
I20260709 13:21:12.483844 22145 tablet_bootstrap.cc:654] T f9dceef9cf6541039db61f39e2798430 P 8507b5a0d0e6434aaa7302085c7f54cb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:12.525573 22145 tablet_bootstrap.cc:492] T f9dceef9cf6541039db61f39e2798430 P 8507b5a0d0e6434aaa7302085c7f54cb: No bootstrap required, opened a new log
I20260709 13:21:12.526183 22145 ts_tablet_manager.cc:1403] T f9dceef9cf6541039db61f39e2798430 P 8507b5a0d0e6434aaa7302085c7f54cb: Time spent bootstrapping tablet: real 0.095s	user 0.008s	sys 0.012s
I20260709 13:21:12.529232 22145 raft_consensus.cc:359] T f9dceef9cf6541039db61f39e2798430 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.529954 22145 raft_consensus.cc:385] T f9dceef9cf6541039db61f39e2798430 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:12.530318 22145 raft_consensus.cc:740] T f9dceef9cf6541039db61f39e2798430 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8507b5a0d0e6434aaa7302085c7f54cb, State: Initialized, Role: FOLLOWER
I20260709 13:21:12.534247 22146 tablet_bootstrap.cc:492] T 1631e507d6714dc187fc9d7f2457aad9 P c48712233a8a4eb0b39bb05cd60137e6: Bootstrap starting.
I20260709 13:21:12.537029 22145 consensus_queue.cc:260] T f9dceef9cf6541039db61f39e2798430 P 8507b5a0d0e6434aaa7302085c7f54cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.541898 22146 tablet_bootstrap.cc:654] T 1631e507d6714dc187fc9d7f2457aad9 P c48712233a8a4eb0b39bb05cd60137e6: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:12.543332 22142 tablet_bootstrap.cc:654] T ea585ca33ebf4b329ce97d386bf3e7bc P 21a7bb1d6ef54b20befdf8ea6d293776: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:12.546892 22145 ts_tablet_manager.cc:1434] T f9dceef9cf6541039db61f39e2798430 P 8507b5a0d0e6434aaa7302085c7f54cb: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.007s
I20260709 13:21:12.547806 21979 heartbeater.cc:499] Master 127.9.116.190:39063 was elected leader, sending a full tablet report...
I20260709 13:21:12.551123 22144 raft_consensus.cc:493] T 0ca42dd42e4d46359256d9256f64d657 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:12.551709 22144 raft_consensus.cc:515] T 0ca42dd42e4d46359256d9256f64d657 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } } peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } }
I20260709 13:21:12.553788 22142 tablet_bootstrap.cc:492] T ea585ca33ebf4b329ce97d386bf3e7bc P 21a7bb1d6ef54b20befdf8ea6d293776: No bootstrap required, opened a new log
I20260709 13:21:12.554380 22144 leader_election.cc:290] T 0ca42dd42e4d46359256d9256f64d657 P 21a7bb1d6ef54b20befdf8ea6d293776 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591), c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529)
I20260709 13:21:12.554572 22142 ts_tablet_manager.cc:1403] T ea585ca33ebf4b329ce97d386bf3e7bc P 21a7bb1d6ef54b20befdf8ea6d293776: Time spent bootstrapping tablet: real 0.100s	user 0.011s	sys 0.003s
I20260709 13:21:12.555610 22145 tablet_bootstrap.cc:492] T 1631e507d6714dc187fc9d7f2457aad9 P 8507b5a0d0e6434aaa7302085c7f54cb: Bootstrap starting.
I20260709 13:21:12.559166 22142 raft_consensus.cc:359] T ea585ca33ebf4b329ce97d386bf3e7bc P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.560181 22142 raft_consensus.cc:385] T ea585ca33ebf4b329ce97d386bf3e7bc P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:12.560547 22142 raft_consensus.cc:740] T ea585ca33ebf4b329ce97d386bf3e7bc P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21a7bb1d6ef54b20befdf8ea6d293776, State: Initialized, Role: FOLLOWER
I20260709 13:21:12.561503 22142 consensus_queue.cc:260] T ea585ca33ebf4b329ce97d386bf3e7bc P 21a7bb1d6ef54b20befdf8ea6d293776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.564922 22145 tablet_bootstrap.cc:654] T 1631e507d6714dc187fc9d7f2457aad9 P 8507b5a0d0e6434aaa7302085c7f54cb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:12.565551 22142 ts_tablet_manager.cc:1434] T ea585ca33ebf4b329ce97d386bf3e7bc P 21a7bb1d6ef54b20befdf8ea6d293776: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.001s
I20260709 13:21:12.566856 22142 tablet_bootstrap.cc:492] T 1631e507d6714dc187fc9d7f2457aad9 P 21a7bb1d6ef54b20befdf8ea6d293776: Bootstrap starting.
I20260709 13:21:12.568750 22146 tablet_bootstrap.cc:492] T 1631e507d6714dc187fc9d7f2457aad9 P c48712233a8a4eb0b39bb05cd60137e6: No bootstrap required, opened a new log
I20260709 13:21:12.569409 22146 ts_tablet_manager.cc:1403] T 1631e507d6714dc187fc9d7f2457aad9 P c48712233a8a4eb0b39bb05cd60137e6: Time spent bootstrapping tablet: real 0.036s	user 0.011s	sys 0.020s
I20260709 13:21:12.572557 22146 raft_consensus.cc:359] T 1631e507d6714dc187fc9d7f2457aad9 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.573416 22146 raft_consensus.cc:385] T 1631e507d6714dc187fc9d7f2457aad9 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:12.573487 22142 tablet_bootstrap.cc:654] T 1631e507d6714dc187fc9d7f2457aad9 P 21a7bb1d6ef54b20befdf8ea6d293776: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:12.573863 22146 raft_consensus.cc:740] T 1631e507d6714dc187fc9d7f2457aad9 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c48712233a8a4eb0b39bb05cd60137e6, State: Initialized, Role: FOLLOWER
I20260709 13:21:12.574928 22146 consensus_queue.cc:260] T 1631e507d6714dc187fc9d7f2457aad9 P c48712233a8a4eb0b39bb05cd60137e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.594333 21954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ca42dd42e4d46359256d9256f64d657" candidate_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" is_pre_election: true
W20260709 13:21:12.598500 22069 leader_election.cc:343] T 0ca42dd42e4d46359256d9256f64d657 P 21a7bb1d6ef54b20befdf8ea6d293776 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:12.616304 22029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ca42dd42e4d46359256d9256f64d657" candidate_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6" is_pre_election: true
W20260709 13:21:12.618383 22069 leader_election.cc:343] T 0ca42dd42e4d46359256d9256f64d657 P 21a7bb1d6ef54b20befdf8ea6d293776 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:12.618949 22146 ts_tablet_manager.cc:1434] T 1631e507d6714dc187fc9d7f2457aad9 P c48712233a8a4eb0b39bb05cd60137e6: Time spent starting tablet: real 0.049s	user 0.009s	sys 0.021s
I20260709 13:21:12.620244 22054 heartbeater.cc:499] Master 127.9.116.190:39063 was elected leader, sending a full tablet report...
I20260709 13:21:12.619184 22069 leader_election.cc:304] T 0ca42dd42e4d46359256d9256f64d657 P 21a7bb1d6ef54b20befdf8ea6d293776 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 21a7bb1d6ef54b20befdf8ea6d293776; no voters: 8507b5a0d0e6434aaa7302085c7f54cb, c48712233a8a4eb0b39bb05cd60137e6
I20260709 13:21:12.621914 22144 raft_consensus.cc:2749] T 0ca42dd42e4d46359256d9256f64d657 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:12.626993 22145 tablet_bootstrap.cc:492] T 1631e507d6714dc187fc9d7f2457aad9 P 8507b5a0d0e6434aaa7302085c7f54cb: No bootstrap required, opened a new log
I20260709 13:21:12.627660 22145 ts_tablet_manager.cc:1403] T 1631e507d6714dc187fc9d7f2457aad9 P 8507b5a0d0e6434aaa7302085c7f54cb: Time spent bootstrapping tablet: real 0.072s	user 0.032s	sys 0.036s
I20260709 13:21:12.628100 22142 tablet_bootstrap.cc:492] T 1631e507d6714dc187fc9d7f2457aad9 P 21a7bb1d6ef54b20befdf8ea6d293776: No bootstrap required, opened a new log
I20260709 13:21:12.628767 22142 ts_tablet_manager.cc:1403] T 1631e507d6714dc187fc9d7f2457aad9 P 21a7bb1d6ef54b20befdf8ea6d293776: Time spent bootstrapping tablet: real 0.062s	user 0.017s	sys 0.039s
I20260709 13:21:12.628090 22146 tablet_bootstrap.cc:492] T 7f2c531192bb4639a60df411741abc7b P c48712233a8a4eb0b39bb05cd60137e6: Bootstrap starting.
I20260709 13:21:12.631160 22145 raft_consensus.cc:359] T 1631e507d6714dc187fc9d7f2457aad9 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.632022 22145 raft_consensus.cc:385] T 1631e507d6714dc187fc9d7f2457aad9 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:12.632481 22145 raft_consensus.cc:740] T 1631e507d6714dc187fc9d7f2457aad9 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8507b5a0d0e6434aaa7302085c7f54cb, State: Initialized, Role: FOLLOWER
I20260709 13:21:12.632054 22142 raft_consensus.cc:359] T 1631e507d6714dc187fc9d7f2457aad9 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.633033 22142 raft_consensus.cc:385] T 1631e507d6714dc187fc9d7f2457aad9 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:12.633400 22142 raft_consensus.cc:740] T 1631e507d6714dc187fc9d7f2457aad9 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21a7bb1d6ef54b20befdf8ea6d293776, State: Initialized, Role: FOLLOWER
I20260709 13:21:12.633481 22145 consensus_queue.cc:260] T 1631e507d6714dc187fc9d7f2457aad9 P 8507b5a0d0e6434aaa7302085c7f54cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.634331 22142 consensus_queue.cc:260] T 1631e507d6714dc187fc9d7f2457aad9 P 21a7bb1d6ef54b20befdf8ea6d293776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.636225 22145 ts_tablet_manager.cc:1434] T 1631e507d6714dc187fc9d7f2457aad9 P 8507b5a0d0e6434aaa7302085c7f54cb: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:21:12.636952 22142 ts_tablet_manager.cc:1434] T 1631e507d6714dc187fc9d7f2457aad9 P 21a7bb1d6ef54b20befdf8ea6d293776: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:21:12.637383 22145 tablet_bootstrap.cc:492] T b8791ec386d148c19bb6fb734287e432 P 8507b5a0d0e6434aaa7302085c7f54cb: Bootstrap starting.
I20260709 13:21:12.638058 22142 tablet_bootstrap.cc:492] T b8791ec386d148c19bb6fb734287e432 P 21a7bb1d6ef54b20befdf8ea6d293776: Bootstrap starting.
I20260709 13:21:12.642400 22146 tablet_bootstrap.cc:654] T 7f2c531192bb4639a60df411741abc7b P c48712233a8a4eb0b39bb05cd60137e6: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:12.645931 22142 tablet_bootstrap.cc:654] T b8791ec386d148c19bb6fb734287e432 P 21a7bb1d6ef54b20befdf8ea6d293776: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:12.648713 22145 tablet_bootstrap.cc:654] T b8791ec386d148c19bb6fb734287e432 P 8507b5a0d0e6434aaa7302085c7f54cb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:12.663187 22145 tablet_bootstrap.cc:492] T b8791ec386d148c19bb6fb734287e432 P 8507b5a0d0e6434aaa7302085c7f54cb: No bootstrap required, opened a new log
I20260709 13:21:12.664038 22145 ts_tablet_manager.cc:1403] T b8791ec386d148c19bb6fb734287e432 P 8507b5a0d0e6434aaa7302085c7f54cb: Time spent bootstrapping tablet: real 0.027s	user 0.018s	sys 0.004s
I20260709 13:21:12.666117 22142 tablet_bootstrap.cc:492] T b8791ec386d148c19bb6fb734287e432 P 21a7bb1d6ef54b20befdf8ea6d293776: No bootstrap required, opened a new log
I20260709 13:21:12.666821 22142 ts_tablet_manager.cc:1403] T b8791ec386d148c19bb6fb734287e432 P 21a7bb1d6ef54b20befdf8ea6d293776: Time spent bootstrapping tablet: real 0.029s	user 0.017s	sys 0.011s
I20260709 13:21:12.667583 22146 tablet_bootstrap.cc:492] T 7f2c531192bb4639a60df411741abc7b P c48712233a8a4eb0b39bb05cd60137e6: No bootstrap required, opened a new log
I20260709 13:21:12.667194 22145 raft_consensus.cc:359] T b8791ec386d148c19bb6fb734287e432 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.668249 22146 ts_tablet_manager.cc:1403] T 7f2c531192bb4639a60df411741abc7b P c48712233a8a4eb0b39bb05cd60137e6: Time spent bootstrapping tablet: real 0.040s	user 0.019s	sys 0.007s
I20260709 13:21:12.668259 22145 raft_consensus.cc:385] T b8791ec386d148c19bb6fb734287e432 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:12.668870 22145 raft_consensus.cc:740] T b8791ec386d148c19bb6fb734287e432 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8507b5a0d0e6434aaa7302085c7f54cb, State: Initialized, Role: FOLLOWER
I20260709 13:21:12.669936 22142 raft_consensus.cc:359] T b8791ec386d148c19bb6fb734287e432 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.669847 22145 consensus_queue.cc:260] T b8791ec386d148c19bb6fb734287e432 P 8507b5a0d0e6434aaa7302085c7f54cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.670711 22142 raft_consensus.cc:385] T b8791ec386d148c19bb6fb734287e432 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:12.671119 22142 raft_consensus.cc:740] T b8791ec386d148c19bb6fb734287e432 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21a7bb1d6ef54b20befdf8ea6d293776, State: Initialized, Role: FOLLOWER
I20260709 13:21:12.671705 22146 raft_consensus.cc:359] T 7f2c531192bb4639a60df411741abc7b P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.672115 22142 consensus_queue.cc:260] T b8791ec386d148c19bb6fb734287e432 P 21a7bb1d6ef54b20befdf8ea6d293776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.672518 22145 ts_tablet_manager.cc:1434] T b8791ec386d148c19bb6fb734287e432 P 8507b5a0d0e6434aaa7302085c7f54cb: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.003s
I20260709 13:21:12.672580 22146 raft_consensus.cc:385] T 7f2c531192bb4639a60df411741abc7b P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:12.673285 22146 raft_consensus.cc:740] T 7f2c531192bb4639a60df411741abc7b P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c48712233a8a4eb0b39bb05cd60137e6, State: Initialized, Role: FOLLOWER
I20260709 13:21:12.673861 22145 tablet_bootstrap.cc:492] T d74919ac686e49e694ca111d3b1a27f6 P 8507b5a0d0e6434aaa7302085c7f54cb: Bootstrap starting.
I20260709 13:21:12.674196 22146 consensus_queue.cc:260] T 7f2c531192bb4639a60df411741abc7b P c48712233a8a4eb0b39bb05cd60137e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.676491 22142 ts_tablet_manager.cc:1434] T b8791ec386d148c19bb6fb734287e432 P 21a7bb1d6ef54b20befdf8ea6d293776: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:21:12.677708 22142 tablet_bootstrap.cc:492] T ba306baff7874796aba9a7c3fd5d628e P 21a7bb1d6ef54b20befdf8ea6d293776: Bootstrap starting.
I20260709 13:21:12.681383 22145 tablet_bootstrap.cc:654] T d74919ac686e49e694ca111d3b1a27f6 P 8507b5a0d0e6434aaa7302085c7f54cb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:12.683629 22146 ts_tablet_manager.cc:1434] T 7f2c531192bb4639a60df411741abc7b P c48712233a8a4eb0b39bb05cd60137e6: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:21:12.684690 22142 tablet_bootstrap.cc:654] T ba306baff7874796aba9a7c3fd5d628e P 21a7bb1d6ef54b20befdf8ea6d293776: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:12.691592 22146 tablet_bootstrap.cc:492] T 5d0c4853c48a42938548ac8810a0bbc4 P c48712233a8a4eb0b39bb05cd60137e6: Bootstrap starting.
I20260709 13:21:12.696686 22145 tablet_bootstrap.cc:492] T d74919ac686e49e694ca111d3b1a27f6 P 8507b5a0d0e6434aaa7302085c7f54cb: No bootstrap required, opened a new log
I20260709 13:21:12.697350 22145 ts_tablet_manager.cc:1403] T d74919ac686e49e694ca111d3b1a27f6 P 8507b5a0d0e6434aaa7302085c7f54cb: Time spent bootstrapping tablet: real 0.024s	user 0.012s	sys 0.007s
I20260709 13:21:12.699352 22146 tablet_bootstrap.cc:654] T 5d0c4853c48a42938548ac8810a0bbc4 P c48712233a8a4eb0b39bb05cd60137e6: Neither blocks nor log segments found. Creating new log.
W20260709 13:21:12.700096 21900 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:21:12.700608 22145 raft_consensus.cc:359] T d74919ac686e49e694ca111d3b1a27f6 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.701229 22145 raft_consensus.cc:385] T d74919ac686e49e694ca111d3b1a27f6 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:12.701464 22145 raft_consensus.cc:740] T d74919ac686e49e694ca111d3b1a27f6 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8507b5a0d0e6434aaa7302085c7f54cb, State: Initialized, Role: FOLLOWER
I20260709 13:21:12.702283 22145 consensus_queue.cc:260] T d74919ac686e49e694ca111d3b1a27f6 P 8507b5a0d0e6434aaa7302085c7f54cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.696689 22142 tablet_bootstrap.cc:492] T ba306baff7874796aba9a7c3fd5d628e P 21a7bb1d6ef54b20befdf8ea6d293776: No bootstrap required, opened a new log
I20260709 13:21:12.703356 22142 ts_tablet_manager.cc:1403] T ba306baff7874796aba9a7c3fd5d628e P 21a7bb1d6ef54b20befdf8ea6d293776: Time spent bootstrapping tablet: real 0.026s	user 0.009s	sys 0.009s
I20260709 13:21:12.704303 22146 tablet_bootstrap.cc:492] T 5d0c4853c48a42938548ac8810a0bbc4 P c48712233a8a4eb0b39bb05cd60137e6: No bootstrap required, opened a new log
I20260709 13:21:12.704736 22146 ts_tablet_manager.cc:1403] T 5d0c4853c48a42938548ac8810a0bbc4 P c48712233a8a4eb0b39bb05cd60137e6: Time spent bootstrapping tablet: real 0.013s	user 0.011s	sys 0.000s
I20260709 13:21:12.705663 22142 raft_consensus.cc:359] T ba306baff7874796aba9a7c3fd5d628e P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.706274 22142 raft_consensus.cc:385] T ba306baff7874796aba9a7c3fd5d628e P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:12.706519 22142 raft_consensus.cc:740] T ba306baff7874796aba9a7c3fd5d628e P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21a7bb1d6ef54b20befdf8ea6d293776, State: Initialized, Role: FOLLOWER
I20260709 13:21:12.706986 22146 raft_consensus.cc:359] T 5d0c4853c48a42938548ac8810a0bbc4 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.707787 22146 raft_consensus.cc:385] T 5d0c4853c48a42938548ac8810a0bbc4 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:12.707298 22142 consensus_queue.cc:260] T ba306baff7874796aba9a7c3fd5d628e P 21a7bb1d6ef54b20befdf8ea6d293776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.708119 22146 raft_consensus.cc:740] T 5d0c4853c48a42938548ac8810a0bbc4 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c48712233a8a4eb0b39bb05cd60137e6, State: Initialized, Role: FOLLOWER
I20260709 13:21:12.709019 22145 ts_tablet_manager.cc:1434] T d74919ac686e49e694ca111d3b1a27f6 P 8507b5a0d0e6434aaa7302085c7f54cb: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:21:12.710269 22145 tablet_bootstrap.cc:492] T 0ca42dd42e4d46359256d9256f64d657 P 8507b5a0d0e6434aaa7302085c7f54cb: Bootstrap starting.
I20260709 13:21:12.709097 22146 consensus_queue.cc:260] T 5d0c4853c48a42938548ac8810a0bbc4 P c48712233a8a4eb0b39bb05cd60137e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.714397 22142 ts_tablet_manager.cc:1434] T ba306baff7874796aba9a7c3fd5d628e P 21a7bb1d6ef54b20befdf8ea6d293776: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 13:21:12.715704 22146 ts_tablet_manager.cc:1434] T 5d0c4853c48a42938548ac8810a0bbc4 P c48712233a8a4eb0b39bb05cd60137e6: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:21:12.716123 22142 tablet_bootstrap.cc:492] T f9dceef9cf6541039db61f39e2798430 P 21a7bb1d6ef54b20befdf8ea6d293776: Bootstrap starting.
I20260709 13:21:12.716950 22146 tablet_bootstrap.cc:492] T b8791ec386d148c19bb6fb734287e432 P c48712233a8a4eb0b39bb05cd60137e6: Bootstrap starting.
I20260709 13:21:12.718500 22145 tablet_bootstrap.cc:654] T 0ca42dd42e4d46359256d9256f64d657 P 8507b5a0d0e6434aaa7302085c7f54cb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:12.725826 22146 tablet_bootstrap.cc:654] T b8791ec386d148c19bb6fb734287e432 P c48712233a8a4eb0b39bb05cd60137e6: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:12.726511 22142 tablet_bootstrap.cc:654] T f9dceef9cf6541039db61f39e2798430 P 21a7bb1d6ef54b20befdf8ea6d293776: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:12.741155 22146 tablet_bootstrap.cc:492] T b8791ec386d148c19bb6fb734287e432 P c48712233a8a4eb0b39bb05cd60137e6: No bootstrap required, opened a new log
I20260709 13:21:12.741797 22146 ts_tablet_manager.cc:1403] T b8791ec386d148c19bb6fb734287e432 P c48712233a8a4eb0b39bb05cd60137e6: Time spent bootstrapping tablet: real 0.025s	user 0.017s	sys 0.001s
I20260709 13:21:12.741994 22142 tablet_bootstrap.cc:492] T f9dceef9cf6541039db61f39e2798430 P 21a7bb1d6ef54b20befdf8ea6d293776: No bootstrap required, opened a new log
I20260709 13:21:12.742318 22145 tablet_bootstrap.cc:492] T 0ca42dd42e4d46359256d9256f64d657 P 8507b5a0d0e6434aaa7302085c7f54cb: No bootstrap required, opened a new log
I20260709 13:21:12.742956 22145 ts_tablet_manager.cc:1403] T 0ca42dd42e4d46359256d9256f64d657 P 8507b5a0d0e6434aaa7302085c7f54cb: Time spent bootstrapping tablet: real 0.033s	user 0.008s	sys 0.022s
I20260709 13:21:12.742743 22142 ts_tablet_manager.cc:1403] T f9dceef9cf6541039db61f39e2798430 P 21a7bb1d6ef54b20befdf8ea6d293776: Time spent bootstrapping tablet: real 0.027s	user 0.012s	sys 0.011s
I20260709 13:21:12.745587 22142 raft_consensus.cc:359] T f9dceef9cf6541039db61f39e2798430 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.745442 22146 raft_consensus.cc:359] T b8791ec386d148c19bb6fb734287e432 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.746269 22142 raft_consensus.cc:385] T f9dceef9cf6541039db61f39e2798430 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:12.746446 22146 raft_consensus.cc:385] T b8791ec386d148c19bb6fb734287e432 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:12.746255 22145 raft_consensus.cc:359] T 0ca42dd42e4d46359256d9256f64d657 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } } peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } }
I20260709 13:21:12.747022 22142 raft_consensus.cc:740] T f9dceef9cf6541039db61f39e2798430 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21a7bb1d6ef54b20befdf8ea6d293776, State: Initialized, Role: FOLLOWER
I20260709 13:21:12.747088 22146 raft_consensus.cc:740] T b8791ec386d148c19bb6fb734287e432 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c48712233a8a4eb0b39bb05cd60137e6, State: Initialized, Role: FOLLOWER
I20260709 13:21:12.747277 22145 raft_consensus.cc:385] T 0ca42dd42e4d46359256d9256f64d657 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:12.747716 22145 raft_consensus.cc:740] T 0ca42dd42e4d46359256d9256f64d657 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8507b5a0d0e6434aaa7302085c7f54cb, State: Initialized, Role: FOLLOWER
I20260709 13:21:12.747938 22142 consensus_queue.cc:260] T f9dceef9cf6541039db61f39e2798430 P 21a7bb1d6ef54b20befdf8ea6d293776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.748224 22146 consensus_queue.cc:260] T b8791ec386d148c19bb6fb734287e432 P c48712233a8a4eb0b39bb05cd60137e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.748503 22145 consensus_queue.cc:260] T 0ca42dd42e4d46359256d9256f64d657 P 8507b5a0d0e6434aaa7302085c7f54cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } } peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } }
I20260709 13:21:12.750321 22142 ts_tablet_manager.cc:1434] T f9dceef9cf6541039db61f39e2798430 P 21a7bb1d6ef54b20befdf8ea6d293776: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:21:12.751467 22142 tablet_bootstrap.cc:492] T d74919ac686e49e694ca111d3b1a27f6 P 21a7bb1d6ef54b20befdf8ea6d293776: Bootstrap starting.
I20260709 13:21:12.752775 22145 ts_tablet_manager.cc:1434] T 0ca42dd42e4d46359256d9256f64d657 P 8507b5a0d0e6434aaa7302085c7f54cb: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 13:21:12.753960 22145 tablet_bootstrap.cc:492] T ba306baff7874796aba9a7c3fd5d628e P 8507b5a0d0e6434aaa7302085c7f54cb: Bootstrap starting.
I20260709 13:21:12.755306 22146 ts_tablet_manager.cc:1434] T b8791ec386d148c19bb6fb734287e432 P c48712233a8a4eb0b39bb05cd60137e6: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:21:12.756441 22146 tablet_bootstrap.cc:492] T f9dceef9cf6541039db61f39e2798430 P c48712233a8a4eb0b39bb05cd60137e6: Bootstrap starting.
I20260709 13:21:12.758687 22142 tablet_bootstrap.cc:654] T d74919ac686e49e694ca111d3b1a27f6 P 21a7bb1d6ef54b20befdf8ea6d293776: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:12.760849 22145 tablet_bootstrap.cc:654] T ba306baff7874796aba9a7c3fd5d628e P 8507b5a0d0e6434aaa7302085c7f54cb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:12.762844 22153 raft_consensus.cc:493] T 1631e507d6714dc187fc9d7f2457aad9 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:12.763600 22153 raft_consensus.cc:515] T 1631e507d6714dc187fc9d7f2457aad9 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.768692 22146 tablet_bootstrap.cc:654] T f9dceef9cf6541039db61f39e2798430 P c48712233a8a4eb0b39bb05cd60137e6: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:12.770699 22142 tablet_bootstrap.cc:492] T d74919ac686e49e694ca111d3b1a27f6 P 21a7bb1d6ef54b20befdf8ea6d293776: No bootstrap required, opened a new log
I20260709 13:21:12.771394 22142 ts_tablet_manager.cc:1403] T d74919ac686e49e694ca111d3b1a27f6 P 21a7bb1d6ef54b20befdf8ea6d293776: Time spent bootstrapping tablet: real 0.020s	user 0.009s	sys 0.010s
I20260709 13:21:12.774328 22153 leader_election.cc:290] T 1631e507d6714dc187fc9d7f2457aad9 P c48712233a8a4eb0b39bb05cd60137e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591), 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131:33869)
I20260709 13:21:12.787511 22142 raft_consensus.cc:359] T d74919ac686e49e694ca111d3b1a27f6 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.790082 22142 raft_consensus.cc:385] T d74919ac686e49e694ca111d3b1a27f6 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:12.790525 22142 raft_consensus.cc:740] T d74919ac686e49e694ca111d3b1a27f6 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21a7bb1d6ef54b20befdf8ea6d293776, State: Initialized, Role: FOLLOWER
I20260709 13:21:12.791476 22142 consensus_queue.cc:260] T d74919ac686e49e694ca111d3b1a27f6 P 21a7bb1d6ef54b20befdf8ea6d293776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.792059 22145 tablet_bootstrap.cc:492] T ba306baff7874796aba9a7c3fd5d628e P 8507b5a0d0e6434aaa7302085c7f54cb: No bootstrap required, opened a new log
I20260709 13:21:12.792989 22145 ts_tablet_manager.cc:1403] T ba306baff7874796aba9a7c3fd5d628e P 8507b5a0d0e6434aaa7302085c7f54cb: Time spent bootstrapping tablet: real 0.039s	user 0.015s	sys 0.023s
I20260709 13:21:12.794137 22142 ts_tablet_manager.cc:1434] T d74919ac686e49e694ca111d3b1a27f6 P 21a7bb1d6ef54b20befdf8ea6d293776: Time spent starting tablet: real 0.022s	user 0.001s	sys 0.005s
I20260709 13:21:12.795289 22142 tablet_bootstrap.cc:492] T 7f2c531192bb4639a60df411741abc7b P 21a7bb1d6ef54b20befdf8ea6d293776: Bootstrap starting.
I20260709 13:21:12.799147 22145 raft_consensus.cc:359] T ba306baff7874796aba9a7c3fd5d628e P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.800118 22145 raft_consensus.cc:385] T ba306baff7874796aba9a7c3fd5d628e P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:12.800559 22145 raft_consensus.cc:740] T ba306baff7874796aba9a7c3fd5d628e P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8507b5a0d0e6434aaa7302085c7f54cb, State: Initialized, Role: FOLLOWER
I20260709 13:21:12.801530 22145 consensus_queue.cc:260] T ba306baff7874796aba9a7c3fd5d628e P 8507b5a0d0e6434aaa7302085c7f54cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.806486 22142 tablet_bootstrap.cc:654] T 7f2c531192bb4639a60df411741abc7b P 21a7bb1d6ef54b20befdf8ea6d293776: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:12.806664 22146 tablet_bootstrap.cc:492] T f9dceef9cf6541039db61f39e2798430 P c48712233a8a4eb0b39bb05cd60137e6: No bootstrap required, opened a new log
I20260709 13:21:12.807451 22146 ts_tablet_manager.cc:1403] T f9dceef9cf6541039db61f39e2798430 P c48712233a8a4eb0b39bb05cd60137e6: Time spent bootstrapping tablet: real 0.051s	user 0.023s	sys 0.004s
I20260709 13:21:12.811255 22145 ts_tablet_manager.cc:1434] T ba306baff7874796aba9a7c3fd5d628e P 8507b5a0d0e6434aaa7302085c7f54cb: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 13:21:12.815060 22103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1631e507d6714dc187fc9d7f2457aad9" candidate_uuid: "c48712233a8a4eb0b39bb05cd60137e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" is_pre_election: true
I20260709 13:21:12.815815 22103 raft_consensus.cc:2468] T 1631e507d6714dc187fc9d7f2457aad9 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c48712233a8a4eb0b39bb05cd60137e6 in term 0.
I20260709 13:21:12.816617 22148 raft_consensus.cc:493] T f9dceef9cf6541039db61f39e2798430 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:12.817210 22148 raft_consensus.cc:515] T f9dceef9cf6541039db61f39e2798430 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.817457 21992 leader_election.cc:304] T 1631e507d6714dc187fc9d7f2457aad9 P c48712233a8a4eb0b39bb05cd60137e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a7bb1d6ef54b20befdf8ea6d293776, c48712233a8a4eb0b39bb05cd60137e6; no voters: 
I20260709 13:21:12.818234 22142 tablet_bootstrap.cc:492] T 7f2c531192bb4639a60df411741abc7b P 21a7bb1d6ef54b20befdf8ea6d293776: No bootstrap required, opened a new log
I20260709 13:21:12.818426 22153 raft_consensus.cc:2804] T 1631e507d6714dc187fc9d7f2457aad9 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:12.818745 22142 ts_tablet_manager.cc:1403] T 7f2c531192bb4639a60df411741abc7b P 21a7bb1d6ef54b20befdf8ea6d293776: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.005s
I20260709 13:21:12.818859 22153 raft_consensus.cc:493] T 1631e507d6714dc187fc9d7f2457aad9 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:12.819396 22153 raft_consensus.cc:3060] T 1631e507d6714dc187fc9d7f2457aad9 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:12.821795 22142 raft_consensus.cc:359] T 7f2c531192bb4639a60df411741abc7b P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.822582 22142 raft_consensus.cc:385] T 7f2c531192bb4639a60df411741abc7b P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:12.822956 22142 raft_consensus.cc:740] T 7f2c531192bb4639a60df411741abc7b P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21a7bb1d6ef54b20befdf8ea6d293776, State: Initialized, Role: FOLLOWER
I20260709 13:21:12.823616 22142 consensus_queue.cc:260] T 7f2c531192bb4639a60df411741abc7b P 21a7bb1d6ef54b20befdf8ea6d293776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.824492 22144 raft_consensus.cc:493] T b8791ec386d148c19bb6fb734287e432 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:12.825116 22144 raft_consensus.cc:515] T b8791ec386d148c19bb6fb734287e432 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.826601 22148 leader_election.cc:290] T f9dceef9cf6541039db61f39e2798430 P 8507b5a0d0e6434aaa7302085c7f54cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529), 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131:33869)
I20260709 13:21:12.827880 22144 leader_election.cc:290] T b8791ec386d148c19bb6fb734287e432 P 21a7bb1d6ef54b20befdf8ea6d293776 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591), c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529)
I20260709 13:21:12.833252 21954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8791ec386d148c19bb6fb734287e432" candidate_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" is_pre_election: true
I20260709 13:21:12.833969 21954 raft_consensus.cc:2468] T b8791ec386d148c19bb6fb734287e432 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21a7bb1d6ef54b20befdf8ea6d293776 in term 0.
I20260709 13:21:12.835356 22145 tablet_bootstrap.cc:492] T 7f2c531192bb4639a60df411741abc7b P 8507b5a0d0e6434aaa7302085c7f54cb: Bootstrap starting.
I20260709 13:21:12.836757 22069 leader_election.cc:304] T b8791ec386d148c19bb6fb734287e432 P 21a7bb1d6ef54b20befdf8ea6d293776 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a7bb1d6ef54b20befdf8ea6d293776, 8507b5a0d0e6434aaa7302085c7f54cb; no voters: 
I20260709 13:21:12.837049 22029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8791ec386d148c19bb6fb734287e432" candidate_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6" is_pre_election: true
I20260709 13:21:12.838038 22029 raft_consensus.cc:2468] T b8791ec386d148c19bb6fb734287e432 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21a7bb1d6ef54b20befdf8ea6d293776 in term 0.
I20260709 13:21:12.839637 22144 raft_consensus.cc:2804] T b8791ec386d148c19bb6fb734287e432 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:12.840123 22144 raft_consensus.cc:493] T b8791ec386d148c19bb6fb734287e432 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:12.840448 22144 raft_consensus.cc:3060] T b8791ec386d148c19bb6fb734287e432 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:12.841351 22142 ts_tablet_manager.cc:1434] T 7f2c531192bb4639a60df411741abc7b P 21a7bb1d6ef54b20befdf8ea6d293776: Time spent starting tablet: real 0.022s	user 0.009s	sys 0.003s
I20260709 13:21:12.810849 22146 raft_consensus.cc:359] T f9dceef9cf6541039db61f39e2798430 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.842234 22146 raft_consensus.cc:385] T f9dceef9cf6541039db61f39e2798430 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:12.844453 22146 raft_consensus.cc:740] T f9dceef9cf6541039db61f39e2798430 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c48712233a8a4eb0b39bb05cd60137e6, State: Initialized, Role: FOLLOWER
I20260709 13:21:12.845511 22146 consensus_queue.cc:260] T f9dceef9cf6541039db61f39e2798430 P c48712233a8a4eb0b39bb05cd60137e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.842480 22142 tablet_bootstrap.cc:492] T 5d0c4853c48a42938548ac8810a0bbc4 P 21a7bb1d6ef54b20befdf8ea6d293776: Bootstrap starting.
I20260709 13:21:12.850064 21954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1631e507d6714dc187fc9d7f2457aad9" candidate_uuid: "c48712233a8a4eb0b39bb05cd60137e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" is_pre_election: true
I20260709 13:21:12.850797 21954 raft_consensus.cc:2468] T 1631e507d6714dc187fc9d7f2457aad9 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c48712233a8a4eb0b39bb05cd60137e6 in term 0.
I20260709 13:21:12.847843 22145 tablet_bootstrap.cc:654] T 7f2c531192bb4639a60df411741abc7b P 8507b5a0d0e6434aaa7302085c7f54cb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:12.843621 22153 raft_consensus.cc:515] T 1631e507d6714dc187fc9d7f2457aad9 P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.854369 22174 raft_consensus.cc:493] T d74919ac686e49e694ca111d3b1a27f6 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:12.855178 21954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1631e507d6714dc187fc9d7f2457aad9" candidate_uuid: "c48712233a8a4eb0b39bb05cd60137e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb"
I20260709 13:21:12.855007 22174 raft_consensus.cc:515] T d74919ac686e49e694ca111d3b1a27f6 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.855538 22103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1631e507d6714dc187fc9d7f2457aad9" candidate_uuid: "c48712233a8a4eb0b39bb05cd60137e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776"
I20260709 13:21:12.855662 22142 tablet_bootstrap.cc:654] T 5d0c4853c48a42938548ac8810a0bbc4 P 21a7bb1d6ef54b20befdf8ea6d293776: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:12.856410 22103 raft_consensus.cc:3060] T 1631e507d6714dc187fc9d7f2457aad9 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:12.853802 22144 raft_consensus.cc:515] T b8791ec386d148c19bb6fb734287e432 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.858295 21953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d74919ac686e49e694ca111d3b1a27f6" candidate_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" is_pre_election: true
I20260709 13:21:12.855822 21954 raft_consensus.cc:3060] T 1631e507d6714dc187fc9d7f2457aad9 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:12.858996 21953 raft_consensus.cc:2468] T d74919ac686e49e694ca111d3b1a27f6 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21a7bb1d6ef54b20befdf8ea6d293776 in term 0.
I20260709 13:21:12.859684 22144 leader_election.cc:290] T b8791ec386d148c19bb6fb734287e432 P 21a7bb1d6ef54b20befdf8ea6d293776 [CANDIDATE]: Term 1 election: Requested vote from peers 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591), c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529)
I20260709 13:21:12.860231 22144 raft_consensus.cc:493] T ea585ca33ebf4b329ce97d386bf3e7bc P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:12.860668 22144 raft_consensus.cc:515] T ea585ca33ebf4b329ce97d386bf3e7bc P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.862330 22029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8791ec386d148c19bb6fb734287e432" candidate_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6"
I20260709 13:21:12.861150 21953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8791ec386d148c19bb6fb734287e432" candidate_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb"
I20260709 13:21:12.863101 22029 raft_consensus.cc:3060] T b8791ec386d148c19bb6fb734287e432 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:12.869474 22146 ts_tablet_manager.cc:1434] T f9dceef9cf6541039db61f39e2798430 P c48712233a8a4eb0b39bb05cd60137e6: Time spent starting tablet: real 0.062s	user 0.006s	sys 0.007s
I20260709 13:21:12.870558 22146 tablet_bootstrap.cc:492] T ea585ca33ebf4b329ce97d386bf3e7bc P c48712233a8a4eb0b39bb05cd60137e6: Bootstrap starting.
I20260709 13:21:12.873726 21954 raft_consensus.cc:2468] T 1631e507d6714dc187fc9d7f2457aad9 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c48712233a8a4eb0b39bb05cd60137e6 in term 1.
I20260709 13:21:12.863988 21953 raft_consensus.cc:3060] T b8791ec386d148c19bb6fb734287e432 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:12.877724 22103 raft_consensus.cc:2468] T 1631e507d6714dc187fc9d7f2457aad9 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c48712233a8a4eb0b39bb05cd60137e6 in term 1.
I20260709 13:21:12.879387 21992 leader_election.cc:304] T 1631e507d6714dc187fc9d7f2457aad9 P c48712233a8a4eb0b39bb05cd60137e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a7bb1d6ef54b20befdf8ea6d293776, c48712233a8a4eb0b39bb05cd60137e6; no voters: 
I20260709 13:21:12.881803 22175 raft_consensus.cc:2804] T 1631e507d6714dc187fc9d7f2457aad9 P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:12.882511 22146 tablet_bootstrap.cc:654] T ea585ca33ebf4b329ce97d386bf3e7bc P c48712233a8a4eb0b39bb05cd60137e6: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:12.882722 22175 raft_consensus.cc:697] T 1631e507d6714dc187fc9d7f2457aad9 P c48712233a8a4eb0b39bb05cd60137e6 [term 1 LEADER]: Becoming Leader. State: Replica: c48712233a8a4eb0b39bb05cd60137e6, State: Running, Role: LEADER
I20260709 13:21:12.863451 22069 leader_election.cc:304] T d74919ac686e49e694ca111d3b1a27f6 P 21a7bb1d6ef54b20befdf8ea6d293776 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a7bb1d6ef54b20befdf8ea6d293776, 8507b5a0d0e6434aaa7302085c7f54cb; no voters: 
I20260709 13:21:12.886296 22175 consensus_queue.cc:237] T 1631e507d6714dc187fc9d7f2457aad9 P c48712233a8a4eb0b39bb05cd60137e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.888702 22174 leader_election.cc:290] T d74919ac686e49e694ca111d3b1a27f6 P 21a7bb1d6ef54b20befdf8ea6d293776 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591), c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529)
I20260709 13:21:12.891208 22174 raft_consensus.cc:2804] T d74919ac686e49e694ca111d3b1a27f6 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:12.891584 22174 raft_consensus.cc:493] T d74919ac686e49e694ca111d3b1a27f6 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:12.892005 22174 raft_consensus.cc:3060] T d74919ac686e49e694ca111d3b1a27f6 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:12.891683 22028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea585ca33ebf4b329ce97d386bf3e7bc" candidate_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6" is_pre_election: true
I20260709 13:21:12.893538 22153 leader_election.cc:290] T 1631e507d6714dc187fc9d7f2457aad9 P c48712233a8a4eb0b39bb05cd60137e6 [CANDIDATE]: Term 1 election: Requested vote from peers 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591), 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131:33869)
I20260709 13:21:12.895066 21954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea585ca33ebf4b329ce97d386bf3e7bc" candidate_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" is_pre_election: true
I20260709 13:21:12.896296 22028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d74919ac686e49e694ca111d3b1a27f6" candidate_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6" is_pre_election: true
I20260709 13:21:12.897183 22102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9dceef9cf6541039db61f39e2798430" candidate_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" is_pre_election: true
I20260709 13:21:12.898138 22102 raft_consensus.cc:2468] T f9dceef9cf6541039db61f39e2798430 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8507b5a0d0e6434aaa7302085c7f54cb in term 0.
W20260709 13:21:12.899398 22069 leader_election.cc:343] T ea585ca33ebf4b329ce97d386bf3e7bc P 21a7bb1d6ef54b20befdf8ea6d293776 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:21:12.900197 22069 leader_election.cc:343] T d74919ac686e49e694ca111d3b1a27f6 P 21a7bb1d6ef54b20befdf8ea6d293776 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:12.902591 22145 tablet_bootstrap.cc:492] T 7f2c531192bb4639a60df411741abc7b P 8507b5a0d0e6434aaa7302085c7f54cb: No bootstrap required, opened a new log
I20260709 13:21:12.903294 22145 ts_tablet_manager.cc:1403] T 7f2c531192bb4639a60df411741abc7b P 8507b5a0d0e6434aaa7302085c7f54cb: Time spent bootstrapping tablet: real 0.068s	user 0.014s	sys 0.001s
I20260709 13:21:12.906699 21953 raft_consensus.cc:2468] T b8791ec386d148c19bb6fb734287e432 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21a7bb1d6ef54b20befdf8ea6d293776 in term 1.
I20260709 13:21:12.907362 21850 catalog_manager.cc:5697] T 1631e507d6714dc187fc9d7f2457aad9 P c48712233a8a4eb0b39bb05cd60137e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "c48712233a8a4eb0b39bb05cd60137e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:12.911185 22148 raft_consensus.cc:493] T ba306baff7874796aba9a7c3fd5d628e P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:12.911727 22148 raft_consensus.cc:515] T ba306baff7874796aba9a7c3fd5d628e P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.912456 22028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9dceef9cf6541039db61f39e2798430" candidate_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6" is_pre_election: true
I20260709 13:21:12.913132 22028 raft_consensus.cc:2468] T f9dceef9cf6541039db61f39e2798430 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8507b5a0d0e6434aaa7302085c7f54cb in term 0.
I20260709 13:21:12.907297 22145 raft_consensus.cc:359] T 7f2c531192bb4639a60df411741abc7b P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.914885 22145 raft_consensus.cc:385] T 7f2c531192bb4639a60df411741abc7b P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:12.915167 22145 raft_consensus.cc:740] T 7f2c531192bb4639a60df411741abc7b P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8507b5a0d0e6434aaa7302085c7f54cb, State: Initialized, Role: FOLLOWER
I20260709 13:21:12.915418 22146 tablet_bootstrap.cc:492] T ea585ca33ebf4b329ce97d386bf3e7bc P c48712233a8a4eb0b39bb05cd60137e6: No bootstrap required, opened a new log
I20260709 13:21:12.915870 22145 consensus_queue.cc:260] T 7f2c531192bb4639a60df411741abc7b P 8507b5a0d0e6434aaa7302085c7f54cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.917246 21920 leader_election.cc:304] T f9dceef9cf6541039db61f39e2798430 P 8507b5a0d0e6434aaa7302085c7f54cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8507b5a0d0e6434aaa7302085c7f54cb, c48712233a8a4eb0b39bb05cd60137e6; no voters: 
I20260709 13:21:12.915921 22146 ts_tablet_manager.cc:1403] T ea585ca33ebf4b329ce97d386bf3e7bc P c48712233a8a4eb0b39bb05cd60137e6: Time spent bootstrapping tablet: real 0.046s	user 0.012s	sys 0.011s
I20260709 13:21:12.862746 22144 leader_election.cc:290] T ea585ca33ebf4b329ce97d386bf3e7bc P 21a7bb1d6ef54b20befdf8ea6d293776 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591), c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529)
I20260709 13:21:12.910636 22029 raft_consensus.cc:2468] T b8791ec386d148c19bb6fb734287e432 P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21a7bb1d6ef54b20befdf8ea6d293776 in term 1.
I20260709 13:21:12.919420 22028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba306baff7874796aba9a7c3fd5d628e" candidate_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6" is_pre_election: true
I20260709 13:21:12.913867 22148 leader_election.cc:290] T ba306baff7874796aba9a7c3fd5d628e P 8507b5a0d0e6434aaa7302085c7f54cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529), 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131:33869)
I20260709 13:21:12.921936 22146 raft_consensus.cc:359] T ea585ca33ebf4b329ce97d386bf3e7bc P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
W20260709 13:21:12.922508 22069 leader_election.cc:343] T ea585ca33ebf4b329ce97d386bf3e7bc P 21a7bb1d6ef54b20befdf8ea6d293776 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:12.922765 22146 raft_consensus.cc:385] T ea585ca33ebf4b329ce97d386bf3e7bc P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:12.923125 22069 leader_election.cc:304] T ea585ca33ebf4b329ce97d386bf3e7bc P 21a7bb1d6ef54b20befdf8ea6d293776 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 21a7bb1d6ef54b20befdf8ea6d293776; no voters: 8507b5a0d0e6434aaa7302085c7f54cb, c48712233a8a4eb0b39bb05cd60137e6
I20260709 13:21:12.923142 22146 raft_consensus.cc:740] T ea585ca33ebf4b329ce97d386bf3e7bc P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c48712233a8a4eb0b39bb05cd60137e6, State: Initialized, Role: FOLLOWER
I20260709 13:21:12.923349 22148 raft_consensus.cc:2804] T f9dceef9cf6541039db61f39e2798430 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:12.924088 22148 raft_consensus.cc:493] T f9dceef9cf6541039db61f39e2798430 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:12.924695 22148 raft_consensus.cc:3060] T f9dceef9cf6541039db61f39e2798430 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:12.926685 22142 tablet_bootstrap.cc:492] T 5d0c4853c48a42938548ac8810a0bbc4 P 21a7bb1d6ef54b20befdf8ea6d293776: No bootstrap required, opened a new log
I20260709 13:21:12.927462 22142 ts_tablet_manager.cc:1403] T 5d0c4853c48a42938548ac8810a0bbc4 P 21a7bb1d6ef54b20befdf8ea6d293776: Time spent bootstrapping tablet: real 0.085s	user 0.025s	sys 0.010s
I20260709 13:21:12.927682 22174 raft_consensus.cc:515] T d74919ac686e49e694ca111d3b1a27f6 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.929596 22069 leader_election.cc:304] T b8791ec386d148c19bb6fb734287e432 P 21a7bb1d6ef54b20befdf8ea6d293776 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a7bb1d6ef54b20befdf8ea6d293776, 8507b5a0d0e6434aaa7302085c7f54cb; no voters: 
I20260709 13:21:12.924129 22144 raft_consensus.cc:2749] T ea585ca33ebf4b329ce97d386bf3e7bc P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:12.931608 22177 raft_consensus.cc:2804] T b8791ec386d148c19bb6fb734287e432 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:12.931277 22142 raft_consensus.cc:359] T 5d0c4853c48a42938548ac8810a0bbc4 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.932309 22142 raft_consensus.cc:385] T 5d0c4853c48a42938548ac8810a0bbc4 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:12.932631 22142 raft_consensus.cc:740] T 5d0c4853c48a42938548ac8810a0bbc4 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21a7bb1d6ef54b20befdf8ea6d293776, State: Initialized, Role: FOLLOWER
I20260709 13:21:12.933504 22142 consensus_queue.cc:260] T 5d0c4853c48a42938548ac8810a0bbc4 P 21a7bb1d6ef54b20befdf8ea6d293776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.935446 22145 ts_tablet_manager.cc:1434] T 7f2c531192bb4639a60df411741abc7b P 8507b5a0d0e6434aaa7302085c7f54cb: Time spent starting tablet: real 0.032s	user 0.009s	sys 0.000s
I20260709 13:21:12.935536 22142 ts_tablet_manager.cc:1434] T 5d0c4853c48a42938548ac8810a0bbc4 P 21a7bb1d6ef54b20befdf8ea6d293776: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:21:12.924242 22146 consensus_queue.cc:260] T ea585ca33ebf4b329ce97d386bf3e7bc P c48712233a8a4eb0b39bb05cd60137e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.936698 22145 tablet_bootstrap.cc:492] T ea585ca33ebf4b329ce97d386bf3e7bc P 8507b5a0d0e6434aaa7302085c7f54cb: Bootstrap starting.
I20260709 13:21:12.936779 22102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba306baff7874796aba9a7c3fd5d628e" candidate_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" is_pre_election: true
I20260709 13:21:12.937529 22102 raft_consensus.cc:2468] T ba306baff7874796aba9a7c3fd5d628e P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8507b5a0d0e6434aaa7302085c7f54cb in term 0.
I20260709 13:21:12.938192 22146 ts_tablet_manager.cc:1434] T ea585ca33ebf4b329ce97d386bf3e7bc P c48712233a8a4eb0b39bb05cd60137e6: Time spent starting tablet: real 0.020s	user 0.000s	sys 0.007s
I20260709 13:21:12.939499 21917 leader_election.cc:304] T ba306baff7874796aba9a7c3fd5d628e P 8507b5a0d0e6434aaa7302085c7f54cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a7bb1d6ef54b20befdf8ea6d293776, 8507b5a0d0e6434aaa7302085c7f54cb; no voters: 
I20260709 13:21:12.939790 22146 tablet_bootstrap.cc:492] T 0ca42dd42e4d46359256d9256f64d657 P c48712233a8a4eb0b39bb05cd60137e6: Bootstrap starting.
I20260709 13:21:12.940606 22183 raft_consensus.cc:2804] T ba306baff7874796aba9a7c3fd5d628e P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:12.941089 22183 raft_consensus.cc:493] T ba306baff7874796aba9a7c3fd5d628e P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:12.941425 22183 raft_consensus.cc:3060] T ba306baff7874796aba9a7c3fd5d628e P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:12.943960 22174 leader_election.cc:290] T d74919ac686e49e694ca111d3b1a27f6 P 21a7bb1d6ef54b20befdf8ea6d293776 [CANDIDATE]: Term 1 election: Requested vote from peers 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591), c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529)
I20260709 13:21:12.945525 21953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d74919ac686e49e694ca111d3b1a27f6" candidate_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb"
I20260709 13:21:12.946201 21953 raft_consensus.cc:3060] T d74919ac686e49e694ca111d3b1a27f6 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:21:12.946278 21920 leader_election.cc:343] T ba306baff7874796aba9a7c3fd5d628e P 8507b5a0d0e6434aaa7302085c7f54cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:12.947636 22146 tablet_bootstrap.cc:654] T 0ca42dd42e4d46359256d9256f64d657 P c48712233a8a4eb0b39bb05cd60137e6: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:12.948251 22148 raft_consensus.cc:515] T f9dceef9cf6541039db61f39e2798430 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.951313 22029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9dceef9cf6541039db61f39e2798430" candidate_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6"
I20260709 13:21:12.951928 22148 leader_election.cc:290] T f9dceef9cf6541039db61f39e2798430 P 8507b5a0d0e6434aaa7302085c7f54cb [CANDIDATE]: Term 1 election: Requested vote from peers c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529), 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131:33869)
I20260709 13:21:12.951947 22029 raft_consensus.cc:3060] T f9dceef9cf6541039db61f39e2798430 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:12.953092 22028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d74919ac686e49e694ca111d3b1a27f6" candidate_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6"
I20260709 13:21:12.957784 21953 raft_consensus.cc:2468] T d74919ac686e49e694ca111d3b1a27f6 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21a7bb1d6ef54b20befdf8ea6d293776 in term 1.
I20260709 13:21:12.959818 22069 leader_election.cc:304] T d74919ac686e49e694ca111d3b1a27f6 P 21a7bb1d6ef54b20befdf8ea6d293776 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a7bb1d6ef54b20befdf8ea6d293776, 8507b5a0d0e6434aaa7302085c7f54cb; no voters: 
I20260709 13:21:12.960618 22146 tablet_bootstrap.cc:492] T 0ca42dd42e4d46359256d9256f64d657 P c48712233a8a4eb0b39bb05cd60137e6: No bootstrap required, opened a new log
W20260709 13:21:12.960809 22069 leader_election.cc:343] T d74919ac686e49e694ca111d3b1a27f6 P 21a7bb1d6ef54b20befdf8ea6d293776 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:12.961212 22146 ts_tablet_manager.cc:1403] T 0ca42dd42e4d46359256d9256f64d657 P c48712233a8a4eb0b39bb05cd60137e6: Time spent bootstrapping tablet: real 0.022s	user 0.009s	sys 0.009s
I20260709 13:21:12.961285 22029 raft_consensus.cc:2468] T f9dceef9cf6541039db61f39e2798430 P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8507b5a0d0e6434aaa7302085c7f54cb in term 1.
I20260709 13:21:12.963033 21920 leader_election.cc:304] T f9dceef9cf6541039db61f39e2798430 P 8507b5a0d0e6434aaa7302085c7f54cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8507b5a0d0e6434aaa7302085c7f54cb, c48712233a8a4eb0b39bb05cd60137e6; no voters: 
I20260709 13:21:12.964087 22148 raft_consensus.cc:2804] T f9dceef9cf6541039db61f39e2798430 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:12.964249 22177 raft_consensus.cc:697] T b8791ec386d148c19bb6fb734287e432 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 LEADER]: Becoming Leader. State: Replica: 21a7bb1d6ef54b20befdf8ea6d293776, State: Running, Role: LEADER
I20260709 13:21:12.964962 22148 raft_consensus.cc:697] T f9dceef9cf6541039db61f39e2798430 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 LEADER]: Becoming Leader. State: Replica: 8507b5a0d0e6434aaa7302085c7f54cb, State: Running, Role: LEADER
I20260709 13:21:12.964849 22146 raft_consensus.cc:359] T 0ca42dd42e4d46359256d9256f64d657 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } } peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } }
I20260709 13:21:12.965225 22177 consensus_queue.cc:237] T b8791ec386d148c19bb6fb734287e432 P 21a7bb1d6ef54b20befdf8ea6d293776 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.965777 22146 raft_consensus.cc:385] T 0ca42dd42e4d46359256d9256f64d657 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:12.966216 22146 raft_consensus.cc:740] T 0ca42dd42e4d46359256d9256f64d657 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c48712233a8a4eb0b39bb05cd60137e6, State: Initialized, Role: FOLLOWER
I20260709 13:21:12.966248 22148 consensus_queue.cc:237] T f9dceef9cf6541039db61f39e2798430 P 8507b5a0d0e6434aaa7302085c7f54cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.967257 22146 consensus_queue.cc:260] T 0ca42dd42e4d46359256d9256f64d657 P c48712233a8a4eb0b39bb05cd60137e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } } peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } }
I20260709 13:21:12.963948 22186 raft_consensus.cc:2804] T d74919ac686e49e694ca111d3b1a27f6 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:12.969643 22186 raft_consensus.cc:697] T d74919ac686e49e694ca111d3b1a27f6 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 LEADER]: Becoming Leader. State: Replica: 21a7bb1d6ef54b20befdf8ea6d293776, State: Running, Role: LEADER
I20260709 13:21:12.970121 22146 ts_tablet_manager.cc:1434] T 0ca42dd42e4d46359256d9256f64d657 P c48712233a8a4eb0b39bb05cd60137e6: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:21:12.971585 22146 tablet_bootstrap.cc:492] T ba306baff7874796aba9a7c3fd5d628e P c48712233a8a4eb0b39bb05cd60137e6: Bootstrap starting.
I20260709 13:21:12.971809 22102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9dceef9cf6541039db61f39e2798430" candidate_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776"
I20260709 13:21:12.972594 22102 raft_consensus.cc:3060] T f9dceef9cf6541039db61f39e2798430 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:12.974926 22145 tablet_bootstrap.cc:654] T ea585ca33ebf4b329ce97d386bf3e7bc P 8507b5a0d0e6434aaa7302085c7f54cb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:12.979480 22146 tablet_bootstrap.cc:654] T ba306baff7874796aba9a7c3fd5d628e P c48712233a8a4eb0b39bb05cd60137e6: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:12.981983 22102 raft_consensus.cc:2468] T f9dceef9cf6541039db61f39e2798430 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8507b5a0d0e6434aaa7302085c7f54cb in term 1.
I20260709 13:21:12.970643 22186 consensus_queue.cc:237] T d74919ac686e49e694ca111d3b1a27f6 P 21a7bb1d6ef54b20befdf8ea6d293776 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.986696 21852 catalog_manager.cc:5697] T f9dceef9cf6541039db61f39e2798430 P 8507b5a0d0e6434aaa7302085c7f54cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129). New cstate: current_term: 1 leader_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:12.957480 22183 raft_consensus.cc:515] T ba306baff7874796aba9a7c3fd5d628e P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.996335 22177 raft_consensus.cc:493] T 5d0c4853c48a42938548ac8810a0bbc4 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:12.996482 22029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba306baff7874796aba9a7c3fd5d628e" candidate_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6"
I20260709 13:21:12.996946 22102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba306baff7874796aba9a7c3fd5d628e" candidate_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776"
I20260709 13:21:12.996896 22177 raft_consensus.cc:515] T 5d0c4853c48a42938548ac8810a0bbc4 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:12.997608 22102 raft_consensus.cc:3060] T ba306baff7874796aba9a7c3fd5d628e P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:21:12.998329 21920 leader_election.cc:343] T ba306baff7874796aba9a7c3fd5d628e P 8507b5a0d0e6434aaa7302085c7f54cb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:12.999230 22177 leader_election.cc:290] T 5d0c4853c48a42938548ac8810a0bbc4 P 21a7bb1d6ef54b20befdf8ea6d293776 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591), c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529)
I20260709 13:21:12.996567 22183 leader_election.cc:290] T ba306baff7874796aba9a7c3fd5d628e P 8507b5a0d0e6434aaa7302085c7f54cb [CANDIDATE]: Term 1 election: Requested vote from peers c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529), 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131:33869)
I20260709 13:21:13.002533 21953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0c4853c48a42938548ac8810a0bbc4" candidate_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" is_pre_election: true
I20260709 13:21:13.002533 22029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0c4853c48a42938548ac8810a0bbc4" candidate_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6" is_pre_election: true
I20260709 13:21:13.004945 22029 raft_consensus.cc:2468] T 5d0c4853c48a42938548ac8810a0bbc4 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21a7bb1d6ef54b20befdf8ea6d293776 in term 0.
I20260709 13:21:13.005395 22175 raft_consensus.cc:493] T ea585ca33ebf4b329ce97d386bf3e7bc P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:13.001825 21850 catalog_manager.cc:5697] T b8791ec386d148c19bb6fb734287e432 P 21a7bb1d6ef54b20befdf8ea6d293776 reported cstate change: term changed from 0 to 1, leader changed from <none> to 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131). New cstate: current_term: 1 leader_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:13.005925 22175 raft_consensus.cc:515] T ea585ca33ebf4b329ce97d386bf3e7bc P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:13.007054 22177 raft_consensus.cc:493] T 7f2c531192bb4639a60df411741abc7b P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:13.007246 22069 leader_election.cc:304] T 5d0c4853c48a42938548ac8810a0bbc4 P 21a7bb1d6ef54b20befdf8ea6d293776 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a7bb1d6ef54b20befdf8ea6d293776, c48712233a8a4eb0b39bb05cd60137e6; no voters: 
I20260709 13:21:13.007660 22177 raft_consensus.cc:515] T 7f2c531192bb4639a60df411741abc7b P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:13.008113 22174 raft_consensus.cc:2804] T 5d0c4853c48a42938548ac8810a0bbc4 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:13.008229 22175 leader_election.cc:290] T ea585ca33ebf4b329ce97d386bf3e7bc P c48712233a8a4eb0b39bb05cd60137e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591), 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131:33869)
I20260709 13:21:13.008740 22174 raft_consensus.cc:493] T 5d0c4853c48a42938548ac8810a0bbc4 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:13.009157 22174 raft_consensus.cc:3060] T 5d0c4853c48a42938548ac8810a0bbc4 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:13.010339 22177 leader_election.cc:290] T 7f2c531192bb4639a60df411741abc7b P 21a7bb1d6ef54b20befdf8ea6d293776 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591), c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529)
I20260709 13:21:13.010536 22103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea585ca33ebf4b329ce97d386bf3e7bc" candidate_uuid: "c48712233a8a4eb0b39bb05cd60137e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" is_pre_election: true
I20260709 13:21:13.011878 22103 raft_consensus.cc:2468] T ea585ca33ebf4b329ce97d386bf3e7bc P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c48712233a8a4eb0b39bb05cd60137e6 in term 0.
I20260709 13:21:13.013242 21992 leader_election.cc:304] T ea585ca33ebf4b329ce97d386bf3e7bc P c48712233a8a4eb0b39bb05cd60137e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a7bb1d6ef54b20befdf8ea6d293776, c48712233a8a4eb0b39bb05cd60137e6; no voters: 
I20260709 13:21:13.014281 22175 raft_consensus.cc:2804] T ea585ca33ebf4b329ce97d386bf3e7bc P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:13.014669 22175 raft_consensus.cc:493] T ea585ca33ebf4b329ce97d386bf3e7bc P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:13.015028 22175 raft_consensus.cc:3060] T ea585ca33ebf4b329ce97d386bf3e7bc P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:13.017220 22174 raft_consensus.cc:515] T 5d0c4853c48a42938548ac8810a0bbc4 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
W20260709 13:21:13.018957 22069 leader_election.cc:343] T 5d0c4853c48a42938548ac8810a0bbc4 P 21a7bb1d6ef54b20befdf8ea6d293776 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:13.021067 21954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f2c531192bb4639a60df411741abc7b" candidate_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" is_pre_election: true
I20260709 13:21:13.011087 21953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea585ca33ebf4b329ce97d386bf3e7bc" candidate_uuid: "c48712233a8a4eb0b39bb05cd60137e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" is_pre_election: true
I20260709 13:21:13.021767 21954 raft_consensus.cc:2468] T 7f2c531192bb4639a60df411741abc7b P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21a7bb1d6ef54b20befdf8ea6d293776 in term 0.
W20260709 13:21:13.023183 21995 leader_election.cc:343] T ea585ca33ebf4b329ce97d386bf3e7bc P c48712233a8a4eb0b39bb05cd60137e6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:13.024827 22029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f2c531192bb4639a60df411741abc7b" candidate_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6" is_pre_election: true
I20260709 13:21:13.024727 22175 raft_consensus.cc:515] T ea585ca33ebf4b329ce97d386bf3e7bc P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:13.025780 22029 raft_consensus.cc:2468] T 7f2c531192bb4639a60df411741abc7b P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21a7bb1d6ef54b20befdf8ea6d293776 in term 0.
I20260709 13:21:13.027621 22069 leader_election.cc:304] T 7f2c531192bb4639a60df411741abc7b P 21a7bb1d6ef54b20befdf8ea6d293776 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a7bb1d6ef54b20befdf8ea6d293776, c48712233a8a4eb0b39bb05cd60137e6; no voters: 
I20260709 13:21:13.028323 21954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea585ca33ebf4b329ce97d386bf3e7bc" candidate_uuid: "c48712233a8a4eb0b39bb05cd60137e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb"
I20260709 13:21:13.028712 22103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea585ca33ebf4b329ce97d386bf3e7bc" candidate_uuid: "c48712233a8a4eb0b39bb05cd60137e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776"
I20260709 13:21:13.029100 22145 tablet_bootstrap.cc:492] T ea585ca33ebf4b329ce97d386bf3e7bc P 8507b5a0d0e6434aaa7302085c7f54cb: No bootstrap required, opened a new log
I20260709 13:21:13.029350 22103 raft_consensus.cc:3060] T ea585ca33ebf4b329ce97d386bf3e7bc P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:21:13.029785 21995 leader_election.cc:343] T ea585ca33ebf4b329ce97d386bf3e7bc P c48712233a8a4eb0b39bb05cd60137e6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:13.029832 22145 ts_tablet_manager.cc:1403] T ea585ca33ebf4b329ce97d386bf3e7bc P 8507b5a0d0e6434aaa7302085c7f54cb: Time spent bootstrapping tablet: real 0.093s	user 0.011s	sys 0.008s
I20260709 13:21:13.031689 22102 raft_consensus.cc:2468] T ba306baff7874796aba9a7c3fd5d628e P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8507b5a0d0e6434aaa7302085c7f54cb in term 1.
I20260709 13:21:13.031862 22146 tablet_bootstrap.cc:492] T ba306baff7874796aba9a7c3fd5d628e P c48712233a8a4eb0b39bb05cd60137e6: No bootstrap required, opened a new log
I20260709 13:21:13.032439 22146 ts_tablet_manager.cc:1403] T ba306baff7874796aba9a7c3fd5d628e P c48712233a8a4eb0b39bb05cd60137e6: Time spent bootstrapping tablet: real 0.061s	user 0.030s	sys 0.015s
I20260709 13:21:13.032397 22177 raft_consensus.cc:2804] T 7f2c531192bb4639a60df411741abc7b P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:13.032992 22177 raft_consensus.cc:493] T 7f2c531192bb4639a60df411741abc7b P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:13.033393 22177 raft_consensus.cc:3060] T 7f2c531192bb4639a60df411741abc7b P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:13.033394 22145 raft_consensus.cc:359] T ea585ca33ebf4b329ce97d386bf3e7bc P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:13.034322 22145 raft_consensus.cc:385] T ea585ca33ebf4b329ce97d386bf3e7bc P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:13.034603 22145 raft_consensus.cc:740] T ea585ca33ebf4b329ce97d386bf3e7bc P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8507b5a0d0e6434aaa7302085c7f54cb, State: Initialized, Role: FOLLOWER
I20260709 13:21:13.036521 21954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0c4853c48a42938548ac8810a0bbc4" candidate_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb"
I20260709 13:21:13.034710 22174 leader_election.cc:290] T 5d0c4853c48a42938548ac8810a0bbc4 P 21a7bb1d6ef54b20befdf8ea6d293776 [CANDIDATE]: Term 1 election: Requested vote from peers 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591), c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529)
I20260709 13:21:13.035522 22145 consensus_queue.cc:260] T ea585ca33ebf4b329ce97d386bf3e7bc P 8507b5a0d0e6434aaa7302085c7f54cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:13.038870 21917 leader_election.cc:304] T ba306baff7874796aba9a7c3fd5d628e P 8507b5a0d0e6434aaa7302085c7f54cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 21a7bb1d6ef54b20befdf8ea6d293776, 8507b5a0d0e6434aaa7302085c7f54cb; no voters: c48712233a8a4eb0b39bb05cd60137e6
W20260709 13:21:13.039002 22069 leader_election.cc:343] T 5d0c4853c48a42938548ac8810a0bbc4 P 21a7bb1d6ef54b20befdf8ea6d293776 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:13.040127 22145 ts_tablet_manager.cc:1434] T ea585ca33ebf4b329ce97d386bf3e7bc P 8507b5a0d0e6434aaa7302085c7f54cb: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:21:13.039101 22103 raft_consensus.cc:2468] T ea585ca33ebf4b329ce97d386bf3e7bc P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c48712233a8a4eb0b39bb05cd60137e6 in term 1.
I20260709 13:21:13.039834 22029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0c4853c48a42938548ac8810a0bbc4" candidate_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6"
I20260709 13:21:13.041457 22029 raft_consensus.cc:3060] T 5d0c4853c48a42938548ac8810a0bbc4 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:13.041558 21992 leader_election.cc:304] T ea585ca33ebf4b329ce97d386bf3e7bc P c48712233a8a4eb0b39bb05cd60137e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 21a7bb1d6ef54b20befdf8ea6d293776, c48712233a8a4eb0b39bb05cd60137e6; no voters: 8507b5a0d0e6434aaa7302085c7f54cb
I20260709 13:21:13.042636 22153 raft_consensus.cc:2804] T ea585ca33ebf4b329ce97d386bf3e7bc P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:13.043135 22153 raft_consensus.cc:697] T ea585ca33ebf4b329ce97d386bf3e7bc P c48712233a8a4eb0b39bb05cd60137e6 [term 1 LEADER]: Becoming Leader. State: Replica: c48712233a8a4eb0b39bb05cd60137e6, State: Running, Role: LEADER
I20260709 13:21:13.043393 22148 raft_consensus.cc:2804] T ba306baff7874796aba9a7c3fd5d628e P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:13.044116 22148 raft_consensus.cc:697] T ba306baff7874796aba9a7c3fd5d628e P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 LEADER]: Becoming Leader. State: Replica: 8507b5a0d0e6434aaa7302085c7f54cb, State: Running, Role: LEADER
I20260709 13:21:13.044028 22153 consensus_queue.cc:237] T ea585ca33ebf4b329ce97d386bf3e7bc P c48712233a8a4eb0b39bb05cd60137e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:13.045038 22148 consensus_queue.cc:237] T ba306baff7874796aba9a7c3fd5d628e P 8507b5a0d0e6434aaa7302085c7f54cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:13.032994 22175 leader_election.cc:290] T ea585ca33ebf4b329ce97d386bf3e7bc P c48712233a8a4eb0b39bb05cd60137e6 [CANDIDATE]: Term 1 election: Requested vote from peers 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591), 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131:33869)
I20260709 13:21:13.049991 22029 raft_consensus.cc:2468] T 5d0c4853c48a42938548ac8810a0bbc4 P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21a7bb1d6ef54b20befdf8ea6d293776 in term 1.
I20260709 13:21:13.051465 22145 tablet_bootstrap.cc:492] T 5d0c4853c48a42938548ac8810a0bbc4 P 8507b5a0d0e6434aaa7302085c7f54cb: Bootstrap starting.
I20260709 13:21:13.053176 22069 leader_election.cc:304] T 5d0c4853c48a42938548ac8810a0bbc4 P 21a7bb1d6ef54b20befdf8ea6d293776 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 21a7bb1d6ef54b20befdf8ea6d293776, c48712233a8a4eb0b39bb05cd60137e6; no voters: 8507b5a0d0e6434aaa7302085c7f54cb
I20260709 13:21:13.052485 21850 catalog_manager.cc:5697] T d74919ac686e49e694ca111d3b1a27f6 P 21a7bb1d6ef54b20befdf8ea6d293776 reported cstate change: term changed from 0 to 1, leader changed from <none> to 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131). New cstate: current_term: 1 leader_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:13.053692 22146 raft_consensus.cc:359] T ba306baff7874796aba9a7c3fd5d628e P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:13.054437 22146 raft_consensus.cc:385] T ba306baff7874796aba9a7c3fd5d628e P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:13.054454 22186 raft_consensus.cc:2804] T 5d0c4853c48a42938548ac8810a0bbc4 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:13.058938 22146 raft_consensus.cc:740] T ba306baff7874796aba9a7c3fd5d628e P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c48712233a8a4eb0b39bb05cd60137e6, State: Initialized, Role: FOLLOWER
I20260709 13:21:13.059355 22186 raft_consensus.cc:697] T 5d0c4853c48a42938548ac8810a0bbc4 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 LEADER]: Becoming Leader. State: Replica: 21a7bb1d6ef54b20befdf8ea6d293776, State: Running, Role: LEADER
I20260709 13:21:13.060115 22145 tablet_bootstrap.cc:654] T 5d0c4853c48a42938548ac8810a0bbc4 P 8507b5a0d0e6434aaa7302085c7f54cb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:13.060519 22186 consensus_queue.cc:237] T 5d0c4853c48a42938548ac8810a0bbc4 P 21a7bb1d6ef54b20befdf8ea6d293776 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:13.060727 22146 consensus_queue.cc:260] T ba306baff7874796aba9a7c3fd5d628e P c48712233a8a4eb0b39bb05cd60137e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:13.064441 22146 ts_tablet_manager.cc:1434] T ba306baff7874796aba9a7c3fd5d628e P c48712233a8a4eb0b39bb05cd60137e6: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:21:13.068357 22146 tablet_bootstrap.cc:492] T d74919ac686e49e694ca111d3b1a27f6 P c48712233a8a4eb0b39bb05cd60137e6: Bootstrap starting.
I20260709 13:21:13.056308 22177 raft_consensus.cc:515] T 7f2c531192bb4639a60df411741abc7b P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:13.071332 21954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f2c531192bb4639a60df411741abc7b" candidate_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb"
I20260709 13:21:13.072151 21954 raft_consensus.cc:3060] T 7f2c531192bb4639a60df411741abc7b P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:13.072381 22029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f2c531192bb4639a60df411741abc7b" candidate_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6"
I20260709 13:21:13.073108 22029 raft_consensus.cc:3060] T 7f2c531192bb4639a60df411741abc7b P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:13.074539 22146 tablet_bootstrap.cc:654] T d74919ac686e49e694ca111d3b1a27f6 P c48712233a8a4eb0b39bb05cd60137e6: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:13.079980 21849 catalog_manager.cc:5697] T ea585ca33ebf4b329ce97d386bf3e7bc P c48712233a8a4eb0b39bb05cd60137e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "c48712233a8a4eb0b39bb05cd60137e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:13.082939 22029 raft_consensus.cc:2468] T 7f2c531192bb4639a60df411741abc7b P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21a7bb1d6ef54b20befdf8ea6d293776 in term 1.
I20260709 13:21:13.084771 22069 leader_election.cc:304] T 7f2c531192bb4639a60df411741abc7b P 21a7bb1d6ef54b20befdf8ea6d293776 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a7bb1d6ef54b20befdf8ea6d293776, c48712233a8a4eb0b39bb05cd60137e6; no voters: 
I20260709 13:21:13.085942 22174 raft_consensus.cc:2804] T 7f2c531192bb4639a60df411741abc7b P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:13.086577 22174 raft_consensus.cc:697] T 7f2c531192bb4639a60df411741abc7b P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 LEADER]: Becoming Leader. State: Replica: 21a7bb1d6ef54b20befdf8ea6d293776, State: Running, Role: LEADER
I20260709 13:21:13.087750 22174 consensus_queue.cc:237] T 7f2c531192bb4639a60df411741abc7b P 21a7bb1d6ef54b20befdf8ea6d293776 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:13.095409 22177 leader_election.cc:290] T 7f2c531192bb4639a60df411741abc7b P 21a7bb1d6ef54b20befdf8ea6d293776 [CANDIDATE]: Term 1 election: Requested vote from peers 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591), c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529)
I20260709 13:21:13.103931 21852 catalog_manager.cc:5697] T ba306baff7874796aba9a7c3fd5d628e P 8507b5a0d0e6434aaa7302085c7f54cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129). New cstate: current_term: 1 leader_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:13.160409 22153 raft_consensus.cc:493] T ba306baff7874796aba9a7c3fd5d628e P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:13.160648 21954 raft_consensus.cc:2468] T 7f2c531192bb4639a60df411741abc7b P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21a7bb1d6ef54b20befdf8ea6d293776 in term 1.
I20260709 13:21:13.161190 22153 raft_consensus.cc:515] T ba306baff7874796aba9a7c3fd5d628e P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:13.165061 21953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba306baff7874796aba9a7c3fd5d628e" candidate_uuid: "c48712233a8a4eb0b39bb05cd60137e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" is_pre_election: true
I20260709 13:21:13.167467 22153 leader_election.cc:290] T ba306baff7874796aba9a7c3fd5d628e P c48712233a8a4eb0b39bb05cd60137e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591), 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131:33869)
I20260709 13:21:13.168825 22103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba306baff7874796aba9a7c3fd5d628e" candidate_uuid: "c48712233a8a4eb0b39bb05cd60137e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" is_pre_election: true
I20260709 13:21:13.169557 22103 raft_consensus.cc:2393] T ba306baff7874796aba9a7c3fd5d628e P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c48712233a8a4eb0b39bb05cd60137e6 in current term 1: Already voted for candidate 8507b5a0d0e6434aaa7302085c7f54cb in this term.
I20260709 13:21:13.170962 21992 leader_election.cc:304] T ba306baff7874796aba9a7c3fd5d628e P c48712233a8a4eb0b39bb05cd60137e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c48712233a8a4eb0b39bb05cd60137e6; no voters: 21a7bb1d6ef54b20befdf8ea6d293776, 8507b5a0d0e6434aaa7302085c7f54cb
I20260709 13:21:13.172043 22153 raft_consensus.cc:3060] T ba306baff7874796aba9a7c3fd5d628e P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:13.171721 21849 catalog_manager.cc:5697] T 5d0c4853c48a42938548ac8810a0bbc4 P 21a7bb1d6ef54b20befdf8ea6d293776 reported cstate change: term changed from 0 to 1, leader changed from <none> to 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131). New cstate: current_term: 1 leader_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:13.174517 21849 catalog_manager.cc:5697] T 7f2c531192bb4639a60df411741abc7b P 21a7bb1d6ef54b20befdf8ea6d293776 reported cstate change: term changed from 0 to 1, leader changed from <none> to 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131). New cstate: current_term: 1 leader_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:13.180907 22153 raft_consensus.cc:2749] T ba306baff7874796aba9a7c3fd5d628e P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:13.182368 22145 tablet_bootstrap.cc:492] T 5d0c4853c48a42938548ac8810a0bbc4 P 8507b5a0d0e6434aaa7302085c7f54cb: No bootstrap required, opened a new log
I20260709 13:21:13.182435 22146 tablet_bootstrap.cc:492] T d74919ac686e49e694ca111d3b1a27f6 P c48712233a8a4eb0b39bb05cd60137e6: No bootstrap required, opened a new log
I20260709 13:21:13.183043 22145 ts_tablet_manager.cc:1403] T 5d0c4853c48a42938548ac8810a0bbc4 P 8507b5a0d0e6434aaa7302085c7f54cb: Time spent bootstrapping tablet: real 0.134s	user 0.054s	sys 0.072s
I20260709 13:21:13.183126 22146 ts_tablet_manager.cc:1403] T d74919ac686e49e694ca111d3b1a27f6 P c48712233a8a4eb0b39bb05cd60137e6: Time spent bootstrapping tablet: real 0.115s	user 0.042s	sys 0.054s
I20260709 13:21:13.185704 22145 raft_consensus.cc:359] T 5d0c4853c48a42938548ac8810a0bbc4 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:13.186298 22145 raft_consensus.cc:385] T 5d0c4853c48a42938548ac8810a0bbc4 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:13.186007 22146 raft_consensus.cc:359] T d74919ac686e49e694ca111d3b1a27f6 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:13.186580 22145 raft_consensus.cc:740] T 5d0c4853c48a42938548ac8810a0bbc4 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8507b5a0d0e6434aaa7302085c7f54cb, State: Initialized, Role: FOLLOWER
I20260709 13:21:13.186816 22146 raft_consensus.cc:385] T d74919ac686e49e694ca111d3b1a27f6 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:13.187157 22146 raft_consensus.cc:740] T d74919ac686e49e694ca111d3b1a27f6 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c48712233a8a4eb0b39bb05cd60137e6, State: Initialized, Role: FOLLOWER
I20260709 13:21:13.187448 22145 consensus_queue.cc:260] T 5d0c4853c48a42938548ac8810a0bbc4 P 8507b5a0d0e6434aaa7302085c7f54cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:13.187893 22146 consensus_queue.cc:260] T d74919ac686e49e694ca111d3b1a27f6 P c48712233a8a4eb0b39bb05cd60137e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:13.189812 22145 ts_tablet_manager.cc:1434] T 5d0c4853c48a42938548ac8810a0bbc4 P 8507b5a0d0e6434aaa7302085c7f54cb: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:21:13.190181 22146 ts_tablet_manager.cc:1434] T d74919ac686e49e694ca111d3b1a27f6 P c48712233a8a4eb0b39bb05cd60137e6: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:21:13.200058 22148 raft_consensus.cc:493] T 0ca42dd42e4d46359256d9256f64d657 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:13.200721 22148 raft_consensus.cc:515] T 0ca42dd42e4d46359256d9256f64d657 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } } peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } }
I20260709 13:21:13.203894 22148 leader_election.cc:290] T 0ca42dd42e4d46359256d9256f64d657 P 8507b5a0d0e6434aaa7302085c7f54cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131:33869), c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529)
I20260709 13:21:13.204353 22029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ca42dd42e4d46359256d9256f64d657" candidate_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6" is_pre_election: true
I20260709 13:21:13.204393 22103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ca42dd42e4d46359256d9256f64d657" candidate_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" is_pre_election: true
I20260709 13:21:13.205166 22029 raft_consensus.cc:2468] T 0ca42dd42e4d46359256d9256f64d657 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8507b5a0d0e6434aaa7302085c7f54cb in term 0.
I20260709 13:21:13.205237 22103 raft_consensus.cc:2468] T 0ca42dd42e4d46359256d9256f64d657 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8507b5a0d0e6434aaa7302085c7f54cb in term 0.
I20260709 13:21:13.206247 21920 leader_election.cc:304] T 0ca42dd42e4d46359256d9256f64d657 P 8507b5a0d0e6434aaa7302085c7f54cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8507b5a0d0e6434aaa7302085c7f54cb, c48712233a8a4eb0b39bb05cd60137e6; no voters: 
I20260709 13:21:13.206995 22148 raft_consensus.cc:2804] T 0ca42dd42e4d46359256d9256f64d657 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:13.207394 22148 raft_consensus.cc:493] T 0ca42dd42e4d46359256d9256f64d657 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:13.207752 22148 raft_consensus.cc:3060] T 0ca42dd42e4d46359256d9256f64d657 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:13.212945 22148 raft_consensus.cc:515] T 0ca42dd42e4d46359256d9256f64d657 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } } peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } }
I20260709 13:21:13.214737 22148 leader_election.cc:290] T 0ca42dd42e4d46359256d9256f64d657 P 8507b5a0d0e6434aaa7302085c7f54cb [CANDIDATE]: Term 1 election: Requested vote from peers 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131:33869), c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529)
I20260709 13:21:13.215576 22103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ca42dd42e4d46359256d9256f64d657" candidate_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776"
I20260709 13:21:13.215775 22029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ca42dd42e4d46359256d9256f64d657" candidate_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6"
I20260709 13:21:13.216152 22103 raft_consensus.cc:3060] T 0ca42dd42e4d46359256d9256f64d657 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:13.216372 22029 raft_consensus.cc:3060] T 0ca42dd42e4d46359256d9256f64d657 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:13.221426 22029 raft_consensus.cc:2468] T 0ca42dd42e4d46359256d9256f64d657 P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8507b5a0d0e6434aaa7302085c7f54cb in term 1.
I20260709 13:21:13.221426 22103 raft_consensus.cc:2468] T 0ca42dd42e4d46359256d9256f64d657 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8507b5a0d0e6434aaa7302085c7f54cb in term 1.
I20260709 13:21:13.222631 21920 leader_election.cc:304] T 0ca42dd42e4d46359256d9256f64d657 P 8507b5a0d0e6434aaa7302085c7f54cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8507b5a0d0e6434aaa7302085c7f54cb, c48712233a8a4eb0b39bb05cd60137e6; no voters: 
I20260709 13:21:13.223457 22148 raft_consensus.cc:2804] T 0ca42dd42e4d46359256d9256f64d657 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:13.223940 22148 raft_consensus.cc:697] T 0ca42dd42e4d46359256d9256f64d657 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 LEADER]: Becoming Leader. State: Replica: 8507b5a0d0e6434aaa7302085c7f54cb, State: Running, Role: LEADER
I20260709 13:21:13.224782 22148 consensus_queue.cc:237] T 0ca42dd42e4d46359256d9256f64d657 P 8507b5a0d0e6434aaa7302085c7f54cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } } peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } }
I20260709 13:21:13.231920 21849 catalog_manager.cc:5697] T 0ca42dd42e4d46359256d9256f64d657 P 8507b5a0d0e6434aaa7302085c7f54cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129). New cstate: current_term: 1 leader_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:13.250687 22148 raft_consensus.cc:493] T ea585ca33ebf4b329ce97d386bf3e7bc P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:13.251343 22148 raft_consensus.cc:515] T ea585ca33ebf4b329ce97d386bf3e7bc P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:13.253134 22148 leader_election.cc:290] T ea585ca33ebf4b329ce97d386bf3e7bc P 8507b5a0d0e6434aaa7302085c7f54cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529), 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131:33869)
I20260709 13:21:13.253968 22029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea585ca33ebf4b329ce97d386bf3e7bc" candidate_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6" is_pre_election: true
I20260709 13:21:13.254115 22103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea585ca33ebf4b329ce97d386bf3e7bc" candidate_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" is_pre_election: true
I20260709 13:21:13.254925 22103 raft_consensus.cc:2393] T ea585ca33ebf4b329ce97d386bf3e7bc P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8507b5a0d0e6434aaa7302085c7f54cb in current term 1: Already voted for candidate c48712233a8a4eb0b39bb05cd60137e6 in this term.
I20260709 13:21:13.255919 21917 leader_election.cc:304] T ea585ca33ebf4b329ce97d386bf3e7bc P 8507b5a0d0e6434aaa7302085c7f54cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8507b5a0d0e6434aaa7302085c7f54cb; no voters: 21a7bb1d6ef54b20befdf8ea6d293776, c48712233a8a4eb0b39bb05cd60137e6
I20260709 13:21:13.256537 22148 raft_consensus.cc:3060] T ea585ca33ebf4b329ce97d386bf3e7bc P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:13.261569 22148 raft_consensus.cc:2749] T ea585ca33ebf4b329ce97d386bf3e7bc P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:13.278453 22153 consensus_queue.cc:1048] T 1631e507d6714dc187fc9d7f2457aad9 P c48712233a8a4eb0b39bb05cd60137e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:13.290350 22175 consensus_queue.cc:1048] T 1631e507d6714dc187fc9d7f2457aad9 P c48712233a8a4eb0b39bb05cd60137e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:13.381647 22029 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1631e507d6714dc187fc9d7f2457aad9"
dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6"
mode: GRACEFUL
new_leader_uuid: "8507b5a0d0e6434aaa7302085c7f54cb"
 from {username='slave'} at 127.0.0.1:46600
I20260709 13:21:13.382161 22029 raft_consensus.cc:606] T 1631e507d6714dc187fc9d7f2457aad9 P c48712233a8a4eb0b39bb05cd60137e6 [term 1 LEADER]: Received request to transfer leadership to TS 8507b5a0d0e6434aaa7302085c7f54cb
I20260709 13:21:13.392359 22148 consensus_queue.cc:1048] T f9dceef9cf6541039db61f39e2798430 P 8507b5a0d0e6434aaa7302085c7f54cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:13.396736 22102 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b8791ec386d148c19bb6fb734287e432"
dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776"
mode: GRACEFUL
new_leader_uuid: "8507b5a0d0e6434aaa7302085c7f54cb"
 from {username='slave'} at 127.0.0.1:53794
I20260709 13:21:13.397467 22102 raft_consensus.cc:606] T b8791ec386d148c19bb6fb734287e432 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 LEADER]: Received request to transfer leadership to TS 8507b5a0d0e6434aaa7302085c7f54cb
I20260709 13:21:13.405480 22148 consensus_queue.cc:1048] T f9dceef9cf6541039db61f39e2798430 P 8507b5a0d0e6434aaa7302085c7f54cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:13.406428 22029 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ea585ca33ebf4b329ce97d386bf3e7bc"
dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6"
mode: GRACEFUL
new_leader_uuid: "8507b5a0d0e6434aaa7302085c7f54cb"
 from {username='slave'} at 127.0.0.1:46600
I20260709 13:21:13.407146 22029 raft_consensus.cc:606] T ea585ca33ebf4b329ce97d386bf3e7bc P c48712233a8a4eb0b39bb05cd60137e6 [term 1 LEADER]: Received request to transfer leadership to TS 8507b5a0d0e6434aaa7302085c7f54cb
I20260709 13:21:13.408267 22186 consensus_queue.cc:1048] T d74919ac686e49e694ca111d3b1a27f6 P 21a7bb1d6ef54b20befdf8ea6d293776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:13.419130 22103 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d74919ac686e49e694ca111d3b1a27f6"
dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776"
mode: GRACEFUL
new_leader_uuid: "c48712233a8a4eb0b39bb05cd60137e6"
 from {username='slave'} at 127.0.0.1:53794
I20260709 13:21:13.419853 22103 raft_consensus.cc:606] T d74919ac686e49e694ca111d3b1a27f6 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 LEADER]: Received request to transfer leadership to TS c48712233a8a4eb0b39bb05cd60137e6
I20260709 13:21:13.425191 22177 consensus_queue.cc:1048] T b8791ec386d148c19bb6fb734287e432 P 21a7bb1d6ef54b20befdf8ea6d293776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:13.429493 22028 raft_consensus.cc:3060] T d74919ac686e49e694ca111d3b1a27f6 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:13.439620 22177 raft_consensus.cc:993] T b8791ec386d148c19bb6fb734287e432 P 21a7bb1d6ef54b20befdf8ea6d293776: : Instructing follower 8507b5a0d0e6434aaa7302085c7f54cb to start an election
I20260709 13:21:13.441671 22186 raft_consensus.cc:1081] T b8791ec386d148c19bb6fb734287e432 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 LEADER]: Signalling peer 8507b5a0d0e6434aaa7302085c7f54cb to start an election
I20260709 13:21:13.442140 21953 raft_consensus.cc:3060] T 5d0c4853c48a42938548ac8810a0bbc4 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:13.447758 22174 consensus_queue.cc:1048] T d74919ac686e49e694ca111d3b1a27f6 P 21a7bb1d6ef54b20befdf8ea6d293776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:13.449512 21952 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ba306baff7874796aba9a7c3fd5d628e"
dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb"
mode: GRACEFUL
new_leader_uuid: "c48712233a8a4eb0b39bb05cd60137e6"
 from {username='slave'} at 127.0.0.1:51346
I20260709 13:21:13.450201 21952 raft_consensus.cc:606] T ba306baff7874796aba9a7c3fd5d628e P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 LEADER]: Received request to transfer leadership to TS c48712233a8a4eb0b39bb05cd60137e6
I20260709 13:21:13.450405 21954 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b8791ec386d148c19bb6fb734287e432"
dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb"
 from {username='slave'} at 127.0.0.1:51332
I20260709 13:21:13.451097 21954 raft_consensus.cc:493] T b8791ec386d148c19bb6fb734287e432 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:13.451479 21954 raft_consensus.cc:3060] T b8791ec386d148c19bb6fb734287e432 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:13.455781 22186 consensus_queue.cc:1048] T b8791ec386d148c19bb6fb734287e432 P 21a7bb1d6ef54b20befdf8ea6d293776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:13.455821 22174 consensus_queue.cc:1048] T 5d0c4853c48a42938548ac8810a0bbc4 P 21a7bb1d6ef54b20befdf8ea6d293776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:13.459105 22103 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7f2c531192bb4639a60df411741abc7b"
dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776"
mode: GRACEFUL
new_leader_uuid: "c48712233a8a4eb0b39bb05cd60137e6"
 from {username='slave'} at 127.0.0.1:53794
I20260709 13:21:13.459770 22103 raft_consensus.cc:606] T 7f2c531192bb4639a60df411741abc7b P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 LEADER]: Received request to transfer leadership to TS c48712233a8a4eb0b39bb05cd60137e6
I20260709 13:21:13.459115 21954 raft_consensus.cc:515] T b8791ec386d148c19bb6fb734287e432 P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:13.464311 21954 leader_election.cc:290] T b8791ec386d148c19bb6fb734287e432 P 8507b5a0d0e6434aaa7302085c7f54cb [CANDIDATE]: Term 2 election: Requested vote from peers c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529), 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131:33869)
I20260709 13:21:13.466394 22027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8791ec386d148c19bb6fb734287e432" candidate_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6"
I20260709 13:21:13.468240 22103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8791ec386d148c19bb6fb734287e432" candidate_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776"
I20260709 13:21:13.468919 22103 raft_consensus.cc:3055] T b8791ec386d148c19bb6fb734287e432 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:13.469254 22103 raft_consensus.cc:740] T b8791ec386d148c19bb6fb734287e432 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 21a7bb1d6ef54b20befdf8ea6d293776, State: Running, Role: LEADER
I20260709 13:21:13.470132 22103 consensus_queue.cc:260] T b8791ec386d148c19bb6fb734287e432 P 21a7bb1d6ef54b20befdf8ea6d293776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:13.471714 22103 raft_consensus.cc:3060] T b8791ec386d148c19bb6fb734287e432 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:13.481973 21954 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f9dceef9cf6541039db61f39e2798430"
dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb"
mode: GRACEFUL
new_leader_uuid: "c48712233a8a4eb0b39bb05cd60137e6"
 from {username='slave'} at 127.0.0.1:51346
I20260709 13:21:13.482977 21954 raft_consensus.cc:606] T f9dceef9cf6541039db61f39e2798430 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 LEADER]: Received request to transfer leadership to TS c48712233a8a4eb0b39bb05cd60137e6
I20260709 13:21:13.486712 22148 consensus_queue.cc:1048] T ba306baff7874796aba9a7c3fd5d628e P 8507b5a0d0e6434aaa7302085c7f54cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:13.495869 22186 raft_consensus.cc:993] T d74919ac686e49e694ca111d3b1a27f6 P 21a7bb1d6ef54b20befdf8ea6d293776: : Instructing follower c48712233a8a4eb0b39bb05cd60137e6 to start an election
I20260709 13:21:13.496390 22174 raft_consensus.cc:1081] T d74919ac686e49e694ca111d3b1a27f6 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 LEADER]: Signalling peer c48712233a8a4eb0b39bb05cd60137e6 to start an election
I20260709 13:21:13.498868 22028 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d74919ac686e49e694ca111d3b1a27f6"
dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6"
 from {username='slave'} at 127.0.0.1:46568
I20260709 13:21:13.499404 22028 raft_consensus.cc:493] T d74919ac686e49e694ca111d3b1a27f6 P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:13.499828 22028 raft_consensus.cc:3060] T d74919ac686e49e694ca111d3b1a27f6 P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:13.503648 22174 consensus_queue.cc:1048] T 5d0c4853c48a42938548ac8810a0bbc4 P 21a7bb1d6ef54b20befdf8ea6d293776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:13.503942 22103 raft_consensus.cc:2468] T b8791ec386d148c19bb6fb734287e432 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8507b5a0d0e6434aaa7302085c7f54cb in term 2.
I20260709 13:21:13.505262 21917 leader_election.cc:304] T b8791ec386d148c19bb6fb734287e432 P 8507b5a0d0e6434aaa7302085c7f54cb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 21a7bb1d6ef54b20befdf8ea6d293776, 8507b5a0d0e6434aaa7302085c7f54cb; no voters: c48712233a8a4eb0b39bb05cd60137e6
I20260709 13:21:13.506563 22148 raft_consensus.cc:2804] T b8791ec386d148c19bb6fb734287e432 P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:13.507095 22148 raft_consensus.cc:697] T b8791ec386d148c19bb6fb734287e432 P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 LEADER]: Becoming Leader. State: Replica: 8507b5a0d0e6434aaa7302085c7f54cb, State: Running, Role: LEADER
I20260709 13:21:13.507933 22148 consensus_queue.cc:237] T b8791ec386d148c19bb6fb734287e432 P 8507b5a0d0e6434aaa7302085c7f54cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:13.508037 22028 raft_consensus.cc:515] T d74919ac686e49e694ca111d3b1a27f6 P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:13.510315 22028 leader_election.cc:290] T d74919ac686e49e694ca111d3b1a27f6 P c48712233a8a4eb0b39bb05cd60137e6 [CANDIDATE]: Term 2 election: Requested vote from peers 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591), 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131:33869)
I20260709 13:21:13.512401 22103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d74919ac686e49e694ca111d3b1a27f6" candidate_uuid: "c48712233a8a4eb0b39bb05cd60137e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776"
I20260709 13:21:13.513337 22103 raft_consensus.cc:3055] T d74919ac686e49e694ca111d3b1a27f6 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:13.513782 22103 raft_consensus.cc:740] T d74919ac686e49e694ca111d3b1a27f6 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 21a7bb1d6ef54b20befdf8ea6d293776, State: Running, Role: LEADER
I20260709 13:21:13.514946 22103 consensus_queue.cc:260] T d74919ac686e49e694ca111d3b1a27f6 P 21a7bb1d6ef54b20befdf8ea6d293776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:13.515827 21953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d74919ac686e49e694ca111d3b1a27f6" candidate_uuid: "c48712233a8a4eb0b39bb05cd60137e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb"
I20260709 13:21:13.516404 22103 raft_consensus.cc:3060] T d74919ac686e49e694ca111d3b1a27f6 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:13.516631 21953 raft_consensus.cc:3060] T d74919ac686e49e694ca111d3b1a27f6 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:13.521466 21852 catalog_manager.cc:5697] T b8791ec386d148c19bb6fb734287e432 P 8507b5a0d0e6434aaa7302085c7f54cb reported cstate change: term changed from 1 to 2, leader changed from 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131) to 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129). New cstate: current_term: 2 leader_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:13.526357 22103 raft_consensus.cc:2468] T d74919ac686e49e694ca111d3b1a27f6 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c48712233a8a4eb0b39bb05cd60137e6 in term 2.
I20260709 13:21:13.527889 21992 leader_election.cc:304] T d74919ac686e49e694ca111d3b1a27f6 P c48712233a8a4eb0b39bb05cd60137e6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a7bb1d6ef54b20befdf8ea6d293776, c48712233a8a4eb0b39bb05cd60137e6; no voters: 
I20260709 13:21:13.528852 22175 raft_consensus.cc:2804] T d74919ac686e49e694ca111d3b1a27f6 P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:13.529342 22175 raft_consensus.cc:697] T d74919ac686e49e694ca111d3b1a27f6 P c48712233a8a4eb0b39bb05cd60137e6 [term 2 LEADER]: Becoming Leader. State: Replica: c48712233a8a4eb0b39bb05cd60137e6, State: Running, Role: LEADER
I20260709 13:21:13.529968 21953 raft_consensus.cc:2468] T d74919ac686e49e694ca111d3b1a27f6 P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c48712233a8a4eb0b39bb05cd60137e6 in term 2.
I20260709 13:21:13.530216 22175 consensus_queue.cc:237] T d74919ac686e49e694ca111d3b1a27f6 P c48712233a8a4eb0b39bb05cd60137e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:13.537029 22184 consensus_queue.cc:1048] T ba306baff7874796aba9a7c3fd5d628e P 8507b5a0d0e6434aaa7302085c7f54cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:13.547163 21852 catalog_manager.cc:5697] T d74919ac686e49e694ca111d3b1a27f6 P c48712233a8a4eb0b39bb05cd60137e6 reported cstate change: term changed from 1 to 2, leader changed from 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131) to c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130). New cstate: current_term: 2 leader_uuid: "c48712233a8a4eb0b39bb05cd60137e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:13.552469 22184 raft_consensus.cc:993] T ba306baff7874796aba9a7c3fd5d628e P 8507b5a0d0e6434aaa7302085c7f54cb: : Instructing follower c48712233a8a4eb0b39bb05cd60137e6 to start an election
I20260709 13:21:13.553040 22184 raft_consensus.cc:1081] T ba306baff7874796aba9a7c3fd5d628e P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 LEADER]: Signalling peer c48712233a8a4eb0b39bb05cd60137e6 to start an election
I20260709 13:21:13.554821 22028 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ba306baff7874796aba9a7c3fd5d628e"
dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6"
 from {username='slave'} at 127.0.0.1:46584
I20260709 13:21:13.555434 22028 raft_consensus.cc:493] T ba306baff7874796aba9a7c3fd5d628e P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:13.555831 22028 raft_consensus.cc:3060] T ba306baff7874796aba9a7c3fd5d628e P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:13.555814 22175 consensus_queue.cc:1048] T ea585ca33ebf4b329ce97d386bf3e7bc P c48712233a8a4eb0b39bb05cd60137e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:13.563532 22028 raft_consensus.cc:515] T ba306baff7874796aba9a7c3fd5d628e P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:13.568168 22153 consensus_queue.cc:1048] T ea585ca33ebf4b329ce97d386bf3e7bc P c48712233a8a4eb0b39bb05cd60137e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:13.568749 22028 leader_election.cc:290] T ba306baff7874796aba9a7c3fd5d628e P c48712233a8a4eb0b39bb05cd60137e6 [CANDIDATE]: Term 2 election: Requested vote from peers 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591), 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131:33869)
I20260709 13:21:13.569995 22103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba306baff7874796aba9a7c3fd5d628e" candidate_uuid: "c48712233a8a4eb0b39bb05cd60137e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776"
I20260709 13:21:13.570717 22103 raft_consensus.cc:3060] T ba306baff7874796aba9a7c3fd5d628e P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:13.572935 21954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba306baff7874796aba9a7c3fd5d628e" candidate_uuid: "c48712233a8a4eb0b39bb05cd60137e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb"
I20260709 13:21:13.573599 21954 raft_consensus.cc:3055] T ba306baff7874796aba9a7c3fd5d628e P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:13.573925 21954 raft_consensus.cc:740] T ba306baff7874796aba9a7c3fd5d628e P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8507b5a0d0e6434aaa7302085c7f54cb, State: Running, Role: LEADER
I20260709 13:21:13.574746 21954 consensus_queue.cc:260] T ba306baff7874796aba9a7c3fd5d628e P 8507b5a0d0e6434aaa7302085c7f54cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:13.576077 21954 raft_consensus.cc:3060] T ba306baff7874796aba9a7c3fd5d628e P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:13.578510 22103 raft_consensus.cc:2468] T ba306baff7874796aba9a7c3fd5d628e P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c48712233a8a4eb0b39bb05cd60137e6 in term 2.
I20260709 13:21:13.579829 21992 leader_election.cc:304] T ba306baff7874796aba9a7c3fd5d628e P c48712233a8a4eb0b39bb05cd60137e6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a7bb1d6ef54b20befdf8ea6d293776, c48712233a8a4eb0b39bb05cd60137e6; no voters: 
I20260709 13:21:13.580901 22178 raft_consensus.cc:2804] T ba306baff7874796aba9a7c3fd5d628e P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:13.581418 22178 raft_consensus.cc:697] T ba306baff7874796aba9a7c3fd5d628e P c48712233a8a4eb0b39bb05cd60137e6 [term 2 LEADER]: Becoming Leader. State: Replica: c48712233a8a4eb0b39bb05cd60137e6, State: Running, Role: LEADER
I20260709 13:21:13.582307 22178 consensus_queue.cc:237] T ba306baff7874796aba9a7c3fd5d628e P c48712233a8a4eb0b39bb05cd60137e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:13.586460 22153 raft_consensus.cc:993] T ea585ca33ebf4b329ce97d386bf3e7bc P c48712233a8a4eb0b39bb05cd60137e6: : Instructing follower 8507b5a0d0e6434aaa7302085c7f54cb to start an election
I20260709 13:21:13.587110 22175 raft_consensus.cc:1081] T ea585ca33ebf4b329ce97d386bf3e7bc P c48712233a8a4eb0b39bb05cd60137e6 [term 1 LEADER]: Signalling peer 8507b5a0d0e6434aaa7302085c7f54cb to start an election
I20260709 13:21:13.588862 21954 raft_consensus.cc:2468] T ba306baff7874796aba9a7c3fd5d628e P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c48712233a8a4eb0b39bb05cd60137e6 in term 2.
I20260709 13:21:13.590641 21953 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ea585ca33ebf4b329ce97d386bf3e7bc"
dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb"
 from {username='slave'} at 127.0.0.1:51340
I20260709 13:21:13.591228 21953 raft_consensus.cc:493] T ea585ca33ebf4b329ce97d386bf3e7bc P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:13.591646 21953 raft_consensus.cc:3060] T ea585ca33ebf4b329ce97d386bf3e7bc P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:13.593182 21852 catalog_manager.cc:5697] T ba306baff7874796aba9a7c3fd5d628e P c48712233a8a4eb0b39bb05cd60137e6 reported cstate change: term changed from 1 to 2, leader changed from 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129) to c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130). New cstate: current_term: 2 leader_uuid: "c48712233a8a4eb0b39bb05cd60137e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:13.597250 21953 raft_consensus.cc:515] T ea585ca33ebf4b329ce97d386bf3e7bc P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:13.599465 21953 leader_election.cc:290] T ea585ca33ebf4b329ce97d386bf3e7bc P 8507b5a0d0e6434aaa7302085c7f54cb [CANDIDATE]: Term 2 election: Requested vote from peers c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529), 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131:33869)
I20260709 13:21:13.600274 22028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea585ca33ebf4b329ce97d386bf3e7bc" candidate_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6"
I20260709 13:21:13.600641 22103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea585ca33ebf4b329ce97d386bf3e7bc" candidate_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776"
I20260709 13:21:13.600980 22028 raft_consensus.cc:3055] T ea585ca33ebf4b329ce97d386bf3e7bc P c48712233a8a4eb0b39bb05cd60137e6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:13.601325 22028 raft_consensus.cc:740] T ea585ca33ebf4b329ce97d386bf3e7bc P c48712233a8a4eb0b39bb05cd60137e6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c48712233a8a4eb0b39bb05cd60137e6, State: Running, Role: LEADER
I20260709 13:21:13.602289 22028 consensus_queue.cc:260] T ea585ca33ebf4b329ce97d386bf3e7bc P c48712233a8a4eb0b39bb05cd60137e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:13.603688 22028 raft_consensus.cc:3060] T ea585ca33ebf4b329ce97d386bf3e7bc P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:13.601317 22103 raft_consensus.cc:3060] T ea585ca33ebf4b329ce97d386bf3e7bc P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:13.606750 22174 consensus_queue.cc:1048] T 7f2c531192bb4639a60df411741abc7b P 21a7bb1d6ef54b20befdf8ea6d293776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:13.611193 22028 raft_consensus.cc:2468] T ea585ca33ebf4b329ce97d386bf3e7bc P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8507b5a0d0e6434aaa7302085c7f54cb in term 2.
I20260709 13:21:13.611193 22103 raft_consensus.cc:2468] T ea585ca33ebf4b329ce97d386bf3e7bc P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8507b5a0d0e6434aaa7302085c7f54cb in term 2.
I20260709 13:21:13.613077 21920 leader_election.cc:304] T ea585ca33ebf4b329ce97d386bf3e7bc P 8507b5a0d0e6434aaa7302085c7f54cb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8507b5a0d0e6434aaa7302085c7f54cb, c48712233a8a4eb0b39bb05cd60137e6; no voters: 
I20260709 13:21:13.613942 22148 raft_consensus.cc:2804] T ea585ca33ebf4b329ce97d386bf3e7bc P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:13.614462 22148 raft_consensus.cc:697] T ea585ca33ebf4b329ce97d386bf3e7bc P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 LEADER]: Becoming Leader. State: Replica: 8507b5a0d0e6434aaa7302085c7f54cb, State: Running, Role: LEADER
I20260709 13:21:13.615222 22148 consensus_queue.cc:237] T ea585ca33ebf4b329ce97d386bf3e7bc P 8507b5a0d0e6434aaa7302085c7f54cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:13.619853 22177 consensus_queue.cc:1048] T 7f2c531192bb4639a60df411741abc7b P 21a7bb1d6ef54b20befdf8ea6d293776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:13.626284 21852 catalog_manager.cc:5697] T ea585ca33ebf4b329ce97d386bf3e7bc P 8507b5a0d0e6434aaa7302085c7f54cb reported cstate change: term changed from 1 to 2, leader changed from c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130) to 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129). New cstate: current_term: 2 leader_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:13.631183 22186 raft_consensus.cc:993] T 7f2c531192bb4639a60df411741abc7b P 21a7bb1d6ef54b20befdf8ea6d293776: : Instructing follower c48712233a8a4eb0b39bb05cd60137e6 to start an election
I20260709 13:21:13.631780 22177 raft_consensus.cc:1081] T 7f2c531192bb4639a60df411741abc7b P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 LEADER]: Signalling peer c48712233a8a4eb0b39bb05cd60137e6 to start an election
I20260709 13:21:13.633677 22028 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7f2c531192bb4639a60df411741abc7b"
dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6"
 from {username='slave'} at 127.0.0.1:46568
I20260709 13:21:13.634280 22028 raft_consensus.cc:493] T 7f2c531192bb4639a60df411741abc7b P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:13.634667 22028 raft_consensus.cc:3060] T 7f2c531192bb4639a60df411741abc7b P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:13.640420 22028 raft_consensus.cc:515] T 7f2c531192bb4639a60df411741abc7b P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:13.642381 22028 leader_election.cc:290] T 7f2c531192bb4639a60df411741abc7b P c48712233a8a4eb0b39bb05cd60137e6 [CANDIDATE]: Term 2 election: Requested vote from peers 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591), 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131:33869)
I20260709 13:21:13.643254 21953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f2c531192bb4639a60df411741abc7b" candidate_uuid: "c48712233a8a4eb0b39bb05cd60137e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb"
I20260709 13:21:13.643558 22103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f2c531192bb4639a60df411741abc7b" candidate_uuid: "c48712233a8a4eb0b39bb05cd60137e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776"
I20260709 13:21:13.643973 21953 raft_consensus.cc:3060] T 7f2c531192bb4639a60df411741abc7b P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:13.644227 22103 raft_consensus.cc:3055] T 7f2c531192bb4639a60df411741abc7b P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:13.644623 22103 raft_consensus.cc:740] T 7f2c531192bb4639a60df411741abc7b P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 21a7bb1d6ef54b20befdf8ea6d293776, State: Running, Role: LEADER
I20260709 13:21:13.645502 22103 consensus_queue.cc:260] T 7f2c531192bb4639a60df411741abc7b P 21a7bb1d6ef54b20befdf8ea6d293776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:13.646822 22103 raft_consensus.cc:3060] T 7f2c531192bb4639a60df411741abc7b P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:13.650547 21953 raft_consensus.cc:2468] T 7f2c531192bb4639a60df411741abc7b P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c48712233a8a4eb0b39bb05cd60137e6 in term 2.
I20260709 13:21:13.651788 21995 leader_election.cc:304] T 7f2c531192bb4639a60df411741abc7b P c48712233a8a4eb0b39bb05cd60137e6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8507b5a0d0e6434aaa7302085c7f54cb, c48712233a8a4eb0b39bb05cd60137e6; no voters: 
I20260709 13:21:13.652508 22153 raft_consensus.cc:2804] T 7f2c531192bb4639a60df411741abc7b P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:13.652854 22103 raft_consensus.cc:2468] T 7f2c531192bb4639a60df411741abc7b P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c48712233a8a4eb0b39bb05cd60137e6 in term 2.
I20260709 13:21:13.652961 22153 raft_consensus.cc:697] T 7f2c531192bb4639a60df411741abc7b P c48712233a8a4eb0b39bb05cd60137e6 [term 2 LEADER]: Becoming Leader. State: Replica: c48712233a8a4eb0b39bb05cd60137e6, State: Running, Role: LEADER
I20260709 13:21:13.653985 22153 consensus_queue.cc:237] T 7f2c531192bb4639a60df411741abc7b P c48712233a8a4eb0b39bb05cd60137e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:13.660363 21852 catalog_manager.cc:5697] T 7f2c531192bb4639a60df411741abc7b P c48712233a8a4eb0b39bb05cd60137e6 reported cstate change: term changed from 1 to 2, leader changed from 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131) to c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130). New cstate: current_term: 2 leader_uuid: "c48712233a8a4eb0b39bb05cd60137e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:13.687345 22148 consensus_queue.cc:1048] T 0ca42dd42e4d46359256d9256f64d657 P 8507b5a0d0e6434aaa7302085c7f54cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:13.700203 22183 consensus_queue.cc:1048] T 0ca42dd42e4d46359256d9256f64d657 P 8507b5a0d0e6434aaa7302085c7f54cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:13.779517 22178 raft_consensus.cc:993] T 1631e507d6714dc187fc9d7f2457aad9 P c48712233a8a4eb0b39bb05cd60137e6: : Instructing follower 8507b5a0d0e6434aaa7302085c7f54cb to start an election
I20260709 13:21:13.780026 22178 raft_consensus.cc:1081] T 1631e507d6714dc187fc9d7f2457aad9 P c48712233a8a4eb0b39bb05cd60137e6 [term 1 LEADER]: Signalling peer 8507b5a0d0e6434aaa7302085c7f54cb to start an election
I20260709 13:21:13.781666 21953 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1631e507d6714dc187fc9d7f2457aad9"
dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb"
 from {username='slave'} at 127.0.0.1:51340
I20260709 13:21:13.782213 21953 raft_consensus.cc:493] T 1631e507d6714dc187fc9d7f2457aad9 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:13.782603 21953 raft_consensus.cc:3060] T 1631e507d6714dc187fc9d7f2457aad9 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:13.787760 21953 raft_consensus.cc:515] T 1631e507d6714dc187fc9d7f2457aad9 P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:13.789703 21953 leader_election.cc:290] T 1631e507d6714dc187fc9d7f2457aad9 P 8507b5a0d0e6434aaa7302085c7f54cb [CANDIDATE]: Term 2 election: Requested vote from peers c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529), 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131:33869)
I20260709 13:21:13.790730 22028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1631e507d6714dc187fc9d7f2457aad9" candidate_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6"
I20260709 13:21:13.790822 22103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1631e507d6714dc187fc9d7f2457aad9" candidate_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776"
I20260709 13:21:13.791450 22028 raft_consensus.cc:3055] T 1631e507d6714dc187fc9d7f2457aad9 P c48712233a8a4eb0b39bb05cd60137e6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:13.791673 22103 raft_consensus.cc:3060] T 1631e507d6714dc187fc9d7f2457aad9 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:13.791831 22028 raft_consensus.cc:740] T 1631e507d6714dc187fc9d7f2457aad9 P c48712233a8a4eb0b39bb05cd60137e6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c48712233a8a4eb0b39bb05cd60137e6, State: Running, Role: LEADER
I20260709 13:21:13.792661 22028 consensus_queue.cc:260] T 1631e507d6714dc187fc9d7f2457aad9 P c48712233a8a4eb0b39bb05cd60137e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:13.793677 22028 raft_consensus.cc:3060] T 1631e507d6714dc187fc9d7f2457aad9 P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:13.797700 22103 raft_consensus.cc:2468] T 1631e507d6714dc187fc9d7f2457aad9 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8507b5a0d0e6434aaa7302085c7f54cb in term 2.
I20260709 13:21:13.798998 21917 leader_election.cc:304] T 1631e507d6714dc187fc9d7f2457aad9 P 8507b5a0d0e6434aaa7302085c7f54cb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a7bb1d6ef54b20befdf8ea6d293776, 8507b5a0d0e6434aaa7302085c7f54cb; no voters: 
I20260709 13:21:13.799376 22028 raft_consensus.cc:2468] T 1631e507d6714dc187fc9d7f2457aad9 P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8507b5a0d0e6434aaa7302085c7f54cb in term 2.
I20260709 13:21:13.799917 22184 raft_consensus.cc:2804] T 1631e507d6714dc187fc9d7f2457aad9 P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:13.800433 22184 raft_consensus.cc:697] T 1631e507d6714dc187fc9d7f2457aad9 P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 LEADER]: Becoming Leader. State: Replica: 8507b5a0d0e6434aaa7302085c7f54cb, State: Running, Role: LEADER
I20260709 13:21:13.801311 22184 consensus_queue.cc:237] T 1631e507d6714dc187fc9d7f2457aad9 P 8507b5a0d0e6434aaa7302085c7f54cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:13.808075 21850 catalog_manager.cc:5697] T 1631e507d6714dc187fc9d7f2457aad9 P 8507b5a0d0e6434aaa7302085c7f54cb reported cstate change: term changed from 1 to 2, leader changed from c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130) to 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129). New cstate: current_term: 2 leader_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:13.921725 22148 raft_consensus.cc:993] T f9dceef9cf6541039db61f39e2798430 P 8507b5a0d0e6434aaa7302085c7f54cb: : Instructing follower c48712233a8a4eb0b39bb05cd60137e6 to start an election
I20260709 13:21:13.922245 22148 raft_consensus.cc:1081] T f9dceef9cf6541039db61f39e2798430 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 LEADER]: Signalling peer c48712233a8a4eb0b39bb05cd60137e6 to start an election
I20260709 13:21:13.924085 22028 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f9dceef9cf6541039db61f39e2798430"
dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6"
 from {username='slave'} at 127.0.0.1:46584
I20260709 13:21:13.924613 22028 raft_consensus.cc:493] T f9dceef9cf6541039db61f39e2798430 P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:13.925036 22028 raft_consensus.cc:3060] T f9dceef9cf6541039db61f39e2798430 P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:13.930308 22028 raft_consensus.cc:515] T f9dceef9cf6541039db61f39e2798430 P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:13.932114 22028 leader_election.cc:290] T f9dceef9cf6541039db61f39e2798430 P c48712233a8a4eb0b39bb05cd60137e6 [CANDIDATE]: Term 2 election: Requested vote from peers 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591), 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131:33869)
I20260709 13:21:13.933002 21953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9dceef9cf6541039db61f39e2798430" candidate_uuid: "c48712233a8a4eb0b39bb05cd60137e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb"
I20260709 13:21:13.933421 22103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9dceef9cf6541039db61f39e2798430" candidate_uuid: "c48712233a8a4eb0b39bb05cd60137e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776"
I20260709 13:21:13.933777 21953 raft_consensus.cc:3055] T f9dceef9cf6541039db61f39e2798430 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:13.934073 22103 raft_consensus.cc:3060] T f9dceef9cf6541039db61f39e2798430 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:13.934158 21953 raft_consensus.cc:740] T f9dceef9cf6541039db61f39e2798430 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8507b5a0d0e6434aaa7302085c7f54cb, State: Running, Role: LEADER
I20260709 13:21:13.935154 21953 consensus_queue.cc:260] T f9dceef9cf6541039db61f39e2798430 P 8507b5a0d0e6434aaa7302085c7f54cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:13.936470 21953 raft_consensus.cc:3060] T f9dceef9cf6541039db61f39e2798430 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:13.941200 22102 raft_consensus.cc:1275] T d74919ac686e49e694ca111d3b1a27f6 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Refusing update from remote peer c48712233a8a4eb0b39bb05cd60137e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:13.942068 22103 raft_consensus.cc:2468] T f9dceef9cf6541039db61f39e2798430 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c48712233a8a4eb0b39bb05cd60137e6 in term 2.
I20260709 13:21:13.942718 22178 consensus_queue.cc:1048] T d74919ac686e49e694ca111d3b1a27f6 P c48712233a8a4eb0b39bb05cd60137e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:13.943411 21992 leader_election.cc:304] T f9dceef9cf6541039db61f39e2798430 P c48712233a8a4eb0b39bb05cd60137e6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a7bb1d6ef54b20befdf8ea6d293776, c48712233a8a4eb0b39bb05cd60137e6; no voters: 
I20260709 13:21:13.944015 21953 raft_consensus.cc:2468] T f9dceef9cf6541039db61f39e2798430 P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c48712233a8a4eb0b39bb05cd60137e6 in term 2.
I20260709 13:21:13.944306 22178 raft_consensus.cc:2804] T f9dceef9cf6541039db61f39e2798430 P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:13.944918 22178 raft_consensus.cc:697] T f9dceef9cf6541039db61f39e2798430 P c48712233a8a4eb0b39bb05cd60137e6 [term 2 LEADER]: Becoming Leader. State: Replica: c48712233a8a4eb0b39bb05cd60137e6, State: Running, Role: LEADER
I20260709 13:21:13.945593 22178 consensus_queue.cc:237] T f9dceef9cf6541039db61f39e2798430 P c48712233a8a4eb0b39bb05cd60137e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:13.954517 21953 raft_consensus.cc:1275] T d74919ac686e49e694ca111d3b1a27f6 P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Refusing update from remote peer c48712233a8a4eb0b39bb05cd60137e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:13.954882 21852 catalog_manager.cc:5697] T f9dceef9cf6541039db61f39e2798430 P c48712233a8a4eb0b39bb05cd60137e6 reported cstate change: term changed from 1 to 2, leader changed from 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129) to c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130). New cstate: current_term: 2 leader_uuid: "c48712233a8a4eb0b39bb05cd60137e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:13.956359 22175 consensus_queue.cc:1048] T d74919ac686e49e694ca111d3b1a27f6 P c48712233a8a4eb0b39bb05cd60137e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:13.998713 22028 raft_consensus.cc:1275] T ea585ca33ebf4b329ce97d386bf3e7bc P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Refusing update from remote peer 8507b5a0d0e6434aaa7302085c7f54cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:14.000452 22148 consensus_queue.cc:1048] T ea585ca33ebf4b329ce97d386bf3e7bc P 8507b5a0d0e6434aaa7302085c7f54cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:14.011317 22103 raft_consensus.cc:1275] T ea585ca33ebf4b329ce97d386bf3e7bc P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Refusing update from remote peer 8507b5a0d0e6434aaa7302085c7f54cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:14.012920 22184 consensus_queue.cc:1048] T ea585ca33ebf4b329ce97d386bf3e7bc P 8507b5a0d0e6434aaa7302085c7f54cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:14.055392 22103 raft_consensus.cc:1275] T ba306baff7874796aba9a7c3fd5d628e P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Refusing update from remote peer c48712233a8a4eb0b39bb05cd60137e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:14.056823 22175 consensus_queue.cc:1048] T ba306baff7874796aba9a7c3fd5d628e P c48712233a8a4eb0b39bb05cd60137e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:14.065121 21953 raft_consensus.cc:1275] T ba306baff7874796aba9a7c3fd5d628e P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Refusing update from remote peer c48712233a8a4eb0b39bb05cd60137e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:14.067839 22153 consensus_queue.cc:1048] T ba306baff7874796aba9a7c3fd5d628e P c48712233a8a4eb0b39bb05cd60137e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:21:14.076566 22103 raft_consensus.cc:1275] T 7f2c531192bb4639a60df411741abc7b P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Refusing update from remote peer c48712233a8a4eb0b39bb05cd60137e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:14.077863 22178 consensus_queue.cc:1048] T 7f2c531192bb4639a60df411741abc7b P c48712233a8a4eb0b39bb05cd60137e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:14.087419 22102 raft_consensus.cc:1275] T b8791ec386d148c19bb6fb734287e432 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Refusing update from remote peer 8507b5a0d0e6434aaa7302085c7f54cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:14.088093 21953 raft_consensus.cc:1275] T 7f2c531192bb4639a60df411741abc7b P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Refusing update from remote peer c48712233a8a4eb0b39bb05cd60137e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:14.089159 22184 consensus_queue.cc:1048] T b8791ec386d148c19bb6fb734287e432 P 8507b5a0d0e6434aaa7302085c7f54cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:21:14.089624 22175 consensus_queue.cc:1048] T 7f2c531192bb4639a60df411741abc7b P c48712233a8a4eb0b39bb05cd60137e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:14.100967 22028 raft_consensus.cc:3060] T b8791ec386d148c19bb6fb734287e432 P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:14.107065 22028 raft_consensus.cc:1275] T b8791ec386d148c19bb6fb734287e432 P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Refusing update from remote peer 8507b5a0d0e6434aaa7302085c7f54cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:14.108834 22226 consensus_queue.cc:1048] T b8791ec386d148c19bb6fb734287e432 P 8507b5a0d0e6434aaa7302085c7f54cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:14.221573 22028 raft_consensus.cc:1275] T 1631e507d6714dc187fc9d7f2457aad9 P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Refusing update from remote peer 8507b5a0d0e6434aaa7302085c7f54cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:14.222942 22226 consensus_queue.cc:1048] T 1631e507d6714dc187fc9d7f2457aad9 P 8507b5a0d0e6434aaa7302085c7f54cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:14.230818 22103 raft_consensus.cc:1275] T 1631e507d6714dc187fc9d7f2457aad9 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Refusing update from remote peer 8507b5a0d0e6434aaa7302085c7f54cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:14.232650 22183 consensus_queue.cc:1048] T 1631e507d6714dc187fc9d7f2457aad9 P 8507b5a0d0e6434aaa7302085c7f54cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:14.387483 21953 raft_consensus.cc:1275] T f9dceef9cf6541039db61f39e2798430 P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Refusing update from remote peer c48712233a8a4eb0b39bb05cd60137e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:14.389091 22175 consensus_queue.cc:1048] T f9dceef9cf6541039db61f39e2798430 P c48712233a8a4eb0b39bb05cd60137e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:14.397241 22103 raft_consensus.cc:1275] T f9dceef9cf6541039db61f39e2798430 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Refusing update from remote peer c48712233a8a4eb0b39bb05cd60137e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:14.399031 22225 consensus_queue.cc:1048] T f9dceef9cf6541039db61f39e2798430 P c48712233a8a4eb0b39bb05cd60137e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:21:16.514375 21850 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45298:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:21:16.516642 21850 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:21:16.571704 21938 tablet_service.cc:1511] Processing CreateTablet for tablet b740d838804d4730b30e939a24d5414d (DEFAULT_TABLE table=soft_deleted_table [id=9654cec777eb43cdbb439e3b6c18c25c]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:21:16.572965 21938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b740d838804d4730b30e939a24d5414d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:16.573589 22015 tablet_service.cc:1511] Processing CreateTablet for tablet b740d838804d4730b30e939a24d5414d (DEFAULT_TABLE table=soft_deleted_table [id=9654cec777eb43cdbb439e3b6c18c25c]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:21:16.574563 22092 tablet_service.cc:1511] Processing CreateTablet for tablet b740d838804d4730b30e939a24d5414d (DEFAULT_TABLE table=soft_deleted_table [id=9654cec777eb43cdbb439e3b6c18c25c]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:21:16.575862 22092 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b740d838804d4730b30e939a24d5414d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:16.576704 22015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b740d838804d4730b30e939a24d5414d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:16.576673 21936 tablet_service.cc:1511] Processing CreateTablet for tablet 74231e641a7e4f7fb5bc39a2a084bb81 (DEFAULT_TABLE table=soft_deleted_table [id=9654cec777eb43cdbb439e3b6c18c25c]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:21:16.578042 21936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74231e641a7e4f7fb5bc39a2a084bb81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:16.582053 22018 tablet_service.cc:1511] Processing CreateTablet for tablet 74231e641a7e4f7fb5bc39a2a084bb81 (DEFAULT_TABLE table=soft_deleted_table [id=9654cec777eb43cdbb439e3b6c18c25c]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:21:16.583731 22018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74231e641a7e4f7fb5bc39a2a084bb81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:16.604866 22093 tablet_service.cc:1511] Processing CreateTablet for tablet 74231e641a7e4f7fb5bc39a2a084bb81 (DEFAULT_TABLE table=soft_deleted_table [id=9654cec777eb43cdbb439e3b6c18c25c]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:21:16.605403 22239 tablet_bootstrap.cc:492] T b740d838804d4730b30e939a24d5414d P 21a7bb1d6ef54b20befdf8ea6d293776: Bootstrap starting.
I20260709 13:21:16.605574 22238 tablet_bootstrap.cc:492] T b740d838804d4730b30e939a24d5414d P 8507b5a0d0e6434aaa7302085c7f54cb: Bootstrap starting.
I20260709 13:21:16.606343 22093 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74231e641a7e4f7fb5bc39a2a084bb81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:16.612118 21938 tablet_service.cc:1511] Processing CreateTablet for tablet b86408eb7a2544ff921e6534948d5b67 (DEFAULT_TABLE table=soft_deleted_table [id=9654cec777eb43cdbb439e3b6c18c25c]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:21:16.613595 21938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b86408eb7a2544ff921e6534948d5b67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:16.613386 22015 tablet_service.cc:1511] Processing CreateTablet for tablet b86408eb7a2544ff921e6534948d5b67 (DEFAULT_TABLE table=soft_deleted_table [id=9654cec777eb43cdbb439e3b6c18c25c]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:21:16.614420 22239 tablet_bootstrap.cc:654] T b740d838804d4730b30e939a24d5414d P 21a7bb1d6ef54b20befdf8ea6d293776: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:16.614751 22015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b86408eb7a2544ff921e6534948d5b67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:16.607530 22240 tablet_bootstrap.cc:492] T b740d838804d4730b30e939a24d5414d P c48712233a8a4eb0b39bb05cd60137e6: Bootstrap starting.
I20260709 13:21:16.621981 22240 tablet_bootstrap.cc:654] T b740d838804d4730b30e939a24d5414d P c48712233a8a4eb0b39bb05cd60137e6: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:16.630040 22238 tablet_bootstrap.cc:654] T b740d838804d4730b30e939a24d5414d P 8507b5a0d0e6434aaa7302085c7f54cb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:16.630740 22092 tablet_service.cc:1511] Processing CreateTablet for tablet b86408eb7a2544ff921e6534948d5b67 (DEFAULT_TABLE table=soft_deleted_table [id=9654cec777eb43cdbb439e3b6c18c25c]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:21:16.635885 22092 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b86408eb7a2544ff921e6534948d5b67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:16.631955 21941 tablet_service.cc:1511] Processing CreateTablet for tablet 400f2b2a8e114d5d9506446a360c4daf (DEFAULT_TABLE table=soft_deleted_table [id=9654cec777eb43cdbb439e3b6c18c25c]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:21:16.640383 21941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 400f2b2a8e114d5d9506446a360c4daf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:16.644902 22016 tablet_service.cc:1511] Processing CreateTablet for tablet 400f2b2a8e114d5d9506446a360c4daf (DEFAULT_TABLE table=soft_deleted_table [id=9654cec777eb43cdbb439e3b6c18c25c]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:21:16.646384 22240 tablet_bootstrap.cc:492] T b740d838804d4730b30e939a24d5414d P c48712233a8a4eb0b39bb05cd60137e6: No bootstrap required, opened a new log
I20260709 13:21:16.647205 22240 ts_tablet_manager.cc:1403] T b740d838804d4730b30e939a24d5414d P c48712233a8a4eb0b39bb05cd60137e6: Time spent bootstrapping tablet: real 0.040s	user 0.006s	sys 0.010s
I20260709 13:21:16.648836 22016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 400f2b2a8e114d5d9506446a360c4daf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:16.655966 22240 raft_consensus.cc:359] T b740d838804d4730b30e939a24d5414d P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:16.658250 22093 tablet_service.cc:1511] Processing CreateTablet for tablet 400f2b2a8e114d5d9506446a360c4daf (DEFAULT_TABLE table=soft_deleted_table [id=9654cec777eb43cdbb439e3b6c18c25c]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:21:16.664940 22240 raft_consensus.cc:385] T b740d838804d4730b30e939a24d5414d P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:16.665266 22240 raft_consensus.cc:740] T b740d838804d4730b30e939a24d5414d P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c48712233a8a4eb0b39bb05cd60137e6, State: Initialized, Role: FOLLOWER
I20260709 13:21:16.665649 22093 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 400f2b2a8e114d5d9506446a360c4daf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:16.666183 22240 consensus_queue.cc:260] T b740d838804d4730b30e939a24d5414d P c48712233a8a4eb0b39bb05cd60137e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:16.668414 21938 tablet_service.cc:1511] Processing CreateTablet for tablet 455c56b836ff4e63949be17b0cf25c40 (DEFAULT_TABLE table=soft_deleted_table [id=9654cec777eb43cdbb439e3b6c18c25c]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:21:16.669894 21938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 455c56b836ff4e63949be17b0cf25c40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:16.668833 22240 ts_tablet_manager.cc:1434] T b740d838804d4730b30e939a24d5414d P c48712233a8a4eb0b39bb05cd60137e6: Time spent starting tablet: real 0.021s	user 0.003s	sys 0.003s
I20260709 13:21:16.664507 22239 tablet_bootstrap.cc:492] T b740d838804d4730b30e939a24d5414d P 21a7bb1d6ef54b20befdf8ea6d293776: No bootstrap required, opened a new log
I20260709 13:21:16.672533 22239 ts_tablet_manager.cc:1403] T b740d838804d4730b30e939a24d5414d P 21a7bb1d6ef54b20befdf8ea6d293776: Time spent bootstrapping tablet: real 0.067s	user 0.017s	sys 0.013s
I20260709 13:21:16.672818 22240 tablet_bootstrap.cc:492] T 400f2b2a8e114d5d9506446a360c4daf P c48712233a8a4eb0b39bb05cd60137e6: Bootstrap starting.
I20260709 13:21:16.677556 22239 raft_consensus.cc:359] T b740d838804d4730b30e939a24d5414d P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:16.678459 22239 raft_consensus.cc:385] T b740d838804d4730b30e939a24d5414d P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:16.678856 22239 raft_consensus.cc:740] T b740d838804d4730b30e939a24d5414d P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21a7bb1d6ef54b20befdf8ea6d293776, State: Initialized, Role: FOLLOWER
I20260709 13:21:16.680058 22239 consensus_queue.cc:260] T b740d838804d4730b30e939a24d5414d P 21a7bb1d6ef54b20befdf8ea6d293776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:16.682369 22238 tablet_bootstrap.cc:492] T b740d838804d4730b30e939a24d5414d P 8507b5a0d0e6434aaa7302085c7f54cb: No bootstrap required, opened a new log
I20260709 13:21:16.682979 22238 ts_tablet_manager.cc:1403] T b740d838804d4730b30e939a24d5414d P 8507b5a0d0e6434aaa7302085c7f54cb: Time spent bootstrapping tablet: real 0.079s	user 0.018s	sys 0.000s
I20260709 13:21:16.686559 22239 ts_tablet_manager.cc:1434] T b740d838804d4730b30e939a24d5414d P 21a7bb1d6ef54b20befdf8ea6d293776: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:21:16.688624 22240 tablet_bootstrap.cc:654] T 400f2b2a8e114d5d9506446a360c4daf P c48712233a8a4eb0b39bb05cd60137e6: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:16.686234 22238 raft_consensus.cc:359] T b740d838804d4730b30e939a24d5414d P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:16.692451 22238 raft_consensus.cc:385] T b740d838804d4730b30e939a24d5414d P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:16.692873 22238 raft_consensus.cc:740] T b740d838804d4730b30e939a24d5414d P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8507b5a0d0e6434aaa7302085c7f54cb, State: Initialized, Role: FOLLOWER
I20260709 13:21:16.693723 22238 consensus_queue.cc:260] T b740d838804d4730b30e939a24d5414d P 8507b5a0d0e6434aaa7302085c7f54cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:16.704804 22238 ts_tablet_manager.cc:1434] T b740d838804d4730b30e939a24d5414d P 8507b5a0d0e6434aaa7302085c7f54cb: Time spent starting tablet: real 0.021s	user 0.014s	sys 0.000s
I20260709 13:21:16.707458 22238 tablet_bootstrap.cc:492] T 74231e641a7e4f7fb5bc39a2a084bb81 P 8507b5a0d0e6434aaa7302085c7f54cb: Bootstrap starting.
I20260709 13:21:16.709777 22092 tablet_service.cc:1511] Processing CreateTablet for tablet 455c56b836ff4e63949be17b0cf25c40 (DEFAULT_TABLE table=soft_deleted_table [id=9654cec777eb43cdbb439e3b6c18c25c]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:21:16.711299 22015 tablet_service.cc:1511] Processing CreateTablet for tablet 455c56b836ff4e63949be17b0cf25c40 (DEFAULT_TABLE table=soft_deleted_table [id=9654cec777eb43cdbb439e3b6c18c25c]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:21:16.713284 22015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 455c56b836ff4e63949be17b0cf25c40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:16.715432 22238 tablet_bootstrap.cc:654] T 74231e641a7e4f7fb5bc39a2a084bb81 P 8507b5a0d0e6434aaa7302085c7f54cb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:16.715268 21936 tablet_service.cc:1511] Processing CreateTablet for tablet 91f78fe291144d05bce820bf9ee9cc03 (DEFAULT_TABLE table=soft_deleted_table [id=9654cec777eb43cdbb439e3b6c18c25c]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:21:16.716974 21936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91f78fe291144d05bce820bf9ee9cc03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:16.711416 22092 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 455c56b836ff4e63949be17b0cf25c40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:16.687793 22239 tablet_bootstrap.cc:492] T 74231e641a7e4f7fb5bc39a2a084bb81 P 21a7bb1d6ef54b20befdf8ea6d293776: Bootstrap starting.
I20260709 13:21:16.741390 22238 tablet_bootstrap.cc:492] T 74231e641a7e4f7fb5bc39a2a084bb81 P 8507b5a0d0e6434aaa7302085c7f54cb: No bootstrap required, opened a new log
I20260709 13:21:16.742033 22238 ts_tablet_manager.cc:1403] T 74231e641a7e4f7fb5bc39a2a084bb81 P 8507b5a0d0e6434aaa7302085c7f54cb: Time spent bootstrapping tablet: real 0.035s	user 0.013s	sys 0.005s
W20260709 13:21:16.748145 21900 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:21:16.748432 22018 tablet_service.cc:1511] Processing CreateTablet for tablet 91f78fe291144d05bce820bf9ee9cc03 (DEFAULT_TABLE table=soft_deleted_table [id=9654cec777eb43cdbb439e3b6c18c25c]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:21:16.749857 22018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91f78fe291144d05bce820bf9ee9cc03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:16.750743 22238 raft_consensus.cc:359] T 74231e641a7e4f7fb5bc39a2a084bb81 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:16.751616 22238 raft_consensus.cc:385] T 74231e641a7e4f7fb5bc39a2a084bb81 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:16.751915 22238 raft_consensus.cc:740] T 74231e641a7e4f7fb5bc39a2a084bb81 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8507b5a0d0e6434aaa7302085c7f54cb, State: Initialized, Role: FOLLOWER
I20260709 13:21:16.752739 22238 consensus_queue.cc:260] T 74231e641a7e4f7fb5bc39a2a084bb81 P 8507b5a0d0e6434aaa7302085c7f54cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:16.755879 22238 ts_tablet_manager.cc:1434] T 74231e641a7e4f7fb5bc39a2a084bb81 P 8507b5a0d0e6434aaa7302085c7f54cb: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.005s
I20260709 13:21:16.758172 22238 tablet_bootstrap.cc:492] T b86408eb7a2544ff921e6534948d5b67 P 8507b5a0d0e6434aaa7302085c7f54cb: Bootstrap starting.
I20260709 13:21:16.762617 22089 tablet_service.cc:1511] Processing CreateTablet for tablet 91f78fe291144d05bce820bf9ee9cc03 (DEFAULT_TABLE table=soft_deleted_table [id=9654cec777eb43cdbb439e3b6c18c25c]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:21:16.764179 22089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91f78fe291144d05bce820bf9ee9cc03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:16.774876 21939 tablet_service.cc:1511] Processing CreateTablet for tablet 300ddb6928744c789723f712d6f9780e (DEFAULT_TABLE table=soft_deleted_table [id=9654cec777eb43cdbb439e3b6c18c25c]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:21:16.776469 21939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 300ddb6928744c789723f712d6f9780e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:16.781487 22245 raft_consensus.cc:493] T b740d838804d4730b30e939a24d5414d P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:16.782130 22245 raft_consensus.cc:515] T b740d838804d4730b30e939a24d5414d P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:16.776418 22240 tablet_bootstrap.cc:492] T 400f2b2a8e114d5d9506446a360c4daf P c48712233a8a4eb0b39bb05cd60137e6: No bootstrap required, opened a new log
I20260709 13:21:16.783478 22240 ts_tablet_manager.cc:1403] T 400f2b2a8e114d5d9506446a360c4daf P c48712233a8a4eb0b39bb05cd60137e6: Time spent bootstrapping tablet: real 0.111s	user 0.030s	sys 0.043s
I20260709 13:21:16.784539 22245 leader_election.cc:290] T b740d838804d4730b30e939a24d5414d P 8507b5a0d0e6434aaa7302085c7f54cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529), 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131:33869)
I20260709 13:21:16.785686 22103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b740d838804d4730b30e939a24d5414d" candidate_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" is_pre_election: true
I20260709 13:21:16.786340 22103 raft_consensus.cc:2468] T b740d838804d4730b30e939a24d5414d P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8507b5a0d0e6434aaa7302085c7f54cb in term 0.
I20260709 13:21:16.786628 22240 raft_consensus.cc:359] T 400f2b2a8e114d5d9506446a360c4daf P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:16.787436 22240 raft_consensus.cc:385] T 400f2b2a8e114d5d9506446a360c4daf P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:16.787726 22240 raft_consensus.cc:740] T 400f2b2a8e114d5d9506446a360c4daf P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c48712233a8a4eb0b39bb05cd60137e6, State: Initialized, Role: FOLLOWER
I20260709 13:21:16.787727 21917 leader_election.cc:304] T b740d838804d4730b30e939a24d5414d P 8507b5a0d0e6434aaa7302085c7f54cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a7bb1d6ef54b20befdf8ea6d293776, 8507b5a0d0e6434aaa7302085c7f54cb; no voters: 
I20260709 13:21:16.788578 22240 consensus_queue.cc:260] T 400f2b2a8e114d5d9506446a360c4daf P c48712233a8a4eb0b39bb05cd60137e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:16.790560 22240 ts_tablet_manager.cc:1434] T 400f2b2a8e114d5d9506446a360c4daf P c48712233a8a4eb0b39bb05cd60137e6: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:21:16.790709 22028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b740d838804d4730b30e939a24d5414d" candidate_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6" is_pre_election: true
I20260709 13:21:16.791471 22028 raft_consensus.cc:2468] T b740d838804d4730b30e939a24d5414d P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8507b5a0d0e6434aaa7302085c7f54cb in term 0.
I20260709 13:21:16.791783 22240 tablet_bootstrap.cc:492] T 74231e641a7e4f7fb5bc39a2a084bb81 P c48712233a8a4eb0b39bb05cd60137e6: Bootstrap starting.
I20260709 13:21:16.799234 22239 tablet_bootstrap.cc:654] T 74231e641a7e4f7fb5bc39a2a084bb81 P 21a7bb1d6ef54b20befdf8ea6d293776: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:16.799785 22240 tablet_bootstrap.cc:654] T 74231e641a7e4f7fb5bc39a2a084bb81 P c48712233a8a4eb0b39bb05cd60137e6: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:16.789885 22015 tablet_service.cc:1511] Processing CreateTablet for tablet 300ddb6928744c789723f712d6f9780e (DEFAULT_TABLE table=soft_deleted_table [id=9654cec777eb43cdbb439e3b6c18c25c]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:21:16.802063 22245 raft_consensus.cc:2804] T b740d838804d4730b30e939a24d5414d P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:16.802592 22245 raft_consensus.cc:493] T b740d838804d4730b30e939a24d5414d P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:16.803073 22015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 300ddb6928744c789723f712d6f9780e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:16.803143 22245 raft_consensus.cc:3060] T b740d838804d4730b30e939a24d5414d P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:16.811441 22238 tablet_bootstrap.cc:654] T b86408eb7a2544ff921e6534948d5b67 P 8507b5a0d0e6434aaa7302085c7f54cb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:16.817430 22092 tablet_service.cc:1511] Processing CreateTablet for tablet 300ddb6928744c789723f712d6f9780e (DEFAULT_TABLE table=soft_deleted_table [id=9654cec777eb43cdbb439e3b6c18c25c]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:21:16.821898 22092 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 300ddb6928744c789723f712d6f9780e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:16.827963 22018 tablet_service.cc:1511] Processing CreateTablet for tablet dfb67aaa32244d76a341136766820549 (DEFAULT_TABLE table=soft_deleted_table [id=9654cec777eb43cdbb439e3b6c18c25c]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:21:16.829156 22239 tablet_bootstrap.cc:492] T 74231e641a7e4f7fb5bc39a2a084bb81 P 21a7bb1d6ef54b20befdf8ea6d293776: No bootstrap required, opened a new log
I20260709 13:21:16.829409 22018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfb67aaa32244d76a341136766820549. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:16.829807 22239 ts_tablet_manager.cc:1403] T 74231e641a7e4f7fb5bc39a2a084bb81 P 21a7bb1d6ef54b20befdf8ea6d293776: Time spent bootstrapping tablet: real 0.142s	user 0.008s	sys 0.004s
I20260709 13:21:16.831560 21936 tablet_service.cc:1511] Processing CreateTablet for tablet dfb67aaa32244d76a341136766820549 (DEFAULT_TABLE table=soft_deleted_table [id=9654cec777eb43cdbb439e3b6c18c25c]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:21:16.834684 22093 tablet_service.cc:1511] Processing CreateTablet for tablet dfb67aaa32244d76a341136766820549 (DEFAULT_TABLE table=soft_deleted_table [id=9654cec777eb43cdbb439e3b6c18c25c]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:21:16.836938 21936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfb67aaa32244d76a341136766820549. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:16.837508 22093 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfb67aaa32244d76a341136766820549. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:16.845804 22226 raft_consensus.cc:493] T 74231e641a7e4f7fb5bc39a2a084bb81 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:16.846459 22226 raft_consensus.cc:515] T 74231e641a7e4f7fb5bc39a2a084bb81 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:16.848887 22226 leader_election.cc:290] T 74231e641a7e4f7fb5bc39a2a084bb81 P 8507b5a0d0e6434aaa7302085c7f54cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529), 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131:33869)
I20260709 13:21:16.850235 22028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74231e641a7e4f7fb5bc39a2a084bb81" candidate_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6" is_pre_election: true
W20260709 13:21:16.852319 21920 leader_election.cc:343] T 74231e641a7e4f7fb5bc39a2a084bb81 P 8507b5a0d0e6434aaa7302085c7f54cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:16.861987 22239 raft_consensus.cc:359] T 74231e641a7e4f7fb5bc39a2a084bb81 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:16.862901 22239 raft_consensus.cc:385] T 74231e641a7e4f7fb5bc39a2a084bb81 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:16.863214 22239 raft_consensus.cc:740] T 74231e641a7e4f7fb5bc39a2a084bb81 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21a7bb1d6ef54b20befdf8ea6d293776, State: Initialized, Role: FOLLOWER
I20260709 13:21:16.863940 22239 consensus_queue.cc:260] T 74231e641a7e4f7fb5bc39a2a084bb81 P 21a7bb1d6ef54b20befdf8ea6d293776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:16.866254 22239 ts_tablet_manager.cc:1434] T 74231e641a7e4f7fb5bc39a2a084bb81 P 21a7bb1d6ef54b20befdf8ea6d293776: Time spent starting tablet: real 0.036s	user 0.004s	sys 0.004s
I20260709 13:21:16.867417 22239 tablet_bootstrap.cc:492] T b86408eb7a2544ff921e6534948d5b67 P 21a7bb1d6ef54b20befdf8ea6d293776: Bootstrap starting.
I20260709 13:21:16.867298 22090 tablet_service.cc:1511] Processing CreateTablet for tablet d481ac177e6345efa62e86898d48fbac (DEFAULT_TABLE table=soft_deleted_table [id=9654cec777eb43cdbb439e3b6c18c25c]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:21:16.868815 22090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d481ac177e6345efa62e86898d48fbac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:16.868996 22015 tablet_service.cc:1511] Processing CreateTablet for tablet d481ac177e6345efa62e86898d48fbac (DEFAULT_TABLE table=soft_deleted_table [id=9654cec777eb43cdbb439e3b6c18c25c]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:21:16.870271 22015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d481ac177e6345efa62e86898d48fbac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:16.871945 22103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74231e641a7e4f7fb5bc39a2a084bb81" candidate_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" is_pre_election: true
I20260709 13:21:16.872650 22103 raft_consensus.cc:2468] T 74231e641a7e4f7fb5bc39a2a084bb81 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8507b5a0d0e6434aaa7302085c7f54cb in term 0.
I20260709 13:21:16.873013 21939 tablet_service.cc:1511] Processing CreateTablet for tablet d481ac177e6345efa62e86898d48fbac (DEFAULT_TABLE table=soft_deleted_table [id=9654cec777eb43cdbb439e3b6c18c25c]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:21:16.873955 21917 leader_election.cc:304] T 74231e641a7e4f7fb5bc39a2a084bb81 P 8507b5a0d0e6434aaa7302085c7f54cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 21a7bb1d6ef54b20befdf8ea6d293776, 8507b5a0d0e6434aaa7302085c7f54cb; no voters: c48712233a8a4eb0b39bb05cd60137e6
I20260709 13:21:16.874367 21939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d481ac177e6345efa62e86898d48fbac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:16.874465 22238 tablet_bootstrap.cc:492] T b86408eb7a2544ff921e6534948d5b67 P 8507b5a0d0e6434aaa7302085c7f54cb: No bootstrap required, opened a new log
I20260709 13:21:16.875156 22238 ts_tablet_manager.cc:1403] T b86408eb7a2544ff921e6534948d5b67 P 8507b5a0d0e6434aaa7302085c7f54cb: Time spent bootstrapping tablet: real 0.117s	user 0.009s	sys 0.005s
I20260709 13:21:16.875844 22250 raft_consensus.cc:2804] T 74231e641a7e4f7fb5bc39a2a084bb81 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:16.876314 22250 raft_consensus.cc:493] T 74231e641a7e4f7fb5bc39a2a084bb81 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:16.876686 22250 raft_consensus.cc:3060] T 74231e641a7e4f7fb5bc39a2a084bb81 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:16.892971 22239 tablet_bootstrap.cc:654] T b86408eb7a2544ff921e6534948d5b67 P 21a7bb1d6ef54b20befdf8ea6d293776: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:16.894229 22250 raft_consensus.cc:515] T 74231e641a7e4f7fb5bc39a2a084bb81 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:16.898924 22103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74231e641a7e4f7fb5bc39a2a084bb81" candidate_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776"
I20260709 13:21:16.899541 22103 raft_consensus.cc:3060] T 74231e641a7e4f7fb5bc39a2a084bb81 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:16.900242 22028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74231e641a7e4f7fb5bc39a2a084bb81" candidate_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6"
W20260709 13:21:16.901671 21920 leader_election.cc:343] T 74231e641a7e4f7fb5bc39a2a084bb81 P 8507b5a0d0e6434aaa7302085c7f54cb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:16.878621 22238 raft_consensus.cc:359] T b86408eb7a2544ff921e6534948d5b67 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:16.903337 22238 raft_consensus.cc:385] T b86408eb7a2544ff921e6534948d5b67 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:16.903625 22238 raft_consensus.cc:740] T b86408eb7a2544ff921e6534948d5b67 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8507b5a0d0e6434aaa7302085c7f54cb, State: Initialized, Role: FOLLOWER
I20260709 13:21:16.905571 22238 consensus_queue.cc:260] T b86408eb7a2544ff921e6534948d5b67 P 8507b5a0d0e6434aaa7302085c7f54cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:16.909230 22238 ts_tablet_manager.cc:1434] T b86408eb7a2544ff921e6534948d5b67 P 8507b5a0d0e6434aaa7302085c7f54cb: Time spent starting tablet: real 0.034s	user 0.005s	sys 0.003s
I20260709 13:21:16.912037 22103 raft_consensus.cc:2468] T 74231e641a7e4f7fb5bc39a2a084bb81 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8507b5a0d0e6434aaa7302085c7f54cb in term 1.
I20260709 13:21:16.910930 22250 leader_election.cc:290] T 74231e641a7e4f7fb5bc39a2a084bb81 P 8507b5a0d0e6434aaa7302085c7f54cb [CANDIDATE]: Term 1 election: Requested vote from peers c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529), 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131:33869)
I20260709 13:21:16.913861 21917 leader_election.cc:304] T 74231e641a7e4f7fb5bc39a2a084bb81 P 8507b5a0d0e6434aaa7302085c7f54cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 21a7bb1d6ef54b20befdf8ea6d293776, 8507b5a0d0e6434aaa7302085c7f54cb; no voters: c48712233a8a4eb0b39bb05cd60137e6
I20260709 13:21:16.915184 22250 raft_consensus.cc:2804] T 74231e641a7e4f7fb5bc39a2a084bb81 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:16.915941 22250 raft_consensus.cc:697] T 74231e641a7e4f7fb5bc39a2a084bb81 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 LEADER]: Becoming Leader. State: Replica: 8507b5a0d0e6434aaa7302085c7f54cb, State: Running, Role: LEADER
I20260709 13:21:16.918269 22250 consensus_queue.cc:237] T 74231e641a7e4f7fb5bc39a2a084bb81 P 8507b5a0d0e6434aaa7302085c7f54cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:16.924388 22240 tablet_bootstrap.cc:492] T 74231e641a7e4f7fb5bc39a2a084bb81 P c48712233a8a4eb0b39bb05cd60137e6: No bootstrap required, opened a new log
I20260709 13:21:16.924115 22245 raft_consensus.cc:515] T b740d838804d4730b30e939a24d5414d P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:16.925189 22240 ts_tablet_manager.cc:1403] T 74231e641a7e4f7fb5bc39a2a084bb81 P c48712233a8a4eb0b39bb05cd60137e6: Time spent bootstrapping tablet: real 0.134s	user 0.034s	sys 0.025s
I20260709 13:21:16.928216 22103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b740d838804d4730b30e939a24d5414d" candidate_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776"
I20260709 13:21:16.928550 22028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b740d838804d4730b30e939a24d5414d" candidate_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6"
I20260709 13:21:16.928869 22103 raft_consensus.cc:3060] T b740d838804d4730b30e939a24d5414d P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:16.929132 22028 raft_consensus.cc:3060] T b740d838804d4730b30e939a24d5414d P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:16.929018 22240 raft_consensus.cc:359] T 74231e641a7e4f7fb5bc39a2a084bb81 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:16.930051 22240 raft_consensus.cc:385] T 74231e641a7e4f7fb5bc39a2a084bb81 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:16.930387 22240 raft_consensus.cc:740] T 74231e641a7e4f7fb5bc39a2a084bb81 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c48712233a8a4eb0b39bb05cd60137e6, State: Initialized, Role: FOLLOWER
I20260709 13:21:16.930521 22245 leader_election.cc:290] T b740d838804d4730b30e939a24d5414d P 8507b5a0d0e6434aaa7302085c7f54cb [CANDIDATE]: Term 1 election: Requested vote from peers c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529), 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131:33869)
I20260709 13:21:16.931838 22240 consensus_queue.cc:260] T 74231e641a7e4f7fb5bc39a2a084bb81 P c48712233a8a4eb0b39bb05cd60137e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:16.934651 22240 ts_tablet_manager.cc:1434] T 74231e641a7e4f7fb5bc39a2a084bb81 P c48712233a8a4eb0b39bb05cd60137e6: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 13:21:16.936980 22028 raft_consensus.cc:2468] T b740d838804d4730b30e939a24d5414d P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8507b5a0d0e6434aaa7302085c7f54cb in term 1.
I20260709 13:21:16.938160 21920 leader_election.cc:304] T b740d838804d4730b30e939a24d5414d P 8507b5a0d0e6434aaa7302085c7f54cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8507b5a0d0e6434aaa7302085c7f54cb, c48712233a8a4eb0b39bb05cd60137e6; no voters: 
I20260709 13:21:16.939065 22226 raft_consensus.cc:2804] T b740d838804d4730b30e939a24d5414d P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:16.940207 22103 raft_consensus.cc:2468] T b740d838804d4730b30e939a24d5414d P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8507b5a0d0e6434aaa7302085c7f54cb in term 1.
I20260709 13:21:16.945452 22240 tablet_bootstrap.cc:492] T b86408eb7a2544ff921e6534948d5b67 P c48712233a8a4eb0b39bb05cd60137e6: Bootstrap starting.
I20260709 13:21:16.945506 22226 raft_consensus.cc:697] T b740d838804d4730b30e939a24d5414d P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 LEADER]: Becoming Leader. State: Replica: 8507b5a0d0e6434aaa7302085c7f54cb, State: Running, Role: LEADER
I20260709 13:21:16.946152 22238 tablet_bootstrap.cc:492] T 455c56b836ff4e63949be17b0cf25c40 P 8507b5a0d0e6434aaa7302085c7f54cb: Bootstrap starting.
I20260709 13:21:16.948563 22239 tablet_bootstrap.cc:492] T b86408eb7a2544ff921e6534948d5b67 P 21a7bb1d6ef54b20befdf8ea6d293776: No bootstrap required, opened a new log
I20260709 13:21:16.951359 22239 ts_tablet_manager.cc:1403] T b86408eb7a2544ff921e6534948d5b67 P 21a7bb1d6ef54b20befdf8ea6d293776: Time spent bootstrapping tablet: real 0.084s	user 0.010s	sys 0.015s
I20260709 13:21:16.949807 22226 consensus_queue.cc:237] T b740d838804d4730b30e939a24d5414d P 8507b5a0d0e6434aaa7302085c7f54cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:16.954959 22239 raft_consensus.cc:359] T b86408eb7a2544ff921e6534948d5b67 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:16.955677 22239 raft_consensus.cc:385] T b86408eb7a2544ff921e6534948d5b67 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:16.955938 22239 raft_consensus.cc:740] T b86408eb7a2544ff921e6534948d5b67 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21a7bb1d6ef54b20befdf8ea6d293776, State: Initialized, Role: FOLLOWER
I20260709 13:21:16.956693 22239 consensus_queue.cc:260] T b86408eb7a2544ff921e6534948d5b67 P 21a7bb1d6ef54b20befdf8ea6d293776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:16.964179 22238 tablet_bootstrap.cc:654] T 455c56b836ff4e63949be17b0cf25c40 P 8507b5a0d0e6434aaa7302085c7f54cb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:16.966956 22240 tablet_bootstrap.cc:654] T b86408eb7a2544ff921e6534948d5b67 P c48712233a8a4eb0b39bb05cd60137e6: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:16.964450 22239 ts_tablet_manager.cc:1434] T b86408eb7a2544ff921e6534948d5b67 P 21a7bb1d6ef54b20befdf8ea6d293776: Time spent starting tablet: real 0.013s	user 0.002s	sys 0.004s
I20260709 13:21:16.970013 22239 tablet_bootstrap.cc:492] T 455c56b836ff4e63949be17b0cf25c40 P 21a7bb1d6ef54b20befdf8ea6d293776: Bootstrap starting.
I20260709 13:21:16.977209 22239 tablet_bootstrap.cc:654] T 455c56b836ff4e63949be17b0cf25c40 P 21a7bb1d6ef54b20befdf8ea6d293776: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:16.979615 22240 tablet_bootstrap.cc:492] T b86408eb7a2544ff921e6534948d5b67 P c48712233a8a4eb0b39bb05cd60137e6: No bootstrap required, opened a new log
I20260709 13:21:16.980276 22240 ts_tablet_manager.cc:1403] T b86408eb7a2544ff921e6534948d5b67 P c48712233a8a4eb0b39bb05cd60137e6: Time spent bootstrapping tablet: real 0.036s	user 0.007s	sys 0.008s
I20260709 13:21:16.982069 22238 tablet_bootstrap.cc:492] T 455c56b836ff4e63949be17b0cf25c40 P 8507b5a0d0e6434aaa7302085c7f54cb: No bootstrap required, opened a new log
I20260709 13:21:16.982663 22238 ts_tablet_manager.cc:1403] T 455c56b836ff4e63949be17b0cf25c40 P 8507b5a0d0e6434aaa7302085c7f54cb: Time spent bootstrapping tablet: real 0.037s	user 0.009s	sys 0.007s
I20260709 13:21:16.983114 22240 raft_consensus.cc:359] T b86408eb7a2544ff921e6534948d5b67 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:16.984112 22240 raft_consensus.cc:385] T b86408eb7a2544ff921e6534948d5b67 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:16.983114 21849 catalog_manager.cc:5697] T 74231e641a7e4f7fb5bc39a2a084bb81 P 8507b5a0d0e6434aaa7302085c7f54cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129). New cstate: current_term: 1 leader_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:16.984519 22240 raft_consensus.cc:740] T b86408eb7a2544ff921e6534948d5b67 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c48712233a8a4eb0b39bb05cd60137e6, State: Initialized, Role: FOLLOWER
I20260709 13:21:16.985713 22240 consensus_queue.cc:260] T b86408eb7a2544ff921e6534948d5b67 P c48712233a8a4eb0b39bb05cd60137e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:16.985981 22238 raft_consensus.cc:359] T 455c56b836ff4e63949be17b0cf25c40 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:16.987026 22238 raft_consensus.cc:385] T 455c56b836ff4e63949be17b0cf25c40 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:16.986033 21849 catalog_manager.cc:5697] T b740d838804d4730b30e939a24d5414d P 8507b5a0d0e6434aaa7302085c7f54cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129). New cstate: current_term: 1 leader_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:16.987398 22238 raft_consensus.cc:740] T 455c56b836ff4e63949be17b0cf25c40 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8507b5a0d0e6434aaa7302085c7f54cb, State: Initialized, Role: FOLLOWER
I20260709 13:21:16.988282 22238 consensus_queue.cc:260] T 455c56b836ff4e63949be17b0cf25c40 P 8507b5a0d0e6434aaa7302085c7f54cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:16.990926 22238 ts_tablet_manager.cc:1434] T 455c56b836ff4e63949be17b0cf25c40 P 8507b5a0d0e6434aaa7302085c7f54cb: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
I20260709 13:21:16.992097 22238 tablet_bootstrap.cc:492] T 91f78fe291144d05bce820bf9ee9cc03 P 8507b5a0d0e6434aaa7302085c7f54cb: Bootstrap starting.
I20260709 13:21:16.995074 22239 tablet_bootstrap.cc:492] T 455c56b836ff4e63949be17b0cf25c40 P 21a7bb1d6ef54b20befdf8ea6d293776: No bootstrap required, opened a new log
I20260709 13:21:16.995797 22239 ts_tablet_manager.cc:1403] T 455c56b836ff4e63949be17b0cf25c40 P 21a7bb1d6ef54b20befdf8ea6d293776: Time spent bootstrapping tablet: real 0.026s	user 0.007s	sys 0.013s
I20260709 13:21:16.997367 22240 ts_tablet_manager.cc:1434] T b86408eb7a2544ff921e6534948d5b67 P c48712233a8a4eb0b39bb05cd60137e6: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 13:21:16.998677 22240 tablet_bootstrap.cc:492] T 91f78fe291144d05bce820bf9ee9cc03 P c48712233a8a4eb0b39bb05cd60137e6: Bootstrap starting.
I20260709 13:21:16.998935 22238 tablet_bootstrap.cc:654] T 91f78fe291144d05bce820bf9ee9cc03 P 8507b5a0d0e6434aaa7302085c7f54cb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:16.999397 22239 raft_consensus.cc:359] T 455c56b836ff4e63949be17b0cf25c40 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.000442 22239 raft_consensus.cc:385] T 455c56b836ff4e63949be17b0cf25c40 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:17.000824 22239 raft_consensus.cc:740] T 455c56b836ff4e63949be17b0cf25c40 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21a7bb1d6ef54b20befdf8ea6d293776, State: Initialized, Role: FOLLOWER
I20260709 13:21:17.001888 22239 consensus_queue.cc:260] T 455c56b836ff4e63949be17b0cf25c40 P 21a7bb1d6ef54b20befdf8ea6d293776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.007586 22240 tablet_bootstrap.cc:654] T 91f78fe291144d05bce820bf9ee9cc03 P c48712233a8a4eb0b39bb05cd60137e6: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:17.007751 22239 ts_tablet_manager.cc:1434] T 455c56b836ff4e63949be17b0cf25c40 P 21a7bb1d6ef54b20befdf8ea6d293776: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.002s
I20260709 13:21:17.009001 22239 tablet_bootstrap.cc:492] T 400f2b2a8e114d5d9506446a360c4daf P 21a7bb1d6ef54b20befdf8ea6d293776: Bootstrap starting.
I20260709 13:21:17.016568 22239 tablet_bootstrap.cc:654] T 400f2b2a8e114d5d9506446a360c4daf P 21a7bb1d6ef54b20befdf8ea6d293776: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:17.022636 22240 tablet_bootstrap.cc:492] T 91f78fe291144d05bce820bf9ee9cc03 P c48712233a8a4eb0b39bb05cd60137e6: No bootstrap required, opened a new log
I20260709 13:21:17.019788 22238 tablet_bootstrap.cc:492] T 91f78fe291144d05bce820bf9ee9cc03 P 8507b5a0d0e6434aaa7302085c7f54cb: No bootstrap required, opened a new log
I20260709 13:21:17.023257 22225 raft_consensus.cc:493] T 74231e641a7e4f7fb5bc39a2a084bb81 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:17.023643 22238 ts_tablet_manager.cc:1403] T 91f78fe291144d05bce820bf9ee9cc03 P 8507b5a0d0e6434aaa7302085c7f54cb: Time spent bootstrapping tablet: real 0.032s	user 0.015s	sys 0.005s
I20260709 13:21:17.023837 22225 raft_consensus.cc:515] T 74231e641a7e4f7fb5bc39a2a084bb81 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.025429 22240 ts_tablet_manager.cc:1403] T 91f78fe291144d05bce820bf9ee9cc03 P c48712233a8a4eb0b39bb05cd60137e6: Time spent bootstrapping tablet: real 0.027s	user 0.011s	sys 0.008s
I20260709 13:21:17.027233 22239 tablet_bootstrap.cc:492] T 400f2b2a8e114d5d9506446a360c4daf P 21a7bb1d6ef54b20befdf8ea6d293776: No bootstrap required, opened a new log
I20260709 13:21:17.027820 22239 ts_tablet_manager.cc:1403] T 400f2b2a8e114d5d9506446a360c4daf P 21a7bb1d6ef54b20befdf8ea6d293776: Time spent bootstrapping tablet: real 0.019s	user 0.014s	sys 0.000s
I20260709 13:21:17.027607 22238 raft_consensus.cc:359] T 91f78fe291144d05bce820bf9ee9cc03 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.028553 22238 raft_consensus.cc:385] T 91f78fe291144d05bce820bf9ee9cc03 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:17.028932 22238 raft_consensus.cc:740] T 91f78fe291144d05bce820bf9ee9cc03 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8507b5a0d0e6434aaa7302085c7f54cb, State: Initialized, Role: FOLLOWER
I20260709 13:21:17.028769 22240 raft_consensus.cc:359] T 91f78fe291144d05bce820bf9ee9cc03 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.029628 22240 raft_consensus.cc:385] T 91f78fe291144d05bce820bf9ee9cc03 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:17.029981 22240 raft_consensus.cc:740] T 91f78fe291144d05bce820bf9ee9cc03 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c48712233a8a4eb0b39bb05cd60137e6, State: Initialized, Role: FOLLOWER
I20260709 13:21:17.029915 22238 consensus_queue.cc:260] T 91f78fe291144d05bce820bf9ee9cc03 P 8507b5a0d0e6434aaa7302085c7f54cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.031018 22239 raft_consensus.cc:359] T 400f2b2a8e114d5d9506446a360c4daf P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.031134 22240 consensus_queue.cc:260] T 91f78fe291144d05bce820bf9ee9cc03 P c48712233a8a4eb0b39bb05cd60137e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.031944 22239 raft_consensus.cc:385] T 400f2b2a8e114d5d9506446a360c4daf P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:17.032337 22239 raft_consensus.cc:740] T 400f2b2a8e114d5d9506446a360c4daf P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21a7bb1d6ef54b20befdf8ea6d293776, State: Initialized, Role: FOLLOWER
I20260709 13:21:17.033071 22238 ts_tablet_manager.cc:1434] T 91f78fe291144d05bce820bf9ee9cc03 P 8507b5a0d0e6434aaa7302085c7f54cb: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:21:17.033202 22239 consensus_queue.cc:260] T 400f2b2a8e114d5d9506446a360c4daf P 21a7bb1d6ef54b20befdf8ea6d293776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.034830 22238 tablet_bootstrap.cc:492] T 300ddb6928744c789723f712d6f9780e P 8507b5a0d0e6434aaa7302085c7f54cb: Bootstrap starting.
I20260709 13:21:17.035858 22239 ts_tablet_manager.cc:1434] T 400f2b2a8e114d5d9506446a360c4daf P 21a7bb1d6ef54b20befdf8ea6d293776: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:21:17.035773 21953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74231e641a7e4f7fb5bc39a2a084bb81" candidate_uuid: "c48712233a8a4eb0b39bb05cd60137e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" is_pre_election: true
I20260709 13:21:17.037076 22239 tablet_bootstrap.cc:492] T 300ddb6928744c789723f712d6f9780e P 21a7bb1d6ef54b20befdf8ea6d293776: Bootstrap starting.
I20260709 13:21:17.044404 22238 tablet_bootstrap.cc:654] T 300ddb6928744c789723f712d6f9780e P 8507b5a0d0e6434aaa7302085c7f54cb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:17.044600 22239 tablet_bootstrap.cc:654] T 300ddb6928744c789723f712d6f9780e P 21a7bb1d6ef54b20befdf8ea6d293776: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:17.047606 22225 leader_election.cc:290] T 74231e641a7e4f7fb5bc39a2a084bb81 P c48712233a8a4eb0b39bb05cd60137e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591), 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131:33869)
I20260709 13:21:17.048734 22103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74231e641a7e4f7fb5bc39a2a084bb81" candidate_uuid: "c48712233a8a4eb0b39bb05cd60137e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" is_pre_election: true
I20260709 13:21:17.049511 22103 raft_consensus.cc:2393] T 74231e641a7e4f7fb5bc39a2a084bb81 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c48712233a8a4eb0b39bb05cd60137e6 in current term 1: Already voted for candidate 8507b5a0d0e6434aaa7302085c7f54cb in this term.
I20260709 13:21:17.050846 21992 leader_election.cc:304] T 74231e641a7e4f7fb5bc39a2a084bb81 P c48712233a8a4eb0b39bb05cd60137e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c48712233a8a4eb0b39bb05cd60137e6; no voters: 21a7bb1d6ef54b20befdf8ea6d293776, 8507b5a0d0e6434aaa7302085c7f54cb
I20260709 13:21:17.052142 22225 raft_consensus.cc:493] T 400f2b2a8e114d5d9506446a360c4daf P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:17.055003 22225 raft_consensus.cc:515] T 400f2b2a8e114d5d9506446a360c4daf P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.056097 22261 raft_consensus.cc:3060] T 74231e641a7e4f7fb5bc39a2a084bb81 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:17.058305 21953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "400f2b2a8e114d5d9506446a360c4daf" candidate_uuid: "c48712233a8a4eb0b39bb05cd60137e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" is_pre_election: true
W20260709 13:21:17.060070 21995 leader_election.cc:343] T 400f2b2a8e114d5d9506446a360c4daf P c48712233a8a4eb0b39bb05cd60137e6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:17.060397 22238 tablet_bootstrap.cc:492] T 300ddb6928744c789723f712d6f9780e P 8507b5a0d0e6434aaa7302085c7f54cb: No bootstrap required, opened a new log
I20260709 13:21:17.061089 22238 ts_tablet_manager.cc:1403] T 300ddb6928744c789723f712d6f9780e P 8507b5a0d0e6434aaa7302085c7f54cb: Time spent bootstrapping tablet: real 0.027s	user 0.013s	sys 0.004s
I20260709 13:21:17.063078 22226 raft_consensus.cc:493] T 91f78fe291144d05bce820bf9ee9cc03 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:17.063657 22226 raft_consensus.cc:515] T 91f78fe291144d05bce820bf9ee9cc03 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.065018 22225 leader_election.cc:290] T 400f2b2a8e114d5d9506446a360c4daf P c48712233a8a4eb0b39bb05cd60137e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591), 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131:33869)
I20260709 13:21:17.065852 22226 leader_election.cc:290] T 91f78fe291144d05bce820bf9ee9cc03 P 8507b5a0d0e6434aaa7302085c7f54cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529), 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131:33869)
I20260709 13:21:17.069252 22239 tablet_bootstrap.cc:492] T 300ddb6928744c789723f712d6f9780e P 21a7bb1d6ef54b20befdf8ea6d293776: No bootstrap required, opened a new log
I20260709 13:21:17.069888 22239 ts_tablet_manager.cc:1403] T 300ddb6928744c789723f712d6f9780e P 21a7bb1d6ef54b20befdf8ea6d293776: Time spent bootstrapping tablet: real 0.033s	user 0.006s	sys 0.014s
I20260709 13:21:17.066346 22261 raft_consensus.cc:2749] T 74231e641a7e4f7fb5bc39a2a084bb81 P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:17.047425 22240 ts_tablet_manager.cc:1434] T 91f78fe291144d05bce820bf9ee9cc03 P c48712233a8a4eb0b39bb05cd60137e6: Time spent starting tablet: real 0.022s	user 0.013s	sys 0.002s
I20260709 13:21:17.066623 22028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91f78fe291144d05bce820bf9ee9cc03" candidate_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6" is_pre_election: true
I20260709 13:21:17.071146 22028 raft_consensus.cc:2468] T 91f78fe291144d05bce820bf9ee9cc03 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8507b5a0d0e6434aaa7302085c7f54cb in term 0.
I20260709 13:21:17.066206 22238 raft_consensus.cc:359] T 300ddb6928744c789723f712d6f9780e P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.067515 22103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91f78fe291144d05bce820bf9ee9cc03" candidate_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" is_pre_election: true
I20260709 13:21:17.072063 22238 raft_consensus.cc:385] T 300ddb6928744c789723f712d6f9780e P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:17.071906 22102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "400f2b2a8e114d5d9506446a360c4daf" candidate_uuid: "c48712233a8a4eb0b39bb05cd60137e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" is_pre_election: true
I20260709 13:21:17.072465 22238 raft_consensus.cc:740] T 300ddb6928744c789723f712d6f9780e P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8507b5a0d0e6434aaa7302085c7f54cb, State: Initialized, Role: FOLLOWER
I20260709 13:21:17.072687 22102 raft_consensus.cc:2468] T 400f2b2a8e114d5d9506446a360c4daf P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c48712233a8a4eb0b39bb05cd60137e6 in term 0.
I20260709 13:21:17.071713 22240 tablet_bootstrap.cc:492] T 455c56b836ff4e63949be17b0cf25c40 P c48712233a8a4eb0b39bb05cd60137e6: Bootstrap starting.
W20260709 13:21:17.073098 21917 leader_election.cc:343] T 91f78fe291144d05bce820bf9ee9cc03 P 8507b5a0d0e6434aaa7302085c7f54cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131:33869): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:17.073263 22238 consensus_queue.cc:260] T 300ddb6928744c789723f712d6f9780e P 8507b5a0d0e6434aaa7302085c7f54cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.073161 22239 raft_consensus.cc:359] T 300ddb6928744c789723f712d6f9780e P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.075017 22239 raft_consensus.cc:385] T 300ddb6928744c789723f712d6f9780e P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:17.075387 22239 raft_consensus.cc:740] T 300ddb6928744c789723f712d6f9780e P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21a7bb1d6ef54b20befdf8ea6d293776, State: Initialized, Role: FOLLOWER
I20260709 13:21:17.075960 22238 ts_tablet_manager.cc:1434] T 300ddb6928744c789723f712d6f9780e P 8507b5a0d0e6434aaa7302085c7f54cb: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.000s
I20260709 13:21:17.076262 22239 consensus_queue.cc:260] T 300ddb6928744c789723f712d6f9780e P 21a7bb1d6ef54b20befdf8ea6d293776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.077396 22238 tablet_bootstrap.cc:492] T 400f2b2a8e114d5d9506446a360c4daf P 8507b5a0d0e6434aaa7302085c7f54cb: Bootstrap starting.
I20260709 13:21:17.074457 21920 leader_election.cc:304] T 91f78fe291144d05bce820bf9ee9cc03 P 8507b5a0d0e6434aaa7302085c7f54cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8507b5a0d0e6434aaa7302085c7f54cb, c48712233a8a4eb0b39bb05cd60137e6; no voters: 21a7bb1d6ef54b20befdf8ea6d293776
I20260709 13:21:17.078966 22239 ts_tablet_manager.cc:1434] T 300ddb6928744c789723f712d6f9780e P 21a7bb1d6ef54b20befdf8ea6d293776: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:21:17.080111 22239 tablet_bootstrap.cc:492] T 91f78fe291144d05bce820bf9ee9cc03 P 21a7bb1d6ef54b20befdf8ea6d293776: Bootstrap starting.
I20260709 13:21:17.082453 22226 raft_consensus.cc:2804] T 91f78fe291144d05bce820bf9ee9cc03 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:17.083062 22226 raft_consensus.cc:493] T 91f78fe291144d05bce820bf9ee9cc03 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:17.083487 22226 raft_consensus.cc:3060] T 91f78fe291144d05bce820bf9ee9cc03 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:17.085034 22238 tablet_bootstrap.cc:654] T 400f2b2a8e114d5d9506446a360c4daf P 8507b5a0d0e6434aaa7302085c7f54cb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:17.086578 22239 tablet_bootstrap.cc:654] T 91f78fe291144d05bce820bf9ee9cc03 P 21a7bb1d6ef54b20befdf8ea6d293776: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:17.087576 21992 leader_election.cc:304] T 400f2b2a8e114d5d9506446a360c4daf P c48712233a8a4eb0b39bb05cd60137e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 21a7bb1d6ef54b20befdf8ea6d293776, c48712233a8a4eb0b39bb05cd60137e6; no voters: 8507b5a0d0e6434aaa7302085c7f54cb
I20260709 13:21:17.101172 22240 tablet_bootstrap.cc:654] T 455c56b836ff4e63949be17b0cf25c40 P c48712233a8a4eb0b39bb05cd60137e6: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:17.104308 22261 raft_consensus.cc:2804] T 400f2b2a8e114d5d9506446a360c4daf P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:17.104763 22261 raft_consensus.cc:493] T 400f2b2a8e114d5d9506446a360c4daf P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:17.105159 22261 raft_consensus.cc:3060] T 400f2b2a8e114d5d9506446a360c4daf P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:17.107764 22239 tablet_bootstrap.cc:492] T 91f78fe291144d05bce820bf9ee9cc03 P 21a7bb1d6ef54b20befdf8ea6d293776: No bootstrap required, opened a new log
I20260709 13:21:17.109732 22239 ts_tablet_manager.cc:1403] T 91f78fe291144d05bce820bf9ee9cc03 P 21a7bb1d6ef54b20befdf8ea6d293776: Time spent bootstrapping tablet: real 0.030s	user 0.014s	sys 0.009s
I20260709 13:21:17.113408 22239 raft_consensus.cc:359] T 91f78fe291144d05bce820bf9ee9cc03 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.117708 22239 raft_consensus.cc:385] T 91f78fe291144d05bce820bf9ee9cc03 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:17.118521 22239 raft_consensus.cc:740] T 91f78fe291144d05bce820bf9ee9cc03 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21a7bb1d6ef54b20befdf8ea6d293776, State: Initialized, Role: FOLLOWER
I20260709 13:21:17.114760 22261 raft_consensus.cc:515] T 400f2b2a8e114d5d9506446a360c4daf P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.120152 22239 consensus_queue.cc:260] T 91f78fe291144d05bce820bf9ee9cc03 P 21a7bb1d6ef54b20befdf8ea6d293776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.117877 22226 raft_consensus.cc:515] T 91f78fe291144d05bce820bf9ee9cc03 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.123416 22240 tablet_bootstrap.cc:492] T 455c56b836ff4e63949be17b0cf25c40 P c48712233a8a4eb0b39bb05cd60137e6: No bootstrap required, opened a new log
I20260709 13:21:17.122959 21953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "400f2b2a8e114d5d9506446a360c4daf" candidate_uuid: "c48712233a8a4eb0b39bb05cd60137e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb"
I20260709 13:21:17.126719 22240 ts_tablet_manager.cc:1403] T 455c56b836ff4e63949be17b0cf25c40 P c48712233a8a4eb0b39bb05cd60137e6: Time spent bootstrapping tablet: real 0.055s	user 0.021s	sys 0.000s
I20260709 13:21:17.127097 22103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91f78fe291144d05bce820bf9ee9cc03" candidate_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776"
W20260709 13:21:17.127599 21995 leader_election.cc:343] T 400f2b2a8e114d5d9506446a360c4daf P c48712233a8a4eb0b39bb05cd60137e6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:17.127732 22103 raft_consensus.cc:3060] T 91f78fe291144d05bce820bf9ee9cc03 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:17.122145 22261 leader_election.cc:290] T 400f2b2a8e114d5d9506446a360c4daf P c48712233a8a4eb0b39bb05cd60137e6 [CANDIDATE]: Term 1 election: Requested vote from peers 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591), 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131:33869)
I20260709 13:21:17.128502 22239 ts_tablet_manager.cc:1434] T 91f78fe291144d05bce820bf9ee9cc03 P 21a7bb1d6ef54b20befdf8ea6d293776: Time spent starting tablet: real 0.018s	user 0.000s	sys 0.006s
I20260709 13:21:17.129645 22239 tablet_bootstrap.cc:492] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776: Bootstrap starting.
I20260709 13:21:17.129990 22240 raft_consensus.cc:359] T 455c56b836ff4e63949be17b0cf25c40 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.130751 22240 raft_consensus.cc:385] T 455c56b836ff4e63949be17b0cf25c40 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:17.131058 22240 raft_consensus.cc:740] T 455c56b836ff4e63949be17b0cf25c40 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c48712233a8a4eb0b39bb05cd60137e6, State: Initialized, Role: FOLLOWER
I20260709 13:21:17.125624 22028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91f78fe291144d05bce820bf9ee9cc03" candidate_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6"
I20260709 13:21:17.131680 22238 tablet_bootstrap.cc:492] T 400f2b2a8e114d5d9506446a360c4daf P 8507b5a0d0e6434aaa7302085c7f54cb: No bootstrap required, opened a new log
I20260709 13:21:17.132020 22028 raft_consensus.cc:3060] T 91f78fe291144d05bce820bf9ee9cc03 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:17.132242 22238 ts_tablet_manager.cc:1403] T 400f2b2a8e114d5d9506446a360c4daf P 8507b5a0d0e6434aaa7302085c7f54cb: Time spent bootstrapping tablet: real 0.055s	user 0.037s	sys 0.012s
I20260709 13:21:17.131958 22240 consensus_queue.cc:260] T 455c56b836ff4e63949be17b0cf25c40 P c48712233a8a4eb0b39bb05cd60137e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.125433 22102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "400f2b2a8e114d5d9506446a360c4daf" candidate_uuid: "c48712233a8a4eb0b39bb05cd60137e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776"
I20260709 13:21:17.134312 22102 raft_consensus.cc:3060] T 400f2b2a8e114d5d9506446a360c4daf P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:17.137025 22240 ts_tablet_manager.cc:1434] T 455c56b836ff4e63949be17b0cf25c40 P c48712233a8a4eb0b39bb05cd60137e6: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 13:21:17.138192 22240 tablet_bootstrap.cc:492] T 300ddb6928744c789723f712d6f9780e P c48712233a8a4eb0b39bb05cd60137e6: Bootstrap starting.
I20260709 13:21:17.138978 22239 tablet_bootstrap.cc:654] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:17.139246 22238 raft_consensus.cc:359] T 400f2b2a8e114d5d9506446a360c4daf P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.140132 22238 raft_consensus.cc:385] T 400f2b2a8e114d5d9506446a360c4daf P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:17.140496 22238 raft_consensus.cc:740] T 400f2b2a8e114d5d9506446a360c4daf P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8507b5a0d0e6434aaa7302085c7f54cb, State: Initialized, Role: FOLLOWER
I20260709 13:21:17.141316 22238 consensus_queue.cc:260] T 400f2b2a8e114d5d9506446a360c4daf P 8507b5a0d0e6434aaa7302085c7f54cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.144029 22238 ts_tablet_manager.cc:1434] T 400f2b2a8e114d5d9506446a360c4daf P 8507b5a0d0e6434aaa7302085c7f54cb: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.001s
I20260709 13:21:17.145512 22238 tablet_bootstrap.cc:492] T d481ac177e6345efa62e86898d48fbac P 8507b5a0d0e6434aaa7302085c7f54cb: Bootstrap starting.
I20260709 13:21:17.150714 22227 raft_consensus.cc:493] T 455c56b836ff4e63949be17b0cf25c40 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:17.158500 22238 tablet_bootstrap.cc:654] T d481ac177e6345efa62e86898d48fbac P 8507b5a0d0e6434aaa7302085c7f54cb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:17.158864 22227 raft_consensus.cc:515] T 455c56b836ff4e63949be17b0cf25c40 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.161254 22227 leader_election.cc:290] T 455c56b836ff4e63949be17b0cf25c40 P 21a7bb1d6ef54b20befdf8ea6d293776 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591), c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529)
I20260709 13:21:17.152282 22239 tablet_bootstrap.cc:492] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776: No bootstrap required, opened a new log
I20260709 13:21:17.162367 22239 ts_tablet_manager.cc:1403] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776: Time spent bootstrapping tablet: real 0.033s	user 0.012s	sys 0.007s
I20260709 13:21:17.165084 21953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "455c56b836ff4e63949be17b0cf25c40" candidate_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" is_pre_election: true
I20260709 13:21:17.165889 21953 raft_consensus.cc:2468] T 455c56b836ff4e63949be17b0cf25c40 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21a7bb1d6ef54b20befdf8ea6d293776 in term 0.
I20260709 13:21:17.166479 22239 raft_consensus.cc:359] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.167450 22239 raft_consensus.cc:385] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:17.167709 22239 raft_consensus.cc:740] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21a7bb1d6ef54b20befdf8ea6d293776, State: Initialized, Role: FOLLOWER
I20260709 13:21:17.167958 22028 raft_consensus.cc:2468] T 91f78fe291144d05bce820bf9ee9cc03 P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8507b5a0d0e6434aaa7302085c7f54cb in term 1.
I20260709 13:21:17.168260 22029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "455c56b836ff4e63949be17b0cf25c40" candidate_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6" is_pre_election: true
I20260709 13:21:17.168383 22239 consensus_queue.cc:260] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.168944 22029 raft_consensus.cc:2468] T 455c56b836ff4e63949be17b0cf25c40 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21a7bb1d6ef54b20befdf8ea6d293776 in term 0.
I20260709 13:21:17.170209 22240 tablet_bootstrap.cc:654] T 300ddb6928744c789723f712d6f9780e P c48712233a8a4eb0b39bb05cd60137e6: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:17.170837 22102 raft_consensus.cc:2468] T 400f2b2a8e114d5d9506446a360c4daf P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c48712233a8a4eb0b39bb05cd60137e6 in term 1.
I20260709 13:21:17.171975 21920 leader_election.cc:304] T 91f78fe291144d05bce820bf9ee9cc03 P 8507b5a0d0e6434aaa7302085c7f54cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8507b5a0d0e6434aaa7302085c7f54cb, c48712233a8a4eb0b39bb05cd60137e6; no voters: 
I20260709 13:21:17.173087 22250 raft_consensus.cc:2804] T 91f78fe291144d05bce820bf9ee9cc03 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:17.125916 22226 leader_election.cc:290] T 91f78fe291144d05bce820bf9ee9cc03 P 8507b5a0d0e6434aaa7302085c7f54cb [CANDIDATE]: Term 1 election: Requested vote from peers c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529), 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131:33869)
I20260709 13:21:17.173646 22250 raft_consensus.cc:697] T 91f78fe291144d05bce820bf9ee9cc03 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 LEADER]: Becoming Leader. State: Replica: 8507b5a0d0e6434aaa7302085c7f54cb, State: Running, Role: LEADER
I20260709 13:21:17.172012 21992 leader_election.cc:304] T 400f2b2a8e114d5d9506446a360c4daf P c48712233a8a4eb0b39bb05cd60137e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 21a7bb1d6ef54b20befdf8ea6d293776, c48712233a8a4eb0b39bb05cd60137e6; no voters: 8507b5a0d0e6434aaa7302085c7f54cb
I20260709 13:21:17.175899 22239 ts_tablet_manager.cc:1434] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.002s
I20260709 13:21:17.174582 22250 consensus_queue.cc:237] T 91f78fe291144d05bce820bf9ee9cc03 P 8507b5a0d0e6434aaa7302085c7f54cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.173425 22103 raft_consensus.cc:2468] T 91f78fe291144d05bce820bf9ee9cc03 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8507b5a0d0e6434aaa7302085c7f54cb in term 1.
I20260709 13:21:17.182694 22239 tablet_bootstrap.cc:492] T d481ac177e6345efa62e86898d48fbac P 21a7bb1d6ef54b20befdf8ea6d293776: Bootstrap starting.
I20260709 13:21:17.175658 22069 leader_election.cc:304] T 455c56b836ff4e63949be17b0cf25c40 P 21a7bb1d6ef54b20befdf8ea6d293776 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a7bb1d6ef54b20befdf8ea6d293776, c48712233a8a4eb0b39bb05cd60137e6; no voters: 
I20260709 13:21:17.186292 22227 raft_consensus.cc:2804] T 455c56b836ff4e63949be17b0cf25c40 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:17.186669 22227 raft_consensus.cc:493] T 455c56b836ff4e63949be17b0cf25c40 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:17.187686 22238 tablet_bootstrap.cc:492] T d481ac177e6345efa62e86898d48fbac P 8507b5a0d0e6434aaa7302085c7f54cb: No bootstrap required, opened a new log
I20260709 13:21:17.188259 22238 ts_tablet_manager.cc:1403] T d481ac177e6345efa62e86898d48fbac P 8507b5a0d0e6434aaa7302085c7f54cb: Time spent bootstrapping tablet: real 0.043s	user 0.016s	sys 0.021s
I20260709 13:21:17.187016 22227 raft_consensus.cc:3060] T 455c56b836ff4e63949be17b0cf25c40 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:17.189848 22239 tablet_bootstrap.cc:654] T d481ac177e6345efa62e86898d48fbac P 21a7bb1d6ef54b20befdf8ea6d293776: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:17.189769 22261 raft_consensus.cc:2804] T 400f2b2a8e114d5d9506446a360c4daf P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:17.190388 22261 raft_consensus.cc:697] T 400f2b2a8e114d5d9506446a360c4daf P c48712233a8a4eb0b39bb05cd60137e6 [term 1 LEADER]: Becoming Leader. State: Replica: c48712233a8a4eb0b39bb05cd60137e6, State: Running, Role: LEADER
I20260709 13:21:17.191290 22261 consensus_queue.cc:237] T 400f2b2a8e114d5d9506446a360c4daf P c48712233a8a4eb0b39bb05cd60137e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.192584 22238 raft_consensus.cc:359] T d481ac177e6345efa62e86898d48fbac P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.193316 22238 raft_consensus.cc:385] T d481ac177e6345efa62e86898d48fbac P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:17.193573 22238 raft_consensus.cc:740] T d481ac177e6345efa62e86898d48fbac P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8507b5a0d0e6434aaa7302085c7f54cb, State: Initialized, Role: FOLLOWER
I20260709 13:21:17.194226 22238 consensus_queue.cc:260] T d481ac177e6345efa62e86898d48fbac P 8507b5a0d0e6434aaa7302085c7f54cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.199728 22238 ts_tablet_manager.cc:1434] T d481ac177e6345efa62e86898d48fbac P 8507b5a0d0e6434aaa7302085c7f54cb: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:21:17.201043 22238 tablet_bootstrap.cc:492] T dfb67aaa32244d76a341136766820549 P 8507b5a0d0e6434aaa7302085c7f54cb: Bootstrap starting.
I20260709 13:21:17.200103 22227 raft_consensus.cc:515] T 455c56b836ff4e63949be17b0cf25c40 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.217157 21953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "455c56b836ff4e63949be17b0cf25c40" candidate_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb"
I20260709 13:21:17.217569 22238 tablet_bootstrap.cc:654] T dfb67aaa32244d76a341136766820549 P 8507b5a0d0e6434aaa7302085c7f54cb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:17.218070 21953 raft_consensus.cc:3060] T 455c56b836ff4e63949be17b0cf25c40 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:17.218757 22239 tablet_bootstrap.cc:492] T d481ac177e6345efa62e86898d48fbac P 21a7bb1d6ef54b20befdf8ea6d293776: No bootstrap required, opened a new log
I20260709 13:21:17.219347 22239 ts_tablet_manager.cc:1403] T d481ac177e6345efa62e86898d48fbac P 21a7bb1d6ef54b20befdf8ea6d293776: Time spent bootstrapping tablet: real 0.037s	user 0.007s	sys 0.006s
I20260709 13:21:17.219599 22227 leader_election.cc:290] T 455c56b836ff4e63949be17b0cf25c40 P 21a7bb1d6ef54b20befdf8ea6d293776 [CANDIDATE]: Term 1 election: Requested vote from peers 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591), c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529)
I20260709 13:21:17.220890 22029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "455c56b836ff4e63949be17b0cf25c40" candidate_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6"
I20260709 13:21:17.221642 22029 raft_consensus.cc:3060] T 455c56b836ff4e63949be17b0cf25c40 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:17.222924 22239 raft_consensus.cc:359] T d481ac177e6345efa62e86898d48fbac P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.223974 22239 raft_consensus.cc:385] T d481ac177e6345efa62e86898d48fbac P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:17.224404 22239 raft_consensus.cc:740] T d481ac177e6345efa62e86898d48fbac P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21a7bb1d6ef54b20befdf8ea6d293776, State: Initialized, Role: FOLLOWER
I20260709 13:21:17.225360 22239 consensus_queue.cc:260] T d481ac177e6345efa62e86898d48fbac P 21a7bb1d6ef54b20befdf8ea6d293776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.226639 22240 tablet_bootstrap.cc:492] T 300ddb6928744c789723f712d6f9780e P c48712233a8a4eb0b39bb05cd60137e6: No bootstrap required, opened a new log
I20260709 13:21:17.227250 22240 ts_tablet_manager.cc:1403] T 300ddb6928744c789723f712d6f9780e P c48712233a8a4eb0b39bb05cd60137e6: Time spent bootstrapping tablet: real 0.089s	user 0.025s	sys 0.009s
I20260709 13:21:17.228201 22239 ts_tablet_manager.cc:1434] T d481ac177e6345efa62e86898d48fbac P 21a7bb1d6ef54b20befdf8ea6d293776: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:21:17.230309 22240 raft_consensus.cc:359] T 300ddb6928744c789723f712d6f9780e P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.231178 22240 raft_consensus.cc:385] T 300ddb6928744c789723f712d6f9780e P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:17.231530 22240 raft_consensus.cc:740] T 300ddb6928744c789723f712d6f9780e P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c48712233a8a4eb0b39bb05cd60137e6, State: Initialized, Role: FOLLOWER
I20260709 13:21:17.232163 22238 tablet_bootstrap.cc:492] T dfb67aaa32244d76a341136766820549 P 8507b5a0d0e6434aaa7302085c7f54cb: No bootstrap required, opened a new log
I20260709 13:21:17.232761 22238 ts_tablet_manager.cc:1403] T dfb67aaa32244d76a341136766820549 P 8507b5a0d0e6434aaa7302085c7f54cb: Time spent bootstrapping tablet: real 0.032s	user 0.007s	sys 0.007s
I20260709 13:21:17.232323 22240 consensus_queue.cc:260] T 300ddb6928744c789723f712d6f9780e P c48712233a8a4eb0b39bb05cd60137e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.235086 22240 ts_tablet_manager.cc:1434] T 300ddb6928744c789723f712d6f9780e P c48712233a8a4eb0b39bb05cd60137e6: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.002s
I20260709 13:21:17.235659 22238 raft_consensus.cc:359] T dfb67aaa32244d76a341136766820549 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.236181 22240 tablet_bootstrap.cc:492] T d481ac177e6345efa62e86898d48fbac P c48712233a8a4eb0b39bb05cd60137e6: Bootstrap starting.
I20260709 13:21:17.236466 22238 raft_consensus.cc:385] T dfb67aaa32244d76a341136766820549 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:17.236809 22238 raft_consensus.cc:740] T dfb67aaa32244d76a341136766820549 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8507b5a0d0e6434aaa7302085c7f54cb, State: Initialized, Role: FOLLOWER
I20260709 13:21:17.237530 22238 consensus_queue.cc:260] T dfb67aaa32244d76a341136766820549 P 8507b5a0d0e6434aaa7302085c7f54cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.239684 22238 ts_tablet_manager.cc:1434] T dfb67aaa32244d76a341136766820549 P 8507b5a0d0e6434aaa7302085c7f54cb: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:21:17.241102 22029 raft_consensus.cc:2468] T 455c56b836ff4e63949be17b0cf25c40 P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21a7bb1d6ef54b20befdf8ea6d293776 in term 1.
I20260709 13:21:17.241418 21953 raft_consensus.cc:2468] T 455c56b836ff4e63949be17b0cf25c40 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21a7bb1d6ef54b20befdf8ea6d293776 in term 1.
I20260709 13:21:17.242921 22069 leader_election.cc:304] T 455c56b836ff4e63949be17b0cf25c40 P 21a7bb1d6ef54b20befdf8ea6d293776 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a7bb1d6ef54b20befdf8ea6d293776, 8507b5a0d0e6434aaa7302085c7f54cb; no voters: 
I20260709 13:21:17.244201 22240 tablet_bootstrap.cc:654] T d481ac177e6345efa62e86898d48fbac P c48712233a8a4eb0b39bb05cd60137e6: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:17.244639 22227 raft_consensus.cc:2804] T 455c56b836ff4e63949be17b0cf25c40 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:17.245245 21852 catalog_manager.cc:5697] T 91f78fe291144d05bce820bf9ee9cc03 P 8507b5a0d0e6434aaa7302085c7f54cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129). New cstate: current_term: 1 leader_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:17.227748 21850 catalog_manager.cc:5697] T 400f2b2a8e114d5d9506446a360c4daf P c48712233a8a4eb0b39bb05cd60137e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "c48712233a8a4eb0b39bb05cd60137e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:17.260704 22227 raft_consensus.cc:697] T 455c56b836ff4e63949be17b0cf25c40 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 LEADER]: Becoming Leader. State: Replica: 21a7bb1d6ef54b20befdf8ea6d293776, State: Running, Role: LEADER
I20260709 13:21:17.261945 22227 consensus_queue.cc:237] T 455c56b836ff4e63949be17b0cf25c40 P 21a7bb1d6ef54b20befdf8ea6d293776 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.270242 22240 tablet_bootstrap.cc:492] T d481ac177e6345efa62e86898d48fbac P c48712233a8a4eb0b39bb05cd60137e6: No bootstrap required, opened a new log
I20260709 13:21:17.271034 22240 ts_tablet_manager.cc:1403] T d481ac177e6345efa62e86898d48fbac P c48712233a8a4eb0b39bb05cd60137e6: Time spent bootstrapping tablet: real 0.035s	user 0.016s	sys 0.016s
I20260709 13:21:17.273865 22240 raft_consensus.cc:359] T d481ac177e6345efa62e86898d48fbac P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.274638 22240 raft_consensus.cc:385] T d481ac177e6345efa62e86898d48fbac P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:17.275058 22240 raft_consensus.cc:740] T d481ac177e6345efa62e86898d48fbac P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c48712233a8a4eb0b39bb05cd60137e6, State: Initialized, Role: FOLLOWER
I20260709 13:21:17.275792 22240 consensus_queue.cc:260] T d481ac177e6345efa62e86898d48fbac P c48712233a8a4eb0b39bb05cd60137e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.276273 21850 catalog_manager.cc:5697] T 455c56b836ff4e63949be17b0cf25c40 P 21a7bb1d6ef54b20befdf8ea6d293776 reported cstate change: term changed from 0 to 1, leader changed from <none> to 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131). New cstate: current_term: 1 leader_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:17.278278 22240 ts_tablet_manager.cc:1434] T d481ac177e6345efa62e86898d48fbac P c48712233a8a4eb0b39bb05cd60137e6: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:21:17.279312 22240 tablet_bootstrap.cc:492] T dfb67aaa32244d76a341136766820549 P c48712233a8a4eb0b39bb05cd60137e6: Bootstrap starting.
I20260709 13:21:17.284899 22240 tablet_bootstrap.cc:654] T dfb67aaa32244d76a341136766820549 P c48712233a8a4eb0b39bb05cd60137e6: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:17.288172 22250 raft_consensus.cc:493] T b86408eb7a2544ff921e6534948d5b67 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:17.288707 22250 raft_consensus.cc:515] T b86408eb7a2544ff921e6534948d5b67 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.289804 22240 tablet_bootstrap.cc:492] T dfb67aaa32244d76a341136766820549 P c48712233a8a4eb0b39bb05cd60137e6: No bootstrap required, opened a new log
I20260709 13:21:17.290470 22240 ts_tablet_manager.cc:1403] T dfb67aaa32244d76a341136766820549 P c48712233a8a4eb0b39bb05cd60137e6: Time spent bootstrapping tablet: real 0.011s	user 0.011s	sys 0.000s
I20260709 13:21:17.292011 22103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86408eb7a2544ff921e6534948d5b67" candidate_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" is_pre_election: true
I20260709 13:21:17.292688 22103 raft_consensus.cc:2468] T b86408eb7a2544ff921e6534948d5b67 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8507b5a0d0e6434aaa7302085c7f54cb in term 0.
I20260709 13:21:17.292775 22029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86408eb7a2544ff921e6534948d5b67" candidate_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6" is_pre_election: true
I20260709 13:21:17.293205 22240 raft_consensus.cc:359] T dfb67aaa32244d76a341136766820549 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.293519 22029 raft_consensus.cc:2468] T b86408eb7a2544ff921e6534948d5b67 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8507b5a0d0e6434aaa7302085c7f54cb in term 0.
I20260709 13:21:17.293852 22240 raft_consensus.cc:385] T dfb67aaa32244d76a341136766820549 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:17.294214 22240 raft_consensus.cc:740] T dfb67aaa32244d76a341136766820549 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c48712233a8a4eb0b39bb05cd60137e6, State: Initialized, Role: FOLLOWER
I20260709 13:21:17.294674 21917 leader_election.cc:304] T b86408eb7a2544ff921e6534948d5b67 P 8507b5a0d0e6434aaa7302085c7f54cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a7bb1d6ef54b20befdf8ea6d293776, 8507b5a0d0e6434aaa7302085c7f54cb; no voters: 
I20260709 13:21:17.294970 22240 consensus_queue.cc:260] T dfb67aaa32244d76a341136766820549 P c48712233a8a4eb0b39bb05cd60137e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.295646 22226 raft_consensus.cc:2804] T b86408eb7a2544ff921e6534948d5b67 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:17.296185 22226 raft_consensus.cc:493] T b86408eb7a2544ff921e6534948d5b67 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:17.295706 22250 leader_election.cc:290] T b86408eb7a2544ff921e6534948d5b67 P 8507b5a0d0e6434aaa7302085c7f54cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529), 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131:33869)
I20260709 13:21:17.296535 22226 raft_consensus.cc:3060] T b86408eb7a2544ff921e6534948d5b67 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:17.297199 22240 ts_tablet_manager.cc:1434] T dfb67aaa32244d76a341136766820549 P c48712233a8a4eb0b39bb05cd60137e6: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:21:17.303539 22226 raft_consensus.cc:515] T b86408eb7a2544ff921e6534948d5b67 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.306527 22029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86408eb7a2544ff921e6534948d5b67" candidate_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6"
I20260709 13:21:17.306809 22103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86408eb7a2544ff921e6534948d5b67" candidate_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776"
I20260709 13:21:17.307224 22029 raft_consensus.cc:3060] T b86408eb7a2544ff921e6534948d5b67 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:17.307492 22103 raft_consensus.cc:3060] T b86408eb7a2544ff921e6534948d5b67 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:17.310954 22226 leader_election.cc:290] T b86408eb7a2544ff921e6534948d5b67 P 8507b5a0d0e6434aaa7302085c7f54cb [CANDIDATE]: Term 1 election: Requested vote from peers c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529), 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131:33869)
I20260709 13:21:17.312732 22103 raft_consensus.cc:2468] T b86408eb7a2544ff921e6534948d5b67 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8507b5a0d0e6434aaa7302085c7f54cb in term 1.
I20260709 13:21:17.313767 21917 leader_election.cc:304] T b86408eb7a2544ff921e6534948d5b67 P 8507b5a0d0e6434aaa7302085c7f54cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a7bb1d6ef54b20befdf8ea6d293776, 8507b5a0d0e6434aaa7302085c7f54cb; no voters: 
I20260709 13:21:17.314442 22029 raft_consensus.cc:2468] T b86408eb7a2544ff921e6534948d5b67 P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8507b5a0d0e6434aaa7302085c7f54cb in term 1.
I20260709 13:21:17.314451 22226 raft_consensus.cc:2804] T b86408eb7a2544ff921e6534948d5b67 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:17.315182 22226 raft_consensus.cc:697] T b86408eb7a2544ff921e6534948d5b67 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 LEADER]: Becoming Leader. State: Replica: 8507b5a0d0e6434aaa7302085c7f54cb, State: Running, Role: LEADER
I20260709 13:21:17.316409 22226 consensus_queue.cc:237] T b86408eb7a2544ff921e6534948d5b67 P 8507b5a0d0e6434aaa7302085c7f54cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.318013 22227 raft_consensus.cc:493] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:17.318537 22227 raft_consensus.cc:515] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.320844 22227 leader_election.cc:290] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591), c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529)
I20260709 13:21:17.321584 21953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfb67aaa32244d76a341136766820549" candidate_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" is_pre_election: true
I20260709 13:21:17.321961 22029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfb67aaa32244d76a341136766820549" candidate_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6" is_pre_election: true
I20260709 13:21:17.322240 21953 raft_consensus.cc:2468] T dfb67aaa32244d76a341136766820549 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21a7bb1d6ef54b20befdf8ea6d293776 in term 0.
I20260709 13:21:17.322639 22029 raft_consensus.cc:2468] T dfb67aaa32244d76a341136766820549 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21a7bb1d6ef54b20befdf8ea6d293776 in term 0.
I20260709 13:21:17.323421 22069 leader_election.cc:304] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a7bb1d6ef54b20befdf8ea6d293776, 8507b5a0d0e6434aaa7302085c7f54cb; no voters: 
I20260709 13:21:17.324640 22227 raft_consensus.cc:2804] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:17.324966 22227 raft_consensus.cc:493] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:17.325177 22227 raft_consensus.cc:3060] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:17.326009 22261 raft_consensus.cc:493] T 300ddb6928744c789723f712d6f9780e P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:17.326027 21850 catalog_manager.cc:5697] T b86408eb7a2544ff921e6534948d5b67 P 8507b5a0d0e6434aaa7302085c7f54cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129). New cstate: current_term: 1 leader_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:17.326602 22261 raft_consensus.cc:515] T 300ddb6928744c789723f712d6f9780e P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.328871 22261 leader_election.cc:290] T 300ddb6928744c789723f712d6f9780e P c48712233a8a4eb0b39bb05cd60137e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591), 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131:33869)
I20260709 13:21:17.329663 21953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300ddb6928744c789723f712d6f9780e" candidate_uuid: "c48712233a8a4eb0b39bb05cd60137e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" is_pre_election: true
I20260709 13:21:17.329973 22103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300ddb6928744c789723f712d6f9780e" candidate_uuid: "c48712233a8a4eb0b39bb05cd60137e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" is_pre_election: true
I20260709 13:21:17.330354 21953 raft_consensus.cc:2468] T 300ddb6928744c789723f712d6f9780e P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c48712233a8a4eb0b39bb05cd60137e6 in term 0.
I20260709 13:21:17.330564 22103 raft_consensus.cc:2468] T 300ddb6928744c789723f712d6f9780e P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c48712233a8a4eb0b39bb05cd60137e6 in term 0.
I20260709 13:21:17.331385 21995 leader_election.cc:304] T 300ddb6928744c789723f712d6f9780e P c48712233a8a4eb0b39bb05cd60137e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8507b5a0d0e6434aaa7302085c7f54cb, c48712233a8a4eb0b39bb05cd60137e6; no voters: 
I20260709 13:21:17.332722 22261 raft_consensus.cc:2804] T 300ddb6928744c789723f712d6f9780e P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:17.333097 22261 raft_consensus.cc:493] T 300ddb6928744c789723f712d6f9780e P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:17.333422 22261 raft_consensus.cc:3060] T 300ddb6928744c789723f712d6f9780e P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:17.334659 22227 raft_consensus.cc:515] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.336851 22227 leader_election.cc:290] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776 [CANDIDATE]: Term 1 election: Requested vote from peers 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591), c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529)
I20260709 13:21:17.337459 21953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfb67aaa32244d76a341136766820549" candidate_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb"
I20260709 13:21:17.337899 22028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfb67aaa32244d76a341136766820549" candidate_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6"
I20260709 13:21:17.338095 21953 raft_consensus.cc:3060] T dfb67aaa32244d76a341136766820549 P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:17.338500 22028 raft_consensus.cc:3060] T dfb67aaa32244d76a341136766820549 P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:17.342265 22261 raft_consensus.cc:515] T 300ddb6928744c789723f712d6f9780e P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.344537 22261 leader_election.cc:290] T 300ddb6928744c789723f712d6f9780e P c48712233a8a4eb0b39bb05cd60137e6 [CANDIDATE]: Term 1 election: Requested vote from peers 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591), 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131:33869)
I20260709 13:21:17.345548 22103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300ddb6928744c789723f712d6f9780e" candidate_uuid: "c48712233a8a4eb0b39bb05cd60137e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776"
I20260709 13:21:17.345731 21954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300ddb6928744c789723f712d6f9780e" candidate_uuid: "c48712233a8a4eb0b39bb05cd60137e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb"
I20260709 13:21:17.346186 22103 raft_consensus.cc:3060] T 300ddb6928744c789723f712d6f9780e P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:17.346264 21954 raft_consensus.cc:3060] T 300ddb6928744c789723f712d6f9780e P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:17.346602 22250 consensus_queue.cc:1048] T 74231e641a7e4f7fb5bc39a2a084bb81 P 8507b5a0d0e6434aaa7302085c7f54cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:17.352176 22028 raft_consensus.cc:2468] T dfb67aaa32244d76a341136766820549 P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21a7bb1d6ef54b20befdf8ea6d293776 in term 1.
I20260709 13:21:17.353503 22069 leader_election.cc:304] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a7bb1d6ef54b20befdf8ea6d293776, c48712233a8a4eb0b39bb05cd60137e6; no voters: 
I20260709 13:21:17.354339 22227 raft_consensus.cc:2804] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:17.354313 22103 raft_consensus.cc:2468] T 300ddb6928744c789723f712d6f9780e P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c48712233a8a4eb0b39bb05cd60137e6 in term 1.
I20260709 13:21:17.355080 22227 raft_consensus.cc:697] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 LEADER]: Becoming Leader. State: Replica: 21a7bb1d6ef54b20befdf8ea6d293776, State: Running, Role: LEADER
I20260709 13:21:17.355473 21953 raft_consensus.cc:2468] T dfb67aaa32244d76a341136766820549 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21a7bb1d6ef54b20befdf8ea6d293776 in term 1.
I20260709 13:21:17.355764 21992 leader_election.cc:304] T 300ddb6928744c789723f712d6f9780e P c48712233a8a4eb0b39bb05cd60137e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a7bb1d6ef54b20befdf8ea6d293776, c48712233a8a4eb0b39bb05cd60137e6; no voters: 
I20260709 13:21:17.355983 22227 consensus_queue.cc:237] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.356870 22225 raft_consensus.cc:2804] T 300ddb6928744c789723f712d6f9780e P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:17.357311 22225 raft_consensus.cc:697] T 300ddb6928744c789723f712d6f9780e P c48712233a8a4eb0b39bb05cd60137e6 [term 1 LEADER]: Becoming Leader. State: Replica: c48712233a8a4eb0b39bb05cd60137e6, State: Running, Role: LEADER
I20260709 13:21:17.358124 22225 consensus_queue.cc:237] T 300ddb6928744c789723f712d6f9780e P c48712233a8a4eb0b39bb05cd60137e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.354074 21954 raft_consensus.cc:2468] T 300ddb6928744c789723f712d6f9780e P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c48712233a8a4eb0b39bb05cd60137e6 in term 1.
I20260709 13:21:17.361575 22226 consensus_queue.cc:1048] T b740d838804d4730b30e939a24d5414d P 8507b5a0d0e6434aaa7302085c7f54cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:17.379356 22225 raft_consensus.cc:493] T d481ac177e6345efa62e86898d48fbac P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:17.379882 22225 raft_consensus.cc:515] T d481ac177e6345efa62e86898d48fbac P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.382596 22225 leader_election.cc:290] T d481ac177e6345efa62e86898d48fbac P c48712233a8a4eb0b39bb05cd60137e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591), 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131:33869)
I20260709 13:21:17.385020 21953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481ac177e6345efa62e86898d48fbac" candidate_uuid: "c48712233a8a4eb0b39bb05cd60137e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" is_pre_election: true
I20260709 13:21:17.385747 21953 raft_consensus.cc:2468] T d481ac177e6345efa62e86898d48fbac P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c48712233a8a4eb0b39bb05cd60137e6 in term 0.
I20260709 13:21:17.387420 22250 consensus_queue.cc:1048] T 74231e641a7e4f7fb5bc39a2a084bb81 P 8507b5a0d0e6434aaa7302085c7f54cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:17.391251 22102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481ac177e6345efa62e86898d48fbac" candidate_uuid: "c48712233a8a4eb0b39bb05cd60137e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" is_pre_election: true
I20260709 13:21:17.392120 22102 raft_consensus.cc:2468] T d481ac177e6345efa62e86898d48fbac P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c48712233a8a4eb0b39bb05cd60137e6 in term 0.
I20260709 13:21:17.393429 21992 leader_election.cc:304] T d481ac177e6345efa62e86898d48fbac P c48712233a8a4eb0b39bb05cd60137e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a7bb1d6ef54b20befdf8ea6d293776, c48712233a8a4eb0b39bb05cd60137e6; no voters: 
I20260709 13:21:17.397267 21849 catalog_manager.cc:5697] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776 reported cstate change: term changed from 0 to 1, leader changed from <none> to 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131). New cstate: current_term: 1 leader_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:17.400439 21850 catalog_manager.cc:5697] T 300ddb6928744c789723f712d6f9780e P c48712233a8a4eb0b39bb05cd60137e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "c48712233a8a4eb0b39bb05cd60137e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:17.403431 22225 raft_consensus.cc:2804] T d481ac177e6345efa62e86898d48fbac P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:17.404110 22225 raft_consensus.cc:493] T d481ac177e6345efa62e86898d48fbac P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:17.406066 22225 raft_consensus.cc:3060] T d481ac177e6345efa62e86898d48fbac P c48712233a8a4eb0b39bb05cd60137e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:17.404601 22250 consensus_queue.cc:1048] T b740d838804d4730b30e939a24d5414d P 8507b5a0d0e6434aaa7302085c7f54cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:17.415915 22225 raft_consensus.cc:515] T d481ac177e6345efa62e86898d48fbac P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.422685 22225 leader_election.cc:290] T d481ac177e6345efa62e86898d48fbac P c48712233a8a4eb0b39bb05cd60137e6 [CANDIDATE]: Term 1 election: Requested vote from peers 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591), 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131:33869)
I20260709 13:21:17.423997 21953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481ac177e6345efa62e86898d48fbac" candidate_uuid: "c48712233a8a4eb0b39bb05cd60137e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb"
I20260709 13:21:17.424858 21953 raft_consensus.cc:3060] T d481ac177e6345efa62e86898d48fbac P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:17.423942 22103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481ac177e6345efa62e86898d48fbac" candidate_uuid: "c48712233a8a4eb0b39bb05cd60137e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776"
I20260709 13:21:17.425676 22103 raft_consensus.cc:3060] T d481ac177e6345efa62e86898d48fbac P 21a7bb1d6ef54b20befdf8ea6d293776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:17.432421 21953 raft_consensus.cc:2468] T d481ac177e6345efa62e86898d48fbac P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c48712233a8a4eb0b39bb05cd60137e6 in term 1.
I20260709 13:21:17.432421 22103 raft_consensus.cc:2468] T d481ac177e6345efa62e86898d48fbac P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c48712233a8a4eb0b39bb05cd60137e6 in term 1.
I20260709 13:21:17.433940 21995 leader_election.cc:304] T d481ac177e6345efa62e86898d48fbac P c48712233a8a4eb0b39bb05cd60137e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8507b5a0d0e6434aaa7302085c7f54cb, c48712233a8a4eb0b39bb05cd60137e6; no voters: 
I20260709 13:21:17.435143 22225 raft_consensus.cc:2804] T d481ac177e6345efa62e86898d48fbac P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:17.435616 22225 raft_consensus.cc:697] T d481ac177e6345efa62e86898d48fbac P c48712233a8a4eb0b39bb05cd60137e6 [term 1 LEADER]: Becoming Leader. State: Replica: c48712233a8a4eb0b39bb05cd60137e6, State: Running, Role: LEADER
I20260709 13:21:17.436420 22225 consensus_queue.cc:237] T d481ac177e6345efa62e86898d48fbac P c48712233a8a4eb0b39bb05cd60137e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.445694 21849 catalog_manager.cc:5697] T d481ac177e6345efa62e86898d48fbac P c48712233a8a4eb0b39bb05cd60137e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130). New cstate: current_term: 1 leader_uuid: "c48712233a8a4eb0b39bb05cd60137e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:17.503049 22250 raft_consensus.cc:493] T 400f2b2a8e114d5d9506446a360c4daf P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:17.503613 22250 raft_consensus.cc:515] T 400f2b2a8e114d5d9506446a360c4daf P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.505641 22250 leader_election.cc:290] T 400f2b2a8e114d5d9506446a360c4daf P 8507b5a0d0e6434aaa7302085c7f54cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529), 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131:33869)
I20260709 13:21:17.506305 22029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "400f2b2a8e114d5d9506446a360c4daf" candidate_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6" is_pre_election: true
I20260709 13:21:17.506749 22103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "400f2b2a8e114d5d9506446a360c4daf" candidate_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" is_pre_election: true
I20260709 13:21:17.507520 22103 raft_consensus.cc:2393] T 400f2b2a8e114d5d9506446a360c4daf P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8507b5a0d0e6434aaa7302085c7f54cb in current term 1: Already voted for candidate c48712233a8a4eb0b39bb05cd60137e6 in this term.
I20260709 13:21:17.508428 21917 leader_election.cc:304] T 400f2b2a8e114d5d9506446a360c4daf P 8507b5a0d0e6434aaa7302085c7f54cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8507b5a0d0e6434aaa7302085c7f54cb; no voters: 21a7bb1d6ef54b20befdf8ea6d293776, c48712233a8a4eb0b39bb05cd60137e6
I20260709 13:21:17.509682 22250 raft_consensus.cc:3060] T 400f2b2a8e114d5d9506446a360c4daf P 8507b5a0d0e6434aaa7302085c7f54cb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:17.510704 22029 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "400f2b2a8e114d5d9506446a360c4daf"
dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6"
mode: GRACEFUL
new_leader_uuid: "8507b5a0d0e6434aaa7302085c7f54cb"
 from {username='slave'} at 127.0.0.1:46600
I20260709 13:21:17.511394 22029 raft_consensus.cc:606] T 400f2b2a8e114d5d9506446a360c4daf P c48712233a8a4eb0b39bb05cd60137e6 [term 1 LEADER]: Received request to transfer leadership to TS 8507b5a0d0e6434aaa7302085c7f54cb
I20260709 13:21:17.515980 22250 raft_consensus.cc:2749] T 400f2b2a8e114d5d9506446a360c4daf P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:17.516613 22103 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "455c56b836ff4e63949be17b0cf25c40"
dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776"
mode: GRACEFUL
new_leader_uuid: "c48712233a8a4eb0b39bb05cd60137e6"
 from {username='slave'} at 127.0.0.1:53794
I20260709 13:21:17.517153 22103 raft_consensus.cc:606] T 455c56b836ff4e63949be17b0cf25c40 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 LEADER]: Received request to transfer leadership to TS c48712233a8a4eb0b39bb05cd60137e6
I20260709 13:21:17.522441 21953 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "91f78fe291144d05bce820bf9ee9cc03"
dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb"
mode: GRACEFUL
new_leader_uuid: "c48712233a8a4eb0b39bb05cd60137e6"
 from {username='slave'} at 127.0.0.1:51346
I20260709 13:21:17.522960 21953 raft_consensus.cc:606] T 91f78fe291144d05bce820bf9ee9cc03 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 LEADER]: Received request to transfer leadership to TS c48712233a8a4eb0b39bb05cd60137e6
I20260709 13:21:17.530893 22103 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dfb67aaa32244d76a341136766820549"
dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776"
mode: GRACEFUL
new_leader_uuid: "c48712233a8a4eb0b39bb05cd60137e6"
 from {username='slave'} at 127.0.0.1:53794
I20260709 13:21:17.531407 22103 raft_consensus.cc:606] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 LEADER]: Received request to transfer leadership to TS c48712233a8a4eb0b39bb05cd60137e6
I20260709 13:21:17.536327 22029 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d481ac177e6345efa62e86898d48fbac"
dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6"
mode: GRACEFUL
new_leader_uuid: "21a7bb1d6ef54b20befdf8ea6d293776"
 from {username='slave'} at 127.0.0.1:46600
I20260709 13:21:17.536916 22029 raft_consensus.cc:606] T d481ac177e6345efa62e86898d48fbac P c48712233a8a4eb0b39bb05cd60137e6 [term 1 LEADER]: Received request to transfer leadership to TS 21a7bb1d6ef54b20befdf8ea6d293776
I20260709 13:21:17.650826 22227 consensus_queue.cc:1048] T 455c56b836ff4e63949be17b0cf25c40 P 21a7bb1d6ef54b20befdf8ea6d293776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:17.657382 22275 raft_consensus.cc:993] T 455c56b836ff4e63949be17b0cf25c40 P 21a7bb1d6ef54b20befdf8ea6d293776: : Instructing follower c48712233a8a4eb0b39bb05cd60137e6 to start an election
I20260709 13:21:17.657912 22277 raft_consensus.cc:1081] T 455c56b836ff4e63949be17b0cf25c40 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 LEADER]: Signalling peer c48712233a8a4eb0b39bb05cd60137e6 to start an election
I20260709 13:21:17.659739 22028 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "455c56b836ff4e63949be17b0cf25c40"
dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6"
 from {username='slave'} at 127.0.0.1:46568
I20260709 13:21:17.660833 22028 raft_consensus.cc:493] T 455c56b836ff4e63949be17b0cf25c40 P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:17.661201 22028 raft_consensus.cc:3060] T 455c56b836ff4e63949be17b0cf25c40 P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:17.664606 22277 consensus_queue.cc:1048] T 455c56b836ff4e63949be17b0cf25c40 P 21a7bb1d6ef54b20befdf8ea6d293776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:17.668169 22028 raft_consensus.cc:515] T 455c56b836ff4e63949be17b0cf25c40 P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.671492 21953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "455c56b836ff4e63949be17b0cf25c40" candidate_uuid: "c48712233a8a4eb0b39bb05cd60137e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb"
I20260709 13:21:17.675508 22028 leader_election.cc:290] T 455c56b836ff4e63949be17b0cf25c40 P c48712233a8a4eb0b39bb05cd60137e6 [CANDIDATE]: Term 2 election: Requested vote from peers 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591), 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131:33869)
I20260709 13:21:17.676805 22225 consensus_queue.cc:1048] T 400f2b2a8e114d5d9506446a360c4daf P c48712233a8a4eb0b39bb05cd60137e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:17.677582 22103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "455c56b836ff4e63949be17b0cf25c40" candidate_uuid: "c48712233a8a4eb0b39bb05cd60137e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776"
I20260709 13:21:17.678242 22103 raft_consensus.cc:3055] T 455c56b836ff4e63949be17b0cf25c40 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:17.678560 22103 raft_consensus.cc:740] T 455c56b836ff4e63949be17b0cf25c40 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 21a7bb1d6ef54b20befdf8ea6d293776, State: Running, Role: LEADER
I20260709 13:21:17.679352 22103 consensus_queue.cc:260] T 455c56b836ff4e63949be17b0cf25c40 P 21a7bb1d6ef54b20befdf8ea6d293776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.680620 22103 raft_consensus.cc:3060] T 455c56b836ff4e63949be17b0cf25c40 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:17.690330 22103 raft_consensus.cc:2468] T 455c56b836ff4e63949be17b0cf25c40 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c48712233a8a4eb0b39bb05cd60137e6 in term 2.
I20260709 13:21:17.691581 21992 leader_election.cc:304] T 455c56b836ff4e63949be17b0cf25c40 P c48712233a8a4eb0b39bb05cd60137e6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 21a7bb1d6ef54b20befdf8ea6d293776, c48712233a8a4eb0b39bb05cd60137e6; no voters: 8507b5a0d0e6434aaa7302085c7f54cb
I20260709 13:21:17.692389 22225 raft_consensus.cc:2804] T 455c56b836ff4e63949be17b0cf25c40 P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:17.692909 22225 raft_consensus.cc:697] T 455c56b836ff4e63949be17b0cf25c40 P c48712233a8a4eb0b39bb05cd60137e6 [term 2 LEADER]: Becoming Leader. State: Replica: c48712233a8a4eb0b39bb05cd60137e6, State: Running, Role: LEADER
I20260709 13:21:17.693646 22261 raft_consensus.cc:993] T 400f2b2a8e114d5d9506446a360c4daf P c48712233a8a4eb0b39bb05cd60137e6: : Instructing follower 8507b5a0d0e6434aaa7302085c7f54cb to start an election
I20260709 13:21:17.693769 22225 consensus_queue.cc:237] T 455c56b836ff4e63949be17b0cf25c40 P c48712233a8a4eb0b39bb05cd60137e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.694463 22264 raft_consensus.cc:1081] T 400f2b2a8e114d5d9506446a360c4daf P c48712233a8a4eb0b39bb05cd60137e6 [term 1 LEADER]: Signalling peer 8507b5a0d0e6434aaa7302085c7f54cb to start an election
I20260709 13:21:17.696452 21954 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "400f2b2a8e114d5d9506446a360c4daf"
dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb"
 from {username='slave'} at 127.0.0.1:51340
I20260709 13:21:17.697242 21954 raft_consensus.cc:493] T 400f2b2a8e114d5d9506446a360c4daf P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:17.697737 21954 raft_consensus.cc:3060] T 400f2b2a8e114d5d9506446a360c4daf P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:17.703992 21850 catalog_manager.cc:5697] T 455c56b836ff4e63949be17b0cf25c40 P c48712233a8a4eb0b39bb05cd60137e6 reported cstate change: term changed from 1 to 2, leader changed from 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131) to c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130). New cstate: current_term: 2 leader_uuid: "c48712233a8a4eb0b39bb05cd60137e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:17.704607 21954 raft_consensus.cc:515] T 400f2b2a8e114d5d9506446a360c4daf P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.705623 21953 raft_consensus.cc:1240] T 400f2b2a8e114d5d9506446a360c4daf P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Rejecting Update request from peer c48712233a8a4eb0b39bb05cd60137e6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:21:17.706702 22225 consensus_queue.cc:1048] T 400f2b2a8e114d5d9506446a360c4daf P c48712233a8a4eb0b39bb05cd60137e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:17.709163 22225 consensus_queue.cc:1059] T 400f2b2a8e114d5d9506446a360c4daf P c48712233a8a4eb0b39bb05cd60137e6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:17.710031 22225 raft_consensus.cc:3055] T 400f2b2a8e114d5d9506446a360c4daf P c48712233a8a4eb0b39bb05cd60137e6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:17.711520 22102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "400f2b2a8e114d5d9506446a360c4daf" candidate_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776"
I20260709 13:21:17.709349 22028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "400f2b2a8e114d5d9506446a360c4daf" candidate_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6"
I20260709 13:21:17.711951 22225 raft_consensus.cc:740] T 400f2b2a8e114d5d9506446a360c4daf P c48712233a8a4eb0b39bb05cd60137e6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c48712233a8a4eb0b39bb05cd60137e6, State: Running, Role: LEADER
I20260709 13:21:17.713367 21954 leader_election.cc:290] T 400f2b2a8e114d5d9506446a360c4daf P 8507b5a0d0e6434aaa7302085c7f54cb [CANDIDATE]: Term 2 election: Requested vote from peers c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529), 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131:33869)
I20260709 13:21:17.713352 22225 consensus_queue.cc:260] T 400f2b2a8e114d5d9506446a360c4daf P c48712233a8a4eb0b39bb05cd60137e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.716385 22225 raft_consensus.cc:3060] T 400f2b2a8e114d5d9506446a360c4daf P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:17.731279 22028 raft_consensus.cc:2468] T 400f2b2a8e114d5d9506446a360c4daf P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8507b5a0d0e6434aaa7302085c7f54cb in term 2.
I20260709 13:21:17.733888 21920 leader_election.cc:304] T 400f2b2a8e114d5d9506446a360c4daf P 8507b5a0d0e6434aaa7302085c7f54cb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8507b5a0d0e6434aaa7302085c7f54cb, c48712233a8a4eb0b39bb05cd60137e6; no voters: 21a7bb1d6ef54b20befdf8ea6d293776
I20260709 13:21:17.734726 22250 raft_consensus.cc:2804] T 400f2b2a8e114d5d9506446a360c4daf P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:17.735281 22250 raft_consensus.cc:697] T 400f2b2a8e114d5d9506446a360c4daf P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 LEADER]: Becoming Leader. State: Replica: 8507b5a0d0e6434aaa7302085c7f54cb, State: Running, Role: LEADER
I20260709 13:21:17.736125 22250 consensus_queue.cc:237] T 400f2b2a8e114d5d9506446a360c4daf P 8507b5a0d0e6434aaa7302085c7f54cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.740458 22225 consensus_queue.cc:1048] T 300ddb6928744c789723f712d6f9780e P c48712233a8a4eb0b39bb05cd60137e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:17.745739 22250 consensus_queue.cc:1048] T 91f78fe291144d05bce820bf9ee9cc03 P 8507b5a0d0e6434aaa7302085c7f54cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:17.747134 21850 catalog_manager.cc:5697] T 400f2b2a8e114d5d9506446a360c4daf P 8507b5a0d0e6434aaa7302085c7f54cb reported cstate change: term changed from 1 to 2, leader changed from c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130) to 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129). New cstate: current_term: 2 leader_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:17.767139 22295 consensus_queue.cc:1048] T 300ddb6928744c789723f712d6f9780e P c48712233a8a4eb0b39bb05cd60137e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:17.769140 22252 consensus_queue.cc:1048] T 91f78fe291144d05bce820bf9ee9cc03 P 8507b5a0d0e6434aaa7302085c7f54cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:17.789460 22252 raft_consensus.cc:993] T 91f78fe291144d05bce820bf9ee9cc03 P 8507b5a0d0e6434aaa7302085c7f54cb: : Instructing follower c48712233a8a4eb0b39bb05cd60137e6 to start an election
I20260709 13:21:17.789932 22252 raft_consensus.cc:1081] T 91f78fe291144d05bce820bf9ee9cc03 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 LEADER]: Signalling peer c48712233a8a4eb0b39bb05cd60137e6 to start an election
I20260709 13:21:17.792156 22028 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "91f78fe291144d05bce820bf9ee9cc03"
dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6"
 from {username='slave'} at 127.0.0.1:46584
I20260709 13:21:17.792747 22028 raft_consensus.cc:493] T 91f78fe291144d05bce820bf9ee9cc03 P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:17.793207 22028 raft_consensus.cc:3060] T 91f78fe291144d05bce820bf9ee9cc03 P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:17.802459 22028 raft_consensus.cc:515] T 91f78fe291144d05bce820bf9ee9cc03 P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.805346 22028 leader_election.cc:290] T 91f78fe291144d05bce820bf9ee9cc03 P c48712233a8a4eb0b39bb05cd60137e6 [CANDIDATE]: Term 2 election: Requested vote from peers 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591), 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131:33869)
I20260709 13:21:17.806034 22102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91f78fe291144d05bce820bf9ee9cc03" candidate_uuid: "c48712233a8a4eb0b39bb05cd60137e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776"
I20260709 13:21:17.807222 22102 raft_consensus.cc:3060] T 91f78fe291144d05bce820bf9ee9cc03 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:17.809626 21954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91f78fe291144d05bce820bf9ee9cc03" candidate_uuid: "c48712233a8a4eb0b39bb05cd60137e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb"
I20260709 13:21:17.810343 21954 raft_consensus.cc:3055] T 91f78fe291144d05bce820bf9ee9cc03 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:17.810645 21954 raft_consensus.cc:740] T 91f78fe291144d05bce820bf9ee9cc03 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8507b5a0d0e6434aaa7302085c7f54cb, State: Running, Role: LEADER
I20260709 13:21:17.811481 21954 consensus_queue.cc:260] T 91f78fe291144d05bce820bf9ee9cc03 P 8507b5a0d0e6434aaa7302085c7f54cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.812758 21954 raft_consensus.cc:3060] T 91f78fe291144d05bce820bf9ee9cc03 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:17.819721 22102 raft_consensus.cc:2468] T 91f78fe291144d05bce820bf9ee9cc03 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c48712233a8a4eb0b39bb05cd60137e6 in term 2.
I20260709 13:21:17.820328 21954 raft_consensus.cc:2468] T 91f78fe291144d05bce820bf9ee9cc03 P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c48712233a8a4eb0b39bb05cd60137e6 in term 2.
I20260709 13:21:17.821197 21992 leader_election.cc:304] T 91f78fe291144d05bce820bf9ee9cc03 P c48712233a8a4eb0b39bb05cd60137e6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a7bb1d6ef54b20befdf8ea6d293776, c48712233a8a4eb0b39bb05cd60137e6; no voters: 
I20260709 13:21:17.822113 22264 raft_consensus.cc:2804] T 91f78fe291144d05bce820bf9ee9cc03 P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:17.822573 22264 raft_consensus.cc:697] T 91f78fe291144d05bce820bf9ee9cc03 P c48712233a8a4eb0b39bb05cd60137e6 [term 2 LEADER]: Becoming Leader. State: Replica: c48712233a8a4eb0b39bb05cd60137e6, State: Running, Role: LEADER
I20260709 13:21:17.823392 22264 consensus_queue.cc:237] T 91f78fe291144d05bce820bf9ee9cc03 P c48712233a8a4eb0b39bb05cd60137e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.832082 21850 catalog_manager.cc:5697] T 91f78fe291144d05bce820bf9ee9cc03 P c48712233a8a4eb0b39bb05cd60137e6 reported cstate change: term changed from 1 to 2, leader changed from 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129) to c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130). New cstate: current_term: 2 leader_uuid: "c48712233a8a4eb0b39bb05cd60137e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:17.853659 22245 consensus_queue.cc:1048] T b86408eb7a2544ff921e6534948d5b67 P 8507b5a0d0e6434aaa7302085c7f54cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:17.863651 22287 consensus_queue.cc:1048] T b86408eb7a2544ff921e6534948d5b67 P 8507b5a0d0e6434aaa7302085c7f54cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:17.893663 22264 consensus_queue.cc:1048] T d481ac177e6345efa62e86898d48fbac P c48712233a8a4eb0b39bb05cd60137e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:17.900074 22295 raft_consensus.cc:993] T d481ac177e6345efa62e86898d48fbac P c48712233a8a4eb0b39bb05cd60137e6: : Instructing follower 21a7bb1d6ef54b20befdf8ea6d293776 to start an election
I20260709 13:21:17.900539 22295 raft_consensus.cc:1081] T d481ac177e6345efa62e86898d48fbac P c48712233a8a4eb0b39bb05cd60137e6 [term 1 LEADER]: Signalling peer 21a7bb1d6ef54b20befdf8ea6d293776 to start an election
I20260709 13:21:17.902977 22103 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d481ac177e6345efa62e86898d48fbac"
dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776"
 from {username='slave'} at 127.0.0.1:53788
I20260709 13:21:17.903538 22103 raft_consensus.cc:493] T d481ac177e6345efa62e86898d48fbac P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:17.903884 22103 raft_consensus.cc:3060] T d481ac177e6345efa62e86898d48fbac P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:17.905732 22295 consensus_queue.cc:1048] T d481ac177e6345efa62e86898d48fbac P c48712233a8a4eb0b39bb05cd60137e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:17.910354 22103 raft_consensus.cc:515] T d481ac177e6345efa62e86898d48fbac P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.911607 22102 raft_consensus.cc:1240] T d481ac177e6345efa62e86898d48fbac P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Rejecting Update request from peer c48712233a8a4eb0b39bb05cd60137e6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:21:17.913290 21954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481ac177e6345efa62e86898d48fbac" candidate_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb"
I20260709 13:21:17.914132 22103 leader_election.cc:290] T d481ac177e6345efa62e86898d48fbac P 21a7bb1d6ef54b20befdf8ea6d293776 [CANDIDATE]: Term 2 election: Requested vote from peers 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591), c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529)
I20260709 13:21:17.915055 22295 consensus_queue.cc:1059] T d481ac177e6345efa62e86898d48fbac P c48712233a8a4eb0b39bb05cd60137e6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:17.915463 22028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481ac177e6345efa62e86898d48fbac" candidate_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6"
I20260709 13:21:17.916932 22028 raft_consensus.cc:3055] T d481ac177e6345efa62e86898d48fbac P c48712233a8a4eb0b39bb05cd60137e6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:17.914001 21954 raft_consensus.cc:3060] T d481ac177e6345efa62e86898d48fbac P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:17.917263 22028 raft_consensus.cc:740] T d481ac177e6345efa62e86898d48fbac P c48712233a8a4eb0b39bb05cd60137e6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c48712233a8a4eb0b39bb05cd60137e6, State: Running, Role: LEADER
I20260709 13:21:17.918076 22028 consensus_queue.cc:260] T d481ac177e6345efa62e86898d48fbac P c48712233a8a4eb0b39bb05cd60137e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.919332 22028 raft_consensus.cc:3060] T d481ac177e6345efa62e86898d48fbac P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:17.925459 21954 raft_consensus.cc:2468] T d481ac177e6345efa62e86898d48fbac P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21a7bb1d6ef54b20befdf8ea6d293776 in term 2.
I20260709 13:21:17.926178 22028 raft_consensus.cc:2468] T d481ac177e6345efa62e86898d48fbac P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21a7bb1d6ef54b20befdf8ea6d293776 in term 2.
W20260709 13:21:17.926800 22261 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 13:21:17.926905 22069 leader_election.cc:304] T d481ac177e6345efa62e86898d48fbac P 21a7bb1d6ef54b20befdf8ea6d293776 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a7bb1d6ef54b20befdf8ea6d293776, 8507b5a0d0e6434aaa7302085c7f54cb; no voters: 
I20260709 13:21:17.927872 22277 raft_consensus.cc:2804] T d481ac177e6345efa62e86898d48fbac P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:17.928380 22277 raft_consensus.cc:697] T d481ac177e6345efa62e86898d48fbac P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 LEADER]: Becoming Leader. State: Replica: 21a7bb1d6ef54b20befdf8ea6d293776, State: Running, Role: LEADER
I20260709 13:21:17.929181 22277 consensus_queue.cc:237] T d481ac177e6345efa62e86898d48fbac P 21a7bb1d6ef54b20befdf8ea6d293776 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.937217 21852 catalog_manager.cc:5697] T d481ac177e6345efa62e86898d48fbac P 21a7bb1d6ef54b20befdf8ea6d293776 reported cstate change: term changed from 1 to 2, leader changed from c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130) to 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131). New cstate: current_term: 2 leader_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:17.953343 22277 consensus_queue.cc:1048] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:17.959601 22227 raft_consensus.cc:993] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776: : Instructing follower c48712233a8a4eb0b39bb05cd60137e6 to start an election
I20260709 13:21:17.960086 22275 raft_consensus.cc:1081] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 LEADER]: Signalling peer c48712233a8a4eb0b39bb05cd60137e6 to start an election
I20260709 13:21:17.961726 22029 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dfb67aaa32244d76a341136766820549"
dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6"
 from {username='slave'} at 127.0.0.1:46568
I20260709 13:21:17.962294 22029 raft_consensus.cc:493] T dfb67aaa32244d76a341136766820549 P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:17.962674 22029 raft_consensus.cc:3060] T dfb67aaa32244d76a341136766820549 P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:17.966655 22277 consensus_queue.cc:1048] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:17.970407 22029 raft_consensus.cc:515] T dfb67aaa32244d76a341136766820549 P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.973341 21953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfb67aaa32244d76a341136766820549" candidate_uuid: "c48712233a8a4eb0b39bb05cd60137e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb"
I20260709 13:21:17.975822 22029 leader_election.cc:290] T dfb67aaa32244d76a341136766820549 P c48712233a8a4eb0b39bb05cd60137e6 [CANDIDATE]: Term 2 election: Requested vote from peers 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591), 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131:33869)
I20260709 13:21:17.977059 22103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfb67aaa32244d76a341136766820549" candidate_uuid: "c48712233a8a4eb0b39bb05cd60137e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776"
I20260709 13:21:17.977705 22103 raft_consensus.cc:3055] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:17.978008 22103 raft_consensus.cc:740] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 21a7bb1d6ef54b20befdf8ea6d293776, State: Running, Role: LEADER
I20260709 13:21:17.978772 22103 consensus_queue.cc:260] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:17.980031 22103 raft_consensus.cc:3060] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:17.992839 22103 raft_consensus.cc:2468] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c48712233a8a4eb0b39bb05cd60137e6 in term 2.
I20260709 13:21:17.994422 21992 leader_election.cc:304] T dfb67aaa32244d76a341136766820549 P c48712233a8a4eb0b39bb05cd60137e6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 21a7bb1d6ef54b20befdf8ea6d293776, c48712233a8a4eb0b39bb05cd60137e6; no voters: 8507b5a0d0e6434aaa7302085c7f54cb
I20260709 13:21:17.995770 22261 raft_consensus.cc:2804] T dfb67aaa32244d76a341136766820549 P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:17.996299 22261 raft_consensus.cc:697] T dfb67aaa32244d76a341136766820549 P c48712233a8a4eb0b39bb05cd60137e6 [term 2 LEADER]: Becoming Leader. State: Replica: c48712233a8a4eb0b39bb05cd60137e6, State: Running, Role: LEADER
I20260709 13:21:17.997116 22261 consensus_queue.cc:237] T dfb67aaa32244d76a341136766820549 P c48712233a8a4eb0b39bb05cd60137e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:18.004366 21852 catalog_manager.cc:5697] T dfb67aaa32244d76a341136766820549 P c48712233a8a4eb0b39bb05cd60137e6 reported cstate change: term changed from 1 to 2, leader changed from 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131) to c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130). New cstate: current_term: 2 leader_uuid: "c48712233a8a4eb0b39bb05cd60137e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:18.107126 22103 raft_consensus.cc:1275] T 455c56b836ff4e63949be17b0cf25c40 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Refusing update from remote peer c48712233a8a4eb0b39bb05cd60137e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:18.108580 22261 consensus_queue.cc:1048] T 455c56b836ff4e63949be17b0cf25c40 P c48712233a8a4eb0b39bb05cd60137e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:18.116093 21954 raft_consensus.cc:3060] T 455c56b836ff4e63949be17b0cf25c40 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:18.124342 21954 raft_consensus.cc:1275] T 455c56b836ff4e63949be17b0cf25c40 P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Refusing update from remote peer c48712233a8a4eb0b39bb05cd60137e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:18.125566 22261 consensus_queue.cc:1048] T 455c56b836ff4e63949be17b0cf25c40 P c48712233a8a4eb0b39bb05cd60137e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:18.156333 22029 raft_consensus.cc:1275] T 400f2b2a8e114d5d9506446a360c4daf P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Refusing update from remote peer 8507b5a0d0e6434aaa7302085c7f54cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:18.157666 22287 consensus_queue.cc:1048] T 400f2b2a8e114d5d9506446a360c4daf P 8507b5a0d0e6434aaa7302085c7f54cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:18.165841 22103 raft_consensus.cc:3060] T 400f2b2a8e114d5d9506446a360c4daf P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:18.173146 22103 raft_consensus.cc:1275] T 400f2b2a8e114d5d9506446a360c4daf P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Refusing update from remote peer 8507b5a0d0e6434aaa7302085c7f54cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:18.174589 22287 consensus_queue.cc:1048] T 400f2b2a8e114d5d9506446a360c4daf P 8507b5a0d0e6434aaa7302085c7f54cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:18.230333 22103 raft_consensus.cc:1275] T 91f78fe291144d05bce820bf9ee9cc03 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Refusing update from remote peer c48712233a8a4eb0b39bb05cd60137e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:18.232138 22261 consensus_queue.cc:1048] T 91f78fe291144d05bce820bf9ee9cc03 P c48712233a8a4eb0b39bb05cd60137e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:18.233047 21954 raft_consensus.cc:1275] T 91f78fe291144d05bce820bf9ee9cc03 P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Refusing update from remote peer c48712233a8a4eb0b39bb05cd60137e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:18.234912 22261 consensus_queue.cc:1048] T 91f78fe291144d05bce820bf9ee9cc03 P c48712233a8a4eb0b39bb05cd60137e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:18.439056 21954 raft_consensus.cc:1275] T d481ac177e6345efa62e86898d48fbac P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Refusing update from remote peer 21a7bb1d6ef54b20befdf8ea6d293776: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:18.440285 22277 consensus_queue.cc:1048] T d481ac177e6345efa62e86898d48fbac P 21a7bb1d6ef54b20befdf8ea6d293776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:18.448244 22029 raft_consensus.cc:1275] T d481ac177e6345efa62e86898d48fbac P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Refusing update from remote peer 21a7bb1d6ef54b20befdf8ea6d293776: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:18.450871 22277 consensus_queue.cc:1048] T d481ac177e6345efa62e86898d48fbac P 21a7bb1d6ef54b20befdf8ea6d293776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:18.492156 21954 raft_consensus.cc:3060] T dfb67aaa32244d76a341136766820549 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:18.500176 21954 raft_consensus.cc:1275] T dfb67aaa32244d76a341136766820549 P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Refusing update from remote peer c48712233a8a4eb0b39bb05cd60137e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:18.501811 22261 consensus_queue.cc:1048] T dfb67aaa32244d76a341136766820549 P c48712233a8a4eb0b39bb05cd60137e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:18.512579 22103 raft_consensus.cc:1275] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Refusing update from remote peer c48712233a8a4eb0b39bb05cd60137e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:18.514098 22261 consensus_queue.cc:1048] T dfb67aaa32244d76a341136766820549 P c48712233a8a4eb0b39bb05cd60137e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:20.541723 21852 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45298:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:21:20.574126 21852 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45298:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:21:20.604352  9682 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
W20260709 13:21:20.761317 22326 log.cc:927] Time spent T d481ac177e6345efa62e86898d48fbac P 21a7bb1d6ef54b20befdf8ea6d293776: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
W20260709 13:21:20.817871 22337 log.cc:927] Time spent T b86408eb7a2544ff921e6534948d5b67 P 21a7bb1d6ef54b20befdf8ea6d293776: Append to log took a long time: real 0.088s	user 0.004s	sys 0.000s
I20260709 13:21:20.785274 22346 tablet.cc:1724] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:20.834851 22359 tablet.cc:1724] T dfb67aaa32244d76a341136766820549 P c48712233a8a4eb0b39bb05cd60137e6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:20.825428 22352 tablet.cc:1724] T b740d838804d4730b30e939a24d5414d P 21a7bb1d6ef54b20befdf8ea6d293776: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:20.858263 21953 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "74231e641a7e4f7fb5bc39a2a084bb81"
dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb"
mode: GRACEFUL
new_leader_uuid: "21a7bb1d6ef54b20befdf8ea6d293776"
 from {username='slave'} at 127.0.0.1:43246
I20260709 13:21:20.859382 21953 raft_consensus.cc:606] T 74231e641a7e4f7fb5bc39a2a084bb81 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 LEADER]: Received request to transfer leadership to TS 21a7bb1d6ef54b20befdf8ea6d293776
W20260709 13:21:20.861088 22339 log.cc:927] Time spent T d481ac177e6345efa62e86898d48fbac P c48712233a8a4eb0b39bb05cd60137e6: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:21:20.861927 22363 tablet.cc:1724] T 455c56b836ff4e63949be17b0cf25c40 P c48712233a8a4eb0b39bb05cd60137e6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:20.817953 22351 tablet.cc:1724] T b740d838804d4730b30e939a24d5414d P 8507b5a0d0e6434aaa7302085c7f54cb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:21:20.895577 22055 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:455c56b836ff4e63949be17b0cf25c40)
W20260709 13:21:20.901190 22055 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:455c56b836ff4e63949be17b0cf25c40)
W20260709 13:21:20.902143 22055 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:455c56b836ff4e63949be17b0cf25c40)
I20260709 13:21:20.895063 22345 tablet.cc:1724] T 91f78fe291144d05bce820bf9ee9cc03 P c48712233a8a4eb0b39bb05cd60137e6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:20.932065 22370 tablet.cc:1724] T d481ac177e6345efa62e86898d48fbac P 8507b5a0d0e6434aaa7302085c7f54cb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:20.932286 22367 tablet.cc:1724] T d481ac177e6345efa62e86898d48fbac P c48712233a8a4eb0b39bb05cd60137e6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:21:20.882340 21992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:33869, user_credentials={real_user=slave}} blocked reactor thread for 46762.6us
W20260709 13:21:20.891424 21995 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.129:35591, user_credentials={real_user=slave}} blocked reactor thread for 65524us
I20260709 13:21:20.971616 22355 tablet.cc:1724] T d481ac177e6345efa62e86898d48fbac P 21a7bb1d6ef54b20befdf8ea6d293776: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:20.992422 22369 tablet.cc:1724] T 74231e641a7e4f7fb5bc39a2a084bb81 P 8507b5a0d0e6434aaa7302085c7f54cb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:21.003850 22372 tablet.cc:1724] T 74231e641a7e4f7fb5bc39a2a084bb81 P 21a7bb1d6ef54b20befdf8ea6d293776: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:21.024618 22368 raft_consensus.cc:993] T 74231e641a7e4f7fb5bc39a2a084bb81 P 8507b5a0d0e6434aaa7302085c7f54cb: : Instructing follower 21a7bb1d6ef54b20befdf8ea6d293776 to start an election
W20260709 13:21:21.025017 21920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:46529, user_credentials={real_user=slave}} blocked reactor thread for 77640.7us
I20260709 13:21:21.025272 22368 raft_consensus.cc:1081] T 74231e641a7e4f7fb5bc39a2a084bb81 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 LEADER]: Signalling peer 21a7bb1d6ef54b20befdf8ea6d293776 to start an election
I20260709 13:21:21.027755 22359 tablet.cc:1724] T b740d838804d4730b30e939a24d5414d P c48712233a8a4eb0b39bb05cd60137e6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:21.029062 22352 tablet.cc:1724] T 91f78fe291144d05bce820bf9ee9cc03 P 21a7bb1d6ef54b20befdf8ea6d293776: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:21.031818 22370 tablet.cc:1724] T 400f2b2a8e114d5d9506446a360c4daf P 8507b5a0d0e6434aaa7302085c7f54cb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:21.031627 22100 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "74231e641a7e4f7fb5bc39a2a084bb81"
dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776"
 from {username='slave'} at 127.0.0.1:53790
I20260709 13:21:21.034585 22100 raft_consensus.cc:493] T 74231e641a7e4f7fb5bc39a2a084bb81 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:21.035252 22100 raft_consensus.cc:3060] T 74231e641a7e4f7fb5bc39a2a084bb81 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:21.040380 22346 tablet.cc:1724] T 455c56b836ff4e63949be17b0cf25c40 P 21a7bb1d6ef54b20befdf8ea6d293776: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:21.081233 22100 raft_consensus.cc:515] T 74231e641a7e4f7fb5bc39a2a084bb81 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:21.094274 22359 tablet.cc:1724] T 400f2b2a8e114d5d9506446a360c4daf P c48712233a8a4eb0b39bb05cd60137e6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:21.094645 22345 tablet.cc:1724] T 74231e641a7e4f7fb5bc39a2a084bb81 P c48712233a8a4eb0b39bb05cd60137e6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:21.096583 22355 tablet.cc:1724] T b86408eb7a2544ff921e6534948d5b67 P 21a7bb1d6ef54b20befdf8ea6d293776: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:21.106357 22363 tablet.cc:1724] T b86408eb7a2544ff921e6534948d5b67 P c48712233a8a4eb0b39bb05cd60137e6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:21.120163 22100 leader_election.cc:290] T 74231e641a7e4f7fb5bc39a2a084bb81 P 21a7bb1d6ef54b20befdf8ea6d293776 [CANDIDATE]: Term 2 election: Requested vote from peers 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591), c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529)
I20260709 13:21:21.145078 22351 tablet.cc:1724] T 455c56b836ff4e63949be17b0cf25c40 P 8507b5a0d0e6434aaa7302085c7f54cb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:21.073819 22352 tablet.cc:1724] T 400f2b2a8e114d5d9506446a360c4daf P 21a7bb1d6ef54b20befdf8ea6d293776: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:21.121031 21951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74231e641a7e4f7fb5bc39a2a084bb81" candidate_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb"
I20260709 13:21:21.150722 21951 raft_consensus.cc:3055] T 74231e641a7e4f7fb5bc39a2a084bb81 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:21.151113 21951 raft_consensus.cc:740] T 74231e641a7e4f7fb5bc39a2a084bb81 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8507b5a0d0e6434aaa7302085c7f54cb, State: Running, Role: LEADER
I20260709 13:21:21.152159 21951 consensus_queue.cc:260] T 74231e641a7e4f7fb5bc39a2a084bb81 P 8507b5a0d0e6434aaa7302085c7f54cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:21.123715 22378 tablet.cc:1724] T b86408eb7a2544ff921e6534948d5b67 P 8507b5a0d0e6434aaa7302085c7f54cb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:21.124586 22027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74231e641a7e4f7fb5bc39a2a084bb81" candidate_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6"
I20260709 13:21:21.153419 22027 raft_consensus.cc:3060] T 74231e641a7e4f7fb5bc39a2a084bb81 P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:21.153731 21951 raft_consensus.cc:3060] T 74231e641a7e4f7fb5bc39a2a084bb81 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:21.136029 22359 tablet.cc:1724] T 300ddb6928744c789723f712d6f9780e P c48712233a8a4eb0b39bb05cd60137e6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:21.124106 22028 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dfb67aaa32244d76a341136766820549"
dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6"
mode: GRACEFUL
new_leader_uuid: "21a7bb1d6ef54b20befdf8ea6d293776"
 from {username='slave'} at 127.0.0.1:38854
I20260709 13:21:21.155045 22028 raft_consensus.cc:606] T dfb67aaa32244d76a341136766820549 P c48712233a8a4eb0b39bb05cd60137e6 [term 2 LEADER]: Received request to transfer leadership to TS 21a7bb1d6ef54b20befdf8ea6d293776
I20260709 13:21:21.151666 22370 tablet.cc:1724] T dfb67aaa32244d76a341136766820549 P 8507b5a0d0e6434aaa7302085c7f54cb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:21.162135 22027 raft_consensus.cc:2468] T 74231e641a7e4f7fb5bc39a2a084bb81 P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21a7bb1d6ef54b20befdf8ea6d293776 in term 2.
I20260709 13:21:21.163337  9682 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 13:21:21.164698 22372 tablet.cc:1724] T 300ddb6928744c789723f712d6f9780e P 21a7bb1d6ef54b20befdf8ea6d293776: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:21.168556  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:21:21.170571 21951 raft_consensus.cc:2468] T 74231e641a7e4f7fb5bc39a2a084bb81 P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21a7bb1d6ef54b20befdf8ea6d293776 in term 2.
I20260709 13:21:21.172223 22069 leader_election.cc:304] T 74231e641a7e4f7fb5bc39a2a084bb81 P 21a7bb1d6ef54b20befdf8ea6d293776 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a7bb1d6ef54b20befdf8ea6d293776, 8507b5a0d0e6434aaa7302085c7f54cb; no voters: 
I20260709 13:21:21.173382 22277 raft_consensus.cc:2804] T 74231e641a7e4f7fb5bc39a2a084bb81 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:21.173862 22277 raft_consensus.cc:697] T 74231e641a7e4f7fb5bc39a2a084bb81 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 LEADER]: Becoming Leader. State: Replica: 21a7bb1d6ef54b20befdf8ea6d293776, State: Running, Role: LEADER
I20260709 13:21:21.174903 22277 consensus_queue.cc:237] T 74231e641a7e4f7fb5bc39a2a084bb81 P 21a7bb1d6ef54b20befdf8ea6d293776 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:21.187824 22351 tablet.cc:1724] T 91f78fe291144d05bce820bf9ee9cc03 P 8507b5a0d0e6434aaa7302085c7f54cb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:21.197140 21954 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0ca42dd42e4d46359256d9256f64d657"
dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb"
mode: GRACEFUL
new_leader_uuid: "21a7bb1d6ef54b20befdf8ea6d293776"
 from {username='slave'} at 127.0.0.1:43246
I20260709 13:21:21.197927 21954 raft_consensus.cc:606] T 0ca42dd42e4d46359256d9256f64d657 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 LEADER]: Received request to transfer leadership to TS 21a7bb1d6ef54b20befdf8ea6d293776
I20260709 13:21:21.197388 22369 tablet.cc:1724] T 300ddb6928744c789723f712d6f9780e P 8507b5a0d0e6434aaa7302085c7f54cb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:21.200150 21851 catalog_manager.cc:5697] T 74231e641a7e4f7fb5bc39a2a084bb81 P 21a7bb1d6ef54b20befdf8ea6d293776 reported cstate change: term changed from 1 to 2, leader changed from 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129) to 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131). New cstate: current_term: 2 leader_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:21.201362 22027 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ba306baff7874796aba9a7c3fd5d628e"
dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6"
mode: GRACEFUL
new_leader_uuid: "21a7bb1d6ef54b20befdf8ea6d293776"
 from {username='slave'} at 127.0.0.1:38854
I20260709 13:21:21.204097 22027 raft_consensus.cc:606] T ba306baff7874796aba9a7c3fd5d628e P c48712233a8a4eb0b39bb05cd60137e6 [term 2 LEADER]: Received request to transfer leadership to TS 21a7bb1d6ef54b20befdf8ea6d293776
I20260709 13:21:21.207306  9682 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:21:21.207952  9682 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:21:21.218178 21852 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 848ddee1690240febd24df3095cdbbab: soft_deleted_table [id=9654cec777eb43cdbb439e3b6c18c25c] alter complete (version 1)
I20260709 13:21:21.336966 22360 raft_consensus.cc:993] T dfb67aaa32244d76a341136766820549 P c48712233a8a4eb0b39bb05cd60137e6: : Instructing follower 21a7bb1d6ef54b20befdf8ea6d293776 to start an election
I20260709 13:21:21.337517 22361 raft_consensus.cc:1081] T dfb67aaa32244d76a341136766820549 P c48712233a8a4eb0b39bb05cd60137e6 [term 2 LEADER]: Signalling peer 21a7bb1d6ef54b20befdf8ea6d293776 to start an election
I20260709 13:21:21.339102 22098 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dfb67aaa32244d76a341136766820549"
dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776"
 from {username='slave'} at 127.0.0.1:53788
I20260709 13:21:21.339609 22098 raft_consensus.cc:493] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:21.339926 22098 raft_consensus.cc:3060] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:21.346248 22098 raft_consensus.cc:515] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:21.349031 22098 leader_election.cc:290] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776 [CANDIDATE]: Term 3 election: Requested vote from peers 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591), c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529)
I20260709 13:21:21.349222 21954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfb67aaa32244d76a341136766820549" candidate_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb"
I20260709 13:21:21.349946 21954 raft_consensus.cc:3060] T dfb67aaa32244d76a341136766820549 P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:21.350760 22360 raft_consensus.cc:993] T ba306baff7874796aba9a7c3fd5d628e P c48712233a8a4eb0b39bb05cd60137e6: : Instructing follower 21a7bb1d6ef54b20befdf8ea6d293776 to start an election
I20260709 13:21:21.351323 22360 raft_consensus.cc:1081] T ba306baff7874796aba9a7c3fd5d628e P c48712233a8a4eb0b39bb05cd60137e6 [term 2 LEADER]: Signalling peer 21a7bb1d6ef54b20befdf8ea6d293776 to start an election
I20260709 13:21:21.351514 22027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfb67aaa32244d76a341136766820549" candidate_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6"
I20260709 13:21:21.352178 22027 raft_consensus.cc:3055] T dfb67aaa32244d76a341136766820549 P c48712233a8a4eb0b39bb05cd60137e6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:21:21.352530 22027 raft_consensus.cc:740] T dfb67aaa32244d76a341136766820549 P c48712233a8a4eb0b39bb05cd60137e6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c48712233a8a4eb0b39bb05cd60137e6, State: Running, Role: LEADER
I20260709 13:21:21.353235 22100 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ba306baff7874796aba9a7c3fd5d628e"
dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776"
 from {username='slave'} at 127.0.0.1:53788
I20260709 13:21:21.353832 22100 raft_consensus.cc:493] T ba306baff7874796aba9a7c3fd5d628e P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:21.353449 22027 consensus_queue.cc:260] T dfb67aaa32244d76a341136766820549 P c48712233a8a4eb0b39bb05cd60137e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:21.354274 22100 raft_consensus.cc:3060] T ba306baff7874796aba9a7c3fd5d628e P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:21.355049 22027 raft_consensus.cc:3060] T dfb67aaa32244d76a341136766820549 P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:21.357532 22371 raft_consensus.cc:993] T 0ca42dd42e4d46359256d9256f64d657 P 8507b5a0d0e6434aaa7302085c7f54cb: : Instructing follower 21a7bb1d6ef54b20befdf8ea6d293776 to start an election
I20260709 13:21:21.358096 22371 raft_consensus.cc:1081] T 0ca42dd42e4d46359256d9256f64d657 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 LEADER]: Signalling peer 21a7bb1d6ef54b20befdf8ea6d293776 to start an election
I20260709 13:21:21.360018 22101 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0ca42dd42e4d46359256d9256f64d657"
dest_uuid: "21a7bb1d6ef54b20befdf8ea6d293776"
 from {username='slave'} at 127.0.0.1:53790
I20260709 13:21:21.360598 22101 raft_consensus.cc:493] T 0ca42dd42e4d46359256d9256f64d657 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:21.361020 22101 raft_consensus.cc:3060] T 0ca42dd42e4d46359256d9256f64d657 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:21.364519 21954 raft_consensus.cc:2468] T dfb67aaa32244d76a341136766820549 P 8507b5a0d0e6434aaa7302085c7f54cb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21a7bb1d6ef54b20befdf8ea6d293776 in term 3.
I20260709 13:21:21.367095 22069 leader_election.cc:304] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a7bb1d6ef54b20befdf8ea6d293776, 8507b5a0d0e6434aaa7302085c7f54cb; no voters: 
I20260709 13:21:21.368005 22277 raft_consensus.cc:2804] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:21:21.368505 22277 raft_consensus.cc:697] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 3 LEADER]: Becoming Leader. State: Replica: 21a7bb1d6ef54b20befdf8ea6d293776, State: Running, Role: LEADER
I20260709 13:21:21.368711 22100 raft_consensus.cc:515] T ba306baff7874796aba9a7c3fd5d628e P 21a7bb1d6ef54b20befdf8ea6d293776 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:21.369582 22277 consensus_queue.cc:237] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:21.371325 22100 leader_election.cc:290] T ba306baff7874796aba9a7c3fd5d628e P 21a7bb1d6ef54b20befdf8ea6d293776 [CANDIDATE]: Term 3 election: Requested vote from peers 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591), c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529)
I20260709 13:21:21.374467 21951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba306baff7874796aba9a7c3fd5d628e" candidate_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb"
I20260709 13:21:21.375079 21951 raft_consensus.cc:3060] T ba306baff7874796aba9a7c3fd5d628e P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:21.370625 22027 raft_consensus.cc:2468] T dfb67aaa32244d76a341136766820549 P c48712233a8a4eb0b39bb05cd60137e6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21a7bb1d6ef54b20befdf8ea6d293776 in term 3.
I20260709 13:21:21.378472 22101 raft_consensus.cc:515] T 0ca42dd42e4d46359256d9256f64d657 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } } peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } }
I20260709 13:21:21.380312 22027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba306baff7874796aba9a7c3fd5d628e" candidate_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6"
I20260709 13:21:21.381032 22027 raft_consensus.cc:3055] T ba306baff7874796aba9a7c3fd5d628e P c48712233a8a4eb0b39bb05cd60137e6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:21:21.381361 22027 raft_consensus.cc:740] T ba306baff7874796aba9a7c3fd5d628e P c48712233a8a4eb0b39bb05cd60137e6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c48712233a8a4eb0b39bb05cd60137e6, State: Running, Role: LEADER
I20260709 13:21:21.382778 21950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ca42dd42e4d46359256d9256f64d657" candidate_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8507b5a0d0e6434aaa7302085c7f54cb"
I20260709 13:21:21.383500 21950 raft_consensus.cc:3055] T 0ca42dd42e4d46359256d9256f64d657 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:21.382956 21851 catalog_manager.cc:5697] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776 reported cstate change: term changed from 2 to 3, leader changed from c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130) to 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131). New cstate: current_term: 3 leader_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:21.383853 21950 raft_consensus.cc:740] T 0ca42dd42e4d46359256d9256f64d657 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8507b5a0d0e6434aaa7302085c7f54cb, State: Running, Role: LEADER
I20260709 13:21:21.383558 22027 consensus_queue.cc:260] T ba306baff7874796aba9a7c3fd5d628e P c48712233a8a4eb0b39bb05cd60137e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:21.384847 21950 consensus_queue.cc:260] T 0ca42dd42e4d46359256d9256f64d657 P 8507b5a0d0e6434aaa7302085c7f54cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } } peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } }
I20260709 13:21:21.385509 22027 raft_consensus.cc:3060] T ba306baff7874796aba9a7c3fd5d628e P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:21.386271 21950 raft_consensus.cc:3060] T 0ca42dd42e4d46359256d9256f64d657 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:21.385322 21951 raft_consensus.cc:2468] T ba306baff7874796aba9a7c3fd5d628e P 8507b5a0d0e6434aaa7302085c7f54cb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21a7bb1d6ef54b20befdf8ea6d293776 in term 3.
I20260709 13:21:21.390098 22069 leader_election.cc:304] T ba306baff7874796aba9a7c3fd5d628e P 21a7bb1d6ef54b20befdf8ea6d293776 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a7bb1d6ef54b20befdf8ea6d293776, 8507b5a0d0e6434aaa7302085c7f54cb; no voters: 
I20260709 13:21:21.391225 22277 raft_consensus.cc:2804] T ba306baff7874796aba9a7c3fd5d628e P 21a7bb1d6ef54b20befdf8ea6d293776 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:21:21.391935 22277 raft_consensus.cc:697] T ba306baff7874796aba9a7c3fd5d628e P 21a7bb1d6ef54b20befdf8ea6d293776 [term 3 LEADER]: Becoming Leader. State: Replica: 21a7bb1d6ef54b20befdf8ea6d293776, State: Running, Role: LEADER
I20260709 13:21:21.393085 22277 consensus_queue.cc:237] T ba306baff7874796aba9a7c3fd5d628e P 21a7bb1d6ef54b20befdf8ea6d293776 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } }
I20260709 13:21:21.394662 22027 raft_consensus.cc:2468] T ba306baff7874796aba9a7c3fd5d628e P c48712233a8a4eb0b39bb05cd60137e6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21a7bb1d6ef54b20befdf8ea6d293776 in term 3.
I20260709 13:21:21.395056 22101 leader_election.cc:290] T 0ca42dd42e4d46359256d9256f64d657 P 21a7bb1d6ef54b20befdf8ea6d293776 [CANDIDATE]: Term 2 election: Requested vote from peers 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591), c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529)
I20260709 13:21:21.400629 22027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ca42dd42e4d46359256d9256f64d657" candidate_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c48712233a8a4eb0b39bb05cd60137e6"
I20260709 13:21:21.401973 22027 raft_consensus.cc:3060] T 0ca42dd42e4d46359256d9256f64d657 P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:21.403475 21950 raft_consensus.cc:2468] T 0ca42dd42e4d46359256d9256f64d657 P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21a7bb1d6ef54b20befdf8ea6d293776 in term 2.
I20260709 13:21:21.401356 21954 raft_consensus.cc:1240] T ba306baff7874796aba9a7c3fd5d628e P 8507b5a0d0e6434aaa7302085c7f54cb [term 3 FOLLOWER]: Rejecting Update request from peer c48712233a8a4eb0b39bb05cd60137e6 for earlier term 2. Current term is 3. Ops: []
I20260709 13:21:21.405552 22069 leader_election.cc:304] T 0ca42dd42e4d46359256d9256f64d657 P 21a7bb1d6ef54b20befdf8ea6d293776 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a7bb1d6ef54b20befdf8ea6d293776, 8507b5a0d0e6434aaa7302085c7f54cb; no voters: 
I20260709 13:21:21.406433 22277 raft_consensus.cc:2804] T 0ca42dd42e4d46359256d9256f64d657 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:21.407022 22277 raft_consensus.cc:697] T 0ca42dd42e4d46359256d9256f64d657 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 LEADER]: Becoming Leader. State: Replica: 21a7bb1d6ef54b20befdf8ea6d293776, State: Running, Role: LEADER
I20260709 13:21:21.406097 21851 catalog_manager.cc:5697] T ba306baff7874796aba9a7c3fd5d628e P 21a7bb1d6ef54b20befdf8ea6d293776 reported cstate change: term changed from 2 to 3, leader changed from c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130) to 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131). New cstate: current_term: 3 leader_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:21.408072 22277 consensus_queue.cc:237] T 0ca42dd42e4d46359256d9256f64d657 P 21a7bb1d6ef54b20befdf8ea6d293776 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } } peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } }
I20260709 13:21:21.409924 22027 raft_consensus.cc:2468] T 0ca42dd42e4d46359256d9256f64d657 P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21a7bb1d6ef54b20befdf8ea6d293776 in term 2.
I20260709 13:21:21.421360 21851 catalog_manager.cc:5697] T 0ca42dd42e4d46359256d9256f64d657 P 21a7bb1d6ef54b20befdf8ea6d293776 reported cstate change: term changed from 1 to 2, leader changed from 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129) to 21a7bb1d6ef54b20befdf8ea6d293776 (127.9.116.131). New cstate: current_term: 2 leader_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a7bb1d6ef54b20befdf8ea6d293776" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 33869 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:21.570168 21954 raft_consensus.cc:1275] T 74231e641a7e4f7fb5bc39a2a084bb81 P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Refusing update from remote peer 21a7bb1d6ef54b20befdf8ea6d293776: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:21:21.571482 22277 consensus_queue.cc:1048] T 74231e641a7e4f7fb5bc39a2a084bb81 P 21a7bb1d6ef54b20befdf8ea6d293776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:21:21.581305 22027 raft_consensus.cc:1275] T 74231e641a7e4f7fb5bc39a2a084bb81 P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Refusing update from remote peer 21a7bb1d6ef54b20befdf8ea6d293776: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:21:21.583206 22277 consensus_queue.cc:1048] T 74231e641a7e4f7fb5bc39a2a084bb81 P 21a7bb1d6ef54b20befdf8ea6d293776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:21:21.803704 22027 raft_consensus.cc:1275] T dfb67aaa32244d76a341136766820549 P c48712233a8a4eb0b39bb05cd60137e6 [term 3 FOLLOWER]: Refusing update from remote peer 21a7bb1d6ef54b20befdf8ea6d293776: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:21:21.805189 22388 consensus_queue.cc:1048] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:21:21.812999 21954 raft_consensus.cc:1275] T dfb67aaa32244d76a341136766820549 P 8507b5a0d0e6434aaa7302085c7f54cb [term 3 FOLLOWER]: Refusing update from remote peer 21a7bb1d6ef54b20befdf8ea6d293776: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:21:21.815443 22277 consensus_queue.cc:1048] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:21:21.830983 22027 raft_consensus.cc:1275] T 0ca42dd42e4d46359256d9256f64d657 P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Refusing update from remote peer 21a7bb1d6ef54b20befdf8ea6d293776: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:21.832468 22277 consensus_queue.cc:1048] T 0ca42dd42e4d46359256d9256f64d657 P 21a7bb1d6ef54b20befdf8ea6d293776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:21.846341 21954 raft_consensus.cc:1275] T 0ca42dd42e4d46359256d9256f64d657 P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Refusing update from remote peer 21a7bb1d6ef54b20befdf8ea6d293776: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:21.847762 22332 consensus_queue.cc:1048] T 0ca42dd42e4d46359256d9256f64d657 P 21a7bb1d6ef54b20befdf8ea6d293776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:21.891825 21954 raft_consensus.cc:1275] T ba306baff7874796aba9a7c3fd5d628e P 8507b5a0d0e6434aaa7302085c7f54cb [term 3 FOLLOWER]: Refusing update from remote peer 21a7bb1d6ef54b20befdf8ea6d293776: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:21:21.893033 22332 consensus_queue.cc:1048] T ba306baff7874796aba9a7c3fd5d628e P 21a7bb1d6ef54b20befdf8ea6d293776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8507b5a0d0e6434aaa7302085c7f54cb" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:21:21.904282 22027 raft_consensus.cc:1275] T ba306baff7874796aba9a7c3fd5d628e P c48712233a8a4eb0b39bb05cd60137e6 [term 3 FOLLOWER]: Refusing update from remote peer 21a7bb1d6ef54b20befdf8ea6d293776: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:21:21.906020 22332 consensus_queue.cc:1048] T ba306baff7874796aba9a7c3fd5d628e P 21a7bb1d6ef54b20befdf8ea6d293776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c48712233a8a4eb0b39bb05cd60137e6" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:21:22.269537  9682 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 13:21:22.275651  9682 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 13:21:22.276047  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:21:22.280653  9682 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 13:21:22.281034  9682 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:21:23.313984  9682 tablet_server.cc:179] TabletServer@127.9.116.129:0 shutting down...
I20260709 13:21:23.344834  9682 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:21:23.345680  9682 tablet_replica.cc:333] T d481ac177e6345efa62e86898d48fbac P 8507b5a0d0e6434aaa7302085c7f54cb: stopping tablet replica
I20260709 13:21:23.346540  9682 raft_consensus.cc:2243] T d481ac177e6345efa62e86898d48fbac P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:23.347119  9682 raft_consensus.cc:2272] T d481ac177e6345efa62e86898d48fbac P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.349411  9682 tablet_replica.cc:333] T 91f78fe291144d05bce820bf9ee9cc03 P 8507b5a0d0e6434aaa7302085c7f54cb: stopping tablet replica
I20260709 13:21:23.350063  9682 raft_consensus.cc:2243] T 91f78fe291144d05bce820bf9ee9cc03 P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:23.350565  9682 raft_consensus.cc:2272] T 91f78fe291144d05bce820bf9ee9cc03 P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.352831  9682 tablet_replica.cc:333] T 455c56b836ff4e63949be17b0cf25c40 P 8507b5a0d0e6434aaa7302085c7f54cb: stopping tablet replica
I20260709 13:21:23.353494  9682 raft_consensus.cc:2243] T 455c56b836ff4e63949be17b0cf25c40 P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:23.353962  9682 raft_consensus.cc:2272] T 455c56b836ff4e63949be17b0cf25c40 P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.356344  9682 tablet_replica.cc:333] T b86408eb7a2544ff921e6534948d5b67 P 8507b5a0d0e6434aaa7302085c7f54cb: stopping tablet replica
I20260709 13:21:23.357012  9682 raft_consensus.cc:2243] T b86408eb7a2544ff921e6534948d5b67 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:23.357846  9682 raft_consensus.cc:2272] T b86408eb7a2544ff921e6534948d5b67 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.360188  9682 tablet_replica.cc:333] T 400f2b2a8e114d5d9506446a360c4daf P 8507b5a0d0e6434aaa7302085c7f54cb: stopping tablet replica
I20260709 13:21:23.360811  9682 raft_consensus.cc:2243] T 400f2b2a8e114d5d9506446a360c4daf P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:23.361775  9682 raft_consensus.cc:2272] T 400f2b2a8e114d5d9506446a360c4daf P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.363947  9682 tablet_replica.cc:333] T b740d838804d4730b30e939a24d5414d P 8507b5a0d0e6434aaa7302085c7f54cb: stopping tablet replica
I20260709 13:21:23.364493  9682 raft_consensus.cc:2243] T b740d838804d4730b30e939a24d5414d P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:23.365290  9682 raft_consensus.cc:2272] T b740d838804d4730b30e939a24d5414d P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.367317  9682 tablet_replica.cc:333] T ea585ca33ebf4b329ce97d386bf3e7bc P 8507b5a0d0e6434aaa7302085c7f54cb: stopping tablet replica
I20260709 13:21:23.367940  9682 raft_consensus.cc:2243] T ea585ca33ebf4b329ce97d386bf3e7bc P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:23.368778  9682 raft_consensus.cc:2272] T ea585ca33ebf4b329ce97d386bf3e7bc P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.371058  9682 tablet_replica.cc:333] T dfb67aaa32244d76a341136766820549 P 8507b5a0d0e6434aaa7302085c7f54cb: stopping tablet replica
I20260709 13:21:23.371635  9682 raft_consensus.cc:2243] T dfb67aaa32244d76a341136766820549 P 8507b5a0d0e6434aaa7302085c7f54cb [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:23.372064  9682 raft_consensus.cc:2272] T dfb67aaa32244d76a341136766820549 P 8507b5a0d0e6434aaa7302085c7f54cb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.374029  9682 tablet_replica.cc:333] T 300ddb6928744c789723f712d6f9780e P 8507b5a0d0e6434aaa7302085c7f54cb: stopping tablet replica
I20260709 13:21:23.374601  9682 raft_consensus.cc:2243] T 300ddb6928744c789723f712d6f9780e P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:23.375056  9682 raft_consensus.cc:2272] T 300ddb6928744c789723f712d6f9780e P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.377156  9682 tablet_replica.cc:333] T 74231e641a7e4f7fb5bc39a2a084bb81 P 8507b5a0d0e6434aaa7302085c7f54cb: stopping tablet replica
I20260709 13:21:23.377763  9682 raft_consensus.cc:2243] T 74231e641a7e4f7fb5bc39a2a084bb81 P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:23.378214  9682 raft_consensus.cc:2272] T 74231e641a7e4f7fb5bc39a2a084bb81 P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.380319  9682 tablet_replica.cc:333] T 7f2c531192bb4639a60df411741abc7b P 8507b5a0d0e6434aaa7302085c7f54cb: stopping tablet replica
I20260709 13:21:23.380887  9682 raft_consensus.cc:2243] T 7f2c531192bb4639a60df411741abc7b P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:23.381407  9682 raft_consensus.cc:2272] T 7f2c531192bb4639a60df411741abc7b P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:23.383836 21995 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 10 similar messages]
I20260709 13:21:23.384603  9682 tablet_replica.cc:333] T 0ca42dd42e4d46359256d9256f64d657 P 8507b5a0d0e6434aaa7302085c7f54cb: stopping tablet replica
I20260709 13:21:23.385601  9682 raft_consensus.cc:2243] T 0ca42dd42e4d46359256d9256f64d657 P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:23.386137  9682 raft_consensus.cc:2272] T 0ca42dd42e4d46359256d9256f64d657 P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:23.387097 21995 consensus_peers.cc:597] T d74919ac686e49e694ca111d3b1a27f6 P c48712233a8a4eb0b39bb05cd60137e6 -> Peer 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591): Couldn't send request to peer 8507b5a0d0e6434aaa7302085c7f54cb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:23.388533  9682 tablet_replica.cc:333] T ba306baff7874796aba9a7c3fd5d628e P 8507b5a0d0e6434aaa7302085c7f54cb: stopping tablet replica
I20260709 13:21:23.389249  9682 raft_consensus.cc:2243] T ba306baff7874796aba9a7c3fd5d628e P 8507b5a0d0e6434aaa7302085c7f54cb [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:23.389716  9682 raft_consensus.cc:2272] T ba306baff7874796aba9a7c3fd5d628e P 8507b5a0d0e6434aaa7302085c7f54cb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.392195  9682 tablet_replica.cc:333] T d74919ac686e49e694ca111d3b1a27f6 P 8507b5a0d0e6434aaa7302085c7f54cb: stopping tablet replica
I20260709 13:21:23.392760  9682 raft_consensus.cc:2243] T d74919ac686e49e694ca111d3b1a27f6 P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:23.393247  9682 raft_consensus.cc:2272] T d74919ac686e49e694ca111d3b1a27f6 P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.395480  9682 tablet_replica.cc:333] T 5d0c4853c48a42938548ac8810a0bbc4 P 8507b5a0d0e6434aaa7302085c7f54cb: stopping tablet replica
I20260709 13:21:23.396042  9682 raft_consensus.cc:2243] T 5d0c4853c48a42938548ac8810a0bbc4 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:23.396481  9682 raft_consensus.cc:2272] T 5d0c4853c48a42938548ac8810a0bbc4 P 8507b5a0d0e6434aaa7302085c7f54cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.398538  9682 tablet_replica.cc:333] T b8791ec386d148c19bb6fb734287e432 P 8507b5a0d0e6434aaa7302085c7f54cb: stopping tablet replica
I20260709 13:21:23.399153  9682 raft_consensus.cc:2243] T b8791ec386d148c19bb6fb734287e432 P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:23.400017  9682 raft_consensus.cc:2272] T b8791ec386d148c19bb6fb734287e432 P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.402652  9682 tablet_replica.cc:333] T 1631e507d6714dc187fc9d7f2457aad9 P 8507b5a0d0e6434aaa7302085c7f54cb: stopping tablet replica
I20260709 13:21:23.403393  9682 raft_consensus.cc:2243] T 1631e507d6714dc187fc9d7f2457aad9 P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:23.404588  9682 raft_consensus.cc:2272] T 1631e507d6714dc187fc9d7f2457aad9 P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.406973  9682 tablet_replica.cc:333] T f9dceef9cf6541039db61f39e2798430 P 8507b5a0d0e6434aaa7302085c7f54cb: stopping tablet replica
I20260709 13:21:23.407523  9682 raft_consensus.cc:2243] T f9dceef9cf6541039db61f39e2798430 P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:23.407940  9682 raft_consensus.cc:2272] T f9dceef9cf6541039db61f39e2798430 P 8507b5a0d0e6434aaa7302085c7f54cb [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:23.422279 22069 consensus_peers.cc:597] T ba306baff7874796aba9a7c3fd5d628e P 21a7bb1d6ef54b20befdf8ea6d293776 -> Peer 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591): Couldn't send request to peer 8507b5a0d0e6434aaa7302085c7f54cb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:23.449966 21995 consensus_peers.cc:597] T 7f2c531192bb4639a60df411741abc7b P c48712233a8a4eb0b39bb05cd60137e6 -> Peer 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591): Couldn't send request to peer 8507b5a0d0e6434aaa7302085c7f54cb. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:35591: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:23.451897 21995 consensus_peers.cc:597] T 455c56b836ff4e63949be17b0cf25c40 P c48712233a8a4eb0b39bb05cd60137e6 -> Peer 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591): Couldn't send request to peer 8507b5a0d0e6434aaa7302085c7f54cb. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:35591: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:23.453811  9682 tablet_server.cc:196] TabletServer@127.9.116.129:0 shutdown complete.
I20260709 13:21:23.490163  9682 tablet_server.cc:179] TabletServer@127.9.116.130:0 shutting down...
I20260709 13:21:23.525383  9682 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:21:23.526338  9682 tablet_replica.cc:333] T d481ac177e6345efa62e86898d48fbac P c48712233a8a4eb0b39bb05cd60137e6: stopping tablet replica
I20260709 13:21:23.527447  9682 raft_consensus.cc:2243] T d481ac177e6345efa62e86898d48fbac P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:23.528095  9682 raft_consensus.cc:2272] T d481ac177e6345efa62e86898d48fbac P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:23.528898 22069 consensus_peers.cc:597] T ba306baff7874796aba9a7c3fd5d628e P 21a7bb1d6ef54b20befdf8ea6d293776 -> Peer c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529): Couldn't send request to peer c48712233a8a4eb0b39bb05cd60137e6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:23.530444  9682 tablet_replica.cc:333] T 455c56b836ff4e63949be17b0cf25c40 P c48712233a8a4eb0b39bb05cd60137e6: stopping tablet replica
I20260709 13:21:23.531349  9682 raft_consensus.cc:2243] T 455c56b836ff4e63949be17b0cf25c40 P c48712233a8a4eb0b39bb05cd60137e6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:23.532186  9682 raft_consensus.cc:2272] T 455c56b836ff4e63949be17b0cf25c40 P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.534466  9682 tablet_replica.cc:333] T 91f78fe291144d05bce820bf9ee9cc03 P c48712233a8a4eb0b39bb05cd60137e6: stopping tablet replica
I20260709 13:21:23.535234  9682 raft_consensus.cc:2243] T 91f78fe291144d05bce820bf9ee9cc03 P c48712233a8a4eb0b39bb05cd60137e6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:23.536042  9682 raft_consensus.cc:2272] T 91f78fe291144d05bce820bf9ee9cc03 P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.538219  9682 tablet_replica.cc:333] T b86408eb7a2544ff921e6534948d5b67 P c48712233a8a4eb0b39bb05cd60137e6: stopping tablet replica
I20260709 13:21:23.538765  9682 raft_consensus.cc:2243] T b86408eb7a2544ff921e6534948d5b67 P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:23.539330  9682 raft_consensus.cc:2272] T b86408eb7a2544ff921e6534948d5b67 P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.541518  9682 tablet_replica.cc:333] T 400f2b2a8e114d5d9506446a360c4daf P c48712233a8a4eb0b39bb05cd60137e6: stopping tablet replica
I20260709 13:21:23.542274  9682 raft_consensus.cc:2243] T 400f2b2a8e114d5d9506446a360c4daf P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:23.542891  9682 raft_consensus.cc:2272] T 400f2b2a8e114d5d9506446a360c4daf P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.545168  9682 tablet_replica.cc:333] T b740d838804d4730b30e939a24d5414d P c48712233a8a4eb0b39bb05cd60137e6: stopping tablet replica
I20260709 13:21:23.545780  9682 raft_consensus.cc:2243] T b740d838804d4730b30e939a24d5414d P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:23.546273  9682 raft_consensus.cc:2272] T b740d838804d4730b30e939a24d5414d P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.548605  9682 tablet_replica.cc:333] T d74919ac686e49e694ca111d3b1a27f6 P c48712233a8a4eb0b39bb05cd60137e6: stopping tablet replica
I20260709 13:21:23.549221  9682 raft_consensus.cc:2243] T d74919ac686e49e694ca111d3b1a27f6 P c48712233a8a4eb0b39bb05cd60137e6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:23.550124  9682 raft_consensus.cc:2272] T d74919ac686e49e694ca111d3b1a27f6 P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.552466  9682 tablet_replica.cc:333] T ba306baff7874796aba9a7c3fd5d628e P c48712233a8a4eb0b39bb05cd60137e6: stopping tablet replica
I20260709 13:21:23.553056  9682 raft_consensus.cc:2243] T ba306baff7874796aba9a7c3fd5d628e P c48712233a8a4eb0b39bb05cd60137e6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:23.553511  9682 raft_consensus.cc:2272] T ba306baff7874796aba9a7c3fd5d628e P c48712233a8a4eb0b39bb05cd60137e6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.555857  9682 tablet_replica.cc:333] T 0ca42dd42e4d46359256d9256f64d657 P c48712233a8a4eb0b39bb05cd60137e6: stopping tablet replica
I20260709 13:21:23.556536  9682 raft_consensus.cc:2243] T 0ca42dd42e4d46359256d9256f64d657 P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:23.556993  9682 raft_consensus.cc:2272] T 0ca42dd42e4d46359256d9256f64d657 P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.559257  9682 tablet_replica.cc:333] T ea585ca33ebf4b329ce97d386bf3e7bc P c48712233a8a4eb0b39bb05cd60137e6: stopping tablet replica
I20260709 13:21:23.559892  9682 raft_consensus.cc:2243] T ea585ca33ebf4b329ce97d386bf3e7bc P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:23.560456  9682 raft_consensus.cc:2272] T ea585ca33ebf4b329ce97d386bf3e7bc P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.562740  9682 tablet_replica.cc:333] T b8791ec386d148c19bb6fb734287e432 P c48712233a8a4eb0b39bb05cd60137e6: stopping tablet replica
I20260709 13:21:23.563432  9682 raft_consensus.cc:2243] T b8791ec386d148c19bb6fb734287e432 P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:23.563920  9682 raft_consensus.cc:2272] T b8791ec386d148c19bb6fb734287e432 P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.566138  9682 tablet_replica.cc:333] T 5d0c4853c48a42938548ac8810a0bbc4 P c48712233a8a4eb0b39bb05cd60137e6: stopping tablet replica
I20260709 13:21:23.566819  9682 raft_consensus.cc:2243] T 5d0c4853c48a42938548ac8810a0bbc4 P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:23.567332  9682 raft_consensus.cc:2272] T 5d0c4853c48a42938548ac8810a0bbc4 P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.569542  9682 tablet_replica.cc:333] T f9dceef9cf6541039db61f39e2798430 P c48712233a8a4eb0b39bb05cd60137e6: stopping tablet replica
I20260709 13:21:23.570124  9682 raft_consensus.cc:2243] T f9dceef9cf6541039db61f39e2798430 P c48712233a8a4eb0b39bb05cd60137e6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:23.571092  9682 raft_consensus.cc:2272] T f9dceef9cf6541039db61f39e2798430 P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.573388  9682 tablet_replica.cc:333] T dfb67aaa32244d76a341136766820549 P c48712233a8a4eb0b39bb05cd60137e6: stopping tablet replica
I20260709 13:21:23.574002  9682 raft_consensus.cc:2243] T dfb67aaa32244d76a341136766820549 P c48712233a8a4eb0b39bb05cd60137e6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:23.574447  9682 raft_consensus.cc:2272] T dfb67aaa32244d76a341136766820549 P c48712233a8a4eb0b39bb05cd60137e6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.576560  9682 tablet_replica.cc:333] T 300ddb6928744c789723f712d6f9780e P c48712233a8a4eb0b39bb05cd60137e6: stopping tablet replica
I20260709 13:21:23.577355  9682 raft_consensus.cc:2243] T 300ddb6928744c789723f712d6f9780e P c48712233a8a4eb0b39bb05cd60137e6 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:23.578241  9682 raft_consensus.cc:2272] T 300ddb6928744c789723f712d6f9780e P c48712233a8a4eb0b39bb05cd60137e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.580633  9682 tablet_replica.cc:333] T 74231e641a7e4f7fb5bc39a2a084bb81 P c48712233a8a4eb0b39bb05cd60137e6: stopping tablet replica
I20260709 13:21:23.581180  9682 raft_consensus.cc:2243] T 74231e641a7e4f7fb5bc39a2a084bb81 P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:23.581664  9682 raft_consensus.cc:2272] T 74231e641a7e4f7fb5bc39a2a084bb81 P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.583751  9682 tablet_replica.cc:333] T 7f2c531192bb4639a60df411741abc7b P c48712233a8a4eb0b39bb05cd60137e6: stopping tablet replica
I20260709 13:21:23.584295  9682 raft_consensus.cc:2243] T 7f2c531192bb4639a60df411741abc7b P c48712233a8a4eb0b39bb05cd60137e6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:23.585201  9682 raft_consensus.cc:2272] T 7f2c531192bb4639a60df411741abc7b P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.587518  9682 tablet_replica.cc:333] T 1631e507d6714dc187fc9d7f2457aad9 P c48712233a8a4eb0b39bb05cd60137e6: stopping tablet replica
I20260709 13:21:23.588152  9682 raft_consensus.cc:2243] T 1631e507d6714dc187fc9d7f2457aad9 P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:23.588668  9682 raft_consensus.cc:2272] T 1631e507d6714dc187fc9d7f2457aad9 P c48712233a8a4eb0b39bb05cd60137e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.633431  9682 tablet_server.cc:196] TabletServer@127.9.116.130:0 shutdown complete.
I20260709 13:21:23.667752  9682 tablet_server.cc:179] TabletServer@127.9.116.131:0 shutting down...
W20260709 13:21:23.683802 22069 consensus_peers.cc:597] T 74231e641a7e4f7fb5bc39a2a084bb81 P 21a7bb1d6ef54b20befdf8ea6d293776 -> Peer 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591): Couldn't send request to peer 8507b5a0d0e6434aaa7302085c7f54cb. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:35591: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:23.684805 22069 consensus_peers.cc:597] T d481ac177e6345efa62e86898d48fbac P 21a7bb1d6ef54b20befdf8ea6d293776 -> Peer 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591): Couldn't send request to peer 8507b5a0d0e6434aaa7302085c7f54cb. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:35591: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:23.688633 22069 consensus_peers.cc:597] T 5d0c4853c48a42938548ac8810a0bbc4 P 21a7bb1d6ef54b20befdf8ea6d293776 -> Peer 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591): Couldn't send request to peer 8507b5a0d0e6434aaa7302085c7f54cb. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:35591: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:23.704381  9682 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:21:23.705197  9682 tablet_replica.cc:333] T d481ac177e6345efa62e86898d48fbac P 21a7bb1d6ef54b20befdf8ea6d293776: stopping tablet replica
I20260709 13:21:23.705911  9682 raft_consensus.cc:2243] T d481ac177e6345efa62e86898d48fbac P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:23.706732  9682 raft_consensus.cc:2272] T d481ac177e6345efa62e86898d48fbac P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.708930  9682 tablet_replica.cc:333] T 91f78fe291144d05bce820bf9ee9cc03 P 21a7bb1d6ef54b20befdf8ea6d293776: stopping tablet replica
I20260709 13:21:23.709547  9682 raft_consensus.cc:2243] T 91f78fe291144d05bce820bf9ee9cc03 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:23.710079  9682 raft_consensus.cc:2272] T 91f78fe291144d05bce820bf9ee9cc03 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.712832  9682 tablet_replica.cc:333] T 455c56b836ff4e63949be17b0cf25c40 P 21a7bb1d6ef54b20befdf8ea6d293776: stopping tablet replica
I20260709 13:21:23.713564  9682 raft_consensus.cc:2243] T 455c56b836ff4e63949be17b0cf25c40 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:23.714154  9682 raft_consensus.cc:2272] T 455c56b836ff4e63949be17b0cf25c40 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:23.714974 22069 consensus_peers.cc:597] T 0ca42dd42e4d46359256d9256f64d657 P 21a7bb1d6ef54b20befdf8ea6d293776 -> Peer 8507b5a0d0e6434aaa7302085c7f54cb (127.9.116.129:35591): Couldn't send request to peer 8507b5a0d0e6434aaa7302085c7f54cb. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:35591: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:23.716509  9682 tablet_replica.cc:333] T b86408eb7a2544ff921e6534948d5b67 P 21a7bb1d6ef54b20befdf8ea6d293776: stopping tablet replica
I20260709 13:21:23.717188  9682 raft_consensus.cc:2243] T b86408eb7a2544ff921e6534948d5b67 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:23.717676  9682 raft_consensus.cc:2272] T b86408eb7a2544ff921e6534948d5b67 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.719880  9682 tablet_replica.cc:333] T 400f2b2a8e114d5d9506446a360c4daf P 21a7bb1d6ef54b20befdf8ea6d293776: stopping tablet replica
I20260709 13:21:23.720491  9682 raft_consensus.cc:2243] T 400f2b2a8e114d5d9506446a360c4daf P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:23.720964  9682 raft_consensus.cc:2272] T 400f2b2a8e114d5d9506446a360c4daf P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.723028  9682 tablet_replica.cc:333] T b740d838804d4730b30e939a24d5414d P 21a7bb1d6ef54b20befdf8ea6d293776: stopping tablet replica
I20260709 13:21:23.723598  9682 raft_consensus.cc:2243] T b740d838804d4730b30e939a24d5414d P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:23.724030  9682 raft_consensus.cc:2272] T b740d838804d4730b30e939a24d5414d P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.726106  9682 tablet_replica.cc:333] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776: stopping tablet replica
I20260709 13:21:23.726691  9682 raft_consensus.cc:2243] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:21:23.727543  9682 raft_consensus.cc:2272] T dfb67aaa32244d76a341136766820549 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.729408  9682 tablet_replica.cc:333] T 300ddb6928744c789723f712d6f9780e P 21a7bb1d6ef54b20befdf8ea6d293776: stopping tablet replica
I20260709 13:21:23.729947  9682 raft_consensus.cc:2243] T 300ddb6928744c789723f712d6f9780e P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:23.730382  9682 raft_consensus.cc:2272] T 300ddb6928744c789723f712d6f9780e P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.732381  9682 tablet_replica.cc:333] T 74231e641a7e4f7fb5bc39a2a084bb81 P 21a7bb1d6ef54b20befdf8ea6d293776: stopping tablet replica
I20260709 13:21:23.732934  9682 raft_consensus.cc:2243] T 74231e641a7e4f7fb5bc39a2a084bb81 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:23.733665  9682 raft_consensus.cc:2272] T 74231e641a7e4f7fb5bc39a2a084bb81 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.736188  9682 tablet_replica.cc:333] T 7f2c531192bb4639a60df411741abc7b P 21a7bb1d6ef54b20befdf8ea6d293776: stopping tablet replica
I20260709 13:21:23.737005  9682 raft_consensus.cc:2243] T 7f2c531192bb4639a60df411741abc7b P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:23.737600  9682 raft_consensus.cc:2272] T 7f2c531192bb4639a60df411741abc7b P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:23.739362 22069 consensus_peers.cc:597] T 5d0c4853c48a42938548ac8810a0bbc4 P 21a7bb1d6ef54b20befdf8ea6d293776 -> Peer c48712233a8a4eb0b39bb05cd60137e6 (127.9.116.130:46529): Couldn't send request to peer c48712233a8a4eb0b39bb05cd60137e6. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.130:46529: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:23.740198  9682 tablet_replica.cc:333] T f9dceef9cf6541039db61f39e2798430 P 21a7bb1d6ef54b20befdf8ea6d293776: stopping tablet replica
I20260709 13:21:23.740875  9682 raft_consensus.cc:2243] T f9dceef9cf6541039db61f39e2798430 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:23.741348  9682 raft_consensus.cc:2272] T f9dceef9cf6541039db61f39e2798430 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.743603  9682 tablet_replica.cc:333] T d74919ac686e49e694ca111d3b1a27f6 P 21a7bb1d6ef54b20befdf8ea6d293776: stopping tablet replica
I20260709 13:21:23.744164  9682 raft_consensus.cc:2243] T d74919ac686e49e694ca111d3b1a27f6 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:23.744638  9682 raft_consensus.cc:2272] T d74919ac686e49e694ca111d3b1a27f6 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.746757  9682 tablet_replica.cc:333] T ba306baff7874796aba9a7c3fd5d628e P 21a7bb1d6ef54b20befdf8ea6d293776: stopping tablet replica
I20260709 13:21:23.747390  9682 raft_consensus.cc:2243] T ba306baff7874796aba9a7c3fd5d628e P 21a7bb1d6ef54b20befdf8ea6d293776 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:21:23.748148  9682 raft_consensus.cc:2272] T ba306baff7874796aba9a7c3fd5d628e P 21a7bb1d6ef54b20befdf8ea6d293776 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.750281  9682 tablet_replica.cc:333] T 5d0c4853c48a42938548ac8810a0bbc4 P 21a7bb1d6ef54b20befdf8ea6d293776: stopping tablet replica
I20260709 13:21:23.750862  9682 raft_consensus.cc:2243] T 5d0c4853c48a42938548ac8810a0bbc4 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:23.751694  9682 raft_consensus.cc:2272] T 5d0c4853c48a42938548ac8810a0bbc4 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.753774  9682 tablet_replica.cc:333] T b8791ec386d148c19bb6fb734287e432 P 21a7bb1d6ef54b20befdf8ea6d293776: stopping tablet replica
I20260709 13:21:23.754323  9682 raft_consensus.cc:2243] T b8791ec386d148c19bb6fb734287e432 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:23.754850  9682 raft_consensus.cc:2272] T b8791ec386d148c19bb6fb734287e432 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.756947  9682 tablet_replica.cc:333] T 1631e507d6714dc187fc9d7f2457aad9 P 21a7bb1d6ef54b20befdf8ea6d293776: stopping tablet replica
I20260709 13:21:23.757484  9682 raft_consensus.cc:2243] T 1631e507d6714dc187fc9d7f2457aad9 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:23.757943  9682 raft_consensus.cc:2272] T 1631e507d6714dc187fc9d7f2457aad9 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.760089  9682 tablet_replica.cc:333] T ea585ca33ebf4b329ce97d386bf3e7bc P 21a7bb1d6ef54b20befdf8ea6d293776: stopping tablet replica
I20260709 13:21:23.760624  9682 raft_consensus.cc:2243] T ea585ca33ebf4b329ce97d386bf3e7bc P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:23.761073  9682 raft_consensus.cc:2272] T ea585ca33ebf4b329ce97d386bf3e7bc P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.763185  9682 tablet_replica.cc:333] T 0ca42dd42e4d46359256d9256f64d657 P 21a7bb1d6ef54b20befdf8ea6d293776: stopping tablet replica
I20260709 13:21:23.763686  9682 raft_consensus.cc:2243] T 0ca42dd42e4d46359256d9256f64d657 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:23.764447  9682 raft_consensus.cc:2272] T 0ca42dd42e4d46359256d9256f64d657 P 21a7bb1d6ef54b20befdf8ea6d293776 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.804210  9682 tablet_server.cc:196] TabletServer@127.9.116.131:0 shutdown complete.
I20260709 13:21:23.838632  9682 master.cc:562] Master@127.9.116.190:39063 shutting down...
I20260709 13:21:23.861352  9682 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 848ddee1690240febd24df3095cdbbab [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:23.861924  9682 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 848ddee1690240febd24df3095cdbbab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.862274  9682 tablet_replica.cc:333] T 00000000000000000000000000000000 P 848ddee1690240febd24df3095cdbbab: stopping tablet replica
I20260709 13:21:23.883245  9682 master.cc:584] Master@127.9.116.190:39063 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (12375 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 13:21:23.937253  9682 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.9.116.190:36011
I20260709 13:21:23.938448  9682 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:21:23.944443 22401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:23.944525 22400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:23.945430 22403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:23.947316  9682 server_base.cc:1061] running on GCE node
I20260709 13:21:23.948212  9682 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:21:23.948401  9682 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:21:23.948513  9682 hybrid_clock.cc:648] HybridClock initialized: now 1783603283948502 us; error 0 us; skew 500 ppm
I20260709 13:21:23.948951  9682 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:21:23.951519  9682 webserver.cc:533] Webserver started at http://127.9.116.190:35997/ using document root <none> and password file <none>
I20260709 13:21:23.952041  9682 fs_manager.cc:362] Metadata directory not provided
I20260709 13:21:23.952193  9682 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:21:23.952419  9682 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:21:23.953492  9682 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602916530764-9682-0/minicluster-data/master-0-root/instance:
uuid: "542aff268f9b4b3abea229f5c457ad3c"
format_stamp: "Formatted at 2026-07-09 13:21:23 on dist-test-slave-1t6m"
I20260709 13:21:23.957958  9682 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.003s
I20260709 13:21:23.961277 22408 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:23.962045  9682 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:21:23.962307  9682 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602916530764-9682-0/minicluster-data/master-0-root
uuid: "542aff268f9b4b3abea229f5c457ad3c"
format_stamp: "Formatted at 2026-07-09 13:21:23 on dist-test-slave-1t6m"
I20260709 13:21:23.962548  9682 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602916530764-9682-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602916530764-9682-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602916530764-9682-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:21:23.972796  9682 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:21:23.974007  9682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:21:24.016537  9682 rpc_server.cc:307] RPC server started. Bound to: 127.9.116.190:36011
I20260709 13:21:24.016644 22459 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.116.190:36011 every 8 connection(s)
I20260709 13:21:24.020453 22460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:24.031634 22460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 542aff268f9b4b3abea229f5c457ad3c: Bootstrap starting.
I20260709 13:21:24.036665 22460 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 542aff268f9b4b3abea229f5c457ad3c: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:24.041040 22460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 542aff268f9b4b3abea229f5c457ad3c: No bootstrap required, opened a new log
I20260709 13:21:24.043365 22460 raft_consensus.cc:359] T 00000000000000000000000000000000 P 542aff268f9b4b3abea229f5c457ad3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "542aff268f9b4b3abea229f5c457ad3c" member_type: VOTER }
I20260709 13:21:24.043752 22460 raft_consensus.cc:385] T 00000000000000000000000000000000 P 542aff268f9b4b3abea229f5c457ad3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:24.043931 22460 raft_consensus.cc:740] T 00000000000000000000000000000000 P 542aff268f9b4b3abea229f5c457ad3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 542aff268f9b4b3abea229f5c457ad3c, State: Initialized, Role: FOLLOWER
I20260709 13:21:24.044492 22460 consensus_queue.cc:260] T 00000000000000000000000000000000 P 542aff268f9b4b3abea229f5c457ad3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "542aff268f9b4b3abea229f5c457ad3c" member_type: VOTER }
I20260709 13:21:24.044917 22460 raft_consensus.cc:399] T 00000000000000000000000000000000 P 542aff268f9b4b3abea229f5c457ad3c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:21:24.045105 22460 raft_consensus.cc:493] T 00000000000000000000000000000000 P 542aff268f9b4b3abea229f5c457ad3c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:21:24.045317 22460 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 542aff268f9b4b3abea229f5c457ad3c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:24.050371 22460 raft_consensus.cc:515] T 00000000000000000000000000000000 P 542aff268f9b4b3abea229f5c457ad3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "542aff268f9b4b3abea229f5c457ad3c" member_type: VOTER }
I20260709 13:21:24.050963 22460 leader_election.cc:304] T 00000000000000000000000000000000 P 542aff268f9b4b3abea229f5c457ad3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 542aff268f9b4b3abea229f5c457ad3c; no voters: 
I20260709 13:21:24.052074 22460 leader_election.cc:290] T 00000000000000000000000000000000 P 542aff268f9b4b3abea229f5c457ad3c [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:21:24.052464 22463 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 542aff268f9b4b3abea229f5c457ad3c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:24.053905 22463 raft_consensus.cc:697] T 00000000000000000000000000000000 P 542aff268f9b4b3abea229f5c457ad3c [term 1 LEADER]: Becoming Leader. State: Replica: 542aff268f9b4b3abea229f5c457ad3c, State: Running, Role: LEADER
I20260709 13:21:24.054620 22463 consensus_queue.cc:237] T 00000000000000000000000000000000 P 542aff268f9b4b3abea229f5c457ad3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "542aff268f9b4b3abea229f5c457ad3c" member_type: VOTER }
I20260709 13:21:24.055222 22460 sys_catalog.cc:565] T 00000000000000000000000000000000 P 542aff268f9b4b3abea229f5c457ad3c [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:21:24.059568 22465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 542aff268f9b4b3abea229f5c457ad3c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 542aff268f9b4b3abea229f5c457ad3c. Latest consensus state: current_term: 1 leader_uuid: "542aff268f9b4b3abea229f5c457ad3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "542aff268f9b4b3abea229f5c457ad3c" member_type: VOTER } }
I20260709 13:21:24.059770 22464 sys_catalog.cc:455] T 00000000000000000000000000000000 P 542aff268f9b4b3abea229f5c457ad3c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "542aff268f9b4b3abea229f5c457ad3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "542aff268f9b4b3abea229f5c457ad3c" member_type: VOTER } }
I20260709 13:21:24.060370 22465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 542aff268f9b4b3abea229f5c457ad3c [sys.catalog]: This master's current role is: LEADER
I20260709 13:21:24.060400 22464 sys_catalog.cc:458] T 00000000000000000000000000000000 P 542aff268f9b4b3abea229f5c457ad3c [sys.catalog]: This master's current role is: LEADER
I20260709 13:21:24.066735 22471 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:21:24.074874 22471 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:21:24.075946  9682 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:21:24.084179 22471 catalog_manager.cc:1383] Generated new cluster ID: c4de4d4d18e742e39396b46e9db029c6
I20260709 13:21:24.084501 22471 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:21:24.118530 22471 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:21:24.119930 22471 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:21:24.141180 22471 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 542aff268f9b4b3abea229f5c457ad3c: Generated new TSK 0
I20260709 13:21:24.141812 22471 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:21:24.207528  9682 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:21:24.213531 22481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:24.215435 22482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:24.218734 22484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:24.219064  9682 server_base.cc:1061] running on GCE node
I20260709 13:21:24.220062  9682 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:21:24.220270  9682 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:21:24.220459  9682 hybrid_clock.cc:648] HybridClock initialized: now 1783603284220423 us; error 0 us; skew 500 ppm
I20260709 13:21:24.221004  9682 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:21:24.223378  9682 webserver.cc:533] Webserver started at http://127.9.116.129:45951/ using document root <none> and password file <none>
I20260709 13:21:24.223872  9682 fs_manager.cc:362] Metadata directory not provided
I20260709 13:21:24.224071  9682 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:21:24.224346  9682 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:21:24.225435  9682 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602916530764-9682-0/minicluster-data/ts-0-root/instance:
uuid: "7df364d160d64ac4b5cdc4f004324f8b"
format_stamp: "Formatted at 2026-07-09 13:21:24 on dist-test-slave-1t6m"
I20260709 13:21:24.229908  9682 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:21:24.233300 22489 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:24.234016  9682 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:21:24.234318  9682 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602916530764-9682-0/minicluster-data/ts-0-root
uuid: "7df364d160d64ac4b5cdc4f004324f8b"
format_stamp: "Formatted at 2026-07-09 13:21:24 on dist-test-slave-1t6m"
I20260709 13:21:24.234611  9682 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602916530764-9682-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602916530764-9682-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602916530764-9682-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:21:24.262332  9682 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:21:24.263736  9682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:21:24.265322  9682 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:21:24.267678  9682 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:21:24.267875  9682 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:24.268193  9682 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:21:24.268380  9682 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:24.309228  9682 rpc_server.cc:307] RPC server started. Bound to: 127.9.116.129:42823
I20260709 13:21:24.309336 22551 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.116.129:42823 every 8 connection(s)
I20260709 13:21:24.314522  9682 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:21:24.321511 22556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:24.322505 22557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:24.326541  9682 server_base.cc:1061] running on GCE node
W20260709 13:21:24.326836 22559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:24.328117 22552 heartbeater.cc:344] Connected to a master server at 127.9.116.190:36011
I20260709 13:21:24.328343  9682 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:21:24.328541 22552 heartbeater.cc:461] Registering TS with master...
W20260709 13:21:24.328655  9682 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:21:24.328989  9682 hybrid_clock.cc:648] HybridClock initialized: now 1783603284328969 us; error 0 us; skew 500 ppm
I20260709 13:21:24.329524 22552 heartbeater.cc:507] Master 127.9.116.190:36011 requested a full tablet report, sending...
I20260709 13:21:24.329591  9682 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:21:24.332062 22425 ts_manager.cc:194] Registered new tserver with Master: 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823)
I20260709 13:21:24.332589  9682 webserver.cc:533] Webserver started at http://127.9.116.130:42059/ using document root <none> and password file <none>
I20260709 13:21:24.333142  9682 fs_manager.cc:362] Metadata directory not provided
I20260709 13:21:24.333351  9682 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:21:24.333624  9682 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:21:24.334189 22425 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51480
I20260709 13:21:24.334964  9682 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602916530764-9682-0/minicluster-data/ts-1-root/instance:
uuid: "e20f6c83ae144d698c26b94c2eda023f"
format_stamp: "Formatted at 2026-07-09 13:21:24 on dist-test-slave-1t6m"
I20260709 13:21:24.339919  9682 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.003s
I20260709 13:21:24.343319 22564 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:24.344136  9682 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:21:24.344462  9682 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602916530764-9682-0/minicluster-data/ts-1-root
uuid: "e20f6c83ae144d698c26b94c2eda023f"
format_stamp: "Formatted at 2026-07-09 13:21:24 on dist-test-slave-1t6m"
I20260709 13:21:24.344772  9682 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602916530764-9682-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602916530764-9682-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602916530764-9682-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:21:24.363716  9682 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:21:24.365031  9682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:21:24.366514  9682 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:21:24.368835  9682 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:21:24.369065  9682 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:24.369318  9682 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:21:24.369488  9682 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:24.410070  9682 rpc_server.cc:307] RPC server started. Bound to: 127.9.116.130:43469
I20260709 13:21:24.410178 22626 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.116.130:43469 every 8 connection(s)
I20260709 13:21:24.414963  9682 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:21:24.421577 22630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:24.422881 22631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:24.426311  9682 server_base.cc:1061] running on GCE node
W20260709 13:21:24.426505 22633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:24.427309 22627 heartbeater.cc:344] Connected to a master server at 127.9.116.190:36011
I20260709 13:21:24.427728 22627 heartbeater.cc:461] Registering TS with master...
I20260709 13:21:24.427793  9682 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:21:24.428134  9682 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:21:24.428328  9682 hybrid_clock.cc:648] HybridClock initialized: now 1783603284428306 us; error 0 us; skew 500 ppm
I20260709 13:21:24.428628 22627 heartbeater.cc:507] Master 127.9.116.190:36011 requested a full tablet report, sending...
I20260709 13:21:24.429013  9682 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:21:24.431033 22425 ts_manager.cc:194] Registered new tserver with Master: e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469)
I20260709 13:21:24.431826  9682 webserver.cc:533] Webserver started at http://127.9.116.131:42489/ using document root <none> and password file <none>
I20260709 13:21:24.432374  9682 fs_manager.cc:362] Metadata directory not provided
I20260709 13:21:24.432569  9682 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:21:24.432696 22425 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51490
I20260709 13:21:24.432858  9682 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:21:24.434353  9682 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602916530764-9682-0/minicluster-data/ts-2-root/instance:
uuid: "2d62e0b282b44003a202b8dcbe6ea6ac"
format_stamp: "Formatted at 2026-07-09 13:21:24 on dist-test-slave-1t6m"
I20260709 13:21:24.439085  9682 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 13:21:24.442550 22638 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:24.443378  9682 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:21:24.443702  9682 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602916530764-9682-0/minicluster-data/ts-2-root
uuid: "2d62e0b282b44003a202b8dcbe6ea6ac"
format_stamp: "Formatted at 2026-07-09 13:21:24 on dist-test-slave-1t6m"
I20260709 13:21:24.444000  9682 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602916530764-9682-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602916530764-9682-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskMUb88c/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602916530764-9682-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:21:24.485396  9682 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:21:24.486651  9682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:21:24.488057  9682 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:21:24.490389  9682 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:21:24.490586  9682 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:24.490940  9682 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:21:24.491124  9682 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:24.532068  9682 rpc_server.cc:307] RPC server started. Bound to: 127.9.116.131:34335
I20260709 13:21:24.532184 22700 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.116.131:34335 every 8 connection(s)
I20260709 13:21:24.547394 22701 heartbeater.cc:344] Connected to a master server at 127.9.116.190:36011
I20260709 13:21:24.547881 22701 heartbeater.cc:461] Registering TS with master...
I20260709 13:21:24.548630 22701 heartbeater.cc:507] Master 127.9.116.190:36011 requested a full tablet report, sending...
I20260709 13:21:24.550648 22425 ts_manager.cc:194] Registered new tserver with Master: 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335)
I20260709 13:21:24.551661  9682 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.0159337s
I20260709 13:21:24.552220 22425 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51502
I20260709 13:21:24.581972 22425 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51508:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:21:24.584707 22425 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:21:24.656637 22592 tablet_service.cc:1511] Processing CreateTablet for tablet c0614d551fc342cead195ccaed2aeac3 (DEFAULT_TABLE table=test-workload [id=95ac8ea50a704d32b677b8fbc3324665]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:21:24.656909 22591 tablet_service.cc:1511] Processing CreateTablet for tablet 56bc1b5c8c3143998cb9921b469c071d (DEFAULT_TABLE table=test-workload [id=95ac8ea50a704d32b677b8fbc3324665]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:21:24.658181 22592 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0614d551fc342cead195ccaed2aeac3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:24.659253 22591 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56bc1b5c8c3143998cb9921b469c071d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:24.659301 22590 tablet_service.cc:1511] Processing CreateTablet for tablet dded1824a3ef49988cc1b941db0f9955 (DEFAULT_TABLE table=test-workload [id=95ac8ea50a704d32b677b8fbc3324665]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:21:24.660844 22590 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dded1824a3ef49988cc1b941db0f9955. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:24.661171 22666 tablet_service.cc:1511] Processing CreateTablet for tablet c0614d551fc342cead195ccaed2aeac3 (DEFAULT_TABLE table=test-workload [id=95ac8ea50a704d32b677b8fbc3324665]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:21:24.662570 22666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0614d551fc342cead195ccaed2aeac3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:24.667963 22665 tablet_service.cc:1511] Processing CreateTablet for tablet 56bc1b5c8c3143998cb9921b469c071d (DEFAULT_TABLE table=test-workload [id=95ac8ea50a704d32b677b8fbc3324665]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:21:24.669473 22665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56bc1b5c8c3143998cb9921b469c071d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:24.672009 22664 tablet_service.cc:1511] Processing CreateTablet for tablet dded1824a3ef49988cc1b941db0f9955 (DEFAULT_TABLE table=test-workload [id=95ac8ea50a704d32b677b8fbc3324665]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:21:24.673482 22664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dded1824a3ef49988cc1b941db0f9955. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:24.679687 22589 tablet_service.cc:1511] Processing CreateTablet for tablet 2e322f5288b64ccca277807d2dc46579 (DEFAULT_TABLE table=test-workload [id=95ac8ea50a704d32b677b8fbc3324665]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:21:24.681246 22589 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e322f5288b64ccca277807d2dc46579. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:24.691567 22663 tablet_service.cc:1511] Processing CreateTablet for tablet 2e322f5288b64ccca277807d2dc46579 (DEFAULT_TABLE table=test-workload [id=95ac8ea50a704d32b677b8fbc3324665]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:21:24.693675 22663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e322f5288b64ccca277807d2dc46579. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:24.693934 22588 tablet_service.cc:1511] Processing CreateTablet for tablet 5e4995abf28446589b7695a1b77ee513 (DEFAULT_TABLE table=test-workload [id=95ac8ea50a704d32b677b8fbc3324665]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:21:24.695573 22588 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e4995abf28446589b7695a1b77ee513. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:24.696447 22662 tablet_service.cc:1511] Processing CreateTablet for tablet 5a1c542f0ae74a718e0b1c70aa641da4 (DEFAULT_TABLE table=test-workload [id=95ac8ea50a704d32b677b8fbc3324665]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:21:24.698563 22662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a1c542f0ae74a718e0b1c70aa641da4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:24.701184 22661 tablet_service.cc:1511] Processing CreateTablet for tablet 5e4995abf28446589b7695a1b77ee513 (DEFAULT_TABLE table=test-workload [id=95ac8ea50a704d32b677b8fbc3324665]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:21:24.702605 22661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e4995abf28446589b7695a1b77ee513. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:24.709045 22590 tablet_service.cc:1511] Processing CreateTablet for tablet 68d74bd4ad624659b92ec79d4560a02c (DEFAULT_TABLE table=test-workload [id=95ac8ea50a704d32b677b8fbc3324665]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:21:24.710561 22590 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68d74bd4ad624659b92ec79d4560a02c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:24.713035 22587 tablet_service.cc:1511] Processing CreateTablet for tablet a9c5f91cbe6f4a6abbd799e94e0985e1 (DEFAULT_TABLE table=test-workload [id=95ac8ea50a704d32b677b8fbc3324665]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:21:24.716760 22587 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9c5f91cbe6f4a6abbd799e94e0985e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:24.718031 22586 tablet_service.cc:1511] Processing CreateTablet for tablet 2dad40e2d06244cebe50ec7433000752 (DEFAULT_TABLE table=test-workload [id=95ac8ea50a704d32b677b8fbc3324665]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:21:24.719519 22586 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2dad40e2d06244cebe50ec7433000752. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:24.724579 22660 tablet_service.cc:1511] Processing CreateTablet for tablet 68d74bd4ad624659b92ec79d4560a02c (DEFAULT_TABLE table=test-workload [id=95ac8ea50a704d32b677b8fbc3324665]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:21:24.726158 22660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68d74bd4ad624659b92ec79d4560a02c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:24.732093 22659 tablet_service.cc:1511] Processing CreateTablet for tablet a9c5f91cbe6f4a6abbd799e94e0985e1 (DEFAULT_TABLE table=test-workload [id=95ac8ea50a704d32b677b8fbc3324665]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:21:24.735745 22659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9c5f91cbe6f4a6abbd799e94e0985e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:24.691237 22591 tablet_service.cc:1511] Processing CreateTablet for tablet 5a1c542f0ae74a718e0b1c70aa641da4 (DEFAULT_TABLE table=test-workload [id=95ac8ea50a704d32b677b8fbc3324665]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:21:24.743855 22591 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a1c542f0ae74a718e0b1c70aa641da4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:24.754235 22658 tablet_service.cc:1511] Processing CreateTablet for tablet 2dad40e2d06244cebe50ec7433000752 (DEFAULT_TABLE table=test-workload [id=95ac8ea50a704d32b677b8fbc3324665]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:21:24.755836 22658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2dad40e2d06244cebe50ec7433000752. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:24.771943 22517 tablet_service.cc:1511] Processing CreateTablet for tablet c0614d551fc342cead195ccaed2aeac3 (DEFAULT_TABLE table=test-workload [id=95ac8ea50a704d32b677b8fbc3324665]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:21:24.774020 22517 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0614d551fc342cead195ccaed2aeac3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:24.774101 22516 tablet_service.cc:1511] Processing CreateTablet for tablet 56bc1b5c8c3143998cb9921b469c071d (DEFAULT_TABLE table=test-workload [id=95ac8ea50a704d32b677b8fbc3324665]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:21:24.775681 22516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56bc1b5c8c3143998cb9921b469c071d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:24.780064 22512 tablet_service.cc:1511] Processing CreateTablet for tablet 5e4995abf28446589b7695a1b77ee513 (DEFAULT_TABLE table=test-workload [id=95ac8ea50a704d32b677b8fbc3324665]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:21:24.781626 22512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e4995abf28446589b7695a1b77ee513. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:24.783951 22513 tablet_service.cc:1511] Processing CreateTablet for tablet 5a1c542f0ae74a718e0b1c70aa641da4 (DEFAULT_TABLE table=test-workload [id=95ac8ea50a704d32b677b8fbc3324665]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:21:24.785426 22513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a1c542f0ae74a718e0b1c70aa641da4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:24.787573 22514 tablet_service.cc:1511] Processing CreateTablet for tablet 2e322f5288b64ccca277807d2dc46579 (DEFAULT_TABLE table=test-workload [id=95ac8ea50a704d32b677b8fbc3324665]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:21:24.789069 22514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e322f5288b64ccca277807d2dc46579. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:24.792071 22515 tablet_service.cc:1511] Processing CreateTablet for tablet dded1824a3ef49988cc1b941db0f9955 (DEFAULT_TABLE table=test-workload [id=95ac8ea50a704d32b677b8fbc3324665]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:21:24.793762 22515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dded1824a3ef49988cc1b941db0f9955. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:24.796001 22511 tablet_service.cc:1511] Processing CreateTablet for tablet 68d74bd4ad624659b92ec79d4560a02c (DEFAULT_TABLE table=test-workload [id=95ac8ea50a704d32b677b8fbc3324665]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:21:24.802304 22510 tablet_service.cc:1511] Processing CreateTablet for tablet a9c5f91cbe6f4a6abbd799e94e0985e1 (DEFAULT_TABLE table=test-workload [id=95ac8ea50a704d32b677b8fbc3324665]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:21:24.804838 22509 tablet_service.cc:1511] Processing CreateTablet for tablet 2dad40e2d06244cebe50ec7433000752 (DEFAULT_TABLE table=test-workload [id=95ac8ea50a704d32b677b8fbc3324665]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:21:24.811435 22511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68d74bd4ad624659b92ec79d4560a02c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:24.813966 22509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2dad40e2d06244cebe50ec7433000752. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:24.815095 22716 tablet_bootstrap.cc:492] T c0614d551fc342cead195ccaed2aeac3 P e20f6c83ae144d698c26b94c2eda023f: Bootstrap starting.
I20260709 13:21:24.817422 22510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9c5f91cbe6f4a6abbd799e94e0985e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:24.822055 22716 tablet_bootstrap.cc:654] T c0614d551fc342cead195ccaed2aeac3 P e20f6c83ae144d698c26b94c2eda023f: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:24.870147 22717 tablet_bootstrap.cc:492] T 56bc1b5c8c3143998cb9921b469c071d P 2d62e0b282b44003a202b8dcbe6ea6ac: Bootstrap starting.
I20260709 13:21:24.881480 22717 tablet_bootstrap.cc:654] T 56bc1b5c8c3143998cb9921b469c071d P 2d62e0b282b44003a202b8dcbe6ea6ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:24.880424 22719 tablet_bootstrap.cc:492] T 56bc1b5c8c3143998cb9921b469c071d P 7df364d160d64ac4b5cdc4f004324f8b: Bootstrap starting.
I20260709 13:21:24.902504 22719 tablet_bootstrap.cc:654] T 56bc1b5c8c3143998cb9921b469c071d P 7df364d160d64ac4b5cdc4f004324f8b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:24.908792 22716 tablet_bootstrap.cc:492] T c0614d551fc342cead195ccaed2aeac3 P e20f6c83ae144d698c26b94c2eda023f: No bootstrap required, opened a new log
I20260709 13:21:24.909334 22716 ts_tablet_manager.cc:1403] T c0614d551fc342cead195ccaed2aeac3 P e20f6c83ae144d698c26b94c2eda023f: Time spent bootstrapping tablet: real 0.095s	user 0.014s	sys 0.004s
I20260709 13:21:24.912292 22716 raft_consensus.cc:359] T c0614d551fc342cead195ccaed2aeac3 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } } peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } }
I20260709 13:21:24.913316 22716 raft_consensus.cc:385] T c0614d551fc342cead195ccaed2aeac3 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:24.913724 22716 raft_consensus.cc:740] T c0614d551fc342cead195ccaed2aeac3 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e20f6c83ae144d698c26b94c2eda023f, State: Initialized, Role: FOLLOWER
I20260709 13:21:24.914760 22716 consensus_queue.cc:260] T c0614d551fc342cead195ccaed2aeac3 P e20f6c83ae144d698c26b94c2eda023f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } } peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } }
I20260709 13:21:24.916196 22717 tablet_bootstrap.cc:492] T 56bc1b5c8c3143998cb9921b469c071d P 2d62e0b282b44003a202b8dcbe6ea6ac: No bootstrap required, opened a new log
I20260709 13:21:24.917013 22717 ts_tablet_manager.cc:1403] T 56bc1b5c8c3143998cb9921b469c071d P 2d62e0b282b44003a202b8dcbe6ea6ac: Time spent bootstrapping tablet: real 0.047s	user 0.014s	sys 0.019s
I20260709 13:21:24.919260 22719 tablet_bootstrap.cc:492] T 56bc1b5c8c3143998cb9921b469c071d P 7df364d160d64ac4b5cdc4f004324f8b: No bootstrap required, opened a new log
I20260709 13:21:24.919936 22719 ts_tablet_manager.cc:1403] T 56bc1b5c8c3143998cb9921b469c071d P 7df364d160d64ac4b5cdc4f004324f8b: Time spent bootstrapping tablet: real 0.040s	user 0.018s	sys 0.000s
I20260709 13:21:24.920310 22627 heartbeater.cc:499] Master 127.9.116.190:36011 was elected leader, sending a full tablet report...
I20260709 13:21:24.920181 22717 raft_consensus.cc:359] T 56bc1b5c8c3143998cb9921b469c071d P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:24.921084 22717 raft_consensus.cc:385] T 56bc1b5c8c3143998cb9921b469c071d P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:24.921403 22717 raft_consensus.cc:740] T 56bc1b5c8c3143998cb9921b469c071d P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d62e0b282b44003a202b8dcbe6ea6ac, State: Initialized, Role: FOLLOWER
I20260709 13:21:24.922200 22717 consensus_queue.cc:260] T 56bc1b5c8c3143998cb9921b469c071d P 2d62e0b282b44003a202b8dcbe6ea6ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:24.923105 22719 raft_consensus.cc:359] T 56bc1b5c8c3143998cb9921b469c071d P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:24.923821 22716 ts_tablet_manager.cc:1434] T c0614d551fc342cead195ccaed2aeac3 P e20f6c83ae144d698c26b94c2eda023f: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.005s
I20260709 13:21:24.924120 22719 raft_consensus.cc:385] T 56bc1b5c8c3143998cb9921b469c071d P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:24.924640 22719 raft_consensus.cc:740] T 56bc1b5c8c3143998cb9921b469c071d P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7df364d160d64ac4b5cdc4f004324f8b, State: Initialized, Role: FOLLOWER
I20260709 13:21:24.925288 22716 tablet_bootstrap.cc:492] T 56bc1b5c8c3143998cb9921b469c071d P e20f6c83ae144d698c26b94c2eda023f: Bootstrap starting.
I20260709 13:21:24.925647 22719 consensus_queue.cc:260] T 56bc1b5c8c3143998cb9921b469c071d P 7df364d160d64ac4b5cdc4f004324f8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:24.931423 22701 heartbeater.cc:499] Master 127.9.116.190:36011 was elected leader, sending a full tablet report...
I20260709 13:21:24.932201 22717 ts_tablet_manager.cc:1434] T 56bc1b5c8c3143998cb9921b469c071d P 2d62e0b282b44003a202b8dcbe6ea6ac: Time spent starting tablet: real 0.015s	user 0.013s	sys 0.001s
I20260709 13:21:24.933004 22716 tablet_bootstrap.cc:654] T 56bc1b5c8c3143998cb9921b469c071d P e20f6c83ae144d698c26b94c2eda023f: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:24.933406 22717 tablet_bootstrap.cc:492] T 68d74bd4ad624659b92ec79d4560a02c P 2d62e0b282b44003a202b8dcbe6ea6ac: Bootstrap starting.
I20260709 13:21:24.934022 22719 ts_tablet_manager.cc:1434] T 56bc1b5c8c3143998cb9921b469c071d P 7df364d160d64ac4b5cdc4f004324f8b: Time spent starting tablet: real 0.014s	user 0.011s	sys 0.000s
I20260709 13:21:24.934810 22552 heartbeater.cc:499] Master 127.9.116.190:36011 was elected leader, sending a full tablet report...
I20260709 13:21:24.940102 22717 tablet_bootstrap.cc:654] T 68d74bd4ad624659b92ec79d4560a02c P 2d62e0b282b44003a202b8dcbe6ea6ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:24.943888 22719 tablet_bootstrap.cc:492] T c0614d551fc342cead195ccaed2aeac3 P 7df364d160d64ac4b5cdc4f004324f8b: Bootstrap starting.
I20260709 13:21:24.952831 22719 tablet_bootstrap.cc:654] T c0614d551fc342cead195ccaed2aeac3 P 7df364d160d64ac4b5cdc4f004324f8b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:24.953908 22717 tablet_bootstrap.cc:492] T 68d74bd4ad624659b92ec79d4560a02c P 2d62e0b282b44003a202b8dcbe6ea6ac: No bootstrap required, opened a new log
I20260709 13:21:24.954547 22717 ts_tablet_manager.cc:1403] T 68d74bd4ad624659b92ec79d4560a02c P 2d62e0b282b44003a202b8dcbe6ea6ac: Time spent bootstrapping tablet: real 0.021s	user 0.015s	sys 0.002s
I20260709 13:21:24.958105 22717 raft_consensus.cc:359] T 68d74bd4ad624659b92ec79d4560a02c P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:24.959136 22717 raft_consensus.cc:385] T 68d74bd4ad624659b92ec79d4560a02c P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:24.959571 22717 raft_consensus.cc:740] T 68d74bd4ad624659b92ec79d4560a02c P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d62e0b282b44003a202b8dcbe6ea6ac, State: Initialized, Role: FOLLOWER
I20260709 13:21:24.960554 22717 consensus_queue.cc:260] T 68d74bd4ad624659b92ec79d4560a02c P 2d62e0b282b44003a202b8dcbe6ea6ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:24.958580 22716 tablet_bootstrap.cc:492] T 56bc1b5c8c3143998cb9921b469c071d P e20f6c83ae144d698c26b94c2eda023f: No bootstrap required, opened a new log
I20260709 13:21:24.963163 22716 ts_tablet_manager.cc:1403] T 56bc1b5c8c3143998cb9921b469c071d P e20f6c83ae144d698c26b94c2eda023f: Time spent bootstrapping tablet: real 0.038s	user 0.023s	sys 0.009s
I20260709 13:21:24.963508 22719 tablet_bootstrap.cc:492] T c0614d551fc342cead195ccaed2aeac3 P 7df364d160d64ac4b5cdc4f004324f8b: No bootstrap required, opened a new log
I20260709 13:21:24.964176 22719 ts_tablet_manager.cc:1403] T c0614d551fc342cead195ccaed2aeac3 P 7df364d160d64ac4b5cdc4f004324f8b: Time spent bootstrapping tablet: real 0.021s	user 0.016s	sys 0.000s
I20260709 13:21:24.966411 22716 raft_consensus.cc:359] T 56bc1b5c8c3143998cb9921b469c071d P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:24.967341 22717 ts_tablet_manager.cc:1434] T 68d74bd4ad624659b92ec79d4560a02c P 2d62e0b282b44003a202b8dcbe6ea6ac: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.001s
I20260709 13:21:24.967231 22719 raft_consensus.cc:359] T c0614d551fc342cead195ccaed2aeac3 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } } peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } }
I20260709 13:21:24.968169 22719 raft_consensus.cc:385] T c0614d551fc342cead195ccaed2aeac3 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:24.968495 22719 raft_consensus.cc:740] T c0614d551fc342cead195ccaed2aeac3 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7df364d160d64ac4b5cdc4f004324f8b, State: Initialized, Role: FOLLOWER
I20260709 13:21:24.967257 22716 raft_consensus.cc:385] T 56bc1b5c8c3143998cb9921b469c071d P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:24.969004 22716 raft_consensus.cc:740] T 56bc1b5c8c3143998cb9921b469c071d P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e20f6c83ae144d698c26b94c2eda023f, State: Initialized, Role: FOLLOWER
I20260709 13:21:24.969573 22719 consensus_queue.cc:260] T c0614d551fc342cead195ccaed2aeac3 P 7df364d160d64ac4b5cdc4f004324f8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } } peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } }
I20260709 13:21:24.969743 22716 consensus_queue.cc:260] T 56bc1b5c8c3143998cb9921b469c071d P e20f6c83ae144d698c26b94c2eda023f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:24.968495 22717 tablet_bootstrap.cc:492] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac: Bootstrap starting.
I20260709 13:21:24.972326 22719 ts_tablet_manager.cc:1434] T c0614d551fc342cead195ccaed2aeac3 P 7df364d160d64ac4b5cdc4f004324f8b: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:21:24.973407 22719 tablet_bootstrap.cc:492] T 2dad40e2d06244cebe50ec7433000752 P 7df364d160d64ac4b5cdc4f004324f8b: Bootstrap starting.
I20260709 13:21:24.978292 22717 tablet_bootstrap.cc:654] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:24.980906 22719 tablet_bootstrap.cc:654] T 2dad40e2d06244cebe50ec7433000752 P 7df364d160d64ac4b5cdc4f004324f8b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:24.982895 22716 ts_tablet_manager.cc:1434] T 56bc1b5c8c3143998cb9921b469c071d P e20f6c83ae144d698c26b94c2eda023f: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 13:21:24.990090 22716 tablet_bootstrap.cc:492] T dded1824a3ef49988cc1b941db0f9955 P e20f6c83ae144d698c26b94c2eda023f: Bootstrap starting.
I20260709 13:21:24.998618 22719 tablet_bootstrap.cc:492] T 2dad40e2d06244cebe50ec7433000752 P 7df364d160d64ac4b5cdc4f004324f8b: No bootstrap required, opened a new log
I20260709 13:21:24.999377 22716 tablet_bootstrap.cc:654] T dded1824a3ef49988cc1b941db0f9955 P e20f6c83ae144d698c26b94c2eda023f: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:24.999418 22719 ts_tablet_manager.cc:1403] T 2dad40e2d06244cebe50ec7433000752 P 7df364d160d64ac4b5cdc4f004324f8b: Time spent bootstrapping tablet: real 0.026s	user 0.015s	sys 0.009s
I20260709 13:21:25.003744 22717 tablet_bootstrap.cc:492] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac: No bootstrap required, opened a new log
I20260709 13:21:25.003291 22719 raft_consensus.cc:359] T 2dad40e2d06244cebe50ec7433000752 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.004350 22719 raft_consensus.cc:385] T 2dad40e2d06244cebe50ec7433000752 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:25.004817 22719 raft_consensus.cc:740] T 2dad40e2d06244cebe50ec7433000752 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7df364d160d64ac4b5cdc4f004324f8b, State: Initialized, Role: FOLLOWER
I20260709 13:21:25.004361 22717 ts_tablet_manager.cc:1403] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac: Time spent bootstrapping tablet: real 0.036s	user 0.012s	sys 0.000s
I20260709 13:21:25.005678 22719 consensus_queue.cc:260] T 2dad40e2d06244cebe50ec7433000752 P 7df364d160d64ac4b5cdc4f004324f8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.007644 22716 tablet_bootstrap.cc:492] T dded1824a3ef49988cc1b941db0f9955 P e20f6c83ae144d698c26b94c2eda023f: No bootstrap required, opened a new log
I20260709 13:21:25.008284 22716 ts_tablet_manager.cc:1403] T dded1824a3ef49988cc1b941db0f9955 P e20f6c83ae144d698c26b94c2eda023f: Time spent bootstrapping tablet: real 0.022s	user 0.017s	sys 0.000s
I20260709 13:21:25.009764 22717 raft_consensus.cc:359] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.010565 22717 raft_consensus.cc:385] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:25.010919 22717 raft_consensus.cc:740] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d62e0b282b44003a202b8dcbe6ea6ac, State: Initialized, Role: FOLLOWER
I20260709 13:21:25.011324 22719 ts_tablet_manager.cc:1434] T 2dad40e2d06244cebe50ec7433000752 P 7df364d160d64ac4b5cdc4f004324f8b: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:21:25.011729 22716 raft_consensus.cc:359] T dded1824a3ef49988cc1b941db0f9955 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.012503 22719 tablet_bootstrap.cc:492] T dded1824a3ef49988cc1b941db0f9955 P 7df364d160d64ac4b5cdc4f004324f8b: Bootstrap starting.
I20260709 13:21:25.013780 22716 raft_consensus.cc:385] T dded1824a3ef49988cc1b941db0f9955 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:25.011729 22717 consensus_queue.cc:260] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.014218 22716 raft_consensus.cc:740] T dded1824a3ef49988cc1b941db0f9955 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e20f6c83ae144d698c26b94c2eda023f, State: Initialized, Role: FOLLOWER
I20260709 13:21:25.015201 22716 consensus_queue.cc:260] T dded1824a3ef49988cc1b941db0f9955 P e20f6c83ae144d698c26b94c2eda023f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.015888 22717 ts_tablet_manager.cc:1434] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.004s
I20260709 13:21:25.016959 22717 tablet_bootstrap.cc:492] T dded1824a3ef49988cc1b941db0f9955 P 2d62e0b282b44003a202b8dcbe6ea6ac: Bootstrap starting.
I20260709 13:21:25.020077 22716 ts_tablet_manager.cc:1434] T dded1824a3ef49988cc1b941db0f9955 P e20f6c83ae144d698c26b94c2eda023f: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:21:25.021219 22716 tablet_bootstrap.cc:492] T 2dad40e2d06244cebe50ec7433000752 P e20f6c83ae144d698c26b94c2eda023f: Bootstrap starting.
I20260709 13:21:25.026541 22719 tablet_bootstrap.cc:654] T dded1824a3ef49988cc1b941db0f9955 P 7df364d160d64ac4b5cdc4f004324f8b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:25.028263 22716 tablet_bootstrap.cc:654] T 2dad40e2d06244cebe50ec7433000752 P e20f6c83ae144d698c26b94c2eda023f: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:25.034775 22717 tablet_bootstrap.cc:654] T dded1824a3ef49988cc1b941db0f9955 P 2d62e0b282b44003a202b8dcbe6ea6ac: Neither blocks nor log segments found. Creating new log.
W20260709 13:21:25.036635 22702 tablet.cc:2406] T 56bc1b5c8c3143998cb9921b469c071d P 2d62e0b282b44003a202b8dcbe6ea6ac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:21:25.036648 22716 tablet_bootstrap.cc:492] T 2dad40e2d06244cebe50ec7433000752 P e20f6c83ae144d698c26b94c2eda023f: No bootstrap required, opened a new log
I20260709 13:21:25.038364 22716 ts_tablet_manager.cc:1403] T 2dad40e2d06244cebe50ec7433000752 P e20f6c83ae144d698c26b94c2eda023f: Time spent bootstrapping tablet: real 0.017s	user 0.013s	sys 0.000s
I20260709 13:21:25.039666 22719 tablet_bootstrap.cc:492] T dded1824a3ef49988cc1b941db0f9955 P 7df364d160d64ac4b5cdc4f004324f8b: No bootstrap required, opened a new log
I20260709 13:21:25.040437 22719 ts_tablet_manager.cc:1403] T dded1824a3ef49988cc1b941db0f9955 P 7df364d160d64ac4b5cdc4f004324f8b: Time spent bootstrapping tablet: real 0.028s	user 0.012s	sys 0.000s
I20260709 13:21:25.040956 22716 raft_consensus.cc:359] T 2dad40e2d06244cebe50ec7433000752 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.041608 22716 raft_consensus.cc:385] T 2dad40e2d06244cebe50ec7433000752 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:25.041865 22716 raft_consensus.cc:740] T 2dad40e2d06244cebe50ec7433000752 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e20f6c83ae144d698c26b94c2eda023f, State: Initialized, Role: FOLLOWER
I20260709 13:21:25.042585 22716 consensus_queue.cc:260] T 2dad40e2d06244cebe50ec7433000752 P e20f6c83ae144d698c26b94c2eda023f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.044718 22716 ts_tablet_manager.cc:1434] T 2dad40e2d06244cebe50ec7433000752 P e20f6c83ae144d698c26b94c2eda023f: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.002s
I20260709 13:21:25.044274 22719 raft_consensus.cc:359] T dded1824a3ef49988cc1b941db0f9955 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.045162 22719 raft_consensus.cc:385] T dded1824a3ef49988cc1b941db0f9955 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:25.045488 22719 raft_consensus.cc:740] T dded1824a3ef49988cc1b941db0f9955 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7df364d160d64ac4b5cdc4f004324f8b, State: Initialized, Role: FOLLOWER
I20260709 13:21:25.045935 22716 tablet_bootstrap.cc:492] T 2e322f5288b64ccca277807d2dc46579 P e20f6c83ae144d698c26b94c2eda023f: Bootstrap starting.
I20260709 13:21:25.046453 22719 consensus_queue.cc:260] T dded1824a3ef49988cc1b941db0f9955 P 7df364d160d64ac4b5cdc4f004324f8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.047508 22724 raft_consensus.cc:493] T 2dad40e2d06244cebe50ec7433000752 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:25.048135 22724 raft_consensus.cc:515] T 2dad40e2d06244cebe50ec7433000752 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.053937 22716 tablet_bootstrap.cc:654] T 2e322f5288b64ccca277807d2dc46579 P e20f6c83ae144d698c26b94c2eda023f: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:25.059665 22724 leader_election.cc:290] T 2dad40e2d06244cebe50ec7433000752 P 7df364d160d64ac4b5cdc4f004324f8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469), 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335)
I20260709 13:21:25.062928 22719 ts_tablet_manager.cc:1434] T dded1824a3ef49988cc1b941db0f9955 P 7df364d160d64ac4b5cdc4f004324f8b: Time spent starting tablet: real 0.022s	user 0.014s	sys 0.010s
I20260709 13:21:25.064509 22719 tablet_bootstrap.cc:492] T 68d74bd4ad624659b92ec79d4560a02c P 7df364d160d64ac4b5cdc4f004324f8b: Bootstrap starting.
I20260709 13:21:25.066653 22717 tablet_bootstrap.cc:492] T dded1824a3ef49988cc1b941db0f9955 P 2d62e0b282b44003a202b8dcbe6ea6ac: No bootstrap required, opened a new log
W20260709 13:21:25.069011 22473 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:21:25.070473 22717 ts_tablet_manager.cc:1403] T dded1824a3ef49988cc1b941db0f9955 P 2d62e0b282b44003a202b8dcbe6ea6ac: Time spent bootstrapping tablet: real 0.054s	user 0.013s	sys 0.005s
I20260709 13:21:25.073145 22719 tablet_bootstrap.cc:654] T 68d74bd4ad624659b92ec79d4560a02c P 7df364d160d64ac4b5cdc4f004324f8b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:25.074354 22722 raft_consensus.cc:493] T dded1824a3ef49988cc1b941db0f9955 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:25.075019 22722 raft_consensus.cc:515] T dded1824a3ef49988cc1b941db0f9955 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.076682 22716 tablet_bootstrap.cc:492] T 2e322f5288b64ccca277807d2dc46579 P e20f6c83ae144d698c26b94c2eda023f: No bootstrap required, opened a new log
I20260709 13:21:25.076920 22717 raft_consensus.cc:359] T dded1824a3ef49988cc1b941db0f9955 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.077409 22716 ts_tablet_manager.cc:1403] T 2e322f5288b64ccca277807d2dc46579 P e20f6c83ae144d698c26b94c2eda023f: Time spent bootstrapping tablet: real 0.032s	user 0.011s	sys 0.008s
I20260709 13:21:25.077726 22717 raft_consensus.cc:385] T dded1824a3ef49988cc1b941db0f9955 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:25.078053 22717 raft_consensus.cc:740] T dded1824a3ef49988cc1b941db0f9955 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d62e0b282b44003a202b8dcbe6ea6ac, State: Initialized, Role: FOLLOWER
I20260709 13:21:25.081049 22716 raft_consensus.cc:359] T 2e322f5288b64ccca277807d2dc46579 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.081954 22716 raft_consensus.cc:385] T 2e322f5288b64ccca277807d2dc46579 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:25.082327 22716 raft_consensus.cc:740] T 2e322f5288b64ccca277807d2dc46579 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e20f6c83ae144d698c26b94c2eda023f, State: Initialized, Role: FOLLOWER
I20260709 13:21:25.083154 22722 leader_election.cc:290] T dded1824a3ef49988cc1b941db0f9955 P e20f6c83ae144d698c26b94c2eda023f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823), 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335)
I20260709 13:21:25.083345 22716 consensus_queue.cc:260] T 2e322f5288b64ccca277807d2dc46579 P e20f6c83ae144d698c26b94c2eda023f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.086076 22716 ts_tablet_manager.cc:1434] T 2e322f5288b64ccca277807d2dc46579 P e20f6c83ae144d698c26b94c2eda023f: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:21:25.087532 22716 tablet_bootstrap.cc:492] T a9c5f91cbe6f4a6abbd799e94e0985e1 P e20f6c83ae144d698c26b94c2eda023f: Bootstrap starting.
I20260709 13:21:25.089449 22676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dad40e2d06244cebe50ec7433000752" candidate_uuid: "7df364d160d64ac4b5cdc4f004324f8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" is_pre_election: true
W20260709 13:21:25.091370 22493 leader_election.cc:343] T 2dad40e2d06244cebe50ec7433000752 P 7df364d160d64ac4b5cdc4f004324f8b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:25.079352 22717 consensus_queue.cc:260] T dded1824a3ef49988cc1b941db0f9955 P 2d62e0b282b44003a202b8dcbe6ea6ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.094498 22716 tablet_bootstrap.cc:654] T a9c5f91cbe6f4a6abbd799e94e0985e1 P e20f6c83ae144d698c26b94c2eda023f: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:25.097047 22717 ts_tablet_manager.cc:1434] T dded1824a3ef49988cc1b941db0f9955 P 2d62e0b282b44003a202b8dcbe6ea6ac: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.001s
I20260709 13:21:25.098341 22717 tablet_bootstrap.cc:492] T 5a1c542f0ae74a718e0b1c70aa641da4 P 2d62e0b282b44003a202b8dcbe6ea6ac: Bootstrap starting.
I20260709 13:21:25.104893 22741 raft_consensus.cc:493] T 56bc1b5c8c3143998cb9921b469c071d P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:25.105309 22722 raft_consensus.cc:493] T 2dad40e2d06244cebe50ec7433000752 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:25.105520 22741 raft_consensus.cc:515] T 56bc1b5c8c3143998cb9921b469c071d P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.105861 22722 raft_consensus.cc:515] T 2dad40e2d06244cebe50ec7433000752 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.108140 22722 leader_election.cc:290] T 2dad40e2d06244cebe50ec7433000752 P e20f6c83ae144d698c26b94c2eda023f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823), 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335)
I20260709 13:21:25.117007 22717 tablet_bootstrap.cc:654] T 5a1c542f0ae74a718e0b1c70aa641da4 P 2d62e0b282b44003a202b8dcbe6ea6ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:25.118213 22741 leader_election.cc:290] T 56bc1b5c8c3143998cb9921b469c071d P e20f6c83ae144d698c26b94c2eda023f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823), 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335)
I20260709 13:21:25.131208 22602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dad40e2d06244cebe50ec7433000752" candidate_uuid: "7df364d160d64ac4b5cdc4f004324f8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e20f6c83ae144d698c26b94c2eda023f" is_pre_election: true
I20260709 13:21:25.132037 22602 raft_consensus.cc:2468] T 2dad40e2d06244cebe50ec7433000752 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7df364d160d64ac4b5cdc4f004324f8b in term 0.
I20260709 13:21:25.133111 22723 raft_consensus.cc:493] T 68d74bd4ad624659b92ec79d4560a02c P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:25.133715 22491 leader_election.cc:304] T 2dad40e2d06244cebe50ec7433000752 P 7df364d160d64ac4b5cdc4f004324f8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7df364d160d64ac4b5cdc4f004324f8b, e20f6c83ae144d698c26b94c2eda023f; no voters: 2d62e0b282b44003a202b8dcbe6ea6ac
I20260709 13:21:25.133841 22723 raft_consensus.cc:515] T 68d74bd4ad624659b92ec79d4560a02c P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.136276 22724 raft_consensus.cc:2804] T 2dad40e2d06244cebe50ec7433000752 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:25.136854 22724 raft_consensus.cc:493] T 2dad40e2d06244cebe50ec7433000752 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:25.137302 22724 raft_consensus.cc:3060] T 2dad40e2d06244cebe50ec7433000752 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:25.147018 22723 leader_election.cc:290] T 68d74bd4ad624659b92ec79d4560a02c P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823), e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469)
I20260709 13:21:25.161185 22717 tablet_bootstrap.cc:492] T 5a1c542f0ae74a718e0b1c70aa641da4 P 2d62e0b282b44003a202b8dcbe6ea6ac: No bootstrap required, opened a new log
I20260709 13:21:25.162132 22717 ts_tablet_manager.cc:1403] T 5a1c542f0ae74a718e0b1c70aa641da4 P 2d62e0b282b44003a202b8dcbe6ea6ac: Time spent bootstrapping tablet: real 0.064s	user 0.020s	sys 0.008s
I20260709 13:21:25.169232 22717 raft_consensus.cc:359] T 5a1c542f0ae74a718e0b1c70aa641da4 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.169726 22716 tablet_bootstrap.cc:492] T a9c5f91cbe6f4a6abbd799e94e0985e1 P e20f6c83ae144d698c26b94c2eda023f: No bootstrap required, opened a new log
I20260709 13:21:25.170096 22717 raft_consensus.cc:385] T 5a1c542f0ae74a718e0b1c70aa641da4 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:25.170560 22716 ts_tablet_manager.cc:1403] T a9c5f91cbe6f4a6abbd799e94e0985e1 P e20f6c83ae144d698c26b94c2eda023f: Time spent bootstrapping tablet: real 0.083s	user 0.018s	sys 0.021s
I20260709 13:21:25.170581 22717 raft_consensus.cc:740] T 5a1c542f0ae74a718e0b1c70aa641da4 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d62e0b282b44003a202b8dcbe6ea6ac, State: Initialized, Role: FOLLOWER
I20260709 13:21:25.172005 22717 consensus_queue.cc:260] T 5a1c542f0ae74a718e0b1c70aa641da4 P 2d62e0b282b44003a202b8dcbe6ea6ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.174496 22717 ts_tablet_manager.cc:1434] T 5a1c542f0ae74a718e0b1c70aa641da4 P 2d62e0b282b44003a202b8dcbe6ea6ac: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 13:21:25.175684 22717 tablet_bootstrap.cc:492] T c0614d551fc342cead195ccaed2aeac3 P 2d62e0b282b44003a202b8dcbe6ea6ac: Bootstrap starting.
I20260709 13:21:25.178298 22724 raft_consensus.cc:515] T 2dad40e2d06244cebe50ec7433000752 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.180763 22724 leader_election.cc:290] T 2dad40e2d06244cebe50ec7433000752 P 7df364d160d64ac4b5cdc4f004324f8b [CANDIDATE]: Term 1 election: Requested vote from peers e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469), 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335)
I20260709 13:21:25.181558 22719 tablet_bootstrap.cc:492] T 68d74bd4ad624659b92ec79d4560a02c P 7df364d160d64ac4b5cdc4f004324f8b: No bootstrap required, opened a new log
I20260709 13:21:25.182138 22719 ts_tablet_manager.cc:1403] T 68d74bd4ad624659b92ec79d4560a02c P 7df364d160d64ac4b5cdc4f004324f8b: Time spent bootstrapping tablet: real 0.118s	user 0.030s	sys 0.054s
I20260709 13:21:25.183358 22717 tablet_bootstrap.cc:654] T c0614d551fc342cead195ccaed2aeac3 P 2d62e0b282b44003a202b8dcbe6ea6ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:25.184521 22602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dad40e2d06244cebe50ec7433000752" candidate_uuid: "7df364d160d64ac4b5cdc4f004324f8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e20f6c83ae144d698c26b94c2eda023f"
I20260709 13:21:25.185235 22602 raft_consensus.cc:3060] T 2dad40e2d06244cebe50ec7433000752 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:25.185314 22719 raft_consensus.cc:359] T 68d74bd4ad624659b92ec79d4560a02c P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.186251 22719 raft_consensus.cc:385] T 68d74bd4ad624659b92ec79d4560a02c P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:25.186589 22719 raft_consensus.cc:740] T 68d74bd4ad624659b92ec79d4560a02c P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7df364d160d64ac4b5cdc4f004324f8b, State: Initialized, Role: FOLLOWER
I20260709 13:21:25.187543 22719 consensus_queue.cc:260] T 68d74bd4ad624659b92ec79d4560a02c P 7df364d160d64ac4b5cdc4f004324f8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.189976 22719 ts_tablet_manager.cc:1434] T 68d74bd4ad624659b92ec79d4560a02c P 7df364d160d64ac4b5cdc4f004324f8b: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:21:25.192999 22527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dded1824a3ef49988cc1b941db0f9955" candidate_uuid: "e20f6c83ae144d698c26b94c2eda023f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b" is_pre_election: true
I20260709 13:21:25.193404 22719 tablet_bootstrap.cc:492] T 5e4995abf28446589b7695a1b77ee513 P 7df364d160d64ac4b5cdc4f004324f8b: Bootstrap starting.
I20260709 13:21:25.193993 22527 raft_consensus.cc:2468] T dded1824a3ef49988cc1b941db0f9955 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e20f6c83ae144d698c26b94c2eda023f in term 0.
I20260709 13:21:25.195304 22526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dad40e2d06244cebe50ec7433000752" candidate_uuid: "e20f6c83ae144d698c26b94c2eda023f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b" is_pre_election: true
I20260709 13:21:25.196542 22526 raft_consensus.cc:2393] T 2dad40e2d06244cebe50ec7433000752 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e20f6c83ae144d698c26b94c2eda023f in current term 1: Already voted for candidate 7df364d160d64ac4b5cdc4f004324f8b in this term.
I20260709 13:21:25.197736 22525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56bc1b5c8c3143998cb9921b469c071d" candidate_uuid: "e20f6c83ae144d698c26b94c2eda023f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b" is_pre_election: true
I20260709 13:21:25.197808 22716 raft_consensus.cc:359] T a9c5f91cbe6f4a6abbd799e94e0985e1 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.198915 22525 raft_consensus.cc:2468] T 56bc1b5c8c3143998cb9921b469c071d P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e20f6c83ae144d698c26b94c2eda023f in term 0.
I20260709 13:21:25.198957 22716 raft_consensus.cc:385] T a9c5f91cbe6f4a6abbd799e94e0985e1 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:25.199534 22716 raft_consensus.cc:740] T a9c5f91cbe6f4a6abbd799e94e0985e1 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e20f6c83ae144d698c26b94c2eda023f, State: Initialized, Role: FOLLOWER
I20260709 13:21:25.200769 22719 tablet_bootstrap.cc:654] T 5e4995abf28446589b7695a1b77ee513 P 7df364d160d64ac4b5cdc4f004324f8b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:25.200361 22716 consensus_queue.cc:260] T a9c5f91cbe6f4a6abbd799e94e0985e1 P e20f6c83ae144d698c26b94c2eda023f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.211808 22716 ts_tablet_manager.cc:1434] T a9c5f91cbe6f4a6abbd799e94e0985e1 P e20f6c83ae144d698c26b94c2eda023f: Time spent starting tablet: real 0.041s	user 0.008s	sys 0.000s
I20260709 13:21:25.211627 22568 leader_election.cc:304] T dded1824a3ef49988cc1b941db0f9955 P e20f6c83ae144d698c26b94c2eda023f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7df364d160d64ac4b5cdc4f004324f8b, e20f6c83ae144d698c26b94c2eda023f; no voters: 
I20260709 13:21:25.212446 22719 tablet_bootstrap.cc:492] T 5e4995abf28446589b7695a1b77ee513 P 7df364d160d64ac4b5cdc4f004324f8b: No bootstrap required, opened a new log
I20260709 13:21:25.213004 22719 ts_tablet_manager.cc:1403] T 5e4995abf28446589b7695a1b77ee513 P 7df364d160d64ac4b5cdc4f004324f8b: Time spent bootstrapping tablet: real 0.020s	user 0.008s	sys 0.008s
I20260709 13:21:25.214522 22742 raft_consensus.cc:2804] T dded1824a3ef49988cc1b941db0f9955 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:25.215006 22742 raft_consensus.cc:493] T dded1824a3ef49988cc1b941db0f9955 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:25.215595 22742 raft_consensus.cc:3060] T dded1824a3ef49988cc1b941db0f9955 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:25.216162 22719 raft_consensus.cc:359] T 5e4995abf28446589b7695a1b77ee513 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.216962 22719 raft_consensus.cc:385] T 5e4995abf28446589b7695a1b77ee513 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:25.217437 22719 raft_consensus.cc:740] T 5e4995abf28446589b7695a1b77ee513 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7df364d160d64ac4b5cdc4f004324f8b, State: Initialized, Role: FOLLOWER
I20260709 13:21:25.218328 22719 consensus_queue.cc:260] T 5e4995abf28446589b7695a1b77ee513 P 7df364d160d64ac4b5cdc4f004324f8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.214962 22568 leader_election.cc:304] T 56bc1b5c8c3143998cb9921b469c071d P e20f6c83ae144d698c26b94c2eda023f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7df364d160d64ac4b5cdc4f004324f8b, e20f6c83ae144d698c26b94c2eda023f; no voters: 
I20260709 13:21:25.219894 22741 raft_consensus.cc:2804] T 56bc1b5c8c3143998cb9921b469c071d P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:25.220292 22741 raft_consensus.cc:493] T 56bc1b5c8c3143998cb9921b469c071d P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:25.220638 22741 raft_consensus.cc:3060] T 56bc1b5c8c3143998cb9921b469c071d P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:25.220974 22716 tablet_bootstrap.cc:492] T 68d74bd4ad624659b92ec79d4560a02c P e20f6c83ae144d698c26b94c2eda023f: Bootstrap starting.
I20260709 13:21:25.221436 22676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dded1824a3ef49988cc1b941db0f9955" candidate_uuid: "e20f6c83ae144d698c26b94c2eda023f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" is_pre_election: true
I20260709 13:21:25.222386 22676 raft_consensus.cc:2468] T dded1824a3ef49988cc1b941db0f9955 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e20f6c83ae144d698c26b94c2eda023f in term 0.
I20260709 13:21:25.224076 22723 raft_consensus.cc:493] T 56bc1b5c8c3143998cb9921b469c071d P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:25.224627 22723 raft_consensus.cc:515] T 56bc1b5c8c3143998cb9921b469c071d P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.225085 22676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dad40e2d06244cebe50ec7433000752" candidate_uuid: "e20f6c83ae144d698c26b94c2eda023f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" is_pre_election: true
I20260709 13:21:25.226457 22675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56bc1b5c8c3143998cb9921b469c071d" candidate_uuid: "e20f6c83ae144d698c26b94c2eda023f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" is_pre_election: true
I20260709 13:21:25.226886 22723 leader_election.cc:290] T 56bc1b5c8c3143998cb9921b469c071d P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823), e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469)
I20260709 13:21:25.227231 22675 raft_consensus.cc:2468] T 56bc1b5c8c3143998cb9921b469c071d P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e20f6c83ae144d698c26b94c2eda023f in term 0.
I20260709 13:21:25.228161 22716 tablet_bootstrap.cc:654] T 68d74bd4ad624659b92ec79d4560a02c P e20f6c83ae144d698c26b94c2eda023f: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:25.230149 22526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56bc1b5c8c3143998cb9921b469c071d" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b" is_pre_election: true
I20260709 13:21:25.231120 22526 raft_consensus.cc:2468] T 56bc1b5c8c3143998cb9921b469c071d P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d62e0b282b44003a202b8dcbe6ea6ac in term 0.
I20260709 13:21:25.232489 22525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68d74bd4ad624659b92ec79d4560a02c" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b" is_pre_election: true
I20260709 13:21:25.233423 22525 raft_consensus.cc:2468] T 68d74bd4ad624659b92ec79d4560a02c P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d62e0b282b44003a202b8dcbe6ea6ac in term 0.
I20260709 13:21:25.234231 22741 raft_consensus.cc:515] T 56bc1b5c8c3143998cb9921b469c071d P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
W20260709 13:21:25.236588 22640 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.9.116.190:36011, user_credentials={real_user=slave}} blocked reactor thread for 56226.9us
I20260709 13:21:25.238240 22719 ts_tablet_manager.cc:1434] T 5e4995abf28446589b7695a1b77ee513 P 7df364d160d64ac4b5cdc4f004324f8b: Time spent starting tablet: real 0.025s	user 0.009s	sys 0.000s
I20260709 13:21:25.239542 22719 tablet_bootstrap.cc:492] T 2e322f5288b64ccca277807d2dc46579 P 7df364d160d64ac4b5cdc4f004324f8b: Bootstrap starting.
I20260709 13:21:25.245045 22675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dad40e2d06244cebe50ec7433000752" candidate_uuid: "7df364d160d64ac4b5cdc4f004324f8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac"
I20260709 13:21:25.246500 22741 leader_election.cc:290] T 56bc1b5c8c3143998cb9921b469c071d P e20f6c83ae144d698c26b94c2eda023f [CANDIDATE]: Term 1 election: Requested vote from peers 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823), 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335)
I20260709 13:21:25.248840 22716 tablet_bootstrap.cc:492] T 68d74bd4ad624659b92ec79d4560a02c P e20f6c83ae144d698c26b94c2eda023f: No bootstrap required, opened a new log
I20260709 13:21:25.249307 22716 ts_tablet_manager.cc:1403] T 68d74bd4ad624659b92ec79d4560a02c P e20f6c83ae144d698c26b94c2eda023f: Time spent bootstrapping tablet: real 0.029s	user 0.018s	sys 0.007s
I20260709 13:21:25.249629 22719 tablet_bootstrap.cc:654] T 2e322f5288b64ccca277807d2dc46579 P 7df364d160d64ac4b5cdc4f004324f8b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:25.251644 22525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56bc1b5c8c3143998cb9921b469c071d" candidate_uuid: "e20f6c83ae144d698c26b94c2eda023f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b"
I20260709 13:21:25.252424 22716 raft_consensus.cc:359] T 68d74bd4ad624659b92ec79d4560a02c P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.253291 22716 raft_consensus.cc:385] T 68d74bd4ad624659b92ec79d4560a02c P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:25.252424 22525 raft_consensus.cc:3060] T 56bc1b5c8c3143998cb9921b469c071d P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:25.253620 22716 raft_consensus.cc:740] T 68d74bd4ad624659b92ec79d4560a02c P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e20f6c83ae144d698c26b94c2eda023f, State: Initialized, Role: FOLLOWER
I20260709 13:21:25.254662 22716 consensus_queue.cc:260] T 68d74bd4ad624659b92ec79d4560a02c P e20f6c83ae144d698c26b94c2eda023f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.257390 22716 ts_tablet_manager.cc:1434] T 68d74bd4ad624659b92ec79d4560a02c P e20f6c83ae144d698c26b94c2eda023f: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:21:25.258478 22716 tablet_bootstrap.cc:492] T 5a1c542f0ae74a718e0b1c70aa641da4 P e20f6c83ae144d698c26b94c2eda023f: Bootstrap starting.
W20260709 13:21:25.259182 22493 leader_election.cc:343] T 2dad40e2d06244cebe50ec7433000752 P 7df364d160d64ac4b5cdc4f004324f8b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:21:25.260200 22568 leader_election.cc:343] T 2dad40e2d06244cebe50ec7433000752 P e20f6c83ae144d698c26b94c2eda023f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:25.236007 22642 leader_election.cc:304] T 56bc1b5c8c3143998cb9921b469c071d P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d62e0b282b44003a202b8dcbe6ea6ac, 7df364d160d64ac4b5cdc4f004324f8b; no voters: 
I20260709 13:21:25.260720 22568 leader_election.cc:304] T 2dad40e2d06244cebe50ec7433000752 P e20f6c83ae144d698c26b94c2eda023f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e20f6c83ae144d698c26b94c2eda023f; no voters: 2d62e0b282b44003a202b8dcbe6ea6ac, 7df364d160d64ac4b5cdc4f004324f8b
I20260709 13:21:25.264881 22723 raft_consensus.cc:2804] T 56bc1b5c8c3143998cb9921b469c071d P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:25.265269 22723 raft_consensus.cc:493] T 56bc1b5c8c3143998cb9921b469c071d P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:25.263721 22602 raft_consensus.cc:2468] T 2dad40e2d06244cebe50ec7433000752 P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7df364d160d64ac4b5cdc4f004324f8b in term 1.
I20260709 13:21:25.265724 22723 raft_consensus.cc:3060] T 56bc1b5c8c3143998cb9921b469c071d P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:25.266084 22741 raft_consensus.cc:2749] T 2dad40e2d06244cebe50ec7433000752 P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:25.267907 22722 raft_consensus.cc:493] T 68d74bd4ad624659b92ec79d4560a02c P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:25.268489 22722 raft_consensus.cc:515] T 68d74bd4ad624659b92ec79d4560a02c P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.268955 22601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68d74bd4ad624659b92ec79d4560a02c" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e20f6c83ae144d698c26b94c2eda023f" is_pre_election: true
I20260709 13:21:25.269676 22601 raft_consensus.cc:2468] T 68d74bd4ad624659b92ec79d4560a02c P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d62e0b282b44003a202b8dcbe6ea6ac in term 0.
I20260709 13:21:25.268996 22600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56bc1b5c8c3143998cb9921b469c071d" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e20f6c83ae144d698c26b94c2eda023f" is_pre_election: true
I20260709 13:21:25.271320 22600 raft_consensus.cc:2393] T 56bc1b5c8c3143998cb9921b469c071d P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2d62e0b282b44003a202b8dcbe6ea6ac in current term 1: Already voted for candidate e20f6c83ae144d698c26b94c2eda023f in this term.
I20260709 13:21:25.271163 22491 leader_election.cc:304] T 2dad40e2d06244cebe50ec7433000752 P 7df364d160d64ac4b5cdc4f004324f8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7df364d160d64ac4b5cdc4f004324f8b, e20f6c83ae144d698c26b94c2eda023f; no voters: 2d62e0b282b44003a202b8dcbe6ea6ac
I20260709 13:21:25.272449 22724 raft_consensus.cc:2804] T 2dad40e2d06244cebe50ec7433000752 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:25.272567 22719 tablet_bootstrap.cc:492] T 2e322f5288b64ccca277807d2dc46579 P 7df364d160d64ac4b5cdc4f004324f8b: No bootstrap required, opened a new log
I20260709 13:21:25.272825 22722 leader_election.cc:290] T 68d74bd4ad624659b92ec79d4560a02c P e20f6c83ae144d698c26b94c2eda023f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823), 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335)
I20260709 13:21:25.273748 22724 raft_consensus.cc:697] T 2dad40e2d06244cebe50ec7433000752 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 LEADER]: Becoming Leader. State: Replica: 7df364d160d64ac4b5cdc4f004324f8b, State: Running, Role: LEADER
I20260709 13:21:25.273788 22719 ts_tablet_manager.cc:1403] T 2e322f5288b64ccca277807d2dc46579 P 7df364d160d64ac4b5cdc4f004324f8b: Time spent bootstrapping tablet: real 0.034s	user 0.013s	sys 0.000s
I20260709 13:21:25.274765 22724 consensus_queue.cc:237] T 2dad40e2d06244cebe50ec7433000752 P 7df364d160d64ac4b5cdc4f004324f8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.276701 22742 raft_consensus.cc:515] T dded1824a3ef49988cc1b941db0f9955 P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.277367 22734 raft_consensus.cc:493] T c0614d551fc342cead195ccaed2aeac3 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:25.278172 22734 raft_consensus.cc:515] T c0614d551fc342cead195ccaed2aeac3 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } } peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } }
I20260709 13:21:25.265726 22642 leader_election.cc:304] T 68d74bd4ad624659b92ec79d4560a02c P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d62e0b282b44003a202b8dcbe6ea6ac, 7df364d160d64ac4b5cdc4f004324f8b; no voters: 
I20260709 13:21:25.265241 22716 tablet_bootstrap.cc:654] T 5a1c542f0ae74a718e0b1c70aa641da4 P e20f6c83ae144d698c26b94c2eda023f: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:25.282095 22734 leader_election.cc:290] T c0614d551fc342cead195ccaed2aeac3 P 7df364d160d64ac4b5cdc4f004324f8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469), 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335)
I20260709 13:21:25.278540 22723 raft_consensus.cc:515] T 56bc1b5c8c3143998cb9921b469c071d P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.285319 22723 leader_election.cc:290] T 56bc1b5c8c3143998cb9921b469c071d P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 election: Requested vote from peers 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823), e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469)
I20260709 13:21:25.285885 22723 raft_consensus.cc:2804] T 68d74bd4ad624659b92ec79d4560a02c P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:25.285930 22526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68d74bd4ad624659b92ec79d4560a02c" candidate_uuid: "e20f6c83ae144d698c26b94c2eda023f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b" is_pre_election: true
I20260709 13:21:25.286336 22723 raft_consensus.cc:493] T 68d74bd4ad624659b92ec79d4560a02c P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:25.286680 22526 raft_consensus.cc:2468] T 68d74bd4ad624659b92ec79d4560a02c P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e20f6c83ae144d698c26b94c2eda023f in term 0.
I20260709 13:21:25.286716 22723 raft_consensus.cc:3060] T 68d74bd4ad624659b92ec79d4560a02c P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:25.297618 22674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68d74bd4ad624659b92ec79d4560a02c" candidate_uuid: "e20f6c83ae144d698c26b94c2eda023f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" is_pre_election: true
I20260709 13:21:25.291826 22601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0614d551fc342cead195ccaed2aeac3" candidate_uuid: "7df364d160d64ac4b5cdc4f004324f8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e20f6c83ae144d698c26b94c2eda023f" is_pre_election: true
I20260709 13:21:25.288236 22600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56bc1b5c8c3143998cb9921b469c071d" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e20f6c83ae144d698c26b94c2eda023f"
I20260709 13:21:25.299516 22600 raft_consensus.cc:2393] T 56bc1b5c8c3143998cb9921b469c071d P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2d62e0b282b44003a202b8dcbe6ea6ac in current term 1: Already voted for candidate e20f6c83ae144d698c26b94c2eda023f in this term.
I20260709 13:21:25.299363 22601 raft_consensus.cc:2468] T c0614d551fc342cead195ccaed2aeac3 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7df364d160d64ac4b5cdc4f004324f8b in term 0.
I20260709 13:21:25.291740 22742 leader_election.cc:290] T dded1824a3ef49988cc1b941db0f9955 P e20f6c83ae144d698c26b94c2eda023f [CANDIDATE]: Term 1 election: Requested vote from peers 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823), 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335)
I20260709 13:21:25.295931 22526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56bc1b5c8c3143998cb9921b469c071d" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b"
I20260709 13:21:25.296661 22676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56bc1b5c8c3143998cb9921b469c071d" candidate_uuid: "e20f6c83ae144d698c26b94c2eda023f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac"
I20260709 13:21:25.303253 22676 raft_consensus.cc:2393] T 56bc1b5c8c3143998cb9921b469c071d P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e20f6c83ae144d698c26b94c2eda023f in current term 1: Already voted for candidate 2d62e0b282b44003a202b8dcbe6ea6ac in this term.
I20260709 13:21:25.303041 22642 leader_election.cc:304] T 56bc1b5c8c3143998cb9921b469c071d P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2d62e0b282b44003a202b8dcbe6ea6ac; no voters: 7df364d160d64ac4b5cdc4f004324f8b, e20f6c83ae144d698c26b94c2eda023f
I20260709 13:21:25.289219 22675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0614d551fc342cead195ccaed2aeac3" candidate_uuid: "7df364d160d64ac4b5cdc4f004324f8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" is_pre_election: true
I20260709 13:21:25.294772 22719 raft_consensus.cc:359] T 2e322f5288b64ccca277807d2dc46579 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.263721 22525 raft_consensus.cc:2468] T 56bc1b5c8c3143998cb9921b469c071d P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e20f6c83ae144d698c26b94c2eda023f in term 1.
W20260709 13:21:25.305779 22493 leader_election.cc:343] T c0614d551fc342cead195ccaed2aeac3 P 7df364d160d64ac4b5cdc4f004324f8b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:25.306002 22719 raft_consensus.cc:385] T 2e322f5288b64ccca277807d2dc46579 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:25.289193 22568 leader_election.cc:304] T 68d74bd4ad624659b92ec79d4560a02c P e20f6c83ae144d698c26b94c2eda023f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7df364d160d64ac4b5cdc4f004324f8b, e20f6c83ae144d698c26b94c2eda023f; no voters: 
I20260709 13:21:25.297345 22717 tablet_bootstrap.cc:492] T c0614d551fc342cead195ccaed2aeac3 P 2d62e0b282b44003a202b8dcbe6ea6ac: No bootstrap required, opened a new log
I20260709 13:21:25.303980 22752 raft_consensus.cc:2749] T 56bc1b5c8c3143998cb9921b469c071d P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:21:25.306475 22719 raft_consensus.cc:740] T 2e322f5288b64ccca277807d2dc46579 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7df364d160d64ac4b5cdc4f004324f8b, State: Initialized, Role: FOLLOWER
I20260709 13:21:25.307981 22717 ts_tablet_manager.cc:1403] T c0614d551fc342cead195ccaed2aeac3 P 2d62e0b282b44003a202b8dcbe6ea6ac: Time spent bootstrapping tablet: real 0.133s	user 0.031s	sys 0.057s
I20260709 13:21:25.308120 22742 raft_consensus.cc:2804] T 68d74bd4ad624659b92ec79d4560a02c P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:25.308583 22742 raft_consensus.cc:493] T 68d74bd4ad624659b92ec79d4560a02c P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:25.308974 22742 raft_consensus.cc:3060] T 68d74bd4ad624659b92ec79d4560a02c P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:25.310359 22424 catalog_manager.cc:5697] T 2dad40e2d06244cebe50ec7433000752 P 7df364d160d64ac4b5cdc4f004324f8b reported cstate change: term changed from 0 to 1, leader changed from <none> to 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129). New cstate: current_term: 1 leader_uuid: "7df364d160d64ac4b5cdc4f004324f8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:25.311699 22717 raft_consensus.cc:359] T c0614d551fc342cead195ccaed2aeac3 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } } peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } }
I20260709 13:21:25.311934 22723 raft_consensus.cc:515] T 68d74bd4ad624659b92ec79d4560a02c P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.312573 22717 raft_consensus.cc:385] T c0614d551fc342cead195ccaed2aeac3 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:25.313064 22717 raft_consensus.cc:740] T c0614d551fc342cead195ccaed2aeac3 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d62e0b282b44003a202b8dcbe6ea6ac, State: Initialized, Role: FOLLOWER
I20260709 13:21:25.313215 22674 raft_consensus.cc:2393] T 68d74bd4ad624659b92ec79d4560a02c P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e20f6c83ae144d698c26b94c2eda023f in current term 1: Already voted for candidate 2d62e0b282b44003a202b8dcbe6ea6ac in this term.
I20260709 13:21:25.313985 22717 consensus_queue.cc:260] T c0614d551fc342cead195ccaed2aeac3 P 2d62e0b282b44003a202b8dcbe6ea6ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } } peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } }
I20260709 13:21:25.314677 22491 leader_election.cc:304] T c0614d551fc342cead195ccaed2aeac3 P 7df364d160d64ac4b5cdc4f004324f8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7df364d160d64ac4b5cdc4f004324f8b, e20f6c83ae144d698c26b94c2eda023f; no voters: 2d62e0b282b44003a202b8dcbe6ea6ac
I20260709 13:21:25.314832 22568 leader_election.cc:304] T 56bc1b5c8c3143998cb9921b469c071d P e20f6c83ae144d698c26b94c2eda023f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7df364d160d64ac4b5cdc4f004324f8b, e20f6c83ae144d698c26b94c2eda023f; no voters: 
I20260709 13:21:25.316087 22525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dded1824a3ef49988cc1b941db0f9955" candidate_uuid: "e20f6c83ae144d698c26b94c2eda023f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b"
I20260709 13:21:25.316102 22734 raft_consensus.cc:2804] T c0614d551fc342cead195ccaed2aeac3 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:25.316744 22717 ts_tablet_manager.cc:1434] T c0614d551fc342cead195ccaed2aeac3 P 2d62e0b282b44003a202b8dcbe6ea6ac: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:21:25.317031 22734 raft_consensus.cc:493] T c0614d551fc342cead195ccaed2aeac3 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:25.316805 22525 raft_consensus.cc:3060] T dded1824a3ef49988cc1b941db0f9955 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:25.317500 22734 raft_consensus.cc:3060] T c0614d551fc342cead195ccaed2aeac3 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:25.318428 22717 tablet_bootstrap.cc:492] T 2dad40e2d06244cebe50ec7433000752 P 2d62e0b282b44003a202b8dcbe6ea6ac: Bootstrap starting.
I20260709 13:21:25.319798 22716 tablet_bootstrap.cc:492] T 5a1c542f0ae74a718e0b1c70aa641da4 P e20f6c83ae144d698c26b94c2eda023f: No bootstrap required, opened a new log
I20260709 13:21:25.320330 22716 ts_tablet_manager.cc:1403] T 5a1c542f0ae74a718e0b1c70aa641da4 P e20f6c83ae144d698c26b94c2eda023f: Time spent bootstrapping tablet: real 0.062s	user 0.008s	sys 0.004s
I20260709 13:21:25.323212 22716 raft_consensus.cc:359] T 5a1c542f0ae74a718e0b1c70aa641da4 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.323925 22716 raft_consensus.cc:385] T 5a1c542f0ae74a718e0b1c70aa641da4 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:25.323604 22742 raft_consensus.cc:515] T 68d74bd4ad624659b92ec79d4560a02c P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.324246 22716 raft_consensus.cc:740] T 5a1c542f0ae74a718e0b1c70aa641da4 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e20f6c83ae144d698c26b94c2eda023f, State: Initialized, Role: FOLLOWER
I20260709 13:21:25.316252 22722 raft_consensus.cc:2804] T 56bc1b5c8c3143998cb9921b469c071d P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:25.325752 22722 raft_consensus.cc:697] T 56bc1b5c8c3143998cb9921b469c071d P e20f6c83ae144d698c26b94c2eda023f [term 1 LEADER]: Becoming Leader. State: Replica: e20f6c83ae144d698c26b94c2eda023f, State: Running, Role: LEADER
I20260709 13:21:25.326251 22717 tablet_bootstrap.cc:654] T 2dad40e2d06244cebe50ec7433000752 P 2d62e0b282b44003a202b8dcbe6ea6ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:25.326615 22722 consensus_queue.cc:237] T 56bc1b5c8c3143998cb9921b469c071d P e20f6c83ae144d698c26b94c2eda023f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.326689 22526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68d74bd4ad624659b92ec79d4560a02c" candidate_uuid: "e20f6c83ae144d698c26b94c2eda023f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b"
I20260709 13:21:25.330365 22752 raft_consensus.cc:493] T 5a1c542f0ae74a718e0b1c70aa641da4 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:25.330751 22526 raft_consensus.cc:3060] T 68d74bd4ad624659b92ec79d4560a02c P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:25.330905 22752 raft_consensus.cc:515] T 5a1c542f0ae74a718e0b1c70aa641da4 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.331172 22674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dded1824a3ef49988cc1b941db0f9955" candidate_uuid: "e20f6c83ae144d698c26b94c2eda023f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac"
I20260709 13:21:25.332032 22674 raft_consensus.cc:3060] T dded1824a3ef49988cc1b941db0f9955 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:25.333284 22752 leader_election.cc:290] T 5a1c542f0ae74a718e0b1c70aa641da4 P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823), e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469)
I20260709 13:21:25.337373 22601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a1c542f0ae74a718e0b1c70aa641da4" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e20f6c83ae144d698c26b94c2eda023f" is_pre_election: true
I20260709 13:21:25.338430 22527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a1c542f0ae74a718e0b1c70aa641da4" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b" is_pre_election: true
W20260709 13:21:25.340081 22642 leader_election.cc:343] T 5a1c542f0ae74a718e0b1c70aa641da4 P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:25.340269 22734 raft_consensus.cc:515] T c0614d551fc342cead195ccaed2aeac3 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } } peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } }
I20260709 13:21:25.342671 22716 consensus_queue.cc:260] T 5a1c542f0ae74a718e0b1c70aa641da4 P e20f6c83ae144d698c26b94c2eda023f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.343991 22601 raft_consensus.cc:2468] T 5a1c542f0ae74a718e0b1c70aa641da4 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d62e0b282b44003a202b8dcbe6ea6ac in term 0.
I20260709 13:21:25.344813 22600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0614d551fc342cead195ccaed2aeac3" candidate_uuid: "7df364d160d64ac4b5cdc4f004324f8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e20f6c83ae144d698c26b94c2eda023f"
I20260709 13:21:25.345443 22600 raft_consensus.cc:3060] T c0614d551fc342cead195ccaed2aeac3 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:25.345986 22640 leader_election.cc:304] T 5a1c542f0ae74a718e0b1c70aa641da4 P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2d62e0b282b44003a202b8dcbe6ea6ac, e20f6c83ae144d698c26b94c2eda023f; no voters: 7df364d160d64ac4b5cdc4f004324f8b
I20260709 13:21:25.347034 22752 raft_consensus.cc:2804] T 5a1c542f0ae74a718e0b1c70aa641da4 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:25.347407 22752 raft_consensus.cc:493] T 5a1c542f0ae74a718e0b1c70aa641da4 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:25.347684 22723 leader_election.cc:290] T 68d74bd4ad624659b92ec79d4560a02c P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 election: Requested vote from peers 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823), e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469)
I20260709 13:21:25.348656 22716 ts_tablet_manager.cc:1434] T 5a1c542f0ae74a718e0b1c70aa641da4 P e20f6c83ae144d698c26b94c2eda023f: Time spent starting tablet: real 0.028s	user 0.009s	sys 0.000s
I20260709 13:21:25.349267 22601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68d74bd4ad624659b92ec79d4560a02c" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e20f6c83ae144d698c26b94c2eda023f"
I20260709 13:21:25.349996 22716 tablet_bootstrap.cc:492] T 5e4995abf28446589b7695a1b77ee513 P e20f6c83ae144d698c26b94c2eda023f: Bootstrap starting.
I20260709 13:21:25.347754 22752 raft_consensus.cc:3060] T 5a1c542f0ae74a718e0b1c70aa641da4 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:25.350081 22601 raft_consensus.cc:2393] T 68d74bd4ad624659b92ec79d4560a02c P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2d62e0b282b44003a202b8dcbe6ea6ac in current term 1: Already voted for candidate e20f6c83ae144d698c26b94c2eda023f in this term.
I20260709 13:21:25.353562 22741 raft_consensus.cc:493] T 2e322f5288b64ccca277807d2dc46579 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:25.353991 22526 raft_consensus.cc:2468] T 68d74bd4ad624659b92ec79d4560a02c P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e20f6c83ae144d698c26b94c2eda023f in term 1.
I20260709 13:21:25.354068 22741 raft_consensus.cc:515] T 2e322f5288b64ccca277807d2dc46579 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.355440 22734 leader_election.cc:290] T c0614d551fc342cead195ccaed2aeac3 P 7df364d160d64ac4b5cdc4f004324f8b [CANDIDATE]: Term 1 election: Requested vote from peers e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469), 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335)
I20260709 13:21:25.356976 22741 leader_election.cc:290] T 2e322f5288b64ccca277807d2dc46579 P e20f6c83ae144d698c26b94c2eda023f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823), 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335)
I20260709 13:21:25.357267 22717 tablet_bootstrap.cc:492] T 2dad40e2d06244cebe50ec7433000752 P 2d62e0b282b44003a202b8dcbe6ea6ac: No bootstrap required, opened a new log
I20260709 13:21:25.357707 22716 tablet_bootstrap.cc:654] T 5e4995abf28446589b7695a1b77ee513 P e20f6c83ae144d698c26b94c2eda023f: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:25.358004 22717 ts_tablet_manager.cc:1403] T 2dad40e2d06244cebe50ec7433000752 P 2d62e0b282b44003a202b8dcbe6ea6ac: Time spent bootstrapping tablet: real 0.040s	user 0.007s	sys 0.012s
I20260709 13:21:25.358839 22527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e322f5288b64ccca277807d2dc46579" candidate_uuid: "e20f6c83ae144d698c26b94c2eda023f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b" is_pre_election: true
I20260709 13:21:25.361527 22717 raft_consensus.cc:359] T 2dad40e2d06244cebe50ec7433000752 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.308578 22719 consensus_queue.cc:260] T 2e322f5288b64ccca277807d2dc46579 P 7df364d160d64ac4b5cdc4f004324f8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.366689 22717 raft_consensus.cc:385] T 2dad40e2d06244cebe50ec7433000752 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:25.366969 22527 raft_consensus.cc:2468] T 2e322f5288b64ccca277807d2dc46579 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e20f6c83ae144d698c26b94c2eda023f in term 0.
I20260709 13:21:25.367161 22717 raft_consensus.cc:740] T 2dad40e2d06244cebe50ec7433000752 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d62e0b282b44003a202b8dcbe6ea6ac, State: Initialized, Role: FOLLOWER
I20260709 13:21:25.368423 22719 ts_tablet_manager.cc:1434] T 2e322f5288b64ccca277807d2dc46579 P 7df364d160d64ac4b5cdc4f004324f8b: Time spent starting tablet: real 0.094s	user 0.007s	sys 0.001s
I20260709 13:21:25.368394 22717 consensus_queue.cc:260] T 2dad40e2d06244cebe50ec7433000752 P 2d62e0b282b44003a202b8dcbe6ea6ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.369319 22675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0614d551fc342cead195ccaed2aeac3" candidate_uuid: "7df364d160d64ac4b5cdc4f004324f8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac"
I20260709 13:21:25.369762 22719 tablet_bootstrap.cc:492] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 7df364d160d64ac4b5cdc4f004324f8b: Bootstrap starting.
I20260709 13:21:25.369946 22675 raft_consensus.cc:3060] T c0614d551fc342cead195ccaed2aeac3 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:25.373349 22717 ts_tablet_manager.cc:1434] T 2dad40e2d06244cebe50ec7433000752 P 2d62e0b282b44003a202b8dcbe6ea6ac: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:21:25.363236 22568 leader_election.cc:304] T 68d74bd4ad624659b92ec79d4560a02c P e20f6c83ae144d698c26b94c2eda023f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7df364d160d64ac4b5cdc4f004324f8b, e20f6c83ae144d698c26b94c2eda023f; no voters: 
I20260709 13:21:25.375581 22568 leader_election.cc:304] T 2e322f5288b64ccca277807d2dc46579 P e20f6c83ae144d698c26b94c2eda023f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7df364d160d64ac4b5cdc4f004324f8b, e20f6c83ae144d698c26b94c2eda023f; no voters: 
I20260709 13:21:25.363385 22423 catalog_manager.cc:5697] T 56bc1b5c8c3143998cb9921b469c071d P e20f6c83ae144d698c26b94c2eda023f reported cstate change: term changed from 0 to 1, leader changed from <none> to e20f6c83ae144d698c26b94c2eda023f (127.9.116.130). New cstate: current_term: 1 leader_uuid: "e20f6c83ae144d698c26b94c2eda023f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:25.377579 22675 raft_consensus.cc:2468] T c0614d551fc342cead195ccaed2aeac3 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7df364d160d64ac4b5cdc4f004324f8b in term 1.
I20260709 13:21:25.379304 22741 raft_consensus.cc:2804] T 68d74bd4ad624659b92ec79d4560a02c P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:25.354612 22600 raft_consensus.cc:2468] T c0614d551fc342cead195ccaed2aeac3 P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7df364d160d64ac4b5cdc4f004324f8b in term 1.
I20260709 13:21:25.374607 22717 tablet_bootstrap.cc:492] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 2d62e0b282b44003a202b8dcbe6ea6ac: Bootstrap starting.
I20260709 13:21:25.380554 22722 raft_consensus.cc:2804] T 2e322f5288b64ccca277807d2dc46579 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:25.380936 22722 raft_consensus.cc:493] T 2e322f5288b64ccca277807d2dc46579 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:25.381290 22722 raft_consensus.cc:3060] T 2e322f5288b64ccca277807d2dc46579 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:25.381163 22493 leader_election.cc:304] T c0614d551fc342cead195ccaed2aeac3 P 7df364d160d64ac4b5cdc4f004324f8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d62e0b282b44003a202b8dcbe6ea6ac, 7df364d160d64ac4b5cdc4f004324f8b; no voters: 
I20260709 13:21:25.382068 22527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68d74bd4ad624659b92ec79d4560a02c" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b"
I20260709 13:21:25.366036 22674 raft_consensus.cc:2468] T dded1824a3ef49988cc1b941db0f9955 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e20f6c83ae144d698c26b94c2eda023f in term 1.
I20260709 13:21:25.382529 22723 raft_consensus.cc:493] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:25.382967 22527 raft_consensus.cc:2393] T 68d74bd4ad624659b92ec79d4560a02c P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2d62e0b282b44003a202b8dcbe6ea6ac in current term 1: Already voted for candidate e20f6c83ae144d698c26b94c2eda023f in this term.
I20260709 13:21:25.383124 22723 raft_consensus.cc:515] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.384182 22734 raft_consensus.cc:2804] T c0614d551fc342cead195ccaed2aeac3 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:25.384955 22734 raft_consensus.cc:697] T c0614d551fc342cead195ccaed2aeac3 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 LEADER]: Becoming Leader. State: Replica: 7df364d160d64ac4b5cdc4f004324f8b, State: Running, Role: LEADER
I20260709 13:21:25.385298 22723 leader_election.cc:290] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823), e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469)
I20260709 13:21:25.385840 22734 consensus_queue.cc:237] T c0614d551fc342cead195ccaed2aeac3 P 7df364d160d64ac4b5cdc4f004324f8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } } peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } }
I20260709 13:21:25.386884 22525 raft_consensus.cc:2468] T dded1824a3ef49988cc1b941db0f9955 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e20f6c83ae144d698c26b94c2eda023f in term 1.
I20260709 13:21:25.387091 22600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e4995abf28446589b7695a1b77ee513" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e20f6c83ae144d698c26b94c2eda023f" is_pre_election: true
I20260709 13:21:25.387928 22742 leader_election.cc:290] T 68d74bd4ad624659b92ec79d4560a02c P e20f6c83ae144d698c26b94c2eda023f [CANDIDATE]: Term 1 election: Requested vote from peers 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823), 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335)
I20260709 13:21:25.387964 22675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e322f5288b64ccca277807d2dc46579" candidate_uuid: "e20f6c83ae144d698c26b94c2eda023f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" is_pre_election: true
I20260709 13:21:25.392752 22719 tablet_bootstrap.cc:654] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 7df364d160d64ac4b5cdc4f004324f8b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:25.392469 22741 raft_consensus.cc:697] T 68d74bd4ad624659b92ec79d4560a02c P e20f6c83ae144d698c26b94c2eda023f [term 1 LEADER]: Becoming Leader. State: Replica: e20f6c83ae144d698c26b94c2eda023f, State: Running, Role: LEADER
I20260709 13:21:25.393504 22741 consensus_queue.cc:237] T 68d74bd4ad624659b92ec79d4560a02c P e20f6c83ae144d698c26b94c2eda023f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.389778 22568 leader_election.cc:304] T dded1824a3ef49988cc1b941db0f9955 P e20f6c83ae144d698c26b94c2eda023f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7df364d160d64ac4b5cdc4f004324f8b, e20f6c83ae144d698c26b94c2eda023f; no voters: 
W20260709 13:21:25.402163 22640 leader_election.cc:343] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:25.366714 22752 raft_consensus.cc:515] T 5a1c542f0ae74a718e0b1c70aa641da4 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.403268 22722 raft_consensus.cc:515] T 2e322f5288b64ccca277807d2dc46579 P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.403586 22642 leader_election.cc:304] T 68d74bd4ad624659b92ec79d4560a02c P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2d62e0b282b44003a202b8dcbe6ea6ac; no voters: 7df364d160d64ac4b5cdc4f004324f8b, e20f6c83ae144d698c26b94c2eda023f
I20260709 13:21:25.405582 22722 leader_election.cc:290] T 2e322f5288b64ccca277807d2dc46579 P e20f6c83ae144d698c26b94c2eda023f [CANDIDATE]: Term 1 election: Requested vote from peers 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823), 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335)
I20260709 13:21:25.405493 22674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68d74bd4ad624659b92ec79d4560a02c" candidate_uuid: "e20f6c83ae144d698c26b94c2eda023f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac"
I20260709 13:21:25.406076 22752 leader_election.cc:290] T 5a1c542f0ae74a718e0b1c70aa641da4 P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 election: Requested vote from peers 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823), e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469)
I20260709 13:21:25.406880 22674 raft_consensus.cc:2393] T 68d74bd4ad624659b92ec79d4560a02c P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e20f6c83ae144d698c26b94c2eda023f in current term 1: Already voted for candidate 2d62e0b282b44003a202b8dcbe6ea6ac in this term.
I20260709 13:21:25.408288 22525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e4995abf28446589b7695a1b77ee513" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b" is_pre_election: true
I20260709 13:21:25.409174 22741 raft_consensus.cc:2804] T dded1824a3ef49988cc1b941db0f9955 P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:25.410691 22741 raft_consensus.cc:697] T dded1824a3ef49988cc1b941db0f9955 P e20f6c83ae144d698c26b94c2eda023f [term 1 LEADER]: Becoming Leader. State: Replica: e20f6c83ae144d698c26b94c2eda023f, State: Running, Role: LEADER
I20260709 13:21:25.408908 22723 raft_consensus.cc:2749] T 68d74bd4ad624659b92ec79d4560a02c P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:21:25.411067 22527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a1c542f0ae74a718e0b1c70aa641da4" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b"
I20260709 13:21:25.412860 22525 raft_consensus.cc:2468] T 5e4995abf28446589b7695a1b77ee513 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d62e0b282b44003a202b8dcbe6ea6ac in term 0.
I20260709 13:21:25.411695 22741 consensus_queue.cc:237] T dded1824a3ef49988cc1b941db0f9955 P e20f6c83ae144d698c26b94c2eda023f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
W20260709 13:21:25.413805 22642 leader_election.cc:343] T 5a1c542f0ae74a718e0b1c70aa641da4 P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:25.419726 22525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e322f5288b64ccca277807d2dc46579" candidate_uuid: "e20f6c83ae144d698c26b94c2eda023f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b"
I20260709 13:21:25.420418 22525 raft_consensus.cc:3060] T 2e322f5288b64ccca277807d2dc46579 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:25.417858 22600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a1c542f0ae74a718e0b1c70aa641da4" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e20f6c83ae144d698c26b94c2eda023f"
I20260709 13:21:25.423020 22600 raft_consensus.cc:3060] T 5a1c542f0ae74a718e0b1c70aa641da4 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:25.424975 22717 tablet_bootstrap.cc:654] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 2d62e0b282b44003a202b8dcbe6ea6ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:25.415999 22642 leader_election.cc:304] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2d62e0b282b44003a202b8dcbe6ea6ac, 7df364d160d64ac4b5cdc4f004324f8b; no voters: e20f6c83ae144d698c26b94c2eda023f
I20260709 13:21:25.427855 22723 raft_consensus.cc:2804] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:25.428390 22723 raft_consensus.cc:493] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:25.428838 22723 raft_consensus.cc:3060] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:25.433043 22525 raft_consensus.cc:2468] T 2e322f5288b64ccca277807d2dc46579 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e20f6c83ae144d698c26b94c2eda023f in term 1.
W20260709 13:21:25.436522 22568 leader_election.cc:343] T 2e322f5288b64ccca277807d2dc46579 P e20f6c83ae144d698c26b94c2eda023f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:25.436902 22716 tablet_bootstrap.cc:492] T 5e4995abf28446589b7695a1b77ee513 P e20f6c83ae144d698c26b94c2eda023f: No bootstrap required, opened a new log
I20260709 13:21:25.437650 22716 ts_tablet_manager.cc:1403] T 5e4995abf28446589b7695a1b77ee513 P e20f6c83ae144d698c26b94c2eda023f: Time spent bootstrapping tablet: real 0.088s	user 0.019s	sys 0.004s
I20260709 13:21:25.440011 22674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e322f5288b64ccca277807d2dc46579" candidate_uuid: "e20f6c83ae144d698c26b94c2eda023f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac"
I20260709 13:21:25.441023 22716 raft_consensus.cc:359] T 5e4995abf28446589b7695a1b77ee513 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.441958 22716 raft_consensus.cc:385] T 5e4995abf28446589b7695a1b77ee513 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:25.442335 22716 raft_consensus.cc:740] T 5e4995abf28446589b7695a1b77ee513 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e20f6c83ae144d698c26b94c2eda023f, State: Initialized, Role: FOLLOWER
I20260709 13:21:25.443583 22716 consensus_queue.cc:260] T 5e4995abf28446589b7695a1b77ee513 P e20f6c83ae144d698c26b94c2eda023f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
W20260709 13:21:25.443956 22568 leader_election.cc:343] T 2e322f5288b64ccca277807d2dc46579 P e20f6c83ae144d698c26b94c2eda023f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:25.448045 22568 leader_election.cc:304] T 2e322f5288b64ccca277807d2dc46579 P e20f6c83ae144d698c26b94c2eda023f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7df364d160d64ac4b5cdc4f004324f8b, e20f6c83ae144d698c26b94c2eda023f; no voters: 2d62e0b282b44003a202b8dcbe6ea6ac
I20260709 13:21:25.448666 22600 raft_consensus.cc:2468] T 5a1c542f0ae74a718e0b1c70aa641da4 P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d62e0b282b44003a202b8dcbe6ea6ac in term 1.
I20260709 13:21:25.449014 22742 raft_consensus.cc:2804] T 2e322f5288b64ccca277807d2dc46579 P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:25.449534 22742 raft_consensus.cc:697] T 2e322f5288b64ccca277807d2dc46579 P e20f6c83ae144d698c26b94c2eda023f [term 1 LEADER]: Becoming Leader. State: Replica: e20f6c83ae144d698c26b94c2eda023f, State: Running, Role: LEADER
I20260709 13:21:25.449828 22719 tablet_bootstrap.cc:492] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 7df364d160d64ac4b5cdc4f004324f8b: No bootstrap required, opened a new log
I20260709 13:21:25.450372 22719 ts_tablet_manager.cc:1403] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 7df364d160d64ac4b5cdc4f004324f8b: Time spent bootstrapping tablet: real 0.081s	user 0.013s	sys 0.021s
I20260709 13:21:25.450177 22640 leader_election.cc:304] T 5a1c542f0ae74a718e0b1c70aa641da4 P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2d62e0b282b44003a202b8dcbe6ea6ac, e20f6c83ae144d698c26b94c2eda023f; no voters: 7df364d160d64ac4b5cdc4f004324f8b
I20260709 13:21:25.451025 22717 tablet_bootstrap.cc:492] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 2d62e0b282b44003a202b8dcbe6ea6ac: No bootstrap required, opened a new log
I20260709 13:21:25.450403 22742 consensus_queue.cc:237] T 2e322f5288b64ccca277807d2dc46579 P e20f6c83ae144d698c26b94c2eda023f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.451530 22717 ts_tablet_manager.cc:1403] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 2d62e0b282b44003a202b8dcbe6ea6ac: Time spent bootstrapping tablet: real 0.077s	user 0.010s	sys 0.004s
I20260709 13:21:25.451452 22752 raft_consensus.cc:2804] T 5a1c542f0ae74a718e0b1c70aa641da4 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:25.454569 22716 ts_tablet_manager.cc:1434] T 5e4995abf28446589b7695a1b77ee513 P e20f6c83ae144d698c26b94c2eda023f: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:21:25.458614 22719 raft_consensus.cc:359] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.459729 22719 raft_consensus.cc:385] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:25.460093 22719 raft_consensus.cc:740] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7df364d160d64ac4b5cdc4f004324f8b, State: Initialized, Role: FOLLOWER
I20260709 13:21:25.439919 22723 raft_consensus.cc:515] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.462328 22719 consensus_queue.cc:260] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 7df364d160d64ac4b5cdc4f004324f8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.465123 22719 ts_tablet_manager.cc:1434] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 7df364d160d64ac4b5cdc4f004324f8b: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.000s
I20260709 13:21:25.465952 22525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e4995abf28446589b7695a1b77ee513" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b"
I20260709 13:21:25.466096 22723 leader_election.cc:290] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 election: Requested vote from peers 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823), e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469)
I20260709 13:21:25.466590 22525 raft_consensus.cc:3060] T 5e4995abf28446589b7695a1b77ee513 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:25.467628 22719 tablet_bootstrap.cc:492] T 5a1c542f0ae74a718e0b1c70aa641da4 P 7df364d160d64ac4b5cdc4f004324f8b: Bootstrap starting.
I20260709 13:21:25.466504 22600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e4995abf28446589b7695a1b77ee513" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e20f6c83ae144d698c26b94c2eda023f"
I20260709 13:21:25.471482 22600 raft_consensus.cc:3060] T 5e4995abf28446589b7695a1b77ee513 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:25.478926 22525 raft_consensus.cc:2468] T 5e4995abf28446589b7695a1b77ee513 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d62e0b282b44003a202b8dcbe6ea6ac in term 1.
I20260709 13:21:25.480525 22642 leader_election.cc:304] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d62e0b282b44003a202b8dcbe6ea6ac, 7df364d160d64ac4b5cdc4f004324f8b; no voters: 
I20260709 13:21:25.481518 22723 raft_consensus.cc:2804] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:25.482265 22723 raft_consensus.cc:697] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 LEADER]: Becoming Leader. State: Replica: 2d62e0b282b44003a202b8dcbe6ea6ac, State: Running, Role: LEADER
I20260709 13:21:25.483505 22723 consensus_queue.cc:237] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.487524 22600 raft_consensus.cc:2468] T 5e4995abf28446589b7695a1b77ee513 P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d62e0b282b44003a202b8dcbe6ea6ac in term 1.
I20260709 13:21:25.487370 22717 raft_consensus.cc:359] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.493990 22717 raft_consensus.cc:385] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:25.494391 22717 raft_consensus.cc:740] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d62e0b282b44003a202b8dcbe6ea6ac, State: Initialized, Role: FOLLOWER
I20260709 13:21:25.496510 22719 tablet_bootstrap.cc:654] T 5a1c542f0ae74a718e0b1c70aa641da4 P 7df364d160d64ac4b5cdc4f004324f8b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:25.489923 22752 raft_consensus.cc:697] T 5a1c542f0ae74a718e0b1c70aa641da4 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 LEADER]: Becoming Leader. State: Replica: 2d62e0b282b44003a202b8dcbe6ea6ac, State: Running, Role: LEADER
I20260709 13:21:25.495280 22717 consensus_queue.cc:260] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 2d62e0b282b44003a202b8dcbe6ea6ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.499315 22717 ts_tablet_manager.cc:1434] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 2d62e0b282b44003a202b8dcbe6ea6ac: Time spent starting tablet: real 0.047s	user 0.009s	sys 0.000s
I20260709 13:21:25.500419 22717 tablet_bootstrap.cc:492] T 2e322f5288b64ccca277807d2dc46579 P 2d62e0b282b44003a202b8dcbe6ea6ac: Bootstrap starting.
I20260709 13:21:25.510166 22752 consensus_queue.cc:237] T 5a1c542f0ae74a718e0b1c70aa641da4 P 2d62e0b282b44003a202b8dcbe6ea6ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.514470 22423 catalog_manager.cc:5697] T c0614d551fc342cead195ccaed2aeac3 P 7df364d160d64ac4b5cdc4f004324f8b reported cstate change: term changed from 0 to 1, leader changed from <none> to 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129). New cstate: current_term: 1 leader_uuid: "7df364d160d64ac4b5cdc4f004324f8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:25.519430 22717 tablet_bootstrap.cc:654] T 2e322f5288b64ccca277807d2dc46579 P 2d62e0b282b44003a202b8dcbe6ea6ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:25.532064 22719 tablet_bootstrap.cc:492] T 5a1c542f0ae74a718e0b1c70aa641da4 P 7df364d160d64ac4b5cdc4f004324f8b: No bootstrap required, opened a new log
I20260709 13:21:25.532593 22719 ts_tablet_manager.cc:1403] T 5a1c542f0ae74a718e0b1c70aa641da4 P 7df364d160d64ac4b5cdc4f004324f8b: Time spent bootstrapping tablet: real 0.065s	user 0.016s	sys 0.018s
I20260709 13:21:25.534210 22717 tablet_bootstrap.cc:492] T 2e322f5288b64ccca277807d2dc46579 P 2d62e0b282b44003a202b8dcbe6ea6ac: No bootstrap required, opened a new log
I20260709 13:21:25.534828 22717 ts_tablet_manager.cc:1403] T 2e322f5288b64ccca277807d2dc46579 P 2d62e0b282b44003a202b8dcbe6ea6ac: Time spent bootstrapping tablet: real 0.035s	user 0.012s	sys 0.000s
I20260709 13:21:25.535415 22719 raft_consensus.cc:359] T 5a1c542f0ae74a718e0b1c70aa641da4 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.535193 22425 catalog_manager.cc:5697] T 2e322f5288b64ccca277807d2dc46579 P e20f6c83ae144d698c26b94c2eda023f reported cstate change: term changed from 0 to 1, leader changed from <none> to e20f6c83ae144d698c26b94c2eda023f (127.9.116.130). New cstate: current_term: 1 leader_uuid: "e20f6c83ae144d698c26b94c2eda023f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:25.536278 22719 raft_consensus.cc:385] T 5a1c542f0ae74a718e0b1c70aa641da4 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:25.536615 22719 raft_consensus.cc:740] T 5a1c542f0ae74a718e0b1c70aa641da4 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7df364d160d64ac4b5cdc4f004324f8b, State: Initialized, Role: FOLLOWER
I20260709 13:21:25.537328 22719 consensus_queue.cc:260] T 5a1c542f0ae74a718e0b1c70aa641da4 P 7df364d160d64ac4b5cdc4f004324f8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.537639 22717 raft_consensus.cc:359] T 2e322f5288b64ccca277807d2dc46579 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.537215 22425 catalog_manager.cc:5697] T 68d74bd4ad624659b92ec79d4560a02c P e20f6c83ae144d698c26b94c2eda023f reported cstate change: term changed from 0 to 1, leader changed from <none> to e20f6c83ae144d698c26b94c2eda023f (127.9.116.130). New cstate: current_term: 1 leader_uuid: "e20f6c83ae144d698c26b94c2eda023f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:25.538357 22717 raft_consensus.cc:385] T 2e322f5288b64ccca277807d2dc46579 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:25.538815 22717 raft_consensus.cc:740] T 2e322f5288b64ccca277807d2dc46579 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d62e0b282b44003a202b8dcbe6ea6ac, State: Initialized, Role: FOLLOWER
I20260709 13:21:25.539877 22719 ts_tablet_manager.cc:1434] T 5a1c542f0ae74a718e0b1c70aa641da4 P 7df364d160d64ac4b5cdc4f004324f8b: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.003s
I20260709 13:21:25.539568 22425 catalog_manager.cc:5697] T dded1824a3ef49988cc1b941db0f9955 P e20f6c83ae144d698c26b94c2eda023f reported cstate change: term changed from 0 to 1, leader changed from <none> to e20f6c83ae144d698c26b94c2eda023f (127.9.116.130). New cstate: current_term: 1 leader_uuid: "e20f6c83ae144d698c26b94c2eda023f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:25.540755 22717 consensus_queue.cc:260] T 2e322f5288b64ccca277807d2dc46579 P 2d62e0b282b44003a202b8dcbe6ea6ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.542344 22717 ts_tablet_manager.cc:1434] T 2e322f5288b64ccca277807d2dc46579 P 2d62e0b282b44003a202b8dcbe6ea6ac: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:21:25.553228 22424 catalog_manager.cc:5697] T 5a1c542f0ae74a718e0b1c70aa641da4 P 2d62e0b282b44003a202b8dcbe6ea6ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131). New cstate: current_term: 1 leader_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:25.555112 22424 catalog_manager.cc:5697] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131). New cstate: current_term: 1 leader_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:25.557065 22752 raft_consensus.cc:493] T 2e322f5288b64ccca277807d2dc46579 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:25.557585 22752 raft_consensus.cc:515] T 2e322f5288b64ccca277807d2dc46579 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.559389 22752 leader_election.cc:290] T 2e322f5288b64ccca277807d2dc46579 P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823), e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469)
I20260709 13:21:25.560364 22525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e322f5288b64ccca277807d2dc46579" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b" is_pre_election: true
I20260709 13:21:25.560843 22600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e322f5288b64ccca277807d2dc46579" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e20f6c83ae144d698c26b94c2eda023f" is_pre_election: true
I20260709 13:21:25.561149 22525 raft_consensus.cc:2393] T 2e322f5288b64ccca277807d2dc46579 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2d62e0b282b44003a202b8dcbe6ea6ac in current term 1: Already voted for candidate e20f6c83ae144d698c26b94c2eda023f in this term.
I20260709 13:21:25.562582 22640 leader_election.cc:304] T 2e322f5288b64ccca277807d2dc46579 P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2d62e0b282b44003a202b8dcbe6ea6ac; no voters: 7df364d160d64ac4b5cdc4f004324f8b, e20f6c83ae144d698c26b94c2eda023f
I20260709 13:21:25.563385 22752 raft_consensus.cc:3060] T 2e322f5288b64ccca277807d2dc46579 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:25.569696 22752 raft_consensus.cc:2749] T 2e322f5288b64ccca277807d2dc46579 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:25.592098 22752 raft_consensus.cc:493] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:25.592617 22752 raft_consensus.cc:515] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.594316 22752 leader_election.cc:290] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823), e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469)
I20260709 13:21:25.595299 22525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c5f91cbe6f4a6abbd799e94e0985e1" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b" is_pre_election: true
I20260709 13:21:25.595449 22600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c5f91cbe6f4a6abbd799e94e0985e1" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e20f6c83ae144d698c26b94c2eda023f" is_pre_election: true
I20260709 13:21:25.596037 22525 raft_consensus.cc:2468] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d62e0b282b44003a202b8dcbe6ea6ac in term 0.
I20260709 13:21:25.596161 22600 raft_consensus.cc:2468] T a9c5f91cbe6f4a6abbd799e94e0985e1 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d62e0b282b44003a202b8dcbe6ea6ac in term 0.
I20260709 13:21:25.597343 22642 leader_election.cc:304] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d62e0b282b44003a202b8dcbe6ea6ac, 7df364d160d64ac4b5cdc4f004324f8b; no voters: 
I20260709 13:21:25.598352 22752 raft_consensus.cc:2804] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:25.598755 22752 raft_consensus.cc:493] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:25.599112 22752 raft_consensus.cc:3060] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:25.604240 22752 raft_consensus.cc:515] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.605892 22752 leader_election.cc:290] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 election: Requested vote from peers 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823), e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469)
I20260709 13:21:25.606755 22525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c5f91cbe6f4a6abbd799e94e0985e1" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b"
I20260709 13:21:25.606922 22600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c5f91cbe6f4a6abbd799e94e0985e1" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e20f6c83ae144d698c26b94c2eda023f"
I20260709 13:21:25.607484 22525 raft_consensus.cc:3060] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:25.607573 22600 raft_consensus.cc:3060] T a9c5f91cbe6f4a6abbd799e94e0985e1 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:25.612794 22525 raft_consensus.cc:2468] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d62e0b282b44003a202b8dcbe6ea6ac in term 1.
I20260709 13:21:25.612795 22600 raft_consensus.cc:2468] T a9c5f91cbe6f4a6abbd799e94e0985e1 P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d62e0b282b44003a202b8dcbe6ea6ac in term 1.
I20260709 13:21:25.614053 22642 leader_election.cc:304] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d62e0b282b44003a202b8dcbe6ea6ac, 7df364d160d64ac4b5cdc4f004324f8b; no voters: 
I20260709 13:21:25.614827 22752 raft_consensus.cc:2804] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:25.615338 22752 raft_consensus.cc:697] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 LEADER]: Becoming Leader. State: Replica: 2d62e0b282b44003a202b8dcbe6ea6ac, State: Running, Role: LEADER
I20260709 13:21:25.616003 22752 consensus_queue.cc:237] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 2d62e0b282b44003a202b8dcbe6ea6ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.624114 22424 catalog_manager.cc:5697] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 2d62e0b282b44003a202b8dcbe6ea6ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131). New cstate: current_term: 1 leader_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:25.706588 22752 raft_consensus.cc:493] T 2dad40e2d06244cebe50ec7433000752 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:25.707321 22752 raft_consensus.cc:515] T 2dad40e2d06244cebe50ec7433000752 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.708340 22674 raft_consensus.cc:3060] T 2dad40e2d06244cebe50ec7433000752 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:25.709590 22752 leader_election.cc:290] T 2dad40e2d06244cebe50ec7433000752 P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823), e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469)
I20260709 13:21:25.710464 22525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dad40e2d06244cebe50ec7433000752" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b" is_pre_election: true
I20260709 13:21:25.710775 22600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dad40e2d06244cebe50ec7433000752" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e20f6c83ae144d698c26b94c2eda023f" is_pre_election: true
I20260709 13:21:25.711661 22600 raft_consensus.cc:2393] T 2dad40e2d06244cebe50ec7433000752 P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2d62e0b282b44003a202b8dcbe6ea6ac in current term 1: Already voted for candidate 7df364d160d64ac4b5cdc4f004324f8b in this term.
I20260709 13:21:25.712705 22640 leader_election.cc:304] T 2dad40e2d06244cebe50ec7433000752 P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2d62e0b282b44003a202b8dcbe6ea6ac; no voters: 7df364d160d64ac4b5cdc4f004324f8b, e20f6c83ae144d698c26b94c2eda023f
I20260709 13:21:25.715628 22752 raft_consensus.cc:2749] T 2dad40e2d06244cebe50ec7433000752 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:25.716782 22734 consensus_queue.cc:1048] T 2dad40e2d06244cebe50ec7433000752 P 7df364d160d64ac4b5cdc4f004324f8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:25.730373 22777 consensus_queue.cc:1048] T 2dad40e2d06244cebe50ec7433000752 P 7df364d160d64ac4b5cdc4f004324f8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:25.740557 22741 consensus_queue.cc:1048] T 56bc1b5c8c3143998cb9921b469c071d P e20f6c83ae144d698c26b94c2eda023f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:25.743341 22741 consensus_queue.cc:1048] T 56bc1b5c8c3143998cb9921b469c071d P e20f6c83ae144d698c26b94c2eda023f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:25.787631 22600 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "56bc1b5c8c3143998cb9921b469c071d"
dest_uuid: "e20f6c83ae144d698c26b94c2eda023f"
mode: GRACEFUL
new_leader_uuid: "7df364d160d64ac4b5cdc4f004324f8b"
 from {username='slave'} at 127.0.0.1:50812
I20260709 13:21:25.788211 22600 raft_consensus.cc:606] T 56bc1b5c8c3143998cb9921b469c071d P e20f6c83ae144d698c26b94c2eda023f [term 1 LEADER]: Received request to transfer leadership to TS 7df364d160d64ac4b5cdc4f004324f8b
I20260709 13:21:25.793546 22600 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dded1824a3ef49988cc1b941db0f9955"
dest_uuid: "e20f6c83ae144d698c26b94c2eda023f"
mode: GRACEFUL
new_leader_uuid: "7df364d160d64ac4b5cdc4f004324f8b"
 from {username='slave'} at 127.0.0.1:50812
I20260709 13:21:25.794101 22600 raft_consensus.cc:606] T dded1824a3ef49988cc1b941db0f9955 P e20f6c83ae144d698c26b94c2eda023f [term 1 LEADER]: Received request to transfer leadership to TS 7df364d160d64ac4b5cdc4f004324f8b
I20260709 13:21:25.799628 22600 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2e322f5288b64ccca277807d2dc46579"
dest_uuid: "e20f6c83ae144d698c26b94c2eda023f"
mode: GRACEFUL
new_leader_uuid: "7df364d160d64ac4b5cdc4f004324f8b"
 from {username='slave'} at 127.0.0.1:50812
I20260709 13:21:25.800187 22600 raft_consensus.cc:606] T 2e322f5288b64ccca277807d2dc46579 P e20f6c83ae144d698c26b94c2eda023f [term 1 LEADER]: Received request to transfer leadership to TS 7df364d160d64ac4b5cdc4f004324f8b
I20260709 13:21:25.813413 22674 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5a1c542f0ae74a718e0b1c70aa641da4"
dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac"
mode: GRACEFUL
new_leader_uuid: "e20f6c83ae144d698c26b94c2eda023f"
 from {username='slave'} at 127.0.0.1:48188
I20260709 13:21:25.813979 22674 raft_consensus.cc:606] T 5a1c542f0ae74a718e0b1c70aa641da4 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 LEADER]: Received request to transfer leadership to TS e20f6c83ae144d698c26b94c2eda023f
I20260709 13:21:25.819312 22674 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5e4995abf28446589b7695a1b77ee513"
dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac"
mode: GRACEFUL
new_leader_uuid: "e20f6c83ae144d698c26b94c2eda023f"
 from {username='slave'} at 127.0.0.1:48188
I20260709 13:21:25.819857 22674 raft_consensus.cc:606] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 LEADER]: Received request to transfer leadership to TS e20f6c83ae144d698c26b94c2eda023f
I20260709 13:21:25.828179 22674 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a9c5f91cbe6f4a6abbd799e94e0985e1"
dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac"
mode: GRACEFUL
new_leader_uuid: "e20f6c83ae144d698c26b94c2eda023f"
 from {username='slave'} at 127.0.0.1:48188
I20260709 13:21:25.828768 22674 raft_consensus.cc:606] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 LEADER]: Received request to transfer leadership to TS e20f6c83ae144d698c26b94c2eda023f
I20260709 13:21:25.842307 22525 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2dad40e2d06244cebe50ec7433000752"
dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b"
mode: GRACEFUL
new_leader_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac"
 from {username='slave'} at 127.0.0.1:48032
I20260709 13:21:25.842926 22525 raft_consensus.cc:606] T 2dad40e2d06244cebe50ec7433000752 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 LEADER]: Received request to transfer leadership to TS 2d62e0b282b44003a202b8dcbe6ea6ac
I20260709 13:21:25.863258 22777 consensus_queue.cc:1048] T c0614d551fc342cead195ccaed2aeac3 P 7df364d160d64ac4b5cdc4f004324f8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:25.874058 22777 consensus_queue.cc:1048] T c0614d551fc342cead195ccaed2aeac3 P 7df364d160d64ac4b5cdc4f004324f8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:25.886981 22777 raft_consensus.cc:493] T 5a1c542f0ae74a718e0b1c70aa641da4 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:25.887526 22777 raft_consensus.cc:515] T 5a1c542f0ae74a718e0b1c70aa641da4 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.889642 22742 consensus_queue.cc:1048] T 2e322f5288b64ccca277807d2dc46579 P e20f6c83ae144d698c26b94c2eda023f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:25.890426 22600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a1c542f0ae74a718e0b1c70aa641da4" candidate_uuid: "7df364d160d64ac4b5cdc4f004324f8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e20f6c83ae144d698c26b94c2eda023f" is_pre_election: true
I20260709 13:21:25.891058 22777 leader_election.cc:290] T 5a1c542f0ae74a718e0b1c70aa641da4 P 7df364d160d64ac4b5cdc4f004324f8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469), 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335)
I20260709 13:21:25.891491 22674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a1c542f0ae74a718e0b1c70aa641da4" candidate_uuid: "7df364d160d64ac4b5cdc4f004324f8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" is_pre_election: true
I20260709 13:21:25.892330 22600 raft_consensus.cc:2393] T 5a1c542f0ae74a718e0b1c70aa641da4 P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7df364d160d64ac4b5cdc4f004324f8b in current term 1: Already voted for candidate 2d62e0b282b44003a202b8dcbe6ea6ac in this term.
I20260709 13:21:25.893420 22491 leader_election.cc:304] T 5a1c542f0ae74a718e0b1c70aa641da4 P 7df364d160d64ac4b5cdc4f004324f8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7df364d160d64ac4b5cdc4f004324f8b; no voters: 2d62e0b282b44003a202b8dcbe6ea6ac, e20f6c83ae144d698c26b94c2eda023f
I20260709 13:21:25.894824 22777 raft_consensus.cc:3060] T 5a1c542f0ae74a718e0b1c70aa641da4 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:25.897457 22742 consensus_queue.cc:1048] T dded1824a3ef49988cc1b941db0f9955 P e20f6c83ae144d698c26b94c2eda023f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:25.903401 22777 raft_consensus.cc:2749] T 5a1c542f0ae74a718e0b1c70aa641da4 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:25.907922 22722 consensus_queue.cc:1048] T 2e322f5288b64ccca277807d2dc46579 P e20f6c83ae144d698c26b94c2eda023f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:25.914949 22760 raft_consensus.cc:993] T dded1824a3ef49988cc1b941db0f9955 P e20f6c83ae144d698c26b94c2eda023f: : Instructing follower 7df364d160d64ac4b5cdc4f004324f8b to start an election
I20260709 13:21:25.915453 22760 raft_consensus.cc:1081] T dded1824a3ef49988cc1b941db0f9955 P e20f6c83ae144d698c26b94c2eda023f [term 1 LEADER]: Signalling peer 7df364d160d64ac4b5cdc4f004324f8b to start an election
I20260709 13:21:25.915288 22722 consensus_queue.cc:1048] T 68d74bd4ad624659b92ec79d4560a02c P e20f6c83ae144d698c26b94c2eda023f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:25.921190 22527 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dded1824a3ef49988cc1b941db0f9955"
dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b"
 from {username='slave'} at 127.0.0.1:48022
I20260709 13:21:25.921815 22527 raft_consensus.cc:493] T dded1824a3ef49988cc1b941db0f9955 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:25.922240 22527 raft_consensus.cc:3060] T dded1824a3ef49988cc1b941db0f9955 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:25.927240 22752 consensus_queue.cc:1048] T 5a1c542f0ae74a718e0b1c70aa641da4 P 2d62e0b282b44003a202b8dcbe6ea6ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:25.928793 22722 raft_consensus.cc:993] T 2e322f5288b64ccca277807d2dc46579 P e20f6c83ae144d698c26b94c2eda023f: : Instructing follower 7df364d160d64ac4b5cdc4f004324f8b to start an election
I20260709 13:21:25.928977 22790 consensus_queue.cc:1048] T dded1824a3ef49988cc1b941db0f9955 P e20f6c83ae144d698c26b94c2eda023f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:25.929385 22742 raft_consensus.cc:1081] T 2e322f5288b64ccca277807d2dc46579 P e20f6c83ae144d698c26b94c2eda023f [term 1 LEADER]: Signalling peer 7df364d160d64ac4b5cdc4f004324f8b to start an election
I20260709 13:21:25.931061 22527 raft_consensus.cc:515] T dded1824a3ef49988cc1b941db0f9955 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.931507 22525 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2e322f5288b64ccca277807d2dc46579"
dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b"
 from {username='slave'} at 127.0.0.1:48022
I20260709 13:21:25.932236 22525 raft_consensus.cc:493] T 2e322f5288b64ccca277807d2dc46579 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:25.932722 22525 raft_consensus.cc:3060] T 2e322f5288b64ccca277807d2dc46579 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:25.935644 22527 leader_election.cc:290] T dded1824a3ef49988cc1b941db0f9955 P 7df364d160d64ac4b5cdc4f004324f8b [CANDIDATE]: Term 2 election: Requested vote from peers e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469), 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335)
I20260709 13:21:25.936051 22674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dded1824a3ef49988cc1b941db0f9955" candidate_uuid: "7df364d160d64ac4b5cdc4f004324f8b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac"
I20260709 13:21:25.936879 22674 raft_consensus.cc:3060] T dded1824a3ef49988cc1b941db0f9955 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:25.947741 22674 raft_consensus.cc:2468] T dded1824a3ef49988cc1b941db0f9955 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7df364d160d64ac4b5cdc4f004324f8b in term 2.
I20260709 13:21:25.947546 22600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dded1824a3ef49988cc1b941db0f9955" candidate_uuid: "7df364d160d64ac4b5cdc4f004324f8b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e20f6c83ae144d698c26b94c2eda023f"
I20260709 13:21:25.949039 22600 raft_consensus.cc:3055] T dded1824a3ef49988cc1b941db0f9955 P e20f6c83ae144d698c26b94c2eda023f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:25.949152 22675 raft_consensus.cc:1240] T dded1824a3ef49988cc1b941db0f9955 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Rejecting Update request from peer e20f6c83ae144d698c26b94c2eda023f for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:21:25.949481 22493 leader_election.cc:304] T dded1824a3ef49988cc1b941db0f9955 P 7df364d160d64ac4b5cdc4f004324f8b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d62e0b282b44003a202b8dcbe6ea6ac, 7df364d160d64ac4b5cdc4f004324f8b; no voters: 
I20260709 13:21:25.952859 22777 raft_consensus.cc:2804] T dded1824a3ef49988cc1b941db0f9955 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:25.952688 22742 consensus_queue.cc:1059] T dded1824a3ef49988cc1b941db0f9955 P e20f6c83ae144d698c26b94c2eda023f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:25.954192 22777 raft_consensus.cc:697] T dded1824a3ef49988cc1b941db0f9955 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 LEADER]: Becoming Leader. State: Replica: 7df364d160d64ac4b5cdc4f004324f8b, State: Running, Role: LEADER
I20260709 13:21:25.955420 22777 consensus_queue.cc:237] T dded1824a3ef49988cc1b941db0f9955 P 7df364d160d64ac4b5cdc4f004324f8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.957785 22752 consensus_queue.cc:1048] T 5a1c542f0ae74a718e0b1c70aa641da4 P 2d62e0b282b44003a202b8dcbe6ea6ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:25.949512 22600 raft_consensus.cc:740] T dded1824a3ef49988cc1b941db0f9955 P e20f6c83ae144d698c26b94c2eda023f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e20f6c83ae144d698c26b94c2eda023f, State: Running, Role: LEADER
I20260709 13:21:25.959550 22525 raft_consensus.cc:515] T 2e322f5288b64ccca277807d2dc46579 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.960474 22752 raft_consensus.cc:993] T 5a1c542f0ae74a718e0b1c70aa641da4 P 2d62e0b282b44003a202b8dcbe6ea6ac: : Instructing follower e20f6c83ae144d698c26b94c2eda023f to start an election
I20260709 13:21:25.960017 22600 consensus_queue.cc:260] T dded1824a3ef49988cc1b941db0f9955 P e20f6c83ae144d698c26b94c2eda023f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.960980 22752 raft_consensus.cc:1081] T 5a1c542f0ae74a718e0b1c70aa641da4 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 LEADER]: Signalling peer e20f6c83ae144d698c26b94c2eda023f to start an election
I20260709 13:21:25.961757 22600 raft_consensus.cc:3060] T dded1824a3ef49988cc1b941db0f9955 P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:25.964210 22525 leader_election.cc:290] T 2e322f5288b64ccca277807d2dc46579 P 7df364d160d64ac4b5cdc4f004324f8b [CANDIDATE]: Term 2 election: Requested vote from peers e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469), 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335)
I20260709 13:21:25.968744 22674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e322f5288b64ccca277807d2dc46579" candidate_uuid: "7df364d160d64ac4b5cdc4f004324f8b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac"
I20260709 13:21:25.968748 22602 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5a1c542f0ae74a718e0b1c70aa641da4"
dest_uuid: "e20f6c83ae144d698c26b94c2eda023f"
 from {username='slave'} at 127.0.0.1:50800
I20260709 13:21:25.970412 22602 raft_consensus.cc:493] T 5a1c542f0ae74a718e0b1c70aa641da4 P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:25.971098 22602 raft_consensus.cc:3060] T 5a1c542f0ae74a718e0b1c70aa641da4 P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:25.975742 22599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e322f5288b64ccca277807d2dc46579" candidate_uuid: "7df364d160d64ac4b5cdc4f004324f8b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e20f6c83ae144d698c26b94c2eda023f"
I20260709 13:21:25.976799 22599 raft_consensus.cc:3055] T 2e322f5288b64ccca277807d2dc46579 P e20f6c83ae144d698c26b94c2eda023f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:25.977205 22599 raft_consensus.cc:740] T 2e322f5288b64ccca277807d2dc46579 P e20f6c83ae144d698c26b94c2eda023f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e20f6c83ae144d698c26b94c2eda023f, State: Running, Role: LEADER
I20260709 13:21:25.969513 22674 raft_consensus.cc:3060] T 2e322f5288b64ccca277807d2dc46579 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:25.978305 22599 consensus_queue.cc:260] T 2e322f5288b64ccca277807d2dc46579 P e20f6c83ae144d698c26b94c2eda023f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.980019 22599 raft_consensus.cc:3060] T 2e322f5288b64ccca277807d2dc46579 P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:25.991707 22760 consensus_queue.cc:1048] T 68d74bd4ad624659b92ec79d4560a02c P e20f6c83ae144d698c26b94c2eda023f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:25.993839 22600 raft_consensus.cc:2468] T dded1824a3ef49988cc1b941db0f9955 P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7df364d160d64ac4b5cdc4f004324f8b in term 2.
W20260709 13:21:25.994496 22742 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 13:21:25.997681 22602 raft_consensus.cc:515] T 5a1c542f0ae74a718e0b1c70aa641da4 P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:25.998898 22601 raft_consensus.cc:1240] T 5a1c542f0ae74a718e0b1c70aa641da4 P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Rejecting Update request from peer 2d62e0b282b44003a202b8dcbe6ea6ac for earlier term 1. Current term is 2. Ops: []
I20260709 13:21:26.002542 22767 consensus_queue.cc:1059] T 5a1c542f0ae74a718e0b1c70aa641da4 P 2d62e0b282b44003a202b8dcbe6ea6ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:26.003304 22599 raft_consensus.cc:2468] T 2e322f5288b64ccca277807d2dc46579 P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7df364d160d64ac4b5cdc4f004324f8b in term 2.
I20260709 13:21:26.003547 22767 raft_consensus.cc:3055] T 5a1c542f0ae74a718e0b1c70aa641da4 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:26.004042 22767 raft_consensus.cc:740] T 5a1c542f0ae74a718e0b1c70aa641da4 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2d62e0b282b44003a202b8dcbe6ea6ac, State: Running, Role: LEADER
I20260709 13:21:25.990343 22674 raft_consensus.cc:2468] T 2e322f5288b64ccca277807d2dc46579 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7df364d160d64ac4b5cdc4f004324f8b in term 2.
I20260709 13:21:26.004863 22767 consensus_queue.cc:260] T 5a1c542f0ae74a718e0b1c70aa641da4 P 2d62e0b282b44003a202b8dcbe6ea6ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:26.005461 22491 leader_election.cc:304] T 2e322f5288b64ccca277807d2dc46579 P 7df364d160d64ac4b5cdc4f004324f8b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7df364d160d64ac4b5cdc4f004324f8b, e20f6c83ae144d698c26b94c2eda023f; no voters: 
I20260709 13:21:26.005944 22525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a1c542f0ae74a718e0b1c70aa641da4" candidate_uuid: "e20f6c83ae144d698c26b94c2eda023f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b"
I20260709 13:21:26.006534 22777 raft_consensus.cc:2804] T 2e322f5288b64ccca277807d2dc46579 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:26.007078 22777 raft_consensus.cc:697] T 2e322f5288b64ccca277807d2dc46579 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 LEADER]: Becoming Leader. State: Replica: 7df364d160d64ac4b5cdc4f004324f8b, State: Running, Role: LEADER
I20260709 13:21:26.007447 22767 raft_consensus.cc:3060] T 5a1c542f0ae74a718e0b1c70aa641da4 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:26.008517 22602 leader_election.cc:290] T 5a1c542f0ae74a718e0b1c70aa641da4 P e20f6c83ae144d698c26b94c2eda023f [CANDIDATE]: Term 2 election: Requested vote from peers 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823), 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335)
I20260709 13:21:26.007941 22777 consensus_queue.cc:237] T 2e322f5288b64ccca277807d2dc46579 P 7df364d160d64ac4b5cdc4f004324f8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:26.009864 22674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a1c542f0ae74a718e0b1c70aa641da4" candidate_uuid: "e20f6c83ae144d698c26b94c2eda023f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac"
I20260709 13:21:26.013290 22423 catalog_manager.cc:5697] T dded1824a3ef49988cc1b941db0f9955 P 7df364d160d64ac4b5cdc4f004324f8b reported cstate change: term changed from 1 to 2, leader changed from e20f6c83ae144d698c26b94c2eda023f (127.9.116.130) to 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129). New cstate: current_term: 2 leader_uuid: "7df364d160d64ac4b5cdc4f004324f8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:26.021927 22674 raft_consensus.cc:2468] T 5a1c542f0ae74a718e0b1c70aa641da4 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e20f6c83ae144d698c26b94c2eda023f in term 2.
I20260709 13:21:26.023731 22568 leader_election.cc:304] T 5a1c542f0ae74a718e0b1c70aa641da4 P e20f6c83ae144d698c26b94c2eda023f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2d62e0b282b44003a202b8dcbe6ea6ac, e20f6c83ae144d698c26b94c2eda023f; no voters: 7df364d160d64ac4b5cdc4f004324f8b
I20260709 13:21:26.024601 22742 raft_consensus.cc:2804] T 5a1c542f0ae74a718e0b1c70aa641da4 P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:26.025045 22742 raft_consensus.cc:697] T 5a1c542f0ae74a718e0b1c70aa641da4 P e20f6c83ae144d698c26b94c2eda023f [term 2 LEADER]: Becoming Leader. State: Replica: e20f6c83ae144d698c26b94c2eda023f, State: Running, Role: LEADER
I20260709 13:21:26.025887 22742 consensus_queue.cc:237] T 5a1c542f0ae74a718e0b1c70aa641da4 P e20f6c83ae144d698c26b94c2eda023f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:26.029173 22423 catalog_manager.cc:5697] T 2e322f5288b64ccca277807d2dc46579 P 7df364d160d64ac4b5cdc4f004324f8b reported cstate change: term changed from 1 to 2, leader changed from e20f6c83ae144d698c26b94c2eda023f (127.9.116.130) to 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129). New cstate: current_term: 2 leader_uuid: "7df364d160d64ac4b5cdc4f004324f8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:26.031596 22767 consensus_queue.cc:1048] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:26.034592 22422 catalog_manager.cc:5697] T 5a1c542f0ae74a718e0b1c70aa641da4 P e20f6c83ae144d698c26b94c2eda023f reported cstate change: term changed from 1 to 2, leader changed from 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131) to e20f6c83ae144d698c26b94c2eda023f (127.9.116.130). New cstate: current_term: 2 leader_uuid: "e20f6c83ae144d698c26b94c2eda023f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:26.040271 22752 raft_consensus.cc:993] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac: : Instructing follower e20f6c83ae144d698c26b94c2eda023f to start an election
I20260709 13:21:26.040874 22752 raft_consensus.cc:1081] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 LEADER]: Signalling peer e20f6c83ae144d698c26b94c2eda023f to start an election
I20260709 13:21:26.043888 22599 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5e4995abf28446589b7695a1b77ee513"
dest_uuid: "e20f6c83ae144d698c26b94c2eda023f"
 from {username='slave'} at 127.0.0.1:50800
I20260709 13:21:26.045461 22599 raft_consensus.cc:493] T 5e4995abf28446589b7695a1b77ee513 P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:26.045933 22599 raft_consensus.cc:3060] T 5e4995abf28446589b7695a1b77ee513 P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:26.051409 22723 consensus_queue.cc:1048] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:26.054142 22599 raft_consensus.cc:515] T 5e4995abf28446589b7695a1b77ee513 P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:26.062238 22599 leader_election.cc:290] T 5e4995abf28446589b7695a1b77ee513 P e20f6c83ae144d698c26b94c2eda023f [CANDIDATE]: Term 2 election: Requested vote from peers 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823), 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335)
I20260709 13:21:26.065101 22527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e4995abf28446589b7695a1b77ee513" candidate_uuid: "e20f6c83ae144d698c26b94c2eda023f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b"
I20260709 13:21:26.065171 22674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e4995abf28446589b7695a1b77ee513" candidate_uuid: "e20f6c83ae144d698c26b94c2eda023f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac"
I20260709 13:21:26.065804 22527 raft_consensus.cc:3060] T 5e4995abf28446589b7695a1b77ee513 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:26.065958 22674 raft_consensus.cc:3055] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:26.066419 22674 raft_consensus.cc:740] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2d62e0b282b44003a202b8dcbe6ea6ac, State: Running, Role: LEADER
I20260709 13:21:26.067333 22674 consensus_queue.cc:260] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:26.068661 22674 raft_consensus.cc:3060] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:26.074435 22674 raft_consensus.cc:2468] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e20f6c83ae144d698c26b94c2eda023f in term 2.
I20260709 13:21:26.075583 22568 leader_election.cc:304] T 5e4995abf28446589b7695a1b77ee513 P e20f6c83ae144d698c26b94c2eda023f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d62e0b282b44003a202b8dcbe6ea6ac, e20f6c83ae144d698c26b94c2eda023f; no voters: 
I20260709 13:21:26.076354 22742 raft_consensus.cc:2804] T 5e4995abf28446589b7695a1b77ee513 P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:26.076463 22527 raft_consensus.cc:2468] T 5e4995abf28446589b7695a1b77ee513 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e20f6c83ae144d698c26b94c2eda023f in term 2.
I20260709 13:21:26.076866 22742 raft_consensus.cc:697] T 5e4995abf28446589b7695a1b77ee513 P e20f6c83ae144d698c26b94c2eda023f [term 2 LEADER]: Becoming Leader. State: Replica: e20f6c83ae144d698c26b94c2eda023f, State: Running, Role: LEADER
I20260709 13:21:26.077808 22742 consensus_queue.cc:237] T 5e4995abf28446589b7695a1b77ee513 P e20f6c83ae144d698c26b94c2eda023f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:26.085547 22422 catalog_manager.cc:5697] T 5e4995abf28446589b7695a1b77ee513 P e20f6c83ae144d698c26b94c2eda023f reported cstate change: term changed from 1 to 2, leader changed from 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131) to e20f6c83ae144d698c26b94c2eda023f (127.9.116.130). New cstate: current_term: 2 leader_uuid: "e20f6c83ae144d698c26b94c2eda023f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:26.164408 22723 consensus_queue.cc:1048] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 2d62e0b282b44003a202b8dcbe6ea6ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:26.170814 22767 raft_consensus.cc:993] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 2d62e0b282b44003a202b8dcbe6ea6ac: : Instructing follower e20f6c83ae144d698c26b94c2eda023f to start an election
I20260709 13:21:26.171334 22752 raft_consensus.cc:1081] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 LEADER]: Signalling peer e20f6c83ae144d698c26b94c2eda023f to start an election
I20260709 13:21:26.173872 22602 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a9c5f91cbe6f4a6abbd799e94e0985e1"
dest_uuid: "e20f6c83ae144d698c26b94c2eda023f"
 from {username='slave'} at 127.0.0.1:50800
I20260709 13:21:26.174500 22602 raft_consensus.cc:493] T a9c5f91cbe6f4a6abbd799e94e0985e1 P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:26.174933 22602 raft_consensus.cc:3060] T a9c5f91cbe6f4a6abbd799e94e0985e1 P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:26.177883 22752 consensus_queue.cc:1048] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 2d62e0b282b44003a202b8dcbe6ea6ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:26.185819 22602 raft_consensus.cc:515] T a9c5f91cbe6f4a6abbd799e94e0985e1 P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:26.188735 22602 leader_election.cc:290] T a9c5f91cbe6f4a6abbd799e94e0985e1 P e20f6c83ae144d698c26b94c2eda023f [CANDIDATE]: Term 2 election: Requested vote from peers 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823), 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335)
I20260709 13:21:26.190040 22527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c5f91cbe6f4a6abbd799e94e0985e1" candidate_uuid: "e20f6c83ae144d698c26b94c2eda023f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b"
I20260709 13:21:26.190824 22527 raft_consensus.cc:3060] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:26.191558 22674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c5f91cbe6f4a6abbd799e94e0985e1" candidate_uuid: "e20f6c83ae144d698c26b94c2eda023f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac"
I20260709 13:21:26.192229 22674 raft_consensus.cc:3055] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:26.192586 22674 raft_consensus.cc:740] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2d62e0b282b44003a202b8dcbe6ea6ac, State: Running, Role: LEADER
I20260709 13:21:26.193413 22674 consensus_queue.cc:260] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 2d62e0b282b44003a202b8dcbe6ea6ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:26.194041 22777 raft_consensus.cc:993] T 2dad40e2d06244cebe50ec7433000752 P 7df364d160d64ac4b5cdc4f004324f8b: : Instructing follower 2d62e0b282b44003a202b8dcbe6ea6ac to start an election
I20260709 13:21:26.194612 22777 raft_consensus.cc:1081] T 2dad40e2d06244cebe50ec7433000752 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 LEADER]: Signalling peer 2d62e0b282b44003a202b8dcbe6ea6ac to start an election
I20260709 13:21:26.194869 22674 raft_consensus.cc:3060] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:26.196434 22676 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2dad40e2d06244cebe50ec7433000752"
dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac"
 from {username='slave'} at 127.0.0.1:48174
I20260709 13:21:26.197049 22676 raft_consensus.cc:493] T 2dad40e2d06244cebe50ec7433000752 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:26.197413 22676 raft_consensus.cc:3060] T 2dad40e2d06244cebe50ec7433000752 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:26.199029 22527 raft_consensus.cc:2468] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e20f6c83ae144d698c26b94c2eda023f in term 2.
I20260709 13:21:26.200394 22568 leader_election.cc:304] T a9c5f91cbe6f4a6abbd799e94e0985e1 P e20f6c83ae144d698c26b94c2eda023f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7df364d160d64ac4b5cdc4f004324f8b, e20f6c83ae144d698c26b94c2eda023f; no voters: 
I20260709 13:21:26.201243 22674 raft_consensus.cc:2468] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e20f6c83ae144d698c26b94c2eda023f in term 2.
I20260709 13:21:26.201486 22742 raft_consensus.cc:2804] T a9c5f91cbe6f4a6abbd799e94e0985e1 P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:26.202946 22742 raft_consensus.cc:697] T a9c5f91cbe6f4a6abbd799e94e0985e1 P e20f6c83ae144d698c26b94c2eda023f [term 2 LEADER]: Becoming Leader. State: Replica: e20f6c83ae144d698c26b94c2eda023f, State: Running, Role: LEADER
I20260709 13:21:26.203837 22742 consensus_queue.cc:237] T a9c5f91cbe6f4a6abbd799e94e0985e1 P e20f6c83ae144d698c26b94c2eda023f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:26.205099 22676 raft_consensus.cc:515] T 2dad40e2d06244cebe50ec7433000752 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:26.208060 22676 leader_election.cc:290] T 2dad40e2d06244cebe50ec7433000752 P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 2 election: Requested vote from peers 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823), e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469)
I20260709 13:21:26.208154 22527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dad40e2d06244cebe50ec7433000752" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b"
I20260709 13:21:26.208992 22527 raft_consensus.cc:3055] T 2dad40e2d06244cebe50ec7433000752 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:26.209419 22527 raft_consensus.cc:740] T 2dad40e2d06244cebe50ec7433000752 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7df364d160d64ac4b5cdc4f004324f8b, State: Running, Role: LEADER
I20260709 13:21:26.208518 22602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dad40e2d06244cebe50ec7433000752" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e20f6c83ae144d698c26b94c2eda023f"
I20260709 13:21:26.210016 22602 raft_consensus.cc:3060] T 2dad40e2d06244cebe50ec7433000752 P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:26.210227 22527 consensus_queue.cc:260] T 2dad40e2d06244cebe50ec7433000752 P 7df364d160d64ac4b5cdc4f004324f8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:26.211460 22527 raft_consensus.cc:3060] T 2dad40e2d06244cebe50ec7433000752 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:26.212239 22422 catalog_manager.cc:5697] T a9c5f91cbe6f4a6abbd799e94e0985e1 P e20f6c83ae144d698c26b94c2eda023f reported cstate change: term changed from 1 to 2, leader changed from 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131) to e20f6c83ae144d698c26b94c2eda023f (127.9.116.130). New cstate: current_term: 2 leader_uuid: "e20f6c83ae144d698c26b94c2eda023f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:26.217032 22602 raft_consensus.cc:2468] T 2dad40e2d06244cebe50ec7433000752 P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d62e0b282b44003a202b8dcbe6ea6ac in term 2.
I20260709 13:21:26.217234 22527 raft_consensus.cc:2468] T 2dad40e2d06244cebe50ec7433000752 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d62e0b282b44003a202b8dcbe6ea6ac in term 2.
I20260709 13:21:26.218365 22640 leader_election.cc:304] T 2dad40e2d06244cebe50ec7433000752 P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d62e0b282b44003a202b8dcbe6ea6ac, e20f6c83ae144d698c26b94c2eda023f; no voters: 
I20260709 13:21:26.219221 22752 raft_consensus.cc:2804] T 2dad40e2d06244cebe50ec7433000752 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:26.219738 22752 raft_consensus.cc:697] T 2dad40e2d06244cebe50ec7433000752 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 LEADER]: Becoming Leader. State: Replica: 2d62e0b282b44003a202b8dcbe6ea6ac, State: Running, Role: LEADER
I20260709 13:21:26.220463 22752 consensus_queue.cc:237] T 2dad40e2d06244cebe50ec7433000752 P 2d62e0b282b44003a202b8dcbe6ea6ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:26.227165 22422 catalog_manager.cc:5697] T 2dad40e2d06244cebe50ec7433000752 P 2d62e0b282b44003a202b8dcbe6ea6ac reported cstate change: term changed from 1 to 2, leader changed from 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129) to 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131). New cstate: current_term: 2 leader_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:26.245721 22760 raft_consensus.cc:993] T 56bc1b5c8c3143998cb9921b469c071d P e20f6c83ae144d698c26b94c2eda023f: : Instructing follower 7df364d160d64ac4b5cdc4f004324f8b to start an election
I20260709 13:21:26.246196 22742 raft_consensus.cc:1081] T 56bc1b5c8c3143998cb9921b469c071d P e20f6c83ae144d698c26b94c2eda023f [term 1 LEADER]: Signalling peer 7df364d160d64ac4b5cdc4f004324f8b to start an election
I20260709 13:21:26.247789 22527 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "56bc1b5c8c3143998cb9921b469c071d"
dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b"
 from {username='slave'} at 127.0.0.1:48022
I20260709 13:21:26.248303 22527 raft_consensus.cc:493] T 56bc1b5c8c3143998cb9921b469c071d P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:26.248618 22527 raft_consensus.cc:3060] T 56bc1b5c8c3143998cb9921b469c071d P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:26.253762 22527 raft_consensus.cc:515] T 56bc1b5c8c3143998cb9921b469c071d P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:26.255652 22527 leader_election.cc:290] T 56bc1b5c8c3143998cb9921b469c071d P 7df364d160d64ac4b5cdc4f004324f8b [CANDIDATE]: Term 2 election: Requested vote from peers e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469), 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335)
I20260709 13:21:26.256656 22602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56bc1b5c8c3143998cb9921b469c071d" candidate_uuid: "7df364d160d64ac4b5cdc4f004324f8b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e20f6c83ae144d698c26b94c2eda023f"
I20260709 13:21:26.256819 22676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56bc1b5c8c3143998cb9921b469c071d" candidate_uuid: "7df364d160d64ac4b5cdc4f004324f8b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac"
I20260709 13:21:26.257387 22602 raft_consensus.cc:3055] T 56bc1b5c8c3143998cb9921b469c071d P e20f6c83ae144d698c26b94c2eda023f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:26.257512 22676 raft_consensus.cc:3060] T 56bc1b5c8c3143998cb9921b469c071d P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:26.257678 22602 raft_consensus.cc:740] T 56bc1b5c8c3143998cb9921b469c071d P e20f6c83ae144d698c26b94c2eda023f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e20f6c83ae144d698c26b94c2eda023f, State: Running, Role: LEADER
I20260709 13:21:26.258633 22602 consensus_queue.cc:260] T 56bc1b5c8c3143998cb9921b469c071d P e20f6c83ae144d698c26b94c2eda023f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:26.259747 22602 raft_consensus.cc:3060] T 56bc1b5c8c3143998cb9921b469c071d P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:26.264145 22676 raft_consensus.cc:2468] T 56bc1b5c8c3143998cb9921b469c071d P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7df364d160d64ac4b5cdc4f004324f8b in term 2.
I20260709 13:21:26.265496 22602 raft_consensus.cc:2468] T 56bc1b5c8c3143998cb9921b469c071d P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7df364d160d64ac4b5cdc4f004324f8b in term 2.
I20260709 13:21:26.265635 22493 leader_election.cc:304] T 56bc1b5c8c3143998cb9921b469c071d P 7df364d160d64ac4b5cdc4f004324f8b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d62e0b282b44003a202b8dcbe6ea6ac, 7df364d160d64ac4b5cdc4f004324f8b; no voters: 
I20260709 13:21:26.266556 22724 raft_consensus.cc:2804] T 56bc1b5c8c3143998cb9921b469c071d P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:26.267181 22724 raft_consensus.cc:697] T 56bc1b5c8c3143998cb9921b469c071d P 7df364d160d64ac4b5cdc4f004324f8b [term 2 LEADER]: Becoming Leader. State: Replica: 7df364d160d64ac4b5cdc4f004324f8b, State: Running, Role: LEADER
I20260709 13:21:26.267973 22724 consensus_queue.cc:237] T 56bc1b5c8c3143998cb9921b469c071d P 7df364d160d64ac4b5cdc4f004324f8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:26.274724 22422 catalog_manager.cc:5697] T 56bc1b5c8c3143998cb9921b469c071d P 7df364d160d64ac4b5cdc4f004324f8b reported cstate change: term changed from 1 to 2, leader changed from e20f6c83ae144d698c26b94c2eda023f (127.9.116.130) to 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129). New cstate: current_term: 2 leader_uuid: "7df364d160d64ac4b5cdc4f004324f8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:26.357052 22602 raft_consensus.cc:1275] T dded1824a3ef49988cc1b941db0f9955 P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Refusing update from remote peer 7df364d160d64ac4b5cdc4f004324f8b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:26.358422 22724 consensus_queue.cc:1048] T dded1824a3ef49988cc1b941db0f9955 P 7df364d160d64ac4b5cdc4f004324f8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:26.366209 22676 raft_consensus.cc:1275] T dded1824a3ef49988cc1b941db0f9955 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Refusing update from remote peer 7df364d160d64ac4b5cdc4f004324f8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:26.367863 22777 consensus_queue.cc:1048] T dded1824a3ef49988cc1b941db0f9955 P 7df364d160d64ac4b5cdc4f004324f8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:26.399039 22602 raft_consensus.cc:1275] T 2e322f5288b64ccca277807d2dc46579 P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Refusing update from remote peer 7df364d160d64ac4b5cdc4f004324f8b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:26.400365 22777 consensus_queue.cc:1048] T 2e322f5288b64ccca277807d2dc46579 P 7df364d160d64ac4b5cdc4f004324f8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:26.412346 22676 raft_consensus.cc:1275] T 2e322f5288b64ccca277807d2dc46579 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Refusing update from remote peer 7df364d160d64ac4b5cdc4f004324f8b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:26.413725 22734 consensus_queue.cc:1048] T 2e322f5288b64ccca277807d2dc46579 P 7df364d160d64ac4b5cdc4f004324f8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:26.517491 22676 raft_consensus.cc:1275] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Refusing update from remote peer e20f6c83ae144d698c26b94c2eda023f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:26.518975 22742 consensus_queue.cc:1048] T 5e4995abf28446589b7695a1b77ee513 P e20f6c83ae144d698c26b94c2eda023f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:26.527554 22527 raft_consensus.cc:1275] T 5e4995abf28446589b7695a1b77ee513 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Refusing update from remote peer e20f6c83ae144d698c26b94c2eda023f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:26.529646 22802 consensus_queue.cc:1048] T 5e4995abf28446589b7695a1b77ee513 P e20f6c83ae144d698c26b94c2eda023f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:21:26.584883 22676 raft_consensus.cc:1275] T 5a1c542f0ae74a718e0b1c70aa641da4 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Refusing update from remote peer e20f6c83ae144d698c26b94c2eda023f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:26.586216 22802 consensus_queue.cc:1048] T 5a1c542f0ae74a718e0b1c70aa641da4 P e20f6c83ae144d698c26b94c2eda023f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:26.594974 22527 raft_consensus.cc:3060] T 5a1c542f0ae74a718e0b1c70aa641da4 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:26.603011 22527 raft_consensus.cc:1275] T 5a1c542f0ae74a718e0b1c70aa641da4 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Refusing update from remote peer e20f6c83ae144d698c26b94c2eda023f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:26.604911 22802 consensus_queue.cc:1048] T 5a1c542f0ae74a718e0b1c70aa641da4 P e20f6c83ae144d698c26b94c2eda023f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:26.727008 22676 raft_consensus.cc:1275] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Refusing update from remote peer e20f6c83ae144d698c26b94c2eda023f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:26.728559 22742 consensus_queue.cc:1048] T a9c5f91cbe6f4a6abbd799e94e0985e1 P e20f6c83ae144d698c26b94c2eda023f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:26.737553 22527 raft_consensus.cc:1275] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Refusing update from remote peer e20f6c83ae144d698c26b94c2eda023f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:26.739127 22760 consensus_queue.cc:1048] T a9c5f91cbe6f4a6abbd799e94e0985e1 P e20f6c83ae144d698c26b94c2eda023f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:26.757009 22527 raft_consensus.cc:1275] T 2dad40e2d06244cebe50ec7433000752 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Refusing update from remote peer 2d62e0b282b44003a202b8dcbe6ea6ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:26.758285 22752 consensus_queue.cc:1048] T 2dad40e2d06244cebe50ec7433000752 P 2d62e0b282b44003a202b8dcbe6ea6ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:26.768846 22602 raft_consensus.cc:1275] T 2dad40e2d06244cebe50ec7433000752 P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Refusing update from remote peer 2d62e0b282b44003a202b8dcbe6ea6ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:26.770968 22752 consensus_queue.cc:1048] T 2dad40e2d06244cebe50ec7433000752 P 2d62e0b282b44003a202b8dcbe6ea6ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:26.867956 22676 raft_consensus.cc:1275] T 56bc1b5c8c3143998cb9921b469c071d P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Refusing update from remote peer 7df364d160d64ac4b5cdc4f004324f8b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:26.869467 22734 consensus_queue.cc:1048] T 56bc1b5c8c3143998cb9921b469c071d P 7df364d160d64ac4b5cdc4f004324f8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:26.873997 22602 raft_consensus.cc:1275] T 56bc1b5c8c3143998cb9921b469c071d P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Refusing update from remote peer 7df364d160d64ac4b5cdc4f004324f8b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:26.875555 22734 consensus_queue.cc:1048] T 56bc1b5c8c3143998cb9921b469c071d P 7df364d160d64ac4b5cdc4f004324f8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:28.871121 22422 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44484:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:21:28.873350 22422 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:21:28.924683 22513 tablet_service.cc:1511] Processing CreateTablet for tablet 0f952d6650154a4a98f94a75b916d44b (DEFAULT_TABLE table=soft_deleted_table [id=0d2ccd2882d348079d7ec3e42d5bdf7f]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:21:28.926064 22513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f952d6650154a4a98f94a75b916d44b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.926605 22588 tablet_service.cc:1511] Processing CreateTablet for tablet 0f952d6650154a4a98f94a75b916d44b (DEFAULT_TABLE table=soft_deleted_table [id=0d2ccd2882d348079d7ec3e42d5bdf7f]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:21:28.927942 22588 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f952d6650154a4a98f94a75b916d44b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.927802 22663 tablet_service.cc:1511] Processing CreateTablet for tablet 0f952d6650154a4a98f94a75b916d44b (DEFAULT_TABLE table=soft_deleted_table [id=0d2ccd2882d348079d7ec3e42d5bdf7f]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:21:28.929139 22663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f952d6650154a4a98f94a75b916d44b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.933391 22659 tablet_service.cc:1511] Processing CreateTablet for tablet ef14a8f3b6344312bd294edcabb2f312 (DEFAULT_TABLE table=soft_deleted_table [id=0d2ccd2882d348079d7ec3e42d5bdf7f]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:21:28.933727 22510 tablet_service.cc:1511] Processing CreateTablet for tablet ef14a8f3b6344312bd294edcabb2f312 (DEFAULT_TABLE table=soft_deleted_table [id=0d2ccd2882d348079d7ec3e42d5bdf7f]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:21:28.934110 22514 tablet_service.cc:1511] Processing CreateTablet for tablet 47e33cdb6a154e8781b3e36a95eed353 (DEFAULT_TABLE table=soft_deleted_table [id=0d2ccd2882d348079d7ec3e42d5bdf7f]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:21:28.934929 22659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef14a8f3b6344312bd294edcabb2f312. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.936013 22590 tablet_service.cc:1511] Processing CreateTablet for tablet 47e33cdb6a154e8781b3e36a95eed353 (DEFAULT_TABLE table=soft_deleted_table [id=0d2ccd2882d348079d7ec3e42d5bdf7f]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:21:28.936292 22591 tablet_service.cc:1511] Processing CreateTablet for tablet ef14a8f3b6344312bd294edcabb2f312 (DEFAULT_TABLE table=soft_deleted_table [id=0d2ccd2882d348079d7ec3e42d5bdf7f]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:21:28.937347 22590 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47e33cdb6a154e8781b3e36a95eed353. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.937945 22591 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef14a8f3b6344312bd294edcabb2f312. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.934974 22510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef14a8f3b6344312bd294edcabb2f312. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.944530 22514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47e33cdb6a154e8781b3e36a95eed353. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.961551 22823 tablet_bootstrap.cc:492] T 0f952d6650154a4a98f94a75b916d44b P 2d62e0b282b44003a202b8dcbe6ea6ac: Bootstrap starting.
I20260709 13:21:28.966413 22665 tablet_service.cc:1511] Processing CreateTablet for tablet 47e33cdb6a154e8781b3e36a95eed353 (DEFAULT_TABLE table=soft_deleted_table [id=0d2ccd2882d348079d7ec3e42d5bdf7f]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:21:28.967900 22665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47e33cdb6a154e8781b3e36a95eed353. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.969444 22512 tablet_service.cc:1511] Processing CreateTablet for tablet 5b97872c6f3b4a288ff8ade4569da6f1 (DEFAULT_TABLE table=soft_deleted_table [id=0d2ccd2882d348079d7ec3e42d5bdf7f]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:21:28.969971 22823 tablet_bootstrap.cc:654] T 0f952d6650154a4a98f94a75b916d44b P 2d62e0b282b44003a202b8dcbe6ea6ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:28.970845 22512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b97872c6f3b4a288ff8ade4569da6f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.973052 22587 tablet_service.cc:1511] Processing CreateTablet for tablet 5b97872c6f3b4a288ff8ade4569da6f1 (DEFAULT_TABLE table=soft_deleted_table [id=0d2ccd2882d348079d7ec3e42d5bdf7f]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:21:28.974412 22587 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b97872c6f3b4a288ff8ade4569da6f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.979777 22663 tablet_service.cc:1511] Processing CreateTablet for tablet 5b97872c6f3b4a288ff8ade4569da6f1 (DEFAULT_TABLE table=soft_deleted_table [id=0d2ccd2882d348079d7ec3e42d5bdf7f]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:21:28.981235 22663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b97872c6f3b4a288ff8ade4569da6f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.984647 22511 tablet_service.cc:1511] Processing CreateTablet for tablet 2f75c1bbbd7b4dc09412c016798ccd50 (DEFAULT_TABLE table=soft_deleted_table [id=0d2ccd2882d348079d7ec3e42d5bdf7f]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:21:28.986158 22511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f75c1bbbd7b4dc09412c016798ccd50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.988216 22590 tablet_service.cc:1511] Processing CreateTablet for tablet 2f75c1bbbd7b4dc09412c016798ccd50 (DEFAULT_TABLE table=soft_deleted_table [id=0d2ccd2882d348079d7ec3e42d5bdf7f]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:21:28.989660 22590 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f75c1bbbd7b4dc09412c016798ccd50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.989810 22658 tablet_service.cc:1511] Processing CreateTablet for tablet 2f75c1bbbd7b4dc09412c016798ccd50 (DEFAULT_TABLE table=soft_deleted_table [id=0d2ccd2882d348079d7ec3e42d5bdf7f]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:21:28.991250 22658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f75c1bbbd7b4dc09412c016798ccd50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.998196 22515 tablet_service.cc:1511] Processing CreateTablet for tablet e1e94cad244443feb24f679271460852 (DEFAULT_TABLE table=soft_deleted_table [id=0d2ccd2882d348079d7ec3e42d5bdf7f]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:21:28.999670 22515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1e94cad244443feb24f679271460852. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:29.009903 22589 tablet_service.cc:1511] Processing CreateTablet for tablet e1e94cad244443feb24f679271460852 (DEFAULT_TABLE table=soft_deleted_table [id=0d2ccd2882d348079d7ec3e42d5bdf7f]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:21:29.011547 22589 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1e94cad244443feb24f679271460852. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:29.014213 22666 tablet_service.cc:1511] Processing CreateTablet for tablet e1e94cad244443feb24f679271460852 (DEFAULT_TABLE table=soft_deleted_table [id=0d2ccd2882d348079d7ec3e42d5bdf7f]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:21:29.015671 22666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1e94cad244443feb24f679271460852. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:29.018239 22825 tablet_bootstrap.cc:492] T 47e33cdb6a154e8781b3e36a95eed353 P 7df364d160d64ac4b5cdc4f004324f8b: Bootstrap starting.
I20260709 13:21:29.026732 22516 tablet_service.cc:1511] Processing CreateTablet for tablet a3b0925412bd4dceabec4f898eb43973 (DEFAULT_TABLE table=soft_deleted_table [id=0d2ccd2882d348079d7ec3e42d5bdf7f]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:21:29.028383 22516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3b0925412bd4dceabec4f898eb43973. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:29.029521 22591 tablet_service.cc:1511] Processing CreateTablet for tablet a3b0925412bd4dceabec4f898eb43973 (DEFAULT_TABLE table=soft_deleted_table [id=0d2ccd2882d348079d7ec3e42d5bdf7f]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:21:29.031090 22591 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3b0925412bd4dceabec4f898eb43973. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:29.035254 22823 tablet_bootstrap.cc:492] T 0f952d6650154a4a98f94a75b916d44b P 2d62e0b282b44003a202b8dcbe6ea6ac: No bootstrap required, opened a new log
I20260709 13:21:29.035985 22823 ts_tablet_manager.cc:1403] T 0f952d6650154a4a98f94a75b916d44b P 2d62e0b282b44003a202b8dcbe6ea6ac: Time spent bootstrapping tablet: real 0.075s	user 0.005s	sys 0.007s
I20260709 13:21:29.040067 22662 tablet_service.cc:1511] Processing CreateTablet for tablet a3b0925412bd4dceabec4f898eb43973 (DEFAULT_TABLE table=soft_deleted_table [id=0d2ccd2882d348079d7ec3e42d5bdf7f]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:21:29.041621 22662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3b0925412bd4dceabec4f898eb43973. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:29.051285 22586 tablet_service.cc:1511] Processing CreateTablet for tablet 93dd95f0dfb8463291a74921c46665dd (DEFAULT_TABLE table=soft_deleted_table [id=0d2ccd2882d348079d7ec3e42d5bdf7f]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:21:29.052819 22586 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93dd95f0dfb8463291a74921c46665dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:29.053977 22659 tablet_service.cc:1511] Processing CreateTablet for tablet 93dd95f0dfb8463291a74921c46665dd (DEFAULT_TABLE table=soft_deleted_table [id=0d2ccd2882d348079d7ec3e42d5bdf7f]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:21:29.055426 22659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93dd95f0dfb8463291a74921c46665dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:29.055984 22515 tablet_service.cc:1511] Processing CreateTablet for tablet 93dd95f0dfb8463291a74921c46665dd (DEFAULT_TABLE table=soft_deleted_table [id=0d2ccd2882d348079d7ec3e42d5bdf7f]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:21:29.057495 22515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93dd95f0dfb8463291a74921c46665dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:29.073846 22512 tablet_service.cc:1511] Processing CreateTablet for tablet 2d4c5545efd2494a81249b0dbbfbc56c (DEFAULT_TABLE table=soft_deleted_table [id=0d2ccd2882d348079d7ec3e42d5bdf7f]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:21:29.075335 22512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d4c5545efd2494a81249b0dbbfbc56c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:29.076540 22825 tablet_bootstrap.cc:654] T 47e33cdb6a154e8781b3e36a95eed353 P 7df364d160d64ac4b5cdc4f004324f8b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.039361 22823 raft_consensus.cc:359] T 0f952d6650154a4a98f94a75b916d44b P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.082329 22823 raft_consensus.cc:385] T 0f952d6650154a4a98f94a75b916d44b P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.082769 22823 raft_consensus.cc:740] T 0f952d6650154a4a98f94a75b916d44b P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d62e0b282b44003a202b8dcbe6ea6ac, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.084031 22823 consensus_queue.cc:260] T 0f952d6650154a4a98f94a75b916d44b P 2d62e0b282b44003a202b8dcbe6ea6ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.082882 22824 tablet_bootstrap.cc:492] T 0f952d6650154a4a98f94a75b916d44b P e20f6c83ae144d698c26b94c2eda023f: Bootstrap starting.
I20260709 13:21:29.096177 22589 tablet_service.cc:1511] Processing CreateTablet for tablet 2d4c5545efd2494a81249b0dbbfbc56c (DEFAULT_TABLE table=soft_deleted_table [id=0d2ccd2882d348079d7ec3e42d5bdf7f]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:21:29.097733 22589 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d4c5545efd2494a81249b0dbbfbc56c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:29.100178 22665 tablet_service.cc:1511] Processing CreateTablet for tablet 2d4c5545efd2494a81249b0dbbfbc56c (DEFAULT_TABLE table=soft_deleted_table [id=0d2ccd2882d348079d7ec3e42d5bdf7f]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:21:29.101617 22665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d4c5545efd2494a81249b0dbbfbc56c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:29.104126 22824 tablet_bootstrap.cc:654] T 0f952d6650154a4a98f94a75b916d44b P e20f6c83ae144d698c26b94c2eda023f: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.110683 22823 ts_tablet_manager.cc:1434] T 0f952d6650154a4a98f94a75b916d44b P 2d62e0b282b44003a202b8dcbe6ea6ac: Time spent starting tablet: real 0.074s	user 0.007s	sys 0.000s
W20260709 13:21:29.115173 22473 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:21:29.119568 22825 tablet_bootstrap.cc:492] T 47e33cdb6a154e8781b3e36a95eed353 P 7df364d160d64ac4b5cdc4f004324f8b: No bootstrap required, opened a new log
I20260709 13:21:29.120268 22825 ts_tablet_manager.cc:1403] T 47e33cdb6a154e8781b3e36a95eed353 P 7df364d160d64ac4b5cdc4f004324f8b: Time spent bootstrapping tablet: real 0.102s	user 0.013s	sys 0.000s
I20260709 13:21:29.122008 22823 tablet_bootstrap.cc:492] T 2f75c1bbbd7b4dc09412c016798ccd50 P 2d62e0b282b44003a202b8dcbe6ea6ac: Bootstrap starting.
I20260709 13:21:29.123490 22825 raft_consensus.cc:359] T 47e33cdb6a154e8781b3e36a95eed353 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.124372 22825 raft_consensus.cc:385] T 47e33cdb6a154e8781b3e36a95eed353 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.124984 22825 raft_consensus.cc:740] T 47e33cdb6a154e8781b3e36a95eed353 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7df364d160d64ac4b5cdc4f004324f8b, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.125917 22825 consensus_queue.cc:260] T 47e33cdb6a154e8781b3e36a95eed353 P 7df364d160d64ac4b5cdc4f004324f8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.128947 22825 ts_tablet_manager.cc:1434] T 47e33cdb6a154e8781b3e36a95eed353 P 7df364d160d64ac4b5cdc4f004324f8b: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:21:29.130071 22825 tablet_bootstrap.cc:492] T 0f952d6650154a4a98f94a75b916d44b P 7df364d160d64ac4b5cdc4f004324f8b: Bootstrap starting.
I20260709 13:21:29.131145 22824 tablet_bootstrap.cc:492] T 0f952d6650154a4a98f94a75b916d44b P e20f6c83ae144d698c26b94c2eda023f: No bootstrap required, opened a new log
I20260709 13:21:29.131759 22824 ts_tablet_manager.cc:1403] T 0f952d6650154a4a98f94a75b916d44b P e20f6c83ae144d698c26b94c2eda023f: Time spent bootstrapping tablet: real 0.052s	user 0.016s	sys 0.000s
I20260709 13:21:29.133301 22823 tablet_bootstrap.cc:654] T 2f75c1bbbd7b4dc09412c016798ccd50 P 2d62e0b282b44003a202b8dcbe6ea6ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.136432 22824 raft_consensus.cc:359] T 0f952d6650154a4a98f94a75b916d44b P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.137580 22824 raft_consensus.cc:385] T 0f952d6650154a4a98f94a75b916d44b P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.137722 22825 tablet_bootstrap.cc:654] T 0f952d6650154a4a98f94a75b916d44b P 7df364d160d64ac4b5cdc4f004324f8b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.137938 22824 raft_consensus.cc:740] T 0f952d6650154a4a98f94a75b916d44b P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e20f6c83ae144d698c26b94c2eda023f, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.138923 22824 consensus_queue.cc:260] T 0f952d6650154a4a98f94a75b916d44b P e20f6c83ae144d698c26b94c2eda023f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.141459 22824 ts_tablet_manager.cc:1434] T 0f952d6650154a4a98f94a75b916d44b P e20f6c83ae144d698c26b94c2eda023f: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:21:29.143170 22823 tablet_bootstrap.cc:492] T 2f75c1bbbd7b4dc09412c016798ccd50 P 2d62e0b282b44003a202b8dcbe6ea6ac: No bootstrap required, opened a new log
I20260709 13:21:29.143999 22824 tablet_bootstrap.cc:492] T 47e33cdb6a154e8781b3e36a95eed353 P e20f6c83ae144d698c26b94c2eda023f: Bootstrap starting.
I20260709 13:21:29.143996 22823 ts_tablet_manager.cc:1403] T 2f75c1bbbd7b4dc09412c016798ccd50 P 2d62e0b282b44003a202b8dcbe6ea6ac: Time spent bootstrapping tablet: real 0.022s	user 0.014s	sys 0.000s
I20260709 13:21:29.146852 22825 tablet_bootstrap.cc:492] T 0f952d6650154a4a98f94a75b916d44b P 7df364d160d64ac4b5cdc4f004324f8b: No bootstrap required, opened a new log
I20260709 13:21:29.147449 22825 ts_tablet_manager.cc:1403] T 0f952d6650154a4a98f94a75b916d44b P 7df364d160d64ac4b5cdc4f004324f8b: Time spent bootstrapping tablet: real 0.018s	user 0.010s	sys 0.004s
I20260709 13:21:29.147977 22823 raft_consensus.cc:359] T 2f75c1bbbd7b4dc09412c016798ccd50 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.148835 22823 raft_consensus.cc:385] T 2f75c1bbbd7b4dc09412c016798ccd50 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.149185 22823 raft_consensus.cc:740] T 2f75c1bbbd7b4dc09412c016798ccd50 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d62e0b282b44003a202b8dcbe6ea6ac, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.150005 22823 consensus_queue.cc:260] T 2f75c1bbbd7b4dc09412c016798ccd50 P 2d62e0b282b44003a202b8dcbe6ea6ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.150776 22825 raft_consensus.cc:359] T 0f952d6650154a4a98f94a75b916d44b P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.151593 22825 raft_consensus.cc:385] T 0f952d6650154a4a98f94a75b916d44b P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.151921 22825 raft_consensus.cc:740] T 0f952d6650154a4a98f94a75b916d44b P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7df364d160d64ac4b5cdc4f004324f8b, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.152575 22823 ts_tablet_manager.cc:1434] T 2f75c1bbbd7b4dc09412c016798ccd50 P 2d62e0b282b44003a202b8dcbe6ea6ac: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:21:29.152701 22825 consensus_queue.cc:260] T 0f952d6650154a4a98f94a75b916d44b P 7df364d160d64ac4b5cdc4f004324f8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.153811 22823 tablet_bootstrap.cc:492] T ef14a8f3b6344312bd294edcabb2f312 P 2d62e0b282b44003a202b8dcbe6ea6ac: Bootstrap starting.
I20260709 13:21:29.155198 22825 ts_tablet_manager.cc:1434] T 0f952d6650154a4a98f94a75b916d44b P 7df364d160d64ac4b5cdc4f004324f8b: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:21:29.156479 22825 tablet_bootstrap.cc:492] T e1e94cad244443feb24f679271460852 P 7df364d160d64ac4b5cdc4f004324f8b: Bootstrap starting.
I20260709 13:21:29.161340 22823 tablet_bootstrap.cc:654] T ef14a8f3b6344312bd294edcabb2f312 P 2d62e0b282b44003a202b8dcbe6ea6ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.166775 22824 tablet_bootstrap.cc:654] T 47e33cdb6a154e8781b3e36a95eed353 P e20f6c83ae144d698c26b94c2eda023f: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.171499 22825 tablet_bootstrap.cc:654] T e1e94cad244443feb24f679271460852 P 7df364d160d64ac4b5cdc4f004324f8b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.181560 22823 tablet_bootstrap.cc:492] T ef14a8f3b6344312bd294edcabb2f312 P 2d62e0b282b44003a202b8dcbe6ea6ac: No bootstrap required, opened a new log
I20260709 13:21:29.182070 22823 ts_tablet_manager.cc:1403] T ef14a8f3b6344312bd294edcabb2f312 P 2d62e0b282b44003a202b8dcbe6ea6ac: Time spent bootstrapping tablet: real 0.028s	user 0.012s	sys 0.013s
I20260709 13:21:29.182435 22824 tablet_bootstrap.cc:492] T 47e33cdb6a154e8781b3e36a95eed353 P e20f6c83ae144d698c26b94c2eda023f: No bootstrap required, opened a new log
I20260709 13:21:29.182579 22825 tablet_bootstrap.cc:492] T e1e94cad244443feb24f679271460852 P 7df364d160d64ac4b5cdc4f004324f8b: No bootstrap required, opened a new log
I20260709 13:21:29.183141 22825 ts_tablet_manager.cc:1403] T e1e94cad244443feb24f679271460852 P 7df364d160d64ac4b5cdc4f004324f8b: Time spent bootstrapping tablet: real 0.027s	user 0.011s	sys 0.006s
I20260709 13:21:29.183146 22824 ts_tablet_manager.cc:1403] T 47e33cdb6a154e8781b3e36a95eed353 P e20f6c83ae144d698c26b94c2eda023f: Time spent bootstrapping tablet: real 0.039s	user 0.016s	sys 0.007s
I20260709 13:21:29.184559 22823 raft_consensus.cc:359] T ef14a8f3b6344312bd294edcabb2f312 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.185187 22823 raft_consensus.cc:385] T ef14a8f3b6344312bd294edcabb2f312 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.185434 22823 raft_consensus.cc:740] T ef14a8f3b6344312bd294edcabb2f312 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d62e0b282b44003a202b8dcbe6ea6ac, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.186121 22823 consensus_queue.cc:260] T ef14a8f3b6344312bd294edcabb2f312 P 2d62e0b282b44003a202b8dcbe6ea6ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.186355 22825 raft_consensus.cc:359] T e1e94cad244443feb24f679271460852 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.186555 22824 raft_consensus.cc:359] T 47e33cdb6a154e8781b3e36a95eed353 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.187233 22825 raft_consensus.cc:385] T e1e94cad244443feb24f679271460852 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.187354 22824 raft_consensus.cc:385] T 47e33cdb6a154e8781b3e36a95eed353 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.187649 22825 raft_consensus.cc:740] T e1e94cad244443feb24f679271460852 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7df364d160d64ac4b5cdc4f004324f8b, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.187709 22824 raft_consensus.cc:740] T 47e33cdb6a154e8781b3e36a95eed353 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e20f6c83ae144d698c26b94c2eda023f, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.188328 22823 ts_tablet_manager.cc:1434] T ef14a8f3b6344312bd294edcabb2f312 P 2d62e0b282b44003a202b8dcbe6ea6ac: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:21:29.188572 22825 consensus_queue.cc:260] T e1e94cad244443feb24f679271460852 P 7df364d160d64ac4b5cdc4f004324f8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.188704 22824 consensus_queue.cc:260] T 47e33cdb6a154e8781b3e36a95eed353 P e20f6c83ae144d698c26b94c2eda023f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.189396 22823 tablet_bootstrap.cc:492] T 5b97872c6f3b4a288ff8ade4569da6f1 P 2d62e0b282b44003a202b8dcbe6ea6ac: Bootstrap starting.
I20260709 13:21:29.191239 22825 ts_tablet_manager.cc:1434] T e1e94cad244443feb24f679271460852 P 7df364d160d64ac4b5cdc4f004324f8b: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
I20260709 13:21:29.192333 22825 tablet_bootstrap.cc:492] T ef14a8f3b6344312bd294edcabb2f312 P 7df364d160d64ac4b5cdc4f004324f8b: Bootstrap starting.
I20260709 13:21:29.197022 22823 tablet_bootstrap.cc:654] T 5b97872c6f3b4a288ff8ade4569da6f1 P 2d62e0b282b44003a202b8dcbe6ea6ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.199108 22825 tablet_bootstrap.cc:654] T ef14a8f3b6344312bd294edcabb2f312 P 7df364d160d64ac4b5cdc4f004324f8b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.208930 22823 tablet_bootstrap.cc:492] T 5b97872c6f3b4a288ff8ade4569da6f1 P 2d62e0b282b44003a202b8dcbe6ea6ac: No bootstrap required, opened a new log
I20260709 13:21:29.209528 22823 ts_tablet_manager.cc:1403] T 5b97872c6f3b4a288ff8ade4569da6f1 P 2d62e0b282b44003a202b8dcbe6ea6ac: Time spent bootstrapping tablet: real 0.020s	user 0.015s	sys 0.004s
I20260709 13:21:29.210711 22825 tablet_bootstrap.cc:492] T ef14a8f3b6344312bd294edcabb2f312 P 7df364d160d64ac4b5cdc4f004324f8b: No bootstrap required, opened a new log
I20260709 13:21:29.211365 22825 ts_tablet_manager.cc:1403] T ef14a8f3b6344312bd294edcabb2f312 P 7df364d160d64ac4b5cdc4f004324f8b: Time spent bootstrapping tablet: real 0.019s	user 0.014s	sys 0.004s
I20260709 13:21:29.211480 22824 ts_tablet_manager.cc:1434] T 47e33cdb6a154e8781b3e36a95eed353 P e20f6c83ae144d698c26b94c2eda023f: Time spent starting tablet: real 0.028s	user 0.001s	sys 0.006s
I20260709 13:21:29.212605 22824 tablet_bootstrap.cc:492] T ef14a8f3b6344312bd294edcabb2f312 P e20f6c83ae144d698c26b94c2eda023f: Bootstrap starting.
I20260709 13:21:29.212574 22823 raft_consensus.cc:359] T 5b97872c6f3b4a288ff8ade4569da6f1 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.213696 22823 raft_consensus.cc:385] T 5b97872c6f3b4a288ff8ade4569da6f1 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.214056 22823 raft_consensus.cc:740] T 5b97872c6f3b4a288ff8ade4569da6f1 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d62e0b282b44003a202b8dcbe6ea6ac, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.214895 22823 consensus_queue.cc:260] T 5b97872c6f3b4a288ff8ade4569da6f1 P 2d62e0b282b44003a202b8dcbe6ea6ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.215197 22825 raft_consensus.cc:359] T ef14a8f3b6344312bd294edcabb2f312 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.216111 22825 raft_consensus.cc:385] T ef14a8f3b6344312bd294edcabb2f312 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.216502 22825 raft_consensus.cc:740] T ef14a8f3b6344312bd294edcabb2f312 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7df364d160d64ac4b5cdc4f004324f8b, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.217505 22823 ts_tablet_manager.cc:1434] T 5b97872c6f3b4a288ff8ade4569da6f1 P 2d62e0b282b44003a202b8dcbe6ea6ac: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:21:29.217389 22825 consensus_queue.cc:260] T ef14a8f3b6344312bd294edcabb2f312 P 7df364d160d64ac4b5cdc4f004324f8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.218621 22823 tablet_bootstrap.cc:492] T e1e94cad244443feb24f679271460852 P 2d62e0b282b44003a202b8dcbe6ea6ac: Bootstrap starting.
I20260709 13:21:29.225363 22823 tablet_bootstrap.cc:654] T e1e94cad244443feb24f679271460852 P 2d62e0b282b44003a202b8dcbe6ea6ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.227147 22825 ts_tablet_manager.cc:1434] T ef14a8f3b6344312bd294edcabb2f312 P 7df364d160d64ac4b5cdc4f004324f8b: Time spent starting tablet: real 0.015s	user 0.010s	sys 0.004s
I20260709 13:21:29.228044 22824 tablet_bootstrap.cc:654] T ef14a8f3b6344312bd294edcabb2f312 P e20f6c83ae144d698c26b94c2eda023f: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.228518 22825 tablet_bootstrap.cc:492] T 2f75c1bbbd7b4dc09412c016798ccd50 P 7df364d160d64ac4b5cdc4f004324f8b: Bootstrap starting.
I20260709 13:21:29.235882 22825 tablet_bootstrap.cc:654] T 2f75c1bbbd7b4dc09412c016798ccd50 P 7df364d160d64ac4b5cdc4f004324f8b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.238474 22823 tablet_bootstrap.cc:492] T e1e94cad244443feb24f679271460852 P 2d62e0b282b44003a202b8dcbe6ea6ac: No bootstrap required, opened a new log
I20260709 13:21:29.239473 22823 ts_tablet_manager.cc:1403] T e1e94cad244443feb24f679271460852 P 2d62e0b282b44003a202b8dcbe6ea6ac: Time spent bootstrapping tablet: real 0.021s	user 0.008s	sys 0.011s
I20260709 13:21:29.242460 22824 tablet_bootstrap.cc:492] T ef14a8f3b6344312bd294edcabb2f312 P e20f6c83ae144d698c26b94c2eda023f: No bootstrap required, opened a new log
I20260709 13:21:29.242619 22823 raft_consensus.cc:359] T e1e94cad244443feb24f679271460852 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.243235 22824 ts_tablet_manager.cc:1403] T ef14a8f3b6344312bd294edcabb2f312 P e20f6c83ae144d698c26b94c2eda023f: Time spent bootstrapping tablet: real 0.031s	user 0.012s	sys 0.010s
I20260709 13:21:29.243525 22823 raft_consensus.cc:385] T e1e94cad244443feb24f679271460852 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.243865 22823 raft_consensus.cc:740] T e1e94cad244443feb24f679271460852 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d62e0b282b44003a202b8dcbe6ea6ac, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.244680 22823 consensus_queue.cc:260] T e1e94cad244443feb24f679271460852 P 2d62e0b282b44003a202b8dcbe6ea6ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.246840 22825 tablet_bootstrap.cc:492] T 2f75c1bbbd7b4dc09412c016798ccd50 P 7df364d160d64ac4b5cdc4f004324f8b: No bootstrap required, opened a new log
I20260709 13:21:29.247349 22823 ts_tablet_manager.cc:1434] T e1e94cad244443feb24f679271460852 P 2d62e0b282b44003a202b8dcbe6ea6ac: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:21:29.246901 22824 raft_consensus.cc:359] T ef14a8f3b6344312bd294edcabb2f312 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.247666 22825 ts_tablet_manager.cc:1403] T 2f75c1bbbd7b4dc09412c016798ccd50 P 7df364d160d64ac4b5cdc4f004324f8b: Time spent bootstrapping tablet: real 0.019s	user 0.012s	sys 0.003s
I20260709 13:21:29.248040 22824 raft_consensus.cc:385] T ef14a8f3b6344312bd294edcabb2f312 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.248582 22823 tablet_bootstrap.cc:492] T 47e33cdb6a154e8781b3e36a95eed353 P 2d62e0b282b44003a202b8dcbe6ea6ac: Bootstrap starting.
I20260709 13:21:29.248463 22824 raft_consensus.cc:740] T ef14a8f3b6344312bd294edcabb2f312 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e20f6c83ae144d698c26b94c2eda023f, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.253518 22824 consensus_queue.cc:260] T ef14a8f3b6344312bd294edcabb2f312 P e20f6c83ae144d698c26b94c2eda023f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.253824 22825 raft_consensus.cc:359] T 2f75c1bbbd7b4dc09412c016798ccd50 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.255039 22825 raft_consensus.cc:385] T 2f75c1bbbd7b4dc09412c016798ccd50 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.255672 22825 raft_consensus.cc:740] T 2f75c1bbbd7b4dc09412c016798ccd50 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7df364d160d64ac4b5cdc4f004324f8b, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.255832 22823 tablet_bootstrap.cc:654] T 47e33cdb6a154e8781b3e36a95eed353 P 2d62e0b282b44003a202b8dcbe6ea6ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.256695 22825 consensus_queue.cc:260] T 2f75c1bbbd7b4dc09412c016798ccd50 P 7df364d160d64ac4b5cdc4f004324f8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.260285 22824 ts_tablet_manager.cc:1434] T ef14a8f3b6344312bd294edcabb2f312 P e20f6c83ae144d698c26b94c2eda023f: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:21:29.260643 22825 ts_tablet_manager.cc:1434] T 2f75c1bbbd7b4dc09412c016798ccd50 P 7df364d160d64ac4b5cdc4f004324f8b: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:21:29.261881 22824 tablet_bootstrap.cc:492] T 2f75c1bbbd7b4dc09412c016798ccd50 P e20f6c83ae144d698c26b94c2eda023f: Bootstrap starting.
I20260709 13:21:29.261947 22825 tablet_bootstrap.cc:492] T 5b97872c6f3b4a288ff8ade4569da6f1 P 7df364d160d64ac4b5cdc4f004324f8b: Bootstrap starting.
I20260709 13:21:29.271031 22825 tablet_bootstrap.cc:654] T 5b97872c6f3b4a288ff8ade4569da6f1 P 7df364d160d64ac4b5cdc4f004324f8b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.272310 22824 tablet_bootstrap.cc:654] T 2f75c1bbbd7b4dc09412c016798ccd50 P e20f6c83ae144d698c26b94c2eda023f: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.274649 22752 raft_consensus.cc:493] T 0f952d6650154a4a98f94a75b916d44b P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:29.275441 22752 raft_consensus.cc:515] T 0f952d6650154a4a98f94a75b916d44b P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.279485 22527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f952d6650154a4a98f94a75b916d44b" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b" is_pre_election: true
I20260709 13:21:29.280200 22527 raft_consensus.cc:2468] T 0f952d6650154a4a98f94a75b916d44b P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d62e0b282b44003a202b8dcbe6ea6ac in term 0.
I20260709 13:21:29.280856 22602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f952d6650154a4a98f94a75b916d44b" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e20f6c83ae144d698c26b94c2eda023f" is_pre_election: true
I20260709 13:21:29.281620 22602 raft_consensus.cc:2468] T 0f952d6650154a4a98f94a75b916d44b P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d62e0b282b44003a202b8dcbe6ea6ac in term 0.
I20260709 13:21:29.281713 22642 leader_election.cc:304] T 0f952d6650154a4a98f94a75b916d44b P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d62e0b282b44003a202b8dcbe6ea6ac, 7df364d160d64ac4b5cdc4f004324f8b; no voters: 
I20260709 13:21:29.283608 22752 leader_election.cc:290] T 0f952d6650154a4a98f94a75b916d44b P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823), e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469)
I20260709 13:21:29.284394 22752 raft_consensus.cc:2804] T 0f952d6650154a4a98f94a75b916d44b P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:29.284848 22752 raft_consensus.cc:493] T 0f952d6650154a4a98f94a75b916d44b P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:29.285269 22752 raft_consensus.cc:3060] T 0f952d6650154a4a98f94a75b916d44b P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.286365 22824 tablet_bootstrap.cc:492] T 2f75c1bbbd7b4dc09412c016798ccd50 P e20f6c83ae144d698c26b94c2eda023f: No bootstrap required, opened a new log
I20260709 13:21:29.287238 22824 ts_tablet_manager.cc:1403] T 2f75c1bbbd7b4dc09412c016798ccd50 P e20f6c83ae144d698c26b94c2eda023f: Time spent bootstrapping tablet: real 0.026s	user 0.011s	sys 0.013s
I20260709 13:21:29.291482 22824 raft_consensus.cc:359] T 2f75c1bbbd7b4dc09412c016798ccd50 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.292567 22824 raft_consensus.cc:385] T 2f75c1bbbd7b4dc09412c016798ccd50 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.292968 22824 raft_consensus.cc:740] T 2f75c1bbbd7b4dc09412c016798ccd50 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e20f6c83ae144d698c26b94c2eda023f, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.294283 22824 consensus_queue.cc:260] T 2f75c1bbbd7b4dc09412c016798ccd50 P e20f6c83ae144d698c26b94c2eda023f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.297685 22824 ts_tablet_manager.cc:1434] T 2f75c1bbbd7b4dc09412c016798ccd50 P e20f6c83ae144d698c26b94c2eda023f: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:21:29.298772 22825 tablet_bootstrap.cc:492] T 5b97872c6f3b4a288ff8ade4569da6f1 P 7df364d160d64ac4b5cdc4f004324f8b: No bootstrap required, opened a new log
I20260709 13:21:29.299198 22824 tablet_bootstrap.cc:492] T e1e94cad244443feb24f679271460852 P e20f6c83ae144d698c26b94c2eda023f: Bootstrap starting.
I20260709 13:21:29.299952 22825 ts_tablet_manager.cc:1403] T 5b97872c6f3b4a288ff8ade4569da6f1 P 7df364d160d64ac4b5cdc4f004324f8b: Time spent bootstrapping tablet: real 0.038s	user 0.012s	sys 0.011s
I20260709 13:21:29.303786 22825 raft_consensus.cc:359] T 5b97872c6f3b4a288ff8ade4569da6f1 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.308430 22825 raft_consensus.cc:385] T 5b97872c6f3b4a288ff8ade4569da6f1 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.309242 22825 raft_consensus.cc:740] T 5b97872c6f3b4a288ff8ade4569da6f1 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7df364d160d64ac4b5cdc4f004324f8b, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.309396 22824 tablet_bootstrap.cc:654] T e1e94cad244443feb24f679271460852 P e20f6c83ae144d698c26b94c2eda023f: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.310987 22825 consensus_queue.cc:260] T 5b97872c6f3b4a288ff8ade4569da6f1 P 7df364d160d64ac4b5cdc4f004324f8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.315024 22825 ts_tablet_manager.cc:1434] T 5b97872c6f3b4a288ff8ade4569da6f1 P 7df364d160d64ac4b5cdc4f004324f8b: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:21:29.318048 22825 tablet_bootstrap.cc:492] T a3b0925412bd4dceabec4f898eb43973 P 7df364d160d64ac4b5cdc4f004324f8b: Bootstrap starting.
I20260709 13:21:29.323848 22824 tablet_bootstrap.cc:492] T e1e94cad244443feb24f679271460852 P e20f6c83ae144d698c26b94c2eda023f: No bootstrap required, opened a new log
I20260709 13:21:29.324749 22824 ts_tablet_manager.cc:1403] T e1e94cad244443feb24f679271460852 P e20f6c83ae144d698c26b94c2eda023f: Time spent bootstrapping tablet: real 0.026s	user 0.014s	sys 0.008s
I20260709 13:21:29.331113 22825 tablet_bootstrap.cc:654] T a3b0925412bd4dceabec4f898eb43973 P 7df364d160d64ac4b5cdc4f004324f8b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.333384 22824 raft_consensus.cc:359] T e1e94cad244443feb24f679271460852 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.334571 22824 raft_consensus.cc:385] T e1e94cad244443feb24f679271460852 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.335039 22824 raft_consensus.cc:740] T e1e94cad244443feb24f679271460852 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e20f6c83ae144d698c26b94c2eda023f, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.336350 22824 consensus_queue.cc:260] T e1e94cad244443feb24f679271460852 P e20f6c83ae144d698c26b94c2eda023f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.339744 22824 ts_tablet_manager.cc:1434] T e1e94cad244443feb24f679271460852 P e20f6c83ae144d698c26b94c2eda023f: Time spent starting tablet: real 0.015s	user 0.012s	sys 0.000s
I20260709 13:21:29.349200 22824 tablet_bootstrap.cc:492] T 93dd95f0dfb8463291a74921c46665dd P e20f6c83ae144d698c26b94c2eda023f: Bootstrap starting.
W20260709 13:21:29.351836 22642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.129:42823, user_credentials={real_user=slave}} blocked reactor thread for 70215.5us
I20260709 13:21:29.358428 22824 tablet_bootstrap.cc:654] T 93dd95f0dfb8463291a74921c46665dd P e20f6c83ae144d698c26b94c2eda023f: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.370051 22811 raft_consensus.cc:493] T 47e33cdb6a154e8781b3e36a95eed353 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:29.370951 22811 raft_consensus.cc:515] T 47e33cdb6a154e8781b3e36a95eed353 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.375300 22824 tablet_bootstrap.cc:492] T 93dd95f0dfb8463291a74921c46665dd P e20f6c83ae144d698c26b94c2eda023f: No bootstrap required, opened a new log
I20260709 13:21:29.375475 22527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47e33cdb6a154e8781b3e36a95eed353" candidate_uuid: "e20f6c83ae144d698c26b94c2eda023f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b" is_pre_election: true
I20260709 13:21:29.376309 22824 ts_tablet_manager.cc:1403] T 93dd95f0dfb8463291a74921c46665dd P e20f6c83ae144d698c26b94c2eda023f: Time spent bootstrapping tablet: real 0.027s	user 0.012s	sys 0.010s
I20260709 13:21:29.376394 22527 raft_consensus.cc:2468] T 47e33cdb6a154e8781b3e36a95eed353 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e20f6c83ae144d698c26b94c2eda023f in term 0.
I20260709 13:21:29.379194 22674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47e33cdb6a154e8781b3e36a95eed353" candidate_uuid: "e20f6c83ae144d698c26b94c2eda023f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" is_pre_election: true
I20260709 13:21:29.380573 22811 leader_election.cc:290] T 47e33cdb6a154e8781b3e36a95eed353 P e20f6c83ae144d698c26b94c2eda023f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823), 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335)
I20260709 13:21:29.380873 22824 raft_consensus.cc:359] T 93dd95f0dfb8463291a74921c46665dd P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.381428 22568 leader_election.cc:304] T 47e33cdb6a154e8781b3e36a95eed353 P e20f6c83ae144d698c26b94c2eda023f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7df364d160d64ac4b5cdc4f004324f8b, e20f6c83ae144d698c26b94c2eda023f; no voters: 
W20260709 13:21:29.382740 22568 leader_election.cc:343] T 47e33cdb6a154e8781b3e36a95eed353 P e20f6c83ae144d698c26b94c2eda023f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:29.384035 22811 raft_consensus.cc:2804] T 47e33cdb6a154e8781b3e36a95eed353 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:29.384663 22811 raft_consensus.cc:493] T 47e33cdb6a154e8781b3e36a95eed353 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:29.385120 22811 raft_consensus.cc:3060] T 47e33cdb6a154e8781b3e36a95eed353 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.387073 22824 raft_consensus.cc:385] T 93dd95f0dfb8463291a74921c46665dd P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.387509 22824 raft_consensus.cc:740] T 93dd95f0dfb8463291a74921c46665dd P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e20f6c83ae144d698c26b94c2eda023f, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.388365 22824 consensus_queue.cc:260] T 93dd95f0dfb8463291a74921c46665dd P e20f6c83ae144d698c26b94c2eda023f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.390102 22825 tablet_bootstrap.cc:492] T a3b0925412bd4dceabec4f898eb43973 P 7df364d160d64ac4b5cdc4f004324f8b: No bootstrap required, opened a new log
I20260709 13:21:29.390915 22825 ts_tablet_manager.cc:1403] T a3b0925412bd4dceabec4f898eb43973 P 7df364d160d64ac4b5cdc4f004324f8b: Time spent bootstrapping tablet: real 0.073s	user 0.027s	sys 0.012s
I20260709 13:21:29.397739 22811 raft_consensus.cc:515] T 47e33cdb6a154e8781b3e36a95eed353 P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.401557 22527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47e33cdb6a154e8781b3e36a95eed353" candidate_uuid: "e20f6c83ae144d698c26b94c2eda023f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b"
I20260709 13:21:29.402388 22527 raft_consensus.cc:3060] T 47e33cdb6a154e8781b3e36a95eed353 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.403867 22825 raft_consensus.cc:359] T a3b0925412bd4dceabec4f898eb43973 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.404989 22825 raft_consensus.cc:385] T a3b0925412bd4dceabec4f898eb43973 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.405364 22825 raft_consensus.cc:740] T a3b0925412bd4dceabec4f898eb43973 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7df364d160d64ac4b5cdc4f004324f8b, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.408052 22825 consensus_queue.cc:260] T a3b0925412bd4dceabec4f898eb43973 P 7df364d160d64ac4b5cdc4f004324f8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.408555 22811 leader_election.cc:290] T 47e33cdb6a154e8781b3e36a95eed353 P e20f6c83ae144d698c26b94c2eda023f [CANDIDATE]: Term 1 election: Requested vote from peers 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823), 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335)
I20260709 13:21:29.409404 22824 ts_tablet_manager.cc:1434] T 93dd95f0dfb8463291a74921c46665dd P e20f6c83ae144d698c26b94c2eda023f: Time spent starting tablet: real 0.033s	user 0.012s	sys 0.016s
I20260709 13:21:29.410615 22824 tablet_bootstrap.cc:492] T a3b0925412bd4dceabec4f898eb43973 P e20f6c83ae144d698c26b94c2eda023f: Bootstrap starting.
I20260709 13:21:29.410419 22676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47e33cdb6a154e8781b3e36a95eed353" candidate_uuid: "e20f6c83ae144d698c26b94c2eda023f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac"
I20260709 13:21:29.411783 22823 tablet_bootstrap.cc:492] T 47e33cdb6a154e8781b3e36a95eed353 P 2d62e0b282b44003a202b8dcbe6ea6ac: No bootstrap required, opened a new log
W20260709 13:21:29.412235 22568 leader_election.cc:343] T 47e33cdb6a154e8781b3e36a95eed353 P e20f6c83ae144d698c26b94c2eda023f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:29.413203 22752 raft_consensus.cc:515] T 0f952d6650154a4a98f94a75b916d44b P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.416574 22602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f952d6650154a4a98f94a75b916d44b" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e20f6c83ae144d698c26b94c2eda023f"
I20260709 13:21:29.416874 22526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f952d6650154a4a98f94a75b916d44b" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b"
I20260709 13:21:29.417200 22602 raft_consensus.cc:3060] T 0f952d6650154a4a98f94a75b916d44b P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.417595 22526 raft_consensus.cc:3060] T 0f952d6650154a4a98f94a75b916d44b P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.412351 22823 ts_tablet_manager.cc:1403] T 47e33cdb6a154e8781b3e36a95eed353 P 2d62e0b282b44003a202b8dcbe6ea6ac: Time spent bootstrapping tablet: real 0.164s	user 0.059s	sys 0.098s
I20260709 13:21:29.417266 22527 raft_consensus.cc:2468] T 47e33cdb6a154e8781b3e36a95eed353 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e20f6c83ae144d698c26b94c2eda023f in term 1.
I20260709 13:21:29.422483 22752 leader_election.cc:290] T 0f952d6650154a4a98f94a75b916d44b P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 election: Requested vote from peers 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823), e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469)
I20260709 13:21:29.428968 22823 raft_consensus.cc:359] T 47e33cdb6a154e8781b3e36a95eed353 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.429780 22823 raft_consensus.cc:385] T 47e33cdb6a154e8781b3e36a95eed353 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.430088 22823 raft_consensus.cc:740] T 47e33cdb6a154e8781b3e36a95eed353 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d62e0b282b44003a202b8dcbe6ea6ac, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.430912 22823 consensus_queue.cc:260] T 47e33cdb6a154e8781b3e36a95eed353 P 2d62e0b282b44003a202b8dcbe6ea6ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.432366 22824 tablet_bootstrap.cc:654] T a3b0925412bd4dceabec4f898eb43973 P e20f6c83ae144d698c26b94c2eda023f: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.433362 22823 ts_tablet_manager.cc:1434] T 47e33cdb6a154e8781b3e36a95eed353 P 2d62e0b282b44003a202b8dcbe6ea6ac: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.000s
I20260709 13:21:29.433961 22602 raft_consensus.cc:2468] T 0f952d6650154a4a98f94a75b916d44b P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d62e0b282b44003a202b8dcbe6ea6ac in term 1.
I20260709 13:21:29.434747 22823 tablet_bootstrap.cc:492] T 93dd95f0dfb8463291a74921c46665dd P 2d62e0b282b44003a202b8dcbe6ea6ac: Bootstrap starting.
I20260709 13:21:29.435549 22568 leader_election.cc:304] T 47e33cdb6a154e8781b3e36a95eed353 P e20f6c83ae144d698c26b94c2eda023f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7df364d160d64ac4b5cdc4f004324f8b, e20f6c83ae144d698c26b94c2eda023f; no voters: 2d62e0b282b44003a202b8dcbe6ea6ac
I20260709 13:21:29.437183 22811 raft_consensus.cc:2804] T 47e33cdb6a154e8781b3e36a95eed353 P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:29.437858 22811 raft_consensus.cc:697] T 47e33cdb6a154e8781b3e36a95eed353 P e20f6c83ae144d698c26b94c2eda023f [term 1 LEADER]: Becoming Leader. State: Replica: e20f6c83ae144d698c26b94c2eda023f, State: Running, Role: LEADER
I20260709 13:21:29.439002 22811 consensus_queue.cc:237] T 47e33cdb6a154e8781b3e36a95eed353 P e20f6c83ae144d698c26b94c2eda023f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.443185 22823 tablet_bootstrap.cc:654] T 93dd95f0dfb8463291a74921c46665dd P 2d62e0b282b44003a202b8dcbe6ea6ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.443591 22640 leader_election.cc:304] T 0f952d6650154a4a98f94a75b916d44b P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d62e0b282b44003a202b8dcbe6ea6ac, e20f6c83ae144d698c26b94c2eda023f; no voters: 
I20260709 13:21:29.444583 22836 raft_consensus.cc:493] T 2f75c1bbbd7b4dc09412c016798ccd50 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:29.445127 22836 raft_consensus.cc:515] T 2f75c1bbbd7b4dc09412c016798ccd50 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.447746 22836 leader_election.cc:290] T 2f75c1bbbd7b4dc09412c016798ccd50 P 7df364d160d64ac4b5cdc4f004324f8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469), 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335)
I20260709 13:21:29.448280 22752 raft_consensus.cc:2804] T 0f952d6650154a4a98f94a75b916d44b P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:29.451069 22825 ts_tablet_manager.cc:1434] T a3b0925412bd4dceabec4f898eb43973 P 7df364d160d64ac4b5cdc4f004324f8b: Time spent starting tablet: real 0.060s	user 0.008s	sys 0.001s
I20260709 13:21:29.452412 22676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f75c1bbbd7b4dc09412c016798ccd50" candidate_uuid: "7df364d160d64ac4b5cdc4f004324f8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" is_pre_election: true
I20260709 13:21:29.452687 22752 raft_consensus.cc:697] T 0f952d6650154a4a98f94a75b916d44b P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 LEADER]: Becoming Leader. State: Replica: 2d62e0b282b44003a202b8dcbe6ea6ac, State: Running, Role: LEADER
I20260709 13:21:29.452495 22825 tablet_bootstrap.cc:492] T 93dd95f0dfb8463291a74921c46665dd P 7df364d160d64ac4b5cdc4f004324f8b: Bootstrap starting.
I20260709 13:21:29.453337 22676 raft_consensus.cc:2468] T 2f75c1bbbd7b4dc09412c016798ccd50 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7df364d160d64ac4b5cdc4f004324f8b in term 0.
I20260709 13:21:29.455303 22843 raft_consensus.cc:493] T 5b97872c6f3b4a288ff8ade4569da6f1 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:29.455852 22843 raft_consensus.cc:515] T 5b97872c6f3b4a288ff8ade4569da6f1 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.453800 22752 consensus_queue.cc:237] T 0f952d6650154a4a98f94a75b916d44b P 2d62e0b282b44003a202b8dcbe6ea6ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.457988 22843 leader_election.cc:290] T 5b97872c6f3b4a288ff8ade4569da6f1 P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823), e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469)
I20260709 13:21:29.451114 22599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f75c1bbbd7b4dc09412c016798ccd50" candidate_uuid: "7df364d160d64ac4b5cdc4f004324f8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e20f6c83ae144d698c26b94c2eda023f" is_pre_election: true
I20260709 13:21:29.459594 22599 raft_consensus.cc:2468] T 2f75c1bbbd7b4dc09412c016798ccd50 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7df364d160d64ac4b5cdc4f004324f8b in term 0.
I20260709 13:21:29.459659 22527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b97872c6f3b4a288ff8ade4569da6f1" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b" is_pre_election: true
I20260709 13:21:29.460379 22527 raft_consensus.cc:2468] T 5b97872c6f3b4a288ff8ade4569da6f1 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d62e0b282b44003a202b8dcbe6ea6ac in term 0.
I20260709 13:21:29.460696 22491 leader_election.cc:304] T 2f75c1bbbd7b4dc09412c016798ccd50 P 7df364d160d64ac4b5cdc4f004324f8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7df364d160d64ac4b5cdc4f004324f8b, e20f6c83ae144d698c26b94c2eda023f; no voters: 
I20260709 13:21:29.462042 22642 leader_election.cc:304] T 5b97872c6f3b4a288ff8ade4569da6f1 P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d62e0b282b44003a202b8dcbe6ea6ac, 7df364d160d64ac4b5cdc4f004324f8b; no voters: 
I20260709 13:21:29.462361 22599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b97872c6f3b4a288ff8ade4569da6f1" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e20f6c83ae144d698c26b94c2eda023f" is_pre_election: true
I20260709 13:21:29.463017 22836 raft_consensus.cc:2804] T 2f75c1bbbd7b4dc09412c016798ccd50 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:29.463802 22836 raft_consensus.cc:493] T 2f75c1bbbd7b4dc09412c016798ccd50 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:21:29.464308 22640 leader_election.cc:343] T 5b97872c6f3b4a288ff8ade4569da6f1 P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:29.464154 22836 raft_consensus.cc:3060] T 2f75c1bbbd7b4dc09412c016798ccd50 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.464511 22843 raft_consensus.cc:2804] T 5b97872c6f3b4a288ff8ade4569da6f1 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:29.466079 22843 raft_consensus.cc:493] T 5b97872c6f3b4a288ff8ade4569da6f1 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:29.468349 22824 tablet_bootstrap.cc:492] T a3b0925412bd4dceabec4f898eb43973 P e20f6c83ae144d698c26b94c2eda023f: No bootstrap required, opened a new log
I20260709 13:21:29.469014 22824 ts_tablet_manager.cc:1403] T a3b0925412bd4dceabec4f898eb43973 P e20f6c83ae144d698c26b94c2eda023f: Time spent bootstrapping tablet: real 0.059s	user 0.011s	sys 0.010s
I20260709 13:21:29.470180 22825 tablet_bootstrap.cc:654] T 93dd95f0dfb8463291a74921c46665dd P 7df364d160d64ac4b5cdc4f004324f8b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.466450 22843 raft_consensus.cc:3060] T 5b97872c6f3b4a288ff8ade4569da6f1 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.471655 22824 raft_consensus.cc:359] T a3b0925412bd4dceabec4f898eb43973 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.472442 22824 raft_consensus.cc:385] T a3b0925412bd4dceabec4f898eb43973 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.472724 22824 raft_consensus.cc:740] T a3b0925412bd4dceabec4f898eb43973 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e20f6c83ae144d698c26b94c2eda023f, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.474290 22824 consensus_queue.cc:260] T a3b0925412bd4dceabec4f898eb43973 P e20f6c83ae144d698c26b94c2eda023f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.475618 22823 tablet_bootstrap.cc:492] T 93dd95f0dfb8463291a74921c46665dd P 2d62e0b282b44003a202b8dcbe6ea6ac: No bootstrap required, opened a new log
I20260709 13:21:29.476192 22823 ts_tablet_manager.cc:1403] T 93dd95f0dfb8463291a74921c46665dd P 2d62e0b282b44003a202b8dcbe6ea6ac: Time spent bootstrapping tablet: real 0.042s	user 0.009s	sys 0.011s
I20260709 13:21:29.476722 22824 ts_tablet_manager.cc:1434] T a3b0925412bd4dceabec4f898eb43973 P e20f6c83ae144d698c26b94c2eda023f: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:21:29.477836 22824 tablet_bootstrap.cc:492] T 5b97872c6f3b4a288ff8ade4569da6f1 P e20f6c83ae144d698c26b94c2eda023f: Bootstrap starting.
I20260709 13:21:29.479367 22823 raft_consensus.cc:359] T 93dd95f0dfb8463291a74921c46665dd P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.480155 22823 raft_consensus.cc:385] T 93dd95f0dfb8463291a74921c46665dd P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.480438 22823 raft_consensus.cc:740] T 93dd95f0dfb8463291a74921c46665dd P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d62e0b282b44003a202b8dcbe6ea6ac, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.481549 22724 raft_consensus.cc:493] T e1e94cad244443feb24f679271460852 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:29.481201 22823 consensus_queue.cc:260] T 93dd95f0dfb8463291a74921c46665dd P 2d62e0b282b44003a202b8dcbe6ea6ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.482131 22724 raft_consensus.cc:515] T e1e94cad244443feb24f679271460852 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.466724 22526 raft_consensus.cc:2468] T 0f952d6650154a4a98f94a75b916d44b P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d62e0b282b44003a202b8dcbe6ea6ac in term 1.
I20260709 13:21:29.484174 22823 ts_tablet_manager.cc:1434] T 93dd95f0dfb8463291a74921c46665dd P 2d62e0b282b44003a202b8dcbe6ea6ac: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:21:29.482725 22836 raft_consensus.cc:515] T 2f75c1bbbd7b4dc09412c016798ccd50 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.488543 22823 tablet_bootstrap.cc:492] T a3b0925412bd4dceabec4f898eb43973 P 2d62e0b282b44003a202b8dcbe6ea6ac: Bootstrap starting.
I20260709 13:21:29.489338 22836 leader_election.cc:290] T 2f75c1bbbd7b4dc09412c016798ccd50 P 7df364d160d64ac4b5cdc4f004324f8b [CANDIDATE]: Term 1 election: Requested vote from peers e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469), 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335)
I20260709 13:21:29.485066 22843 raft_consensus.cc:515] T 5b97872c6f3b4a288ff8ade4569da6f1 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.490438 22676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f75c1bbbd7b4dc09412c016798ccd50" candidate_uuid: "7df364d160d64ac4b5cdc4f004324f8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac"
I20260709 13:21:29.491109 22676 raft_consensus.cc:3060] T 2f75c1bbbd7b4dc09412c016798ccd50 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.490438 22602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f75c1bbbd7b4dc09412c016798ccd50" candidate_uuid: "7df364d160d64ac4b5cdc4f004324f8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e20f6c83ae144d698c26b94c2eda023f"
I20260709 13:21:29.492331 22602 raft_consensus.cc:3060] T 2f75c1bbbd7b4dc09412c016798ccd50 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.492622 22601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b97872c6f3b4a288ff8ade4569da6f1" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e20f6c83ae144d698c26b94c2eda023f"
W20260709 13:21:29.494405 22640 leader_election.cc:343] T 5b97872c6f3b4a288ff8ade4569da6f1 P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:29.496286 22599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e94cad244443feb24f679271460852" candidate_uuid: "7df364d160d64ac4b5cdc4f004324f8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e20f6c83ae144d698c26b94c2eda023f" is_pre_election: true
I20260709 13:21:29.496188 22843 leader_election.cc:290] T 5b97872c6f3b4a288ff8ade4569da6f1 P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 election: Requested vote from peers 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823), e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469)
I20260709 13:21:29.497088 22599 raft_consensus.cc:2468] T e1e94cad244443feb24f679271460852 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7df364d160d64ac4b5cdc4f004324f8b in term 0.
I20260709 13:21:29.498634 22491 leader_election.cc:304] T e1e94cad244443feb24f679271460852 P 7df364d160d64ac4b5cdc4f004324f8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7df364d160d64ac4b5cdc4f004324f8b, e20f6c83ae144d698c26b94c2eda023f; no voters: 
I20260709 13:21:29.499140 22676 raft_consensus.cc:2468] T 2f75c1bbbd7b4dc09412c016798ccd50 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7df364d160d64ac4b5cdc4f004324f8b in term 1.
I20260709 13:21:29.500018 22836 raft_consensus.cc:493] T ef14a8f3b6344312bd294edcabb2f312 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:29.500974 22493 leader_election.cc:304] T 2f75c1bbbd7b4dc09412c016798ccd50 P 7df364d160d64ac4b5cdc4f004324f8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d62e0b282b44003a202b8dcbe6ea6ac, 7df364d160d64ac4b5cdc4f004324f8b; no voters: 
I20260709 13:21:29.498908 22526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b97872c6f3b4a288ff8ade4569da6f1" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b"
I20260709 13:21:29.501933 22526 raft_consensus.cc:3060] T 5b97872c6f3b4a288ff8ade4569da6f1 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.496201 22823 tablet_bootstrap.cc:654] T a3b0925412bd4dceabec4f898eb43973 P 2d62e0b282b44003a202b8dcbe6ea6ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.502717 22825 tablet_bootstrap.cc:492] T 93dd95f0dfb8463291a74921c46665dd P 7df364d160d64ac4b5cdc4f004324f8b: No bootstrap required, opened a new log
I20260709 13:21:29.503404 22825 ts_tablet_manager.cc:1403] T 93dd95f0dfb8463291a74921c46665dd P 7df364d160d64ac4b5cdc4f004324f8b: Time spent bootstrapping tablet: real 0.051s	user 0.012s	sys 0.006s
I20260709 13:21:29.503937 22724 leader_election.cc:290] T e1e94cad244443feb24f679271460852 P 7df364d160d64ac4b5cdc4f004324f8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469), 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335)
I20260709 13:21:29.504559 22724 raft_consensus.cc:2804] T e1e94cad244443feb24f679271460852 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:29.504943 22724 raft_consensus.cc:493] T e1e94cad244443feb24f679271460852 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:29.500823 22836 raft_consensus.cc:515] T ef14a8f3b6344312bd294edcabb2f312 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.505319 22724 raft_consensus.cc:3060] T e1e94cad244443feb24f679271460852 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.507182 22676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e94cad244443feb24f679271460852" candidate_uuid: "7df364d160d64ac4b5cdc4f004324f8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" is_pre_election: true
I20260709 13:21:29.513718 22676 raft_consensus.cc:2468] T e1e94cad244443feb24f679271460852 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7df364d160d64ac4b5cdc4f004324f8b in term 0.
I20260709 13:21:29.509236 22824 tablet_bootstrap.cc:654] T 5b97872c6f3b4a288ff8ade4569da6f1 P e20f6c83ae144d698c26b94c2eda023f: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.515668 22836 leader_election.cc:290] T ef14a8f3b6344312bd294edcabb2f312 P 7df364d160d64ac4b5cdc4f004324f8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469), 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335)
I20260709 13:21:29.516462 22836 raft_consensus.cc:2804] T 2f75c1bbbd7b4dc09412c016798ccd50 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:29.517004 22836 raft_consensus.cc:697] T 2f75c1bbbd7b4dc09412c016798ccd50 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 LEADER]: Becoming Leader. State: Replica: 7df364d160d64ac4b5cdc4f004324f8b, State: Running, Role: LEADER
I20260709 13:21:29.506446 22825 raft_consensus.cc:359] T 93dd95f0dfb8463291a74921c46665dd P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.518214 22825 raft_consensus.cc:385] T 93dd95f0dfb8463291a74921c46665dd P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.518818 22825 raft_consensus.cc:740] T 93dd95f0dfb8463291a74921c46665dd P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7df364d160d64ac4b5cdc4f004324f8b, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.519074 22843 raft_consensus.cc:493] T ef14a8f3b6344312bd294edcabb2f312 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:29.519676 22843 raft_consensus.cc:515] T ef14a8f3b6344312bd294edcabb2f312 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.519826 22825 consensus_queue.cc:260] T 93dd95f0dfb8463291a74921c46665dd P 7df364d160d64ac4b5cdc4f004324f8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.522697 22825 ts_tablet_manager.cc:1434] T 93dd95f0dfb8463291a74921c46665dd P 7df364d160d64ac4b5cdc4f004324f8b: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.002s
I20260709 13:21:29.523726 22825 tablet_bootstrap.cc:492] T 2d4c5545efd2494a81249b0dbbfbc56c P 7df364d160d64ac4b5cdc4f004324f8b: Bootstrap starting.
I20260709 13:21:29.512614 22823 tablet_bootstrap.cc:492] T a3b0925412bd4dceabec4f898eb43973 P 2d62e0b282b44003a202b8dcbe6ea6ac: No bootstrap required, opened a new log
I20260709 13:21:29.524726 22823 ts_tablet_manager.cc:1403] T a3b0925412bd4dceabec4f898eb43973 P 2d62e0b282b44003a202b8dcbe6ea6ac: Time spent bootstrapping tablet: real 0.036s	user 0.013s	sys 0.001s
I20260709 13:21:29.529320 22422 catalog_manager.cc:5697] T 47e33cdb6a154e8781b3e36a95eed353 P e20f6c83ae144d698c26b94c2eda023f reported cstate change: term changed from 0 to 1, leader changed from <none> to e20f6c83ae144d698c26b94c2eda023f (127.9.116.130). New cstate: current_term: 1 leader_uuid: "e20f6c83ae144d698c26b94c2eda023f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:29.512414 22526 raft_consensus.cc:2468] T 5b97872c6f3b4a288ff8ade4569da6f1 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d62e0b282b44003a202b8dcbe6ea6ac in term 1.
I20260709 13:21:29.530671 22825 tablet_bootstrap.cc:654] T 2d4c5545efd2494a81249b0dbbfbc56c P 7df364d160d64ac4b5cdc4f004324f8b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.533807 22823 raft_consensus.cc:359] T a3b0925412bd4dceabec4f898eb43973 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.540114 22823 raft_consensus.cc:385] T a3b0925412bd4dceabec4f898eb43973 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.540496 22823 raft_consensus.cc:740] T a3b0925412bd4dceabec4f898eb43973 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d62e0b282b44003a202b8dcbe6ea6ac, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.537179 22824 tablet_bootstrap.cc:492] T 5b97872c6f3b4a288ff8ade4569da6f1 P e20f6c83ae144d698c26b94c2eda023f: No bootstrap required, opened a new log
I20260709 13:21:29.541339 22823 consensus_queue.cc:260] T a3b0925412bd4dceabec4f898eb43973 P 2d62e0b282b44003a202b8dcbe6ea6ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.518206 22836 consensus_queue.cc:237] T 2f75c1bbbd7b4dc09412c016798ccd50 P 7df364d160d64ac4b5cdc4f004324f8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.541464 22423 catalog_manager.cc:5697] T 0f952d6650154a4a98f94a75b916d44b P 2d62e0b282b44003a202b8dcbe6ea6ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131). New cstate: current_term: 1 leader_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:29.544013 22823 ts_tablet_manager.cc:1434] T a3b0925412bd4dceabec4f898eb43973 P 2d62e0b282b44003a202b8dcbe6ea6ac: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.003s
I20260709 13:21:29.545089 22823 tablet_bootstrap.cc:492] T 2d4c5545efd2494a81249b0dbbfbc56c P 2d62e0b282b44003a202b8dcbe6ea6ac: Bootstrap starting.
I20260709 13:21:29.545183 22811 raft_consensus.cc:493] T 93dd95f0dfb8463291a74921c46665dd P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:29.535380 22724 raft_consensus.cc:515] T e1e94cad244443feb24f679271460852 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.541532 22824 ts_tablet_manager.cc:1403] T 5b97872c6f3b4a288ff8ade4569da6f1 P e20f6c83ae144d698c26b94c2eda023f: Time spent bootstrapping tablet: real 0.064s	user 0.013s	sys 0.005s
I20260709 13:21:29.547178 22811 raft_consensus.cc:515] T 93dd95f0dfb8463291a74921c46665dd P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.549430 22599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef14a8f3b6344312bd294edcabb2f312" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e20f6c83ae144d698c26b94c2eda023f" is_pre_election: true
I20260709 13:21:29.550186 22599 raft_consensus.cc:2468] T ef14a8f3b6344312bd294edcabb2f312 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d62e0b282b44003a202b8dcbe6ea6ac in term 0.
I20260709 13:21:29.551574 22676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93dd95f0dfb8463291a74921c46665dd" candidate_uuid: "e20f6c83ae144d698c26b94c2eda023f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" is_pre_election: true
I20260709 13:21:29.551743 22640 leader_election.cc:304] T ef14a8f3b6344312bd294edcabb2f312 P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d62e0b282b44003a202b8dcbe6ea6ac, e20f6c83ae144d698c26b94c2eda023f; no voters: 
I20260709 13:21:29.552240 22724 leader_election.cc:290] T e1e94cad244443feb24f679271460852 P 7df364d160d64ac4b5cdc4f004324f8b [CANDIDATE]: Term 1 election: Requested vote from peers e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469), 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335)
I20260709 13:21:29.552325 22676 raft_consensus.cc:2468] T 93dd95f0dfb8463291a74921c46665dd P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e20f6c83ae144d698c26b94c2eda023f in term 0.
I20260709 13:21:29.552613 22752 raft_consensus.cc:2804] T ef14a8f3b6344312bd294edcabb2f312 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:29.553439 22752 raft_consensus.cc:493] T ef14a8f3b6344312bd294edcabb2f312 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:29.553838 22752 raft_consensus.cc:3060] T ef14a8f3b6344312bd294edcabb2f312 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.526610 22602 raft_consensus.cc:2468] T 2f75c1bbbd7b4dc09412c016798ccd50 P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7df364d160d64ac4b5cdc4f004324f8b in term 1.
I20260709 13:21:29.555477 22811 leader_election.cc:290] T 93dd95f0dfb8463291a74921c46665dd P e20f6c83ae144d698c26b94c2eda023f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823), 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335)
I20260709 13:21:29.556788 22823 tablet_bootstrap.cc:654] T 2d4c5545efd2494a81249b0dbbfbc56c P 2d62e0b282b44003a202b8dcbe6ea6ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.550016 22824 raft_consensus.cc:359] T 5b97872c6f3b4a288ff8ade4569da6f1 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.557081 22568 leader_election.cc:304] T 93dd95f0dfb8463291a74921c46665dd P e20f6c83ae144d698c26b94c2eda023f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d62e0b282b44003a202b8dcbe6ea6ac, e20f6c83ae144d698c26b94c2eda023f; no voters: 
I20260709 13:21:29.558593 22843 leader_election.cc:290] T ef14a8f3b6344312bd294edcabb2f312 P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823), e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469)
I20260709 13:21:29.553848 22825 tablet_bootstrap.cc:492] T 2d4c5545efd2494a81249b0dbbfbc56c P 7df364d160d64ac4b5cdc4f004324f8b: No bootstrap required, opened a new log
I20260709 13:21:29.557416 22824 raft_consensus.cc:385] T 5b97872c6f3b4a288ff8ade4569da6f1 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.560698 22824 raft_consensus.cc:740] T 5b97872c6f3b4a288ff8ade4569da6f1 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e20f6c83ae144d698c26b94c2eda023f, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.560910 22825 ts_tablet_manager.cc:1403] T 2d4c5545efd2494a81249b0dbbfbc56c P 7df364d160d64ac4b5cdc4f004324f8b: Time spent bootstrapping tablet: real 0.037s	user 0.013s	sys 0.004s
I20260709 13:21:29.561928 22811 raft_consensus.cc:2804] T 93dd95f0dfb8463291a74921c46665dd P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:29.562484 22811 raft_consensus.cc:493] T 93dd95f0dfb8463291a74921c46665dd P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:29.562981 22811 raft_consensus.cc:3060] T 93dd95f0dfb8463291a74921c46665dd P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.566574 22825 raft_consensus.cc:359] T 2d4c5545efd2494a81249b0dbbfbc56c P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.569433 22825 raft_consensus.cc:385] T 2d4c5545efd2494a81249b0dbbfbc56c P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.570871 22825 raft_consensus.cc:740] T 2d4c5545efd2494a81249b0dbbfbc56c P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7df364d160d64ac4b5cdc4f004324f8b, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.572041 22825 consensus_queue.cc:260] T 2d4c5545efd2494a81249b0dbbfbc56c P 7df364d160d64ac4b5cdc4f004324f8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
W20260709 13:21:29.574677 22493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.131:34335, user_credentials={real_user=slave}} blocked reactor thread for 73784us
I20260709 13:21:29.561805 22824 consensus_queue.cc:260] T 5b97872c6f3b4a288ff8ade4569da6f1 P e20f6c83ae144d698c26b94c2eda023f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.579262 22676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef14a8f3b6344312bd294edcabb2f312" candidate_uuid: "7df364d160d64ac4b5cdc4f004324f8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" is_pre_election: true
I20260709 13:21:29.580281 22674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e94cad244443feb24f679271460852" candidate_uuid: "7df364d160d64ac4b5cdc4f004324f8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac"
I20260709 13:21:29.580953 22674 raft_consensus.cc:3060] T e1e94cad244443feb24f679271460852 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.581040 22824 ts_tablet_manager.cc:1434] T 5b97872c6f3b4a288ff8ade4569da6f1 P e20f6c83ae144d698c26b94c2eda023f: Time spent starting tablet: real 0.034s	user 0.005s	sys 0.001s
I20260709 13:21:29.577183 22825 ts_tablet_manager.cc:1434] T 2d4c5545efd2494a81249b0dbbfbc56c P 7df364d160d64ac4b5cdc4f004324f8b: Time spent starting tablet: real 0.015s	user 0.002s	sys 0.004s
I20260709 13:21:29.586413 22823 tablet_bootstrap.cc:492] T 2d4c5545efd2494a81249b0dbbfbc56c P 2d62e0b282b44003a202b8dcbe6ea6ac: No bootstrap required, opened a new log
I20260709 13:21:29.582626 22824 tablet_bootstrap.cc:492] T 2d4c5545efd2494a81249b0dbbfbc56c P e20f6c83ae144d698c26b94c2eda023f: Bootstrap starting.
I20260709 13:21:29.587137 22823 ts_tablet_manager.cc:1403] T 2d4c5545efd2494a81249b0dbbfbc56c P 2d62e0b282b44003a202b8dcbe6ea6ac: Time spent bootstrapping tablet: real 0.042s	user 0.027s	sys 0.010s
I20260709 13:21:29.573776 22752 raft_consensus.cc:515] T ef14a8f3b6344312bd294edcabb2f312 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.589583 22752 leader_election.cc:290] T ef14a8f3b6344312bd294edcabb2f312 P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 election: Requested vote from peers 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823), e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469)
I20260709 13:21:29.590076 22823 raft_consensus.cc:359] T 2d4c5545efd2494a81249b0dbbfbc56c P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.590683 22676 raft_consensus.cc:2393] T ef14a8f3b6344312bd294edcabb2f312 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7df364d160d64ac4b5cdc4f004324f8b in current term 1: Already voted for candidate 2d62e0b282b44003a202b8dcbe6ea6ac in this term.
I20260709 13:21:29.591027 22823 raft_consensus.cc:385] T 2d4c5545efd2494a81249b0dbbfbc56c P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.591547 22823 raft_consensus.cc:740] T 2d4c5545efd2494a81249b0dbbfbc56c P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d62e0b282b44003a202b8dcbe6ea6ac, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.592398 22823 consensus_queue.cc:260] T 2d4c5545efd2494a81249b0dbbfbc56c P 2d62e0b282b44003a202b8dcbe6ea6ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.595662 22823 ts_tablet_manager.cc:1434] T 2d4c5545efd2494a81249b0dbbfbc56c P 2d62e0b282b44003a202b8dcbe6ea6ac: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.002s
I20260709 13:21:29.596944 22602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef14a8f3b6344312bd294edcabb2f312" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e20f6c83ae144d698c26b94c2eda023f"
I20260709 13:21:29.597712 22602 raft_consensus.cc:3060] T ef14a8f3b6344312bd294edcabb2f312 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.601344 22847 raft_consensus.cc:493] T 5b97872c6f3b4a288ff8ade4569da6f1 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:29.601950 22847 raft_consensus.cc:515] T 5b97872c6f3b4a288ff8ade4569da6f1 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.602747 22811 raft_consensus.cc:515] T 93dd95f0dfb8463291a74921c46665dd P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.604540 22847 leader_election.cc:290] T 5b97872c6f3b4a288ff8ade4569da6f1 P e20f6c83ae144d698c26b94c2eda023f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823), 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335)
I20260709 13:21:29.608055 22824 tablet_bootstrap.cc:654] T 2d4c5545efd2494a81249b0dbbfbc56c P e20f6c83ae144d698c26b94c2eda023f: Neither blocks nor log segments found. Creating new log.
W20260709 13:21:29.632570 22491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.116.130:43469, user_credentials={real_user=slave}} blocked reactor thread for 134010us
I20260709 13:21:29.640666 22676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b97872c6f3b4a288ff8ade4569da6f1" candidate_uuid: "e20f6c83ae144d698c26b94c2eda023f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" is_pre_election: true
I20260709 13:21:29.641294 22642 leader_election.cc:304] T 5b97872c6f3b4a288ff8ade4569da6f1 P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2d62e0b282b44003a202b8dcbe6ea6ac, 7df364d160d64ac4b5cdc4f004324f8b; no voters: e20f6c83ae144d698c26b94c2eda023f
I20260709 13:21:29.642228 22599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef14a8f3b6344312bd294edcabb2f312" candidate_uuid: "7df364d160d64ac4b5cdc4f004324f8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e20f6c83ae144d698c26b94c2eda023f" is_pre_election: true
I20260709 13:21:29.644289 22526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93dd95f0dfb8463291a74921c46665dd" candidate_uuid: "e20f6c83ae144d698c26b94c2eda023f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b" is_pre_election: true
I20260709 13:21:29.644579 22527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b97872c6f3b4a288ff8ade4569da6f1" candidate_uuid: "e20f6c83ae144d698c26b94c2eda023f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b" is_pre_election: true
I20260709 13:21:29.644994 22526 raft_consensus.cc:2468] T 93dd95f0dfb8463291a74921c46665dd P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e20f6c83ae144d698c26b94c2eda023f in term 0.
I20260709 13:21:29.645514 22527 raft_consensus.cc:2393] T 5b97872c6f3b4a288ff8ade4569da6f1 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e20f6c83ae144d698c26b94c2eda023f in current term 1: Already voted for candidate 2d62e0b282b44003a202b8dcbe6ea6ac in this term.
I20260709 13:21:29.647073 22527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93dd95f0dfb8463291a74921c46665dd" candidate_uuid: "e20f6c83ae144d698c26b94c2eda023f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b"
I20260709 13:21:29.641657 22676 raft_consensus.cc:2393] T 5b97872c6f3b4a288ff8ade4569da6f1 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e20f6c83ae144d698c26b94c2eda023f in current term 1: Already voted for candidate 2d62e0b282b44003a202b8dcbe6ea6ac in this term.
I20260709 13:21:29.646442 22674 raft_consensus.cc:2468] T e1e94cad244443feb24f679271460852 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7df364d160d64ac4b5cdc4f004324f8b in term 1.
I20260709 13:21:29.649811 22527 raft_consensus.cc:3060] T 93dd95f0dfb8463291a74921c46665dd P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.656252 22811 leader_election.cc:290] T 93dd95f0dfb8463291a74921c46665dd P e20f6c83ae144d698c26b94c2eda023f [CANDIDATE]: Term 1 election: Requested vote from peers 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823), 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335)
I20260709 13:21:29.659379 22493 leader_election.cc:304] T e1e94cad244443feb24f679271460852 P 7df364d160d64ac4b5cdc4f004324f8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d62e0b282b44003a202b8dcbe6ea6ac, 7df364d160d64ac4b5cdc4f004324f8b; no voters: 
I20260709 13:21:29.655021 22752 raft_consensus.cc:2804] T 5b97872c6f3b4a288ff8ade4569da6f1 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:29.660332 22855 raft_consensus.cc:2804] T e1e94cad244443feb24f679271460852 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:29.661383 22752 raft_consensus.cc:697] T 5b97872c6f3b4a288ff8ade4569da6f1 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 LEADER]: Becoming Leader. State: Replica: 2d62e0b282b44003a202b8dcbe6ea6ac, State: Running, Role: LEADER
I20260709 13:21:29.661763 22855 raft_consensus.cc:697] T e1e94cad244443feb24f679271460852 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 LEADER]: Becoming Leader. State: Replica: 7df364d160d64ac4b5cdc4f004324f8b, State: Running, Role: LEADER
I20260709 13:21:29.662698 22855 consensus_queue.cc:237] T e1e94cad244443feb24f679271460852 P 7df364d160d64ac4b5cdc4f004324f8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.664171 22752 consensus_queue.cc:237] T 5b97872c6f3b4a288ff8ade4569da6f1 P 2d62e0b282b44003a202b8dcbe6ea6ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.669656 22526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef14a8f3b6344312bd294edcabb2f312" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b" is_pre_election: true
I20260709 13:21:29.670501 22526 raft_consensus.cc:2468] T ef14a8f3b6344312bd294edcabb2f312 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d62e0b282b44003a202b8dcbe6ea6ac in term 0.
I20260709 13:21:29.671692 22568 leader_election.cc:304] T 5b97872c6f3b4a288ff8ade4569da6f1 P e20f6c83ae144d698c26b94c2eda023f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e20f6c83ae144d698c26b94c2eda023f; no voters: 2d62e0b282b44003a202b8dcbe6ea6ac, 7df364d160d64ac4b5cdc4f004324f8b
I20260709 13:21:29.672567 22526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef14a8f3b6344312bd294edcabb2f312" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b"
I20260709 13:21:29.673099 22811 raft_consensus.cc:3060] T 5b97872c6f3b4a288ff8ade4569da6f1 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.673964 22491 leader_election.cc:304] T ef14a8f3b6344312bd294edcabb2f312 P 7df364d160d64ac4b5cdc4f004324f8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7df364d160d64ac4b5cdc4f004324f8b; no voters: 2d62e0b282b44003a202b8dcbe6ea6ac, e20f6c83ae144d698c26b94c2eda023f
I20260709 13:21:29.673403 22526 raft_consensus.cc:3060] T ef14a8f3b6344312bd294edcabb2f312 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.679677 22674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93dd95f0dfb8463291a74921c46665dd" candidate_uuid: "e20f6c83ae144d698c26b94c2eda023f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac"
I20260709 13:21:29.684556 22599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e94cad244443feb24f679271460852" candidate_uuid: "7df364d160d64ac4b5cdc4f004324f8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e20f6c83ae144d698c26b94c2eda023f"
I20260709 13:21:29.686985 22599 raft_consensus.cc:3060] T e1e94cad244443feb24f679271460852 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.688010 22824 tablet_bootstrap.cc:492] T 2d4c5545efd2494a81249b0dbbfbc56c P e20f6c83ae144d698c26b94c2eda023f: No bootstrap required, opened a new log
I20260709 13:21:29.688576 22824 ts_tablet_manager.cc:1403] T 2d4c5545efd2494a81249b0dbbfbc56c P e20f6c83ae144d698c26b94c2eda023f: Time spent bootstrapping tablet: real 0.106s	user 0.011s	sys 0.009s
I20260709 13:21:29.691506 22674 raft_consensus.cc:3060] T 93dd95f0dfb8463291a74921c46665dd P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.694051 22824 raft_consensus.cc:359] T 2d4c5545efd2494a81249b0dbbfbc56c P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.694859 22824 raft_consensus.cc:385] T 2d4c5545efd2494a81249b0dbbfbc56c P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.695116 22824 raft_consensus.cc:740] T 2d4c5545efd2494a81249b0dbbfbc56c P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e20f6c83ae144d698c26b94c2eda023f, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.695927 22824 consensus_queue.cc:260] T 2d4c5545efd2494a81249b0dbbfbc56c P e20f6c83ae144d698c26b94c2eda023f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.699088 22824 ts_tablet_manager.cc:1434] T 2d4c5545efd2494a81249b0dbbfbc56c P e20f6c83ae144d698c26b94c2eda023f: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:21:29.702384 22811 raft_consensus.cc:2749] T 5b97872c6f3b4a288ff8ade4569da6f1 P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:29.704115 22602 raft_consensus.cc:2468] T ef14a8f3b6344312bd294edcabb2f312 P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d62e0b282b44003a202b8dcbe6ea6ac in term 1.
I20260709 13:21:29.705775 22640 leader_election.cc:304] T ef14a8f3b6344312bd294edcabb2f312 P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d62e0b282b44003a202b8dcbe6ea6ac, e20f6c83ae144d698c26b94c2eda023f; no voters: 
I20260709 13:21:29.706713 22843 raft_consensus.cc:2804] T ef14a8f3b6344312bd294edcabb2f312 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:29.708894 22674 raft_consensus.cc:2468] T 93dd95f0dfb8463291a74921c46665dd P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e20f6c83ae144d698c26b94c2eda023f in term 1.
I20260709 13:21:29.707237 22843 raft_consensus.cc:697] T ef14a8f3b6344312bd294edcabb2f312 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 LEADER]: Becoming Leader. State: Replica: 2d62e0b282b44003a202b8dcbe6ea6ac, State: Running, Role: LEADER
I20260709 13:21:29.711119 22843 consensus_queue.cc:237] T ef14a8f3b6344312bd294edcabb2f312 P 2d62e0b282b44003a202b8dcbe6ea6ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.724023 22848 raft_consensus.cc:493] T a3b0925412bd4dceabec4f898eb43973 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:29.724845 22848 raft_consensus.cc:515] T a3b0925412bd4dceabec4f898eb43973 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.725391 22527 raft_consensus.cc:2468] T 93dd95f0dfb8463291a74921c46665dd P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e20f6c83ae144d698c26b94c2eda023f in term 1.
I20260709 13:21:29.732167 22848 leader_election.cc:290] T a3b0925412bd4dceabec4f898eb43973 P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823), e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469)
I20260709 13:21:29.734026 22601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3b0925412bd4dceabec4f898eb43973" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e20f6c83ae144d698c26b94c2eda023f" is_pre_election: true
I20260709 13:21:29.734843 22601 raft_consensus.cc:2468] T a3b0925412bd4dceabec4f898eb43973 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d62e0b282b44003a202b8dcbe6ea6ac in term 0.
I20260709 13:21:29.735965 22640 leader_election.cc:304] T a3b0925412bd4dceabec4f898eb43973 P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d62e0b282b44003a202b8dcbe6ea6ac, e20f6c83ae144d698c26b94c2eda023f; no voters: 
I20260709 13:21:29.736982 22752 raft_consensus.cc:2804] T a3b0925412bd4dceabec4f898eb43973 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:29.737407 22752 raft_consensus.cc:493] T a3b0925412bd4dceabec4f898eb43973 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:29.737818 22752 raft_consensus.cc:3060] T a3b0925412bd4dceabec4f898eb43973 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.715323 22568 leader_election.cc:304] T 93dd95f0dfb8463291a74921c46665dd P e20f6c83ae144d698c26b94c2eda023f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d62e0b282b44003a202b8dcbe6ea6ac, e20f6c83ae144d698c26b94c2eda023f; no voters: 
I20260709 13:21:29.741566 22848 raft_consensus.cc:493] T 47e33cdb6a154e8781b3e36a95eed353 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:29.742328 22848 raft_consensus.cc:515] T 47e33cdb6a154e8781b3e36a95eed353 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.742519 22424 catalog_manager.cc:5697] T 5b97872c6f3b4a288ff8ade4569da6f1 P 2d62e0b282b44003a202b8dcbe6ea6ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131). New cstate: current_term: 1 leader_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:29.744690 22848 leader_election.cc:290] T 47e33cdb6a154e8781b3e36a95eed353 P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823), e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469)
I20260709 13:21:29.744437 22424 catalog_manager.cc:5697] T ef14a8f3b6344312bd294edcabb2f312 P 2d62e0b282b44003a202b8dcbe6ea6ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131). New cstate: current_term: 1 leader_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:29.745847 22601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47e33cdb6a154e8781b3e36a95eed353" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e20f6c83ae144d698c26b94c2eda023f" is_pre_election: true
I20260709 13:21:29.748781 22811 raft_consensus.cc:2804] T 93dd95f0dfb8463291a74921c46665dd P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:29.751900 22811 raft_consensus.cc:697] T 93dd95f0dfb8463291a74921c46665dd P e20f6c83ae144d698c26b94c2eda023f [term 1 LEADER]: Becoming Leader. State: Replica: e20f6c83ae144d698c26b94c2eda023f, State: Running, Role: LEADER
I20260709 13:21:29.752769 22599 raft_consensus.cc:2468] T e1e94cad244443feb24f679271460852 P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7df364d160d64ac4b5cdc4f004324f8b in term 1.
I20260709 13:21:29.753358 22811 consensus_queue.cc:237] T 93dd95f0dfb8463291a74921c46665dd P e20f6c83ae144d698c26b94c2eda023f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.764446 22422 catalog_manager.cc:5697] T e1e94cad244443feb24f679271460852 P 7df364d160d64ac4b5cdc4f004324f8b reported cstate change: term changed from 0 to 1, leader changed from <none> to 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129). New cstate: current_term: 1 leader_uuid: "7df364d160d64ac4b5cdc4f004324f8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:29.768208 22422 catalog_manager.cc:5697] T 2f75c1bbbd7b4dc09412c016798ccd50 P 7df364d160d64ac4b5cdc4f004324f8b reported cstate change: term changed from 0 to 1, leader changed from <none> to 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129). New cstate: current_term: 1 leader_uuid: "7df364d160d64ac4b5cdc4f004324f8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:29.771152 22527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3b0925412bd4dceabec4f898eb43973" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b" is_pre_election: true
I20260709 13:21:29.771917 22527 raft_consensus.cc:2468] T a3b0925412bd4dceabec4f898eb43973 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d62e0b282b44003a202b8dcbe6ea6ac in term 0.
I20260709 13:21:29.774479 22527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47e33cdb6a154e8781b3e36a95eed353" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b" is_pre_election: true
I20260709 13:21:29.775395 22527 raft_consensus.cc:2393] T 47e33cdb6a154e8781b3e36a95eed353 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2d62e0b282b44003a202b8dcbe6ea6ac in current term 1: Already voted for candidate e20f6c83ae144d698c26b94c2eda023f in this term.
I20260709 13:21:29.777365 22642 leader_election.cc:304] T 47e33cdb6a154e8781b3e36a95eed353 P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2d62e0b282b44003a202b8dcbe6ea6ac; no voters: 7df364d160d64ac4b5cdc4f004324f8b, e20f6c83ae144d698c26b94c2eda023f
I20260709 13:21:29.778223 22848 raft_consensus.cc:3060] T 47e33cdb6a154e8781b3e36a95eed353 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.783143 22423 catalog_manager.cc:5697] T 93dd95f0dfb8463291a74921c46665dd P e20f6c83ae144d698c26b94c2eda023f reported cstate change: term changed from 0 to 1, leader changed from <none> to e20f6c83ae144d698c26b94c2eda023f (127.9.116.130). New cstate: current_term: 1 leader_uuid: "e20f6c83ae144d698c26b94c2eda023f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:29.788416 22848 raft_consensus.cc:2749] T 47e33cdb6a154e8781b3e36a95eed353 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:29.792654 22526 raft_consensus.cc:2468] T ef14a8f3b6344312bd294edcabb2f312 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d62e0b282b44003a202b8dcbe6ea6ac in term 1.
I20260709 13:21:29.793365 22860 raft_consensus.cc:2749] T ef14a8f3b6344312bd294edcabb2f312 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:29.793537 22752 raft_consensus.cc:515] T a3b0925412bd4dceabec4f898eb43973 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.795979 22752 leader_election.cc:290] T a3b0925412bd4dceabec4f898eb43973 P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 election: Requested vote from peers 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823), e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469)
I20260709 13:21:29.797255 22599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3b0925412bd4dceabec4f898eb43973" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e20f6c83ae144d698c26b94c2eda023f"
I20260709 13:21:29.798413 22526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3b0925412bd4dceabec4f898eb43973" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b"
I20260709 13:21:29.799690 22599 raft_consensus.cc:3060] T a3b0925412bd4dceabec4f898eb43973 P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.799912 22526 raft_consensus.cc:3060] T a3b0925412bd4dceabec4f898eb43973 P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.807071 22526 raft_consensus.cc:2468] T a3b0925412bd4dceabec4f898eb43973 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d62e0b282b44003a202b8dcbe6ea6ac in term 1.
I20260709 13:21:29.807076 22599 raft_consensus.cc:2468] T a3b0925412bd4dceabec4f898eb43973 P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d62e0b282b44003a202b8dcbe6ea6ac in term 1.
I20260709 13:21:29.808416 22640 leader_election.cc:304] T a3b0925412bd4dceabec4f898eb43973 P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d62e0b282b44003a202b8dcbe6ea6ac, e20f6c83ae144d698c26b94c2eda023f; no voters: 
I20260709 13:21:29.809193 22752 raft_consensus.cc:2804] T a3b0925412bd4dceabec4f898eb43973 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:29.809716 22752 raft_consensus.cc:697] T a3b0925412bd4dceabec4f898eb43973 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 LEADER]: Becoming Leader. State: Replica: 2d62e0b282b44003a202b8dcbe6ea6ac, State: Running, Role: LEADER
I20260709 13:21:29.810704 22752 consensus_queue.cc:237] T a3b0925412bd4dceabec4f898eb43973 P 2d62e0b282b44003a202b8dcbe6ea6ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.818514 22423 catalog_manager.cc:5697] T a3b0925412bd4dceabec4f898eb43973 P 2d62e0b282b44003a202b8dcbe6ea6ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131). New cstate: current_term: 1 leader_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:29.926919 22860 raft_consensus.cc:493] T 2d4c5545efd2494a81249b0dbbfbc56c P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:29.927424 22860 raft_consensus.cc:515] T 2d4c5545efd2494a81249b0dbbfbc56c P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.929270 22860 leader_election.cc:290] T 2d4c5545efd2494a81249b0dbbfbc56c P 7df364d160d64ac4b5cdc4f004324f8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469), 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335)
I20260709 13:21:29.930145 22599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d4c5545efd2494a81249b0dbbfbc56c" candidate_uuid: "7df364d160d64ac4b5cdc4f004324f8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e20f6c83ae144d698c26b94c2eda023f" is_pre_election: true
I20260709 13:21:29.930553 22674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d4c5545efd2494a81249b0dbbfbc56c" candidate_uuid: "7df364d160d64ac4b5cdc4f004324f8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" is_pre_election: true
I20260709 13:21:29.931005 22599 raft_consensus.cc:2468] T 2d4c5545efd2494a81249b0dbbfbc56c P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7df364d160d64ac4b5cdc4f004324f8b in term 0.
I20260709 13:21:29.931246 22674 raft_consensus.cc:2468] T 2d4c5545efd2494a81249b0dbbfbc56c P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7df364d160d64ac4b5cdc4f004324f8b in term 0.
I20260709 13:21:29.932257 22491 leader_election.cc:304] T 2d4c5545efd2494a81249b0dbbfbc56c P 7df364d160d64ac4b5cdc4f004324f8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7df364d160d64ac4b5cdc4f004324f8b, e20f6c83ae144d698c26b94c2eda023f; no voters: 
I20260709 13:21:29.933058 22860 raft_consensus.cc:2804] T 2d4c5545efd2494a81249b0dbbfbc56c P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:29.933473 22860 raft_consensus.cc:493] T 2d4c5545efd2494a81249b0dbbfbc56c P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:29.933817 22860 raft_consensus.cc:3060] T 2d4c5545efd2494a81249b0dbbfbc56c P 7df364d160d64ac4b5cdc4f004324f8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.937561 22870 consensus_queue.cc:1048] T 47e33cdb6a154e8781b3e36a95eed353 P e20f6c83ae144d698c26b94c2eda023f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:29.947137 22860 raft_consensus.cc:515] T 2d4c5545efd2494a81249b0dbbfbc56c P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.950485 22599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d4c5545efd2494a81249b0dbbfbc56c" candidate_uuid: "7df364d160d64ac4b5cdc4f004324f8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e20f6c83ae144d698c26b94c2eda023f"
I20260709 13:21:29.951205 22599 raft_consensus.cc:3060] T 2d4c5545efd2494a81249b0dbbfbc56c P e20f6c83ae144d698c26b94c2eda023f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.959530 22752 consensus_queue.cc:1048] T 0f952d6650154a4a98f94a75b916d44b P 2d62e0b282b44003a202b8dcbe6ea6ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:29.973892 22860 leader_election.cc:290] T 2d4c5545efd2494a81249b0dbbfbc56c P 7df364d160d64ac4b5cdc4f004324f8b [CANDIDATE]: Term 1 election: Requested vote from peers e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469), 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335)
I20260709 13:21:29.961488 22599 raft_consensus.cc:2468] T 2d4c5545efd2494a81249b0dbbfbc56c P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7df364d160d64ac4b5cdc4f004324f8b in term 1.
I20260709 13:21:29.975744 22491 leader_election.cc:304] T 2d4c5545efd2494a81249b0dbbfbc56c P 7df364d160d64ac4b5cdc4f004324f8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7df364d160d64ac4b5cdc4f004324f8b, e20f6c83ae144d698c26b94c2eda023f; no voters: 
I20260709 13:21:29.976680 22860 raft_consensus.cc:2804] T 2d4c5545efd2494a81249b0dbbfbc56c P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:29.978077 22860 raft_consensus.cc:697] T 2d4c5545efd2494a81249b0dbbfbc56c P 7df364d160d64ac4b5cdc4f004324f8b [term 1 LEADER]: Becoming Leader. State: Replica: 7df364d160d64ac4b5cdc4f004324f8b, State: Running, Role: LEADER
I20260709 13:21:29.978832 22676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d4c5545efd2494a81249b0dbbfbc56c" candidate_uuid: "7df364d160d64ac4b5cdc4f004324f8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac"
I20260709 13:21:29.979557 22676 raft_consensus.cc:3060] T 2d4c5545efd2494a81249b0dbbfbc56c P 2d62e0b282b44003a202b8dcbe6ea6ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.978989 22860 consensus_queue.cc:237] T 2d4c5545efd2494a81249b0dbbfbc56c P 7df364d160d64ac4b5cdc4f004324f8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:29.993644 22811 consensus_queue.cc:1048] T 47e33cdb6a154e8781b3e36a95eed353 P e20f6c83ae144d698c26b94c2eda023f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:30.000934 22676 raft_consensus.cc:2468] T 2d4c5545efd2494a81249b0dbbfbc56c P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7df364d160d64ac4b5cdc4f004324f8b in term 1.
I20260709 13:21:30.009074 22422 catalog_manager.cc:5697] T 2d4c5545efd2494a81249b0dbbfbc56c P 7df364d160d64ac4b5cdc4f004324f8b reported cstate change: term changed from 0 to 1, leader changed from <none> to 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129). New cstate: current_term: 1 leader_uuid: "7df364d160d64ac4b5cdc4f004324f8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:30.012756 22752 consensus_queue.cc:1048] T 0f952d6650154a4a98f94a75b916d44b P 2d62e0b282b44003a202b8dcbe6ea6ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:30.027835 22860 consensus_queue.cc:1048] T 2f75c1bbbd7b4dc09412c016798ccd50 P 7df364d160d64ac4b5cdc4f004324f8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:30.038964 22860 consensus_queue.cc:1048] T 2f75c1bbbd7b4dc09412c016798ccd50 P 7df364d160d64ac4b5cdc4f004324f8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:30.070995 22674 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0f952d6650154a4a98f94a75b916d44b"
dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac"
mode: GRACEFUL
new_leader_uuid: "7df364d160d64ac4b5cdc4f004324f8b"
 from {username='slave'} at 127.0.0.1:48188
I20260709 13:21:30.071521 22674 raft_consensus.cc:606] T 0f952d6650154a4a98f94a75b916d44b P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 LEADER]: Received request to transfer leadership to TS 7df364d160d64ac4b5cdc4f004324f8b
I20260709 13:21:30.076507 22674 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ef14a8f3b6344312bd294edcabb2f312"
dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac"
mode: GRACEFUL
new_leader_uuid: "7df364d160d64ac4b5cdc4f004324f8b"
 from {username='slave'} at 127.0.0.1:48188
I20260709 13:21:30.076992 22674 raft_consensus.cc:606] T ef14a8f3b6344312bd294edcabb2f312 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 LEADER]: Received request to transfer leadership to TS 7df364d160d64ac4b5cdc4f004324f8b
I20260709 13:21:30.081918 22599 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "47e33cdb6a154e8781b3e36a95eed353"
dest_uuid: "e20f6c83ae144d698c26b94c2eda023f"
mode: GRACEFUL
new_leader_uuid: "7df364d160d64ac4b5cdc4f004324f8b"
 from {username='slave'} at 127.0.0.1:50812
I20260709 13:21:30.082463 22599 raft_consensus.cc:606] T 47e33cdb6a154e8781b3e36a95eed353 P e20f6c83ae144d698c26b94c2eda023f [term 1 LEADER]: Received request to transfer leadership to TS 7df364d160d64ac4b5cdc4f004324f8b
I20260709 13:21:30.087951 22674 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5b97872c6f3b4a288ff8ade4569da6f1"
dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac"
mode: GRACEFUL
new_leader_uuid: "7df364d160d64ac4b5cdc4f004324f8b"
 from {username='slave'} at 127.0.0.1:48188
I20260709 13:21:30.088462 22674 raft_consensus.cc:606] T 5b97872c6f3b4a288ff8ade4569da6f1 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 LEADER]: Received request to transfer leadership to TS 7df364d160d64ac4b5cdc4f004324f8b
I20260709 13:21:30.093436 22526 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2f75c1bbbd7b4dc09412c016798ccd50"
dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b"
mode: GRACEFUL
new_leader_uuid: "e20f6c83ae144d698c26b94c2eda023f"
 from {username='slave'} at 127.0.0.1:48032
I20260709 13:21:30.093967 22526 raft_consensus.cc:606] T 2f75c1bbbd7b4dc09412c016798ccd50 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 LEADER]: Received request to transfer leadership to TS e20f6c83ae144d698c26b94c2eda023f
I20260709 13:21:30.099757 22526 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e1e94cad244443feb24f679271460852"
dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b"
mode: GRACEFUL
new_leader_uuid: "e20f6c83ae144d698c26b94c2eda023f"
 from {username='slave'} at 127.0.0.1:48032
I20260709 13:21:30.100277 22526 raft_consensus.cc:606] T e1e94cad244443feb24f679271460852 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 LEADER]: Received request to transfer leadership to TS e20f6c83ae144d698c26b94c2eda023f
I20260709 13:21:30.105041 22674 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a3b0925412bd4dceabec4f898eb43973"
dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac"
mode: GRACEFUL
new_leader_uuid: "e20f6c83ae144d698c26b94c2eda023f"
 from {username='slave'} at 127.0.0.1:48188
I20260709 13:21:30.105556 22674 raft_consensus.cc:606] T a3b0925412bd4dceabec4f898eb43973 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 LEADER]: Received request to transfer leadership to TS e20f6c83ae144d698c26b94c2eda023f
I20260709 13:21:30.112907 22526 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2d4c5545efd2494a81249b0dbbfbc56c"
dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b"
mode: GRACEFUL
new_leader_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac"
 from {username='slave'} at 127.0.0.1:48032
I20260709 13:21:30.113554 22526 raft_consensus.cc:606] T 2d4c5545efd2494a81249b0dbbfbc56c P 7df364d160d64ac4b5cdc4f004324f8b [term 1 LEADER]: Received request to transfer leadership to TS 2d62e0b282b44003a202b8dcbe6ea6ac
I20260709 13:21:30.125356 22843 consensus_queue.cc:1048] T ef14a8f3b6344312bd294edcabb2f312 P 2d62e0b282b44003a202b8dcbe6ea6ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:30.132700 22752 raft_consensus.cc:993] T ef14a8f3b6344312bd294edcabb2f312 P 2d62e0b282b44003a202b8dcbe6ea6ac: : Instructing follower 7df364d160d64ac4b5cdc4f004324f8b to start an election
I20260709 13:21:30.133172 22752 raft_consensus.cc:1081] T ef14a8f3b6344312bd294edcabb2f312 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 LEADER]: Signalling peer 7df364d160d64ac4b5cdc4f004324f8b to start an election
I20260709 13:21:30.138501 22526 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ef14a8f3b6344312bd294edcabb2f312"
dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b"
 from {username='slave'} at 127.0.0.1:48026
I20260709 13:21:30.140100 22848 consensus_queue.cc:1048] T ef14a8f3b6344312bd294edcabb2f312 P 2d62e0b282b44003a202b8dcbe6ea6ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:30.142937 22526 raft_consensus.cc:493] T ef14a8f3b6344312bd294edcabb2f312 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:30.143471 22526 raft_consensus.cc:3060] T ef14a8f3b6344312bd294edcabb2f312 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:30.145758 22752 consensus_queue.cc:1048] T 5b97872c6f3b4a288ff8ade4569da6f1 P 2d62e0b282b44003a202b8dcbe6ea6ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:30.151553 22526 raft_consensus.cc:515] T ef14a8f3b6344312bd294edcabb2f312 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:30.154909 22526 leader_election.cc:290] T ef14a8f3b6344312bd294edcabb2f312 P 7df364d160d64ac4b5cdc4f004324f8b [CANDIDATE]: Term 2 election: Requested vote from peers e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469), 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335)
I20260709 13:21:30.154945 22601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef14a8f3b6344312bd294edcabb2f312" candidate_uuid: "7df364d160d64ac4b5cdc4f004324f8b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e20f6c83ae144d698c26b94c2eda023f"
I20260709 13:21:30.155323 22674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef14a8f3b6344312bd294edcabb2f312" candidate_uuid: "7df364d160d64ac4b5cdc4f004324f8b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac"
I20260709 13:21:30.155846 22674 raft_consensus.cc:3055] T ef14a8f3b6344312bd294edcabb2f312 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:30.156165 22674 raft_consensus.cc:740] T ef14a8f3b6344312bd294edcabb2f312 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2d62e0b282b44003a202b8dcbe6ea6ac, State: Running, Role: LEADER
I20260709 13:21:30.157001 22674 consensus_queue.cc:260] T ef14a8f3b6344312bd294edcabb2f312 P 2d62e0b282b44003a202b8dcbe6ea6ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:30.157903 22674 raft_consensus.cc:3060] T ef14a8f3b6344312bd294edcabb2f312 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:30.157714 22847 consensus_queue.cc:1048] T 93dd95f0dfb8463291a74921c46665dd P e20f6c83ae144d698c26b94c2eda023f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:30.162974 22848 raft_consensus.cc:993] T 5b97872c6f3b4a288ff8ade4569da6f1 P 2d62e0b282b44003a202b8dcbe6ea6ac: : Instructing follower 7df364d160d64ac4b5cdc4f004324f8b to start an election
I20260709 13:21:30.163496 22843 raft_consensus.cc:1081] T 5b97872c6f3b4a288ff8ade4569da6f1 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 LEADER]: Signalling peer 7df364d160d64ac4b5cdc4f004324f8b to start an election
I20260709 13:21:30.167986 22527 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5b97872c6f3b4a288ff8ade4569da6f1"
dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b"
 from {username='slave'} at 127.0.0.1:48026
I20260709 13:21:30.168619 22527 raft_consensus.cc:493] T 5b97872c6f3b4a288ff8ade4569da6f1 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:30.168963 22527 raft_consensus.cc:3060] T 5b97872c6f3b4a288ff8ade4569da6f1 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:30.171419 22843 consensus_queue.cc:1048] T 5b97872c6f3b4a288ff8ade4569da6f1 P 2d62e0b282b44003a202b8dcbe6ea6ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:30.174507 22847 consensus_queue.cc:1048] T 93dd95f0dfb8463291a74921c46665dd P e20f6c83ae144d698c26b94c2eda023f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:30.175977 22527 raft_consensus.cc:515] T 5b97872c6f3b4a288ff8ade4569da6f1 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:30.167053 22674 raft_consensus.cc:2468] T ef14a8f3b6344312bd294edcabb2f312 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7df364d160d64ac4b5cdc4f004324f8b in term 2.
I20260709 13:21:30.180384 22493 leader_election.cc:304] T ef14a8f3b6344312bd294edcabb2f312 P 7df364d160d64ac4b5cdc4f004324f8b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2d62e0b282b44003a202b8dcbe6ea6ac, 7df364d160d64ac4b5cdc4f004324f8b; no voters: e20f6c83ae144d698c26b94c2eda023f
I20260709 13:21:30.181370 22854 raft_consensus.cc:2804] T ef14a8f3b6344312bd294edcabb2f312 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:30.181552 22601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b97872c6f3b4a288ff8ade4569da6f1" candidate_uuid: "7df364d160d64ac4b5cdc4f004324f8b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e20f6c83ae144d698c26b94c2eda023f"
I20260709 13:21:30.182040 22854 raft_consensus.cc:697] T ef14a8f3b6344312bd294edcabb2f312 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 LEADER]: Becoming Leader. State: Replica: 7df364d160d64ac4b5cdc4f004324f8b, State: Running, Role: LEADER
I20260709 13:21:30.184010 22527 leader_election.cc:290] T 5b97872c6f3b4a288ff8ade4569da6f1 P 7df364d160d64ac4b5cdc4f004324f8b [CANDIDATE]: Term 2 election: Requested vote from peers e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469), 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335)
I20260709 13:21:30.183559 22854 consensus_queue.cc:237] T ef14a8f3b6344312bd294edcabb2f312 P 7df364d160d64ac4b5cdc4f004324f8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:30.187256 22674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b97872c6f3b4a288ff8ade4569da6f1" candidate_uuid: "7df364d160d64ac4b5cdc4f004324f8b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac"
I20260709 13:21:30.187947 22674 raft_consensus.cc:3055] T 5b97872c6f3b4a288ff8ade4569da6f1 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:30.188247 22674 raft_consensus.cc:740] T 5b97872c6f3b4a288ff8ade4569da6f1 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2d62e0b282b44003a202b8dcbe6ea6ac, State: Running, Role: LEADER
I20260709 13:21:30.189060 22674 consensus_queue.cc:260] T 5b97872c6f3b4a288ff8ade4569da6f1 P 2d62e0b282b44003a202b8dcbe6ea6ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:30.190162 22674 raft_consensus.cc:3060] T 5b97872c6f3b4a288ff8ade4569da6f1 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:30.197677 22674 raft_consensus.cc:2468] T 5b97872c6f3b4a288ff8ade4569da6f1 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7df364d160d64ac4b5cdc4f004324f8b in term 2.
I20260709 13:21:30.198949 22493 leader_election.cc:304] T 5b97872c6f3b4a288ff8ade4569da6f1 P 7df364d160d64ac4b5cdc4f004324f8b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2d62e0b282b44003a202b8dcbe6ea6ac, 7df364d160d64ac4b5cdc4f004324f8b; no voters: e20f6c83ae144d698c26b94c2eda023f
I20260709 13:21:30.200667 22854 raft_consensus.cc:2804] T 5b97872c6f3b4a288ff8ade4569da6f1 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:30.201112 22854 raft_consensus.cc:697] T 5b97872c6f3b4a288ff8ade4569da6f1 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 LEADER]: Becoming Leader. State: Replica: 7df364d160d64ac4b5cdc4f004324f8b, State: Running, Role: LEADER
I20260709 13:21:30.201949 22854 consensus_queue.cc:237] T 5b97872c6f3b4a288ff8ade4569da6f1 P 7df364d160d64ac4b5cdc4f004324f8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:30.206501 22854 consensus_queue.cc:1048] T e1e94cad244443feb24f679271460852 P 7df364d160d64ac4b5cdc4f004324f8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:30.214699 22860 raft_consensus.cc:993] T e1e94cad244443feb24f679271460852 P 7df364d160d64ac4b5cdc4f004324f8b: : Instructing follower e20f6c83ae144d698c26b94c2eda023f to start an election
I20260709 13:21:30.215557 22860 raft_consensus.cc:1081] T e1e94cad244443feb24f679271460852 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 LEADER]: Signalling peer e20f6c83ae144d698c26b94c2eda023f to start an election
I20260709 13:21:30.216094 22423 catalog_manager.cc:5697] T ef14a8f3b6344312bd294edcabb2f312 P 7df364d160d64ac4b5cdc4f004324f8b reported cstate change: term changed from 1 to 2, leader changed from 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131) to 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129). New cstate: current_term: 2 leader_uuid: "7df364d160d64ac4b5cdc4f004324f8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:30.218314 22601 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e1e94cad244443feb24f679271460852"
dest_uuid: "e20f6c83ae144d698c26b94c2eda023f"
 from {username='slave'} at 127.0.0.1:50796
I20260709 13:21:30.219029 22601 raft_consensus.cc:493] T e1e94cad244443feb24f679271460852 P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:30.218617 22423 catalog_manager.cc:5697] T 5b97872c6f3b4a288ff8ade4569da6f1 P 7df364d160d64ac4b5cdc4f004324f8b reported cstate change: term changed from 1 to 2, leader changed from 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131) to 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129). New cstate: current_term: 2 leader_uuid: "7df364d160d64ac4b5cdc4f004324f8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:30.219447 22601 raft_consensus.cc:3060] T e1e94cad244443feb24f679271460852 P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:30.220225 22860 consensus_queue.cc:1048] T e1e94cad244443feb24f679271460852 P 7df364d160d64ac4b5cdc4f004324f8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:30.227636 22601 raft_consensus.cc:515] T e1e94cad244443feb24f679271460852 P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:30.230767 22526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e94cad244443feb24f679271460852" candidate_uuid: "e20f6c83ae144d698c26b94c2eda023f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b"
I20260709 13:21:30.231415 22526 raft_consensus.cc:3055] T e1e94cad244443feb24f679271460852 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:30.231818 22526 raft_consensus.cc:740] T e1e94cad244443feb24f679271460852 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7df364d160d64ac4b5cdc4f004324f8b, State: Running, Role: LEADER
I20260709 13:21:30.232627 22526 consensus_queue.cc:260] T e1e94cad244443feb24f679271460852 P 7df364d160d64ac4b5cdc4f004324f8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:30.233794 22526 raft_consensus.cc:3060] T e1e94cad244443feb24f679271460852 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:30.235610 22599 raft_consensus.cc:1240] T e1e94cad244443feb24f679271460852 P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Rejecting Update request from peer 7df364d160d64ac4b5cdc4f004324f8b for earlier term 1. Current term is 2. Ops: []
I20260709 13:21:30.237197 22601 leader_election.cc:290] T e1e94cad244443feb24f679271460852 P e20f6c83ae144d698c26b94c2eda023f [CANDIDATE]: Term 2 election: Requested vote from peers 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823), 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335)
I20260709 13:21:30.241019 22674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e94cad244443feb24f679271460852" candidate_uuid: "e20f6c83ae144d698c26b94c2eda023f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac"
I20260709 13:21:30.241647 22674 raft_consensus.cc:3060] T e1e94cad244443feb24f679271460852 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:30.246281 22526 raft_consensus.cc:2468] T e1e94cad244443feb24f679271460852 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e20f6c83ae144d698c26b94c2eda023f in term 2.
I20260709 13:21:30.246667 22674 raft_consensus.cc:2468] T e1e94cad244443feb24f679271460852 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e20f6c83ae144d698c26b94c2eda023f in term 2.
I20260709 13:21:30.247771 22568 leader_election.cc:304] T e1e94cad244443feb24f679271460852 P e20f6c83ae144d698c26b94c2eda023f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7df364d160d64ac4b5cdc4f004324f8b, e20f6c83ae144d698c26b94c2eda023f; no voters: 
I20260709 13:21:30.249229 22847 raft_consensus.cc:2804] T e1e94cad244443feb24f679271460852 P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:30.249722 22847 raft_consensus.cc:697] T e1e94cad244443feb24f679271460852 P e20f6c83ae144d698c26b94c2eda023f [term 2 LEADER]: Becoming Leader. State: Replica: e20f6c83ae144d698c26b94c2eda023f, State: Running, Role: LEADER
I20260709 13:21:30.250540 22847 consensus_queue.cc:237] T e1e94cad244443feb24f679271460852 P e20f6c83ae144d698c26b94c2eda023f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:30.257828 22424 catalog_manager.cc:5697] T e1e94cad244443feb24f679271460852 P e20f6c83ae144d698c26b94c2eda023f reported cstate change: term changed from 1 to 2, leader changed from 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129) to e20f6c83ae144d698c26b94c2eda023f (127.9.116.130). New cstate: current_term: 2 leader_uuid: "e20f6c83ae144d698c26b94c2eda023f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:30.380998 22843 consensus_queue.cc:1048] T a3b0925412bd4dceabec4f898eb43973 P 2d62e0b282b44003a202b8dcbe6ea6ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:21:30.383966 22843 consensus_queue.cc:1048] T a3b0925412bd4dceabec4f898eb43973 P 2d62e0b282b44003a202b8dcbe6ea6ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:30.404315 22854 consensus_queue.cc:1048] T 2d4c5545efd2494a81249b0dbbfbc56c P 7df364d160d64ac4b5cdc4f004324f8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:30.424476 22752 raft_consensus.cc:993] T a3b0925412bd4dceabec4f898eb43973 P 2d62e0b282b44003a202b8dcbe6ea6ac: : Instructing follower e20f6c83ae144d698c26b94c2eda023f to start an election
I20260709 13:21:30.425040 22752 raft_consensus.cc:1081] T a3b0925412bd4dceabec4f898eb43973 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 LEADER]: Signalling peer e20f6c83ae144d698c26b94c2eda023f to start an election
I20260709 13:21:30.435973 22602 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a3b0925412bd4dceabec4f898eb43973"
dest_uuid: "e20f6c83ae144d698c26b94c2eda023f"
 from {username='slave'} at 127.0.0.1:50800
I20260709 13:21:30.436872 22602 raft_consensus.cc:493] T a3b0925412bd4dceabec4f898eb43973 P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:30.437426 22602 raft_consensus.cc:3060] T a3b0925412bd4dceabec4f898eb43973 P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:30.440781 22843 raft_consensus.cc:993] T 0f952d6650154a4a98f94a75b916d44b P 2d62e0b282b44003a202b8dcbe6ea6ac: : Instructing follower 7df364d160d64ac4b5cdc4f004324f8b to start an election
I20260709 13:21:30.441483 22843 raft_consensus.cc:1081] T 0f952d6650154a4a98f94a75b916d44b P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 LEADER]: Signalling peer 7df364d160d64ac4b5cdc4f004324f8b to start an election
I20260709 13:21:30.443755 22527 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0f952d6650154a4a98f94a75b916d44b"
dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b"
 from {username='slave'} at 127.0.0.1:48026
I20260709 13:21:30.444336 22527 raft_consensus.cc:493] T 0f952d6650154a4a98f94a75b916d44b P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:30.444756 22527 raft_consensus.cc:3060] T 0f952d6650154a4a98f94a75b916d44b P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:30.448380 22860 raft_consensus.cc:993] T 2f75c1bbbd7b4dc09412c016798ccd50 P 7df364d160d64ac4b5cdc4f004324f8b: : Instructing follower e20f6c83ae144d698c26b94c2eda023f to start an election
I20260709 13:21:30.449188 22855 raft_consensus.cc:1081] T 2f75c1bbbd7b4dc09412c016798ccd50 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 LEADER]: Signalling peer e20f6c83ae144d698c26b94c2eda023f to start an election
I20260709 13:21:30.452366 22600 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2f75c1bbbd7b4dc09412c016798ccd50"
dest_uuid: "e20f6c83ae144d698c26b94c2eda023f"
 from {username='slave'} at 127.0.0.1:50796
I20260709 13:21:30.452966 22600 raft_consensus.cc:493] T 2f75c1bbbd7b4dc09412c016798ccd50 P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:30.453323 22600 raft_consensus.cc:3060] T 2f75c1bbbd7b4dc09412c016798ccd50 P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:30.455657 22602 raft_consensus.cc:515] T a3b0925412bd4dceabec4f898eb43973 P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:30.456861 22599 raft_consensus.cc:1240] T a3b0925412bd4dceabec4f898eb43973 P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Rejecting Update request from peer 2d62e0b282b44003a202b8dcbe6ea6ac for earlier term 1. Current term is 2. Ops: []
I20260709 13:21:30.458303 22602 leader_election.cc:290] T a3b0925412bd4dceabec4f898eb43973 P e20f6c83ae144d698c26b94c2eda023f [CANDIDATE]: Term 2 election: Requested vote from peers 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823), 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335)
I20260709 13:21:30.460682 22848 consensus_queue.cc:1059] T a3b0925412bd4dceabec4f898eb43973 P 2d62e0b282b44003a202b8dcbe6ea6ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:30.461745 22752 raft_consensus.cc:3055] T a3b0925412bd4dceabec4f898eb43973 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:30.462103 22752 raft_consensus.cc:740] T a3b0925412bd4dceabec4f898eb43973 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2d62e0b282b44003a202b8dcbe6ea6ac, State: Running, Role: LEADER
I20260709 13:21:30.458498 22527 raft_consensus.cc:515] T 0f952d6650154a4a98f94a75b916d44b P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:30.462352 22526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3b0925412bd4dceabec4f898eb43973" candidate_uuid: "e20f6c83ae144d698c26b94c2eda023f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b"
I20260709 13:21:30.463614 22526 raft_consensus.cc:3060] T a3b0925412bd4dceabec4f898eb43973 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:30.463413 22752 consensus_queue.cc:260] T a3b0925412bd4dceabec4f898eb43973 P 2d62e0b282b44003a202b8dcbe6ea6ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:30.465279 22674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3b0925412bd4dceabec4f898eb43973" candidate_uuid: "e20f6c83ae144d698c26b94c2eda023f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac"
I20260709 13:21:30.466594 22527 leader_election.cc:290] T 0f952d6650154a4a98f94a75b916d44b P 7df364d160d64ac4b5cdc4f004324f8b [CANDIDATE]: Term 2 election: Requested vote from peers e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469), 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335)
I20260709 13:21:30.471670 22752 raft_consensus.cc:3060] T a3b0925412bd4dceabec4f898eb43973 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:30.472496 22602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f952d6650154a4a98f94a75b916d44b" candidate_uuid: "7df364d160d64ac4b5cdc4f004324f8b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e20f6c83ae144d698c26b94c2eda023f"
I20260709 13:21:30.473179 22602 raft_consensus.cc:3060] T 0f952d6650154a4a98f94a75b916d44b P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:30.475746 22676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f952d6650154a4a98f94a75b916d44b" candidate_uuid: "7df364d160d64ac4b5cdc4f004324f8b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac"
I20260709 13:21:30.476747 22676 raft_consensus.cc:3055] T 0f952d6650154a4a98f94a75b916d44b P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:30.477157 22676 raft_consensus.cc:740] T 0f952d6650154a4a98f94a75b916d44b P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2d62e0b282b44003a202b8dcbe6ea6ac, State: Running, Role: LEADER
I20260709 13:21:30.478485 22676 consensus_queue.cc:260] T 0f952d6650154a4a98f94a75b916d44b P 2d62e0b282b44003a202b8dcbe6ea6ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:30.480578 22676 raft_consensus.cc:3060] T 0f952d6650154a4a98f94a75b916d44b P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:30.481220 22526 raft_consensus.cc:2468] T a3b0925412bd4dceabec4f898eb43973 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e20f6c83ae144d698c26b94c2eda023f in term 2.
I20260709 13:21:30.483703 22568 leader_election.cc:304] T a3b0925412bd4dceabec4f898eb43973 P e20f6c83ae144d698c26b94c2eda023f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7df364d160d64ac4b5cdc4f004324f8b, e20f6c83ae144d698c26b94c2eda023f; no voters: 
I20260709 13:21:30.484817 22811 raft_consensus.cc:2804] T a3b0925412bd4dceabec4f898eb43973 P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:30.485532 22811 raft_consensus.cc:697] T a3b0925412bd4dceabec4f898eb43973 P e20f6c83ae144d698c26b94c2eda023f [term 2 LEADER]: Becoming Leader. State: Replica: e20f6c83ae144d698c26b94c2eda023f, State: Running, Role: LEADER
I20260709 13:21:30.486765 22811 consensus_queue.cc:237] T a3b0925412bd4dceabec4f898eb43973 P e20f6c83ae144d698c26b94c2eda023f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:30.492295 22602 raft_consensus.cc:2468] T 0f952d6650154a4a98f94a75b916d44b P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7df364d160d64ac4b5cdc4f004324f8b in term 2.
I20260709 13:21:30.493501 22600 raft_consensus.cc:515] T 2f75c1bbbd7b4dc09412c016798ccd50 P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:30.494270 22491 leader_election.cc:304] T 0f952d6650154a4a98f94a75b916d44b P 7df364d160d64ac4b5cdc4f004324f8b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7df364d160d64ac4b5cdc4f004324f8b, e20f6c83ae144d698c26b94c2eda023f; no voters: 
I20260709 13:21:30.495537 22855 raft_consensus.cc:2804] T 0f952d6650154a4a98f94a75b916d44b P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:30.496024 22855 raft_consensus.cc:697] T 0f952d6650154a4a98f94a75b916d44b P 7df364d160d64ac4b5cdc4f004324f8b [term 2 LEADER]: Becoming Leader. State: Replica: 7df364d160d64ac4b5cdc4f004324f8b, State: Running, Role: LEADER
I20260709 13:21:30.497288 22855 consensus_queue.cc:237] T 0f952d6650154a4a98f94a75b916d44b P 7df364d160d64ac4b5cdc4f004324f8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:30.501616 22860 consensus_queue.cc:1048] T 2d4c5545efd2494a81249b0dbbfbc56c P 7df364d160d64ac4b5cdc4f004324f8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:30.503305 22526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f75c1bbbd7b4dc09412c016798ccd50" candidate_uuid: "e20f6c83ae144d698c26b94c2eda023f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b"
I20260709 13:21:30.504325 22526 raft_consensus.cc:3055] T 2f75c1bbbd7b4dc09412c016798ccd50 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:30.504232 22424 catalog_manager.cc:5697] T a3b0925412bd4dceabec4f898eb43973 P e20f6c83ae144d698c26b94c2eda023f reported cstate change: term changed from 1 to 2, leader changed from 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131) to e20f6c83ae144d698c26b94c2eda023f (127.9.116.130). New cstate: current_term: 2 leader_uuid: "e20f6c83ae144d698c26b94c2eda023f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:30.505353 22600 leader_election.cc:290] T 2f75c1bbbd7b4dc09412c016798ccd50 P e20f6c83ae144d698c26b94c2eda023f [CANDIDATE]: Term 2 election: Requested vote from peers 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823), 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335)
I20260709 13:21:30.507194 22675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f75c1bbbd7b4dc09412c016798ccd50" candidate_uuid: "e20f6c83ae144d698c26b94c2eda023f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac"
I20260709 13:21:30.504863 22526 raft_consensus.cc:740] T 2f75c1bbbd7b4dc09412c016798ccd50 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7df364d160d64ac4b5cdc4f004324f8b, State: Running, Role: LEADER
I20260709 13:21:30.507968 22675 raft_consensus.cc:3060] T 2f75c1bbbd7b4dc09412c016798ccd50 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:30.509070 22526 consensus_queue.cc:260] T 2f75c1bbbd7b4dc09412c016798ccd50 P 7df364d160d64ac4b5cdc4f004324f8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:30.512674 22674 raft_consensus.cc:2468] T a3b0925412bd4dceabec4f898eb43973 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e20f6c83ae144d698c26b94c2eda023f in term 2.
I20260709 13:21:30.515093 22526 raft_consensus.cc:3060] T 2f75c1bbbd7b4dc09412c016798ccd50 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:30.527539 22676 raft_consensus.cc:2468] T 0f952d6650154a4a98f94a75b916d44b P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7df364d160d64ac4b5cdc4f004324f8b in term 2.
I20260709 13:21:30.532713 22425 catalog_manager.cc:5697] T 0f952d6650154a4a98f94a75b916d44b P 7df364d160d64ac4b5cdc4f004324f8b reported cstate change: term changed from 1 to 2, leader changed from 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131) to 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129). New cstate: current_term: 2 leader_uuid: "7df364d160d64ac4b5cdc4f004324f8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:30.535799 22870 raft_consensus.cc:993] T 47e33cdb6a154e8781b3e36a95eed353 P e20f6c83ae144d698c26b94c2eda023f: : Instructing follower 7df364d160d64ac4b5cdc4f004324f8b to start an election
I20260709 13:21:30.536396 22870 raft_consensus.cc:1081] T 47e33cdb6a154e8781b3e36a95eed353 P e20f6c83ae144d698c26b94c2eda023f [term 1 LEADER]: Signalling peer 7df364d160d64ac4b5cdc4f004324f8b to start an election
I20260709 13:21:30.538179 22527 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "47e33cdb6a154e8781b3e36a95eed353"
dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b"
 from {username='slave'} at 127.0.0.1:48022
I20260709 13:21:30.538746 22527 raft_consensus.cc:493] T 47e33cdb6a154e8781b3e36a95eed353 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:30.539132 22527 raft_consensus.cc:3060] T 47e33cdb6a154e8781b3e36a95eed353 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:30.541201 22675 raft_consensus.cc:2468] T 2f75c1bbbd7b4dc09412c016798ccd50 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e20f6c83ae144d698c26b94c2eda023f in term 2.
I20260709 13:21:30.543455 22568 leader_election.cc:304] T 2f75c1bbbd7b4dc09412c016798ccd50 P e20f6c83ae144d698c26b94c2eda023f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d62e0b282b44003a202b8dcbe6ea6ac, e20f6c83ae144d698c26b94c2eda023f; no voters: 
I20260709 13:21:30.544342 22811 raft_consensus.cc:2804] T 2f75c1bbbd7b4dc09412c016798ccd50 P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:30.544883 22811 raft_consensus.cc:697] T 2f75c1bbbd7b4dc09412c016798ccd50 P e20f6c83ae144d698c26b94c2eda023f [term 2 LEADER]: Becoming Leader. State: Replica: e20f6c83ae144d698c26b94c2eda023f, State: Running, Role: LEADER
I20260709 13:21:30.546290 22860 raft_consensus.cc:993] T 2d4c5545efd2494a81249b0dbbfbc56c P 7df364d160d64ac4b5cdc4f004324f8b: : Instructing follower 2d62e0b282b44003a202b8dcbe6ea6ac to start an election
I20260709 13:21:30.546840 22860 raft_consensus.cc:1081] T 2d4c5545efd2494a81249b0dbbfbc56c P 7df364d160d64ac4b5cdc4f004324f8b [term 1 LEADER]: Signalling peer 2d62e0b282b44003a202b8dcbe6ea6ac to start an election
I20260709 13:21:30.548022 22811 consensus_queue.cc:237] T 2f75c1bbbd7b4dc09412c016798ccd50 P e20f6c83ae144d698c26b94c2eda023f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:30.548835 22527 raft_consensus.cc:515] T 47e33cdb6a154e8781b3e36a95eed353 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:30.553907 22526 raft_consensus.cc:2468] T 2f75c1bbbd7b4dc09412c016798ccd50 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e20f6c83ae144d698c26b94c2eda023f in term 2.
I20260709 13:21:30.551538 22602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47e33cdb6a154e8781b3e36a95eed353" candidate_uuid: "7df364d160d64ac4b5cdc4f004324f8b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e20f6c83ae144d698c26b94c2eda023f"
I20260709 13:21:30.558019 22602 raft_consensus.cc:3055] T 47e33cdb6a154e8781b3e36a95eed353 P e20f6c83ae144d698c26b94c2eda023f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:30.558357 22602 raft_consensus.cc:740] T 47e33cdb6a154e8781b3e36a95eed353 P e20f6c83ae144d698c26b94c2eda023f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e20f6c83ae144d698c26b94c2eda023f, State: Running, Role: LEADER
I20260709 13:21:30.559249 22602 consensus_queue.cc:260] T 47e33cdb6a154e8781b3e36a95eed353 P e20f6c83ae144d698c26b94c2eda023f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:30.560473 22423 catalog_manager.cc:5697] T 2f75c1bbbd7b4dc09412c016798ccd50 P e20f6c83ae144d698c26b94c2eda023f reported cstate change: term changed from 1 to 2, leader changed from 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129) to e20f6c83ae144d698c26b94c2eda023f (127.9.116.130). New cstate: current_term: 2 leader_uuid: "e20f6c83ae144d698c26b94c2eda023f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:30.562270 22527 leader_election.cc:290] T 47e33cdb6a154e8781b3e36a95eed353 P 7df364d160d64ac4b5cdc4f004324f8b [CANDIDATE]: Term 2 election: Requested vote from peers e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469), 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131:34335)
I20260709 13:21:30.560438 22602 raft_consensus.cc:3060] T 47e33cdb6a154e8781b3e36a95eed353 P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:30.564545 22673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47e33cdb6a154e8781b3e36a95eed353" candidate_uuid: "7df364d160d64ac4b5cdc4f004324f8b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac"
I20260709 13:21:30.565200 22673 raft_consensus.cc:3060] T 47e33cdb6a154e8781b3e36a95eed353 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:30.557484 22675 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2d4c5545efd2494a81249b0dbbfbc56c"
dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac"
 from {username='slave'} at 127.0.0.1:48174
I20260709 13:21:30.567517 22675 raft_consensus.cc:493] T 2d4c5545efd2494a81249b0dbbfbc56c P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:30.568105 22675 raft_consensus.cc:3060] T 2d4c5545efd2494a81249b0dbbfbc56c P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:30.571477 22602 raft_consensus.cc:2468] T 47e33cdb6a154e8781b3e36a95eed353 P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7df364d160d64ac4b5cdc4f004324f8b in term 2.
I20260709 13:21:30.572739 22491 leader_election.cc:304] T 47e33cdb6a154e8781b3e36a95eed353 P 7df364d160d64ac4b5cdc4f004324f8b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7df364d160d64ac4b5cdc4f004324f8b, e20f6c83ae144d698c26b94c2eda023f; no voters: 
I20260709 13:21:30.573809 22860 raft_consensus.cc:2804] T 47e33cdb6a154e8781b3e36a95eed353 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:30.574342 22860 raft_consensus.cc:697] T 47e33cdb6a154e8781b3e36a95eed353 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 LEADER]: Becoming Leader. State: Replica: 7df364d160d64ac4b5cdc4f004324f8b, State: Running, Role: LEADER
I20260709 13:21:30.574750 22673 raft_consensus.cc:2468] T 47e33cdb6a154e8781b3e36a95eed353 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7df364d160d64ac4b5cdc4f004324f8b in term 2.
I20260709 13:21:30.575385 22860 consensus_queue.cc:237] T 47e33cdb6a154e8781b3e36a95eed353 P 7df364d160d64ac4b5cdc4f004324f8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:30.579149 22602 raft_consensus.cc:3060] T ef14a8f3b6344312bd294edcabb2f312 P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:30.581437 22675 raft_consensus.cc:515] T 2d4c5545efd2494a81249b0dbbfbc56c P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:30.584616 22526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d4c5545efd2494a81249b0dbbfbc56c" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b"
I20260709 13:21:30.584872 22675 leader_election.cc:290] T 2d4c5545efd2494a81249b0dbbfbc56c P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 2 election: Requested vote from peers 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823), e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469)
I20260709 13:21:30.585222 22526 raft_consensus.cc:3055] T 2d4c5545efd2494a81249b0dbbfbc56c P 7df364d160d64ac4b5cdc4f004324f8b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:30.585615 22600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d4c5545efd2494a81249b0dbbfbc56c" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e20f6c83ae144d698c26b94c2eda023f"
I20260709 13:21:30.586314 22600 raft_consensus.cc:3060] T 2d4c5545efd2494a81249b0dbbfbc56c P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:30.585668 22526 raft_consensus.cc:740] T 2d4c5545efd2494a81249b0dbbfbc56c P 7df364d160d64ac4b5cdc4f004324f8b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7df364d160d64ac4b5cdc4f004324f8b, State: Running, Role: LEADER
I20260709 13:21:30.586356 22602 raft_consensus.cc:1275] T ef14a8f3b6344312bd294edcabb2f312 P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Refusing update from remote peer 7df364d160d64ac4b5cdc4f004324f8b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:30.588068 22526 consensus_queue.cc:260] T 2d4c5545efd2494a81249b0dbbfbc56c P 7df364d160d64ac4b5cdc4f004324f8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:30.587857 22423 catalog_manager.cc:5697] T 47e33cdb6a154e8781b3e36a95eed353 P 7df364d160d64ac4b5cdc4f004324f8b reported cstate change: term changed from 1 to 2, leader changed from e20f6c83ae144d698c26b94c2eda023f (127.9.116.130) to 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129). New cstate: current_term: 2 leader_uuid: "7df364d160d64ac4b5cdc4f004324f8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:30.588358 22860 consensus_queue.cc:1048] T ef14a8f3b6344312bd294edcabb2f312 P 7df364d160d64ac4b5cdc4f004324f8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:30.589553 22526 raft_consensus.cc:3060] T 2d4c5545efd2494a81249b0dbbfbc56c P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:30.596001 22600 raft_consensus.cc:2468] T 2d4c5545efd2494a81249b0dbbfbc56c P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d62e0b282b44003a202b8dcbe6ea6ac in term 2.
I20260709 13:21:30.597860 22640 leader_election.cc:304] T 2d4c5545efd2494a81249b0dbbfbc56c P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d62e0b282b44003a202b8dcbe6ea6ac, e20f6c83ae144d698c26b94c2eda023f; no voters: 
I20260709 13:21:30.598039 22526 raft_consensus.cc:2468] T 2d4c5545efd2494a81249b0dbbfbc56c P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d62e0b282b44003a202b8dcbe6ea6ac in term 2.
I20260709 13:21:30.598829 22752 raft_consensus.cc:2804] T 2d4c5545efd2494a81249b0dbbfbc56c P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:30.599336 22752 raft_consensus.cc:697] T 2d4c5545efd2494a81249b0dbbfbc56c P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 LEADER]: Becoming Leader. State: Replica: 2d62e0b282b44003a202b8dcbe6ea6ac, State: Running, Role: LEADER
I20260709 13:21:30.600356 22752 consensus_queue.cc:237] T 2d4c5545efd2494a81249b0dbbfbc56c P 2d62e0b282b44003a202b8dcbe6ea6ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:30.609294 22675 raft_consensus.cc:1275] T ef14a8f3b6344312bd294edcabb2f312 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Refusing update from remote peer 7df364d160d64ac4b5cdc4f004324f8b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:30.611536 22860 consensus_queue.cc:1048] T ef14a8f3b6344312bd294edcabb2f312 P 7df364d160d64ac4b5cdc4f004324f8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:30.612143 22422 catalog_manager.cc:5697] T 2d4c5545efd2494a81249b0dbbfbc56c P 2d62e0b282b44003a202b8dcbe6ea6ac reported cstate change: term changed from 1 to 2, leader changed from 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129) to 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131). New cstate: current_term: 2 leader_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:30.640539 22602 raft_consensus.cc:3060] T 5b97872c6f3b4a288ff8ade4569da6f1 P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:30.645725 22602 raft_consensus.cc:1275] T 5b97872c6f3b4a288ff8ade4569da6f1 P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Refusing update from remote peer 7df364d160d64ac4b5cdc4f004324f8b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:30.647101 22860 consensus_queue.cc:1048] T 5b97872c6f3b4a288ff8ade4569da6f1 P 7df364d160d64ac4b5cdc4f004324f8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:30.660316 22675 raft_consensus.cc:1275] T 5b97872c6f3b4a288ff8ade4569da6f1 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Refusing update from remote peer 7df364d160d64ac4b5cdc4f004324f8b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:30.661789 22854 consensus_queue.cc:1048] T 5b97872c6f3b4a288ff8ade4569da6f1 P 7df364d160d64ac4b5cdc4f004324f8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:30.664796 22526 raft_consensus.cc:1275] T e1e94cad244443feb24f679271460852 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Refusing update from remote peer e20f6c83ae144d698c26b94c2eda023f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:30.666513 22870 consensus_queue.cc:1048] T e1e94cad244443feb24f679271460852 P e20f6c83ae144d698c26b94c2eda023f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:30.682617 22673 raft_consensus.cc:1275] T e1e94cad244443feb24f679271460852 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Refusing update from remote peer e20f6c83ae144d698c26b94c2eda023f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:30.685099 22811 consensus_queue.cc:1048] T e1e94cad244443feb24f679271460852 P e20f6c83ae144d698c26b94c2eda023f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:30.914006 22602 raft_consensus.cc:1275] T 0f952d6650154a4a98f94a75b916d44b P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Refusing update from remote peer 7df364d160d64ac4b5cdc4f004324f8b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:30.915295 22855 consensus_queue.cc:1048] T 0f952d6650154a4a98f94a75b916d44b P 7df364d160d64ac4b5cdc4f004324f8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:30.926046 22675 raft_consensus.cc:1275] T 0f952d6650154a4a98f94a75b916d44b P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Refusing update from remote peer 7df364d160d64ac4b5cdc4f004324f8b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:30.927618 22855 consensus_queue.cc:1048] T 0f952d6650154a4a98f94a75b916d44b P 7df364d160d64ac4b5cdc4f004324f8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:30.958658 22526 raft_consensus.cc:1275] T a3b0925412bd4dceabec4f898eb43973 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Refusing update from remote peer e20f6c83ae144d698c26b94c2eda023f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:30.960424 22847 consensus_queue.cc:1048] T a3b0925412bd4dceabec4f898eb43973 P e20f6c83ae144d698c26b94c2eda023f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:21:30.972445 22675 raft_consensus.cc:1275] T a3b0925412bd4dceabec4f898eb43973 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Refusing update from remote peer e20f6c83ae144d698c26b94c2eda023f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:30.973897 22847 consensus_queue.cc:1048] T a3b0925412bd4dceabec4f898eb43973 P e20f6c83ae144d698c26b94c2eda023f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:31.001509 22526 raft_consensus.cc:1275] T 2d4c5545efd2494a81249b0dbbfbc56c P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Refusing update from remote peer 2d62e0b282b44003a202b8dcbe6ea6ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:31.003095 22752 consensus_queue.cc:1048] T 2d4c5545efd2494a81249b0dbbfbc56c P 2d62e0b282b44003a202b8dcbe6ea6ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:31.012223 22602 raft_consensus.cc:1275] T 2d4c5545efd2494a81249b0dbbfbc56c P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Refusing update from remote peer 2d62e0b282b44003a202b8dcbe6ea6ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:31.015595 22752 consensus_queue.cc:1048] T 2d4c5545efd2494a81249b0dbbfbc56c P 2d62e0b282b44003a202b8dcbe6ea6ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:31.033479 22602 raft_consensus.cc:1275] T 47e33cdb6a154e8781b3e36a95eed353 P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Refusing update from remote peer 7df364d160d64ac4b5cdc4f004324f8b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:31.034960 22855 consensus_queue.cc:1048] T 47e33cdb6a154e8781b3e36a95eed353 P 7df364d160d64ac4b5cdc4f004324f8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:31.037704 22675 raft_consensus.cc:1275] T 2f75c1bbbd7b4dc09412c016798ccd50 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Refusing update from remote peer e20f6c83ae144d698c26b94c2eda023f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:31.039340 22870 consensus_queue.cc:1048] T 2f75c1bbbd7b4dc09412c016798ccd50 P e20f6c83ae144d698c26b94c2eda023f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:31.046406 22673 raft_consensus.cc:1275] T 47e33cdb6a154e8781b3e36a95eed353 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Refusing update from remote peer 7df364d160d64ac4b5cdc4f004324f8b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:31.048404 22836 consensus_queue.cc:1048] T 47e33cdb6a154e8781b3e36a95eed353 P 7df364d160d64ac4b5cdc4f004324f8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:21:31.055097 22526 raft_consensus.cc:1275] T 2f75c1bbbd7b4dc09412c016798ccd50 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Refusing update from remote peer e20f6c83ae144d698c26b94c2eda023f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:31.061975 22847 consensus_queue.cc:1048] T 2f75c1bbbd7b4dc09412c016798ccd50 P e20f6c83ae144d698c26b94c2eda023f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:33.117096 22423 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44484:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:21:33.137068 22423 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44484:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:21:33.217860  9682 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:21:33.371518 22917 log.cc:927] Time spent T 47e33cdb6a154e8781b3e36a95eed353 P 7df364d160d64ac4b5cdc4f004324f8b: Append to log took a long time: real 0.082s	user 0.002s	sys 0.000s
W20260709 13:21:33.439886 22914 log.cc:927] Time spent T 2f75c1bbbd7b4dc09412c016798ccd50 P e20f6c83ae144d698c26b94c2eda023f: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:21:33.441095 22950 tablet.cc:1724] T 93dd95f0dfb8463291a74921c46665dd P e20f6c83ae144d698c26b94c2eda023f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:33.462167 22955 tablet.cc:1724] T 5b97872c6f3b4a288ff8ade4569da6f1 P 7df364d160d64ac4b5cdc4f004324f8b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:33.529495 22959 tablet.cc:1724] T 2d4c5545efd2494a81249b0dbbfbc56c P 2d62e0b282b44003a202b8dcbe6ea6ac: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:33.529800 22957 tablet.cc:1724] T 93dd95f0dfb8463291a74921c46665dd P 2d62e0b282b44003a202b8dcbe6ea6ac: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:33.540380 22960 tablet.cc:1724] T 93dd95f0dfb8463291a74921c46665dd P 7df364d160d64ac4b5cdc4f004324f8b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:21:33.531543 22944 log.cc:927] Time spent T 2d4c5545efd2494a81249b0dbbfbc56c P 7df364d160d64ac4b5cdc4f004324f8b: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
W20260709 13:21:33.567210 22491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.130:43469, user_credentials={real_user=slave}} blocked reactor thread for 65123us
W20260709 13:21:33.588059 22954 log.cc:927] Time spent T 47e33cdb6a154e8781b3e36a95eed353 P e20f6c83ae144d698c26b94c2eda023f: Append to log took a long time: real 0.073s	user 0.004s	sys 0.000s
I20260709 13:21:33.615650 22964 tablet.cc:1724] T 2d4c5545efd2494a81249b0dbbfbc56c P e20f6c83ae144d698c26b94c2eda023f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:33.616235 22966 tablet.cc:1724] T 0f952d6650154a4a98f94a75b916d44b P 2d62e0b282b44003a202b8dcbe6ea6ac: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:33.618517 22525 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "56bc1b5c8c3143998cb9921b469c071d"
dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b"
mode: GRACEFUL
new_leader_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac"
 from {username='slave'} at 127.0.0.1:51132
I20260709 13:21:33.617698 22963 tablet.cc:1724] T 0f952d6650154a4a98f94a75b916d44b P 7df364d160d64ac4b5cdc4f004324f8b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:33.619311 22525 raft_consensus.cc:606] T 56bc1b5c8c3143998cb9921b469c071d P 7df364d160d64ac4b5cdc4f004324f8b [term 2 LEADER]: Received request to transfer leadership to TS 2d62e0b282b44003a202b8dcbe6ea6ac
I20260709 13:21:33.599478 22961 tablet.cc:1724] T a3b0925412bd4dceabec4f898eb43973 P e20f6c83ae144d698c26b94c2eda023f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:33.646090 22967 tablet.cc:1724] T a3b0925412bd4dceabec4f898eb43973 P 2d62e0b282b44003a202b8dcbe6ea6ac: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:33.649544 22959 tablet.cc:1724] T 5b97872c6f3b4a288ff8ade4569da6f1 P 2d62e0b282b44003a202b8dcbe6ea6ac: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:33.646535 22957 tablet.cc:1724] T 2f75c1bbbd7b4dc09412c016798ccd50 P 2d62e0b282b44003a202b8dcbe6ea6ac: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:33.679805 22968 tablet.cc:1724] T 5b97872c6f3b4a288ff8ade4569da6f1 P e20f6c83ae144d698c26b94c2eda023f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:33.681412 22972 tablet.cc:1724] T e1e94cad244443feb24f679271460852 P 7df364d160d64ac4b5cdc4f004324f8b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:33.687886 22960 tablet.cc:1724] T a3b0925412bd4dceabec4f898eb43973 P 7df364d160d64ac4b5cdc4f004324f8b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:33.651239 22950 tablet.cc:1724] T 2f75c1bbbd7b4dc09412c016798ccd50 P e20f6c83ae144d698c26b94c2eda023f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:33.701581 22955 tablet.cc:1724] T 47e33cdb6a154e8781b3e36a95eed353 P 7df364d160d64ac4b5cdc4f004324f8b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:33.702739 22964 tablet.cc:1724] T 0f952d6650154a4a98f94a75b916d44b P e20f6c83ae144d698c26b94c2eda023f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:33.714046 22968 tablet.cc:1724] T 47e33cdb6a154e8781b3e36a95eed353 P e20f6c83ae144d698c26b94c2eda023f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:33.720217 22600 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5e4995abf28446589b7695a1b77ee513"
dest_uuid: "e20f6c83ae144d698c26b94c2eda023f"
mode: GRACEFUL
new_leader_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac"
 from {username='slave'} at 127.0.0.1:53616
I20260709 13:21:33.720903 22600 raft_consensus.cc:606] T 5e4995abf28446589b7695a1b77ee513 P e20f6c83ae144d698c26b94c2eda023f [term 2 LEADER]: Received request to transfer leadership to TS 2d62e0b282b44003a202b8dcbe6ea6ac
I20260709 13:21:33.722683  9682 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:21:33.723362  9682 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:21:33.706930 22961 tablet.cc:1724] T e1e94cad244443feb24f679271460852 P e20f6c83ae144d698c26b94c2eda023f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:33.720911 22959 tablet.cc:1724] T e1e94cad244443feb24f679271460852 P 2d62e0b282b44003a202b8dcbe6ea6ac: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:21:33.730909 22493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.116.131:34335, user_credentials={real_user=slave}} blocked reactor thread for 235404us
I20260709 13:21:33.740919 22960 tablet.cc:1724] T 2f75c1bbbd7b4dc09412c016798ccd50 P 7df364d160d64ac4b5cdc4f004324f8b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:33.741551 22950 tablet.cc:1724] T ef14a8f3b6344312bd294edcabb2f312 P e20f6c83ae144d698c26b94c2eda023f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:33.759702 22963 tablet.cc:1724] T 2d4c5545efd2494a81249b0dbbfbc56c P 7df364d160d64ac4b5cdc4f004324f8b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:33.761896 22972 tablet.cc:1724] T ef14a8f3b6344312bd294edcabb2f312 P 7df364d160d64ac4b5cdc4f004324f8b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:33.773491 22966 tablet.cc:1724] T ef14a8f3b6344312bd294edcabb2f312 P 2d62e0b282b44003a202b8dcbe6ea6ac: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:33.779243 22959 tablet.cc:1724] T 47e33cdb6a154e8781b3e36a95eed353 P 2d62e0b282b44003a202b8dcbe6ea6ac: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:21:33.798074 22901 raft_consensus.cc:993] T 56bc1b5c8c3143998cb9921b469c071d P 7df364d160d64ac4b5cdc4f004324f8b: : Instructing follower 2d62e0b282b44003a202b8dcbe6ea6ac to start an election
I20260709 13:21:33.798695 22901 raft_consensus.cc:1081] T 56bc1b5c8c3143998cb9921b469c071d P 7df364d160d64ac4b5cdc4f004324f8b [term 2 LEADER]: Signalling peer 2d62e0b282b44003a202b8dcbe6ea6ac to start an election
I20260709 13:21:33.807184 22669 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "56bc1b5c8c3143998cb9921b469c071d"
dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac"
 from {username='slave'} at 127.0.0.1:48174
I20260709 13:21:33.807828 22669 raft_consensus.cc:493] T 56bc1b5c8c3143998cb9921b469c071d P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:33.808202 22669 raft_consensus.cc:3060] T 56bc1b5c8c3143998cb9921b469c071d P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:33.816612 22669 raft_consensus.cc:515] T 56bc1b5c8c3143998cb9921b469c071d P 2d62e0b282b44003a202b8dcbe6ea6ac [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:33.819839 22523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56bc1b5c8c3143998cb9921b469c071d" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b"
I20260709 13:21:33.820554 22523 raft_consensus.cc:3055] T 56bc1b5c8c3143998cb9921b469c071d P 7df364d160d64ac4b5cdc4f004324f8b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:21:33.820917 22523 raft_consensus.cc:740] T 56bc1b5c8c3143998cb9921b469c071d P 7df364d160d64ac4b5cdc4f004324f8b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7df364d160d64ac4b5cdc4f004324f8b, State: Running, Role: LEADER
I20260709 13:21:33.821874 22523 consensus_queue.cc:260] T 56bc1b5c8c3143998cb9921b469c071d P 7df364d160d64ac4b5cdc4f004324f8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:33.823282 22523 raft_consensus.cc:3060] T 56bc1b5c8c3143998cb9921b469c071d P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:33.824170 22600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56bc1b5c8c3143998cb9921b469c071d" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e20f6c83ae144d698c26b94c2eda023f"
I20260709 13:21:33.824743 22600 raft_consensus.cc:3060] T 56bc1b5c8c3143998cb9921b469c071d P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:33.829077 22669 leader_election.cc:290] T 56bc1b5c8c3143998cb9921b469c071d P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 3 election: Requested vote from peers 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823), e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469)
I20260709 13:21:33.833060 22600 raft_consensus.cc:2468] T 56bc1b5c8c3143998cb9921b469c071d P e20f6c83ae144d698c26b94c2eda023f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d62e0b282b44003a202b8dcbe6ea6ac in term 3.
I20260709 13:21:33.834596 22640 leader_election.cc:304] T 56bc1b5c8c3143998cb9921b469c071d P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d62e0b282b44003a202b8dcbe6ea6ac, e20f6c83ae144d698c26b94c2eda023f; no voters: 
I20260709 13:21:33.835608 22937 raft_consensus.cc:2804] T 56bc1b5c8c3143998cb9921b469c071d P 2d62e0b282b44003a202b8dcbe6ea6ac [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:21:33.836088 22523 raft_consensus.cc:2468] T 56bc1b5c8c3143998cb9921b469c071d P 7df364d160d64ac4b5cdc4f004324f8b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d62e0b282b44003a202b8dcbe6ea6ac in term 3.
I20260709 13:21:33.836122 22937 raft_consensus.cc:697] T 56bc1b5c8c3143998cb9921b469c071d P 2d62e0b282b44003a202b8dcbe6ea6ac [term 3 LEADER]: Becoming Leader. State: Replica: 2d62e0b282b44003a202b8dcbe6ea6ac, State: Running, Role: LEADER
I20260709 13:21:33.837503 22937 consensus_queue.cc:237] T 56bc1b5c8c3143998cb9921b469c071d P 2d62e0b282b44003a202b8dcbe6ea6ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:33.852015 22423 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 542aff268f9b4b3abea229f5c457ad3c: soft_deleted_table [id=0d2ccd2882d348079d7ec3e42d5bdf7f] alter complete (version 1)
I20260709 13:21:33.861670 22423 catalog_manager.cc:5697] T 56bc1b5c8c3143998cb9921b469c071d P 2d62e0b282b44003a202b8dcbe6ea6ac reported cstate change: term changed from 2 to 3, leader changed from 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129) to 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131). New cstate: current_term: 3 leader_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:33.967262 22947 raft_consensus.cc:993] T 5e4995abf28446589b7695a1b77ee513 P e20f6c83ae144d698c26b94c2eda023f: : Instructing follower 2d62e0b282b44003a202b8dcbe6ea6ac to start an election
I20260709 13:21:33.967965 22947 raft_consensus.cc:1081] T 5e4995abf28446589b7695a1b77ee513 P e20f6c83ae144d698c26b94c2eda023f [term 2 LEADER]: Signalling peer 2d62e0b282b44003a202b8dcbe6ea6ac to start an election
I20260709 13:21:33.969527 22669 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5e4995abf28446589b7695a1b77ee513"
dest_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac"
 from {username='slave'} at 127.0.0.1:48178
I20260709 13:21:33.970324 22669 raft_consensus.cc:493] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:33.970827 22669 raft_consensus.cc:3060] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:33.979600 22669 raft_consensus.cc:515] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:33.982023 22669 leader_election.cc:290] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 3 election: Requested vote from peers 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823), e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469)
I20260709 13:21:33.982957 22523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e4995abf28446589b7695a1b77ee513" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7df364d160d64ac4b5cdc4f004324f8b"
I20260709 13:21:33.983695 22523 raft_consensus.cc:3060] T 5e4995abf28446589b7695a1b77ee513 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:33.987262 22600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e4995abf28446589b7695a1b77ee513" candidate_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e20f6c83ae144d698c26b94c2eda023f"
I20260709 13:21:33.988443 22600 raft_consensus.cc:3055] T 5e4995abf28446589b7695a1b77ee513 P e20f6c83ae144d698c26b94c2eda023f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:21:33.989055 22600 raft_consensus.cc:740] T 5e4995abf28446589b7695a1b77ee513 P e20f6c83ae144d698c26b94c2eda023f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e20f6c83ae144d698c26b94c2eda023f, State: Running, Role: LEADER
I20260709 13:21:33.991930 22600 consensus_queue.cc:260] T 5e4995abf28446589b7695a1b77ee513 P e20f6c83ae144d698c26b94c2eda023f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:33.993482 22523 raft_consensus.cc:2468] T 5e4995abf28446589b7695a1b77ee513 P 7df364d160d64ac4b5cdc4f004324f8b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d62e0b282b44003a202b8dcbe6ea6ac in term 3.
I20260709 13:21:33.994410 22600 raft_consensus.cc:3060] T 5e4995abf28446589b7695a1b77ee513 P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:33.994930 22642 leader_election.cc:304] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d62e0b282b44003a202b8dcbe6ea6ac, 7df364d160d64ac4b5cdc4f004324f8b; no voters: 
I20260709 13:21:33.995741 22937 raft_consensus.cc:2804] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:21:33.996181 22937 raft_consensus.cc:697] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 3 LEADER]: Becoming Leader. State: Replica: 2d62e0b282b44003a202b8dcbe6ea6ac, State: Running, Role: LEADER
I20260709 13:21:33.997033 22937 consensus_queue.cc:237] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } }
I20260709 13:21:34.003132 22600 raft_consensus.cc:2468] T 5e4995abf28446589b7695a1b77ee513 P e20f6c83ae144d698c26b94c2eda023f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d62e0b282b44003a202b8dcbe6ea6ac in term 3.
I20260709 13:21:34.008987 22424 catalog_manager.cc:5697] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac reported cstate change: term changed from 2 to 3, leader changed from e20f6c83ae144d698c26b94c2eda023f (127.9.116.130) to 2d62e0b282b44003a202b8dcbe6ea6ac (127.9.116.131). New cstate: current_term: 3 leader_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d62e0b282b44003a202b8dcbe6ea6ac" member_type: VOTER last_known_addr { host: "127.9.116.131" port: 34335 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:34.272550 22600 raft_consensus.cc:1275] T 56bc1b5c8c3143998cb9921b469c071d P e20f6c83ae144d698c26b94c2eda023f [term 3 FOLLOWER]: Refusing update from remote peer 2d62e0b282b44003a202b8dcbe6ea6ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:21:34.273690 22937 consensus_queue.cc:1048] T 56bc1b5c8c3143998cb9921b469c071d P 2d62e0b282b44003a202b8dcbe6ea6ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:21:34.285060 22523 raft_consensus.cc:1275] T 56bc1b5c8c3143998cb9921b469c071d P 7df364d160d64ac4b5cdc4f004324f8b [term 3 FOLLOWER]: Refusing update from remote peer 2d62e0b282b44003a202b8dcbe6ea6ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:21:34.286451 22752 consensus_queue.cc:1048] T 56bc1b5c8c3143998cb9921b469c071d P 2d62e0b282b44003a202b8dcbe6ea6ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:21:34.392609 22600 raft_consensus.cc:1275] T 5e4995abf28446589b7695a1b77ee513 P e20f6c83ae144d698c26b94c2eda023f [term 3 FOLLOWER]: Refusing update from remote peer 2d62e0b282b44003a202b8dcbe6ea6ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:21:34.393818 22752 consensus_queue.cc:1048] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "e20f6c83ae144d698c26b94c2eda023f" member_type: VOTER last_known_addr { host: "127.9.116.130" port: 43469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:21:34.402642 22523 raft_consensus.cc:1275] T 5e4995abf28446589b7695a1b77ee513 P 7df364d160d64ac4b5cdc4f004324f8b [term 3 FOLLOWER]: Refusing update from remote peer 2d62e0b282b44003a202b8dcbe6ea6ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:21:34.405517 22985 consensus_queue.cc:1048] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "7df364d160d64ac4b5cdc4f004324f8b" member_type: VOTER last_known_addr { host: "127.9.116.129" port: 42823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:21:34.782291  9682 tablet_server.cc:179] TabletServer@127.9.116.129:0 shutting down...
I20260709 13:21:34.810070  9682 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:21:34.810953  9682 tablet_replica.cc:333] T 2d4c5545efd2494a81249b0dbbfbc56c P 7df364d160d64ac4b5cdc4f004324f8b: stopping tablet replica
I20260709 13:21:34.811672  9682 raft_consensus.cc:2243] T 2d4c5545efd2494a81249b0dbbfbc56c P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:34.812186  9682 raft_consensus.cc:2272] T 2d4c5545efd2494a81249b0dbbfbc56c P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.815256  9682 tablet_replica.cc:333] T 5b97872c6f3b4a288ff8ade4569da6f1 P 7df364d160d64ac4b5cdc4f004324f8b: stopping tablet replica
I20260709 13:21:34.816154  9682 raft_consensus.cc:2243] T 5b97872c6f3b4a288ff8ade4569da6f1 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:34.817034  9682 raft_consensus.cc:2272] T 5b97872c6f3b4a288ff8ade4569da6f1 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.819502  9682 tablet_replica.cc:333] T 2f75c1bbbd7b4dc09412c016798ccd50 P 7df364d160d64ac4b5cdc4f004324f8b: stopping tablet replica
I20260709 13:21:34.820115  9682 raft_consensus.cc:2243] T 2f75c1bbbd7b4dc09412c016798ccd50 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:34.820564  9682 raft_consensus.cc:2272] T 2f75c1bbbd7b4dc09412c016798ccd50 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.822561  9682 tablet_replica.cc:333] T ef14a8f3b6344312bd294edcabb2f312 P 7df364d160d64ac4b5cdc4f004324f8b: stopping tablet replica
I20260709 13:21:34.823220  9682 raft_consensus.cc:2243] T ef14a8f3b6344312bd294edcabb2f312 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:34.824012  9682 raft_consensus.cc:2272] T ef14a8f3b6344312bd294edcabb2f312 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.825976  9682 tablet_replica.cc:333] T e1e94cad244443feb24f679271460852 P 7df364d160d64ac4b5cdc4f004324f8b: stopping tablet replica
I20260709 13:21:34.826516  9682 raft_consensus.cc:2243] T e1e94cad244443feb24f679271460852 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:34.827018  9682 raft_consensus.cc:2272] T e1e94cad244443feb24f679271460852 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.828958  9682 tablet_replica.cc:333] T 0f952d6650154a4a98f94a75b916d44b P 7df364d160d64ac4b5cdc4f004324f8b: stopping tablet replica
I20260709 13:21:34.829494  9682 raft_consensus.cc:2243] T 0f952d6650154a4a98f94a75b916d44b P 7df364d160d64ac4b5cdc4f004324f8b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:34.830225  9682 raft_consensus.cc:2272] T 0f952d6650154a4a98f94a75b916d44b P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.832315  9682 tablet_replica.cc:333] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 7df364d160d64ac4b5cdc4f004324f8b: stopping tablet replica
I20260709 13:21:34.832921  9682 raft_consensus.cc:2243] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:34.833391  9682 raft_consensus.cc:2272] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.835597  9682 tablet_replica.cc:333] T 68d74bd4ad624659b92ec79d4560a02c P 7df364d160d64ac4b5cdc4f004324f8b: stopping tablet replica
I20260709 13:21:34.836155  9682 raft_consensus.cc:2243] T 68d74bd4ad624659b92ec79d4560a02c P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:34.836630  9682 raft_consensus.cc:2272] T 68d74bd4ad624659b92ec79d4560a02c P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.838914  9682 tablet_replica.cc:333] T 47e33cdb6a154e8781b3e36a95eed353 P 7df364d160d64ac4b5cdc4f004324f8b: stopping tablet replica
I20260709 13:21:34.839521  9682 raft_consensus.cc:2243] T 47e33cdb6a154e8781b3e36a95eed353 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:34.840314  9682 raft_consensus.cc:2272] T 47e33cdb6a154e8781b3e36a95eed353 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.842466  9682 tablet_replica.cc:333] T 93dd95f0dfb8463291a74921c46665dd P 7df364d160d64ac4b5cdc4f004324f8b: stopping tablet replica
I20260709 13:21:34.843082  9682 raft_consensus.cc:2243] T 93dd95f0dfb8463291a74921c46665dd P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:34.843554  9682 raft_consensus.cc:2272] T 93dd95f0dfb8463291a74921c46665dd P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.845695  9682 tablet_replica.cc:333] T 5a1c542f0ae74a718e0b1c70aa641da4 P 7df364d160d64ac4b5cdc4f004324f8b: stopping tablet replica
I20260709 13:21:34.846504  9682 raft_consensus.cc:2243] T 5a1c542f0ae74a718e0b1c70aa641da4 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:34.847144  9682 raft_consensus.cc:2272] T 5a1c542f0ae74a718e0b1c70aa641da4 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:34.849086 22642 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 9 similar messages]
I20260709 13:21:34.850381  9682 tablet_replica.cc:333] T 2dad40e2d06244cebe50ec7433000752 P 7df364d160d64ac4b5cdc4f004324f8b: stopping tablet replica
I20260709 13:21:34.851369  9682 raft_consensus.cc:2243] T 2dad40e2d06244cebe50ec7433000752 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:34.852016  9682 raft_consensus.cc:2272] T 2dad40e2d06244cebe50ec7433000752 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:34.852710 22642 consensus_peers.cc:597] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac -> Peer 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823): Couldn't send request to peer 7df364d160d64ac4b5cdc4f004324f8b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:34.855067  9682 tablet_replica.cc:333] T c0614d551fc342cead195ccaed2aeac3 P 7df364d160d64ac4b5cdc4f004324f8b: stopping tablet replica
I20260709 13:21:34.855971  9682 raft_consensus.cc:2243] T c0614d551fc342cead195ccaed2aeac3 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:34.857206  9682 raft_consensus.cc:2272] T c0614d551fc342cead195ccaed2aeac3 P 7df364d160d64ac4b5cdc4f004324f8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.860157  9682 tablet_replica.cc:333] T 5e4995abf28446589b7695a1b77ee513 P 7df364d160d64ac4b5cdc4f004324f8b: stopping tablet replica
I20260709 13:21:34.860764  9682 raft_consensus.cc:2243] T 5e4995abf28446589b7695a1b77ee513 P 7df364d160d64ac4b5cdc4f004324f8b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:34.861248  9682 raft_consensus.cc:2272] T 5e4995abf28446589b7695a1b77ee513 P 7df364d160d64ac4b5cdc4f004324f8b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.864233  9682 tablet_replica.cc:333] T dded1824a3ef49988cc1b941db0f9955 P 7df364d160d64ac4b5cdc4f004324f8b: stopping tablet replica
I20260709 13:21:34.864902  9682 raft_consensus.cc:2243] T dded1824a3ef49988cc1b941db0f9955 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:34.865643  9682 raft_consensus.cc:2272] T dded1824a3ef49988cc1b941db0f9955 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.867589  9682 tablet_replica.cc:333] T a3b0925412bd4dceabec4f898eb43973 P 7df364d160d64ac4b5cdc4f004324f8b: stopping tablet replica
I20260709 13:21:34.868016  9682 raft_consensus.cc:2243] T a3b0925412bd4dceabec4f898eb43973 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:34.868407  9682 raft_consensus.cc:2272] T a3b0925412bd4dceabec4f898eb43973 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.870343  9682 tablet_replica.cc:333] T 2e322f5288b64ccca277807d2dc46579 P 7df364d160d64ac4b5cdc4f004324f8b: stopping tablet replica
I20260709 13:21:34.870769  9682 raft_consensus.cc:2243] T 2e322f5288b64ccca277807d2dc46579 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:34.871531  9682 raft_consensus.cc:2272] T 2e322f5288b64ccca277807d2dc46579 P 7df364d160d64ac4b5cdc4f004324f8b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.873466  9682 tablet_replica.cc:333] T 56bc1b5c8c3143998cb9921b469c071d P 7df364d160d64ac4b5cdc4f004324f8b: stopping tablet replica
I20260709 13:21:34.873869  9682 raft_consensus.cc:2243] T 56bc1b5c8c3143998cb9921b469c071d P 7df364d160d64ac4b5cdc4f004324f8b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:34.874212  9682 raft_consensus.cc:2272] T 56bc1b5c8c3143998cb9921b469c071d P 7df364d160d64ac4b5cdc4f004324f8b [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:34.900408 22642 connection.cc:573] client connection to 127.9.116.129:42823 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260709 13:21:34.901094 22642 consensus_peers.cc:597] T 56bc1b5c8c3143998cb9921b469c071d P 2d62e0b282b44003a202b8dcbe6ea6ac -> Peer 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823): Couldn't send request to peer 7df364d160d64ac4b5cdc4f004324f8b. Status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:34.916916  9682 tablet_server.cc:196] TabletServer@127.9.116.129:0 shutdown complete.
W20260709 13:21:34.935287 22568 consensus_peers.cc:597] T 2f75c1bbbd7b4dc09412c016798ccd50 P e20f6c83ae144d698c26b94c2eda023f -> Peer 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823): Couldn't send request to peer 7df364d160d64ac4b5cdc4f004324f8b. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:42823: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:34.956818  9682 tablet_server.cc:179] TabletServer@127.9.116.130:0 shutting down...
W20260709 13:21:34.983577 22640 consensus_peers.cc:597] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac -> Peer e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469): Couldn't send request to peer e20f6c83ae144d698c26b94c2eda023f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:34.988677 22568 consensus_peers.cc:597] T 68d74bd4ad624659b92ec79d4560a02c P e20f6c83ae144d698c26b94c2eda023f -> Peer 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823): Couldn't send request to peer 7df364d160d64ac4b5cdc4f004324f8b. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:42823: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:35.003257  9682 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:21:35.004281  9682 tablet_replica.cc:333] T 2d4c5545efd2494a81249b0dbbfbc56c P e20f6c83ae144d698c26b94c2eda023f: stopping tablet replica
I20260709 13:21:35.005400  9682 raft_consensus.cc:2243] T 2d4c5545efd2494a81249b0dbbfbc56c P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:35.006111  9682 raft_consensus.cc:2272] T 2d4c5545efd2494a81249b0dbbfbc56c P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.009305  9682 tablet_replica.cc:333] T 5b97872c6f3b4a288ff8ade4569da6f1 P e20f6c83ae144d698c26b94c2eda023f: stopping tablet replica
I20260709 13:21:35.010211  9682 raft_consensus.cc:2243] T 5b97872c6f3b4a288ff8ade4569da6f1 P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:35.010857  9682 raft_consensus.cc:2272] T 5b97872c6f3b4a288ff8ade4569da6f1 P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.013747  9682 tablet_replica.cc:333] T e1e94cad244443feb24f679271460852 P e20f6c83ae144d698c26b94c2eda023f: stopping tablet replica
I20260709 13:21:35.014617  9682 raft_consensus.cc:2243] T e1e94cad244443feb24f679271460852 P e20f6c83ae144d698c26b94c2eda023f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:35.015741  9682 raft_consensus.cc:2272] T e1e94cad244443feb24f679271460852 P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.018822  9682 tablet_replica.cc:333] T 2f75c1bbbd7b4dc09412c016798ccd50 P e20f6c83ae144d698c26b94c2eda023f: stopping tablet replica
I20260709 13:21:35.019783  9682 raft_consensus.cc:2243] T 2f75c1bbbd7b4dc09412c016798ccd50 P e20f6c83ae144d698c26b94c2eda023f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:35.020890  9682 raft_consensus.cc:2272] T 2f75c1bbbd7b4dc09412c016798ccd50 P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.024188  9682 tablet_replica.cc:333] T ef14a8f3b6344312bd294edcabb2f312 P e20f6c83ae144d698c26b94c2eda023f: stopping tablet replica
I20260709 13:21:35.025117  9682 raft_consensus.cc:2243] T ef14a8f3b6344312bd294edcabb2f312 P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:35.025738  9682 raft_consensus.cc:2272] T ef14a8f3b6344312bd294edcabb2f312 P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.028790  9682 tablet_replica.cc:333] T 0f952d6650154a4a98f94a75b916d44b P e20f6c83ae144d698c26b94c2eda023f: stopping tablet replica
I20260709 13:21:35.029670  9682 raft_consensus.cc:2243] T 0f952d6650154a4a98f94a75b916d44b P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:35.030333  9682 raft_consensus.cc:2272] T 0f952d6650154a4a98f94a75b916d44b P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.033538  9682 tablet_replica.cc:333] T 68d74bd4ad624659b92ec79d4560a02c P e20f6c83ae144d698c26b94c2eda023f: stopping tablet replica
I20260709 13:21:35.034427  9682 raft_consensus.cc:2243] T 68d74bd4ad624659b92ec79d4560a02c P e20f6c83ae144d698c26b94c2eda023f [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:35.035598  9682 raft_consensus.cc:2272] T 68d74bd4ad624659b92ec79d4560a02c P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.038596  9682 tablet_replica.cc:333] T a9c5f91cbe6f4a6abbd799e94e0985e1 P e20f6c83ae144d698c26b94c2eda023f: stopping tablet replica
I20260709 13:21:35.039469  9682 raft_consensus.cc:2243] T a9c5f91cbe6f4a6abbd799e94e0985e1 P e20f6c83ae144d698c26b94c2eda023f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:35.040674  9682 raft_consensus.cc:2272] T a9c5f91cbe6f4a6abbd799e94e0985e1 P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.043823  9682 tablet_replica.cc:333] T 5e4995abf28446589b7695a1b77ee513 P e20f6c83ae144d698c26b94c2eda023f: stopping tablet replica
I20260709 13:21:35.044673  9682 raft_consensus.cc:2243] T 5e4995abf28446589b7695a1b77ee513 P e20f6c83ae144d698c26b94c2eda023f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:35.045264  9682 raft_consensus.cc:2272] T 5e4995abf28446589b7695a1b77ee513 P e20f6c83ae144d698c26b94c2eda023f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.048967  9682 tablet_replica.cc:333] T dded1824a3ef49988cc1b941db0f9955 P e20f6c83ae144d698c26b94c2eda023f: stopping tablet replica
I20260709 13:21:35.049599  9682 raft_consensus.cc:2243] T dded1824a3ef49988cc1b941db0f9955 P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:35.050204  9682 raft_consensus.cc:2272] T dded1824a3ef49988cc1b941db0f9955 P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.052709  9682 tablet_replica.cc:333] T 56bc1b5c8c3143998cb9921b469c071d P e20f6c83ae144d698c26b94c2eda023f: stopping tablet replica
I20260709 13:21:35.053349  9682 raft_consensus.cc:2243] T 56bc1b5c8c3143998cb9921b469c071d P e20f6c83ae144d698c26b94c2eda023f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:35.053882  9682 raft_consensus.cc:2272] T 56bc1b5c8c3143998cb9921b469c071d P e20f6c83ae144d698c26b94c2eda023f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.056764  9682 tablet_replica.cc:333] T a3b0925412bd4dceabec4f898eb43973 P e20f6c83ae144d698c26b94c2eda023f: stopping tablet replica
I20260709 13:21:35.057464  9682 raft_consensus.cc:2243] T a3b0925412bd4dceabec4f898eb43973 P e20f6c83ae144d698c26b94c2eda023f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:35.058319  9682 raft_consensus.cc:2272] T a3b0925412bd4dceabec4f898eb43973 P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.060693  9682 tablet_replica.cc:333] T 2e322f5288b64ccca277807d2dc46579 P e20f6c83ae144d698c26b94c2eda023f: stopping tablet replica
I20260709 13:21:35.061298  9682 raft_consensus.cc:2243] T 2e322f5288b64ccca277807d2dc46579 P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:35.061813  9682 raft_consensus.cc:2272] T 2e322f5288b64ccca277807d2dc46579 P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.064198  9682 tablet_replica.cc:333] T 47e33cdb6a154e8781b3e36a95eed353 P e20f6c83ae144d698c26b94c2eda023f: stopping tablet replica
I20260709 13:21:35.064941  9682 raft_consensus.cc:2243] T 47e33cdb6a154e8781b3e36a95eed353 P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:35.065430  9682 raft_consensus.cc:2272] T 47e33cdb6a154e8781b3e36a95eed353 P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.068167  9682 tablet_replica.cc:333] T 93dd95f0dfb8463291a74921c46665dd P e20f6c83ae144d698c26b94c2eda023f: stopping tablet replica
I20260709 13:21:35.068815  9682 raft_consensus.cc:2243] T 93dd95f0dfb8463291a74921c46665dd P e20f6c83ae144d698c26b94c2eda023f [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:35.069864  9682 raft_consensus.cc:2272] T 93dd95f0dfb8463291a74921c46665dd P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:35.070197 22640 consensus_peers.cc:597] T 2d4c5545efd2494a81249b0dbbfbc56c P 2d62e0b282b44003a202b8dcbe6ea6ac -> Peer e20f6c83ae144d698c26b94c2eda023f (127.9.116.130:43469): Couldn't send request to peer e20f6c83ae144d698c26b94c2eda023f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:35.072263  9682 tablet_replica.cc:333] T 5a1c542f0ae74a718e0b1c70aa641da4 P e20f6c83ae144d698c26b94c2eda023f: stopping tablet replica
I20260709 13:21:35.072970  9682 raft_consensus.cc:2243] T 5a1c542f0ae74a718e0b1c70aa641da4 P e20f6c83ae144d698c26b94c2eda023f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:35.073963  9682 raft_consensus.cc:2272] T 5a1c542f0ae74a718e0b1c70aa641da4 P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.076275  9682 tablet_replica.cc:333] T 2dad40e2d06244cebe50ec7433000752 P e20f6c83ae144d698c26b94c2eda023f: stopping tablet replica
I20260709 13:21:35.076802  9682 raft_consensus.cc:2243] T 2dad40e2d06244cebe50ec7433000752 P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:35.077291  9682 raft_consensus.cc:2272] T 2dad40e2d06244cebe50ec7433000752 P e20f6c83ae144d698c26b94c2eda023f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.079620  9682 tablet_replica.cc:333] T c0614d551fc342cead195ccaed2aeac3 P e20f6c83ae144d698c26b94c2eda023f: stopping tablet replica
I20260709 13:21:35.080135  9682 raft_consensus.cc:2243] T c0614d551fc342cead195ccaed2aeac3 P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:35.080600  9682 raft_consensus.cc:2272] T c0614d551fc342cead195ccaed2aeac3 P e20f6c83ae144d698c26b94c2eda023f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.126474  9682 tablet_server.cc:196] TabletServer@127.9.116.130:0 shutdown complete.
I20260709 13:21:35.161340  9682 tablet_server.cc:179] TabletServer@127.9.116.131:0 shutting down...
W20260709 13:21:35.178876 22642 consensus_peers.cc:597] T 2d4c5545efd2494a81249b0dbbfbc56c P 2d62e0b282b44003a202b8dcbe6ea6ac -> Peer 7df364d160d64ac4b5cdc4f004324f8b (127.9.116.129:42823): Couldn't send request to peer 7df364d160d64ac4b5cdc4f004324f8b. Status: Network error: Client connection negotiation failed: client connection to 127.9.116.129:42823: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:35.192162  9682 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:21:35.193048  9682 tablet_replica.cc:333] T 2d4c5545efd2494a81249b0dbbfbc56c P 2d62e0b282b44003a202b8dcbe6ea6ac: stopping tablet replica
I20260709 13:21:35.194036  9682 raft_consensus.cc:2243] T 2d4c5545efd2494a81249b0dbbfbc56c P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:35.195135  9682 raft_consensus.cc:2272] T 2d4c5545efd2494a81249b0dbbfbc56c P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.197224  9682 tablet_replica.cc:333] T 47e33cdb6a154e8781b3e36a95eed353 P 2d62e0b282b44003a202b8dcbe6ea6ac: stopping tablet replica
I20260709 13:21:35.197750  9682 raft_consensus.cc:2243] T 47e33cdb6a154e8781b3e36a95eed353 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:35.198145  9682 raft_consensus.cc:2272] T 47e33cdb6a154e8781b3e36a95eed353 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.200169  9682 tablet_replica.cc:333] T e1e94cad244443feb24f679271460852 P 2d62e0b282b44003a202b8dcbe6ea6ac: stopping tablet replica
I20260709 13:21:35.200707  9682 raft_consensus.cc:2243] T e1e94cad244443feb24f679271460852 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:35.201089  9682 raft_consensus.cc:2272] T e1e94cad244443feb24f679271460852 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.203013  9682 tablet_replica.cc:333] T 5b97872c6f3b4a288ff8ade4569da6f1 P 2d62e0b282b44003a202b8dcbe6ea6ac: stopping tablet replica
I20260709 13:21:35.203495  9682 raft_consensus.cc:2243] T 5b97872c6f3b4a288ff8ade4569da6f1 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:35.203907  9682 raft_consensus.cc:2272] T 5b97872c6f3b4a288ff8ade4569da6f1 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.205737  9682 tablet_replica.cc:333] T ef14a8f3b6344312bd294edcabb2f312 P 2d62e0b282b44003a202b8dcbe6ea6ac: stopping tablet replica
I20260709 13:21:35.206197  9682 raft_consensus.cc:2243] T ef14a8f3b6344312bd294edcabb2f312 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:35.206580  9682 raft_consensus.cc:2272] T ef14a8f3b6344312bd294edcabb2f312 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.208475  9682 tablet_replica.cc:333] T 2f75c1bbbd7b4dc09412c016798ccd50 P 2d62e0b282b44003a202b8dcbe6ea6ac: stopping tablet replica
I20260709 13:21:35.208957  9682 raft_consensus.cc:2243] T 2f75c1bbbd7b4dc09412c016798ccd50 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:35.209323  9682 raft_consensus.cc:2272] T 2f75c1bbbd7b4dc09412c016798ccd50 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.211223  9682 tablet_replica.cc:333] T 0f952d6650154a4a98f94a75b916d44b P 2d62e0b282b44003a202b8dcbe6ea6ac: stopping tablet replica
I20260709 13:21:35.211733  9682 raft_consensus.cc:2243] T 0f952d6650154a4a98f94a75b916d44b P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:35.212105  9682 raft_consensus.cc:2272] T 0f952d6650154a4a98f94a75b916d44b P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.213937  9682 tablet_replica.cc:333] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 2d62e0b282b44003a202b8dcbe6ea6ac: stopping tablet replica
I20260709 13:21:35.214440  9682 raft_consensus.cc:2243] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:35.214947  9682 raft_consensus.cc:2272] T a9c5f91cbe6f4a6abbd799e94e0985e1 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.216951  9682 tablet_replica.cc:333] T 2dad40e2d06244cebe50ec7433000752 P 2d62e0b282b44003a202b8dcbe6ea6ac: stopping tablet replica
I20260709 13:21:35.217626  9682 raft_consensus.cc:2243] T 2dad40e2d06244cebe50ec7433000752 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:35.218458  9682 raft_consensus.cc:2272] T 2dad40e2d06244cebe50ec7433000752 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.220463  9682 tablet_replica.cc:333] T c0614d551fc342cead195ccaed2aeac3 P 2d62e0b282b44003a202b8dcbe6ea6ac: stopping tablet replica
I20260709 13:21:35.220937  9682 raft_consensus.cc:2243] T c0614d551fc342cead195ccaed2aeac3 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:35.221309  9682 raft_consensus.cc:2272] T c0614d551fc342cead195ccaed2aeac3 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.223307  9682 tablet_replica.cc:333] T dded1824a3ef49988cc1b941db0f9955 P 2d62e0b282b44003a202b8dcbe6ea6ac: stopping tablet replica
I20260709 13:21:35.223821  9682 raft_consensus.cc:2243] T dded1824a3ef49988cc1b941db0f9955 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:35.224212  9682 raft_consensus.cc:2272] T dded1824a3ef49988cc1b941db0f9955 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.226112  9682 tablet_replica.cc:333] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac: stopping tablet replica
I20260709 13:21:35.226572  9682 raft_consensus.cc:2243] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:21:35.227414  9682 raft_consensus.cc:2272] T 5e4995abf28446589b7695a1b77ee513 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.230110  9682 tablet_replica.cc:333] T 68d74bd4ad624659b92ec79d4560a02c P 2d62e0b282b44003a202b8dcbe6ea6ac: stopping tablet replica
I20260709 13:21:35.230638  9682 raft_consensus.cc:2243] T 68d74bd4ad624659b92ec79d4560a02c P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:35.231101  9682 raft_consensus.cc:2272] T 68d74bd4ad624659b92ec79d4560a02c P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.232997  9682 tablet_replica.cc:333] T 93dd95f0dfb8463291a74921c46665dd P 2d62e0b282b44003a202b8dcbe6ea6ac: stopping tablet replica
I20260709 13:21:35.233436  9682 raft_consensus.cc:2243] T 93dd95f0dfb8463291a74921c46665dd P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:35.233822  9682 raft_consensus.cc:2272] T 93dd95f0dfb8463291a74921c46665dd P 2d62e0b282b44003a202b8dcbe6ea6ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.235795  9682 tablet_replica.cc:333] T 5a1c542f0ae74a718e0b1c70aa641da4 P 2d62e0b282b44003a202b8dcbe6ea6ac: stopping tablet replica
I20260709 13:21:35.236261  9682 raft_consensus.cc:2243] T 5a1c542f0ae74a718e0b1c70aa641da4 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:35.236673  9682 raft_consensus.cc:2272] T 5a1c542f0ae74a718e0b1c70aa641da4 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.238610  9682 tablet_replica.cc:333] T a3b0925412bd4dceabec4f898eb43973 P 2d62e0b282b44003a202b8dcbe6ea6ac: stopping tablet replica
I20260709 13:21:35.239120  9682 raft_consensus.cc:2243] T a3b0925412bd4dceabec4f898eb43973 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:35.239518  9682 raft_consensus.cc:2272] T a3b0925412bd4dceabec4f898eb43973 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.241353  9682 tablet_replica.cc:333] T 2e322f5288b64ccca277807d2dc46579 P 2d62e0b282b44003a202b8dcbe6ea6ac: stopping tablet replica
I20260709 13:21:35.241961  9682 raft_consensus.cc:2243] T 2e322f5288b64ccca277807d2dc46579 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:35.242479  9682 raft_consensus.cc:2272] T 2e322f5288b64ccca277807d2dc46579 P 2d62e0b282b44003a202b8dcbe6ea6ac [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.245311  9682 tablet_replica.cc:333] T 56bc1b5c8c3143998cb9921b469c071d P 2d62e0b282b44003a202b8dcbe6ea6ac: stopping tablet replica
I20260709 13:21:35.245930  9682 raft_consensus.cc:2243] T 56bc1b5c8c3143998cb9921b469c071d P 2d62e0b282b44003a202b8dcbe6ea6ac [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:21:35.246918  9682 raft_consensus.cc:2272] T 56bc1b5c8c3143998cb9921b469c071d P 2d62e0b282b44003a202b8dcbe6ea6ac [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.284173  9682 tablet_server.cc:196] TabletServer@127.9.116.131:0 shutdown complete.
I20260709 13:21:35.317545  9682 master.cc:562] Master@127.9.116.190:36011 shutting down...
I20260709 13:21:35.339203  9682 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 542aff268f9b4b3abea229f5c457ad3c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:35.339794  9682 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 542aff268f9b4b3abea229f5c457ad3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.340155  9682 tablet_replica.cc:333] T 00000000000000000000000000000000 P 542aff268f9b4b3abea229f5c457ad3c: stopping tablet replica
I20260709 13:21:35.360719  9682 master.cc:584] Master@127.9.116.190:36011 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (11475 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (23851 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (378715 ms total)
[  PASSED  ] 7 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] LeaderRebalancerTest.AddTserver

 1 FAILED TEST
I20260709 13:21:35.416056  9682 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 401 messages since previous log ~14 seconds ago
I20260709 13:21:35.416324  9682 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 401 messages since previous log ~14 seconds ago
I20260709 13:21:35.416517  9682 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 400 messages since previous log ~14 seconds ago
I20260709 13:21:35.416721  9682 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~33 seconds ago
I20260709 13:21:35.416915  9682 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~33 seconds ago
I20260709 13:21:35.417128  9682 logging.cc:436] LogThrottler Service Queue Overflow: suppressed but not reported on 7 messages since previous log ~182 seconds ago
I20260709 13:21:35.417326  9682 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/service_pool.cc:136: suppressed but not reported on 7 messages since previous log ~182 seconds ago
I20260709 13:21:35.417531  9682 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 5 messages since previous log ~160 seconds ago
I20260709 13:21:35.417793  9682 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 6 messages since previous log ~0 seconds ago
I20260709 13:21:35.418102  9682 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 53 messages since previous log ~10 seconds ago
